aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/malloc.go
AgeCommit message (Collapse)Author
2022-03-31runtime: add wrappers for sys* functions and consolidate docsMichael Anthony Knyszek
This change lifts all non-platform-specific code out of sys* functions for each platform up into wrappers, and moves documentation about the OS virtual memory abstraction layer from malloc.go to mem.go, which contains those wrappers. Change-Id: Ie803e4447403eaafc508b34b53a1a47d6cee9388 Reviewed-on: https://go-review.googlesource.com/c/go/+/393398 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Knyszek <mknyszek@google.com>
2022-03-31runtime: remove intermediate fields in memstats for ReadMemStatsMichael Anthony Knyszek
Currently, the ReadMemStats (really this is all happening in readmemstats_m, but that's just a direct call from ReadMemStats) call chain first populates some fields in memstats, then copies those into the final MemStats location. This used to make a lot of sense when memstats' structure aligned with MemStats, and the values were just copied from one to other. Sometime in the last few releases, we switched to populating the MemStats manually because a lot of fields had diverged from their internal representation. Now, we're left with a lot of fields in memstats that pollute the structure: they only exist to be updated for the sake of ReadMemStats. Since we're going to be adding more fields to memstats in further CLs, this is a good opportunity to clean up. As a result of this change, updatememstats, which used to just update the aforementioned intermediate fields in memstats, is no longer necessary, so it is removed. Change-Id: Ifabfb3ac3002641105af62e9509a6351165dcd87 Reviewed-on: https://go-review.googlesource.com/c/go/+/393397 Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-12-06runtime: set iOS addr space to 40 bits with incremental pageallocMichael Anthony Knyszek
In iOS <14, the address space is strictly limited to 8 GiB, or 33 bits. As a result, the page allocator also assumes all heap memory lives in this region. This is especially necessary because the page allocator has a PROT_NONE mapping proportional to the size of the usable address space, so this keeps that mapping very small. However starting with iOS 14, this restriction is relaxed, and mmap may start returning addresses outside of the <14 range. Today this means that in iOS 14 and later, users experience an error in the page allocator when a heap arena is mapped outside of the old range. This change increases the ios/arm64 heapAddrBits to 40 while simultaneously making ios/arm64 use the 64-bit pagealloc implementation (with reservations and incremental mapping) to accommodate both iOS versions <14 and 14+. Once iOS <14 is deprecated, we can remove these exceptions and treat ios/arm64 like any other arm64 platform. This change also makes the BaseChunkIdx expression a little bit easier to read, while we're here. Fixes #46860. Change-Id: I13865f799777739109585f14f1cc49d6d57e096b Reviewed-on: https://go-review.googlesource.com/c/go/+/344401 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Austin Clements <austin@google.com>
2021-11-02runtime, syscall: add calls to asan functionsfanzha02
Add explicit address sanitizer instrumentation to the runtime and syscall packages. The compiler does not instrument the runtime package. It does instrument the syscall package, but we need to add a couple of cases that it can't see. Refer to the implementation of the asan malloc runtime library, this patch also allocates extra memory as the redzone, around the returned memory region, and marks the redzone as unaddressable to detect the overflows or underflows. Updates #44853. Change-Id: I2753d1cc1296935a66bf521e31ce91e35fcdf798 Reviewed-on: https://go-review.googlesource.com/c/go/+/298614 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: fannie zhang <Fannie.Zhang@arm.com>
2021-10-29runtime: clean up allocation zeroingMichael Anthony Knyszek
Currently, the runtime zeroes allocations in several ways. First, small object spans are always zeroed if they come from mheap, and their slots are zeroed later in mallocgc if needed. Second, large object spans (objects that have their own spans) plumb the need for zeroing down into mheap. Thirdly, large objects that have no pointers have their zeroing delayed until after preemption is reenabled, but before returning in mallocgc. All of this has two consequences: 1. Spans for small objects that come from mheap are sometimes unnecessarily zeroed, even if the mallocgc call that created them doesn't need the object slot to be zeroed. 2. This is all messy and difficult to reason about. This CL simplifies this code, resolving both (1) and (2). First, it recognizes that zeroing in mheap is unnecessary for small object spans; mallocgc and its callees in mcache and mcentral, by design, are *always* able to deal with non-zeroed spans. They must, for they deal with recycled spans all the time. Once this fact is made clear, the only remaining use of zeroing in mheap is for large objects. As a result, this CL lifts mheap zeroing for large objects into mallocgc, to parallel all the other codepaths in mallocgc. This is makes the large object allocation code less surprising. Next, this CL sets the flag for the delayed zeroing explicitly in the one case where it matters, and inverts and renames the flag from isZeroed to delayZeroing. Finally, it adds a check to make sure that only pointer-free allocations take the delayed zeroing codepath, as an extra safety measure. Benchmark results: https://perf.golang.org/search?q=upload:20211028.8 Inspired by tapir.liu@gmail.com's CL 343470. Change-Id: I7e1296adc19ce8a02c8d93a0a5082aefb2673e8f Reviewed-on: https://go-review.googlesource.com/c/go/+/359477 Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
2021-10-07runtime,sync: using fastrandn instead of modulo reductionMeng Zhuo
fastrandn is ~50% faster than fastrand() % n. `ack -v 'fastrand\(\)\s?\%'` finds all modulo on fastrand() name old time/op new time/op delta Fastrandn/2 2.86ns ± 0% 1.59ns ± 0% -44.35% (p=0.000 n=9+10) Fastrandn/3 2.87ns ± 1% 1.59ns ± 0% -44.41% (p=0.000 n=10+9) Fastrandn/4 2.87ns ± 1% 1.58ns ± 1% -45.10% (p=0.000 n=10+10) Fastrandn/5 2.86ns ± 1% 1.58ns ± 1% -44.84% (p=0.000 n=10+10) Change-Id: Ic91f5ca9b9e3b65127bc34792b62fd64fbd13b5c Reviewed-on: https://go-review.googlesource.com/c/go/+/353269 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2021-09-28runtime: add mp parameter for getMCacheLeonard Wang
Since all callers of getMCache appear to have mp available, we pass the mp to getMCache, and reduce one call to getg. And after modification, getMCache is also inlined. Change-Id: Ib7880c118336acc026ecd7c60c5a88722c3ddfc7 Reviewed-on: https://go-review.googlesource.com/c/go/+/349329 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Trust: Michael Knyszek <mknyszek@google.com> Trust: Carlos Amedee <carlos@golang.org>
2021-07-22[dev.typeparams] all: merge master (798ec73) into dev.typeparamsMatthew Dempsky
Merge List: + 2021-07-22 798ec73519 runtime: don't clear timerModifiedEarliest if adjustTimers is 0 + 2021-07-22 fdb45acd1f runtime: move mem profile sampling into m-acquired section + 2021-07-21 3e48c0381f reflect: add missing copyright header + 2021-07-21 48c88f1b1b reflect: add Value.CanConvert + 2021-07-20 9e26569293 cmd/go: don't add C compiler ID to hash for standard library + 2021-07-20 d568e6e075 runtime/debug: skip TestPanicOnFault on netbsd/arm Change-Id: I87e1cd4614bb3b00807f18dfdd02664dcaecaebd
2021-07-22runtime: move mem profile sampling into m-acquired sectionDavid Chase
It was not safe to do mcache profiling updates outside the critical section, but we got lucky because the runtime was not preemptible. Adding chunked memory clearing (CL 270943) created preemption opportunities, which led to corruption of runtime data structures. Fixes #47304. Fixes #47302. Change-Id: I461615470d62328a83ccbac537fbdc6dcde81c85 Reviewed-on: https://go-review.googlesource.com/c/go/+/336449 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2021-06-17[dev.typeparams] runtime: replace Goarch* constants with internal/goarch ↵Michael Anthony Knyszek
versions [generated] [git-generate] cd src/runtime gofmt -w -r "sys.Goarch386 -> goarch.Is386" . gofmt -w -r "sys.GoarchAmd64 -> goarch.IsAmd64" . gofmt -w -r "sys.GoarchAmd64p32 -> goarch.IsAmd64p32" . gofmt -w -r "sys.GoarchArm -> goarch.IsArm" . gofmt -w -r "sys.GoarchArmbe -> goarch.IsArmbe" . gofmt -w -r "sys.GoarchArm64 -> goarch.IsArm64" . gofmt -w -r "sys.GoarchArm64be -> goarch.IsArm64be" . gofmt -w -r "sys.GoarchPpc64 -> goarch.IsPpc64" . gofmt -w -r "sys.GoarchPpc64le -> goarch.IsPpc64le" . gofmt -w -r "sys.GoarchMips -> goarch.IsMips" . gofmt -w -r "sys.GoarchMipsle -> goarch.IsMipsle" . gofmt -w -r "sys.GoarchMips64 -> goarch.IsMips64" . gofmt -w -r "sys.GoarchMips64le -> goarch.IsMips64le" . gofmt -w -r "sys.GoarchMips64p32 -> goarch.IsMips64p32" . gofmt -w -r "sys.GoarchMips64p32le -> goarch.IsMips64p32le" . gofmt -w -r "sys.GoarchPpc -> goarch.IsPpc" . gofmt -w -r "sys.GoarchRiscv -> goarch.IsRiscv" . gofmt -w -r "sys.GoarchRiscv64 -> goarch.IsRiscv64" . gofmt -w -r "sys.GoarchS390 -> goarch.IsS390" . gofmt -w -r "sys.GoarchS390x -> goarch.IsS390x" . gofmt -w -r "sys.GoarchSparc -> goarch.IsSparc" . gofmt -w -r "sys.GoarchSparc64 -> goarch.IsSparc64" . gofmt -w -r "sys.GoarchWasm -> goarch.IsWasm" . goimports -w *.go Change-Id: I9d88e1284efabaeb0ee3733cba6286247d078c85 Reviewed-on: https://go-review.googlesource.com/c/go/+/328345 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-06-17[dev.typeparams] runtime: replace Goos* constants with internal/goos ↵Michael Anthony Knyszek
versions [generated] [git-generate] cd src/runtime gofmt -w -r "sys.GoosAix -> goos.IsAix" . gofmt -w -r "sys.GoosAndroid -> goos.IsAndroid" . gofmt -w -r "sys.GoosDarwin -> goos.IsDarwin" . gofmt -w -r "sys.GoosDragonfly -> goos.IsDragonfly" . gofmt -w -r "sys.GoosFreebsd -> goos.IsFreebsd" . gofmt -w -r "sys.GoosHurd -> goos.IsHurd" . gofmt -w -r "sys.GoosIllumos -> goos.IsIllumos" . gofmt -w -r "sys.GoosIos -> goos.IsIos" . gofmt -w -r "sys.GoosJs -> goos.IsJs" . gofmt -w -r "sys.GoosLinux -> goos.IsLinux" . gofmt -w -r "sys.GoosNacl -> goos.IsNacl" . gofmt -w -r "sys.GoosNetbsd -> goos.IsNetbsd" . gofmt -w -r "sys.GoosOpenbsd -> goos.IsOpenbsd" . gofmt -w -r "sys.GoosPlan9 -> goos.IsPlan9" . gofmt -w -r "sys.GoosSolaris -> goos.IsSolaris" . gofmt -w -r "sys.GoosWindows -> goos.IsWindows" . gofmt -w -r "sys.GoosZos -> goos.IsZos" . goimports -w *.go Change-Id: I42bed2907317ed409812e5a3e2897c88a5d36f24 Reviewed-on: https://go-review.googlesource.com/c/go/+/328344 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-06-17[dev.typeparams] runtime: fix import sort order [generated]Michael Anthony Knyszek
[git-generate] cd src/runtime goimports -w *.go Change-Id: I1387af0f2fd1a213dc2f4c122e83a8db0fcb15f0 Reviewed-on: https://go-review.googlesource.com/c/go/+/329189 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-06-17[dev.typeparams] runtime: replace uses of runtime/internal/sys.PtrSize with ↵Michael Anthony Knyszek
internal/goarch.PtrSize [generated] [git-generate] cd src/runtime/internal/math gofmt -w -r "sys.PtrSize -> goarch.PtrSize" . goimports -w *.go cd ../.. gofmt -w -r "sys.PtrSize -> goarch.PtrSize" . goimports -w *.go Change-Id: I43491cdd54d2e06d4d04152b3d213851b7d6d423 Reviewed-on: https://go-review.googlesource.com/c/go/+/328337 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-06-11[dev.typeparams] runtime: simplify defer record allocationCherry Mui
Now that deferred functions are always argumentless and defer records are no longer with arguments, defer record can be fixed size (just the _defer struct). This allows us to simplify the allocation of defer records, specifically, remove the defer classes and the pools of different sized defers. Change-Id: Icc4b16afc23b38262ca9dd1f7369ad40874cf701 Reviewed-on: https://go-review.googlesource.com/c/go/+/326062 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-05-26runtime,cmd/link/internal/ld: fix typostyltr
Change-Id: I558590cef7e2311aadbdcb4088033e350d3aae32 GitHub-Last-Rev: 513944a6238e0e32e2a2c266b70f7d50c9db508d GitHub-Pull-Request: golang/go#46389 Reviewed-on: https://go-review.googlesource.com/c/go/+/322809 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-04-30runtime: break up large calls to memclrNoHeapPointers to allow preemptionDavid Chase
If something "huge" is allocated, and the zeroing is trivial (no pointers involved) then zero it by chunks in a loop so that preemption can occur, not all in a single non-preemptible call. Benchmarking suggests that 256K is the best chunk size. Updates #42642. Change-Id: I94015e467eaa098c59870e479d6d83bc88efbfb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/270943 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29runtime: top align tinyallocs in race modeKeith Randall
Top align allocations in tinyalloc buckets when in race mode. This will make checkptr checks more reliable, because any code that modifies a pointer past the end of the object will trigger a checkptr error. No test, because we need -race for this to actually kick in. We could add it to the race detector tests, but the race detector tests are all geared towards race detector reports, not checkptr reports. Mucking with parsing reports is more than a test is worth. Fixes #38872 Change-Id: Ie56f0fbd1a9385539f6631fd1ac40c3de5600154 Reviewed-on: https://go-review.googlesource.com/c/go/+/315029 Trust: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-29runtime: use 4 MiB heap arenas on iOSMichael Anthony Knyszek
iOS arm64 is a 64-bit platform but with a strictly 32-bit address space (technically 33 bits, but the bottom half is unavailable to the application). Since address space is limited, use 4 MiB arenas instead of 64 MiB arenas. No changes are needed to the arena index because it's still relatively small; this change just brings iOS more in line with 32-bit platforms. Change-Id: I484e2d273d896fd0a57cd5c25012df0aef160290 Reviewed-on: https://go-review.googlesource.com/c/go/+/270538 Trust: Michael Knyszek <mknyszek@google.com> Trust: Emmanuel Odeke <emmanuel@orijtech.com> Trust: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-30runtime: fix typos in commentsLizzzcai
Change-Id: Ia70e8bdc6d2cf1195d7a3b5d33f180ae2db73e29 Reviewed-on: https://go-review.googlesource.com/c/go/+/305369 Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Ben Shi <powerman1st@163.com> Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-03-15runtime: prepare arenas for use incrementallyMichael Anthony Knyszek
This change moves the call of sysMap from (*mheap).sysAlloc into (*mheap).grow, so we only sysMap what we're going to use in the near future (thanks to the curArena mechanism). The purpose of this change is to better support systems with strict overcommit rules which generally accept reserved memory but not prepared memory (see malloc.go for exact descriptions of these states). This move requires changing linearAlloc to only optionally map memory. In one case, with mheap.heapArenaAlloc, we do want it to map memory. But now in the other case, with mheap.arena, we don't, because we want grow to take care of it. The risk with this change is we may make more syscalls than before on systems with 64 MiB arenas, but because heap growth is relatively rare this is unlikely to be a noticable issue. We also bound the amount of syscalls made by only extending curArena (and thus mapping) by pallocChunkPages*pageSize which is 4 MiB. Fixes #42612. Change-Id: I736df696afe78ddb1a747a896caa0db8726027e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/270537 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-11-05runtime: avoid a bit of unneeded work when MemProfileRate==1Brad Fitzpatrick
Change-Id: I1dc355bcaeb0e5fb06a7fddc4cf5db596d22e0b3 Reviewed-on: https://go-review.googlesource.com/c/go/+/236148 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Austin Clements <austin@google.com>
2020-11-02runtime: make getMCache inlineableMichael Anthony Knyszek
This change moves the responsibility of throwing if an mcache is not available to the caller, because the inlining cost of throw is set very high in the compiler. Even if it was reduced down to the cost of a usual function call, it would still be too expensive, so just move it out. This choice also makes sense in the context of #42339 since we're going to have to handle the case where we don't have an mcache to update stats in a few contexts anyhow. Also, add getMCache to the list of functions that should be inlined to prevent future regressions. getMCache is called on the allocation fast path and because its not inlined actually causes a significant regression (~10%) in some microbenchmarks. Fixes #42305. Change-Id: I64ac5e4f26b730bd4435ea1069a4a50f55411ced Reviewed-on: https://go-review.googlesource.com/c/go/+/267157 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-30runtime: add heap lock assertionsMichael Pratt
Some functions that required holding the heap lock _or_ world stop have been simplified to simply requiring the heap lock. This is conceptually simpler and taking the heap lock during world stop is guaranteed to not contend. This was only done on functions already called on the systemstack to avoid too many extra systemstack calls in GC. Updates #40677 Change-Id: I15aa1dadcdd1a81aac3d2a9ecad6e7d0377befdc Reviewed-on: https://go-review.googlesource.com/c/go/+/250262 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Trust: Michael Pratt <mpratt@google.com>
2020-10-30runtime: allocate at desired address when race detector is onCherry Zhang
Currently, on all supported platforms, the race detector (LLVM TSAN) expects the Go heap is at 0xc000000000 - 0xe000000000. Move the raceenabled condition first, so we always allocate there. This means on Linux/ARM64 when race detector is on we will allocate to 0xc000000000 - 0xe000000000, instead of 0x4000000000. The old address is meant for 39-bit VMA. But the race detector only supports 48-bit VMA anyway. So this is fine. Change-Id: I51ac8eff68297b37c8c651a93145cc94f83a939d Reviewed-on: https://go-review.googlesource.com/c/go/+/266372 Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-26runtime: move malloc stats into consistentHeapStatsMichael Anthony Knyszek
This change moves the mcache-local malloc stats into the consistentHeapStats structure so the malloc stats can be managed consistently with the memory stats. The one exception here is tinyAllocs for which moving that into the global stats would incur several atomic writes on the fast path. Microbenchmarks for just one CPU core have shown a 50% loss in throughput. Since tiny allocation counnt isn't exposed anyway and is always blindly added to both allocs and frees, let that stay inconsistent and flush the tiny allocation count every so often. Change-Id: I2a4b75f209c0e659b9c0db081a3287bf227c10ca Reviewed-on: https://go-review.googlesource.com/c/go/+/247039 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-26runtime: add helper for getting an mcache in allocation contextsMichael Anthony Knyszek
This change adds a function getMCache which returns the current P's mcache if it's available, and otherwise tries to get mcache0 if we're bootstrapping. This function will come in handy as we need to replicate this behavior in multiple places in future changes. Change-Id: I536073d6f6dc6c6390269e613ead9f8bcb6e7f98 Reviewed-on: https://go-review.googlesource.com/c/go/+/246976 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-26runtime: break down memstats.gc_sysMichael Anthony Knyszek
This change breaks apart gc_sys into three distinct pieces. Two of those pieces are pieces which come from heap_sys since they're allocated from the page heap. The rest comes from memory mapped from e.g. persistentalloc which better fits the purpose of a sysMemStat. Also, rename gc_sys to gcMiscSys. Change-Id: I098789170052511e7b31edbcdc9a53e5c24573f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/246973 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-26runtime: delineate which memstats are system stats with a typeMichael Anthony Knyszek
This change modifies the type of several mstats fields to be a new type: sysMemStat. This type has the same structure as the fields used to have. The purpose of this change is to make it very clear which stats may be used in various functions for accounting (usually the platform-specific sys* functions, but there are others). Currently there's an implicit understanding that the *uint64 value passed to these functions is some kind of statistic whose value is atomically managed. This understanding isn't inherently problematic, but we're about to change how some stats (which currently use mSysStatInc and mSysStatDec) work, so we want to make it very clear what the various requirements are around "sysStat". This change also removes mSysStatInc and mSysStatDec in favor of a method on sysMemStat. Note that those two functions were originally written the way they were because atomic 64-bit adds required a valid G on ARM, but this hasn't been the case for a very long time (since golang.org/cl/14204, but even before then it wasn't clear if mutexes required a valid G anymore). Today we implement 64-bit adds on ARM with a spinlock table. Change-Id: I4e9b37cf14afc2ae20cf736e874eb0064af086d7 Reviewed-on: https://go-review.googlesource.com/c/go/+/246971 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-26runtime: rename mcache fields to match Go styleMichael Anthony Knyszek
This change renames a bunch of malloc statistics stored in the mcache that are all named with the "local_" prefix. It also renames largeAlloc to allocLarge to prevent a naming conflict, and next_sample because it would be the last mcache field with the old C naming style. Change-Id: I29695cb83b397a435ede7e9ad5c3c9be72767ea3 Reviewed-on: https://go-review.googlesource.com/c/go/+/246969 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-26runtime: make nlargealloc and largealloc mcache fieldsMichael Anthony Knyszek
This change makes nlargealloc and largealloc into mcache fields just like nlargefree and largefree. These local fields become the new source-of-truth. This change also moves the accounting for these fields out of allocSpan (which is an inappropriate place for it -- this accounting generally happens much closer to the point of allocation) and into largeAlloc. This move is partially possible now that we can call gcController.revise at that point. Furthermore, this change moves largeAlloc into mcache.go and makes it a method of mcache. While there's a little bit of a mismatch here because largeAlloc barely interacts with the mcache, it helps solidify the mcache as the first allocation layer and provides a clear place to aggregate and manage statistics. Change-Id: I37b5e648710733bb4c04430b71e96700e438587a Reviewed-on: https://go-review.googlesource.com/c/go/+/246965 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-14runtime: implement GODEBUG=inittrace=1 supportMartin Möhrmann
Setting inittrace=1 causes the runtime to emit a single line to standard error for each package with init work, summarizing the execution time and memory allocation. The emitted debug information for init functions can be used to find bottlenecks or regressions in Go startup performance. Packages with no init function work (user defined or compiler generated) are omitted. Tracing plugin inits is not supported as they can execute concurrently. This would make the implementation of tracing more complex while adding support for a very rare use case. Plugin inits can be traced separately by testing a main package importing the plugins package imports explicitly. $ GODEBUG=inittrace=1 go test init internal/bytealg @0.008 ms, 0 ms clock, 0 bytes, 0 allocs init runtime @0.059 ms, 0.026 ms clock, 0 bytes, 0 allocs init math @0.19 ms, 0.001 ms clock, 0 bytes, 0 allocs init errors @0.22 ms, 0.004 ms clock, 0 bytes, 0 allocs init strconv @0.24 ms, 0.002 ms clock, 32 bytes, 2 allocs init sync @0.28 ms, 0.003 ms clock, 16 bytes, 1 allocs init unicode @0.44 ms, 0.11 ms clock, 23328 bytes, 24 allocs ... Inspired by stapelberg@google.com who instrumented doInit in a prototype to measure init times with GDB. Fixes #41378 Change-Id: Ic37c6a0cfc95488de9e737f5e346b8dbb39174e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/254659 Trust: Martin Möhrmann <moehrmann@google.com> Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-10-06runtime: enable more address bits on macOS/ARM64Cherry Zhang
Apparently macOS/ARM64 has 47-bit addresses, instead of 33-bit as on ios/ARM64. Enable more address bits. Updates #38485. Change-Id: I8aa64ba22a3933e3d9c4fffd17d902b5f31c30e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/256918 Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-10-01runtime: align 12-byte objects to 8 bytes on 32-bit systemsMichael Anthony Knyszek
Currently on 32-bit systems 8-byte fields in a struct have an alignment of 4 bytes, which means that atomic instructions may fault. This issue is tracked in #36606. Our current workaround is to allocate memory and put any such atomically accessed fields at the beginning of the object. This workaround fails because the tiny allocator might not align the object right. This case specifically only happens with 12-byte objects because a type's size is rounded up to its alignment. So if e.g. we have a type like: type obj struct { a uint64 b byte } then its size will be 12 bytes, because "a" will require a 4 byte alignment. This argument may be extended to all objects of size 9-15 bytes. So, make this workaround work by specifically aligning such objects to 8 bytes on 32-bit systems. This change leaves a TODO to remove the code once #36606 gets resolved. It also adds a test which will presumably no longer be necessary (the compiler should enforce the right alignment) when it gets resolved as well. Fixes #37262. Change-Id: I3a34e5b014b3c37ed2e5e75e62d71d8640aa42bc Reviewed-on: https://go-review.googlesource.com/c/go/+/254057 Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com>
2020-09-23all: add GOOS=iosCherry Zhang
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin" build tag, like GOOS=android matches "linux" and GOOS=illumos matches "solaris". Only ios/arm64 is supported (ios/amd64 is not). GOOS=ios and GOOS=darwin remain essentially the same at this point. They will diverge at later time, to differentiate macOS and iOS. Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"), except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"), it remains GOOS=="darwin". Updates #38485. Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c Reviewed-on: https://go-review.googlesource.com/c/go/+/254740 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-08-17runtime: clean up old mcentral codeMichael Anthony Knyszek
This change deletes the old mcentral implementation from the code base and the newMCentralImpl feature flag along with it. Updates #37487. Change-Id: Ibca8f722665f0865051f649ffe699cbdbfdcfcf2 Reviewed-on: https://go-review.googlesource.com/c/go/+/221184 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-06-27runtime: ensure arenaBaseOffset makes it into DWARF (for viewcore)Keith Randall
This constant does not make it into DWARF because it is an ideal constant larger than maxint (1<<63-1). DWARF has no way to represent signed values that large. Define a different typed constant that is unsigned and so can represent this constant properly. Viewcore needs this constant to interrogate the heap data structures. In addition, the sign of arenaBaseOffset changed in 1.15, and providing a new name lets viewcore detect the sign change easily. Change-Id: I4274a2f6e79ebbf1411e85d64758fac1672fb96b Reviewed-on: https://go-review.googlesource.com/c/go/+/240198 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-06-08runtime: always mark span when marking an objectAustin Clements
The page sweeper depends on spans being marked if any object in the span is marked, but currently only greyobject does this. gcmarknewobject and wbBufFlush1 also mark objects, but neither set span marks. As a result, if there are live objects on a span, but they're all marked via allocation or write barriers, then the span itself won't be marked and the page reclaimer will free the span, ultimately leading to memory corruption when the memory for those live allocations gets reused. Fix this by making gcmarknewobject and wbBufFlush1 also mark pages. No test because I have no idea how to reliably (or even unreliably) trigger this. Fixes #39432. Performance is a wash or very slightly worse. I benchmarked the gcmarknewobject and wbBufFlush1 changes independently and both showed a slight performance improvement, so I'm going to call this noise. name old time/op new time/op delta BiogoIgor 15.9s ± 2% 15.9s ± 2% ~ (p=0.758 n=25+25) BiogoKrishna 15.7s ± 3% 15.7s ± 3% ~ (p=0.382 n=21+21) BleveIndexBatch100 4.94s ± 3% 5.07s ± 4% +2.63% (p=0.000 n=25+25) CompileTemplate 204ms ± 1% 205ms ± 1% +0.43% (p=0.000 n=21+23) CompileUnicode 77.8ms ± 1% 78.1ms ± 1% ~ (p=0.130 n=23+23) CompileGoTypes 731ms ± 1% 733ms ± 1% +0.30% (p=0.006 n=22+22) CompileCompiler 3.64s ± 2% 3.65s ± 3% ~ (p=0.179 n=24+25) CompileSSA 8.44s ± 1% 8.46s ± 1% +0.30% (p=0.003 n=22+23) CompileFlate 132ms ± 1% 133ms ± 1% ~ (p=0.098 n=22+22) CompileGoParser 164ms ± 1% 164ms ± 1% +0.37% (p=0.000 n=21+23) CompileReflect 455ms ± 1% 457ms ± 2% +0.50% (p=0.002 n=20+22) CompileTar 182ms ± 2% 182ms ± 1% ~ (p=0.382 n=22+22) CompileXML 245ms ± 3% 245ms ± 1% ~ (p=0.070 n=21+23) CompileStdCmd 16.5s ± 2% 16.5s ± 3% ~ (p=0.486 n=23+23) FoglemanFauxGLRenderRotateBoat 12.9s ± 1% 13.0s ± 1% +0.97% (p=0.000 n=21+24) FoglemanPathTraceRenderGopherIter1 18.6s ± 1% 18.7s ± 0% ~ (p=0.083 n=23+24) GopherLuaKNucleotide 28.4s ± 1% 29.3s ± 1% +2.84% (p=0.000 n=25+25) MarkdownRenderXHTML 252ms ± 0% 251ms ± 1% -0.50% (p=0.000 n=23+24) Tile38WithinCircle100kmRequest 516µs ± 2% 516µs ± 2% ~ (p=0.763 n=24+25) Tile38IntersectsCircle100kmRequest 689µs ± 2% 689µs ± 2% ~ (p=0.617 n=24+24) Tile38KNearestLimit100Request 608µs ± 1% 606µs ± 2% -0.35% (p=0.030 n=19+22) [Geo mean] 522ms 524ms +0.41% https://perf.golang.org/search?q=upload:20200606.4 Change-Id: I8b331f310dbfaba0468035f207467c8403005bf5 Reviewed-on: https://go-review.googlesource.com/c/go/+/236817 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-05-14runtime: make maxOffAddr reflect the actual address space upper boundMichael Anthony Knyszek
Currently maxOffAddr is defined in terms of the whole 64-bit address space, assuming that it's all supported, by using ^uintptr(0) as the maximal address in the offset space. In reality, the maximal address in the offset space is (1<<heapAddrBits)-1 because we don't have more than that actually available to us on a given platform. On most platforms this is fine, because arenaBaseOffset is just connecting two segments of address space, but on AIX we use it as an actual offset for the starting address of the available address space, which is limited. This means using ^uintptr(0) as the maximal address in the offset address space causes wrap-around, especially when we just want to represent a range approximately like [addr, infinity), which today we do by using maxOffAddr. To fix this, we define maxOffAddr more appropriately, in terms of (1<<heapAddrBits)-1. This change also redefines arenaBaseOffset to not be the negation of the virtual address corresponding to address zero in the virtual address space, but instead directly as the virtual address corresponding to zero. This matches the existing documentation more closely and makes the logic around arenaBaseOffset decidedly simpler, especially when trying to reason about its use on AIX. Fixes #38966. Change-Id: I1336e5036a39de846f64cc2d253e8536dee57611 Reviewed-on: https://go-review.googlesource.com/c/go/+/233497 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-05-07runtime: avoid overflow from linearAllocMichael Anthony Knyszek
Currently linearAlloc manages an exclusive "end" address for the top of its reserved space. While unlikely for a linearAlloc to be allocated with an "end" address hitting the top of the address space, it is possible and could lead to overflow. Avoid overflow by chopping off the last byte from the linearAlloc if it's bumping up against the top of the address space defensively. In practice, this means that if 32-bit platforms map the top of the address space and use the linearAlloc to acquire arenas, the top arena will not be usable. Fixes #35954. Change-Id: I512cddcd34fd1ab15cb6ca92bbf899fc1ef22ff6 Reviewed-on: https://go-review.googlesource.com/c/go/+/231338 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-04-27runtime: add new mcentral implementationMichael Anthony Knyszek
Currently mcentral is implemented as a couple of linked lists of spans protected by a lock. Unfortunately this design leads to significant lock contention. The span ownership model is also confusing and complicated. In-use spans jump between being owned by multiple sources, generally some combination of a gcSweepBuf, a concurrent sweeper, an mcentral or an mcache. So first to address contention, this change replaces those linked lists with gcSweepBufs which have an atomic fast path. Then, we change up the ownership model: a span may be simultaneously owned only by an mcentral and the page reclaimer. Otherwise, an mcentral (which now consists of sweep bufs), a sweeper, or an mcache are the sole owners of a span at any given time. This dramatically simplifies reasoning about span ownership in the runtime. As a result of this new ownership model, sweeping is now driven by walking over the mcentrals rather than having its own global list of spans. Because we no longer have a global list and we traditionally haven't used the mcentrals for large object spans, we no longer have anywhere to put large objects. So, this change also makes it so that we keep large object spans in the appropriate mcentral lists. In terms of the static lock ranking, we add the spanSet spine locks in pretty much the same place as the mcentral locks, since they have the potential to be manipulated both on the allocation and sweep paths, like the mcentral locks. This new implementation is turned on by default via a feature flag called go115NewMCentralImpl. Benchmark results for 1 KiB allocation throughput (5 runs each): name \ MiB/s go113 go114 gotip gotip+this-patch AllocKiB-1 1.71k ± 1% 1.68k ± 1% 1.59k ± 2% 1.71k ± 1% AllocKiB-2 2.46k ± 1% 2.51k ± 1% 2.54k ± 1% 2.93k ± 1% AllocKiB-4 4.27k ± 1% 4.41k ± 2% 4.33k ± 1% 5.01k ± 2% AllocKiB-8 4.38k ± 3% 5.24k ± 1% 5.46k ± 1% 8.23k ± 1% AllocKiB-12 4.38k ± 3% 4.49k ± 1% 5.10k ± 1% 10.04k ± 0% AllocKiB-16 4.31k ± 1% 4.14k ± 3% 4.22k ± 0% 10.42k ± 0% AllocKiB-20 4.26k ± 1% 3.98k ± 1% 4.09k ± 1% 10.46k ± 3% AllocKiB-24 4.20k ± 1% 3.97k ± 1% 4.06k ± 1% 10.74k ± 1% AllocKiB-28 4.15k ± 0% 4.00k ± 0% 4.20k ± 0% 10.76k ± 1% Fixes #37487. Change-Id: I92d47355acacf9af2c41bf080c08a8c1638ba210 Reviewed-on: https://go-review.googlesource.com/c/go/+/221182 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-04-21runtime: add bitmap-based markrootSpans implementationMichael Anthony Knyszek
Currently markrootSpans, the scanning routine which scans span specials (particularly finalizers) as roots, uses sweepSpans to shard work and find spans to mark. However, as part of a future CL to change span ownership and how mcentral works, we want to avoid having markrootSpans use the sweep bufs to find specials, so in this change we introduce a new mechanism. Much like for the page reclaimer, we set up a per-page bitmap where the first page for a span is marked if the span contains any specials, and unmarked if it has no specials. This bitmap is updated by addspecial, removespecial, and during sweeping. markrootSpans then shards this bitmap into mark work and markers iterate over the bitmap looking for spans with specials to mark. Unlike the page reclaimer, we don't need to use the pageInUse bits because having a special implies that a span is in-use. While in terms of computational complexity this design is technically worse, because it needs to iterate over the mapped heap, in practice this iteration is very fast (we can skip over large swathes of the heap very quickly) and we only look at spans that have any specials at all, rather than having to touch each span. This new implementation of markrootSpans is behind a feature flag called go115NewMarkrootSpans. Updates #37487. Change-Id: I8ea07b6c11059f6d412fe419e0ab512d989377b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/221178 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-04-17runtime: use mcache0 if no P in profileallocIan Lance Taylor
A case that I missed in CL 205239: profilealloc can be called at program startup if GOMAXPROCS is large enough. Fixes #38474 Change-Id: I2f089fc6ec00c376680e1c0b8a2557b62789dd7f Reviewed-on: https://go-review.googlesource.com/c/go/+/228420 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-04-07runtime: static lock ranking for the runtime (enabled by GOEXPERIMENT)Dan Scales
I took some of the infrastructure from Austin's lock logging CR https://go-review.googlesource.com/c/go/+/192704 (with deadlock detection from the logs), and developed a setup to give static lock ranking for runtime locks. Static lock ranking establishes a documented total ordering among locks, and then reports an error if the total order is violated. This can happen if a deadlock happens (by acquiring a sequence of locks in different orders), or if just one side of a possible deadlock happens. Lock ordering deadlocks cannot happen as long as the lock ordering is followed. Along the way, I found a deadlock involving the new timer code, which Ian fixed via https://go-review.googlesource.com/c/go/+/207348, as well as two other potential deadlocks. See the constants at the top of runtime/lockrank.go to show the static lock ranking that I ended up with, along with some comments. This is great documentation of the current intended lock ordering when acquiring multiple locks in the runtime. I also added an array lockPartialOrder[] which shows and enforces the current partial ordering among locks (which is embedded within the total ordering). This is more specific about the dependencies among locks. I don't try to check the ranking within a lock class with multiple locks that can be acquired at the same time (i.e. check the ranking when multiple hchan locks are acquired). Currently, I am doing a lockInit() call to set the lock rank of most locks. Any lock that is not otherwise initialized is assumed to be a leaf lock (a very high rank lock), so that eliminates the need to do anything for a bunch of locks (including all architecture-dependent locks). For two locks, root.lock and notifyList.lock (only in the runtime/sema.go file), it is not as easy to do lock initialization, so instead, I am passing the lock rank with the lock calls. For Windows compilation, I needed to increase the StackGuard size from 896 to 928 because of the new lock-rank checking functions. Checking of the static lock ranking is enabled by setting GOEXPERIMENT=staticlockranking before doing a run. To make sure that the static lock ranking code has no overhead in memory or CPU when not enabled by GOEXPERIMENT, I changed 'go build/install' so that it defines a build tag (with the same name) whenever any experiment has been baked into the toolchain (by checking Expstring()). This allows me to avoid increasing the size of the 'mutex' type when static lock ranking is not enabled. Fixes #38029 Change-Id: I154217ff307c47051f8dae9c2a03b53081acd83a Reviewed-on: https://go-review.googlesource.com/c/go/+/207619 Reviewed-by: Dan Scales <danscales@google.com> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Dan Scales <danscales@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-20runtime: use divRoundUpAustin Clements
There are a handful of places where the runtime wants to round up the result of a division. We just introduced a helper to do this. This CL replaces all of the hand-coded round-ups (that I could find) with this helper. Change-Id: I465d152157ff0f3cad40c0aa57491e4f2de510ad Reviewed-on: https://go-review.googlesource.com/c/go/+/224385 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-24runtime: remove mcache field from mIan Lance Taylor
Having an mcache field in both m and p is confusing, so remove it from m. Always use mcache field from p. Use new variable mcache0 during bootstrap. Change-Id: If2cba9f8bb131d911d512b61fd883a86cf62cc98 Reviewed-on: https://go-review.googlesource.com/c/go/+/205239 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-02-05runtime: rewrite a comment in malloc.goJerrin Shaji George
This commit changes the wording of a comment in malloc.go that describes how span objects are zeroed to make it more clear. Change-Id: I07722df1e101af3cbf8680ad07437d4a230b0168 GitHub-Last-Rev: 0e909898c709a9119cea7dbd80c25d9d7a73e22b GitHub-Pull-Request: golang/go#37008 Reviewed-on: https://go-review.googlesource.com/c/go/+/217618 Reviewed-by: Austin Clements <austin@google.com>
2020-01-19runtime: add support for linux/riscv64Joel Sing
Based on riscv-go port. Updates #27532 Change-Id: If522807a382130be3c8d40f4b4c1131d1de7c9e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/204632 Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-16runtime: add arenaBaseOffset on aix/ppc64Clément Chigot
On AIX, addresses returned by mmap are between 0x0a00000000000000 and 0x0afffffffffffff. The previous solution to handle these large addresses was to increase the arena size up to 60 bits addresses, cf CL 138736. However, with the new page allocator, the 60bit heap addresses are causing huge memory allocations, especially by (s *pageAlloc).init. mmap and munmap syscalls dealing with these allocations are reducing performances of every Go programs. In order to avoid these allocations, arenaBaseOffset is set to 0x0a00000000000000 and heap addresses are on 48bit, as others operating systems. Updates: #35451 Change-Id: Ice916b8578f76703428ec12a82024147a7592bc0 Reviewed-on: https://go-review.googlesource.com/c/go/+/206841 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-08runtime: remove unnecessary large parameter to mheap_.allocMichael Anthony Knyszek
mheap_.alloc currently accepts both a spanClass and a "large" parameter indicating whether the allocation is large. These are redundant, since spanClass.sizeclass() == 0 is an equivalent way to determine this and is already used in mheap_.alloc. There are no places in the runtime where the size class could be non-zero and large == true. Updates #35112. Change-Id: Ie66facf8f0faca6f4cd3d20a8ac4bc259e11823d Reviewed-on: https://go-review.googlesource.com/c/go/+/196639 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-11-08runtime: define maximum supported physical page and huge page sizesMichael Anthony Knyszek
This change defines a maximum supported physical and huge page size in the runtime based on the new page allocator's implementation, and uses them where appropriate. Furthemore, if the system exceeds the maximum supported huge page size, we simply ignore it silently. It also fixes a huge-page-related test which is only triggered by a condition which is definitely wrong. Finally, it adds a few TODOs related to code clean-up and supporting larger huge page sizes. Updates #35112. Fixes #35431. Change-Id: Ie4348afb6bf047cce2c1433576d1514720d8230f Reviewed-on: https://go-review.googlesource.com/c/go/+/205937 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 21:39:09 +0000