aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/mfinal.go
AgeCommit message (Collapse)Author
2025-06-13runtime: remove unused unique.runtime_blockUntilEmptyFinalizerQueueCherry Mui
It was added in CL 650256, and then the use in the unique package was removed in CL 650697. Change-Id: Id95f5dff7e11a2dc3eb544fda2586a305d3d91ab Reviewed-on: https://go-review.googlesource.com/c/go/+/681476 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-05-20runtime: report finalizer and cleanup queue length with checkfinalizer>0Michael Anthony Knyszek
This change adds tracking for approximate finalizer and cleanup queue lengths. These lengths are reported once every GC cycle as a single line printed to stderr when GODEBUG=checkfinalizer>0. This change lays the groundwork for runtime/metrics metrics to produce the same values. For #72948. For #72950. Change-Id: I081721238a0fc4c7e5bee2dbaba6cfb4120d1a33 Reviewed-on: https://go-review.googlesource.com/c/go/+/671437 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-05-20runtime: annotate checkfinalizers reports with source and type infoMichael Anthony Knyszek
This change adds a new special kind called CheckFinalizer which is used to annotate finalizers and cleanups with extra information about where that cleanup or finalizer came from. For #72949. Change-Id: I3c1ace7bd580293961b7f0ea30345a6ce956d340 Reviewed-on: https://go-review.googlesource.com/c/go/+/662135 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-05-08runtime: schedule cleanups across multiple goroutinesMichael Anthony Knyszek
This change splits the finalizer and cleanup queues and implements a new lock-free blocking queue for cleanups. The basic design is as follows: The cleanup queue is organized in fixed-sized blocks. Individual cleanup functions are queued, but only whole blocks are dequeued. Enqueuing cleanups places them in P-local cleanup blocks. These are flushed to the full list as they get full. Cleanups can only be enqueued by an active sweeper. Dequeuing cleanups always dequeues entire blocks from the full list. Cleanup blocks can be dequeued and executed at any time. The very last active sweeper in the sweep phase is responsible for flushing all local cleanup blocks to the full list. It can do this without any synchronization because the next GC can't start yet, so we can be very certain that nobody else will be accessing the local blocks. Cleanup blocks are stored off-heap because the need to be allocated by the sweeper, which is called from heap allocation paths. As a result, the GC treats cleanup blocks as roots, just like finalizer blocks. Flushes to the full list signal to the scheduler that cleanup goroutines should be awoken. Every time the scheduler goes to wake up a cleanup goroutine and there were more signals than goroutines to wake, it then forwards this signal to runtime.AddCleanup, so that it creates another goroutine the next time it is called, up to gomaxprocs goroutines. The signals here are a little convoluted, but exist because the sweeper and the scheduler cannot safely create new goroutines. For #71772. For #71825. Change-Id: Ie839fde2b67e1b79ac1426be0ea29a8d923a62cc Reviewed-on: https://go-review.googlesource.com/c/go/+/650697 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2025-05-06unique: use a bespoke canonicalization map and runtime.AddCleanupMichael Anthony Knyszek
This change moves the unique package away from using a concurrent map and instead toward a bespoke concurrent canonicalization map. The map holds all its keys weakly, though keys may be looked up by value. The result is the strong pointer for the canonical value. Entries in the map are automatically cleaned up once the canonical reference no longer exists. Why do this? There's a problem with the current implementation when it comes to chains of unique.Handle: because the unique map will have a unique.Handle stored in its keys, each nested handle must be cleaned up 1 GC at a time. It takes N GC cycles, at minimum, to clean up a nested chain of N handles. This implementation, where the *only* value in the set is weakly-held, does not have this problem. The entire chain is dropped at once. The canon map implementation is a stripped-down version of HashTrieMap. The weak set implementation also has lower memory overheads by virtue of the fact that keys are all stored weakly. Whereas the previous map had both a T and a weak.Pointer[T], this *only* has a weak.Pointer[T]. The canonicalization map is a better abstraction overall and dramatically simplifies the unique.Make code. While we're here, delete the background goroutine and switch to runtime.AddCleanup. This is a step toward fixing #71772. We still need some kind of back-pressure mechanism, which will be implemented in a follow-up CL. For #71772. Fixes #71846. Change-Id: I5b2ee04ebfc7f6dd24c2c4a959dd0f6a8af24ca4 Reviewed-on: https://go-review.googlesource.com/c/go/+/650256 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2025-04-23runtime: move some malloc constants to internal/runtime/gcMichael Anthony Knyszek
These constants are needed by some future generator programs. Change-Id: I5dccd009cbb3b2f321523bc0d8eaeb4c82e5df81 Reviewed-on: https://go-review.googlesource.com/c/go/+/655276 Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-24runtime: rename runfinq to runFinalizersAndCleanupsMichael Pratt
Users see this frame in tracebacks and it serves as a hint that what is running here is a finalizer or cleanup. But runfinq is a rather dense name. We can give it a more obvious name to help users realize what it is. For #73011. Change-Id: I6a6a636ce9a493fd00d4b4c60c23f2b1c96d3568 Reviewed-on: https://go-review.googlesource.com/c/go/+/660296 Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-02-19runtime: minor mfinal.go code cleanupMichael Anthony Knyszek
This change moves finBlockSize into mfinal.go and renames finblock to finBlock. Change-Id: I20a0bc3907e7b028a2caa5d2fe8cf3f76332c871 Reviewed-on: https://go-review.googlesource.com/c/go/+/650695 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-12-07runtime: improve AddCleanup documentationAustin Clements
Steer people from SetFinalizer to AddCleanup. Address some of the *non*-constraints on AddCleanup. Add some of the subtlety from the SetFinalizer documentation to the AddCleanup documentation. Updates #67535. Updates #70425. Change-Id: I8d13b756ca866051b8a5c19327fd5a76f5e0f3d7 Reviewed-on: https://go-review.googlesource.com/c/go/+/634318 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Austin Clements <austin@google.com>
2024-11-16runtime: implement AddCleanupCarlos Amedee
This change introduces AddCleanup to the runtime package. AddCleanup attaches a cleanup function to an pointer to an object. The Stop method on Cleanups will be implemented in a followup CL. AddCleanup is intended to be an incremental improvement over SetFinalizer and will result in SetFinalizer being deprecated. For #67535 Change-Id: I99645152e3fdcee85fcf42a4f312c6917e8aecb1 Reviewed-on: https://go-review.googlesource.com/c/go/+/627695 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-11-15runtime: validate all calls to SetFinalizerCarlos Amedee
This change moves the check for a change in the memory management system to after the SetFinalizer parameters have been validated. Moving the check ensures that invalid parameters will never pass the validation checks. Change-Id: I9f1d3454f891f7b147c0d86b6720297172e08ef9 Reviewed-on: https://go-review.googlesource.com/c/go/+/625035 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-07-23runtime,internal: move runtime/internal/sys to internal/runtime/sysDavid Chase
Cleanup and friction reduction For #65355. Change-Id: Ia14c9dc584a529a35b97801dd3e95b9acc99a511 Reviewed-on: https://go-review.googlesource.com/c/go/+/600436 Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
2024-06-07runtime: add a note on SetFinalizer doc about method receiverAnuraag Agrawal
A method receiver can be collected during the execution of that method. This does make sense when thinking about how the GC would work, but also seems not very obvious, so a point in the docs can increase the chance of avoiding issues caused by missing KeepAlive of method receivers. Change-Id: I6817237dd022b5729dbdcda1b9f70c7059605575 GitHub-Last-Rev: 878bf3fde7b4ed66be0302ffdd5e704b4cd70a6c GitHub-Pull-Request: golang/go#67777 Reviewed-on: https://go-review.googlesource.com/c/go/+/589735 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-04-09runtime: remove the allocheaders GOEXPERIMENTMichael Anthony Knyszek
This change removes the allocheaders, deleting all the old code and merging mbitmap_allocheaders.go back into mbitmap.go. This change also deletes the SetType benchmarks which were already broken in the new GOEXPERIMENT (it's harder to set up than before). We weren't really watching these benchmarks at all, and they don't provide additional test coverage. Change-Id: I135497201c3259087c5cd3722ed3fbe24791d25d Reviewed-on: https://go-review.googlesource.com/c/go/+/567200 Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-04-02all: use kind* of abiqiulaidongfeng
For #59670 Change-Id: Id66e102f13e529dd041b68ce869026a56f0a1b9b GitHub-Last-Rev: 43aa9376f72bc02a9d86518cdc99494a6b2f8573 GitHub-Pull-Request: golang/go#65564 Reviewed-on: https://go-review.googlesource.com/c/go/+/562298 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Austin Clements <austin@google.com>
2024-03-25runtime: migrate internal/atomic to internal/runtimeAndy Pan
For #65355 Change-Id: I65dd090fb99de9b231af2112c5ccb0eb635db2be Reviewed-on: https://go-review.googlesource.com/c/go/+/560155 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ibrahim Bazoka <ibrahimbazoka729@gmail.com> Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
2024-03-04runtime: use .Pointers() instead of manual checkingPouriya
Change-Id: Ib78c1513616089f4942297cd17212b1b11871fd5 GitHub-Last-Rev: f97fe5b5bffffe25dc31de7964588640cb70ec41 GitHub-Pull-Request: golang/go#65819 Reviewed-on: https://go-review.googlesource.com/c/go/+/565515 Reviewed-by: Jorropo <jorropo.pgm@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-11-21sync: do not unnecessarily keep alive functions wrapped by ↵Carlo Alberto Ferraris
Once(Func|Value|Values) The function passed to OnceFunc/OnceValue/OnceValues may transitively keep more allocations alive. As the passed function is guaranteed to be called at most once, it is safe to drop it after the first call is complete. This avoids keeping the passed function (and anything it transitively references) alive until the returned function is GCed. Change-Id: I2faf397b481d2f693ab3aea8e2981b02adbc7a21 Reviewed-on: https://go-review.googlesource.com/c/go/+/481515 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: qiulaidongfeng <2645477756@qq.com>
2023-11-17runtime: put allocation headers back at the start the objectMichael Anthony Knyszek
A persistent performance regression was discovered on perf.golang.org/dashboard and this was narrowed down to the switch to footers. Using allocation headers instead resolves the issue. The benchmark results for allocation footers weren't realistic, because they were performed on a machine with enough L3 cache that it completely hid the additional cache miss introduced by allocation footers. This means that in some corner cases the Go runtime may no longer allocate 16-byte aligned memory. Note however that this property was *mostly* incidental and never guaranteed in any documentation. Allocation headers were tested widely within Google and no issues were found, so we're fairly confident that this will not affect very many users. Nonetheless, by Hyrum's Law some code might depend on it. A follow-up change will add a GODEBUG flag that ensures 16 byte alignment at the potential cost of some additional memory use. Users experiencing both a performance regression and an alignment issue can also disable the GOEXPERIMENT at build time. This reverts commit 1e250a219900651dad27f29eab0877eee4afd5b9. Change-Id: Ia7d62a9c60d1773c8b6d33322ee33a80ef814943 Reviewed-on: https://go-review.googlesource.com/c/go/+/543255 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-11-09runtime: make alloc headers footers insteadMichael Anthony Knyszek
The previous CL in this series (CL 437955) adds the allocation headers experiment. However, this experiment puts the headers at the beginning of each allocation, which decreases the default allocator alignment that users can rely upon. Historically, Go's memory allocator has implicitly provided 16-byte alignment (except for sizes where it doesn't make sense, like 8 or 24 bytes), so it's not unlikely that users are depending on it. It also complicates other changes that want higher alignment. For example, the sync/atomic.Uint64Pair proposal would (hypothetically; it's not yet accepted) introduce a type with 16-byte alignment. The malloc fast path will require extra code to consider alignment and will waste memory for any value containing such a type. This change moves the allocation header to the end of the span's allocation slot instead of the beginning. This means worse locality for the GC when scanning, but it's still an overall win. It also means that objects will still have the 16-byte alignment we've provided thus far. This is broken out in a separate change just becauase it ended up that way during development. But I've chosen to leave it this way in case we want to try and move allocation headers to the front of objects again. Below are the benchmark results of this CL series, comparing the performance of this CL with GOEXPERIMENT=allocheaders vs. without this CL series. name old time/op new time/op delta BiogoIgor 12.5s ± 0% 12.4s ± 2% ~ (p=0.079 n=9+10) BiogoKrishna 12.8s ±10% 12.4s ±10% ~ (p=0.182 n=9+10) BleveIndexBatch100 4.54s ± 3% 4.60s ± 3% ~ (p=0.050 n=9+9) EtcdPut 21.1ms ± 2% 21.3ms ± 4% ~ (p=0.669 n=7+10) EtcdSTM 107ms ± 3% 108ms ± 2% ~ (p=0.497 n=9+10) GoBuildKubelet 34.1s ± 3% 33.1s ± 2% -3.08% (p=0.000 n=10+10) GoBuildKubeletLink 7.94s ± 2% 7.95s ± 2% ~ (p=0.631 n=10+10) GoBuildIstioctl 33.2s ± 1% 31.7s ± 0% -4.37% (p=0.000 n=9+9) GoBuildIstioctlLink 8.07s ± 1% 8.05s ± 1% ~ (p=0.356 n=9+10) GoBuildFrontend 12.1s ± 0% 11.5s ± 1% -4.43% (p=0.000 n=8+10) GoBuildFrontendLink 1.20s ± 2% 1.20s ± 2% ~ (p=0.905 n=9+10) GopherLuaKNucleotide 19.9s ± 0% 19.5s ± 1% -1.95% (p=0.000 n=9+10) MarkdownRenderXHTML 194ms ± 5% 194ms ± 2% ~ (p=0.931 n=9+9) Tile38QueryLoad 518µs ± 1% 508µs ± 1% -1.93% (p=0.000 n=9+8) name old average-RSS-bytes new average-RSS-bytes delta BiogoIgor 66.2MB ± 3% 65.6MB ± 1% ~ (p=0.156 n=10+9) BiogoKrishna 4.34GB ± 2% 4.34GB ± 1% ~ (p=0.315 n=10+9) BleveIndexBatch100 189MB ± 3% 186MB ± 3% ~ (p=0.052 n=10+10) EtcdPut 105MB ± 5% 107MB ± 6% ~ (p=0.579 n=10+10) EtcdSTM 92.1MB ± 5% 93.2MB ± 4% ~ (p=0.353 n=10+10) GoBuildKubelet 2.07GB ± 1% 2.07GB ± 1% ~ (p=0.436 n=10+10) GoBuildIstioctl 1.44GB ± 1% 1.46GB ± 1% +0.96% (p=0.001 n=10+10) GoBuildFrontend 522MB ± 1% 512MB ± 2% -1.98% (p=0.000 n=10+10) GopherLuaKNucleotide 37.4MB ± 5% 36.4MB ± 4% -2.53% (p=0.035 n=10+10) MarkdownRenderXHTML 21.2MB ± 1% 20.9MB ± 3% -1.53% (p=0.003 n=8+10) Tile38QueryLoad 6.39GB ± 2% 6.24GB ± 2% -2.40% (p=0.000 n=10+10) name old peak-RSS-bytes new peak-RSS-bytes delta BiogoIgor 88.5MB ± 4% 88.4MB ± 3% ~ (p=0.971 n=10+10) BiogoKrishna 4.48GB ± 0% 4.42GB ± 0% -1.49% (p=0.000 n=10+10) BleveIndexBatch100 268MB ± 3% 265MB ± 4% ~ (p=0.315 n=9+10) EtcdPut 147MB ± 9% 146MB ± 5% ~ (p=0.853 n=10+10) EtcdSTM 119MB ± 6% 120MB ± 5% ~ (p=0.796 n=10+10) GopherLuaKNucleotide 43.1MB ±17% 40.7MB ±12% ~ (p=0.075 n=10+10) MarkdownRenderXHTML 21.2MB ± 1% 21.1MB ± 3% ~ (p=0.511 n=9+10) Tile38QueryLoad 6.65GB ± 4% 6.52GB ± 2% -1.93% (p=0.009 n=10+10) name old peak-VM-bytes new peak-VM-bytes delta BiogoIgor 1.33GB ± 0% 1.33GB ± 0% -0.16% (p=0.000 n=10+10) BiogoKrishna 5.77GB ± 0% 5.69GB ± 0% -1.23% (p=0.000 n=10+10) BleveIndexBatch100 2.62GB ± 0% 2.61GB ± 0% -0.13% (p=0.000 n=7+10) EtcdPut 12.1GB ± 0% 12.1GB ± 0% ~ (p=0.160 n=8+10) EtcdSTM 12.1GB ± 0% 12.1GB ± 0% -0.02% (p=0.000 n=10+10) GopherLuaKNucleotide 1.26GB ± 0% 1.26GB ± 0% -0.09% (p=0.000 n=10+10) MarkdownRenderXHTML 1.26GB ± 0% 1.26GB ± 0% -0.08% (p=0.000 n=10+10) Tile38QueryLoad 7.89GB ± 4% 7.76GB ± 1% -1.70% (p=0.008 n=10+8) name old p50-latency-ns new p50-latency-ns delta EtcdPut 20.1M ± 5% 20.2M ± 4% ~ (p=0.529 n=10+10) EtcdSTM 79.8M ± 4% 79.9M ± 4% ~ (p=0.971 n=10+10) Tile38QueryLoad 215k ± 1% 210k ± 3% -2.04% (p=0.021 n=8+10) name old p90-latency-ns new p90-latency-ns delta EtcdPut 31.9M ± 6% 32.0M ± 7% ~ (p=0.780 n=9+10) EtcdSTM 220M ± 6% 220M ± 2% ~ (p=1.000 n=10+10) Tile38QueryLoad 622k ± 2% 646k ± 2% +3.83% (p=0.000 n=10+10) name old p99-latency-ns new p99-latency-ns delta EtcdPut 47.6M ±32% 51.4M ±28% ~ (p=0.529 n=10+10) EtcdSTM 452M ± 2% 457M ± 2% ~ (p=0.182 n=9+10) Tile38QueryLoad 5.04M ± 2% 4.91M ± 3% -2.56% (p=0.001 n=9+9) name old ops/s new ops/s delta EtcdPut 46.1k ± 2% 45.7k ± 4% ~ (p=0.475 n=7+10) EtcdSTM 9.18k ± 5% 9.20k ± 3% ~ (p=0.971 n=10+10) Tile38QueryLoad 17.4k ± 1% 17.7k ± 1% +1.97% (p=0.000 n=9+8) Change-Id: I637f48fb9e8c181912db785ae9186d7f16769870 Reviewed-on: https://go-review.googlesource.com/c/go/+/537886 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-11-09runtime: implement experiment to replace heap bitmap with alloc headersMichael Anthony Knyszek
This change replaces the 1-bit-per-word heap bitmap for most size classes with allocation headers for objects that contain pointers. The header consists of a single pointer to a type. All allocations with headers are treated as implicitly containing one or more instances of the type in the header. As the name implies, headers are usually stored as the first word of an object. There are two additional exceptions to where headers are stored and how they're used. Objects smaller than 512 bytes do not have headers. Instead, a heap bitmap is reserved at the end of spans for objects of this size. A full word of overhead is too much for these small objects. The bitmap is of the same format of the old bitmap, minus the noMorePtrs bits which are unnecessary. All the objects <512 bytes have a bitmap less than a pointer-word in size, and that was the granularity at which noMorePtrs could stop scanning early anyway. Objects that are larger than 32 KiB (which have their own span) have their headers stored directly in the span, to allow power-of-two-sized allocations to not spill over into an extra page. The full implementation is behind GOEXPERIMENT=allocheaders. The purpose of this change is performance. First and foremost, with headers we no longer have to unroll pointer/scalar data at allocation time for most size classes. Small size classes still need some unrolling, but their bitmaps are small so we can optimize that case fairly well. Larger objects effectively have their pointer/scalar data unrolled on-demand from type data, which is much more compactly represented and results in less TLB pressure. Furthermore, since the headers are usually right next to the object and where we're about to start scanning, we get an additional temporal locality benefit in the data cache when looking up type metadata. The pointer/scalar data is now effectively unrolled on-demand, but it's also simpler to unroll than before; that unrolled data is never written anywhere, and for arrays we get the benefit of retreading the same data per element, as opposed to looking it up from scratch for each pointer-word of bitmap. Lastly, because we no longer have a heap bitmap that spans the entire heap, there's a flat 1.5% memory use reduction. This is balanced slightly by some objects possibly being bumped up a size class, but most objects are not tightly optimized to size class sizes so there's some memory to spare, making the header basically free in those cases. See the follow-up CL which turns on this experiment by default for benchmark results. (CL 538217.) Change-Id: I4c9034ee200650d06d8bdecd579d5f7c1bbf1fc5 Reviewed-on: https://go-review.googlesource.com/c/go/+/437955 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-08runtime: add available godoc linkcui fliter
Change-Id: Ifb4844efddcb0369b0302eeab72394eeaf5c8072 Reviewed-on: https://go-review.googlesource.com/c/go/+/540022 Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: shuang cui <imcusg@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-06cmd/compile: pass only the itab to the assertI2I-style functionsKeith Randall
It is currently slightly better to pass the whole interface to these functions, so that we don't need to spill/restore the data word across the function call. I'm adding a cache in front of these calls, which means we'll no longer need a spill/restore in the common case, so it is better to just pass the itab word. It also makes unifying the logic between I2I and I2I2 versions easier. Change-Id: I3c3e9fbb1e54890482840d76a1df79f4325bb5bc Reviewed-on: https://go-review.googlesource.com/c/go/+/528075 Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2023-09-18all: clean unnecessary castsJes Cok
Run 'unconvert -safe -apply' (https://github.com/mdempsky/unconvert) Change-Id: I24b7cd7d286cddce86431d8470d15c5f3f0d1106 GitHub-Last-Rev: 022e75384c08bb899a8951ba0daffa0f2e14d5a7 GitHub-Pull-Request: golang/go#62662 Reviewed-on: https://go-review.googlesource.com/c/go/+/528696 Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-19runtime: replace raw traceEv with traceBlockReason in goparkMichael Anthony Knyszek
This change adds traceBlockReason which leaks fewer implementation details of the tracer to the runtime. Currently, gopark is called with an explicit trace event, but this leaks details about trace internals throughout the runtime. This change will make it easier to change out the trace implementation. Change-Id: Id633e1704d2c8838c6abd1214d9695537c4ac7db Reviewed-on: https://go-review.googlesource.com/c/go/+/494185 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-19runtime: implement Pinner API for object pinningSven Anderson
Some C APIs require the use or structures that contain pointers to buffers (iovec, io_uring, ...). The pointer passing rules would require that these buffers are allocated in C memory and to process this data with Go libraries it would need to be copied. In order to provide a zero-copy way to use these C APIs, this CL implements a Pinner API that allows to pin Go objects, which guarantees that the garbage collector does not move these objects while pinned. This allows to relax the pointer passing rules so that pinned pointers can be stored in C allocated memory or can be contained in Go memory that is passed to C functions. The Pin() method accepts pointers to objects of any type and unsafe.Pointer. Slices and arrays can be pinned by calling Pin() with the pointer to the first element. Pinning of maps is not supported. If the GC collects unreachable Pinner holding pinned objects it panics. If Pin() is called with the other non-pointer types it panics as well. Performance considerations: This change has no impact on execution time on existing code, because checks are only done in code paths, that would panic otherwise. The memory footprint on existing code is one pointer per memory span. Fixes: #46787 Signed-off-by: Sven Anderson <sven@anderson.de> Change-Id: I110031fe789b92277ae45a9455624687bd1c54f2 Reviewed-on: https://go-review.googlesource.com/c/go/+/367296 Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-11runtime: move per-type types to internal/abiDavid Chase
Change-Id: I1f031f0f83a94bebe41d3978a91a903dc5bcda66 Reviewed-on: https://go-review.googlesource.com/c/go/+/489276 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-11runtime: redefine _type to abi.Type; add rtype for methods.David Chase
Change-Id: I1c478b704d84811caa209006c657dda82d9c4cf9 Reviewed-on: https://go-review.googlesource.com/c/go/+/488435 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
2023-05-05internal/abi: refactor (basic) type struct into one definitionDavid Chase
This touches a lot of files, which is bad, but it is also good, since there's N copies of this information commoned into 1. The new files in internal/abi are copied from the end of the stack; ultimately this will all end up being used. Change-Id: Ia252c0055aaa72ca569411ef9f9e96e3d610889e Reviewed-on: https://go-review.googlesource.com/c/go/+/462995 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2022-11-18all: add missing periods in commentscui fliter
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29 Reviewed-on: https://go-review.googlesource.com/c/go/+/449757 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Joedian Reid <joedian@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-15runtime: clarify finalizer semantics for tiny objectsChangkun Ou
This change clarifies that a finalizer is not guaranteed to run, not only for zero bytes objects but also tiny objects (< 16bytes). Fixes #46827 Change-Id: I193e77f6f024c79110604f86bcb1a28b16cf98ec Reviewed-on: https://go-review.googlesource.com/c/go/+/337391 Run-TryBot: Changkun Ou <mail@changkun.de> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-10-12arena: add experimental arena packageMichael Anthony Knyszek
This change adds the arenas package and a function to reflect for allocating from an arena via reflection, but all the new API is placed behind a GOEXPERIMENT. For #51317. Change-Id: I026d46294e26ab386d74625108c19a0024fbcedc Reviewed-on: https://go-review.googlesource.com/c/go/+/423361 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-05runtime: refactor finalizer goroutine statusLeonard Wang
Use an atomic.Uint32 to represent the state of finalizer goroutine. fingStatus will only be changed to fingWake in non fingWait state, so it is safe to set fingRunningFinalizer status in runfinq. name old time/op new time/op delta Finalizer-8 592µs ± 4% 561µs ± 1% -5.22% (p=0.000 n=10+10) FinalizerRun-8 694ns ± 6% 675ns ± 7% ~ (p=0.059 n=9+8) Change-Id: I7e4da30cec98ce99f7d8cf4c97f933a8a2d1cae1 Reviewed-on: https://go-review.googlesource.com/c/go/+/400134 Reviewed-by: Joedian Reid <joedian@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-08-19runtime: add and use runtime/internal/sys.NotInHeapCuong Manh Le
Updates #46731 Change-Id: Ic2208c8bb639aa1e390be0d62e2bd799ecf20654 Reviewed-on: https://go-review.googlesource.com/c/go/+/421878 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2022-08-04runtime: add mayAcquire annotation for finlockAustin Clements
We're missing lock edges to finlock that happen only rarely. Anything that calls mallocgc can potentially trigger sweeping, which can potentially queue a finalizer, which acquires finlock. While this can happen on any malloc, it happens relatively rarely, so we simply haven't seen some of the lock edges that could happen. Add a mayAcquire annotation to mallocgc to capture the possibility of acquiring finlock. With this change, we add "fin" to the set of "malloc" locks. Several of these edges were already there, but not quite all of them. This was found by inspecting the rank graph for things that didn't make sense. For #53789. Change-Id: Idc10ce6f250596b0c07ba07ac93f2198fb38c22b Reviewed-on: https://go-review.googlesource.com/c/go/+/418717 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-06runtime, sync, sync/atomic: document happens-before guaranteesRuss Cox
A few of these are copied from the memory model doc. Many are entirely new, following discussion on #47141. See https://research.swtch.com/gomm for background. The rule we are establishing is that each type that is meant to help synchronize a Go program should document its happens-before guarantees. For #50859. Change-Id: I947c40639b263abe67499fa74f68711a97873a39 Reviewed-on: https://go-review.googlesource.com/c/go/+/381316 Auto-Submit: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-05-03runtime: decrease STW pause for goroutine profileRhys Hiltner
The goroutine profile needs to stop the world to get a consistent snapshot of all goroutines in the app. Leaving the world stopped while iterating over allgs leads to a pause proportional to the number of goroutines in the app (or its high-water mark). Instead, do only a fixed amount of bookkeeping while the world is stopped. Install a barrier so the scheduler confirms that a goroutine appears in the profile, with its stack recorded exactly as it was during the stop-the-world pause, before it allows that goroutine to execute. Iterate over allgs while the app resumes normal operations, adding each to the profile unless they've been scheduled in the meantime (and so have profiled themselves). Stop the world a second time to remove the barrier and do a fixed amount of cleanup work. This increases both the fixed overhead and per-goroutine CPU-time cost of GoroutineProfile. It also increases the wall-clock latency of the call to GoroutineProfile, since the scheduler may interrupt it to execute other goroutines. name old time/op new time/op delta GoroutineProfile/small/loaded-8 1.05ms ± 5% 4.99ms ±31% +376.85% (p=0.000 n=10+9) GoroutineProfile/sparse/loaded-8 1.04ms ± 4% 3.61ms ±27% +246.61% (p=0.000 n=10+10) GoroutineProfile/large/loaded-8 7.69ms ±17% 20.35ms ± 4% +164.50% (p=0.000 n=10+10) GoroutineProfile/small/idle 958µs ± 0% 1820µs ±23% +89.91% (p=0.000 n=10+10) GoroutineProfile/sparse/idle-8 1.00ms ± 3% 1.52ms ±17% +51.18% (p=0.000 n=10+10) GoroutineProfile/small/idle-8 1.01ms ± 4% 1.47ms ± 7% +45.28% (p=0.000 n=9+9) GoroutineProfile/sparse/idle 980µs ± 1% 1403µs ± 2% +43.22% (p=0.000 n=9+10) GoroutineProfile/large/idle-8 7.19ms ± 8% 8.43ms ±21% +17.22% (p=0.011 n=10+10) PingPongHog 511ns ± 8% 585ns ± 9% +14.39% (p=0.000 n=10+10) GoroutineProfile/large/idle 6.71ms ± 0% 7.58ms ± 3% +13.08% (p=0.000 n=8+10) PingPongHog-8 469ns ± 8% 509ns ±12% +8.62% (p=0.010 n=9+10) WakeupParallelSyscall/5µs 216µs ± 4% 229µs ± 3% +6.06% (p=0.000 n=10+9) WakeupParallelSyscall/5µs-8 147µs ± 1% 149µs ± 2% +1.12% (p=0.009 n=10+10) WakeupParallelSyscall/2µs-8 140µs ± 0% 142µs ± 1% +1.11% (p=0.001 n=10+9) WakeupParallelSyscall/50µs-8 236µs ± 0% 238µs ± 1% +1.08% (p=0.000 n=9+10) WakeupParallelSyscall/1µs-8 138µs ± 0% 140µs ± 1% +1.05% (p=0.013 n=10+9) Matmult 8.52ns ± 1% 8.61ns ± 0% +0.98% (p=0.002 n=10+8) WakeupParallelSyscall/10µs-8 157µs ± 1% 158µs ± 1% +0.58% (p=0.003 n=10+8) CreateGoroutinesSingle-8 328ns ± 0% 330ns ± 1% +0.57% (p=0.000 n=9+9) WakeupParallelSpinning/100µs-8 343µs ± 0% 344µs ± 1% +0.30% (p=0.015 n=8+8) WakeupParallelSyscall/20µs-8 178µs ± 0% 178µs ± 0% +0.18% (p=0.043 n=10+9) StackGrowthDeep-8 22.8µs ± 0% 22.9µs ± 0% +0.12% (p=0.006 n=10+10) StackGrowth 1.06µs ± 0% 1.06µs ± 0% +0.09% (p=0.000 n=8+9) WakeupParallelSpinning/0s 10.7µs ± 0% 10.7µs ± 0% +0.08% (p=0.000 n=9+9) WakeupParallelSpinning/5µs 30.7µs ± 0% 30.7µs ± 0% +0.04% (p=0.000 n=10+10) WakeupParallelSpinning/100µs 411µs ± 0% 411µs ± 0% +0.03% (p=0.000 n=10+9) WakeupParallelSpinning/2µs 18.7µs ± 0% 18.7µs ± 0% +0.02% (p=0.026 n=10+10) WakeupParallelSpinning/20µs-8 93.0µs ± 0% 93.0µs ± 0% +0.01% (p=0.021 n=9+10) StackGrowth-8 216ns ± 0% 216ns ± 0% ~ (p=0.209 n=10+10) CreateGoroutinesParallel-8 49.5ns ± 2% 49.3ns ± 1% ~ (p=0.591 n=10+10) CreateGoroutinesSingle 699ns ±20% 748ns ±19% ~ (p=0.353 n=10+10) WakeupParallelSpinning/0s-8 15.9µs ± 2% 16.0µs ± 3% ~ (p=0.315 n=10+10) WakeupParallelSpinning/1µs 14.6µs ± 0% 14.6µs ± 0% ~ (p=0.513 n=10+10) WakeupParallelSpinning/2µs-8 24.2µs ± 3% 24.1µs ± 2% ~ (p=0.971 n=10+10) WakeupParallelSpinning/10µs 50.7µs ± 0% 50.7µs ± 0% ~ (p=0.101 n=10+10) WakeupParallelSpinning/20µs 90.7µs ± 0% 90.7µs ± 0% ~ (p=0.898 n=10+10) WakeupParallelSpinning/50µs 211µs ± 0% 211µs ± 0% ~ (p=0.382 n=10+10) WakeupParallelSyscall/0s-8 137µs ± 1% 138µs ± 0% ~ (p=0.075 n=10+10) WakeupParallelSyscall/1µs 216µs ± 1% 219µs ± 3% ~ (p=0.065 n=10+9) WakeupParallelSyscall/2µs 214µs ± 7% 219µs ± 1% ~ (p=0.101 n=10+8) WakeupParallelSyscall/50µs 317µs ± 5% 326µs ± 4% ~ (p=0.123 n=10+10) WakeupParallelSyscall/100µs 450µs ± 9% 459µs ± 8% ~ (p=0.247 n=10+10) WakeupParallelSyscall/100µs-8 337µs ± 0% 338µs ± 1% ~ (p=0.089 n=10+10) WakeupParallelSpinning/5µs-8 32.2µs ± 0% 32.2µs ± 0% -0.05% (p=0.026 n=9+10) WakeupParallelSpinning/50µs-8 216µs ± 0% 216µs ± 0% -0.12% (p=0.004 n=10+10) WakeupParallelSpinning/1µs-8 20.6µs ± 0% 20.5µs ± 0% -0.22% (p=0.014 n=10+10) WakeupParallelSpinning/10µs-8 54.5µs ± 0% 54.2µs ± 1% -0.57% (p=0.000 n=10+10) CreateGoroutines-8 213ns ± 1% 211ns ± 1% -0.86% (p=0.002 n=10+10) CreateGoroutinesCapture 1.03µs ± 0% 1.02µs ± 0% -0.91% (p=0.000 n=10+10) CreateGoroutinesCapture-8 1.32µs ± 1% 1.31µs ± 1% -1.06% (p=0.001 n=10+9) CreateGoroutines 188ns ± 0% 186ns ± 0% -1.06% (p=0.000 n=9+10) CreateGoroutinesParallel 188ns ± 0% 186ns ± 0% -1.27% (p=0.000 n=8+10) WakeupParallelSyscall/0s 210µs ± 3% 207µs ± 3% -1.60% (p=0.043 n=10+10) StackGrowthDeep 121µs ± 1% 119µs ± 1% -1.70% (p=0.000 n=9+10) Matmult-8 1.82ns ± 3% 1.78ns ± 3% -2.16% (p=0.020 n=10+10) WakeupParallelSyscall/20µs 281µs ± 3% 269µs ± 4% -4.44% (p=0.000 n=10+10) WakeupParallelSyscall/10µs 239µs ± 3% 228µs ± 9% -4.70% (p=0.001 n=10+10) GoroutineProfile/sparse-nil/idle-8 485µs ± 2% 12µs ± 4% -97.56% (p=0.000 n=10+10) GoroutineProfile/small-nil/idle-8 484µs ± 2% 12µs ± 1% -97.60% (p=0.000 n=10+7) GoroutineProfile/small-nil/loaded-8 487µs ± 2% 11µs ± 3% -97.68% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/loaded-8 507µs ± 4% 11µs ± 6% -97.78% (p=0.000 n=10+10) GoroutineProfile/large-nil/idle-8 709µs ± 2% 11µs ± 4% -98.38% (p=0.000 n=10+10) GoroutineProfile/large-nil/loaded-8 717µs ± 2% 11µs ± 3% -98.43% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/idle 465µs ± 3% 1µs ± 1% -99.84% (p=0.000 n=10+10) GoroutineProfile/small-nil/idle 493µs ± 3% 1µs ± 0% -99.85% (p=0.000 n=10+9) GoroutineProfile/large-nil/idle 716µs ± 1% 1µs ± 2% -99.89% (p=0.000 n=7+10) name old alloc/op new alloc/op delta CreateGoroutinesCapture 144B ± 0% 144B ± 0% ~ (all equal) CreateGoroutinesCapture-8 144B ± 0% 144B ± 0% ~ (all equal) name old allocs/op new allocs/op delta CreateGoroutinesCapture 5.00 ± 0% 5.00 ± 0% ~ (all equal) CreateGoroutinesCapture-8 5.00 ± 0% 5.00 ± 0% ~ (all equal) name old p50-ns new p50-ns delta GoroutineProfile/small/loaded-8 1.01M ± 3% 3.87M ±45% +282.15% (p=0.000 n=10+10) GoroutineProfile/sparse/loaded-8 1.02M ± 3% 2.43M ±41% +138.42% (p=0.000 n=10+10) GoroutineProfile/large/loaded-8 7.43M ±16% 17.28M ± 2% +132.43% (p=0.000 n=10+10) GoroutineProfile/small/idle 956k ± 0% 1559k ±16% +63.03% (p=0.000 n=10+10) GoroutineProfile/small/idle-8 1.01M ± 3% 1.45M ± 7% +44.31% (p=0.000 n=10+9) GoroutineProfile/sparse/idle 977k ± 1% 1399k ± 2% +43.20% (p=0.000 n=10+10) GoroutineProfile/sparse/idle-8 1.00M ± 3% 1.41M ± 3% +40.47% (p=0.000 n=10+10) GoroutineProfile/large/idle-8 6.97M ± 1% 8.41M ±25% +20.54% (p=0.003 n=8+10) GoroutineProfile/large/idle 6.71M ± 1% 7.46M ± 4% +11.15% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/idle-8 483k ± 3% 13k ± 3% -97.41% (p=0.000 n=10+9) GoroutineProfile/small-nil/idle-8 483k ± 2% 12k ± 1% -97.43% (p=0.000 n=10+8) GoroutineProfile/small-nil/loaded-8 484k ± 3% 10k ± 2% -97.93% (p=0.000 n=10+8) GoroutineProfile/sparse-nil/loaded-8 492k ± 2% 10k ± 4% -97.97% (p=0.000 n=10+8) GoroutineProfile/large-nil/idle-8 708k ± 2% 12k ±15% -98.36% (p=0.000 n=10+10) GoroutineProfile/large-nil/loaded-8 714k ± 2% 10k ± 2% -98.60% (p=0.000 n=10+8) GoroutineProfile/sparse-nil/idle 459k ± 1% 1k ± 1% -99.85% (p=0.000 n=10+10) GoroutineProfile/small-nil/idle 477k ± 1% 1k ± 0% -99.85% (p=0.000 n=10+9) GoroutineProfile/large-nil/idle 712k ± 1% 1k ± 1% -99.90% (p=0.000 n=7+10) name old p90-ns new p90-ns delta GoroutineProfile/small/loaded-8 1.13M ±10% 7.49M ±35% +562.07% (p=0.000 n=10+10) GoroutineProfile/sparse/loaded-8 1.10M ±12% 4.58M ±31% +318.02% (p=0.000 n=10+9) GoroutineProfile/large/loaded-8 8.78M ±24% 27.83M ± 2% +217.00% (p=0.000 n=10+10) GoroutineProfile/small/idle 967k ± 0% 2909k ±50% +200.91% (p=0.000 n=10+10) GoroutineProfile/sparse/idle-8 1.02M ± 3% 1.96M ±76% +92.99% (p=0.000 n=10+10) GoroutineProfile/small/idle-8 1.07M ±17% 1.55M ±12% +45.23% (p=0.000 n=10+10) GoroutineProfile/sparse/idle 992k ± 1% 1417k ± 3% +42.79% (p=0.000 n=9+10) GoroutineProfile/large/idle 6.73M ± 0% 7.99M ± 8% +18.80% (p=0.000 n=8+10) GoroutineProfile/large/idle-8 8.20M ±25% 9.18M ±25% ~ (p=0.315 n=10+10) GoroutineProfile/sparse-nil/idle-8 495k ± 3% 13k ± 1% -97.36% (p=0.000 n=10+9) GoroutineProfile/small-nil/idle-8 494k ± 2% 13k ± 3% -97.36% (p=0.000 n=10+10) GoroutineProfile/small-nil/loaded-8 496k ± 2% 13k ± 1% -97.41% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/loaded-8 544k ±11% 13k ± 1% -97.62% (p=0.000 n=10+9) GoroutineProfile/large-nil/idle-8 724k ± 1% 13k ± 3% -98.20% (p=0.000 n=10+10) GoroutineProfile/large-nil/loaded-8 729k ± 3% 13k ± 2% -98.23% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/idle 476k ± 4% 1k ± 1% -99.85% (p=0.000 n=9+10) GoroutineProfile/small-nil/idle 537k ±10% 1k ± 0% -99.87% (p=0.000 n=10+9) GoroutineProfile/large-nil/idle 729k ± 0% 1k ± 1% -99.90% (p=0.000 n=7+10) name old p99-ns new p99-ns delta GoroutineProfile/sparse/loaded-8 1.27M ±33% 20.49M ±17% +1514.61% (p=0.000 n=10+10) GoroutineProfile/small/loaded-8 1.37M ±29% 20.48M ±23% +1399.35% (p=0.000 n=10+10) GoroutineProfile/large/loaded-8 9.76M ±23% 39.98M ±22% +309.52% (p=0.000 n=10+8) GoroutineProfile/small/idle 976k ± 1% 3367k ±55% +244.94% (p=0.000 n=10+10) GoroutineProfile/sparse/idle-8 1.03M ± 3% 2.50M ±65% +142.30% (p=0.000 n=10+10) GoroutineProfile/small/idle-8 1.17M ±34% 1.70M ±14% +45.15% (p=0.000 n=10+10) GoroutineProfile/sparse/idle 1.02M ± 3% 1.45M ± 4% +42.64% (p=0.000 n=9+10) GoroutineProfile/large/idle 6.92M ± 2% 9.00M ± 7% +29.98% (p=0.000 n=8+9) GoroutineProfile/large/idle-8 8.74M ±23% 9.90M ±24% ~ (p=0.190 n=10+10) GoroutineProfile/sparse-nil/idle-8 508k ± 4% 16k ± 2% -96.90% (p=0.000 n=10+9) GoroutineProfile/small-nil/idle-8 508k ± 4% 16k ± 3% -96.91% (p=0.000 n=10+9) GoroutineProfile/small-nil/loaded-8 542k ± 5% 15k ±15% -97.15% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/loaded-8 649k ±16% 15k ±18% -97.67% (p=0.000 n=10+10) GoroutineProfile/large-nil/idle-8 738k ± 2% 16k ± 2% -97.86% (p=0.000 n=10+10) GoroutineProfile/large-nil/loaded-8 765k ± 4% 15k ±17% -98.03% (p=0.000 n=10+10) GoroutineProfile/sparse-nil/idle 539k ±26% 1k ±17% -99.84% (p=0.000 n=10+10) GoroutineProfile/small-nil/idle 659k ±25% 1k ± 0% -99.84% (p=0.000 n=10+8) GoroutineProfile/large-nil/idle 760k ± 2% 1k ±22% -99.88% (p=0.000 n=9+10) Fixes #33250 For #50794 Change-Id: I862a2bc4e991cec485f21a6fce4fca84f2c6435b Reviewed-on: https://go-review.googlesource.com/c/go/+/387415 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-11all: gofmt main repoRuss Cox
[This CL is part of a sequence implementing the proposal #51082. The design doc is at https://go.dev/s/godocfmt-design.] Run the updated gofmt, which reformats doc comments, on the main repository. Vendored files are excluded. For #51082. Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407 Reviewed-on: https://go-review.googlesource.com/c/go/+/384268 Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-05all: separate doc comment from //go: directivesRuss Cox
A future change to gofmt will rewrite // Doc comment. //go:foo to // Doc comment. // //go:foo Apply that change preemptively to all comments (not necessarily just doc comments). For #51082. Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9 Reviewed-on: https://go-review.googlesource.com/c/go/+/384260 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-03runtime: count spill slot for frame size at finalizer callCherry Mui
The finalizer is called using reflectcall. When register ABI is used, the finalizer's argument is passed in register(s). But the frame size calculation does not include the spill slot. When the argument actually spills, it may clobber the caller's stack frame. This CL fixes it. Change-Id: Ibcc7507c518ba65c1c5a7759e5cab0ae3fc7efce Reviewed-on: https://go-review.googlesource.com/c/go/+/389574 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-08-12[dev.typeparams] all: merge master (46fd547) into dev.typeparamsMatthew Dempsky
Conflicts: - src/go/types/check_test.go CL 324730 on dev.typeparams changed the directory paths in TestCheck, TestExamples, and TestFixedbugs and renamed checkFiles to testFiles; whereas CL 337529 on master added a new test case just above them and that used checkFiles. Merge List: + 2021-08-12 46fd547d89 internal/goversion: update Version to 1.18 + 2021-08-12 5805efc78e doc/go1.17: remove draft notice + 2021-08-12 39634e7dae CONTRIBUTORS: update for the Go 1.17 release + 2021-08-12 095bb790e1 os/exec: re-enable LookPathTest/16 + 2021-08-11 dea23e9ca8 src/make.*: make --no-clean flag a no-op that prints a warning + 2021-08-11 d4c0ed26ac doc/go1.17: linker passes -I to extld as -Wl,--dynamic-linker + 2021-08-10 1f9c9d8530 doc: use "high address/low address" instead of "top/bottom" + 2021-08-09 f1dce319ff cmd/go: with -mod=vendor, don't panic if there are duplicate requirements + 2021-08-09 7aeaad5c86 runtime/cgo: when using msan explicitly unpoison cgoCallers + 2021-08-08 507cc341ec doc: add example for conversion from slice expressions to array ptr + 2021-08-07 891547e2d4 doc/go1.17: fix a typo introduced in CL 335135 + 2021-08-06 8eaf4d16bc make.bash: do not overwrite GO_LDSO if already set + 2021-08-06 63b968f4f8 doc/go1.17: clarify Modules changes + 2021-08-06 70546f6404 runtime: allow arm64 SEH to be called if illegal instruction + 2021-08-05 fd45e267c2 runtime: warn that KeepAlive is not an unsafe.Pointer workaround + 2021-08-04 6e738868a7 net/http: speed up and deflake TestCancelRequestWhenSharingConnection + 2021-08-02 8a7ee4c51e io/fs: don't use absolute path in DirEntry.Name doc + 2021-07-31 b8ca6e59ed all: gofmt + 2021-07-30 b7a85e0003 net/http/httputil: close incoming ReverseProxy request body + 2021-07-29 70fd4e47d7 runtime: avoid possible preemption when returning from Go to C + 2021-07-28 9eee0ed439 cmd/go: fix go.mod file name printed in error messages for replacements + 2021-07-28 b39e0f461c runtime: don't crash on nil pointers in checkptrAlignment + 2021-07-27 7cd10c1149 cmd/go: use .mod instead of .zip to determine if version has go.mod file + 2021-07-27 c8cf0f74e4 cmd/go: add missing flag in UsageLine + 2021-07-27 7ba8e796c9 testing: clarify T.Name returns a distinct name of the running test + 2021-07-27 33ff155970 go/types: preserve untyped constants on the RHS of a shift expression + 2021-07-26 840e583ff3 runtime: correct variable name in comment + 2021-07-26 bfbb288574 runtime: remove adjustTimers counter + 2021-07-26 9c81fd53b3 cmd/vet: add missing copyright header Change-Id: Ia80604d24c6f4205265683024e3100769cf32065
2021-08-05runtime: warn that KeepAlive is not an unsafe.Pointer workaroundMatthew Dempsky
Even experienced users occasionally mistake that runtime.KeepAlive can be used as a workaround for following the unsafe.Pointer safety rules, but it cannot. Add an explicit warning to this effect to dissuade users from trying to use it as such. Fixes #47562. Change-Id: I842e33a3e1c080933c6b1bd1b6318448adbf495c Reviewed-on: https://go-review.googlesource.com/c/go/+/340269 Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-06-17[dev.typeparams] runtime: fix import sort order [generated]Michael Anthony Knyszek
[git-generate] cd src/runtime goimports -w *.go Change-Id: I1387af0f2fd1a213dc2f4c122e83a8db0fcb15f0 Reviewed-on: https://go-review.googlesource.com/c/go/+/329189 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-06-17[dev.typeparams] runtime: replace uses of runtime/internal/sys.PtrSize with ↵Michael Anthony Knyszek
internal/goarch.PtrSize [generated] [git-generate] cd src/runtime/internal/math gofmt -w -r "sys.PtrSize -> goarch.PtrSize" . goimports -w *.go cd ../.. gofmt -w -r "sys.PtrSize -> goarch.PtrSize" . goimports -w *.go Change-Id: I43491cdd54d2e06d4d04152b3d213851b7d6d423 Reviewed-on: https://go-review.googlesource.com/c/go/+/328337 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-03-11runtime: support register ABI for finalizersMichael Anthony Knyszek
This change modifies runfinq to properly pass arguments to finalizers in registers via reflectcall. For #40724. Change-Id: I414c0eff466ef315a0eb10507994e598dd29ccb2 Reviewed-on: https://go-review.googlesource.com/c/go/+/300112 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-23cmd/compile: simplify assert{E,I}2I{,2} calling conventionsCuong Manh Le
This CL rebases CL 273694 on top of master with @mdempsky's permission. For assertE2I and assertI2I, there's no need to pass through the interface's data pointer: it's always going to come back unmodified. For assertE2I2 and assertI2I2, there's no need for an extra bool result parameter: it's redundant with testing the returned interface value for nil. Change-Id: Ic92d4409ad381952f875d3d74b8cf11c32702fa6 Reviewed-on: https://go-review.googlesource.com/c/go/+/292892 Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-02-16[dev.regabi] reflect: support for register ABI on amd64 for reflect.(Value).CallMichael Anthony Knyszek
This change adds support for the new register ABI on amd64 to reflect.(Value).Call. If internal/abi's register counts are non-zero, reflect will try to set up arguments in registers on the Call path. Note that because the register ABI becomes ABI0 with zero registers available, this should keep working as it did before. This change does not add any tests for the register ABI case because there's no way to do so at the moment. For #40724. Change-Id: I8aa089a5aa5a31b72e56b3d9388dd3f82203985b Reviewed-on: https://go-review.googlesource.com/c/go/+/272568 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-28Revert "cmd/compile: split exported/non-exported methods for interface type"Cuong Manh Le
This reverts commit 8f26b57f9afc238bdecb9b7030bc2f4364093885. Reason for revert: break a bunch of code, include standard library. Fixes #42123 Change-Id: Ife90ecbafd2cb395623d1db555fbfc9c1b0098e0 Reviewed-on: https://go-review.googlesource.com/c/go/+/264026 Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2020-10-26runtime: break down memstats.gc_sysMichael Anthony Knyszek
This change breaks apart gc_sys into three distinct pieces. Two of those pieces are pieces which come from heap_sys since they're allocated from the page heap. The rest comes from memory mapped from e.g. persistentalloc which better fits the purpose of a sysMemStat. Also, rename gc_sys to gcMiscSys. Change-Id: I098789170052511e7b31edbcdc9a53e5c24573f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/246973 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>