aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/runtime2.go
AgeCommit message (Collapse)Author
2023-08-07runtime: avoid relying on the unwinder in deferreturnMatthew Dempsky
This CL changes deferreturn so that it never needs to invoke the unwinder. Instead, in the unusual case that we recover into a frame with pending open-coded defers, we now save the extra state needed to find them in g.param. Change-Id: Ied35f6c1063fee5b6044cc37b2bccd3f90682fe6 Reviewed-on: https://go-review.googlesource.com/c/go/+/515856 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Matthew Dempsky <mdempsky@google.com>
2023-08-07runtime, cmd/compile: optimize open-coded defersMatthew Dempsky
This CL optimizes open-coded defers in two ways: 1. It modifies local variable sorting to place all open-coded defer closure slots in order, so that rather than requiring the metadata to contain each offset individually, we just need a single offset to the first slot. 2. Because the slots are in ascending order and can be directly indexed, we can get rid of the count of how many defers are in the frame. Instead, we just find the top set bit in the active defers bitmask, and load the corresponding closure. Change-Id: I6f912295a492211023a9efe12c94a14f449d86ad Reviewed-on: https://go-review.googlesource.com/c/go/+/516199 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-08-07runtime, cmd/compile: prune unused _defer fieldsMatthew Dempsky
These fields are no longer needed since go.dev/cl/513837. Change-Id: I980fc9db998c293e930094bbb87e8c8f1654e39c Reviewed-on: https://go-review.googlesource.com/c/go/+/516198 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-07-31runtime: refactor defer processingMatthew Dempsky
This CL refactors gopanic, Goexit, and deferreturn to share a common state machine for processing pending defers. The new state machine removes a lot of redundant code and does overall less work. It should also make it easier to implement further optimizations (e.g., TODOs added in this CL). Change-Id: I71d3cc8878a6f951d8633505424a191536c8e6b3 Reviewed-on: https://go-review.googlesource.com/c/go/+/513837 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-24runtime: cache inner pinner on PMichael Anthony Knyszek
This change caches the *pinner on the P to pool it and reduce the chance that a new allocation is made. It also makes the *pinner no longer drop its refs array on unpin, also to avoid reallocating. The Pinner benchmark results before and after this CL are attached at the bottom of the commit message. Note that these results are biased toward the current change because of the last two benchmark changes. Reusing the pinner in the benchmark itself achieves similar performance before this change. The benchmark results thus basically just confirm that this change does cache the inner pinner in a useful way. Using the previous benchmarks there's actually a slight regression from the extra check in the cache, however the long pole is still setPinned itself. name old time/op new time/op delta PinnerPinUnpinBatch-8 42.2µs ± 2% 41.5µs ± 1% ~ (p=0.056 n=5+5) PinnerPinUnpinBatchDouble-8 367µs ± 1% 350µs ± 1% -4.67% (p=0.008 n=5+5) PinnerPinUnpinBatchTiny-8 108µs ± 0% 102µs ± 1% -6.22% (p=0.008 n=5+5) PinnerPinUnpin-8 592ns ± 8% 40ns ± 1% -93.29% (p=0.008 n=5+5) PinnerPinUnpinTiny-8 693ns ± 9% 39ns ± 1% -94.31% (p=0.008 n=5+5) PinnerPinUnpinDouble-8 843ns ± 5% 124ns ± 3% -85.24% (p=0.008 n=5+5) PinnerPinUnpinParallel-8 1.11µs ± 5% 0.00µs ± 0% -99.55% (p=0.008 n=5+5) PinnerPinUnpinParallelTiny-8 1.12µs ± 8% 0.00µs ± 1% -99.55% (p=0.008 n=5+5) PinnerPinUnpinParallelDouble-8 1.79µs ± 4% 0.58µs ± 6% -67.36% (p=0.008 n=5+5) PinnerIsPinnedOnPinned-8 5.78ns ± 0% 5.80ns ± 1% ~ (p=0.548 n=5+5) PinnerIsPinnedOnUnpinned-8 4.99ns ± 1% 4.98ns ± 0% ~ (p=0.841 n=5+5) PinnerIsPinnedOnPinnedParallel-8 0.71ns ± 0% 0.71ns ± 0% ~ (p=0.175 n=5+5) PinnerIsPinnedOnUnpinnedParallel-8 0.67ns ± 1% 0.66ns ± 0% ~ (p=0.167 n=5+5) name old alloc/op new alloc/op delta PinnerPinUnpinBatch-8 20.1kB ± 0% 20.0kB ± 0% -0.32% (p=0.008 n=5+5) PinnerPinUnpinBatchDouble-8 52.7kB ± 0% 52.7kB ± 0% -0.12% (p=0.008 n=5+5) PinnerPinUnpinBatchTiny-8 20.1kB ± 0% 20.0kB ± 0% -0.32% (p=0.008 n=5+5) PinnerPinUnpin-8 64.0B ± 0% 0.0B -100.00% (p=0.008 n=5+5) PinnerPinUnpinTiny-8 64.0B ± 0% 0.0B -100.00% (p=0.008 n=5+5) PinnerPinUnpinDouble-8 64.0B ± 0% 0.0B -100.00% (p=0.008 n=5+5) PinnerPinUnpinParallel-8 64.0B ± 0% 0.0B -100.00% (p=0.008 n=5+5) PinnerPinUnpinParallelTiny-8 64.0B ± 0% 0.0B -100.00% (p=0.008 n=5+5) PinnerPinUnpinParallelDouble-8 64.0B ± 0% 0.0B -100.00% (p=0.008 n=5+5) PinnerIsPinnedOnPinned-8 0.00B 0.00B ~ (all equal) PinnerIsPinnedOnUnpinned-8 0.00B 0.00B ~ (all equal) PinnerIsPinnedOnPinnedParallel-8 0.00B 0.00B ~ (all equal) PinnerIsPinnedOnUnpinnedParallel-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta PinnerPinUnpinBatch-8 9.00 ± 0% 8.00 ± 0% -11.11% (p=0.008 n=5+5) PinnerPinUnpinBatchDouble-8 11.0 ± 0% 10.0 ± 0% -9.09% (p=0.008 n=5+5) PinnerPinUnpinBatchTiny-8 9.00 ± 0% 8.00 ± 0% -11.11% (p=0.008 n=5+5) PinnerPinUnpin-8 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) PinnerPinUnpinTiny-8 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) PinnerPinUnpinDouble-8 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) PinnerPinUnpinParallel-8 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) PinnerPinUnpinParallelTiny-8 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) PinnerPinUnpinParallelDouble-8 1.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5) PinnerIsPinnedOnPinned-8 0.00 0.00 ~ (all equal) PinnerIsPinnedOnUnpinned-8 0.00 0.00 ~ (all equal) PinnerIsPinnedOnPinnedParallel-8 0.00 0.00 ~ (all equal) PinnerIsPinnedOnUnpinnedParallel-8 0.00 0.00 ~ (all equal) For #46787. Change-Id: I0cdfad77b189c425868944a4faeff3d5b97417b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/497615 Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ansiwen <ansiwen@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-05-19runtime: replace raw traceEv with traceBlockReason in goparkMichael Anthony Knyszek
This change adds traceBlockReason which leaks fewer implementation details of the tracer to the runtime. Currently, gopark is called with an explicit trace event, but this leaks details about trace internals throughout the runtime. This change will make it easier to change out the trace implementation. Change-Id: Id633e1704d2c8838c6abd1214d9695537c4ac7db Reviewed-on: https://go-review.googlesource.com/c/go/+/494185 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-17runtime/cgo: store M for C-created thread in pthread keyCherry Mui
This reapplies CL 485500, with a fix drafted in CL 492987 incorporated. CL 485500 is reverted due to #60004 and #60007. #60004 is fixed in CL 492743. #60007 is fixed in CL 492987 (incorporated in this CL). [Original CL 485500 description] This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 492987 description] On the first call into Go from a C thread, currently we set the g0 stack's high bound imprecisely based on the SP. With CL 485500, we keep the M and don't recompute the stack bounds when it calls into Go again. If the first call is made when the C thread uses some deep stack, but a subsequent call is made with a shallower stack, the SP may be above g0.stack.hi. This is usually okay as we don't check usually stack.hi. One place where we do check for stack.hi is in the signal handler, in adjustSignalStack. In particular, C TSAN delivers signals on the g0 stack (instead of the usual signal stack). If the SP is above g0.stack.hi, we don't see it is on the g0 stack, and throws. This CL makes it get an accurate stack upper bound with the pthread API (on the platforms where it is available). Also add some debug print for the "handler not on signal stack" throw. Fixes #51676. Fixes #59294. Fixes #59678. Fixes #60007. Change-Id: Ie51c8e81ade34ec81d69fd7bce1fe0039a470776 Reviewed-on: https://go-review.googlesource.com/c/go/+/495855 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-17runtime: capture per-p trace state in a typeMichael Anthony Knyszek
More tightening up of the tracer's interface. Change-Id: I992141c7f30e5c2d5d77d1fcd6817d35bc6e5f6d Reviewed-on: https://go-review.googlesource.com/c/go/+/494191 Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-17runtime: capture per-m trace state in a typeMichael Anthony Knyszek
More tightening up of the tracer's interface. While we're here, clarify why waittraceskip isn't included by explaining what the wait* fields in the M are really for. Change-Id: I0e7b4cac79fb77a7a0b3ca6b6cc267668e3610bc Reviewed-on: https://go-review.googlesource.com/c/go/+/494190 Auto-Submit: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-17runtime: capture per-g trace state in a typeMichael Anthony Knyszek
More tightening up of the tracer's interface. This increases the size of each G very slightly, which isn't great, but we stay within the same size class, so actually memory use will be unchanged. Change-Id: I7d1f5798edcf437c212beb1e1a2619eab833aafb Reviewed-on: https://go-review.googlesource.com/c/go/+/494188 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-10all: fix a lot of commentscui fliter
Fix comments, including duplicate is, wrong phrases and articles, misspellings, etc. Change-Id: I8bfea53b9b275e649757cc4bee6a8a026ed9c7a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/493035 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-05-05Revert "runtime/cgo: store M for C-created thread in pthread key"Chressie Himpel
This reverts CL 485500. Reason for revert: This breaks internal tests at Google, see b/280861579 and b/280820455. Change-Id: I426278d400f7611170918fc07c524cb059b9cc55 Reviewed-on: https://go-review.googlesource.com/c/go/+/492995 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Chressie Himpel <chressie@google.com>
2023-04-26runtime/cgo: store M for C-created thread in pthread keyMichael Pratt
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 485500 description] CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. Fixes #51676. Fixes #59294. Fixes #59678. Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca Reviewed-on: https://go-review.googlesource.com/c/go/+/485500 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-21internal/abi, runtime, cmd: merge funcFlag_* consts into internal/abiAustin Clements
For #59670. Change-Id: Ie784ba4dd2701e4f455e1abde4a6bfebee4b1387 Reviewed-on: https://go-review.googlesource.com/c/go/+/485496 Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Austin Clements <austin@google.com> Auto-Submit: Austin Clements <austin@google.com>
2023-04-21internal/abi, runtime, cmd: merge funcID_* consts into internal/abiAustin Clements
For #59670. Change-Id: I517e97ea74cf232e5cfbb77b127fa8804f74d84b Reviewed-on: https://go-review.googlesource.com/c/go/+/485495 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com>
2023-04-17Revert "runtime/cgo: store M for C-created thread in pthread key"Michael Pratt
This reverts CL 481061. Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers race detection on `g->stacklo` because the synchronization is in Go, which isn't instrumented. For #51676. For #59294. For #59678. Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47 Reviewed-on: https://go-review.googlesource.com/c/go/+/485275 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2023-04-03runtime/cgo: store M for C-created thread in pthread keydoujiang24
This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. Fixes #51676. Fixes #59294. Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494 Reviewed-on: https://go-review.googlesource.com/c/go/+/481061 Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31Revert "runtime/cgo: store M for C-created thread in pthread key"Cherry Mui
This reverts CL 392854. Reason for revert: caused #59294, which was derived from google internal tests. The attempted fix of #59294 caused more breakage. Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba Reviewed-on: https://go-review.googlesource.com/c/go/+/481060 Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-24runtime/cgo: store M for C-created thread in pthread keydoujiang24
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz Fixes #51676 Change-Id: I380702fe2f9b6b401b2d6f04b0aba990f4b9ee6c GitHub-Last-Rev: 93dc64ad98e5583372e41f65ee4b7ab78b5aff51 GitHub-Pull-Request: golang/go#51679 Reviewed-on: https://go-review.googlesource.com/c/go/+/392854 Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: thepudds <thepudds1460@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-21runtime: for deep stacks, print both the top 50 and bottom 50 framesAustin Clements
This is relatively easy using the new traceback iterator. Ancestor tracebacks are now limited to 50 frames. We could keep that at 100, but the fact that it used 100 before seemed arbitrary and unnecessary. Fixes #7181 Updates #54466 Change-Id: If693045881d84848f17e568df275a5105b6f1cb0 Reviewed-on: https://go-review.googlesource.com/c/go/+/475960 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10runtime: delete gentracebackAustin Clements
Printing is the only remaining functionality of gentraceback. Move this into the traceback printing code and eliminate gentraceback. This lets us simplify the logic, which fixes at least one minor bug: previously, if inline unwinding pushed the total printed count over _TracebackMaxFrames, we would print extra frames and then fail to print "additional frames elided". The cumulative performance effect of the series of changes starting with "add a benchmark of Callers" (CL 472956) is: goos: linux goarch: amd64 pkg: runtime cpu: Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz │ baseline │ unwinder │ │ sec/op │ sec/op vs base │ Callers/cached-48 1.464µ ± 1% 1.684µ ± 1% +15.03% (p=0.000 n=20) Callers/inlined-48 1.391µ ± 1% 1.536µ ± 1% +10.42% (p=0.000 n=20) Callers/no-cache-48 10.50µ ± 1% 11.11µ ± 0% +5.82% (p=0.000 n=20) StackCopyPtr-48 88.74m ± 1% 81.22m ± 2% -8.48% (p=0.000 n=20) StackCopy-48 80.90m ± 1% 70.56m ± 1% -12.78% (p=0.000 n=20) StackCopyNoCache-48 2.458m ± 1% 2.209m ± 1% -10.15% (p=0.000 n=20) StackCopyWithStkobj-48 26.81m ± 1% 25.66m ± 1% -4.28% (p=0.000 n=20) geomean 518.8µ 512.9µ -1.14% The performance impact of intermediate CLs in this sequence varies a lot as we went through many refactorings. The slowdown in Callers comes primarily from the introduction of unwinder because that doesn't get inlined and results in somewhat worse code generation in code that's extremely hot in those microbenchmarks. The performance gains on stack copying come mostly from replacing callbacks with direct use of the unwinder. Updates #54466. Fixes #32383. Change-Id: I4970603b2861633eecec30545e852688bc7cc9a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/468301 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10runtime: implement traceback iteratorAustin Clements
Currently, all stack walking logic is in one venerable, large, and very, very complicated function: runtime.gentraceback. This function has three distinct operating modes: printing, populating a PC buffer, or invoking a callback. And it has three different modes of unwinding: physical Go frames, inlined Go frames, and cgo frames. It also has several flags. All of this logic is very interwoven. This CL reimplements the monolithic gentraceback function as an "unwinder" type with an iterator API. It moves all of the logic for stack walking into this new type, and gentraceback is now a much-simplified wrapper around the new unwinder type that still implements printing, populating a PC buffer, and invoking a callback. Follow-up CLs will replace uses of gentraceback with direct uses of unwinder. Exposing traceback functionality as an iterator API will enable a lot of follow-up work such as simplifying the open-coded defer implementation (which should in turn help with #26813 and #37233), printing the bottom of deep stacks (#7181), and eliminating the small limit on CPU stacks in profiles (#56029). Fixes #54466. Change-Id: I36e046dc423c9429c4f286d47162af61aff49a0d Reviewed-on: https://go-review.googlesource.com/c/go/+/458218 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Austin Clements <austin@google.com>
2023-03-10runtime: create an API for unwinding inlined framesAustin Clements
We've replicated the code to expand inlined frames in many places in the runtime at this point. This CL adds a simple iterator API that abstracts this out. We also use this to try out a new idea for structuring tests of runtime internals: rather than exporting this whole internal data type and API, we write the test in package runtime and import the few bits of std we need. The idea is that, for tests of internals, it's easier to inject public APIs from std than it is to export non-public APIs from runtime. This is discussed more in #55108. For #54466. Change-Id: Iebccc04ff59a1509694a8ac0e0d3984e49121339 Reviewed-on: https://go-review.googlesource.com/c/go/+/466096 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Austin Clements <austin@google.com>
2023-02-21runtime: record parent goroutine ID, and print it in stack tracesNick Ripley
Fixes #38651 Change-Id: Id46d684ee80e208c018791a06c26f304670ed159 Reviewed-on: https://go-review.googlesource.com/c/go/+/435337 Run-TryBot: Nick Ripley <nick.ripley@datadoghq.com> Reviewed-by: Ethan Reesor <ethan.reesor@gmail.com> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-18runtime: add page tracerMichael Knyszek
This change adds a new GODEBUG flag called pagetrace that writes a low-overhead trace of how pages of memory are managed by the Go runtime. The page tracer is kept behind a GOEXPERIMENT flag due to a potential security risk for setuid binaries. Change-Id: I6f4a2447d02693c25214400846a5d2832ad6e5c0 Reviewed-on: https://go-review.googlesource.com/c/go/+/444157 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-21runtime: gofmtCherry Mui
Change-Id: Ib9bea9e42d8e99b83dc64450baf9fede15156514 Reviewed-on: https://go-review.googlesource.com/c/go/+/444615 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-10-18runtime: always keep global reference to mp until mexit completesMichael Pratt
Ms are allocated via standard heap allocation (`new(m)`), which means we must keep them alive (i.e., reachable by the GC) until we are completely done using them. Ms are primarily reachable through runtime.allm. However, runtime.mexit drops the M from allm fairly early, long before it is done using the M structure. If that was the last reference to the M, it is now at risk of being freed by the GC and used for some other allocation, leading to memory corruption. Ms with a Go-allocated stack coincidentally already keep a reference to the M in sched.freem, so that the stack can be freed lazily. This reference has the side effect of keeping this Ms reachable. However, Ms with an OS stack skip this and are at risk of corruption. Fix this lifetime by extending sched.freem use to all Ms, with the value of mp.freeWait determining whether the stack needs to be freed or not. Fixes #56243. Change-Id: Ic0c01684775f5646970df507111c9abaac0ba52e Reviewed-on: https://go-review.googlesource.com/c/go/+/443716 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-10-14cmd/compile,cmd/link,runtime: add start line numbers to func metadataMichael Pratt
This adds the function "start line number" to runtime._func and runtime.inlinedCall objects. The "start line number" is the line number of the func keyword or TEXT directive for assembly. Subtracting the start line number from PC line number provides the relative line offset of a PC from the the start of the function. This helps with source stability by allowing code above the function to move without invalidating samples within the function. Encoding start line rather than relative lines directly is convenient because the pprof format already contains a start line field. This CL uses a straightforward encoding of explictly including a start line field in every _func and inlinedCall. It is possible that we could compress this further in the future. e.g., functions with a prologue usually have <line of PC 0> == <start line>. In runtime.test, 95% of functions have <line of PC 0> == <start line>. According to bent, this is geomean +0.83% binary size vs master and -0.31% binary size vs 1.19. Note that //line directives can change the file and line numbers arbitrarily. The encoded start line is as adjusted by //line directives. Since this can change in the middle of a function, `line - start line` offset calculations may not be meaningful if //line directives are in use. For #55022. Change-Id: Iaabbc6dd4f85ffdda294266ef982ae838cc692f6 Reviewed-on: https://go-review.googlesource.com/c/go/+/429638 Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-09-27runtime/trace: add missing events for the locked g in extra M.doujiang24
Extra Ms may lead to the "no consistent ordering of events possible" error when parsing trace file with cgo enabled, since: 1. The gs in the extra Ms may be in `_Gdead` status while starting trace by invoking `runtime.StartTrace`, 2. and these gs will trigger `traceEvGoSysExit` events in `runtime.exitsyscall` when invoking go functions from c, 3. then, the events of those gs are under non-consistent ordering, due to missing the previous events. Add two events, `traceEvGoCreate` and `traceEvGoInSyscall`, in `runtime.StartTrace`, will make the trace parser happy. Fixes #29707 Change-Id: I2fd9d1713cda22f0ddb36efe1ab351f88da10881 GitHub-Last-Rev: 7bbfddb81b70041250e3c59ce53bea44f7afd2c3 GitHub-Pull-Request: golang/go#54974 Reviewed-on: https://go-review.googlesource.com/c/go/+/429858 Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: xie cui <523516579@qq.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com>
2022-09-16runtime/metrics: add /sync/mutex/wait/total:seconds metricMichael Anthony Knyszek
This change adds a metric to the runtime/metrics package which tracks total mutex wait time for sync.Mutex and sync.RWMutex. The purpose of this metric is to be able to quickly get an idea of the total mutex wait time. The implementation of this metric piggybacks off of the existing G runnable tracking infrastructure, as well as the wait reason set on a G when it goes into _Gwaiting. Fixes #49881. Change-Id: I4691abf64ac3574bec69b4d7d4428b1573130517 Reviewed-on: https://go-review.googlesource.com/c/go/+/427618 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-16runtime: set G wait reason more consistentlyMichael Anthony Knyszek
Currently, wait reasons are set somewhat inconsistently. In a follow-up CL, we're going to want to rely on the wait reason being there for casgstatus, so the status quo isn't really going to work for that. Plus this inconsistency means there are a whole bunch of cases where we could be more specific about the G's status but aren't. So, this change adds a new function, casGToWaiting which is like casgstatus but also sets the wait reason. The goal is that by using this API it'll be harder to forget to set a wait reason (or the lack thereof will at least be explicit). This change then updates all casgstatus(gp, ..., _Gwaiting) calls to casGToWaiting(gp, ..., waitReasonX) instead. For a number of these cases, we're missing a wait reason, and it wouldn't hurt to add a wait reason for them, so this change also adds those wait reasons. For #49881. Change-Id: Ia95e06ecb74ed17bb7bb94f1a362ebfe6bec1518 Reviewed-on: https://go-review.googlesource.com/c/go/+/427617 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-16runtime: make the wait reason for a g blocked on a mutex more specificMichael Anthony Knyszek
This change adds 3 new waitReasons that correspond to sync.Mutex.Lock, sync.RWMutex.RLock, and sync.RWMutex.Lock that are plumbed down into semacquire1 by exporting new functions to the sync package from the runtime. Currently these three functions show up as "semacquire" in backtraces which isn't very clear, though the stack trace itself should reveal what's really going on. This represents a minor improvement to backtrace readability, though blocking on an RWMutex.w.Lock will still show up as blocking on a regular mutex (I suppose technically it is). This is a step toward helping the runtime identify when a goroutine is blocked on a mutex of some kind. For #49881. Change-Id: Ia409b4d27e117fe4bfdc25fa541e9c58d6d587b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/427616 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-09-16runtime/metrics: add CPU statsMichael Anthony Knyszek
This changes adds a breakdown for estimated CPU usage by time. These estimates are not based on real on-CPU counters, so each metric has a disclaimer explaining so. They can, however, be more reasonably compared to a total CPU time metric that this change also adds. Fixes #47216. Change-Id: I125006526be9f8e0d609200e193da5a78d9935be Reviewed-on: https://go-review.googlesource.com/c/go/+/404307 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Josh MacDonald <jmacd@lightstep.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-07runtime: rename _func.entryoff to entryOffMichael Pratt
To match _func.nameOff. Change-Id: I75e71cadaa0f7ca8844d1b49950673797b227074 Reviewed-on: https://go-review.googlesource.com/c/go/+/428658 Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com>
2022-09-07runtime: rename _func.nameoff and friends to nameOffMichael Pratt
Switch to the more Go-style name to match inlinedCall.nameOff. Change-Id: I2115b27af8309e1ead7d61ecc65fe4fc966030f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/428657 Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-09-05runtime: convert g.atomicstatus to internal atomic typeAndy Pan
Note that this changes some unsynchronized operations of g.atomicstatus to synchronized operations. Updates #53821 Change-Id: If249d62420ea09fbec39b570942f96c63669c333 Reviewed-on: https://go-review.googlesource.com/c/go/+/425363 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-09-02runtime: document implicit fields at the end of _funcMichael Pratt
Currently these are only implicitly documented by the code that generates and consumes them. Add explicit documentation. Change-Id: I25a1294f04dc11785242643bec83370c66ff7a20 Reviewed-on: https://go-review.googlesource.com/c/go/+/427954 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2022-09-02runtime: consolidate stkframe and its methods into stkframe.goAustin Clements
The stkframe struct and its methods are strewn across different source files. Since they actually have a pretty coherent theme at this point, migrate it all into a new file, stkframe.go. There are no code changes in this CL. For #54466, albeit rather indirectly. Change-Id: Ibe53fc4b1106d131005e1c9d491be838a8f14211 Reviewed-on: https://go-review.googlesource.com/c/go/+/424516 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Austin Clements <austin@google.com> Auto-Submit: Austin Clements <austin@google.com>
2022-09-02runtime: replace stkframe.arglen/argmap with methodsAustin Clements
Currently, stkframe.arglen and stkframe.argmap are populated by gentraceback under a particular set of circumstances. But because they can be constructed from other fields in stkframe, they don't need to be computed eagerly at all. They're also rather misleading, as they're only part of computing the actual argument map and most callers should be using getStackMap, which does the rest of the work. This CL drops these fields from stkframe. It shifts the functions that used to compute them, getArgInfoFast and getArgInfo, into corresponding methods stkframe.argBytes and stkframe.argMapInternal. argBytes is expected to be used by callers that need to know only the argument frame size, while argMapInternal is used only by argBytes and getStackMap. We also move some of the logic from getStackMap into argMapInternal because the previous split of responsibilities didn't make much sense. This lets us return just a bitvector from argMapInternal, rather than both a bitvector, which carries a size, and an "actually use this size". The getArgInfoFast function was inlined before (and inl_test checked this). We drop that requirement from stkframe.argBytes because the uses of this have shifted and now it's only called from heap dumping (which never happens) and conservative stack frame scanning (which very, very rarely happens). There will be a few follow-up clean-up CLs. For #54466. This is a nice clean-up on its own, but it also serves to remove pointers from the traceback state that would eventually become troublesome write barriers once we stack-rip gentraceback. Change-Id: I107f98ed8e7b00185c081de425bbf24af02a4163 Reviewed-on: https://go-review.googlesource.com/c/go/+/424514 Run-TryBot: Austin Clements <austin@google.com> Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-02runtime: document stkframeAustin Clements
The meaning of some of the fields in stkframe is actually quite subtle. Change-Id: Iac765ff6fbf4c3b7c9f2453f5b4a2e5e640f5750 Reviewed-on: https://go-review.googlesource.com/c/go/+/424256 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Austin Clements <austin@google.com>
2022-08-31runtime: convert p.numTimers and p.deletedTimers to internal atomic typesAndy Pan
Note that this changes the non-atomic operations in p.destroy() to atomic operations. For #53821 Change-Id: I7bba77c9a2287ba697c87cce2c79293e4d1b3334 Reviewed-on: https://go-review.googlesource.com/c/go/+/425774 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-08-26runtime: drop padding alignment field for timeHistogramCuong Manh Le
After CL 419449, timeHistogram always have 8-byte alignment. Change-Id: I93145502bcafa1712b811b1a6d62da5d54d0db42 Reviewed-on: https://go-review.googlesource.com/c/go/+/425777 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2022-08-26runtime: convert forcegcstate.idle to internal atomic typeAndy Pan
Note that this changes a few unsynchronized operations of forcegcstate.idle to synchronized operations. Updates #53821 Change-Id: I041654cc84a188fad45e2df7abce3a434f9a1f15 Reviewed-on: https://go-review.googlesource.com/c/go/+/425361 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-08-26runtime: convert g.selectDone to atomic typehopehook
On the write side, g.selectDone has been converted from non-atomic to atomic access. For #53821. Change-Id: Iac46bc6acce7eed51dfd990285dd57f0d58b4ae2 Reviewed-on: https://go-review.googlesource.com/c/go/+/425414 Run-TryBot: hopehook <hopehook@qq.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-26runtime: convert p.statsSeq to internal atomic typehopehook
For #53821. Change-Id: I1cab3671a29c218b8a927aba9064e63b65900173 Reviewed-on: https://go-review.googlesource.com/c/go/+/425416 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: hopehook <hopehook@qq.com>
2022-08-18runtime: remove p padding field for atomic fields alignmentCuong Manh Le
CL 424396 and CL 424397 changed timer0When/timerModifiedEarliest to atomic.Uint64, just they're guaranted to have 64-bit alignment. Change-Id: Idaff1059da2aac84520b9b0e34f9721a74dbba5a Reviewed-on: https://go-review.googlesource.com/c/go/+/424794 Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-08-18runtime: convert timer0When/timerModifiedEarliest to atomic.Int64Cuong Manh Le
So they match with when/nextwhen fields of timer struct. Updates #53821 Change-Id: Iad0cceb129796745774facfbbfe5756df3a320b4 Reviewed-on: https://go-review.googlesource.com/c/go/+/423117 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2022-08-18runtime: convert m.cgoCallersUse to atomic typeCuong Manh Le
Updates #53821 Change-Id: I99b01f8e91b798e73275635c8a63fcdc4a8df9f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/423888 Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2022-08-18runtime: convert p.timerModifiedEarliest to atomic typeCuong Manh Le
Updates #53821 Change-Id: Iac0d7a3871d9e3ee0ba38ee7ab989faca9c89666 Reviewed-on: https://go-review.googlesource.com/c/go/+/424397 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-08-17runtime: convert p.timer0When to atomic typeCuong Manh Le
Updates #53821 Change-Id: I523ec61116d290ecf7b7e3eb96e468695766cb4d Reviewed-on: https://go-review.googlesource.com/c/go/+/424396 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>