aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/testdata/testprogcgo
AgeCommit message (Collapse)Author
2025-12-23runtime: fix nGsyscallNoP accountingMichael Anthony Knyszek
CL 726964 has two bugs. One is fairly obvious. Where there was previous a decrement of nGsyscallNoP in exitsyscallTryGetP, it added a call to addGSyscallNoP. Oops. The other is more subtle. In needm we set isExtraInC to false very early. This will cause exitsyscall (via cgocallbackg) to decrement nGsyscallNoP when the thread never had a corresponding increment. (It could not have, otherwise it would not have called needm, on Linux anyway.) The fix is simple: increment nGsyscallNoP. CL 726964 actually removed this increment erroneously. I'm pretty sure I removed it because the first bug was the real issue, and removing this increment "fixed it" in the context of the test. I was right that this case was subtle, but wrong about how. Fixes #76435. Change-Id: I1ff1dfbf43bd4cf536b0965da370fee58e3f8753 Reviewed-on: https://go-review.googlesource.com/c/go/+/732320 Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-12-08runtime: disable TestNotInGoMetricCallback on FreeBSD + raceMichael Anthony Knyszek
cgo + race is not supported on FreeBSD. Change-Id: I38abeccaaabfcc104d1d5a077fb99646dc4be792 Reviewed-on: https://go-review.googlesource.com/c/go/+/728120 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2025-12-08runtime: use atomic C types with atomic C functionsJoel Sing
Mark types as _Atomic - fixes breakage introduced in CL 726964 across most LLVM based platforms/builders. Change-Id: I5e64b9ccb0cf5244977a787a52ee124bc03c10de Reviewed-on: https://go-review.googlesource.com/c/go/+/728040 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2025-12-05runtime: don't count nGsyscallNoP for extra Ms in CMichael Anthony Knyszek
In #76435, it turns out that the new metric /sched/goroutines/not-in-go:goroutines counts C threads that have called into Go before (on Linux) as not-in-go goroutines. The reason for this is that the M is still attached to the C thread on Linux as an optimization, so we don't go through all the trouble of detaching the M and, of course, decrementing nGsyscallNoP. There's an easy fix to this accounting issue. The flag on the M, isExtraInC, says whether a thread with an extra M attached no longer has any Go on its (logical) stack. When we take the P from an M in this state, we simply just don't increment nGsyscallNoP. When it calls back into Go, we similarly skip the decrement to nGsyscallNoP. This is more efficient than alternatives, like always updating nGsyscallNoP in cgocallbackg, since that would add a new read-modify-write atomic onto that fast path. It does mean we count threads in C with a P still attached as not-in-go, but this transient in most real programs, assuming the thread indeed does not call back into Go any time soon. Fixes #76435. Change-Id: Id05563bacbe35d3fae17d67fb5ed45fa43fa0548 Reviewed-on: https://go-review.googlesource.com/c/go/+/726964 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-07-29internal/goexperiment: delete RangeFunc goexperimentCherry Mui
It is now always enabeld. The GOEXPERIMENT doesn't control anything. Remove. Change-Id: I50eb09f4537f90ec28152eb59a5a689127843fce Reviewed-on: https://go-review.googlesource.com/c/go/+/684838 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-06-27runtime: remove arbitrary 5-second timeout in TestNeedmDeadlockCherry Mui
The NeedmDeadlock test program currently has a 5-second timeout, which is sort of arbitrary. It is long enough in regular mode (which usually takes 0.0X seconds), but not quite so for configurations like ASAN. Instead of using an arbitrary timeout, just use the test's deadline. The test program is invoked with testenv.Command, which will send it a SIGQUIT before the deadline expires. Fixes #56420 (at least for the asan builder). Change-Id: I0b13651cb07241401837ca2e60eaa1b83275b093 Reviewed-on: https://go-review.googlesource.com/c/go/+/684697 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-05-19runtime: check for gsignal in asancall/msancall/racecallMichael Pratt
asancall and msancall are reachable from the signal handler, where we are running on gsignal. Currently, these calls will use the g0 stack in this case, but if the interrupted code was running on g0 this will corrupt the stack and likely cause a crash. As far as I know, racecall is not reachable from the signal handler, but I have updated it as well for consistency. This is the most straightforward fix, though it would be nice to eventually migrate these wrappers to asmcgocall, which already handled this case. Fixes #71395. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15,gotip-linux-amd64-race Change-Id: I6a6a636ccba826dd53e31c0e85b5d42fb1e98d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/643875 Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-14runtime: only set isExtraInC if there are no Go frames leftMichael Pratt
mp.isExtraInC is intended to indicate that this M has no Go frames at all; it is entirely executing in C. If there was a cgocallback to Go and then a cgocall to C, such that the leaf frames are C, that is fine. e.g., traceback can handle this fine with SetCgoTraceback (or by simply skipping the C frames). However, we currently mismanage isExtraInC, unconditionally setting it on return from cgocallback. This means that if there are two levels of cgocallback, we end up running Go code with isExtraInC set. 1. C-created thread calls into Go function 1 (via cgocallback). 2. Go function 1 calls into C function 1 (via cgocall). 3. C function 1 calls into Go function 2 (via cgocallback). 4. Go function 2 returns back to C function 1 (returning via the remainder of cgocallback). 5. C function 1 returns back to Go function 1 (returning via the remainder of cgocall). 6. Go function 1 is now running with mp.isExtraInC == true. The fix is simple; only set isExtraInC on return from cgocallback if there are no more Go frames. There can't be more Go frames unless there is an active cgocall out of the Go frames. Fixes #72870. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest Change-Id: I6a6a636c4e7ba75a29639d7036c5af3738033467 Reviewed-on: https://go-review.googlesource.com/c/go/+/658035 Reviewed-by: Cherry Mui <cherryyz@google.com> Commit-Queue: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-10-18runtime/testdata: fix for C23 nullptr keywordJoseph Myers
src/runtime/testdata/testprogcgo/threadprof.go contains C code with a variable called nullptr. This conflicts with the nullptr keyword in the C23 revision of the C standard (showing up as gccgo test build failures when updating GCC to use C23 by default when building C code). Rename that variable to nullpointer to avoid the clash with the keyword (any other name that's not a keyword would work just as well). Change-Id: Ida5ef371a3f856c611409884e185c3d5ded8e86c GitHub-Last-Rev: 2ec464703be0507a67a077741789a37511d197e4 GitHub-Pull-Request: golang/go#69927 Reviewed-on: https://go-review.googlesource.com/c/go/+/620955 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-09-11runtime: Goexit on C-created thread report more useful error messageCuong Manh Le
This reverts CL 609296, with the fix for failing builders. Fixes #68275 Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-nocgo,gotip-darwin-amd64-nocgo,gotip-linux-ppc64_power10,gotip-linux-ppc64_power8 Change-Id: I0f539ee7b0be720642eee8885946edccd9c6e04e Reviewed-on: https://go-review.googlesource.com/c/go/+/612335 Reviewed-by: Tim King <taking@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Commit-Queue: Ian Lance Taylor <iant@google.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: qiu laidongfeng2 <2645477756@qq.com>
2024-09-10Revert "runtime: Goexit on C-created thread report more useful error message"Tim King
This reverts CL 602296. Reason for revert: Failing on several builders. Change-Id: I889c566d34294032c330d4f9402300ad0d5d3bf5 Reviewed-on: https://go-review.googlesource.com/c/go/+/611919 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-09-10runtime: Goexit on C-created thread report more useful error messageqiulaidongfeng
Fixes #68275 Change-Id: I47b7a2092f1b4d48aebf437db4e329815c956bb9 GitHub-Last-Rev: b89bf3cab7f9f7611122f535914f2788564643c5 GitHub-Pull-Request: golang/go#69126 Reviewed-on: https://go-review.googlesource.com/c/go/+/609296 Reviewed-by: Tim King <taking@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-08-16cmd/cgo: enable #cgo noescape/nocallbackdoujiang24
In Go 1.22 we added code to the go/build package to ignore #cgo noescape and nocallback directives. That permits us to enable these directives in Go 1.24. Also, this fixed a Bug in CL 497837: After retiring _Cgo_use for parameters, the compiler will treat the parameters, start from the second, as non-alive. Then, they will be marked as scalar in stackmap, which means the pointer won't be copied correctly in copystack. Fixes #56378. Fixes #63739. Change-Id: I46e773240f8a467c3c4ba201dc5b4ee473cf6e3e GitHub-Last-Rev: 42fcc506d6a7681ef24ac36a5904b57bda4b15cd GitHub-Pull-Request: golang/go#66879 Reviewed-on: https://go-review.googlesource.com/c/go/+/579955 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-05-17runtime: fix coro interactions with thread-locked goroutinesMichael Anthony Knyszek
This change fixes problems with thread-locked goroutines using newcoro/coroswitch/etc. Currently, the coro paths do not consider thread-locked goroutines at all and can quickly result in broken scheduler state or lost/leaked goroutines. One possible fix to these issues is to fall back on goroutine+channel semantics, but that turns out to be fairly complicated to implement and results in significant performance cliffs. More complex thread-lock state donation tricks also result in some fairly complicated state tracking that doesn't seem worth it given the use-cases of iter.Pull (and even then, there will be performance cliffs). This change implements a much simpler, but more restrictive semantics. In particular, thread-lock state is tied to the coro at the first call to newcoro (i.e. iter.Pull). From then on, the invariant is that if the coro has any thread-lock state *or* a goroutine calling into coroswitch has any thread-lock state, that the full gamut of thread-lock state must remain the same as it was when newcoro was called (the full gamut meaning internal and external lock counts as well as the identity of the thread that was locked to). This semantics allows the common cases to be always fast, but comes with a non-orthogonality caveat. Specifically, when iter.Pull is used in conjunction with thread-locked goroutines, complex cases (passing next between goroutines or passing yield between goroutines) are likely to fail. Simple cases, where any number of iter.Pull iterators are used in a straightforward way (nested, in series, etc.) from the same goroutine, will work and will be guaranteed to be fast regardless of thread-lock state. This is a compromise for the near-term and we may consider lifting the restrictions imposed by this CL in the future. Fixes #65889. Fixes #65946. Change-Id: I3fb5791e36a61f5ded50226a229a79d28739b24e Reviewed-on: https://go-review.googlesource.com/c/go/+/583675 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Austin Clements <austin@google.com>
2024-05-17internal/trace: remove remnanats of v1 tracerCarlos Amedee
This change removes unused parts of the v1 tracer in preperation of the move of the v2 tracer into the trace package. Updates #67367 Change-Id: I3e53a8afdef72dc90c2d5b514380d1077d284bc7 Reviewed-on: https://go-review.googlesource.com/c/go/+/584537 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17runtime: gofmt -w -sJes Cok
Change-Id: I2eac85b502df9851df294f8d46c7845f635dde9b GitHub-Last-Rev: 3c8382442a0fadb355be9e4656942c2e03db2391 GitHub-Pull-Request: golang/go#64198 Reviewed-on: https://go-review.googlesource.com/c/go/+/542697 Run-TryBot: Jes Cok <xigua67damn@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-02cmd/cgo: disable #cgo noescape/nocallback until Go 1.23Russ Cox
Go 1.21 and earlier do not understand this line, causing "go mod vendor" of //go:build go1.22-tagged code that uses this feature to fail. The solution is to include the go/build change to skip over the line in Go 1.22 (making "go mod vendor" from Go 1.22 onward work with this change) and then wait to deploy the cgo change until Go 1.23, at which point Go 1.21 and earlier will be unsupported. For #56378. Fixes #63293. Change-Id: Ifa08b134eac5a6aa15d67dad0851f00e15e1e58b Reviewed-on: https://go-review.googlesource.com/c/go/+/539235 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-10-26runtime: clear g0 stack bounds in dropmMichael Pratt
After CL 527715, needm uses callbackUpdateSystemStack to set the stack bounds for g0 on an M from the extra M list. Since callbackUpdateSystemStack is also used for recursive cgocallback, it does nothing if the stack is already in bounds. Currently, the stack bounds in an extra M may contain stale bounds from a previous thread that used this M and then returned it to the extra list in dropm. Typically a new thread will not have an overlapping stack with an old thread, but because the old thread has exited there is a small chance that the C memory allocator will allocate the new thread's stack partially or fully overlapping with the old thread's stack. If this occurs, then callbackUpdateSystemStack will not update the stack bounds. If in addition, the overlap is partial such that SP on cgocallback is close to the recorded stack lower bound, then Go may quickly "overflow" the stack and crash with "morestack on g0". Fix this by clearing the stack bounds in dropm, which ensures that callbackUpdateSystemStack will unconditionally update the bounds in needm. For #62440. Change-Id: Ic9e2052c2090dd679ed716d1a23a86d66cbcada7 Reviewed-on: https://go-review.googlesource.com/c/go/+/537695 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> TryBot-Bypass: Michael Pratt <mpratt@google.com>
2023-10-23runtime: allow for 10 more threads in TestWindowsStackMemory*Cuong Manh Le
CL 473415 allowed 5 more threads in TestWindowsStackMemory, to cover sysmon and any new threads in future. However, during go1.22 dev cycle, the test becomes flaky again, failing in windows-386 builder a couple of times in CL 535975 and CL 536175 (and maybe others that haven't caught). This CL increases the extra threads from 5 to 10, hopefully to make the test stable again for windows-386. The theory is that Go process load a bunch of DLLs, which may start their own threads. We could investigate more deeply if the test still be flaky with 10 extra threads. Fixes #58570 Change-Id: I255d0d31ed554859a5046fa76dfae1ba89a89aa3 Reviewed-on: https://go-review.googlesource.com/c/go/+/536058 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-09-12runtime: set _XOPEN_SORUCE_EXTENDED to 1 for aixMichael Pratt
aix requires that _XOPEN_SOURCE_EXTENDED is set to a number, not simply defined. For #62440. Change-Id: Iee221d558b5ad5b8dcb874d4d9fdf94593f7d0a1 Reviewed-on: https://go-review.googlesource.com/c/go/+/527797 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2023-09-12runtime: set _XOPEN_SOURCE_EXTENDED for netbsdMichael Pratt
If _XOPEN_SOURCE is defined, then netbsd also requires _XOPEN_SOURCE_EXTENDED to define stack_t. For #62440. Change-Id: Ib05658c3ca7fae1f6b051566e713ce7bc7c037a0 Reviewed-on: https://go-review.googlesource.com/c/go/+/527775 Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-12runtime: allow update of system stack bounds on callback from C threadMichael Pratt
[This is a redo of CL 525455 with the test fixed on darwin by defining _XOPEN_SOURCE, and disabled with android, musl, and openbsd, which do not provide getcontext.] Since CL 495855, Ms are cached for C threads calling into Go, including the stack bounds of the system stack. Some C libraries (e.g., coroutine libraries) do manual stack management and may change stacks between calls to Go on the same thread. Changing the stack if there is more Go up the stack would be problematic. But if the calls are completely independent there is no particular reason for Go to care about the changing stack boundary. Thus, this CL allows the stack bounds to change in such cases. The primary downside here (besides additional complexity) is that normal systems that do not manipulate the stack may not notice unintentional stack corruption as quickly as before. Note that callbackUpdateSystemStack is written to be usable for the initial setup in needm as well as updating the stack in cgocallbackg. Fixes #62440. For #62130. Change-Id: I0fe0134f865932bbaff1fc0da377c35c013bd768 Reviewed-on: https://go-review.googlesource.com/c/go/+/527715 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-09-11Revert "runtime: allow update of system stack bounds on callback from C thread"Michael Pratt
This reverts CL 525455. The test fails to build on darwin, alpine, and android. For #62440. Change-Id: I39c6b1e16499bd61e0f166de6c6efe7a07961e62 Reviewed-on: https://go-review.googlesource.com/c/go/+/527317 Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-09-11runtime: allow update of system stack bounds on callback from C threadMichael Pratt
Since CL 495855, Ms are cached for C threads calling into Go, including the stack bounds of the system stack. Some C libraries (e.g., coroutine libraries) do manual stack management and may change stacks between calls to Go on the same thread. Changing the stack if there is more Go up the stack would be problematic. But if the calls are completely independent there is no particular reason for Go to care about the changing stack boundary. Thus, this CL allows the stack bounds to change in such cases. The primary downside here (besides additional complexity) is that normal systems that do not manipulate the stack may not notice unintentional stack corruption as quickly as before. Note that callbackUpdateSystemStack is written to be usable for the initial setup in needm as well as updating the stack in cgocallbackg. Fixes #62440. For #62130. Change-Id: I7841b056acea1111bdae3b718345a3bd3961b4a8 Reviewed-on: https://go-review.googlesource.com/c/go/+/525455 Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-05runtime: don't let the tests leave core files behindIan Lance Taylor
Also add a check that we didn't leave any core files behind. Change-Id: I30444ef43ad1a8cc1cacd3b75280f2128e104939 Reviewed-on: https://go-review.googlesource.com/c/go/+/525175 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-08-25cmd/cgo: add #cgo noescape/nocallback annotationsdoujiang24
When passing pointers of Go objects from Go to C, the cgo command generate _Cgo_use(pN) for the unsafe.Pointer type arguments, so that the Go compiler will escape these object to heap. Since the C function may callback to Go, then the Go stack might grow/shrink, that means the pointers that the C function have will be invalid. After adding the #cgo noescape annotation for a C function, the cgo command won't generate _Cgo_use(pN), and the Go compiler won't force the object escape to heap. After adding the #cgo nocallback annotation for a C function, which means the C function won't callback to Go, if it do callback to Go, the Go process will crash. Fixes #56378 Change-Id: Ifdca070584e0d349c7b12276270e50089e481f7a GitHub-Last-Rev: f1a17b08b0590eca2670e404bbfedad5461df72f GitHub-Pull-Request: golang/go#60399 Reviewed-on: https://go-review.googlesource.com/c/go/+/497837 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-08-22all: remove redundant string conversions when formatting []byte with %scui fliter
Change-Id: I603051a3174b139ffb81d20d42979c7f3f04a09a Reviewed-on: https://go-review.googlesource.com/c/go/+/521136 Run-TryBot: Filippo Valsorda <filippo@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> Reviewed-by: Filippo Valsorda <filippo@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-05runtime: move Segv and TgkillSegv to testprogCherry Mui
The non-cgo test points Segv and TgkillSegv are currently in testprogcgo. Although the test points don't explicitly use cgo, being a cgo program, there is still some C code that runs when the test point is invoked, such as thread creation code. For the cgo test points, sometimes we fail to unwind the stack if C code is involved. For the non-cgo ones, we want to always be able to unwind the stack, so we check for stack unwinding failures. But if a signal is landed in the small piece of C code mentioned above, we may still fail to unwind. Move the non-cgo test points to a pure-Go program to avoid this problem. May fix #52963. Updates #59029, #59443, #59492. Change-Id: I35d99a0dd4c7cdb627e2083d2414887a24a2822d Reviewed-on: https://go-review.googlesource.com/c/go/+/500535 Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-06-01runtime: only increment extraMInUse when actually in useMichael Pratt
Currently lockextra always increments extraMInUse, even if the M won't be used (or doesn't even exist), such as in addExtraM. addExtraM fails to decrement extraMInUse, so it stays elevated forever. Fix this bug and simplify the model by moving extraMInUse out of lockextra to getExtraM, where we know the M will actually be used. While we're here, remove the nilokay argument from getExtraM, which is always false. Fixes #60540. Change-Id: I7a5d97456b3bc6ea1baeb06b5b2975e3b8dd96a0 Reviewed-on: https://go-review.googlesource.com/c/go/+/499677 Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-17runtime/cgo: store M for C-created thread in pthread keyCherry Mui
This reapplies CL 485500, with a fix drafted in CL 492987 incorporated. CL 485500 is reverted due to #60004 and #60007. #60004 is fixed in CL 492743. #60007 is fixed in CL 492987 (incorporated in this CL). [Original CL 485500 description] This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 492987 description] On the first call into Go from a C thread, currently we set the g0 stack's high bound imprecisely based on the SP. With CL 485500, we keep the M and don't recompute the stack bounds when it calls into Go again. If the first call is made when the C thread uses some deep stack, but a subsequent call is made with a shallower stack, the SP may be above g0.stack.hi. This is usually okay as we don't check usually stack.hi. One place where we do check for stack.hi is in the signal handler, in adjustSignalStack. In particular, C TSAN delivers signals on the g0 stack (instead of the usual signal stack). If the SP is above g0.stack.hi, we don't see it is on the g0 stack, and throws. This CL makes it get an accurate stack upper bound with the pthread API (on the platforms where it is available). Also add some debug print for the "handler not on signal stack" throw. Fixes #51676. Fixes #59294. Fixes #59678. Fixes #60007. Change-Id: Ie51c8e81ade34ec81d69fd7bce1fe0039a470776 Reviewed-on: https://go-review.googlesource.com/c/go/+/495855 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-12runtime: remove crash_cgo_test CgoRaceSignal timeoutIan Lance Taylor
The test had a 5 second timeout. Running the test on a Darwin system sometimes took less than 5 seconds but often took up to 8 seconds. We don't need a timeout anyhow. Instead, use testenv.Command to run the program, which uses the test timeout. Fixes #59807 Change-Id: Ibf3eda9702731bf98601782f4abd11c3caa0bf40 Reviewed-on: https://go-review.googlesource.com/c/go/+/494456 Run-TryBot: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-05-05Revert "runtime/cgo: store M for C-created thread in pthread key"Chressie Himpel
This reverts CL 485500. Reason for revert: This breaks internal tests at Google, see b/280861579 and b/280820455. Change-Id: I426278d400f7611170918fc07c524cb059b9cc55 Reviewed-on: https://go-review.googlesource.com/c/go/+/492995 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Chressie Himpel <chressie@google.com>
2023-04-26runtime/cgo: store M for C-created thread in pthread keyMichael Pratt
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 485500 description] CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. Fixes #51676. Fixes #59294. Fixes #59678. Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca Reviewed-on: https://go-review.googlesource.com/c/go/+/485500 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-24runtime: fix CgoRaceprof and CgoRaceSignal build failuresBryan C. Mills
TestRaceProf and TestRaceSignal were changed to run on all platforms that support the race detector as of CL 487575, but the testprogcgo source files needed to run the test rely on POSIX threads and were still build-constrained to only linux/amd64 and freebsd/amd64. Since the C test program appears to require only POSIX APIs, update the constraint to build the source file on all Unix platforms, and update the tests to skip on Windows. This may slightly increase testprogcgo build time on Unix platforms that do not support the race detector. Change-Id: I704dd496d475a3cd2e2da2a09c7d2e3bb8e96d02 Reviewed-on: https://go-review.googlesource.com/c/go/+/488115 Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>
2023-04-21runtime: in __tsan_fini tell scheduler we are entering non-Go codeIan Lance Taylor
__tsan_fini will call exit which will call destructors which may in principle call back into Go functions. Prepare the scheduler by calling entersyscall before __tsan_fini. Fixes #59711 Change-Id: Ic4df8fba3014bafa516739408ccfc30aba4f22ad Reviewed-on: https://go-review.googlesource.com/c/go/+/486615 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-04-17Revert "runtime/cgo: store M for C-created thread in pthread key"Michael Pratt
This reverts CL 481061. Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers race detection on `g->stacklo` because the synchronization is in Go, which isn't instrumented. For #51676. For #59294. For #59678. Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47 Reviewed-on: https://go-review.googlesource.com/c/go/+/485275 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2023-04-17Revert "runtime: correct GoCheckBindM's C declaration in EnsureBindM test"Michael Pratt
This reverts CL 482975. Reason for revert: CL 481061 causes C TSAN failures and must be reverted. See CL 485275. This CL depends on CL 481061. For #59678. Change-Id: I4599e93d536149bcec94a5a1542533107699514f Reviewed-on: https://go-review.googlesource.com/c/go/+/485317 Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com>
2023-04-06runtime: correct GoCheckBindM's C declaration in EnsureBindM testCherry Mui
The test file has a C declaration which doesn't match the actual definition. Remove it and include "_cgo_export.h" to have the right declaration. Change-Id: Iddf6d8883ee0e439147c7027029dd3e352ef090d Reviewed-on: https://go-review.googlesource.com/c/go/+/482975 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-04-03runtime/cgo: store M for C-created thread in pthread keydoujiang24
This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. Fixes #51676. Fixes #59294. Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494 Reviewed-on: https://go-review.googlesource.com/c/go/+/481061 Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31Revert "runtime/cgo: store M for C-created thread in pthread key"Cherry Mui
This reverts CL 392854. Reason for revert: caused #59294, which was derived from google internal tests. The attempted fix of #59294 caused more breakage. Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba Reviewed-on: https://go-review.googlesource.com/c/go/+/481060 Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-30runtime/trace: use regular unwinding for cgo callbacksFelix Geisendörfer
Introduce a new m.incgocallback field that is true while C code calls into Go code. Use it in the tracer in order to fallback to the default unwinder instead of frame pointer unwinding for this scenario. The existing fields (incgo, ncgo) were not sufficient to detect the case where a thread created in C calls into Go code. Motivation: 1. Take advantage of a cgo symbolizer, if registered, to unwind through C stacks without frame pointers. 2. Reduce the chance of crashes. It seems unsafe to follow frame pointers when there could be C code that was compiled without frame pointers. Removing the curgp.m.incgocallback check in traceStackID shows the following minor differences between frame pointer unwinding and the default unwinder when there is no cgo symbolizer involved. trace_test.go:60: "goCalledFromCThread": got stack: main.goCalledFromCThread /src/runtime/testdata/testprogcgo/trace.go:58 _cgoexp_45c15a3efb3a_goCalledFromCThread _cgo_gotypes.go:694 runtime.cgocallbackg1 /src/runtime/cgocall.go:318 runtime.cgocallbackg /src/runtime/cgocall.go:236 runtime.cgocallback /src/runtime/asm_amd64.s:998 crosscall2 /src/runtime/cgo/asm_amd64.s:30 want stack: main.goCalledFromCThread /src/runtime/testdata/testprogcgo/trace.go:58 _cgoexp_45c15a3efb3a_goCalledFromCThread _cgo_gotypes.go:694 runtime.cgocallbackg1 /src/runtime/cgocall.go:318 runtime.cgocallbackg /src/runtime/cgocall.go:236 runtime.cgocallback /src/runtime/asm_amd64.s:998 trace_test.go:60: "goCalledFromC": got stack: main.goCalledFromC /src/runtime/testdata/testprogcgo/trace.go:51 _cgoexp_45c15a3efb3a_goCalledFromC _cgo_gotypes.go:687 runtime.cgocallbackg1 /src/runtime/cgocall.go:318 runtime.cgocallbackg /src/runtime/cgocall.go:236 runtime.cgocallback /src/runtime/asm_amd64.s:998 crosscall2 /src/runtime/cgo/asm_amd64.s:30 runtime.asmcgocall /src/runtime/asm_amd64.s:848 main._Cfunc_cCalledFromGo _cgo_gotypes.go:263 main.goCalledFromGo /src/runtime/testdata/testprogcgo/trace.go:46 main.Trace /src/runtime/testdata/testprogcgo/trace.go:37 main.main /src/runtime/testdata/testprogcgo/main.go:34 want stack: main.goCalledFromC /src/runtime/testdata/testprogcgo/trace.go:51 _cgoexp_45c15a3efb3a_goCalledFromC _cgo_gotypes.go:687 runtime.cgocallbackg1 /src/runtime/cgocall.go:318 runtime.cgocallbackg /src/runtime/cgocall.go:236 runtime.cgocallback /src/runtime/asm_amd64.s:998 runtime.systemstack_switch /src/runtime/asm_amd64.s:463 runtime.cgocall /src/runtime/cgocall.go:168 main._Cfunc_cCalledFromGo _cgo_gotypes.go:263 main.goCalledFromGo /src/runtime/testdata/testprogcgo/trace.go:46 main.Trace /src/runtime/testdata/testprogcgo/trace.go:37 main.main /src/runtime/testdata/testprogcgo/main.go:34 For #16638 Change-Id: I95fa27a3170c5abd923afc6eadab4eae777ced31 Reviewed-on: https://go-review.googlesource.com/c/go/+/474916 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
2023-03-24runtime/cgo: store M for C-created thread in pthread keydoujiang24
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz Fixes #51676 Change-Id: I380702fe2f9b6b401b2d6f04b0aba990f4b9ee6c GitHub-Last-Rev: 93dc64ad98e5583372e41f65ee4b7ab78b5aff51 GitHub-Pull-Request: golang/go#51679 Reviewed-on: https://go-review.googlesource.com/c/go/+/392854 Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: thepudds <thepudds1460@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-07runtime: allow for 5 more threads in TestWindowsStackMemory*Alex Brainman
Original version of TestWindowsStackMemory did not consider sysmon and other threads running during the test. Allow for 5 extra threads in this test - this should cover any new threads in the future. Fixes #58570 Change-Id: I215790f9b94ff40a32ddd7aa54af715d1dc391c6 Reviewed-on: https://go-review.googlesource.com/c/go/+/473415 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Alex Brainman <alex.brainman@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-02runtime: eliminate arbitrary timeout in TestCgoLockOSThreadExitBryan C. Mills
This test previously failed if running a new pthread took longer than a hard-coded 100ms. On some slow or heavily-loaded builders, that scheduling latency is too short. Since the point of this test is to verify that the background thread is not reused after it terminates (see #20395), the arbitrary time limit does not seem helpful: if the background thread fails to terminate the test will time out on its own, and if the main goroutine is scheduled on the background thread the test will fail regardless of how long it takes. Fixes #58247. Change-Id: I626af52aac55af7a4c0e7829798573c479750c20 Reviewed-on: https://go-review.googlesource.com/c/go/+/464735 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-01runtime: fix missing error print in TestCgoSigfwdAustin Clements
The result of the call to fmt.Errorf was unused. It was clearly intending to print the message, not simply construct an error. Change-Id: I14856214c521a51fe4b45690e6c35fbb17e66577 Reviewed-on: https://go-review.googlesource.com/c/go/+/443375 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-17runtime: run TestCgoSigfwd on all Unix platformsAustin Clements
This test was originally Linux-only, but there doesn't seem to be anything Linux-specific in it. Change-Id: I0f8519eff5dbed97f5e21e1c8e5ab0d747d51df3 Reviewed-on: https://go-review.googlesource.com/c/go/+/443073 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-17runtime: improve coverage of TestCgoSigfwdAustin Clements
Currently, TestCgoSigfwd will pass incorrectly if the SIGSEGV that originates in Go mistakenly goes to the C SIGSEGV handler. Fix this by adding a signal-atomic variable that tracks what the expected behavior is. Change-Id: Id2a9fa3b209299dccf90bb60720b89ad96838a9c Reviewed-on: https://go-review.googlesource.com/c/go/+/443072 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-17misc/cgo/testsigfwd: move to runtime/testprog/testprogcgoAustin Clements
This migrates testsigfwd, which uses some one-off build infrastructure, to be part of the runtime's testprogcgo. The test is largely unchanged. Because it's part of a larger binary, this CL renames a few things and gates the constructor-time signal handler registration on an environment variable. This CL also replaces an errant fmt.Errorf with fmt.Fprintf. For #37486, since it eliminates a non-go-test from dist. Change-Id: I0efd146ea0a0a3f0b361431349a419af0f0ecc61 Reviewed-on: https://go-review.googlesource.com/c/go/+/443068 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-30runtime/trace: skip tests if parsing fails with timestamp errordoujiang24
already skips tests in case of the timestamp error, eg. #97757 Change-Id: Ia696e83cba2e3ed50181a8100b964847092a7365 GitHub-Last-Rev: 8e5f607e14f6a15ed6da5f205c4ca67a4adb6fc8 GitHub-Pull-Request: golang/go#55918 Reviewed-on: https://go-review.googlesource.com/c/go/+/435855 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2022-09-27runtime/trace: add missing events for the locked g in extra M.doujiang24
Extra Ms may lead to the "no consistent ordering of events possible" error when parsing trace file with cgo enabled, since: 1. The gs in the extra Ms may be in `_Gdead` status while starting trace by invoking `runtime.StartTrace`, 2. and these gs will trigger `traceEvGoSysExit` events in `runtime.exitsyscall` when invoking go functions from c, 3. then, the events of those gs are under non-consistent ordering, due to missing the previous events. Add two events, `traceEvGoCreate` and `traceEvGoInSyscall`, in `runtime.StartTrace`, will make the trace parser happy. Fixes #29707 Change-Id: I2fd9d1713cda22f0ddb36efe1ab351f88da10881 GitHub-Last-Rev: 7bbfddb81b70041250e3c59ce53bea44f7afd2c3 GitHub-Pull-Request: golang/go#54974 Reviewed-on: https://go-review.googlesource.com/c/go/+/429858 Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: xie cui <523516579@qq.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com>