aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/asm_arm.s
AgeCommit message (Collapse)Author
2026-03-18runtime: implement part of library initialization in Goqmuntal
All architectures supporting c-shared and c-archive share the same initialization code in assembly, and most of it can be implemented in pure Go. Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64-longtest,gotip-linux-ppc64le_power10,gotip-linux-riscv64,gotip-linux-loong64,gotip-linux-s390x Change-Id: Iaa9fb7d6f9ca8785f1098461646d607ef6b00d47 Reviewed-on: https://go-review.googlesource.com/c/go/+/706417 Auto-Submit: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2026-03-18runtime: make asmcgocall more robust to missing Gqmuntal
Being able to call asmcgocall without a G is useful for code shared between different stages of the runtime initialization and thread creation. Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_15,gotip-linux-mips64le,gotip-linux-ppc64le_power10,gotip-linux-riscv64,gotip-openbsd-ppc64,gotip-openbsd-amd64 Change-Id: Ic427764de197e648e8b9987c98c3b7521512cc5c Reviewed-on: https://go-review.googlesource.com/c/go/+/750541 Reviewed-by: Carlos Amedee <carlos@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2025-10-20runtime: wrap procyield assembly and check for 0Michael Anthony Knyszek
procyield will currently loop infinitely if passed 0 on several platforms. This change sidesteps this bug by renaming procyield to procyieldAsm, and adding a wrapper named procyield that checks for cycles == 0. The benefit of this structure is that procyield called with a constant cycle count of 0 will be inlined and constant folded away, the expected behavior of a procyield of 0 cycles. A follow-up change will fix the assembly to not have this footgun anymore. Change-Id: I7068abfeb961bc0fa475e216836f7c0e46b38373 Reviewed-on: https://go-review.googlesource.com/c/go/+/712663 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2025-08-06all: remove support for windows/armqiulaidongfeng
Also CL 690655 for golang.org/x/sys. For #71671 Change-Id: Iceb369dec5affb944a39d07cdabfd7add6f1f319 Reviewed-on: https://go-review.googlesource.com/c/go/+/648795 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Mark Freeman <markfreeman@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-29cmd/compile: move arm32 over to new bounds check strategyKeith Randall
Change-Id: I529edd805875a4833cabcf4692f0c6d4163b07d2 Reviewed-on: https://go-review.googlesource.com/c/go/+/682398 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-26runtime: remove ret field from gobufKeith Randall
It's not used for anything. Change-Id: I031b3cdfe52b6b1cff4b3cb6713ffe588084542f Reviewed-on: https://go-review.googlesource.com/c/go/+/652276 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-25cmd/compile, runtime: use PC of deferreturn for panic transferDavid Chase
this removes the old conditional-on-register-value handshake from the deferproc/deferprocstack logic. The "line" for the recovery-exit frame itself (not the defers that it runs) is the closing brace of the function. Reduces code size slightly (e.g. go command is 0.2% smaller) Sample output showing effect of this change, also what sort of code it requires to observe the effect: ``` package main import "os" func main() { g(len(os.Args) - 1) // stack[0] } var gi int var pi *int = &gi //go:noinline func g(i int) { switch i { case 0: defer func() { println("g0", i) q() // stack[2] if i == 0 }() for j := *pi; j < 1; j++ { defer func() { println("recover0", recover().(string)) }() } default: for j := *pi; j < 1; j++ { defer func() { println("g1", i) q() // stack[2] if i == 1 }() } defer func() { println("recover1", recover().(string)) }() } p() } // stack[1] (deferreturn) //go:noinline func p() { panic("p()") } //go:noinline func q() { panic("q()") // stack[3] } /* Sample output for "./foo foo": recover1 p() g1 1 panic: q() goroutine 1 [running]: main.q() .../main.go:46 +0x2c main.g.func3() .../main.go:29 +0x48 main.g(0x1?) .../main.go:37 +0x68 main.main() .../main.go:6 +0x28 */ ``` Change-Id: Ie39ea62ecc244213500380ea06d44024cadc2317 Reviewed-on: https://go-review.googlesource.com/c/go/+/650795 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-23runtime: add crash stack support for armMauri de Souza Meneguzzo
Change-Id: Ide4002d1cf82f2daaf7261b367c391dedbbf7719 GitHub-Last-Rev: 80ee248c3e34529e7a522acc97db9fb69c82dffb GitHub-Pull-Request: golang/go#65308 Reviewed-on: https://go-review.googlesource.com/c/go/+/558699 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-20all: add floating point option for ARM targetsLudi Rehak
This change introduces new options to set the floating point mode on ARM targets. The GOARM version number can optionally be followed by ',hardfloat' or ',softfloat' to select whether to use hardware instructions or software emulation for floating point computations, respectively. For example, GOARM=7,softfloat. Previously, software floating point support was limited to GOARM=5. With these options, software floating point is now extended to all ARM versions, including GOARM=6 and 7. This change also extends hardware floating point to GOARM=5. GOARM=5 defaults to softfloat and GOARM=6 and 7 default to hardfloat. For #61588 Change-Id: I23dc86fbd0733b262004a2ed001e1032cf371e94 Reviewed-on: https://go-review.googlesource.com/c/go/+/514907 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-06-14all: fix spelling errorsAlexander Yastrebov
Fix spelling errors discovered using https://github.com/codespell-project/codespell. Errors in data files and vendored packages are ignored. Change-Id: I83c7818222f2eea69afbd270c15b7897678131dc GitHub-Last-Rev: 3491615b1b82832cc0064f535786546e89aa6184 GitHub-Pull-Request: golang/go#60758 Reviewed-on: https://go-review.googlesource.com/c/go/+/502576 Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-06-10runtime: fix typosJes Cok
Change-Id: If13f4d4bc545f78e3eb8c23cf2e63f0eb273d71f GitHub-Last-Rev: 32ca70f52a5c3dd66f18535c5e595e66afb3903c GitHub-Pull-Request: golang/go#60703 Reviewed-on: https://go-review.googlesource.com/c/go/+/502055 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-17runtime/cgo: store M for C-created thread in pthread keyCherry Mui
This reapplies CL 485500, with a fix drafted in CL 492987 incorporated. CL 485500 is reverted due to #60004 and #60007. #60004 is fixed in CL 492743. #60007 is fixed in CL 492987 (incorporated in this CL). [Original CL 485500 description] This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 492987 description] On the first call into Go from a C thread, currently we set the g0 stack's high bound imprecisely based on the SP. With CL 485500, we keep the M and don't recompute the stack bounds when it calls into Go again. If the first call is made when the C thread uses some deep stack, but a subsequent call is made with a shallower stack, the SP may be above g0.stack.hi. This is usually okay as we don't check usually stack.hi. One place where we do check for stack.hi is in the signal handler, in adjustSignalStack. In particular, C TSAN delivers signals on the g0 stack (instead of the usual signal stack). If the SP is above g0.stack.hi, we don't see it is on the g0 stack, and throws. This CL makes it get an accurate stack upper bound with the pthread API (on the platforms where it is available). Also add some debug print for the "handler not on signal stack" throw. Fixes #51676. Fixes #59294. Fixes #59678. Fixes #60007. Change-Id: Ie51c8e81ade34ec81d69fd7bce1fe0039a470776 Reviewed-on: https://go-review.googlesource.com/c/go/+/495855 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-05Revert "runtime/cgo: store M for C-created thread in pthread key"Chressie Himpel
This reverts CL 485500. Reason for revert: This breaks internal tests at Google, see b/280861579 and b/280820455. Change-Id: I426278d400f7611170918fc07c524cb059b9cc55 Reviewed-on: https://go-review.googlesource.com/c/go/+/492995 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Chressie Himpel <chressie@google.com>
2023-04-26runtime/cgo: store M for C-created thread in pthread keyMichael Pratt
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 485500 description] CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. Fixes #51676. Fixes #59294. Fixes #59678. Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca Reviewed-on: https://go-review.googlesource.com/c/go/+/485500 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-21runtime: tidy _Stack* constant namingAustin Clements
For #59670. Change-Id: I0efa743edc08e48dc8d906803ba45e9f641369db Reviewed-on: https://go-review.googlesource.com/c/go/+/486977 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com>
2023-04-20Revert "runtime: tidy _Stack* constant naming"Austin Clements
This reverts commit CL 486381. Submitted out of order and breaks bootstrap. Change-Id: Ia472111cb966e884a48f8ee3893b3bf4b4f4f875 Reviewed-on: https://go-review.googlesource.com/c/go/+/486915 Reviewed-by: David Chase <drchase@google.com> TryBot-Bypass: Austin Clements <austin@google.com>
2023-04-20runtime: tidy _Stack* constant namingAustin Clements
For #59670. Change-Id: I4476d6f92663e8a825d063d6e6a7fc9a2ac99d4d Reviewed-on: https://go-review.googlesource.com/c/go/+/486381 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-04-17Revert "runtime/cgo: store M for C-created thread in pthread key"Michael Pratt
This reverts CL 481061. Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers race detection on `g->stacklo` because the synchronization is in Go, which isn't instrumented. For #51676. For #59294. For #59678. Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47 Reviewed-on: https://go-review.googlesource.com/c/go/+/485275 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2023-04-03runtime/cgo: store M for C-created thread in pthread keydoujiang24
This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. Fixes #51676. Fixes #59294. Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494 Reviewed-on: https://go-review.googlesource.com/c/go/+/481061 Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31Revert "runtime/cgo: store M for C-created thread in pthread key"Cherry Mui
This reverts CL 392854. Reason for revert: caused #59294, which was derived from google internal tests. The attempted fix of #59294 caused more breakage. Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba Reviewed-on: https://go-review.googlesource.com/c/go/+/481060 Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31Revert "runtime: fix ARM assembly code in cgocallback"Cherry Mui
This reverts CL 479255. Reason for revert: need to revert CL 392854, and this caused a conflict. Change-Id: I6cb105c62e51b47de3f652df5f5ee92673a93919 Reviewed-on: https://go-review.googlesource.com/c/go/+/481058 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-24runtime: fix ARM assembly code in cgocallbackCherry Mui
A comparison instruction was missing in CL 392854. Should fix ARM builders. For #51676. Change-Id: Ica27a99be10e595bab4fad35e2e6c00a1c68a662 Reviewed-on: https://go-review.googlesource.com/c/go/+/479255 TryBot-Bypass: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-03-24runtime/cgo: store M for C-created thread in pthread keydoujiang24
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz Fixes #51676 Change-Id: I380702fe2f9b6b401b2d6f04b0aba990f4b9ee6c GitHub-Last-Rev: 93dc64ad98e5583372e41f65ee4b7ab78b5aff51 GitHub-Pull-Request: golang/go#51679 Reviewed-on: https://go-review.googlesource.com/c/go/+/392854 Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: thepudds <thepudds1460@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-24cmd/compile: batch write barrier callsKeith Randall
Have the write barrier call return a pointer to a buffer into which the generated code records pointers that need write barrier treatment. Change-Id: I7871764298e0aa1513de417010c8d46b296b199e Reviewed-on: https://go-review.googlesource.com/c/go/+/447781 Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Bypass: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-17cmd/compile: move raw writes out of write barrier codeKeith Randall
Previously, the write barrier calls themselves did the actual writes to memory. Instead, move those writes out to a common location that both the wb-enabled and wb-disabled code paths share. This enables us to optimize the write barrier path without having to worry about performing the actual writes. Change-Id: Ia71ab651908ec124cc33141afb52e4ca19733ac6 Reviewed-on: https://go-review.googlesource.com/c/go/+/447780 Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Bypass: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-17runtime: remove the restriction that write barrier ptrs come in pairsKeith Randall
Future CLs will remove the invariant that pointers are always put in the write barrier in pairs. The behavior of the assembly code changes a bit, where instead of writing the pointers unconditionally and then checking for overflow, check for overflow first and then write the pointers. Also changed the write barrier flush function to not take the src/dst as arguments. Change-Id: I2ef708038367b7b82ea67cbaf505a1d5904c775c Reviewed-on: https://go-review.googlesource.com/c/go/+/447779 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Bypass: Keith Randall <khr@golang.org>
2022-08-25runtime: mark morestack_noctxt SPWRITE on LR architecturesCherry Mui
On LR architectures, morestack (and morestack_noctxt) are called with a special calling convention, where the caller doesn't save LR on stack but passes it as a register, which morestack will save to g.sched.lr. The stack unwinder currently doesn't understand it, and would fail to unwind from it. morestack already writes SP (as it switches stack), but morestack_noctxt (which tailcalls morestack) doesn't. If a profiling signal lands right in morestack_noctxt, the unwinder will try to unwind the stack and go off, and possibly crash. Marking morestack_noctxt SPWRITE stops the unwinding. Ideally we could teach the unwinder about the special calling convention, or change the calling convention to be less special (so the unwinder doesn't need to fetch a register from the signal context). This is a stop-gap solution, to stop the unwinder from crashing. Fixes #54332. Change-Id: I75295f2e27ddcf05f1ea0b541aedcb9000ae7576 Reviewed-on: https://go-review.googlesource.com/c/go/+/425396 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-08-16runtime: make asmcgocall g0/gsignal checks consistentJoel Sing
In asmcgocall() we need to switch to the g0 stack if we're not already on the g0 stack or the gsignal stack. The prefered way of doing this is to check gsignal first, then g0, since if we are going to switch to g0 we will need g0 handy (thus avoiding a second load). Rewrite/reorder 386 and amd64 to check gsignal first - this shaves a few assembly instructions off and makes the order consistent with arm, arm64, mips64 and ppc64. Add missing gsignal checks to mips, riscv64 and s390x. Change-Id: I1b027bf393c25e0c33e1d8eb80de67e4a0a3f561 Reviewed-on: https://go-review.googlesource.com/c/go/+/335869 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-08-03[dev.typeparams] runtime,cmd/compile,cmd/link: replace jmpdefer with a loopAustin Clements
Currently, deferreturn runs deferred functions by backing up its return PC to the deferreturn call, and then effectively tail-calling the deferred function (via jmpdefer). The effect of this is that the deferred function appears to be called directly from the deferee, and when it returns, the deferee calls deferreturn again so it can run the next deferred function if necessary. This unusual flow control leads to a large number of special cases and complications all over the tool chain. This used to be necessary because deferreturn copied the deferred function's argument frame directly into its caller's frame and then had to invoke that call as if it had been called from its caller's frame so it could access it arguments. But now that we've simplified defer processing so the runtime only deals with argument-less closures, this approach is no longer necessary. This CL simplifies all of this by making deferreturn simply call deferred functions in a loop. This eliminates the need for jmpdefer, so we can delete a bunch of per-architecture assembly code. This eliminates several special cases on Wasm, since it couldn't support these calling shenanigans directly and thus had to simulate the loop a different way. Now Wasm can largely work the way the other platforms do. This eliminates the per-architecture Ginsnopdefer operation. On PPC64, this was necessary to reload the TOC pointer after the tail call (since TOC pointers in general make tail calls impossible). The tail call is gone, and in the case where we do force a jump to the deferreturn call when recovering from an open-coded defer, we go through gogo (via runtime.recovery), which handles the TOC. On other platforms, we needed a NOP so traceback didn't get confused by seeing the return to the CALL instruction, rather than the usual return to the instruction following the CALL instruction. Now we don't inject a return to the CALL instruction at all, so this NOP is also unnecessary. The one potential effect of this is that deferreturn could now appear in stack traces from deferred functions. However, this could already happen from open-coded defers, so we've long since marked deferreturn as a "wrapper" so it gets elided not only from printed stack traces, but from runtime.Callers*. This is a retry of CL 337652 because we had to back out its parent. There are no changes in this version. Change-Id: I3f54b7fec1d7ccac71cc6cf6835c6a46b7e5fb6c Reviewed-on: https://go-review.googlesource.com/c/go/+/339397 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-07-30[dev.typeparams] Revert "[dev.typeparams] runtime,cmd/compile,cmd/link: ↵Austin Clements
replace jmpdefer with a loop" This reverts CL 227652. I'm reverting CL 337651 and this builds on top of it. Change-Id: I03ce363be44c2a3defff2e43e7b1aad83386820d Reviewed-on: https://go-review.googlesource.com/c/go/+/338709 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-07-30[dev.typeparams] runtime,cmd/compile,cmd/link: replace jmpdefer with a loopAustin Clements
Currently, deferreturn runs deferred functions by backing up its return PC to the deferreturn call, and then effectively tail-calling the deferred function (via jmpdefer). The effect of this is that the deferred function appears to be called directly from the deferee, and when it returns, the deferee calls deferreturn again so it can run the next deferred function if necessary. This unusual flow control leads to a large number of special cases and complications all over the tool chain. This used to be necessary because deferreturn copied the deferred function's argument frame directly into its caller's frame and then had to invoke that call as if it had been called from its caller's frame so it could access it arguments. But now that we've simplified defer processing so the runtime only deals with argument-less closures, this approach is no longer necessary. This CL simplifies all of this by making deferreturn simply call deferred functions in a loop. This eliminates the need for jmpdefer, so we can delete a bunch of per-architecture assembly code. This eliminates several special cases on Wasm, since it couldn't support these calling shenanigans directly and thus had to simulate the loop a different way. Now Wasm can largely work the way the other platforms do. This eliminates the per-architecture Ginsnopdefer operation. On PPC64, this was necessary to reload the TOC pointer after the tail call (since TOC pointers in general make tail calls impossible). The tail call is gone, and in the case where we do force a jump to the deferreturn call when recovering from an open-coded defer, we go through gogo (via runtime.recovery), which handles the TOC. On other platforms, we needed a NOP so traceback didn't get confused by seeing the return to the CALL instruction, rather than the usual return to the instruction following the CALL instruction. Now we don't inject a return to the CALL instruction at all, so this NOP is also unnecessary. The one potential effect of this is that deferreturn could now appear in stack traces from deferred functions. However, this could already happen from open-coded defers, so we've long since marked deferreturn as a "wrapper" so it gets elided not only from printed stack traces, but from runtime.Callers*. Change-Id: Ie9f700cd3fb774f498c9edce363772a868407bf7 Reviewed-on: https://go-review.googlesource.com/c/go/+/337652 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-06-08[dev.typeparams] cmd/compile, runtime: remove the siz argument of ↵Cherry Mui
newproc/deferproc newproc/deferproc takes a siz argument for the go'd/deferred function's argument size. Now it is always zero. Remove the argument. Change-Id: If1bb8d427e34015ccec0ba10dbccaae96757fa8c Reviewed-on: https://go-review.googlesource.com/c/go/+/325917 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-05-21[dev.typeparams] runtime: fix newproc arg size on ARMCherry Mui
At runtime startup it calls newproc from assembly code to start the main goroutine. runtime.main has no arguments, so the arg size should be 0, instead of 8. While here, use clearer code sequence to open the frame. Change-Id: I2bbb26a83521ea867897530b86a85b22a3c8be9d Reviewed-on: https://go-review.googlesource.com/c/go/+/321957 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-05-09runtime: switch openbsd/arm to pthreadsJoel Sing
This switches openbsd/arm to thread creation via pthreads, rather than doing direct system calls. Update #36435 Change-Id: Ia8749e3723a9967905c33b6d93dfd9be797a486c Reviewed-on: https://go-review.googlesource.com/c/go/+/315790 Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-05-05runtime: remove redundant save_g call in mcall for armJoel Sing
The setg call a few lines earlier has already performed the same iscgo check and called save_g if necessary. Change-Id: I6e7c44cef4e0397d6001a3d5b7e334cdfbc3ce22 Reviewed-on: https://go-review.googlesource.com/c/go/+/316929 Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-21cmd/compile: allow conversion from slice to array ptrJosh Bleecher Snyder
Panic if the slice is too short. Updates #395 Change-Id: I90f4bff2da5d8f3148ba06d2482084f32b25c29a Reviewed-on: https://go-review.googlesource.com/c/go/+/301650 Trust: Josh Bleecher Snyder <josharian@gmail.com> Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-02-19runtime: fix spurious stack overflow detectionRuss Cox
The regabi builders are unhappy about badctxt calling throw calling systemstack calling gosave_systemstack_switch calling badctxt, all nosplit, repeating. This wouldn't actually happen since after one systemstack we'd end up on the system stack and the next one wouldn't call gosave_systemstack_switch at all. The badctxt call itself is in a very unlikely assertion failure inside gosave_systemstack_switch. Keep the assertion check but call runtime.abort instead on failure, breaking the detected (but not real) cycle. Change-Id: Iaf5c0fc065783b8c1c6d0f62d848f023a0714b96 Reviewed-on: https://go-review.googlesource.com/c/go/+/294069 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: use TOPFRAME to identify top-of-frame functionsRuss Cox
No change to actual runtime, but helps reduce the laundry list of functions. mcall, morestack, and asmcgocall are not actually top-of-frame, so those need more attention in follow-up CLs. mstart moved to assembly so that it can be marked TOPFRAME. Since TOPFRAME also tells DWARF consumers not to unwind this way, this change should also improve debuggers a marginal amount. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: If1e0d46ca973de5e46b62948d076f675f285b5d9 Reviewed-on: https://go-review.googlesource.com/c/go/+/288802 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: use FuncInfo SPWRITE flag to identify untraceable profile samplesRuss Cox
The old code was very clever about predicting whether a traceback was safe. That cleverness has not aged well. In particular, the setsSP function is missing a bunch of functions that write to SP and will confuse traceback. And one such function - jmpdefer - was handled as a special case in gentraceback instead of simply listing it in setsSP. Throw away all the clever prediction about whether traceback will crash. Instead, make traceback NOT crash, by checking whether the function being walked writes to SP. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: I3d55fe257a22745e4919ac4dc9a9378c984ba0da Reviewed-on: https://go-review.googlesource.com/c/go/+/288801 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: remove unnecessary writes to gp.sched.gRuss Cox
A g's sched.g is set in newproc1: newg.sched.g = guintptr(unsafe.Pointer(newg)) After that, it never changes. Yet lots of assembly code does "g.sched.g = g" unnecessarily. Remove all those lines to avoid confusion about whether it ever changes. Also, split gogo into two functions, one that does the nil g check and a second that does the actual switch. This way, if the nil g check fails, we get a stack trace showing the call stack that led to the failure. (The SP write would otherwise cause the stack trace to abort.) Also restore the proper nil g check in a handful of assembly functions. (There is little point in checking for nil g *after* installing it as the real g.) Change-Id: I22866b093f901f765de1d074e36eeec10366abfb Reviewed-on: https://go-review.googlesource.com/c/go/+/292109 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: unify asmcgocall and systemstack traceback setupRuss Cox
Both asmcgocall and systemstack need to save the calling Go code's context for use by traceback, but they do it differently. Systemstack's appraoch is better, because it doesn't require a special case in traceback. So make them both use that. While we are here, the fake mstart caller in systemstack is no longer needed and can be removed. (traceback knows to stop in systemstack because of the writes to SP.) Also remove the fake mstarts in sys_windows_*.s. And while we are there, fix the control flow guard code in sys_windows_arm.s. The current code is using pointers to a stack frame that technically is gone once we hit the RET instruction. Clearly it's working OK, but better not to depend on data below SP being preserved, even for just a few instructions. Store the value we need in other registers instead. (This code is only used for pushing a sigpanic call, which does not actually return to the site of the fault and therefore doesn't need to preserve any of the registers.) This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: Id1e3ef5e54f7ad786e4b87043f2626eba7c3bbd9 Reviewed-on: https://go-review.googlesource.com/c/go/+/288799 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: clean up system calls during cgo callback initRuss Cox
During a cgocallback, the runtime calls needm to get an m. The calls made during needm cannot themselves assume that there is an m or a g (which is attached to the m). In the old days of making direct system calls, the only thing you had to do for such functions was mark them //go:nosplit, to avoid the use of g in the stack split prologue. But now, on operating systems that make system calls through shared libraries and use code that saves state in the g or m before doing so, it's not safe to assume g exists. In fact, it is not even safe to call getg(), because it might fault deferencing the TLS storage to find the g pointer (that storage may not be initialized yet, at least on Windows, and perhaps on other systems in the future). The specific routines that are problematic are usleep and osyield, which are called during lock contention in lockextra, called from needm. All this is rather subtle and hidden, so in addition to fixing the problem on Windows, this CL makes the fact of not running on a g much clearer by introducing variants usleep_no_g and osyield_no_g whose names should make clear that there is no g. And then we can remove the various sketchy getg() == nil checks in the existing routines. As part of this cleanup, this CL also deletes onosstack on Windows. onosstack is from back when the runtime was implemented in C. It predates systemstack but does essentially the same thing. Instead of having two different copies of this code, we can use systemstack consistently. This way we need not port onosstack to each architecture. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: I3352de1fd0a3c26267c6e209063e6e86abd26187 Reviewed-on: https://go-review.googlesource.com/c/go/+/288793 Trust: Russ Cox <rsc@golang.org> Trust: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
2021-02-16[dev.regabi] reflect: support for register ABI on amd64 for reflect.(Value).CallMichael Anthony Knyszek
This change adds support for the new register ABI on amd64 to reflect.(Value).Call. If internal/abi's register counts are non-zero, reflect will try to set up arguments in registers on the Call path. Note that because the register ABI becomes ABI0 with zero registers available, this should keep working as it did before. This change does not add any tests for the register ABI case because there's no way to do so at the moment. For #40724. Change-Id: I8aa089a5aa5a31b72e56b3d9388dd3f82203985b Reviewed-on: https://go-review.googlesource.com/c/go/+/272568 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2021-02-05[dev.regabi] runtime: delete gosave functionCherry Zhang
The runtime.gosave function is not used anywhere. Delete. Note: there is also a gosave<> function, which is actually used and not deleted. Change-Id: I64149a7afdd217de26d1e6396233f2becfad7153 Reviewed-on: https://go-review.googlesource.com/c/go/+/289719 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-10-26runtime,cmd/cgo: simplify C -> Go call pathAustin Clements
This redesigns the way calls work from C to exported Go functions. It removes several steps from the call path, makes cmd/cgo no longer sensitive to the Go calling convention, and eliminates the use of reflectcall from cgo. In order to avoid generating a large amount of FFI glue between the C and Go ABIs, the cgo tool has long depended on generating a C function that marshals the arguments into a struct, and then the actual ABI switch happens in functions with fixed signatures that simply take a pointer to this struct. In a way, this CL simply pushes this idea further. Currently, the cgo tool generates this argument struct in the exact layout of the Go stack frame and depends on reflectcall to unpack it into the appropriate Go call (even though it's actually reflectcall'ing a function generated by cgo). In this CL, we decouple this struct from the Go stack layout. Instead, cgo generates a Go function that takes the struct, unpacks it, and calls the exported function. Since this generated function has a generic signature (like the rest of the call path), we don't need reflectcall and can instead depend on the Go compiler itself to implement the call to the exported Go function. One complication is that syscall.NewCallback on Windows, which converts a Go function into a C function pointer, depends on cgocallback's current dynamic calling approach since the signatures of the callbacks aren't known statically. For this specific case, we continue to depend on reflectcall. Really, the current approach makes some overly simplistic assumptions about translating the C ABI to the Go ABI. Now we're at least in a much better position to do a proper ABI translation. For comparison, the current cgo call path looks like: GoF (generated C function) -> crosscall2 (in cgo/asm_*.s) -> _cgoexp_GoF (generated Go function) -> cgocallback (in asm_*.s) -> cgocallback_gofunc (in asm_*.s) -> cgocallbackg (in cgocall.go) -> cgocallbackg1 (in cgocall.go) -> reflectcall (in asm_*.s) -> _cgoexpwrap_GoF (generated Go function) -> p.GoF Now the call path looks like: GoF (generated C function) -> crosscall2 (in cgo/asm_*.s) -> cgocallback (in asm_*.s) -> cgocallbackg (in cgocall.go) -> cgocallbackg1 (in cgocall.go) -> _cgoexp_GoF (generated Go function) -> p.GoF Notably: 1. We combine _cgoexp_GoF and _cgoexpwrap_GoF and move the combined operation to the end of the sequence. This combined function also handles reflectcall's previous role. 2. We combined cgocallback and cgocallback_gofunc since the only purpose of having both was to convert a raw PC into a Go function value. We instead construct the Go function value in cgocallbackg1. 3. cgocallbackg1 no longer reaches backwards through the stack to get the arguments to cgocallback_gofunc. Instead, we just pass the arguments down. 4. Currently, we need an explicit msanwrite to mark the results struct as written because reflectcall doesn't do this. Now, the results are written by regular Go assignments, so the Go compiler generates the necessary MSAN annotations. This also means we no longer need to track the size of the arguments frame. Updates #40724, since now we don't need to teach cgo about the register ABI or change how it uses reflectcall. Change-Id: I7840489a2597962aeb670e0c1798a16a7359c94f Reviewed-on: https://go-review.googlesource.com/c/go/+/258938 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-09all: remove the nacl port (part 1)Brad Fitzpatrick
You were a useful port and you've served your purpose. Thanks for all the play. A subsequent CL will remove amd64p32 (including assembly files and toolchain bits) and remaining bits. The amd64p32 removal will be separated into its own CL in case we want to support the Linux x32 ABI in the future and want our old amd64p32 support as a starting point. Updates #30439 Change-Id: Ia3a0c7d49804adc87bf52a4dea7e3d3007f2b1cd Reviewed-on: https://go-review.googlesource.com/c/go/+/199499 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-29runtime: switch default order of hashing algorithmsKeith Randall
Currently the standard hasher is memhash, which checks whether aes instructions are available, and if so redirects to aeshash. With this CL, we call aeshash directly, which then redirects to the fallback hash if aes instructions are not available. This reduces the overhead for the hash function in the common case, as it requires just one call instead of two. On architectures which have no assembly hasher, it's a single jump slower. Thanks to Martin for this idea. name old time/op new time/op delta BigKeyMap-4 22.6ns ± 1% 21.1ns ± 2% -6.55% (p=0.000 n=9+10) Change-Id: Ib7ca77b63d28222eb0189bc3d7130531949d853c Reviewed-on: https://go-review.googlesource.com/c/go/+/190998 Reviewed-by: Martin Möhrmann <moehrmann@google.com>
2019-05-09runtime: fix vet complaints for linux/386Russ Cox
Working toward making the tree vet-safe instead of having so many exceptions in cmd/vet/all/whitelist. This CL makes "GOOS=linux GOARCH=386 go vet -unsafeptr=false runtime" happy, while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too. For #31916. Change-Id: I3e5586a7ff6e359357350d0602c2259493280ded Reviewed-on: https://go-review.googlesource.com/c/go/+/176099 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-04-20runtime: move linux specific code into linux specific filesMaya Rashish
Allows us to stop whitelisting this error on many OS/arch combinations XXX I'm not sure I am running vet correctly, and testing all platforms right. Change-Id: I29f548bd5f4a63bd13c4d0667d4209c75c886fd9 GitHub-Last-Rev: 52f6ff4a6b986e86f8b26c3d19da7707d39f1664 GitHub-Pull-Request: golang/go#31583 Reviewed-on: https://go-review.googlesource.com/c/go/+/173157 Run-TryBot: Benny Siegert <bsiegert@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2019-04-15cmd/link, runtime: mark goexit as the top of the call stackMichael Munday
This CL adds a new attribute, TOPFRAME, which can be used to mark functions that should be treated as being at the top of the call stack. The function `runtime.goexit` has been marked this way on architectures that use a link register. This will stop programs that use DWARF to unwind the call stack from unwinding past `runtime.goexit` on architectures that use a link register. For example, it eliminates "corrupt stack?" warnings when generating a backtrace that hits `runtime.goexit` in GDB on s390x. Similar code should be added for non-link-register architectures (i.e. amd64, 386). They mark the top of the call stack slightly differently to link register architectures so I haven't added that code (they need to mark "rip" as undefined). Fixes #24385. Change-Id: I15b4c69ac75b491daa0acf0d981cb80eb06488de Reviewed-on: https://go-review.googlesource.com/c/go/+/169726 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>