aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/asm_riscv64.s
AgeCommit message (Collapse)Author
2026-03-18runtime: implement part of library initialization in Goqmuntal
All architectures supporting c-shared and c-archive share the same initialization code in assembly, and most of it can be implemented in pure Go. Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64-longtest,gotip-linux-ppc64le_power10,gotip-linux-riscv64,gotip-linux-loong64,gotip-linux-s390x Change-Id: Iaa9fb7d6f9ca8785f1098461646d607ef6b00d47 Reviewed-on: https://go-review.googlesource.com/c/go/+/706417 Auto-Submit: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2026-03-18runtime: make asmcgocall more robust to missing Gqmuntal
Being able to call asmcgocall without a G is useful for code shared between different stages of the runtime initialization and thread creation. Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_15,gotip-linux-mips64le,gotip-linux-ppc64le_power10,gotip-linux-riscv64,gotip-openbsd-ppc64,gotip-openbsd-amd64 Change-Id: Ic427764de197e648e8b9987c98c3b7521512cc5c Reviewed-on: https://go-review.googlesource.com/c/go/+/750541 Reviewed-by: Carlos Amedee <carlos@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2026-02-28runtime/cgo: add abi_riscv64.h for callee-saved register macrosGeorge Adams
Add abi_riscv64.h with SAVE_GPR/SAVE_FPR macros for saving and restoring the RISC-V callee-saved registers, following the pattern established by the other abi_*.h headers. Refactor asm_riscv64.s (cgo crosscall2), asm_riscv64.s (runtime _rt0_riscv64_lib), and race_riscv64.s (racecallbackthunk) to use the new macros, replacing inline register save/restore sequences. Change-Id: I83cef17a35c57fc8b5e045e7181460283ae5b423 Reviewed-on: https://go-review.googlesource.com/c/go/+/749900 Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Russ Cox <rsc@golang.org>
2026-01-27cmd/compile, runtime: avoid improper control transfer instruction hints on ↵wangboyao
riscv64 On RISC-V the JAL and JALR instructions provide Return Address Stack(RAS) prediction hints based on the registers used (as per section 2.5.1 of the RISC-V ISA manual). When a JALR instruction uses X1 or X5 as the source register, it hints that a pop should occur. When making a function call, avoid the use of X5 as a source register since this results in the RAS performing a pop-then-push instead of a push, breaking call/return pairing and significantly degrading front-end branch prediction performance. Based on test result of golang.org/x/benchmarks/json on SpacemiT K1, fix version has a performance improvement of about 7% Fixes #76654 Change-Id: I867c8d7cfb54f5decbe176f3ab3bb3d78af1cf64 Reviewed-on: https://go-review.googlesource.com/c/go/+/726760 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au>
2025-11-17cmd/internal/sys,internal/goarch,runtime: enable the use of compressed ↵Joel Sing
instructions on riscv64 Enable the use of compressed instructions on riscv64 by reducing the PC quantum to two bytes and reducing the minimum instruction length to two bytes. Change gostartcall on riscv64 to land at two times the PC quantum into goexit, so that we retain four byte alignment and revise the NOP instructions in goexit to ensure that they are never compressed. Additionally, adjust PCALIGN so that it correctly handles two byte offsets. Fixes #47560 Updates #71105 Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64 Change-Id: I4329a8fbfcb4de636aadaeadabb826bc22698640 Reviewed-on: https://go-review.googlesource.com/c/go/+/523477 Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-by: Mark Freeman <markfreeman@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
2025-10-20runtime: wrap procyield assembly and check for 0Michael Anthony Knyszek
procyield will currently loop infinitely if passed 0 on several platforms. This change sidesteps this bug by renaming procyield to procyieldAsm, and adding a wrapper named procyield that checks for cycles == 0. The benefit of this structure is that procyield called with a constant cycle count of 0 will be inlined and constant folded away, the expected behavior of a procyield of 0 cycles. A follow-up change will fix the assembly to not have this footgun anymore. Change-Id: I7068abfeb961bc0fa475e216836f7c0e46b38373 Reviewed-on: https://go-review.googlesource.com/c/go/+/712663 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2025-10-17runtime: unify riscv64 library entry pointqmuntal
Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64 Change-Id: I6470dfc5c9e03dfe5fc535605fdd7d861b9ba2f3 Reviewed-on: https://go-review.googlesource.com/c/go/+/706415 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-08-04cmd/compile: move riscv64 over to new bounds check strategyKeith Randall
Change-Id: Idd9eaf051aa57f7fef7049c12085926030c35d70 Reviewed-on: https://go-review.googlesource.com/c/go/+/682401 Reviewed-by: Mark Freeman <mark@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-26runtime: remove ret field from gobufKeith Randall
It's not used for anything. Change-Id: I031b3cdfe52b6b1cff4b3cb6713ffe588084542f Reviewed-on: https://go-review.googlesource.com/c/go/+/652276 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-25cmd/compile, runtime: use PC of deferreturn for panic transferDavid Chase
this removes the old conditional-on-register-value handshake from the deferproc/deferprocstack logic. The "line" for the recovery-exit frame itself (not the defers that it runs) is the closing brace of the function. Reduces code size slightly (e.g. go command is 0.2% smaller) Sample output showing effect of this change, also what sort of code it requires to observe the effect: ``` package main import "os" func main() { g(len(os.Args) - 1) // stack[0] } var gi int var pi *int = &gi //go:noinline func g(i int) { switch i { case 0: defer func() { println("g0", i) q() // stack[2] if i == 0 }() for j := *pi; j < 1; j++ { defer func() { println("recover0", recover().(string)) }() } default: for j := *pi; j < 1; j++ { defer func() { println("g1", i) q() // stack[2] if i == 1 }() } defer func() { println("recover1", recover().(string)) }() } p() } // stack[1] (deferreturn) //go:noinline func p() { panic("p()") } //go:noinline func q() { panic("q()") // stack[3] } /* Sample output for "./foo foo": recover1 p() g1 1 panic: q() goroutine 1 [running]: main.q() .../main.go:46 +0x2c main.g.func3() .../main.go:29 +0x48 main.g(0x1?) .../main.go:37 +0x68 main.main() .../main.go:6 +0x28 */ ``` Change-Id: Ie39ea62ecc244213500380ea06d44024cadc2317 Reviewed-on: https://go-review.googlesource.com/c/go/+/650795 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-19all: implement plugin build mode for riscv64Meng Zhuo
Change-Id: I8d7bbeebbf4a46f2fd8d630b1edbaf79b8ffccc5 Reviewed-on: https://go-review.googlesource.com/c/go/+/420114 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> TryBot-Bypass: Joel Sing <joel@sing.id.au>
2024-09-05runtime: use regabi for riscv64 cputicksMeng Zhuo
goos: linux goarch: riscv64 pkg: runtime cpu: Spacemit(R) X60 │ select.old.log │ select.new.log │ │ sec/op │ sec/op vs base │ SelectUncontended 490.5n ± 0% 486.8n ± 0% -0.77% (p=0.000 n=10) SelectSyncContended 2.754µ ± 0% 2.726µ ± 0% -1.02% (p=0.000 n=10) SelectAsyncContended 488.2n ± 0% 484.2n ± 0% -0.84% (p=0.000 n=10) SelectNonblock 112.2n ± 0% 111.5n ± 0% -0.58% (p=0.000 n=10) SelectProdCons 1.420µ ± 0% 1.417µ ± 0% ~ (p=0.069 n=10) GoroutineSelect 10.79m ± 3% 10.74m ± 3% ~ (p=0.529 n=10) geomean 3.228µ 3.208µ -0.63% Change-Id: Idb519ef8b2872284dca6dbf1cf94c3fff65bfd37 Reviewed-on: https://go-review.googlesource.com/c/go/+/610095 Reviewed-by: abner chenc <chenguoqi@loongson.cn> Reviewed-by: Joel Sing <joel@sing.id.au> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-09all: clean up addition of constants in riscv64 assemblyJoel Sing
Use ADD with constants, instead of ADDI. Also use SUB with a positive constant rather than ADD with a negative constant. The resulting assembly is still the same. Change-Id: Ife10bf5ae4122e525f0e7d41b5e463e748236a9c Reviewed-on: https://go-review.googlesource.com/c/go/+/540136 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: M Zhuo <mzh@golangcn.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Mark Ryan <markdryan@rivosinc.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Joel Sing <joel@sing.id.au>
2023-11-03runtime: remove getcallerpc on riscv64Joel Sing
This was converted to a compiler intrinsic and no longer needs to exist in assembly. Change-Id: I7495c435d4642e0e71d8f7677d70af3a3ca2a6ba Reviewed-on: https://go-review.googlesource.com/c/go/+/539195 Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com>
2023-11-02runtime: add crash stack support for riscv64Joel Sing
Change-Id: Ib89a71e20f9c6b86c97814c75cb427e9bd7075e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/538735 Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Joel Sing <joel@sing.id.au>
2023-10-04runtime: add runtime support for openbsd/riscv64 portJoel Sing
Updates #55999 Change-Id: I0e80f80d49696a00d979f85230d482e24d4c2d7c Reviewed-on: https://go-review.googlesource.com/c/go/+/518626 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Aaron Bieber <deftly@gmail.com>
2023-06-14all: fix spelling errorsAlexander Yastrebov
Fix spelling errors discovered using https://github.com/codespell-project/codespell. Errors in data files and vendored packages are ignored. Change-Id: I83c7818222f2eea69afbd270c15b7897678131dc GitHub-Last-Rev: 3491615b1b82832cc0064f535786546e89aa6184 GitHub-Pull-Request: golang/go#60758 Reviewed-on: https://go-review.googlesource.com/c/go/+/502576 Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-06-10runtime: fix typosJes Cok
Change-Id: If13f4d4bc545f78e3eb8c23cf2e63f0eb273d71f GitHub-Last-Rev: 32ca70f52a5c3dd66f18535c5e595e66afb3903c GitHub-Pull-Request: golang/go#60703 Reviewed-on: https://go-review.googlesource.com/c/go/+/502055 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-17runtime/cgo: store M for C-created thread in pthread keyCherry Mui
This reapplies CL 485500, with a fix drafted in CL 492987 incorporated. CL 485500 is reverted due to #60004 and #60007. #60004 is fixed in CL 492743. #60007 is fixed in CL 492987 (incorporated in this CL). [Original CL 485500 description] This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 492987 description] On the first call into Go from a C thread, currently we set the g0 stack's high bound imprecisely based on the SP. With CL 485500, we keep the M and don't recompute the stack bounds when it calls into Go again. If the first call is made when the C thread uses some deep stack, but a subsequent call is made with a shallower stack, the SP may be above g0.stack.hi. This is usually okay as we don't check usually stack.hi. One place where we do check for stack.hi is in the signal handler, in adjustSignalStack. In particular, C TSAN delivers signals on the g0 stack (instead of the usual signal stack). If the SP is above g0.stack.hi, we don't see it is on the g0 stack, and throws. This CL makes it get an accurate stack upper bound with the pthread API (on the platforms where it is available). Also add some debug print for the "handler not on signal stack" throw. Fixes #51676. Fixes #59294. Fixes #59678. Fixes #60007. Change-Id: Ie51c8e81ade34ec81d69fd7bce1fe0039a470776 Reviewed-on: https://go-review.googlesource.com/c/go/+/495855 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-05Revert "runtime/cgo: store M for C-created thread in pthread key"Chressie Himpel
This reverts CL 485500. Reason for revert: This breaks internal tests at Google, see b/280861579 and b/280820455. Change-Id: I426278d400f7611170918fc07c524cb059b9cc55 Reviewed-on: https://go-review.googlesource.com/c/go/+/492995 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Chressie Himpel <chressie@google.com>
2023-04-26runtime/cgo: store M for C-created thread in pthread keyMichael Pratt
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and CL 485316 incorporated. CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 482975 is a followup fix to a C declaration in testprogcgo. CL 485315 is a followup fix for x_cgo_getstackbound on Illumos. CL 485316 is a followup cleanup for ppc64 assembly. [Original CL 481061 description] This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. [CL 485500 description] CL 479915 passed the G to _cgo_getstackbound for direct updates to gp.stack.lo. A G can be reused on a new thread after the previous thread exited. This could trigger the C TSAN race detector because it couldn't see the synchronization in Go (lockextra) preventing the same G from being used on multiple threads at the same time. We work around this by passing the address of a stack variable to _cgo_getstackbound rather than the G. The stack is generally unique per thread, so TSAN won't see the same address from multiple threads. Even if stacks are reused across threads by pthread, C TSAN should see the synchonization in the stack allocator. A regression test is added to misc/cgo/testsanitizer. Fixes #51676. Fixes #59294. Fixes #59678. Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca Reviewed-on: https://go-review.googlesource.com/c/go/+/485500 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-21runtime: tidy _Stack* constant namingAustin Clements
For #59670. Change-Id: I0efa743edc08e48dc8d906803ba45e9f641369db Reviewed-on: https://go-review.googlesource.com/c/go/+/486977 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com>
2023-04-20Revert "runtime: tidy _Stack* constant naming"Austin Clements
This reverts commit CL 486381. Submitted out of order and breaks bootstrap. Change-Id: Ia472111cb966e884a48f8ee3893b3bf4b4f4f875 Reviewed-on: https://go-review.googlesource.com/c/go/+/486915 Reviewed-by: David Chase <drchase@google.com> TryBot-Bypass: Austin Clements <austin@google.com>
2023-04-20runtime: tidy _Stack* constant namingAustin Clements
For #59670. Change-Id: I4476d6f92663e8a825d063d6e6a7fc9a2ac99d4d Reviewed-on: https://go-review.googlesource.com/c/go/+/486381 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-04-17Revert "runtime/cgo: store M for C-created thread in pthread key"Michael Pratt
This reverts CL 481061. Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers race detection on `g->stacklo` because the synchronization is in Go, which isn't instrumented. For #51676. For #59294. For #59678. Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47 Reviewed-on: https://go-review.googlesource.com/c/go/+/485275 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2023-04-03runtime/cgo: store M for C-created thread in pthread keydoujiang24
This reapplies CL 392854, with the followup fixes in CL 479255, CL 479915, and CL 481057 incorporated. CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go calls by binding the M to the C thread. See below for its description. CL 479255 is a followup fix for a small bug in ARM assembly code. CL 479915 is another followup fix to address C to Go calls after the C code uses some stack, but that CL is also buggy. CL 481057, by Michael Knyszek, is a followup fix for a memory leak bug of CL 479915. [Original CL 392854 description] In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [CL 479915 description] Currently, when C calls into Go the first time, we grab an M using needm, which sets m.g0's stack bounds using the SP. We don't know how big the stack is, so we simply assume 32K. Previously, when the Go function returns to C, we drop the M, and the next time C calls into Go, we put a new stack bound on the g0 based on the current SP. After CL 392854, we don't drop the M, and the next time C calls into Go, we reuse the same g0, without recomputing the stack bounds. If the C code uses quite a bit of stack space before calling into Go, the SP may be well below the 32K stack bound we assumed, so the runtime thinks the g0 stack overflows. This CL makes needm get a more accurate stack bound from pthread. (In some platforms this may still be a guess as we don't know exactly where we are in the C stack), but it is probably better than simply assuming 32K. Fixes #51676. Fixes #59294. Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494 Reviewed-on: https://go-review.googlesource.com/c/go/+/481061 Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31Revert "runtime/cgo: store M for C-created thread in pthread key"Cherry Mui
This reverts CL 392854. Reason for revert: caused #59294, which was derived from google internal tests. The attempted fix of #59294 caused more breakage. Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba Reviewed-on: https://go-review.googlesource.com/c/go/+/481060 Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-24runtime/cgo: store M for C-created thread in pthread keydoujiang24
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls. So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call. Instead, we only dropm while the C thread exits, so the extra M won't leak. When invoking a Go function from C: Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor. And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits. When returning back to C: Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C. This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows. This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread. For the newly added BenchmarkCGoInCThread, some benchmark results: 1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz Fixes #51676 Change-Id: I380702fe2f9b6b401b2d6f04b0aba990f4b9ee6c GitHub-Last-Rev: 93dc64ad98e5583372e41f65ee4b7ab78b5aff51 GitHub-Pull-Request: golang/go#51679 Reviewed-on: https://go-review.googlesource.com/c/go/+/392854 Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: thepudds <thepudds1460@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-24cmd/compile: batch write barrier callsKeith Randall
Have the write barrier call return a pointer to a buffer into which the generated code records pointers that need write barrier treatment. Change-Id: I7871764298e0aa1513de417010c8d46b296b199e Reviewed-on: https://go-review.googlesource.com/c/go/+/447781 Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Bypass: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-17cmd/compile: move raw writes out of write barrier codeKeith Randall
Previously, the write barrier calls themselves did the actual writes to memory. Instead, move those writes out to a common location that both the wb-enabled and wb-disabled code paths share. This enables us to optimize the write barrier path without having to worry about performing the actual writes. Change-Id: Ia71ab651908ec124cc33141afb52e4ca19733ac6 Reviewed-on: https://go-review.googlesource.com/c/go/+/447780 Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Bypass: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-17runtime: remove the restriction that write barrier ptrs come in pairsKeith Randall
Future CLs will remove the invariant that pointers are always put in the write barrier in pairs. The behavior of the assembly code changes a bit, where instead of writing the pointers unconditionally and then checking for overflow, check for overflow first and then write the pointers. Also changed the write barrier flush function to not take the src/dst as arguments. Change-Id: I2ef708038367b7b82ea67cbaf505a1d5904c775c Reviewed-on: https://go-review.googlesource.com/c/go/+/447779 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Bypass: Keith Randall <khr@golang.org>
2022-10-26all: delete riscv64 non-register ABI fallback pathWayne Zuo
Change-Id: I9e997b59ffb868575b780b9660df1f5ac322b79a Reviewed-on: https://go-review.googlesource.com/c/go/+/443556 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org> Reviewed-by: David Chase <drchase@google.com>
2022-08-25runtime: mark morestack_noctxt SPWRITE on LR architecturesCherry Mui
On LR architectures, morestack (and morestack_noctxt) are called with a special calling convention, where the caller doesn't save LR on stack but passes it as a register, which morestack will save to g.sched.lr. The stack unwinder currently doesn't understand it, and would fail to unwind from it. morestack already writes SP (as it switches stack), but morestack_noctxt (which tailcalls morestack) doesn't. If a profiling signal lands right in morestack_noctxt, the unwinder will try to unwind the stack and go off, and possibly crash. Marking morestack_noctxt SPWRITE stops the unwinding. Ideally we could teach the unwinder about the special calling convention, or change the calling convention to be less special (so the unwinder doesn't need to fetch a register from the signal context). This is a stop-gap solution, to stop the unwinder from crashing. Fixes #54332. Change-Id: I75295f2e27ddcf05f1ea0b541aedcb9000ae7576 Reviewed-on: https://go-review.googlesource.com/c/go/+/425396 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-04-24runtime: add ABIInternal to strhash and memhash on riscv64Meng Zhuo
This CL fixes regression of strhash and memhash on riscv64 Change-Id: Icc10431a8199c8b1eb7b440cb42be4e53420e171 Reviewed-on: https://go-review.googlesource.com/c/go/+/362134 Run-TryBot: mzh <mzh@golangcn.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Russ Cox <rsc@golang.org>
2022-03-30runtime: add runtime changes for register ABI on riscv64Meng Zhuo
This CL adds - spill functions used by runtime - ABIInternal to functions Adding new stubs_riscv64 file to eliminate vet issues while compiling. Change-Id: I2a9f6088a1cd2d9708f26b2d97895b4e5f9f87e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/360296 Trust: mzh <mzh@golangcn.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-12-29runtime: invalid negative frequency while tracingMeng Zhuo
The riscv64 Hifive Unmatched is the only platform that failed on testcase TestAnalyzeAnnotations occasionally after CL 332954 had merged. The failure happens when ticks per second (freq) is over 1e12 which causing the timestamps of two events are same. There are 2 reasons causing big frequency: 1. RDCYCLE is HART based according to the riscv manual which makes negative ticks delta 2. negative float64 -> uint64 is undefined and "lucky" negative float is too big to handle for trace For #46737 Change-Id: I1f3c1ac31aae249969000c719c32aaf5a66d29a5 Reviewed-on: https://go-review.googlesource.com/c/go/+/373034 Trust: Zhuo Meng <mzh@golangcn.org> Run-TryBot: Zhuo Meng <mzh@golangcn.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-11-03cmd/asm,cmd/compile,runtime: stop using X3 (aka GP) on riscv64Joel Sing
The X3 (aka GP) register will potentially be loaded with the __global_pointer$ symbol during program start up (usually by the dynamic linker). As such, non-Go code may depend on the contents of GP and calculate offsets based on it, including code called via cgo and signal handlers installed by non-Go code. As such, stop using the X3 register so that there are fewer issues interacting between Go and non-Go code. While here remove the X4 (TP) name from the assembler such that any references must use the 'TP' name. This should reduce the likelihood of accidental use (like we do for the 'g' register). The same applies for X3 (GP) when the -shared flag is given. Updates #47100 Change-Id: I72e82b5ca3f80c46a781781345ca0432a4111b74 Reviewed-on: https://go-review.googlesource.com/c/go/+/351859 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-08-18runtime: use RDCYCLE for cputicks on riscv64Meng Zhuo
Use RDCYCLE instruction instead of RDTIME emulation Change-Id: Id7b3de42a36d2d1b163c39cc79870eee7c840ad5 Reviewed-on: https://go-review.googlesource.com/c/go/+/332954 Trust: Meng Zhuo <mzh@golangcn.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-08-16runtime: make asmcgocall g0/gsignal checks consistentJoel Sing
In asmcgocall() we need to switch to the g0 stack if we're not already on the g0 stack or the gsignal stack. The prefered way of doing this is to check gsignal first, then g0, since if we are going to switch to g0 we will need g0 handy (thus avoiding a second load). Rewrite/reorder 386 and amd64 to check gsignal first - this shaves a few assembly instructions off and makes the order consistent with arm, arm64, mips64 and ppc64. Add missing gsignal checks to mips, riscv64 and s390x. Change-Id: I1b027bf393c25e0c33e1d8eb80de67e4a0a3f561 Reviewed-on: https://go-review.googlesource.com/c/go/+/335869 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-08-03[dev.typeparams] runtime,cmd/compile,cmd/link: replace jmpdefer with a loopAustin Clements
Currently, deferreturn runs deferred functions by backing up its return PC to the deferreturn call, and then effectively tail-calling the deferred function (via jmpdefer). The effect of this is that the deferred function appears to be called directly from the deferee, and when it returns, the deferee calls deferreturn again so it can run the next deferred function if necessary. This unusual flow control leads to a large number of special cases and complications all over the tool chain. This used to be necessary because deferreturn copied the deferred function's argument frame directly into its caller's frame and then had to invoke that call as if it had been called from its caller's frame so it could access it arguments. But now that we've simplified defer processing so the runtime only deals with argument-less closures, this approach is no longer necessary. This CL simplifies all of this by making deferreturn simply call deferred functions in a loop. This eliminates the need for jmpdefer, so we can delete a bunch of per-architecture assembly code. This eliminates several special cases on Wasm, since it couldn't support these calling shenanigans directly and thus had to simulate the loop a different way. Now Wasm can largely work the way the other platforms do. This eliminates the per-architecture Ginsnopdefer operation. On PPC64, this was necessary to reload the TOC pointer after the tail call (since TOC pointers in general make tail calls impossible). The tail call is gone, and in the case where we do force a jump to the deferreturn call when recovering from an open-coded defer, we go through gogo (via runtime.recovery), which handles the TOC. On other platforms, we needed a NOP so traceback didn't get confused by seeing the return to the CALL instruction, rather than the usual return to the instruction following the CALL instruction. Now we don't inject a return to the CALL instruction at all, so this NOP is also unnecessary. The one potential effect of this is that deferreturn could now appear in stack traces from deferred functions. However, this could already happen from open-coded defers, so we've long since marked deferreturn as a "wrapper" so it gets elided not only from printed stack traces, but from runtime.Callers*. This is a retry of CL 337652 because we had to back out its parent. There are no changes in this version. Change-Id: I3f54b7fec1d7ccac71cc6cf6835c6a46b7e5fb6c Reviewed-on: https://go-review.googlesource.com/c/go/+/339397 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-07-30[dev.typeparams] Revert "[dev.typeparams] runtime,cmd/compile,cmd/link: ↵Austin Clements
replace jmpdefer with a loop" This reverts CL 227652. I'm reverting CL 337651 and this builds on top of it. Change-Id: I03ce363be44c2a3defff2e43e7b1aad83386820d Reviewed-on: https://go-review.googlesource.com/c/go/+/338709 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-07-30[dev.typeparams] runtime,cmd/compile,cmd/link: replace jmpdefer with a loopAustin Clements
Currently, deferreturn runs deferred functions by backing up its return PC to the deferreturn call, and then effectively tail-calling the deferred function (via jmpdefer). The effect of this is that the deferred function appears to be called directly from the deferee, and when it returns, the deferee calls deferreturn again so it can run the next deferred function if necessary. This unusual flow control leads to a large number of special cases and complications all over the tool chain. This used to be necessary because deferreturn copied the deferred function's argument frame directly into its caller's frame and then had to invoke that call as if it had been called from its caller's frame so it could access it arguments. But now that we've simplified defer processing so the runtime only deals with argument-less closures, this approach is no longer necessary. This CL simplifies all of this by making deferreturn simply call deferred functions in a loop. This eliminates the need for jmpdefer, so we can delete a bunch of per-architecture assembly code. This eliminates several special cases on Wasm, since it couldn't support these calling shenanigans directly and thus had to simulate the loop a different way. Now Wasm can largely work the way the other platforms do. This eliminates the per-architecture Ginsnopdefer operation. On PPC64, this was necessary to reload the TOC pointer after the tail call (since TOC pointers in general make tail calls impossible). The tail call is gone, and in the case where we do force a jump to the deferreturn call when recovering from an open-coded defer, we go through gogo (via runtime.recovery), which handles the TOC. On other platforms, we needed a NOP so traceback didn't get confused by seeing the return to the CALL instruction, rather than the usual return to the instruction following the CALL instruction. Now we don't inject a return to the CALL instruction at all, so this NOP is also unnecessary. The one potential effect of this is that deferreturn could now appear in stack traces from deferred functions. However, this could already happen from open-coded defers, so we've long since marked deferreturn as a "wrapper" so it gets elided not only from printed stack traces, but from runtime.Callers*. Change-Id: Ie9f700cd3fb774f498c9edce363772a868407bf7 Reviewed-on: https://go-review.googlesource.com/c/go/+/337652 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-07-30[dev.typeparams] runtime: use func() for deferred functionsAustin Clements
Prior to regabi, a deferred function could have any signature, so the runtime always manipulated them as funcvals. Now, a deferred function is always func(). Hence, this CL makes the runtime's manipulation of deferred functions more type-safe by using func() directly instead of *funcval. Change-Id: Ib55f38ed49107f74149725c65044e4690761971d Reviewed-on: https://go-review.googlesource.com/c/go/+/337650 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-06-08[dev.typeparams] cmd/compile, runtime: remove the siz argument of ↵Cherry Mui
newproc/deferproc newproc/deferproc takes a siz argument for the go'd/deferred function's argument size. Now it is always zero. Remove the argument. Change-Id: If1bb8d427e34015ccec0ba10dbccaae96757fa8c Reviewed-on: https://go-review.googlesource.com/c/go/+/325917 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-21cmd/compile: allow conversion from slice to array ptrJosh Bleecher Snyder
Panic if the slice is too short. Updates #395 Change-Id: I90f4bff2da5d8f3148ba06d2482084f32b25c29a Reviewed-on: https://go-review.googlesource.com/c/go/+/301650 Trust: Josh Bleecher Snyder <josharian@gmail.com> Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-02-19runtime: fix spurious stack overflow detectionRuss Cox
The regabi builders are unhappy about badctxt calling throw calling systemstack calling gosave_systemstack_switch calling badctxt, all nosplit, repeating. This wouldn't actually happen since after one systemstack we'd end up on the system stack and the next one wouldn't call gosave_systemstack_switch at all. The badctxt call itself is in a very unlikely assertion failure inside gosave_systemstack_switch. Keep the assertion check but call runtime.abort instead on failure, breaking the detected (but not real) cycle. Change-Id: Iaf5c0fc065783b8c1c6d0f62d848f023a0714b96 Reviewed-on: https://go-review.googlesource.com/c/go/+/294069 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: use TOPFRAME to identify top-of-frame functionsRuss Cox
No change to actual runtime, but helps reduce the laundry list of functions. mcall, morestack, and asmcgocall are not actually top-of-frame, so those need more attention in follow-up CLs. mstart moved to assembly so that it can be marked TOPFRAME. Since TOPFRAME also tells DWARF consumers not to unwind this way, this change should also improve debuggers a marginal amount. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: If1e0d46ca973de5e46b62948d076f675f285b5d9 Reviewed-on: https://go-review.googlesource.com/c/go/+/288802 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: remove unnecessary writes to gp.sched.gRuss Cox
A g's sched.g is set in newproc1: newg.sched.g = guintptr(unsafe.Pointer(newg)) After that, it never changes. Yet lots of assembly code does "g.sched.g = g" unnecessarily. Remove all those lines to avoid confusion about whether it ever changes. Also, split gogo into two functions, one that does the nil g check and a second that does the actual switch. This way, if the nil g check fails, we get a stack trace showing the call stack that led to the failure. (The SP write would otherwise cause the stack trace to abort.) Also restore the proper nil g check in a handful of assembly functions. (There is little point in checking for nil g *after* installing it as the real g.) Change-Id: I22866b093f901f765de1d074e36eeec10366abfb Reviewed-on: https://go-review.googlesource.com/c/go/+/292109 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: unify asmcgocall and systemstack traceback setupRuss Cox
Both asmcgocall and systemstack need to save the calling Go code's context for use by traceback, but they do it differently. Systemstack's appraoch is better, because it doesn't require a special case in traceback. So make them both use that. While we are here, the fake mstart caller in systemstack is no longer needed and can be removed. (traceback knows to stop in systemstack because of the writes to SP.) Also remove the fake mstarts in sys_windows_*.s. And while we are there, fix the control flow guard code in sys_windows_arm.s. The current code is using pointers to a stack frame that technically is gone once we hit the RET instruction. Clearly it's working OK, but better not to depend on data below SP being preserved, even for just a few instructions. Store the value we need in other registers instead. (This code is only used for pushing a sigpanic call, which does not actually return to the site of the fault and therefore doesn't need to preserve any of the registers.) This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: Id1e3ef5e54f7ad786e4b87043f2626eba7c3bbd9 Reviewed-on: https://go-review.googlesource.com/c/go/+/288799 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-18runtime: unbreak linux/riscv64 following regabi mergeJoel Sing
Unbreak the linux/riscv64 port by storing the zero value register to memory, rather than the current code that is moving a zero intermediate to the stack pointer register (ideally this should be caught by the assembler). This was broken in CL#272568. On riscv64 a zero immediate value cannot be moved directly to memory, rather a register needs to be loaded with zero and then stored. Alternatively, the the zero value register (aka X0) can be used directly. Change-Id: Id57121541d50c9993cec5c2270b638b184ab9bc1 Reviewed-on: https://go-review.googlesource.com/c/go/+/292894 Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org>