aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/sys_linux_amd64.s
AgeCommit message (Collapse)Author
2022-10-18runtime: always keep global reference to mp until mexit completesMichael Pratt
Ms are allocated via standard heap allocation (`new(m)`), which means we must keep them alive (i.e., reachable by the GC) until we are completely done using them. Ms are primarily reachable through runtime.allm. However, runtime.mexit drops the M from allm fairly early, long before it is done using the M structure. If that was the last reference to the M, it is now at risk of being freed by the GC and used for some other allocation, leading to memory corruption. Ms with a Go-allocated stack coincidentally already keep a reference to the M in sched.freem, so that the stack can be freed lazily. This reference has the side effect of keeping this Ms reachable. However, Ms with an OS stack skip this and are at risk of corruption. Fix this lifetime by extending sched.freem use to all Ms, with the value of mp.freeWait determining whether the stack needs to be freed or not. Fixes #56243. Change-Id: Ic0c01684775f5646970df507111c9abaac0ba52e Reviewed-on: https://go-review.googlesource.com/c/go/+/443716 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-10-07runtime: move epoll syscalls to runtime/internal/syscallAndrew Pogrebnoy
This change moves Linux epoll's syscalls implementation to the "runtime/internal/syscall" package. The intention in this CL was to minimise behavioural changes but make the code more generalised. This also will allow adding new syscalls (like epoll_pwait2) without the need to implement assembly stubs for each arch. It also drops epoll_create as not all architectures provide this call. epoll_create1 was added to the kernel in version 2.6.27 and Go requires Linux kernel version 2.6.32 or later since Go 1.18. So it is safe to always use epoll_create1. This is a resubmit as the previous CL 421994 was reverted due to test failures after the merge with the master. The issue was fixed in CL 438615 For #53824 For #51087 Change-Id: I1bd0f23a85b4f9b80178c5dd36fd3e95ff4f9648 Reviewed-on: https://go-review.googlesource.com/c/go/+/440115 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2022-09-30Revert "runtime: move epoll syscalls to runtime/internal/syscall"Michael Pratt
This reverts CL 421994. Reason for revert: breaks runtime.TestCheckPtr2 For #53824 For #51087 Change-Id: I044ea4d6efdffe0a4b7fb0d2bb3717d9f391fc59 Reviewed-on: https://go-review.googlesource.com/c/go/+/437295 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-09-30runtime: move epoll syscalls to runtime/internal/syscallAndrew Pogrebnoy
This change moves Linux epoll's syscalls implementation to the "runtime/internal/syscall" package. The intention in this CL was to minimise behavioural changes but make the code more generalised. This also will allow adding new syscalls (like epoll_pwait2) without the need to implement assembly stubs for each arch. It also drops epoll_create as not all architectures provide this call. epoll_create1 was added to the kernel in version 2.6.27 and Go requires Linux kernel version 2.6.32 or later since Go 1.18. So it is safe to always use epoll_create1. For #53824 For #51087 Change-Id: I9a6a26b7f2075a38e041de1bab4691da0ecb94fc Reviewed-on: https://go-review.googlesource.com/c/go/+/421994 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com>
2022-04-28runtime: mark sigtramp as TOPFRAMEMichael Pratt
Currently throw() in the signal handler results in "fatal error: unknown return pc from runtime.sigreturn ...". Marking sigtramp as TOPFRAME allows gentraceback to stop tracebacks at the end of a signal handler, since there is not much beyond sigtramp. This is just done on Linux for now, but may apply to other Unix systems as well. Change-Id: I96edcb945283f417a5bfe00ce2fb2b1a0d578692 Reviewed-on: https://go-review.googlesource.com/c/go/+/402190 Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-04-26runtime: use ABIInternal for most calls to sigtrampgoMichael Pratt
sigtramp on openbsd-arm64 is teetering on the edge of the nosplit stack limit. Add more headroom by calling sigtrampgo using ABIInternal, which eliminates a 48-byte ABI wrapper frame. openbsd-amd64 has slightly more space, but is also close to the limit, so convert it as well. Other operating systems don't have it as bad, but many have nearly identical implementations of sigtramp, so I have converted them as well. I've omitted darwin-arm64 and solaris, as those are quite different and would benefit from not needing ifdef for both cases. For #51485. Change-Id: I70512645d4208b346a59d5e5d03836a45833b1d7 Reviewed-on: https://go-review.googlesource.com/c/go/+/390814 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-03runtime: remove fallback to pipe on platforms with pipe2Tobias Klauser
On Linux, the minimum required kernel version for Go 1.18 was be changed to 2.6.32, see #45964. The pipe2 syscall was added in 2.6.27. All other platforms already provide the pipe2 syscall in the minimum supported version: - DragonFly BSD added it in version 4.2, see https://www.dragonflybsd.org/release42/ - FreeBSD added it in version 10.0, see https://www.freebsd.org/cgi/man.cgi?pipe(2)#end - NetBSD added it in version 6.0, see https://man.netbsd.org/pipe2.2#HISTORY - OpenBSD added it in version 5.7, see https://man.openbsd.org/pipe.2#HISTORY - Illumos supports it since 2013, see https://www.illumos.org/issues/3714 - Solaris supports it since 11.4 This also allows to remove setNonblock which was only used in the pipe fallback path on these platforms. Change-Id: I1f40d32fd3065d74e22af77b9ff2292b9cf66706 Reviewed-on: https://go-review.googlesource.com/c/go/+/389354 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-09-27runtime: use per-thread profiler for SetCgoTraceback platformsRhys Hiltner
Updates #35057 Change-Id: I61d772a2cbfb27540fb70c14676c68593076ca94 Reviewed-on: https://go-review.googlesource.com/c/go/+/342054 Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Knyszek <mknyszek@google.com>
2021-09-27runtime: add timer_create syscalls for LinuxRhys Hiltner
Updates #35057 Change-Id: Id702b502fa4e4005ba1e450a945bc4420a8a8b8c Reviewed-on: https://go-review.googlesource.com/c/go/+/342052 Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Than McIntosh <thanm@google.com>
2021-06-11[dev.typeparams] all: always enable regabig on AMD64Cherry Mui
Always enable regabig on AMD64, which enables the G register and the X15 zero register. Remove the fallback path. Also remove the regabig GOEXPERIMENT. On AMD64 it is always enabled (this CL). Other architectures already have a G register, except for 386, where there are too few registers and it is unlikely that we will reserve one. (If we really do, we can just add a new experiment). Change-Id: I229cac0060f48fe58c9fdaabd38d6fa16b8a0855 Reviewed-on: https://go-review.googlesource.com/c/go/+/327272 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-05-21[dev.typeparams] runtime: use internal/abi.FuncPCABI0 to take address of ↵Cherry Mui
assembly functions There are a few assembly functions in the runtime that are marked as ABIInternal, solely because funcPC can get the right address. The functions themselves do not actually follow ABIInternal (or irrelevant). Now we have internal/abi.FuncPCABI0, use that, and un-mark the functions. Also un-mark assembly functions that are only called in assembly. For them, it only matters if the caller and callee are consistent. Change-Id: I240e126ac13cb362f61ff8482057ee9f53c24097 Reviewed-on: https://go-review.googlesource.com/c/go/+/321950 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29runtime: remove linux-amd64 walltime functionIan Lance Taylor
It's never called. Change-Id: I8956743b21301816b5f37a9b34e3f50ef7b2e70a Reviewed-on: https://go-review.googlesource.com/c/go/+/314771 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29runtime: rename walltime1 to walltimeIan Lance Taylor
Change-Id: Iec9de5ca56eb68d524bbaa0668515dbd09ad38a1 Reviewed-on: https://go-review.googlesource.com/c/go/+/314770 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-15runtime: unify C->Go ABI transitionsAustin Clements
The previous CL introduced macros for transitions from the Windows ABI to the Go ABI. This CL does the same for SysV and uses them in almost all places where we transition from the C ABI to the Go ABI. Compared to Windows, this transition is much simpler and I didn't find any places that were getting it wrong. But this does let us unify a lot of code nicely and introduces some degree of abstraction around these ABI transitions. Change-Id: Ib6bdecafce587ce18fca4c8300fcf401284a2bcd Reviewed-on: https://go-review.googlesource.com/c/go/+/309930 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-08cmd/asm,runtime: reduce spellings of GOEXPERIMENTsAustin Clements
Currently, the objabi.Experiment fields use Go-standard CamelCase, the GOEXPERIMENT environment variable flags and build tags use all lowercase, and the asm macros use upper-case with underscores. This CL makes asm use the lowercase names for macros so there is one less spelling, e.g., GOEXPERIMENT_regabiargs. This also makes them consistent with the GOOS_* and GOARCH_* macros, which also use lower case. Change-Id: I305cd89af5e8cd1a89cc148746c034bcfd76db3c Reviewed-on: https://go-review.googlesource.com/c/go/+/307816 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-04-01runtime: fix uses of ABIInternal PCs in assemblyAustin Clements
The covers three kinds of uses: 1. Calls of closures from assembly. These are always ABIInternal calls without wrappers. I went through every indirect call in the runtime and I think mcall is the only case of assembly calling a Go closure in a way that's affected by ABIInternal. systemstack also calls a closure, but it takes no arguments. 2. Calls of Go functions that expect raw ABIInternal pointers. I also only found one of these: callbackasm1 -> cgocallback on Windows. These are trickier to find, though. 3. Finally, I found one case on NetBSD where new OS threads were directly calling the Go runtime entry-point from assembly via a PC, rather than going through a wrapper. This meant new threads may not have special registers set up. In this case, a change on all other OSes had already forced new thread entry to go through an ABI wrapper, so I just caught NetBSD up with that change. With this change, I'm able to run a "hello world" with GOEXPERIMENT=regabi,regabiargs. For #40724. Change-Id: I2a6d0e530c4fd4edf13484d923891c6160d683aa Reviewed-on: https://go-review.googlesource.com/c/go/+/305669 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-03-18all: explode GOEXPERIMENT=regabi into 5 sub-experimentsAustin Clements
This separates GOEXPERIMENT=regabi into five sub-experiments: regabiwrappers, regabig, regabireflect, regabidefer, and regabiargs. Setting GOEXPERIMENT=regabi now implies the working subset of these (currently, regabiwrappers, regabig, and regabireflect). This simplifies testing, helps derisk the register ABI project, and will also help with performance comparisons. This replaces the -abiwrap flag to the compiler and linker with the regabiwrappers experiment. As part of this, regabiargs now enables registers for all calls in the compiler. Previously, this was statically disabled in regabiEnabledForAllCompilation, but now that we can control it independently, this isn't necessary. For #40724. Change-Id: I5171e60cda6789031f2ef034cc2e7c5d62459122 Reviewed-on: https://go-review.googlesource.com/c/go/+/302070 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com>
2021-02-08[dev.regabi] runtime: use g register in some assembly functions on AMD64Cherry Zhang
Now that we have a g register, just use it. Note: functions that can be called from ABI0 context (e.g. morestack) is unchanged. Functions that switch g is also unchanged, because we need to set the new g in both the register and TLS. TODO: other OSes. Change-Id: I692a82a7caa8417ff620a59676a6275f56747b94 Reviewed-on: https://go-review.googlesource.com/c/go/+/289718 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-02-08[dev.regabi] cmd/compile, runtime: reserve R14 as g registers on AMD64Cherry Zhang
This is a proof-of-concept change for using the g register on AMD64. getg is now lowered to R14 in the new ABI. The g register is not yet used in all places where it can be used (e.g. stack bounds check, runtime assembly code). Change-Id: I10123ddf38e31782cf58bafcdff170aee0ff0d1b Reviewed-on: https://go-review.googlesource.com/c/go/+/289196 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: David Chase <drchase@google.com>
2020-10-30reflect,runtime: use internal ABI for selected ASM routines, attempt 2Than McIntosh
[This is a roll-forward of CL 262319, with a fix for some Darwin test failures]. Change the definitions of selected runtime assembly routines from ABI0 (the default) to ABIInternal. The ABIInternal def is intended to indicate that these functions don't follow the existing Go runtime ABI. In addition, convert the assembly reference to runtime.main (from runtime.mainPC) to ABIInternal. Finally, for functions such as "runtime.duffzero" that are called directly from generated code, make sure that the compiler looks up the correct ABI version. This is intended to support the register abi work, however these changes should not have any issues even when GOEXPERIMENT=regabi is not in effect. Updates #27539, #40724. Change-Id: Idf507f1c06176073563845239e1a54dad51a9ea9 Reviewed-on: https://go-review.googlesource.com/c/go/+/266638 Trust: Than McIntosh <thanm@google.com> Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-29Revert "reflect,runtime: use internal ABI for selected ASM routines"Than McIntosh
This reverts commit 50af50d136551e2009b2b52e829570536271cdaa. Reason for revert: Causes failures in the runtime package test on Darwin, apparently. Change-Id: I006bc1b3443fa7207e92fb4a93e3fb438d4d3de3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266257 Trust: Than McIntosh <thanm@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-29reflect,runtime: use internal ABI for selected ASM routinesThan McIntosh
Change the definitions of selected runtime assembly routines from ABI0 (the default) to ABIInternal. The ABIInternal def is intended to indicate that these functions don't follow the existing Go runtime ABI. In addition, convert the assembly reference to runtime.main (from runtime.mainPC) to ABIInternal. Finally, for functions such as "runtime.duffzero" that are called directly from generated code, make sure that the compiler looks up the correct ABI version. This is intended to support the register abi work, however these changes should not have any issues even when GOEXPERIMENT=regabi is not in effect. Updates #27539, #40724. Change-Id: I9846f8dcaccc95718cf2e61a18b7e924a0677e4c Reviewed-on: https://go-review.googlesource.com/c/go/+/262319 Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-30runtime: don't crash if vsyscall and vdso are disabled on x86_64Andrei Vagin
If vdso is disabled, the goruntime calls gettimeofday from vsyscall, but if vsyscall is disabled too, all golang binaries crash: SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0xffffffffff600000} --- killed by SIGSEGV (core dumped) ++ vsyscall doesn't work as it was designed for a long time due to security reasons and now vsyscall is a little more expensive than real syscalls: https://github.com/torvalds/linux/commit/5cec93c216db This patch reworks the code to call syscalls if the vdso library isn't available. Change-Id: I16cbf3f49871bea91e26af1f49aa0ae2fbd3a01d GitHub-Last-Rev: 1d133cd30a5dee1fea9aee0fb4ea0b07e0e87f2a GitHub-Pull-Request: golang/go#41681 Reviewed-on: https://go-review.googlesource.com/c/go/+/257982 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Pratt <mpratt@google.com>
2020-08-19runtime: reduce syscall when call runtime.clonecui
Change-Id: I3ea398fd86aae4c86557dd6fff65d90a6f756890 GitHub-Last-Rev: 4c295388f7b5e6768ffd2530337f78b4c75a9310 GitHub-Pull-Request: golang/go#40392 Reviewed-on: https://go-review.googlesource.com/c/go/+/244626 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-16crypto,internal/bytealg: fix assembly that clobbers BPKeith Randall
BP should be callee-save. It will be saved automatically if there is a nonzero frame size. Otherwise, we need to avoid this register. Change-Id: If3f551efa42d830c8793d9f0183cb8daad7a2ab5 Reviewed-on: https://go-review.googlesource.com/c/go/+/248260 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-13runtime: revert signal stack mlockingAustin Clements
Go 1.14 included a (rather awful) workaround for a Linux kernel bug that corrupted vector registers on x86 CPUs during signal delivery (https://bugzilla.kernel.org/show_bug.cgi?id=205663). This bug was introduced in Linux 5.2 and fixed in 5.3.15, 5.4.2 and all 5.5 and later kernels. The fix was also back-ported by major distros. This workaround was necessary, but had unfortunate downsides, including causing Go programs to exceed the mlock ulimit in many configurations (#37436). We're reasonably confident that by the Go 1.16 release, the number of systems running affected kernels will be vanishingly small. Hence, this CL removes this workaround. This effectively reverts CLs 209597 (version parser), 209899 (mlock top of signal stack), 210299 (better failure message), 223121 (soft mlock failure handling), and 244059 (special-case patched Ubuntu kernels). The one thing we keep is the osArchInit function. It's empty everywhere now, but is a reasonable hook to have. Updates #35326, #35777 (the original register corruption bugs). Updates #40184 (request to revert in 1.15). Fixes #35979. Change-Id: Ie213270837095576f1f3ef46bf3de187dc486c50 Reviewed-on: https://go-review.googlesource.com/c/go/+/246200 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-10runtime: make nanotime1 reentrantCherry Zhang
Currently, nanotime1 (and walltime1) is not reentrant, in that it sets m.vdsoSP at entry and clears it at exit. If a signal lands in between, and nanotime1 is called from the signal handler, it will clear m.vdsoSP while we are still in nanotime1. If (in the unlikely event) it is signaled again, m.vdsoSP will be wrong, which may cause the stack unwinding code to crash. This CL makes it reentrant, by saving/restoring the previous vdsoPC and vdsoSP, instead of setting it to 0 at exit. TODO: have some way to test? Change-Id: I9ee53b251f1d8a5a489c71d4b4c0df1dee70c3e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/246763 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-02-13runtime: correct caller PC/SP offsets in walltime1/nanotime1Cherry Zhang
In walltime1/nanotime1, we save the caller's PC and SP for stack unwinding. The code does that assumed zero frame size. Now that the frame size is not zero, correct the offset. Rewrite it in a way that doesn't depend on hard-coded frame size. May fix #37127. Change-Id: I47d6d54fc3499d7d5946c3f6a2dbd24fbd679de1 Reviewed-on: https://go-review.googlesource.com/c/go/+/219118 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-12-20runtime: make sure BP is saved in nanotime1/walltime1, else frame pointer ↵Dan Scales
may not be preserved nanotime1 and walltime1 do not preserve BP on linux amd64. Previously, this did not cause a problem, because nanotime/walltime do preserve the BP. But now with mid-stack inlining, nanotime/walltime are usually inlined, so BP is not preserved. So, the BP is now wrong in any function after a call to nanotime()/walltime() on amd64. That means the frame pointer on the stack can be wrong for any further function call made after the nanotime() call (notably runtime.main and various GC functions). [386 doesn't use framepointer.] Fix is to set a frame size of 8 for nanotime1 and walltime1, which means the standard prolog/epilog that saves/restore BP in the stack frame is added. I noticed this while investigating issue 16638 (use frame pointers for runtime.Callers). This change would needed for progress on that issue (which doesn't have a high priority). Verified that this fix works/is useful for issue 16638. Change-Id: I19e19ef2c1a517d737a34928baae034f2eb0b2c2 Reviewed-on: https://go-review.googlesource.com/c/go/+/212079 Run-TryBot: Dan Scales <danscales@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-05runtime: mlock top of signal stack on Linux 5.2–5.4.1Austin Clements
Linux 5.2 introduced a bug that can corrupt vector registers on return from a signal if the signal stack isn't faulted in: https://bugzilla.kernel.org/show_bug.cgi?id=205663 This CL works around this by mlocking the top page of all Go signal stacks on the affected kernels. Fixes #35326, #35777 Change-Id: I77c80a2baa4780827633f92f464486caa222295d Reviewed-on: https://go-review.googlesource.com/c/go/+/209899 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-10-26runtime: M-targeted signals for LinuxAustin Clements
We'll add a test once all of the POSIX platforms are done. For #10958, #24543. Change-Id: If7e3f14e8391791364877629bf415d9f8e788b0a Reviewed-on: https://go-review.googlesource.com/c/go/+/201401 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-21runtime: change read and write to return negative errno valueIan Lance Taylor
The internal read and write functions used to return -1 on error; change them to return a negative errno value instead. This will be used by later CLs in this series. For most targets this is a simplification, although for ones that call into libc it is a complication. Updates #27707 Change-Id: Id02bf9487f03e7e88e4f2b85e899e986738697ad Reviewed-on: https://go-review.googlesource.com/c/go/+/171823 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-20runtime: define nonblockingPipeIan Lance Taylor
This requires defining pipe, pipe2, and setNonblock for various platforms. The new function is currently only used on AIX. It will be used by later CLs in this series. Updates #27707 Change-Id: Id2f987b66b4c66a3ef40c22484ff1d14f58e9b31 Reviewed-on: https://go-review.googlesource.com/c/go/+/171822 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-04runtime: wrap nanotime, walltime, and writeAustin Clements
In preparation for general faketime support, this renames the existing nanotime, walltime, and write functions to nanotime1, walltime1, and write1 and wraps them with trivial Go functions. This will let us inject different implementations on all platforms when faketime is enabled. Updates #30439. Change-Id: Ice5ccc513a32a6d89ea051638676d3ee05b00418 Reviewed-on: https://go-review.googlesource.com/c/go/+/192738 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-29cmd/link/ld,cmd/internal/obj,runtime: make the Android TLS offset dynamicElias Naur
We're going to need a different TLS offset for Android Q, so the static offsets used for 386 and amd64 are no longer viable on Android. Introduce runtime·tls_g and use that for indexing into TLS storage. As an added benefit, we can then merge the TLS setup code for all android GOARCHs. While we're at it, remove a bunch of android special cases no longer needed. Updates #29674 Updates #29249 (perhaps fixes it) Change-Id: I77c7385aec7de8f1f6a4da7c9c79999157e39572 Reviewed-on: https://go-review.googlesource.com/c/go/+/169817 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-10-03all: this big patch remove whitespace from assembly filesZhou Peng
Don't worry, this patch just remove trailing whitespace from assembly files, and does not touch any logical changes. Change-Id: Ia724ac0b1abf8bc1e41454bdc79289ef317c165d Reviewed-on: https://go-review.googlesource.com/c/113595 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-09-18runtime: use MADV_FREE on Linux if availableTobias Klauser
On Linux, sysUnused currently uses madvise(MADV_DONTNEED) to signal the kernel that a range of allocated memory contains unneeded data. After a successful call, the range (but not the data it contained before the call to madvise) is still available but the first access to that range will unconditionally incur a page fault (needed to 0-fill the range). A faster alternative is MADV_FREE, available since Linux 4.5. The mechanism is very similar, but the page fault will only be incurred if the kernel, between the call to madvise and the first access, decides to reuse that memory for something else. In sysUnused, test whether MADV_FREE is supported and fall back to MADV_DONTNEED in case it isn't. This requires making the return value of the madvise syscall available to the caller, so change runtime.madvise to return it. Fixes #23687 Change-Id: I962c3429000dd9f4a00846461ad128b71201bb04 Reviewed-on: https://go-review.googlesource.com/135395 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-09-07runtime: use tgkill for raiseMichael Pratt
raise uses tkill to send a signal to the current thread. For this use, tgkill is functionally equivalent to tkill expect that it also takes the pid as the first argument. Using tgkill makes it simpler to run a Go program in a strict sandbox. With kill and tgkill, the sandbox policy (e.g., seccomp) can prevent the program from sending signals to other processes by checking that the first argument == getpid(). With tkill, the policy must whitelist all tids in the process, which is effectively impossible given Go's dynamic thread creation. Fixes #27548 Change-Id: I8ed282ef1f7215b02ef46de144493e36454029ea Reviewed-on: https://go-review.googlesource.com/133975 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-04-24runtime: change GNU/Linux usleep to use nanosleepIan Lance Taylor
Ever since we added sleep to the runtime back in 2008, we've implemented it on GNU/Linux with the select (or pselect or pselect6) system call. But the Linux kernel has a nanosleep system call, which should be a tiny bit more efficient since it doesn't have to check to see whether there are any file descriptors. So use it. Change-Id: Icc3430baca46b082a4d33f97c6c47e25fa91cb9a Reviewed-on: https://go-review.googlesource.com/108538 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-03-15runtime: use Android O friendly faccessat syscall on linux/amd64Tobias Klauser
The Android O seccomp policy disallows the access syscall on amd64, see https://android.googlesource.com/platform/bionic/+/android-4.2.2_r1.2/libc/SYSCALLS.TXT Use the faccessat syscall with AT_FDCWD instead to achieve the same behavior. Updates #24403 Change-Id: I9db847c1c0f33987a3479b3f96e721fb9588cde2 Reviewed-on: https://go-review.googlesource.com/100877 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-03-07runtime: get traceback from VDSO codeIan Lance Taylor
Currently if a profiling signal arrives while executing within a VDSO the profiler will report _ExternalCode, which is needlessly confusing for a pure Go program. Change the VDSO calling code to record the caller's PC/SP, so that we can do a traceback from that point. If that fails for some reason, report _VDSO rather than _ExternalCode, which should at least point in the right direction. This adds some instructions to the code that calls the VDSO, but the slowdown is reasonably negligible: name old time/op new time/op delta ClockVDSOAndFallbackPaths/vDSO-8 40.5ns ± 2% 41.3ns ± 1% +1.85% (p=0.002 n=10+10) ClockVDSOAndFallbackPaths/Fallback-8 41.9ns ± 1% 43.5ns ± 1% +3.84% (p=0.000 n=9+9) TimeNow-8 41.5ns ± 3% 41.5ns ± 2% ~ (p=0.723 n=10+10) Fixes #24142 Change-Id: Iacd935db3c4c782150b3809aaa675a71799b1c9c Reviewed-on: https://go-review.googlesource.com/97315 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-03-07runtime: change from rt_sigaction to sigactionIan Lance Taylor
This normalizes the Linux code to act like other targets. The size argument to the rt_sigaction system call is pushed to a single function, sysSigaction. This is intended as a simplification step for CL 93875 for #14327. Change-Id: I594788e235f0da20e16e8a028e27ac8c883907c4 Reviewed-on: https://go-review.googlesource.com/99077 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-03-05runtime: rename vdso symbols to use camel caseIan Lance Taylor
This was originally C code using names with underscores, which were retained when the code was rewritten into Go. Change the code to use Go-like camel case names. The names that come from the ELF ABI are left unchanged. Change-Id: I181bc5dd81284c07bc67b7df4635f4734b41d646 Reviewed-on: https://go-review.googlesource.com/98520 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-02-27runtime: simplify walltime/nanotime on linux/{386,amd64}Tobias Klauser
Avoid an unnecessary MOVL/MOVQ. Follow CL 97377 Change-Id: Ic43976d6b0cece3ed455496d18aedd67e0337d3f Reviewed-on: https://go-review.googlesource.com/97358 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-02-17runtime: remove unused getrlimit functionTobias Klauser
Follow CL 93655 which removed the (commented-out) usage of this function. Also remove unused constant _RLIMIT_AS and type rlimit. Change-Id: Ifb6e6b2104f4c2555269f8ced72bfcae24f5d5e9 Reviewed-on: https://go-review.googlesource.com/94775 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-02-14runtime: add symbol for AT_FDCWD on Linux amd64 and mips64xTobias Klauser
Also order the syscall number list by numerically for mips64x. Follow-up for CL 92895. Change-Id: I5f01f8c626132a06160997fce8a2aef0c486bb1c Reviewed-on: https://go-review.googlesource.com/93616 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-02-13runtime: use Android O friendly syscalls on 64-bit machinesJason A. Donenfeld
Android O disallows open on 64-bit, so let's use openat with AT_FDCWD to achieve the same behavior. Android O disallows epoll_wait on 64-bit, so let's use epoll_pwait with the last argument as NULL to achieve the same behavior. See here: https://android.googlesource.com/platform/bionic/+/master/libc/seccomp/arm64_app_policy.cpp https://android.googlesource.com/platform/bionic/+/master/libc/seccomp/mips64_app_policy.cpp https://android.googlesource.com/platform/bionic/+/master/libc/seccomp/x86_64_app_policy.cpp Fixes #23750 Change-Id: If8d5a663357471e5d2c1f516151344a9d05b188a Reviewed-on: https://go-review.googlesource.com/92895 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-11-14runtime: call amd64 VDSO entry points on large stackIan Lance Taylor
If the Linux kernel was built with CONFIG_OPTIMIZE_INLINING=n and was built with hardening options turned on, GCC will insert a stack probe in the VDSO function that requires a full page of stack space. The stack probe can corrupt memory if another thread is using it. Avoid sporadic crashes by calling the VDSO on the g0 or gsignal stack. While we're at it, align the stack as C code expects. We've been getting away with a misaligned stack, but it's possible that the VDSO code will change in the future to break that assumption. Benchmarks show a 11% hit on time.Now, but it's only 6ns. name old time/op new time/op delta AfterFunc-12 1.66ms ± 0% 1.66ms ± 1% ~ (p=0.905 n=9+10) After-12 1.90ms ± 6% 1.86ms ± 0% -2.05% (p=0.012 n=10+8) Stop-12 113µs ± 3% 115µs ± 2% +1.60% (p=0.017 n=9+10) SimultaneousAfterFunc-12 145µs ± 1% 144µs ± 0% -0.68% (p=0.002 n=10+8) StartStop-12 39.5µs ± 3% 40.4µs ± 5% +2.19% (p=0.023 n=10+10) Reset-12 10.2µs ± 0% 10.4µs ± 0% +2.45% (p=0.000 n=10+9) Sleep-12 190µs ± 1% 190µs ± 1% ~ (p=0.971 n=10+10) Ticker-12 4.68ms ± 2% 4.64ms ± 2% -0.83% (p=0.043 n=9+10) Now-12 48.4ns ±11% 54.0ns ±11% +11.42% (p=0.017 n=10+10) NowUnixNano-12 48.5ns ±13% 56.9ns ± 8% +17.30% (p=0.000 n=10+10) Format-12 489ns ±11% 504ns ± 6% ~ (p=0.289 n=10+10) FormatNow-12 436ns ±23% 480ns ±13% +10.25% (p=0.026 n=9+10) MarshalJSON-12 656ns ±14% 587ns ±24% ~ (p=0.063 n=10+10) MarshalText-12 647ns ± 7% 638ns ± 9% ~ (p=0.516 n=10+10) Parse-12 348ns ± 8% 328ns ± 9% -5.66% (p=0.030 n=10+10) ParseDuration-12 136ns ± 9% 140ns ±11% ~ (p=0.425 n=10+10) Hour-12 14.8ns ± 6% 15.6ns ±11% ~ (p=0.085 n=10+10) Second-12 14.0ns ± 6% 14.3ns ±12% ~ (p=0.443 n=10+10) Year-12 32.4ns ±11% 33.4ns ± 6% ~ (p=0.492 n=10+10) Day-12 41.5ns ± 9% 42.3ns ±12% ~ (p=0.239 n=10+10) Fixes #20427 Change-Id: Ia395cbb863215f4499b8e7ef95f4b99f51090911 Reviewed-on: https://go-review.googlesource.com/76990 Reviewed-by: Austin Clements <austin@google.com>
2017-10-18runtime: separate error result for mmapAustin Clements
Currently mmap returns an unsafe.Pointer that encodes OS errors as values less than 4096. In practice this is okay, but it borders on being really unsafe: for example, the value has to be checked immediately after return and if stack copying were ever to observe such a value, it would panic. It's also not remotely idiomatic. Fix this by making mmap return a separate pointer value and error, like a normal Go function. Updates #22218. Change-Id: Iefd965095ffc82cc91118872753a5d39d785c3a6 Reviewed-on: https://go-review.googlesource.com/71270 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-11runtime: make it possible to exit Go-created threadsAustin Clements
Currently, threads created by the runtime exist until the whole program exits. For #14592 and #20395, we want to be able to exit and clean up threads created by the runtime. This commit implements that mechanism. The main difficulty is how to clean up the g0 stack. In cgo mode and on Solaris and Windows where the OS manages thread stacks, we simply arrange to return from mstart and let the system clean up the thread. If the runtime allocated the g0 stack, then we use a new exitThread syscall wrapper that arranges to clear a flag in the M once the stack can safely be reaped and call the thread termination syscall. exitThread is based on the existing exit1 wrapper, which was always meant to terminate the calling thread. However, exit1 has never been used since it was introduced 9 years ago, so it was broken on several platforms. exitThread also has the additional complication of having to flag that the stack is unused, which requires some tricks on platforms that use the stack for syscalls. This still leaves the problem of how to reap the unused g0 stacks. For this, we move the M from allm to a new freem list as part of the M exiting. Later, allocm scans the freem list, finds Ms that are marked as done with their stack, removes these from the list and frees their g0 stacks. This also allows these Ms to be garbage collected. This CL does not yet use any of this functionality. Follow-up CLs will. Likewise, there are no new tests in this CL because we'll need follow-up functionality to test it. Change-Id: Ic851ee74227b6d39c6fc1219fc71b45d3004bc63 Reviewed-on: https://go-review.googlesource.com/46037 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 16:27:13 +0000