aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/sys_linux_386.s
AgeCommit message (Collapse)Author
2023-03-24runtime: add "sigaction" to sigreturn symbol nameMichael Pratt
In order to identify the sigreturn function, gdb looks for "__restore_rt". However because that symbol is sometimes missing from the symbol table, it also performs the same instruction matching as libgcc, but only in symbols containing "sigaction" (it expects sigaction to preceed __restore_rt). To match this heuristic, we add __sigaction to the sigreturn symbol name. Fixes #25218. Change-Id: I09cb231ad23f668d451f31dd5633f782355fc91d Reviewed-on: https://go-review.googlesource.com/c/go/+/479096 Auto-Submit: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2022-10-18runtime: always keep global reference to mp until mexit completesMichael Pratt
Ms are allocated via standard heap allocation (`new(m)`), which means we must keep them alive (i.e., reachable by the GC) until we are completely done using them. Ms are primarily reachable through runtime.allm. However, runtime.mexit drops the M from allm fairly early, long before it is done using the M structure. If that was the last reference to the M, it is now at risk of being freed by the GC and used for some other allocation, leading to memory corruption. Ms with a Go-allocated stack coincidentally already keep a reference to the M in sched.freem, so that the stack can be freed lazily. This reference has the side effect of keeping this Ms reachable. However, Ms with an OS stack skip this and are at risk of corruption. Fix this lifetime by extending sched.freem use to all Ms, with the value of mp.freeWait determining whether the stack needs to be freed or not. Fixes #56243. Change-Id: Ic0c01684775f5646970df507111c9abaac0ba52e Reviewed-on: https://go-review.googlesource.com/c/go/+/443716 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-10-07runtime: move epoll syscalls to runtime/internal/syscallAndrew Pogrebnoy
This change moves Linux epoll's syscalls implementation to the "runtime/internal/syscall" package. The intention in this CL was to minimise behavioural changes but make the code more generalised. This also will allow adding new syscalls (like epoll_pwait2) without the need to implement assembly stubs for each arch. It also drops epoll_create as not all architectures provide this call. epoll_create1 was added to the kernel in version 2.6.27 and Go requires Linux kernel version 2.6.32 or later since Go 1.18. So it is safe to always use epoll_create1. This is a resubmit as the previous CL 421994 was reverted due to test failures after the merge with the master. The issue was fixed in CL 438615 For #53824 For #51087 Change-Id: I1bd0f23a85b4f9b80178c5dd36fd3e95ff4f9648 Reviewed-on: https://go-review.googlesource.com/c/go/+/440115 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com>
2022-09-30Revert "runtime: move epoll syscalls to runtime/internal/syscall"Michael Pratt
This reverts CL 421994. Reason for revert: breaks runtime.TestCheckPtr2 For #53824 For #51087 Change-Id: I044ea4d6efdffe0a4b7fb0d2bb3717d9f391fc59 Reviewed-on: https://go-review.googlesource.com/c/go/+/437295 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-09-30runtime: move epoll syscalls to runtime/internal/syscallAndrew Pogrebnoy
This change moves Linux epoll's syscalls implementation to the "runtime/internal/syscall" package. The intention in this CL was to minimise behavioural changes but make the code more generalised. This also will allow adding new syscalls (like epoll_pwait2) without the need to implement assembly stubs for each arch. It also drops epoll_create as not all architectures provide this call. epoll_create1 was added to the kernel in version 2.6.27 and Go requires Linux kernel version 2.6.32 or later since Go 1.18. So it is safe to always use epoll_create1. For #53824 For #51087 Change-Id: I9a6a26b7f2075a38e041de1bab4691da0ecb94fc Reviewed-on: https://go-review.googlesource.com/c/go/+/421994 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com>
2022-04-28runtime: mark sigtramp as TOPFRAMEMichael Pratt
Currently throw() in the signal handler results in "fatal error: unknown return pc from runtime.sigreturn ...". Marking sigtramp as TOPFRAME allows gentraceback to stop tracebacks at the end of a signal handler, since there is not much beyond sigtramp. This is just done on Linux for now, but may apply to other Unix systems as well. Change-Id: I96edcb945283f417a5bfe00ce2fb2b1a0d578692 Reviewed-on: https://go-review.googlesource.com/c/go/+/402190 Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-03runtime: remove fallback to pipe on platforms with pipe2Tobias Klauser
On Linux, the minimum required kernel version for Go 1.18 was be changed to 2.6.32, see #45964. The pipe2 syscall was added in 2.6.27. All other platforms already provide the pipe2 syscall in the minimum supported version: - DragonFly BSD added it in version 4.2, see https://www.dragonflybsd.org/release42/ - FreeBSD added it in version 10.0, see https://www.freebsd.org/cgi/man.cgi?pipe(2)#end - NetBSD added it in version 6.0, see https://man.netbsd.org/pipe2.2#HISTORY - OpenBSD added it in version 5.7, see https://man.openbsd.org/pipe.2#HISTORY - Illumos supports it since 2013, see https://www.illumos.org/issues/3714 - Solaris supports it since 11.4 This also allows to remove setNonblock which was only used in the pipe fallback path on these platforms. Change-Id: I1f40d32fd3065d74e22af77b9ff2292b9cf66706 Reviewed-on: https://go-review.googlesource.com/c/go/+/389354 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-09-27runtime: add timer_create syscalls for LinuxRhys Hiltner
Updates #35057 Change-Id: Id702b502fa4e4005ba1e450a945bc4420a8a8b8c Reviewed-on: https://go-review.googlesource.com/c/go/+/342052 Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Than McIntosh <thanm@google.com>
2021-04-29runtime: rename walltime1 to walltimeIan Lance Taylor
Change-Id: Iec9de5ca56eb68d524bbaa0668515dbd09ad38a1 Reviewed-on: https://go-review.googlesource.com/c/go/+/314770 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-15runtime: unify C->Go ABI transitionsAustin Clements
The previous CL introduced macros for transitions from the Windows ABI to the Go ABI. This CL does the same for SysV and uses them in almost all places where we transition from the C ABI to the Go ABI. Compared to Windows, this transition is much simpler and I didn't find any places that were getting it wrong. But this does let us unify a lot of code nicely and introduces some degree of abstraction around these ABI transitions. Change-Id: Ib6bdecafce587ce18fca4c8300fcf401284a2bcd Reviewed-on: https://go-review.googlesource.com/c/go/+/309930 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-08-13runtime: revert signal stack mlockingAustin Clements
Go 1.14 included a (rather awful) workaround for a Linux kernel bug that corrupted vector registers on x86 CPUs during signal delivery (https://bugzilla.kernel.org/show_bug.cgi?id=205663). This bug was introduced in Linux 5.2 and fixed in 5.3.15, 5.4.2 and all 5.5 and later kernels. The fix was also back-ported by major distros. This workaround was necessary, but had unfortunate downsides, including causing Go programs to exceed the mlock ulimit in many configurations (#37436). We're reasonably confident that by the Go 1.16 release, the number of systems running affected kernels will be vanishingly small. Hence, this CL removes this workaround. This effectively reverts CLs 209597 (version parser), 209899 (mlock top of signal stack), 210299 (better failure message), 223121 (soft mlock failure handling), and 244059 (special-case patched Ubuntu kernels). The one thing we keep is the osArchInit function. It's empty everywhere now, but is a reasonable hook to have. Updates #35326, #35777 (the original register corruption bugs). Updates #40184 (request to revert in 1.15). Fixes #35979. Change-Id: Ie213270837095576f1f3ef46bf3de187dc486c50 Reviewed-on: https://go-review.googlesource.com/c/go/+/246200 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-10runtime: make nanotime1 reentrantCherry Zhang
Currently, nanotime1 (and walltime1) is not reentrant, in that it sets m.vdsoSP at entry and clears it at exit. If a signal lands in between, and nanotime1 is called from the signal handler, it will clear m.vdsoSP while we are still in nanotime1. If (in the unlikely event) it is signaled again, m.vdsoSP will be wrong, which may cause the stack unwinding code to crash. This CL makes it reentrant, by saving/restoring the previous vdsoPC and vdsoSP, instead of setting it to 0 at exit. TODO: have some way to test? Change-Id: I9ee53b251f1d8a5a489c71d4b4c0df1dee70c3e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/246763 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-02-13runtime: correct caller PC/SP offsets in walltime1/nanotime1Cherry Zhang
In walltime1/nanotime1, we save the caller's PC and SP for stack unwinding. The code does that assumed zero frame size. Now that the frame size is not zero, correct the offset. Rewrite it in a way that doesn't depend on hard-coded frame size. May fix #37127. Change-Id: I47d6d54fc3499d7d5946c3f6a2dbd24fbd679de1 Reviewed-on: https://go-review.googlesource.com/c/go/+/219118 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-12-09runtime: mlock top of signal stack on both amd64 and 386Austin Clements
CL 209899 worked around an issue that corrupts vector registers in recent versions of the Linux kernel by mlocking the top page of every signal stack on amd64. However, the underlying issue also affects the XMM registers on 386. This CL applies the mlock fix to both amd64 and 386. Fixes #35777 (again). Change-Id: I9886f2dc4c23625421296bd5518d5fd3288bfe48 Reviewed-on: https://go-review.googlesource.com/c/go/+/210345 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-26runtime: M-targeted signals for LinuxAustin Clements
We'll add a test once all of the POSIX platforms are done. For #10958, #24543. Change-Id: If7e3f14e8391791364877629bf415d9f8e788b0a Reviewed-on: https://go-review.googlesource.com/c/go/+/201401 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-21runtime: change read and write to return negative errno valueIan Lance Taylor
The internal read and write functions used to return -1 on error; change them to return a negative errno value instead. This will be used by later CLs in this series. For most targets this is a simplification, although for ones that call into libc it is a complication. Updates #27707 Change-Id: Id02bf9487f03e7e88e4f2b85e899e986738697ad Reviewed-on: https://go-review.googlesource.com/c/go/+/171823 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-20runtime: define nonblockingPipeIan Lance Taylor
This requires defining pipe, pipe2, and setNonblock for various platforms. The new function is currently only used on AIX. It will be used by later CLs in this series. Updates #27707 Change-Id: Id2f987b66b4c66a3ef40c22484ff1d14f58e9b31 Reviewed-on: https://go-review.googlesource.com/c/go/+/171822 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-04runtime: wrap nanotime, walltime, and writeAustin Clements
In preparation for general faketime support, this renames the existing nanotime, walltime, and write functions to nanotime1, walltime1, and write1 and wraps them with trivial Go functions. This will let us inject different implementations on all platforms when faketime is enabled. Updates #30439. Change-Id: Ice5ccc513a32a6d89ea051638676d3ee05b00418 Reviewed-on: https://go-review.googlesource.com/c/go/+/192738 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09runtime: fix vet complaints for linux/386Russ Cox
Working toward making the tree vet-safe instead of having so many exceptions in cmd/vet/all/whitelist. This CL makes "GOOS=linux GOARCH=386 go vet -unsafeptr=false runtime" happy, while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too. For #31916. Change-Id: I3e5586a7ff6e359357350d0602c2259493280ded Reviewed-on: https://go-review.googlesource.com/c/go/+/176099 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-03-29cmd/link/ld,cmd/internal/obj,runtime: make the Android TLS offset dynamicElias Naur
We're going to need a different TLS offset for Android Q, so the static offsets used for 386 and amd64 are no longer viable on Android. Introduce runtime·tls_g and use that for indexing into TLS storage. As an added benefit, we can then merge the TLS setup code for all android GOARCHs. While we're at it, remove a bunch of android special cases no longer needed. Updates #29674 Updates #29249 (perhaps fixes it) Change-Id: I77c7385aec7de8f1f6a4da7c9c79999157e39572 Reviewed-on: https://go-review.googlesource.com/c/go/+/169817 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-09-18runtime: use MADV_FREE on Linux if availableTobias Klauser
On Linux, sysUnused currently uses madvise(MADV_DONTNEED) to signal the kernel that a range of allocated memory contains unneeded data. After a successful call, the range (but not the data it contained before the call to madvise) is still available but the first access to that range will unconditionally incur a page fault (needed to 0-fill the range). A faster alternative is MADV_FREE, available since Linux 4.5. The mechanism is very similar, but the page fault will only be incurred if the kernel, between the call to madvise and the first access, decides to reuse that memory for something else. In sysUnused, test whether MADV_FREE is supported and fall back to MADV_DONTNEED in case it isn't. This requires making the return value of the madvise syscall available to the caller, so change runtime.madvise to return it. Fixes #23687 Change-Id: I962c3429000dd9f4a00846461ad128b71201bb04 Reviewed-on: https://go-review.googlesource.com/135395 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-09-07runtime: use tgkill for raiseMichael Pratt
raise uses tkill to send a signal to the current thread. For this use, tgkill is functionally equivalent to tkill expect that it also takes the pid as the first argument. Using tgkill makes it simpler to run a Go program in a strict sandbox. With kill and tgkill, the sandbox policy (e.g., seccomp) can prevent the program from sending signals to other processes by checking that the first argument == getpid(). With tkill, the policy must whitelist all tids in the process, which is effectively impossible given Go's dynamic thread creation. Fixes #27548 Change-Id: I8ed282ef1f7215b02ef46de144493e36454029ea Reviewed-on: https://go-review.googlesource.com/133975 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-04-24runtime: change GNU/Linux usleep to use nanosleepIan Lance Taylor
Ever since we added sleep to the runtime back in 2008, we've implemented it on GNU/Linux with the select (or pselect or pselect6) system call. But the Linux kernel has a nanosleep system call, which should be a tiny bit more efficient since it doesn't have to check to see whether there are any file descriptors. So use it. Change-Id: Icc3430baca46b082a4d33f97c6c47e25fa91cb9a Reviewed-on: https://go-review.googlesource.com/108538 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-03-07runtime: get traceback from VDSO codeIan Lance Taylor
Currently if a profiling signal arrives while executing within a VDSO the profiler will report _ExternalCode, which is needlessly confusing for a pure Go program. Change the VDSO calling code to record the caller's PC/SP, so that we can do a traceback from that point. If that fails for some reason, report _VDSO rather than _ExternalCode, which should at least point in the right direction. This adds some instructions to the code that calls the VDSO, but the slowdown is reasonably negligible: name old time/op new time/op delta ClockVDSOAndFallbackPaths/vDSO-8 40.5ns ± 2% 41.3ns ± 1% +1.85% (p=0.002 n=10+10) ClockVDSOAndFallbackPaths/Fallback-8 41.9ns ± 1% 43.5ns ± 1% +3.84% (p=0.000 n=9+9) TimeNow-8 41.5ns ± 3% 41.5ns ± 2% ~ (p=0.723 n=10+10) Fixes #24142 Change-Id: Iacd935db3c4c782150b3809aaa675a71799b1c9c Reviewed-on: https://go-review.googlesource.com/97315 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-03-05runtime: rename vdso symbols to use camel caseIan Lance Taylor
This was originally C code using names with underscores, which were retained when the code was rewritten into Go. Change the code to use Go-like camel case names. The names that come from the ELF ABI are left unchanged. Change-Id: I181bc5dd81284c07bc67b7df4635f4734b41d646 Reviewed-on: https://go-review.googlesource.com/98520 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-02-27runtime: simplify walltime/nanotime on linux/{386,amd64}Tobias Klauser
Avoid an unnecessary MOVL/MOVQ. Follow CL 97377 Change-Id: Ic43976d6b0cece3ed455496d18aedd67e0337d3f Reviewed-on: https://go-review.googlesource.com/97358 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-02-17runtime: remove unused getrlimit functionTobias Klauser
Follow CL 93655 which removed the (commented-out) usage of this function. Also remove unused constant _RLIMIT_AS and type rlimit. Change-Id: Ifb6e6b2104f4c2555269f8ced72bfcae24f5d5e9 Reviewed-on: https://go-review.googlesource.com/94775 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2017-11-14runtime: call amd64 VDSO entry points on large stackIan Lance Taylor
If the Linux kernel was built with CONFIG_OPTIMIZE_INLINING=n and was built with hardening options turned on, GCC will insert a stack probe in the VDSO function that requires a full page of stack space. The stack probe can corrupt memory if another thread is using it. Avoid sporadic crashes by calling the VDSO on the g0 or gsignal stack. While we're at it, align the stack as C code expects. We've been getting away with a misaligned stack, but it's possible that the VDSO code will change in the future to break that assumption. Benchmarks show a 11% hit on time.Now, but it's only 6ns. name old time/op new time/op delta AfterFunc-12 1.66ms ± 0% 1.66ms ± 1% ~ (p=0.905 n=9+10) After-12 1.90ms ± 6% 1.86ms ± 0% -2.05% (p=0.012 n=10+8) Stop-12 113µs ± 3% 115µs ± 2% +1.60% (p=0.017 n=9+10) SimultaneousAfterFunc-12 145µs ± 1% 144µs ± 0% -0.68% (p=0.002 n=10+8) StartStop-12 39.5µs ± 3% 40.4µs ± 5% +2.19% (p=0.023 n=10+10) Reset-12 10.2µs ± 0% 10.4µs ± 0% +2.45% (p=0.000 n=10+9) Sleep-12 190µs ± 1% 190µs ± 1% ~ (p=0.971 n=10+10) Ticker-12 4.68ms ± 2% 4.64ms ± 2% -0.83% (p=0.043 n=9+10) Now-12 48.4ns ±11% 54.0ns ±11% +11.42% (p=0.017 n=10+10) NowUnixNano-12 48.5ns ±13% 56.9ns ± 8% +17.30% (p=0.000 n=10+10) Format-12 489ns ±11% 504ns ± 6% ~ (p=0.289 n=10+10) FormatNow-12 436ns ±23% 480ns ±13% +10.25% (p=0.026 n=9+10) MarshalJSON-12 656ns ±14% 587ns ±24% ~ (p=0.063 n=10+10) MarshalText-12 647ns ± 7% 638ns ± 9% ~ (p=0.516 n=10+10) Parse-12 348ns ± 8% 328ns ± 9% -5.66% (p=0.030 n=10+10) ParseDuration-12 136ns ± 9% 140ns ±11% ~ (p=0.425 n=10+10) Hour-12 14.8ns ± 6% 15.6ns ±11% ~ (p=0.085 n=10+10) Second-12 14.0ns ± 6% 14.3ns ±12% ~ (p=0.443 n=10+10) Year-12 32.4ns ±11% 33.4ns ± 6% ~ (p=0.492 n=10+10) Day-12 41.5ns ± 9% 42.3ns ±12% ~ (p=0.239 n=10+10) Fixes #20427 Change-Id: Ia395cbb863215f4499b8e7ef95f4b99f51090911 Reviewed-on: https://go-review.googlesource.com/76990 Reviewed-by: Austin Clements <austin@google.com>
2017-10-18runtime: separate error result for mmapAustin Clements
Currently mmap returns an unsafe.Pointer that encodes OS errors as values less than 4096. In practice this is okay, but it borders on being really unsafe: for example, the value has to be checked immediately after return and if stack copying were ever to observe such a value, it would panic. It's also not remotely idiomatic. Fix this by making mmap return a separate pointer value and error, like a normal Go function. Updates #22218. Change-Id: Iefd965095ffc82cc91118872753a5d39d785c3a6 Reviewed-on: https://go-review.googlesource.com/71270 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-13runtime: use vDSO on linux/386 to improve time.Now performanceFrank Somers
This change adds support for accelerating time.Now by using the __vdso_clock_gettime fast-path via the vDSO on linux/386 if it is available. When the vDSO path to the clocks is available, it is typically 5x-10x faster than the syscall path (see benchmark extract below). Two such calls are made for each time.Now() call on most platforms as of go 1.9. - Add vdso_linux_386.go, containing the ELF32 definitions for use by vdso_linux.go, the maximum array size, and the symbols to be located in the vDSO. - Modify runtime.walltime and runtime.nanotime to check for and use the vDSO fast-path if available, or fall back to the existing syscall path. - Reduce the stack reservations for runtime.walltime and runtime.monotime from 32 to 16 bytes. It appears the syscall path actually only needed 8 bytes, but 16 is now needed to cover the syscall and vDSO paths. - Remove clearing DX from the syscall paths as clock_gettime only takes 2 args (BX, CX in syscall calling convention), so there should be no need to clear DX. The included BenchmarkTimeNow was run with -cpu=1 -count=20 on an "Intel(R) Celeron(R) CPU J1900 @ 1.99GHz", comparing released go 1.9.1 vs this change. This shows a gain in performance on linux/386 (6.89x), and that no regression occurred on linux/amd64 due to this change. Kernel: linux/i686, GOOS=linux GOARCH=386 name old time/op new time/op delta TimeNow 978ns ± 0% 142ns ± 0% -85.48% (p=0.000 n=16+20) Kernel: linux/x86_64, GOOS=linux GOARCH=amd64 name old time/op new time/op delta TimeNow 125ns ± 0% 125ns ± 0% ~ (all equal) Gains are more dramatic in virtualized environments, presumably due to the overhead of virtualizing the syscall. Fixes #22190 Change-Id: I2f83ce60cb1b8b310c9ced0706bb463c1b3aedf8 Reviewed-on: https://go-review.googlesource.com/69390 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-11runtime: make it possible to exit Go-created threadsAustin Clements
Currently, threads created by the runtime exist until the whole program exits. For #14592 and #20395, we want to be able to exit and clean up threads created by the runtime. This commit implements that mechanism. The main difficulty is how to clean up the g0 stack. In cgo mode and on Solaris and Windows where the OS manages thread stacks, we simply arrange to return from mstart and let the system clean up the thread. If the runtime allocated the g0 stack, then we use a new exitThread syscall wrapper that arranges to clear a flag in the M once the stack can safely be reaped and call the thread termination syscall. exitThread is based on the existing exit1 wrapper, which was always meant to terminate the calling thread. However, exit1 has never been used since it was introduced 9 years ago, so it was broken on several platforms. exitThread also has the additional complication of having to flag that the stack is unused, which requires some tricks on platforms that use the stack for syscalls. This still leaves the problem of how to reap the unused g0 stacks. For this, we move the M from allm to a new freem list as part of the M exiting. Later, allocm scans the freem list, finds Ms that are marked as done with their stack, removes these from the list and frees their g0 stacks. This also allows these Ms to be garbage collected. This CL does not yet use any of this functionality. Follow-up CLs will. Likewise, there are no new tests in this CL because we'll need follow-up functionality to test it. Change-Id: Ic851ee74227b6d39c6fc1219fc71b45d3004bc63 Reviewed-on: https://go-review.googlesource.com/46037 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-08-30runtime: add symbols for Linux syscall numbers on 386/amd64Chris Ball
Matches other architectures by using names for syscalls instead of numbers directly. Fixes #20499. Change-Id: I63d606b0b1fe6fb517fd994a7542a3f38d80dd54 Reviewed-on: https://go-review.googlesource.com/44213 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-08-18runtime: fix usleep by correctly setting nanoseconds parameter for pselect6pvoicu
Fixes #21518 Change-Id: Idd67e3f0410d0ce991b34dcc0c8f15e0d5c529c9 Reviewed-on: https://go-review.googlesource.com/56850 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-05-23runtime: use pselect6 for usleep on linux/386Austin Clements
Commit 4dcba023c6 replaced select with pselect6 on linux/amd64 and linux/arm, but it turns out the Android emulator uses linux/386. This makes the equivalent change there, too. Fixes #20409 more. Change-Id: If542d6ade06309aab8758d5f5f6edec201ca7670 Reviewed-on: https://go-review.googlesource.com/44011 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-04-21runtime: inform arena placement using sbrk(0)Austin Clements
On 32-bit architectures (or if we fail to map a 64-bit-style arena), we try to map the heap arena just above the end of the process image. While we can accept any address, using lower addresses is preferable because lower addresses cause us to map less of the heap bitmap. However, if a program is linked against C code that has global constructors, those constructors may call brk/sbrk to allocate memory (e.g., many C malloc implementations do this for small allocations). The brk also starts just above the process image, so this may adjust the brk past the beginning of where we want to put the heap arena. In this case, the kernel will pick a different address for the arena and it will usually be very high (at least, as these things go in a 32-bit address space). Fix this by consulting the current value of the brk and using this in addition to the end of the process image to compute the initial arena placement. This is implemented only on Linux currently, since we have no evidence that it's an issue on any other OSes. Fixes #19831. Change-Id: Id64b45d08d8c91e4f50d92d0339146250b04f2f8 Reviewed-on: https://go-review.googlesource.com/39810 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-02-21runtime: update assembly var names after monotonic time changesJosh Bleecher Snyder
Change-Id: I721045120a4df41462c02252e2e5e8529ae2d694 Reviewed-on: https://go-review.googlesource.com/37303 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-02-03time: record monotonic clock reading in time.Now, for more accurate comparisonsRuss Cox
See https://golang.org/design/12914-monotonic for details. Fixes #12914. Change-Id: I80edc2e6c012b4ace7161c84cf067d444381a009 Reviewed-on: https://go-review.googlesource.com/36255 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Caleb Spare <cespare@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-12-15runtime: preserve callee-saved C registers in sigtrampBryan C. Mills
This fixes Linux and the *BSD platforms on 386/amd64. A few OS/arch combinations were already saving registers and/or doing something that doesn't clearly resemble the SysV C ABI; those have been left alone. Fixes #18328. Change-Id: I6398f6c71020de108fc8b26ca5946f0ba0258667 Reviewed-on: https://go-review.googlesource.com/34501 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-01runtime: align stack pointer in sigfwdBryan C. Mills
sigfwd calls an arbitrary C signal handler function. The System V ABI for x86_64 (and the most recent revision of the ABI for i386) requires the stack to be 16-byte aligned. Fixes: #17641 Change-Id: I77f53d4a8c29c1b0fe8cfbcc8d5381c4e6f75a6b Reviewed-on: https://go-review.googlesource.com/32107 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-09-30runtime, syscall: use FP instead of SP for parametersMatthew Dempsky
Consistently access function parameters using the FP pseudo-register instead of SP (e.g., x+0(FP) instead of x+4(SP) or x+8(SP), depending on register size). Two reasons: 1) doc/asm says the SP pseudo-register should use negative offsets in the range [-framesize, 0), and 2) cmd/vet only validates parameter offsets when indexed from the FP pseudo-register. No binary changes to the compiled object files for any of the affected package/OS/arch combinations. Change-Id: I0efc6079bc7519fcea588c114ec6a39b245d68b0 Reviewed-on: https://go-review.googlesource.com/30085 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-09-24runtime: unify some signal handling functionsIan Lance Taylor
Unify the OS-specific versions of msigsave, msigrestore, sigblock, updatesigmask, and unblocksig into single versions in signal_unix.go. To do this, make sigprocmask work the same way on all systems, which required adding a definition of sigprocmask for linux and openbsd. Also add a single OS-specific function sigmaskToSigset. Change-Id: I7cbf75131dddb57eeefe648ef845b0791404f785 Reviewed-on: https://go-review.googlesource.com/29689 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org>
2016-08-25all: fix assembly vet issuesJosh Bleecher Snyder
Add missing function prototypes. Fix function prototypes. Use FP references instead of SP references. Fix variable names. Update comments. Clean up whitespace. (Not for vet.) All fairly minor fixes to make vet happy. Updates #11041 Change-Id: Ifab2cdf235ff61cdc226ab1d84b8467b5ac9446c Reviewed-on: https://go-review.googlesource.com/27713 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-04-01runtime: support symbolic backtrace of C code in a cgo crashIan Lance Taylor
The new function runtime.SetCgoTraceback may be used to register stack traceback and symbolizer functions, written in C, to do a stack traceback from cgo code. There is a sample implementation of runtime.SetCgoSymbolizer at github.com/ianlancetaylor/cgosymbolizer. Just importing that package is sufficient to get symbolic C backtraces. Currently only supported on linux/amd64. Change-Id: If96ee2eb41c6c7379d407b9561b87557bfe47341 Reviewed-on: https://go-review.googlesource.com/17761 Reviewed-by: Austin Clements <austin@google.com>
2016-03-28runtime: use set_thread_area instead of modify_ldt on linux/386Shinji Tanaka
linux/386 depends on modify_ldt system call, but recent Linux kernels can disable this system call. Any Go programs built as linux/386 crash with the message 'Trace/breakpoint trap'. The kernel config CONFIG_MODIFY_LDT_SYSCALL, which control enable/disable modify_ldt, is disabled on Amazon Linux 2016.03. This fixes this problem by using set_thread_area instead of modify_ldt on linux/386. Fixes #14795. Change-Id: I0cc5139e40e9e5591945164156a77b6bdff2c7f1 Reviewed-on: https://go-review.googlesource.com/21190 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Minux Ma <minux@golang.org>
2016-03-02all: single space after period.Brad Fitzpatrick
The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-24runtime, syscall: switch linux/386 to use int 0x80Shenghou Ma
Like bionic, musl also doesn't provide vsyscall helper in %gs:0x10, and as int $0x80 is as fast as calling %gs:0x10, just use int $0x80 always. Because we're no longer using vsyscall in VDSO, get rid of VDSO code for linux/386 too. Fixes #14476. Change-Id: I00ec8652060700e0a3c9b524bfe3c16a810263f6 Reviewed-on: https://go-review.googlesource.com/19833 Run-TryBot: Minux Ma <minux@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-11-18runtime: handle volatility of CX when dynamically linking on linux/386Michael Hudson-Doyle
Mostly by avoiding CX entirely, sometimes by reloading it. I also vetted the assembly in other packages, it's all fine. Change-Id: I50059669aaaa04efa303cf22ac228f9d14d83db0 Reviewed-on: https://go-review.googlesource.com/16386 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-11-17runtime, syscall: use int $0x80 to invoke syscalls on android/386Michael Hudson-Doyle
golang.org/cl/16796 broke android/386 by assuming behaviour specific to glibc's dynamic linker. Copy bionic by using int $0x80 to invoke syscalls on android/386 as the old alternative (CALL *runtime_vdso(SB)) cannot be compiled without text relocations, which we want to get rid of on android. Also remove "CALL *runtime_vdso(SB)" variant from the syscall package. Change-Id: I6c01849f8dcbd073d000ddc8f13948a836b8b261 Reviewed-on: https://go-review.googlesource.com/16996 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2015-11-15cmd/go, runtime: always use position-independent code to invoke vsyscall ↵Michael Hudson-Doyle
helper on linux/386 golang.org/cl/16346 changed the runtime on linux/386 to invoke the vsyscall helper via a PIC sequence (CALL 0x10(GS)) when dynamically linking. But it's actually quite easy to make that code sequence work all the time, so do that, and remove the ugly machinery that passed the buildmode from the go tool to the assembly. This means enlarging m.tls so that we can safely access 0x10(GS) (GS is set to &m.tls + 4, so 0x10(GS) accesses m_tls[5]). Change-Id: I1345c34029b149cb5f25320bf19a3cdd73a056fa Reviewed-on: https://go-review.googlesource.com/16796 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-11-12runtime: update newosproc asm to access m.id directlyMatthew Dempsky
darwin/386, freebsd/386, and linux/386 use a setldt system call to setup each M's thread-local storage area, and they need access to the M's id for this. The current code copies m.id into m.tls[0] (and this logic has been cargo culted to OSes like NetBSD and OpenBSD, which don't even need m.id to configure TLS), and then the 386 assembly loads m.tls[0]... but since the assembly code already has a pointer to the M, it might as well just load m.id directly. Change-Id: I1a7278f1ec8ebda8d1de3aa3a61993070e3a8cdf Reviewed-on: https://go-review.googlesource.com/16881 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 14:41:58 +0000