aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/internal
AgeCommit message (Collapse)Author
2021-04-12runtime: using wyhash for memhashFallback on 64bit platformMeng Zhuo
wyhash is a general hash function that: 1. About 8-70% faster that internal maphash 2. Passed Smhasher, BigCrush and PractRand tests name old time/op new time/op delta Hash5 28.9ns ± 0% 30.0ns ± 0% +3.77% (p=0.000 n=9+10) Hash16 32.4ns ± 0% 30.2ns ± 0% -6.74% (p=0.000 n=10+8) Hash64 52.4ns ± 0% 43.4ns ± 0% -17.20% (p=0.000 n=9+10) Hash1024 415ns ± 0% 258ns ± 2% -37.89% (p=0.000 n=10+10) Hash65536 24.9µs ± 0% 14.6µs ± 0% -41.22% (p=0.000 n=9+9) HashStringSpeed 50.2ns ± 4% 47.8ns ± 4% -4.88% (p=0.000 n=10+10) HashBytesSpeed 90.1ns ± 7% 78.3ns ± 4% -13.06% (p=0.000 n=10+10) HashInt32Speed 33.3ns ± 6% 33.6ns ± 4% ~ (p=0.071 n=10+10) HashInt64Speed 32.7ns ± 3% 34.0ns ± 3% +4.05% (p=0.000 n=9+10) HashStringArraySpeed 131ns ± 2% 117ns ± 5% -10.32% (p=0.000 n=9+10) FastrandHashiter 72.2ns ± 1% 75.7ns ±10% +4.87% (p=0.019 n=8+10) name old speed new speed delta Hash5 173MB/s ± 0% 167MB/s ± 0% -3.63% (p=0.000 n=9+10) Hash16 494MB/s ± 0% 530MB/s ± 0% +7.23% (p=0.000 n=10+8) Hash64 1.22GB/s ± 0% 1.48GB/s ± 0% +20.77% (p=0.000 n=9+10) Hash1024 2.47GB/s ± 0% 3.97GB/s ± 2% +61.01% (p=0.000 n=8+10) Hash65536 2.64GB/s ± 0% 4.48GB/s ± 0% +70.13% (p=0.000 n=9+9) Change-Id: I76af4e2bc1995a18149d11983ea8a149c132865e Reviewed-on: https://go-review.googlesource.com/c/go/+/279612 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2021-04-08runtime: drop haveexperiment, sys.GOEXPERIMENTAustin Clements
We have ways to statically access experiments now, so we don't need a relatively clunky string-parsing dynamic way to do it. Change-Id: I5d75480916eef4bde2c30d5fe30593180da77ff2 Reviewed-on: https://go-review.googlesource.com/c/go/+/307815 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-04-05runtime: extend internal atomics to comply with sync/atomicRuslan Andreev
The CV add changes according to TODO in Go source-code. Internal atomic set does not comply with sync/atomic library and has shortage operations for signed integers. This patch extend internal atomic set by Int32 and Int64 operations. It's implemented new aliases and asm versions of operations. As a result Cas64 was replaced by Casint64 in findRunnableGCWorker without type casting. Another purpose is unified structure of internal atomics' source code. Before, assembly impementations for different archs were in different files. For example, filename for AMD64 was asm_amd64.s, but filename for RISC-V was atomic_riscv64.s. Some arches have both files without any meaning. So, assembly files were merged and renamed to atomic_{$ARCH}.s filenames. Change-Id: I29a05a7cbf5f4a9cc146e8315536c038af545677 Reviewed-on: https://go-review.googlesource.com/c/go/+/289152 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-03-11cmd: move GOEXPERIMENT knob from make.bash to cmd/goMatthew Dempsky
This CL changes GOEXPERIMENT to act like other GO[CONFIG] environment variables. Namely, that it can be set at make.bash time to provide a default value used by the toolchain, but then can be manually set when running either cmd/go or the individual tools (compiler, assembler, linker). For example, it's now possible to test rsc.io/tmp/fieldtrack by simply running: GOEXPERIMENT=fieldtrack go test -gcflags=-l rsc.io/tmp/fieldtrack \ -ldflags=-k=rsc.io/tmp/fieldtrack.tracked without needing to re-run make.bash. (-gcflags=-l is needed because the compiler's inlining abilities have improved, so calling a function with a for loop is no longer sufficient to suppress inlining.) Fixes #42681. Change-Id: I2cf8995d5d0d05f6785a2ee1d3b54b2cfb3331ca Reviewed-on: https://go-review.googlesource.com/c/go/+/300991 Trust: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-02-20all: go fmt std cmd (but revert vendor)Russ Cox
Make all our package sources use Go 1.17 gofmt format (adding //go:build lines). Part of //go:build change (#41184). See https://golang.org/design/draft-gobuild Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/294430 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-02-19runtime: move sys.DefaultGoroot to runtime.defaultGOROOTRuss Cox
The default GOROOT has nothing to do with system details. Move it next to its one use in package runtime. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: I1a601fad6335336b4616b834bb21bd8437ee1313 Reviewed-on: https://go-review.googlesource.com/c/go/+/288796 Trust: Russ Cox <rsc@golang.org> Trust: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
2021-02-19runtime: document, clean up internal/sysRuss Cox
Document what the values in internal/sys mean. Remove various special cases for arm64 in the code using StackAlign. Delete Uintreg - it was for GOARCH=amd64p32, which was specific to GOOS=nacl and has been retired. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: I40e8fa07b4e192298b6536b98a72a751951a4383 Reviewed-on: https://go-review.googlesource.com/c/go/+/288795 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-05cmd/compile: improve atomic swap intrinsics on arm64Jonathan Swinney
ARMv8.1 has added new instructions for atomic memory operations. This change builds on the previous change which added support for atomic add, 0a7ac93c27c9ade79fe0f66ae0bb81484c241ae5, to include similar support for atomic-compare-and-swap, atomic-swap, atomic-or, and atomic-and intrinsics. Since the new instructions are not guaranteed to be present, we guard their usages with a branch on a CPU feature. Peformance on an ARMv8.1 machine: name old time/op new time/op delta CompareAndSwap-16 37.9ns ±16% 24.1ns ± 4% -36.44% (p=0.000 n=10+9) CompareAndSwap64-16 38.6ns ±15% 24.1ns ± 3% -37.47% (p=0.000 n=10+10) name old time/op new time/op delta Swap-16 46.9ns ±32% 12.5ns ± 6% -73.40% (p=0.000 n=10+10) Swap64-16 53.4ns ± 1% 12.5ns ± 6% -76.56% (p=0.000 n=10+10) name old time/op new time/op delta Or8-16 8.81ns ± 0% 5.61ns ± 0% -36.32% (p=0.000 n=10+10) Or-16 7.21ns ± 0% 5.61ns ± 0% -22.19% (p=0.000 n=10+10) Or8Parallel-16 59.8ns ± 3% 12.5ns ± 2% -79.10% (p=0.000 n=10+10) OrParallel-16 51.7ns ± 3% 12.5ns ± 2% -75.84% (p=0.000 n=10+10) name old time/op new time/op delta And8-16 8.81ns ± 0% 5.61ns ± 0% -36.32% (p=0.000 n=10+10) And-16 7.21ns ± 0% 5.61ns ± 0% -22.19% (p=0.000 n=10+10) And8Parallel-16 59.1ns ± 6% 12.8ns ± 3% -78.33% (p=0.000 n=10+10) AndParallel-16 51.4ns ± 7% 12.8ns ± 3% -75.03% (p=0.000 n=10+10) Performance on an ARMv8.0 machine (no atomics instructions): name old time/op new time/op delta CompareAndSwap-16 61.3ns ± 0% 62.4ns ± 0% +1.70% (p=0.000 n=8+9) CompareAndSwap64-16 62.0ns ± 3% 61.3ns ± 2% ~ (p=0.093 n=10+10) name old time/op new time/op delta Swap-16 127ns ± 2% 131ns ± 2% +2.91% (p=0.001 n=10+10) Swap64-16 128ns ± 1% 131ns ± 2% +2.43% (p=0.001 n=10+10) name old time/op new time/op delta Or8-16 14.9ns ± 0% 15.3ns ± 0% +2.68% (p=0.000 n=10+10) Or-16 11.8ns ± 0% 12.3ns ± 0% +4.24% (p=0.000 n=10+10) Or8Parallel-16 137ns ± 1% 144ns ± 1% +4.97% (p=0.000 n=10+10) OrParallel-16 128ns ± 1% 136ns ± 1% +6.34% (p=0.000 n=10+10) name old time/op new time/op delta And8-16 14.9ns ± 0% 15.3ns ± 0% +2.68% (p=0.000 n=10+10) And-16 11.8ns ± 0% 12.3ns ± 0% +4.24% (p=0.000 n=10+10) And8Parallel-16 134ns ± 2% 141ns ± 1% +5.29% (p=0.000 n=10+10) AndParallel-16 125ns ± 2% 134ns ± 1% +7.10% (p=0.000 n=10+10) Fixes #39304 Change-Id: Idaca68701d4751650be6b4bedca3d57f51571712 Reviewed-on: https://go-review.googlesource.com/c/go/+/234217 Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Trust: fannie zhang <Fannie.Zhang@arm.com>
2020-10-23runtime/internal/atomic: add 32-bit And/OrMichael Pratt
These will be used in a following CL to perform larger bit clear and bit set than And8/Or8. Change-Id: I60f7b1099e29b69eb64add77564faee862880a8d Reviewed-on: https://go-review.googlesource.com/c/go/+/260977 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Trust: Michael Pratt <mpratt@google.com>
2020-10-21cmd/compiler,cmd/go,sync: add internal {LoadAcq,StoreRel}64 on ppc64Paul E. Murphy
Add an internal atomic intrinsic for load with acquire semantics (extending LoadAcq to 64b) and add LoadAcquintptr for internal use within the sync package. For other arches, this remaps to the appropriate atomic.Load{,64} intrinsic which should not alter code generation. Similarly, add StoreRel{uintptr,64} for consistency, and inline. Finally, add an exception to allow sync to directly use the runtime/internal/atomic package which avoids more convoluted workarounds (contributed by Lynn Boger). In an extreme example, sync.(*Pool).pin consumes 20% of wall time during fmt tests. This is reduced to 5% on ppc64le/power9. From the fmt benchmarks on ppc64le: name old time/op new time/op delta SprintfPadding 468ns ± 0% 451ns ± 0% -3.63% SprintfEmpty 73.3ns ± 0% 51.9ns ± 0% -29.20% SprintfString 135ns ± 0% 122ns ± 0% -9.63% SprintfTruncateString 232ns ± 0% 214ns ± 0% -7.76% SprintfTruncateBytes 216ns ± 0% 202ns ± 0% -6.48% SprintfSlowParsingPath 162ns ± 0% 142ns ± 0% -12.35% SprintfQuoteString 1.00µs ± 0% 0.99µs ± 0% -1.39% SprintfInt 117ns ± 0% 104ns ± 0% -11.11% SprintfIntInt 190ns ± 0% 175ns ± 0% -7.89% SprintfPrefixedInt 232ns ± 0% 212ns ± 0% -8.62% SprintfFloat 270ns ± 0% 255ns ± 0% -5.56% SprintfComplex 1.01µs ± 0% 0.99µs ± 0% -1.68% SprintfBoolean 127ns ± 0% 111ns ± 0% -12.60% SprintfHexString 220ns ± 0% 198ns ± 0% -10.00% SprintfHexBytes 261ns ± 0% 252ns ± 0% -3.45% SprintfBytes 600ns ± 0% 590ns ± 0% -1.67% SprintfStringer 684ns ± 0% 658ns ± 0% -3.80% SprintfStructure 2.57µs ± 0% 2.57µs ± 0% -0.12% ManyArgs 669ns ± 0% 646ns ± 0% -3.44% FprintInt 140ns ± 0% 136ns ± 0% -2.86% FprintfBytes 184ns ± 0% 181ns ± 0% -1.63% FprintIntNoAlloc 140ns ± 0% 136ns ± 0% -2.86% ScanInts 929µs ± 0% 921µs ± 0% -0.79% ScanRecursiveInt 122ms ± 0% 121ms ± 0% -0.11% ScanRecursiveIntReaderWrapper 122ms ± 0% 122ms ± 0% -0.18% Change-Id: I4d66780261b57b06ef600229e475462e7313f0d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/253748 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Keith Randall <khr@golang.org> Trust: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-16runtime/internal/atomic: panic nicely on unaligned 64-bit atomicsAustin Clements
On 386 and arm, unaligned 64-bit atomics aren't safe, so we check for this and panic. Currently, we panic by dereferencing nil, which may be expedient but is pretty user-hostile since it gives no hint of what the actual problem was. This CL replaces this with an actual panic. The only subtlety here is now the atomic assembly implementations are calling back into Go, so they have to play nicely with stack maps and stack scanning. On 386, this just requires declaring NO_LOCAL_POINTERS. On arm, this is somewhat more complicated: first, we have to move the alignment check into the functions that have Go signatures. Then we have to support both the tail call from these functions to the underlying implementation (which requires that they have no frame) and the call into Go to panic (which requires that they have a frame). We resolve this by forcing them to have no frame and setting up the frame manually just before the panic call. Change-Id: I19f1e860045df64088013db37a18acea47342c69 Reviewed-on: https://go-review.googlesource.com/c/go/+/262778 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-10-16runtime/internal/atomic: drop package prefixesAustin Clements
This drops package prefixes from the assembly code on 386 and arm. In addition to just being nicer, this allows the assembler to automatically pick up the argument stack map from the Go signatures of these functions. This doesn't matter right now because these functions never call back out to Go, but prepares us for the next CL. Change-Id: I90fed7d4dd63ad49274529c62804211b6390e2e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/262777 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-09-23all: add GOOS=iosCherry Zhang
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin" build tag, like GOOS=android matches "linux" and GOOS=illumos matches "solaris". Only ios/arm64 is supported (ios/amd64 is not). GOOS=ios and GOOS=darwin remain essentially the same at this point. They will diverge at later time, to differentiate macOS and iOS. Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"), except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"), it remains GOOS=="darwin". Updates #38485. Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c Reviewed-on: https://go-review.googlesource.com/c/go/+/254740 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-08-24runtime: add test for StorepNoWB param leakingCuong Manh Le
CL 249962 added wasm StorepNoWB implementation in assembly, it's now like all other architectures. This CL adds a general test that the second param of StorepNoWB must be force to escape. Fixes #40975 Change-Id: I1eccc7e50a3ec742a1912d65f25b15f9f5ad9241 Reviewed-on: https://go-review.googlesource.com/c/go/+/249761 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-08-23runtime: implement StorepNoWB for wasm in assemblyKeith Randall
The second argument of StorepNoWB must be forced to escape. The current Go code does not explicitly enforce that property. By implementing in assembly, and not using go:noescape, we force the issue. Test is in CL 249761. Issue #40975. This CL is needed for CL 249917, which changes how go:notinheap works and breaks the previous StorepNoWB wasm code. I checked for other possible errors like this. This is the only go:notinheap that isn't in the runtime itself. Change-Id: I43400a806662655727c4a3baa8902b63bdc9fa57 Reviewed-on: https://go-review.googlesource.com/c/go/+/249962 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-04-03runtime: eliminate redundant load in Xchg and Xadd on arm64Xiangdong Ji
Loading arguments of Xchg(64) and Xadd(64) functions to registers could be done only once. Change-Id: Iaf0a695ec9c6a221dfa755855edb68c476978a5b Reviewed-on: https://go-review.googlesource.com/c/go/+/227001 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-15cmd/asm,cmd/internal/obj/riscv: add atomic memory operation instructionsJoel Sing
Use instructions in place of currently used defines. Updates #36765 Change-Id: I00bb59e77b1aace549d7857cc9721ba2cb4ac6ca Reviewed-on: https://go-review.googlesource.com/c/go/+/220541 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-15cmd/asm,cmd/internal/obj/riscv: add LR/SC instructionsJoel Sing
Add support for Load-Reserved (LR) and Store-Conditional (SC) instructions. Use instructions in place of currently used defines. Updates #36765 Change-Id: I77e660639802293ece40cfde4865ac237e3308d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/220540 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-15cmd/internal/obj/riscv: add FENCE instructionJoel Sing
Also remove #define's that were previously in use. Updates #36765 Change-Id: I90b6a8629c78f549012f3f6c5f3b325336182712 Reviewed-on: https://go-review.googlesource.com/c/go/+/220539 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-19runtime: add support for linux/riscv64Joel Sing
Based on riscv-go port. Updates #27532 Change-Id: If522807a382130be3c8d40f4b4c1131d1de7c9e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/204632 Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-11cmd/compile, runtime: intrinsify atomic And8 and Or8 on s390xMichael Munday
Intrinsify these functions to match other platforms. Update the sequence of instructions used in the assembly implementations to match the intrinsics. Also, add a micro benchmark so we can more easily measure the performance of these two functions: name old time/op new time/op delta And8-8 5.33ns ± 7% 2.55ns ± 8% -52.12% (p=0.000 n=20+20) And8Parallel-8 7.39ns ± 5% 3.74ns ± 4% -49.34% (p=0.000 n=20+20) Or8-8 4.84ns ±15% 2.64ns ±11% -45.50% (p=0.000 n=20+20) Or8Parallel-8 7.27ns ± 3% 3.84ns ± 4% -47.10% (p=0.000 n=19+20) By using a 'rotate then xor selected bits' instruction combined with either a 'load and and' or a 'load and or' instruction we can implement And8 and Or8 with far fewer instructions. Replacing 'compare and swap' with atomic instructions may also improve performance when there is contention. Change-Id: I28bb8032052b73ae8ccdf6e4c612d2877085fa01 Reviewed-on: https://go-review.googlesource.com/c/go/+/204277 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-08runtime: copy some functions from math/bits to runtime/internal/sysDavid Chase
CL 201765 activated calls from the runtime to functions in math/bits. When coverage and race detection were simultaneously enabled, this caused a crash when the covered+race-checked code in math/bits was called from the runtime before there was even a P. PS Win for gdlv in helping sort this out. TODO - next CL intrinsifies the new functions in runtime/internal/sys TODO/Would-be-nice - Ctz64 and TrailingZeros64 are the same function; 386.s is intrinsified; clean all that up. Fixes #35461. Updates #35112. Change-Id: I750a54dba493130ad3e68a06530ede7687d41e1d Reviewed-on: https://go-review.googlesource.com/c/go/+/206199 Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-01[dev.link] all: merge branch 'master' into dev.linkThan McIntosh
Fixed a couple of minor conflicts in lib.go and deadcode.go relating to debug logging. Change-Id: I58335fc42ab1f1f3409fd8354da4f26419e8fb22
2019-10-31runtime/internal/atomic: add tests for And8 and Or8Michael Munday
Add some simple unit tests for these atomic operations. These can't catch all the bugs that are possible with these operations but at least they provide some coverage. Change-Id: I94b9f451fcc9fecdb2a1448c5357b019563ad275 Reviewed-on: https://go-review.googlesource.com/c/go/+/204317 Run-TryBot: Michael Munday <mike.munday@ibm.com> Reviewed-by: Austin Clements <austin@google.com>
2019-10-29runtime/internal/atomic: add Store8Austin Clements
We already have Load8, And8, and Or8. For #10958, #24543, but makes sense on its own. Change-Id: I478529fc643edc57efdeccaae413c99edd19b2eb Reviewed-on: https://go-review.googlesource.com/c/go/+/203283 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-21[dev.link] cmd/link: do not put static symbols into name lookup tableCherry Zhang
Since the previous CL, we will not reference static symbols by name. Therefore no need to put them into the name lookup table. On Linux/ARM, in runtime/internal/atomic/sys_linux_arm.s, the kernelcas function has a definition and a reference written in two different forms, one with package prefix, one without. This way, the assembler cannot know they are the same symbol, only the linker knows. This is quite unusual, unify the names to so the assembler can resolve it to index. Change-Id: Ie7223097be6a3b65f3fa43ed4575da9972ef5b69 Reviewed-on: https://go-review.googlesource.com/c/go/+/201998 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-10all: remove nacl (part 3, more amd64p32)Brad Fitzpatrick
Part 1: CL 199499 (GOOS nacl) Part 2: CL 200077 (amd64p32 files, toolchain) Part 3: stuff that arguably should've been part of Part 2, but I forgot one of my grep patterns when splitting the original CL up into two parts. This one might also have interesting stuff to resurrect for any future x32 ABI support. Updates #30439 Change-Id: I2b4143374a253a003666f3c69e776b7e456bdb9c Reviewed-on: https://go-review.googlesource.com/c/go/+/200318 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-09all: remove the nacl port (part 2, amd64p32 + toolchain)Brad Fitzpatrick
This is part two if the nacl removal. Part 1 was CL 199499. This CL removes amd64p32 support, which might be useful in the future if we implement the x32 ABI. It also removes the nacl bits in the toolchain, and some remaining nacl bits. Updates #30439 Change-Id: I2475d5bb066d1b474e00e40d95b520e7c2e286e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/200077 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-09all: remove the nacl port (part 1)Brad Fitzpatrick
You were a useful port and you've served your purpose. Thanks for all the play. A subsequent CL will remove amd64p32 (including assembly files and toolchain bits) and remaining bits. The amd64p32 removal will be separated into its own CL in case we want to support the Linux x32 ABI in the future and want our old amd64p32 support as a starting point. Updates #30439 Change-Id: Ia3a0c7d49804adc87bf52a4dea7e3d3007f2b1cd Reviewed-on: https://go-review.googlesource.com/c/go/+/199499 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-06-06runtime/internal/atomic: remove erroneous ABI wrappersAustin Clements
CL 179862 introduced go:linkname directives to create ABI wrappers for Store and Store64 on s390x, but a concurrent change (CL 180439) replaced the Go definitions of these functions with assembly definitions. This resulted in conflicting definitions for the ABI0 symbols, which led to a bootstrap linking failure. Fix this by removing the now-incorrect go:linkname directives for Store and Store64. This should fix the linux-s390x builders. Updates #31230. Change-Id: I8de8c03c23412fc217d428c0018cc56eb2f9996f Reviewed-on: https://go-review.googlesource.com/c/go/+/181078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-06runtime/internal/atomic: export more ABI0 wrappersAustin Clements
Somehow I missed these two functions in CL 179863. This should fix the linux-arm builders. Updates #31230. Change-Id: I3f8bef3fac331b505a55c0850b0fbc799b7c06c5 Reviewed-on: https://go-review.googlesource.com/c/go/+/181077 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-06-06runtime: mark all Go symbols called from assembly in other packagesAustin Clements
This marks all Go symbols called from assembly in other packages with "go:linkname" directives to ensure they get ABI wrappers. Now that we have this go:linkname convention, this also removes the abi0Syms definition in the runtime, which was used to give morestackc an ABI0 wrapper. Instead, we now just mark morestackc with a go:linkname directive. This was tested with buildall.bash in the default configuration, with -race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't test cgo on non-Linux configurations, I manually grepped for runtime symbols in runtime/cgo. Updates #31230. Change-Id: I6c8aa56be2ca6802dfa2bf159e49c411b9071bf1 Reviewed-on: https://go-review.googlesource.com/c/go/+/179862 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-06-06cmd/compile, runtime: make atomic loads/stores sequentially consistent on s390xMichael Munday
The z/Architecture does not guarantee that a load following a store will not be reordered with that store, unless they access the same address. Therefore if we want to ensure the sequential consistency of atomic loads and stores we need to perform serialization operations after atomic stores. We do not need to serialize in the runtime when using StoreRel[ease] and LoadAcq[uire]. The z/Architecture already provides sufficient ordering guarantees for these operations. name old time/op new time/op delta AtomicLoad64-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal) AtomicStore64-16 0.51ns ± 0% 0.60ns ± 9% +16.47% (p=0.000 n=17+20) AtomicLoad-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal) AtomicStore-16 0.51ns ± 0% 0.60ns ± 9% +16.50% (p=0.000 n=18+20) Fixes #32428. Change-Id: I88d19a4010c46070e4fff4b41587efe4c628d4d9 Reviewed-on: https://go-review.googlesource.com/c/go/+/180439 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-03cmd/compile,runtime/internal/atomic: add Load8Austin Clements
Change-Id: Id52a5730cf9207ee7ccebac4ef12791dc5720e7c Reviewed-on: https://go-review.googlesource.com/c/go/+/172283 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-05-03runtime: remove sys.HugePageSizeMichael Anthony Knyszek
sys.HugePageSize was superceded in the last commit by physHugePageSize which is determined dynamically by querying the operating system. For #30333. Change-Id: I827bfca8bdb347e989cead31564a8fffe56c66ff Reviewed-on: https://go-review.googlesource.com/c/go/+/173757 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-04-30all: add new GOOS=illumos, split out of GOOS=solarisJoshua M. Clulow
Like GOOS=android which implies the "linux" build tag, GOOS=illumos implies the "solaris" build tag. This lets the existing ecosystem of packages still work on illumos, but still permits packages to start differentiating between solaris and illumos. Fixes #20603 Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622 Reviewed-on: https://go-review.googlesource.com/c/go/+/174457 Run-TryBot: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-17runtime/internal/atomic: remove bad go:noescape annotations on LoadpMatthew Dempsky
The //go:noescape directive says that arguments don't leak at all, which is too aggressive of a claim for functions that return pointers derived from their parameters. Remove the directive for now. Long term fix will require a new directive that allows more fine-grained control over escape analysis information supplied for functions implemented in assembly. Also, update the BAD comments in the test cases for Loadp: we really want that *ptr leaks to the result parameter, not that *ptr leaks to the heap. Updates #31525. Change-Id: Ibfa61f2b70daa7ed3223056b57eeee777eef2e31 Reviewed-on: https://go-review.googlesource.com/c/go/+/172578 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-04-16runtime/internal/atomic: fix s390x's StorepNoWB implementationMatthew Dempsky
Same as CL 170323, but for s390x instead of wasm. Fixes #31495. Change-Id: Ie39f649f5e33690375a8bcb1bc3b92d912ca4398 Reviewed-on: https://go-review.googlesource.com/c/go/+/172417 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-02runtime/internal/atomic: fix wasm's StorepNoWB implementationMatthew Dempsky
Package unsafe's safety rules require that pointers converted to uintptr must be converted back to pointer-type before being stored into memory. In particular, storing a pointer into a non-pointer-typed expression does not guarantee the pointer stays valid, even if the expression refers to a pointer-typed variable. wasm's StorepNoWB implementation violates these rules by storing a pointer through a uintptr-typed expression. This happens to work today because esc.go is lenient in its implementation of package unsafe's rules, but my escape analysis rewrite follows them more rigorously, which causes val to be treated as a non-leaking parameter. This CL fixes the issue by using a *T-typed expression, where T is marked //go:notinheap so that the compiler still omits the write barrier as appropriate. Updates #23109. Change-Id: I49bc5474dbaa95729e5c93201493afe692591bc8 Reviewed-on: https://go-review.googlesource.com/c/go/+/170323 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-01-09cmd/dist, cmd/link, runtime: fix stack size when cross-compiling aix/ppc64Clément Chigot
This commit allows to cross-compiling aix/ppc64. The nosplit limit must twice as large as on others platforms because of AIX syscalls. The stack limit, especially stackGuardMultiplier, was set by cmd/dist during the bootstrap and doesn't depend on GOOS/GOARCH target. Fixes #29572 Change-Id: Id51e38885e1978d981aa9e14972eaec17294322e Reviewed-on: https://go-review.googlesource.com/c/157117 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-05runtime/internal/sys: regenerate zgoos_*.go filesTobias Klauser
zgoos_aix.go is missing GoosJs, the order of GoosAndroid and GoosAix is mixed up in all files and GoosHurd was added after CL 146023 introduced GOOS=hurd. Change-Id: I7e2f5a15645272e9020cfca86e44c364fc072a2b Reviewed-on: https://go-review.googlesource.com/c/147397 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-23cmd/compile, runtime: add new lightweight atomics for ppc64xCarlos Eduardo Seo
This change creates the infrastructure for new lightweight atomics primitives in runtime/internal/atomic: - LoadAcq, for load-acquire - StoreRel, for store-release - CasRel, for Compare-and-Swap-release and implements them for ppc64x. There is visible performance improvement in producer-consumer scenarios, like BenchmarkChanProdCons*: benchmark old ns/op new ns/op delta BenchmarkChanProdCons0-48 2034 2034 +0.00% BenchmarkChanProdCons10-48 1798 1608 -10.57% BenchmarkChanProdCons100-48 1596 1585 -0.69% BenchmarkChanProdConsWork0-48 2084 2046 -1.82% BenchmarkChanProdConsWork10-48 1829 1668 -8.80% BenchmarkChanProdConsWork100-48 1650 1650 +0.00% Fixes #21348 Change-Id: I1f6ce377e4a0fe4bd7f5f775e8036f50070ad8db Reviewed-on: https://go-review.googlesource.com/c/142277 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-10-15cmd/compile: add intrinsics for runtime/internal/math on 386 and amd64Martin Möhrmann
Add generic, 386 and amd64 specific ops and SSA rules for multiplication with overflow and branching based on overflow flags. Use these to intrinsify runtime/internal/math.MulUinptr. On amd64 mul, overflow := math.MulUintptr(a, b) if overflow { is lowered to two instructions: MULQ SI JO 0x10ee35c No codegen tests as codegen can not currently test unexported internal runtime functions. amd64: name old time/op new time/op delta MulUintptr/small 1.16ns ± 5% 0.88ns ± 6% -24.36% (p=0.000 n=19+20) MulUintptr/large 10.7ns ± 1% 1.1ns ± 1% -89.28% (p=0.000 n=17+19) Change-Id: If60739a86f820e5044d677276c21df90d3c7a86a Reviewed-on: https://go-review.googlesource.com/c/141820 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-10-15runtime/internal/math: add multiplication with overflow checkMartin Möhrmann
This CL adds a new internal math package for use by the runtime. The new package exports a MulUintptr function with uintptr arguments a and b and returns uintptr(a*b) and whether the full-width product x*y does overflow the uintptr value range (uintptr(x*y) != x*y). Uses of MulUinptr in the runtime and intrinsics for performance will be added in followup CLs. Updates #21588 Change-Id: Ia5a02eeabc955249118e4edf68c67d9fc0858058 Reviewed-on: https://go-review.googlesource.com/c/91755 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-09-28go/build, runtime/internal/sys: add GOOS=aixClément Chigot
This is the first commit of a series that will add AIX as an operating system target for ppc64 architecture. Updates #25893 Change-Id: I865b67a9c98277c11c1a56107be404ac5253277d Reviewed-on: https://go-review.googlesource.com/138115 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-08-24runtime: replace sys.CacheLineSize by corresponding internal/cpu const and varsMartin Möhrmann
sys here is runtime/internal/sys. Replace uses of sys.CacheLineSize for padding by cpu.CacheLinePad or cpu.CacheLinePadSize. Replace other uses of sys.CacheLineSize by cpu.CacheLineSize. Remove now unused sys.CacheLineSize. Updates #25203 Change-Id: I1daf410fe8f6c0493471c2ceccb9ca0a5a75ed8f Reviewed-on: https://go-review.googlesource.com/126601 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-06-21cmd/compile: improve atomic add intrinsics with ARMv8.1 new instructionWei Xiao
ARMv8.1 has added new instruction (LDADDAL) for atomic memory operations. This CL improves existing atomic add intrinsics with the new instruction. Since the new instruction is only guaranteed to be present after ARMv8.1, we guard its usage with a conditional on CPU feature. Performance result on ARMv8.1 machine: name old time/op new time/op delta Xadd-224 1.05µs ± 6% 0.02µs ± 4% -98.06% (p=0.000 n=10+8) Xadd64-224 1.05µs ± 3% 0.02µs ±13% -98.10% (p=0.000 n=9+10) [Geo mean] 1.05µs 0.02µs -98.08% Performance result on ARMv8.0 machine: name old time/op new time/op delta Xadd-46 538ns ± 1% 541ns ± 1% +0.62% (p=0.000 n=9+9) Xadd64-46 505ns ± 1% 508ns ± 0% +0.48% (p=0.003 n=9+8) [Geo mean] 521ns 524ns +0.55% Change-Id: If4b5d8d0e2d6f84fe1492a4f5de0789910ad0ee9 Reviewed-on: https://go-review.googlesource.com/81877 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-06-12runtime/internal/sys: rename Wasm to WASMRichard Musiol
This commit changes sys.Wasm to sys.WASM, as requested on https://groups.google.com/forum/#!topic/golang-dev/VquDxlhjPkg Change-Id: I30a208c34576a8bb49b9beb524203d71df8fdf1c Reviewed-on: https://go-review.googlesource.com/118395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 22:58:39 +0000