aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/internal/atomic
AgeCommit message (Collapse)Author
2020-10-16runtime/internal/atomic: drop package prefixesAustin Clements
This drops package prefixes from the assembly code on 386 and arm. In addition to just being nicer, this allows the assembler to automatically pick up the argument stack map from the Go signatures of these functions. This doesn't matter right now because these functions never call back out to Go, but prepares us for the next CL. Change-Id: I90fed7d4dd63ad49274529c62804211b6390e2e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/262777 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-08-24runtime: add test for StorepNoWB param leakingCuong Manh Le
CL 249962 added wasm StorepNoWB implementation in assembly, it's now like all other architectures. This CL adds a general test that the second param of StorepNoWB must be force to escape. Fixes #40975 Change-Id: I1eccc7e50a3ec742a1912d65f25b15f9f5ad9241 Reviewed-on: https://go-review.googlesource.com/c/go/+/249761 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-08-23runtime: implement StorepNoWB for wasm in assemblyKeith Randall
The second argument of StorepNoWB must be forced to escape. The current Go code does not explicitly enforce that property. By implementing in assembly, and not using go:noescape, we force the issue. Test is in CL 249761. Issue #40975. This CL is needed for CL 249917, which changes how go:notinheap works and breaks the previous StorepNoWB wasm code. I checked for other possible errors like this. This is the only go:notinheap that isn't in the runtime itself. Change-Id: I43400a806662655727c4a3baa8902b63bdc9fa57 Reviewed-on: https://go-review.googlesource.com/c/go/+/249962 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-04-03runtime: eliminate redundant load in Xchg and Xadd on arm64Xiangdong Ji
Loading arguments of Xchg(64) and Xadd(64) functions to registers could be done only once. Change-Id: Iaf0a695ec9c6a221dfa755855edb68c476978a5b Reviewed-on: https://go-review.googlesource.com/c/go/+/227001 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-15cmd/asm,cmd/internal/obj/riscv: add atomic memory operation instructionsJoel Sing
Use instructions in place of currently used defines. Updates #36765 Change-Id: I00bb59e77b1aace549d7857cc9721ba2cb4ac6ca Reviewed-on: https://go-review.googlesource.com/c/go/+/220541 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-15cmd/asm,cmd/internal/obj/riscv: add LR/SC instructionsJoel Sing
Add support for Load-Reserved (LR) and Store-Conditional (SC) instructions. Use instructions in place of currently used defines. Updates #36765 Change-Id: I77e660639802293ece40cfde4865ac237e3308d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/220540 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-15cmd/internal/obj/riscv: add FENCE instructionJoel Sing
Also remove #define's that were previously in use. Updates #36765 Change-Id: I90b6a8629c78f549012f3f6c5f3b325336182712 Reviewed-on: https://go-review.googlesource.com/c/go/+/220539 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-19runtime: add support for linux/riscv64Joel Sing
Based on riscv-go port. Updates #27532 Change-Id: If522807a382130be3c8d40f4b4c1131d1de7c9e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/204632 Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-11cmd/compile, runtime: intrinsify atomic And8 and Or8 on s390xMichael Munday
Intrinsify these functions to match other platforms. Update the sequence of instructions used in the assembly implementations to match the intrinsics. Also, add a micro benchmark so we can more easily measure the performance of these two functions: name old time/op new time/op delta And8-8 5.33ns ± 7% 2.55ns ± 8% -52.12% (p=0.000 n=20+20) And8Parallel-8 7.39ns ± 5% 3.74ns ± 4% -49.34% (p=0.000 n=20+20) Or8-8 4.84ns ±15% 2.64ns ±11% -45.50% (p=0.000 n=20+20) Or8Parallel-8 7.27ns ± 3% 3.84ns ± 4% -47.10% (p=0.000 n=19+20) By using a 'rotate then xor selected bits' instruction combined with either a 'load and and' or a 'load and or' instruction we can implement And8 and Or8 with far fewer instructions. Replacing 'compare and swap' with atomic instructions may also improve performance when there is contention. Change-Id: I28bb8032052b73ae8ccdf6e4c612d2877085fa01 Reviewed-on: https://go-review.googlesource.com/c/go/+/204277 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-01[dev.link] all: merge branch 'master' into dev.linkThan McIntosh
Fixed a couple of minor conflicts in lib.go and deadcode.go relating to debug logging. Change-Id: I58335fc42ab1f1f3409fd8354da4f26419e8fb22
2019-10-31runtime/internal/atomic: add tests for And8 and Or8Michael Munday
Add some simple unit tests for these atomic operations. These can't catch all the bugs that are possible with these operations but at least they provide some coverage. Change-Id: I94b9f451fcc9fecdb2a1448c5357b019563ad275 Reviewed-on: https://go-review.googlesource.com/c/go/+/204317 Run-TryBot: Michael Munday <mike.munday@ibm.com> Reviewed-by: Austin Clements <austin@google.com>
2019-10-29runtime/internal/atomic: add Store8Austin Clements
We already have Load8, And8, and Or8. For #10958, #24543, but makes sense on its own. Change-Id: I478529fc643edc57efdeccaae413c99edd19b2eb Reviewed-on: https://go-review.googlesource.com/c/go/+/203283 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-21[dev.link] cmd/link: do not put static symbols into name lookup tableCherry Zhang
Since the previous CL, we will not reference static symbols by name. Therefore no need to put them into the name lookup table. On Linux/ARM, in runtime/internal/atomic/sys_linux_arm.s, the kernelcas function has a definition and a reference written in two different forms, one with package prefix, one without. This way, the assembler cannot know they are the same symbol, only the linker knows. This is quite unusual, unify the names to so the assembler can resolve it to index. Change-Id: Ie7223097be6a3b65f3fa43ed4575da9972ef5b69 Reviewed-on: https://go-review.googlesource.com/c/go/+/201998 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-10all: remove nacl (part 3, more amd64p32)Brad Fitzpatrick
Part 1: CL 199499 (GOOS nacl) Part 2: CL 200077 (amd64p32 files, toolchain) Part 3: stuff that arguably should've been part of Part 2, but I forgot one of my grep patterns when splitting the original CL up into two parts. This one might also have interesting stuff to resurrect for any future x32 ABI support. Updates #30439 Change-Id: I2b4143374a253a003666f3c69e776b7e456bdb9c Reviewed-on: https://go-review.googlesource.com/c/go/+/200318 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-09all: remove the nacl port (part 2, amd64p32 + toolchain)Brad Fitzpatrick
This is part two if the nacl removal. Part 1 was CL 199499. This CL removes amd64p32 support, which might be useful in the future if we implement the x32 ABI. It also removes the nacl bits in the toolchain, and some remaining nacl bits. Updates #30439 Change-Id: I2475d5bb066d1b474e00e40d95b520e7c2e286e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/200077 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-06-06runtime/internal/atomic: remove erroneous ABI wrappersAustin Clements
CL 179862 introduced go:linkname directives to create ABI wrappers for Store and Store64 on s390x, but a concurrent change (CL 180439) replaced the Go definitions of these functions with assembly definitions. This resulted in conflicting definitions for the ABI0 symbols, which led to a bootstrap linking failure. Fix this by removing the now-incorrect go:linkname directives for Store and Store64. This should fix the linux-s390x builders. Updates #31230. Change-Id: I8de8c03c23412fc217d428c0018cc56eb2f9996f Reviewed-on: https://go-review.googlesource.com/c/go/+/181078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-06runtime/internal/atomic: export more ABI0 wrappersAustin Clements
Somehow I missed these two functions in CL 179863. This should fix the linux-arm builders. Updates #31230. Change-Id: I3f8bef3fac331b505a55c0850b0fbc799b7c06c5 Reviewed-on: https://go-review.googlesource.com/c/go/+/181077 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-06-06runtime: mark all Go symbols called from assembly in other packagesAustin Clements
This marks all Go symbols called from assembly in other packages with "go:linkname" directives to ensure they get ABI wrappers. Now that we have this go:linkname convention, this also removes the abi0Syms definition in the runtime, which was used to give morestackc an ABI0 wrapper. Instead, we now just mark morestackc with a go:linkname directive. This was tested with buildall.bash in the default configuration, with -race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't test cgo on non-Linux configurations, I manually grepped for runtime symbols in runtime/cgo. Updates #31230. Change-Id: I6c8aa56be2ca6802dfa2bf159e49c411b9071bf1 Reviewed-on: https://go-review.googlesource.com/c/go/+/179862 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-06-06cmd/compile, runtime: make atomic loads/stores sequentially consistent on s390xMichael Munday
The z/Architecture does not guarantee that a load following a store will not be reordered with that store, unless they access the same address. Therefore if we want to ensure the sequential consistency of atomic loads and stores we need to perform serialization operations after atomic stores. We do not need to serialize in the runtime when using StoreRel[ease] and LoadAcq[uire]. The z/Architecture already provides sufficient ordering guarantees for these operations. name old time/op new time/op delta AtomicLoad64-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal) AtomicStore64-16 0.51ns ± 0% 0.60ns ± 9% +16.47% (p=0.000 n=17+20) AtomicLoad-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal) AtomicStore-16 0.51ns ± 0% 0.60ns ± 9% +16.50% (p=0.000 n=18+20) Fixes #32428. Change-Id: I88d19a4010c46070e4fff4b41587efe4c628d4d9 Reviewed-on: https://go-review.googlesource.com/c/go/+/180439 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-03cmd/compile,runtime/internal/atomic: add Load8Austin Clements
Change-Id: Id52a5730cf9207ee7ccebac4ef12791dc5720e7c Reviewed-on: https://go-review.googlesource.com/c/go/+/172283 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-04-17runtime/internal/atomic: remove bad go:noescape annotations on LoadpMatthew Dempsky
The //go:noescape directive says that arguments don't leak at all, which is too aggressive of a claim for functions that return pointers derived from their parameters. Remove the directive for now. Long term fix will require a new directive that allows more fine-grained control over escape analysis information supplied for functions implemented in assembly. Also, update the BAD comments in the test cases for Loadp: we really want that *ptr leaks to the result parameter, not that *ptr leaks to the heap. Updates #31525. Change-Id: Ibfa61f2b70daa7ed3223056b57eeee777eef2e31 Reviewed-on: https://go-review.googlesource.com/c/go/+/172578 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-04-16runtime/internal/atomic: fix s390x's StorepNoWB implementationMatthew Dempsky
Same as CL 170323, but for s390x instead of wasm. Fixes #31495. Change-Id: Ie39f649f5e33690375a8bcb1bc3b92d912ca4398 Reviewed-on: https://go-review.googlesource.com/c/go/+/172417 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-02runtime/internal/atomic: fix wasm's StorepNoWB implementationMatthew Dempsky
Package unsafe's safety rules require that pointers converted to uintptr must be converted back to pointer-type before being stored into memory. In particular, storing a pointer into a non-pointer-typed expression does not guarantee the pointer stays valid, even if the expression refers to a pointer-typed variable. wasm's StorepNoWB implementation violates these rules by storing a pointer through a uintptr-typed expression. This happens to work today because esc.go is lenient in its implementation of package unsafe's rules, but my escape analysis rewrite follows them more rigorously, which causes val to be treated as a non-leaking parameter. This CL fixes the issue by using a *T-typed expression, where T is marked //go:notinheap so that the compiler still omits the write barrier as appropriate. Updates #23109. Change-Id: I49bc5474dbaa95729e5c93201493afe692591bc8 Reviewed-on: https://go-review.googlesource.com/c/go/+/170323 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-10-23cmd/compile, runtime: add new lightweight atomics for ppc64xCarlos Eduardo Seo
This change creates the infrastructure for new lightweight atomics primitives in runtime/internal/atomic: - LoadAcq, for load-acquire - StoreRel, for store-release - CasRel, for Compare-and-Swap-release and implements them for ppc64x. There is visible performance improvement in producer-consumer scenarios, like BenchmarkChanProdCons*: benchmark old ns/op new ns/op delta BenchmarkChanProdCons0-48 2034 2034 +0.00% BenchmarkChanProdCons10-48 1798 1608 -10.57% BenchmarkChanProdCons100-48 1596 1585 -0.69% BenchmarkChanProdConsWork0-48 2084 2046 -1.82% BenchmarkChanProdConsWork10-48 1829 1668 -8.80% BenchmarkChanProdConsWork100-48 1650 1650 +0.00% Fixes #21348 Change-Id: I1f6ce377e4a0fe4bd7f5f775e8036f50070ad8db Reviewed-on: https://go-review.googlesource.com/c/142277 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-08-24runtime: replace sys.CacheLineSize by corresponding internal/cpu const and varsMartin Möhrmann
sys here is runtime/internal/sys. Replace uses of sys.CacheLineSize for padding by cpu.CacheLinePad or cpu.CacheLinePadSize. Replace other uses of sys.CacheLineSize by cpu.CacheLineSize. Remove now unused sys.CacheLineSize. Updates #25203 Change-Id: I1daf410fe8f6c0493471c2ceccb9ca0a5a75ed8f Reviewed-on: https://go-review.googlesource.com/126601 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-06-21cmd/compile: improve atomic add intrinsics with ARMv8.1 new instructionWei Xiao
ARMv8.1 has added new instruction (LDADDAL) for atomic memory operations. This CL improves existing atomic add intrinsics with the new instruction. Since the new instruction is only guaranteed to be present after ARMv8.1, we guard its usage with a conditional on CPU feature. Performance result on ARMv8.1 machine: name old time/op new time/op delta Xadd-224 1.05µs ± 6% 0.02µs ± 4% -98.06% (p=0.000 n=10+8) Xadd64-224 1.05µs ± 3% 0.02µs ±13% -98.10% (p=0.000 n=9+10) [Geo mean] 1.05µs 0.02µs -98.08% Performance result on ARMv8.0 machine: name old time/op new time/op delta Xadd-46 538ns ± 1% 541ns ± 1% +0.62% (p=0.000 n=9+9) Xadd64-46 505ns ± 1% 508ns ± 0% +0.48% (p=0.003 n=9+8) [Geo mean] 521ns 524ns +0.55% Change-Id: If4b5d8d0e2d6f84fe1492a4f5de0789910ad0ee9 Reviewed-on: https://go-review.googlesource.com/81877 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-06-01all: update comment URLs from HTTP to HTTPS, where possibleTim Cooper
Each URL was manually verified to ensure it did not serve up incorrect content. Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df Reviewed-on: https://go-review.googlesource.com/115798 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-05-08cmd/compile: add some generic composite type optimizationsMichael Munday
Propagate values through some wide Zero/Move operations. Among other things this allows us to optimize some kinds of array initialization. For example, the following code no longer requires a temporary be allocated on the stack. Instead it writes the values directly into the return value. func f(i uint32) [4]uint32 { return [4]uint32{i, i+1, i+2, i+3} } The return value is unnecessarily cleared but removing that is probably a task for dead store analysis (I think it needs to be able to match multiple Store ops to wide Zero ops). In order to reliably remove stack variables that are rendered unnecessary by these new rules I've added a new generic version of the unread autos elimination pass. These rules are triggered more than 5000 times when building and testing the standard library. Updates #15925 (fixes for arrays of up to 4 elements). Updates #24386 (fixes for up to 4 kept elements). Updates #24416. compilebench results: name old time/op new time/op delta Template 353ms ± 5% 359ms ± 3% ~ (p=0.143 n=10+10) Unicode 219ms ± 1% 217ms ± 4% ~ (p=0.740 n=7+10) GoTypes 1.26s ± 1% 1.26s ± 2% ~ (p=0.549 n=9+10) Compiler 6.00s ± 1% 6.08s ± 1% +1.42% (p=0.000 n=9+8) SSA 15.3s ± 2% 15.6s ± 1% +2.43% (p=0.000 n=10+10) Flate 237ms ± 2% 240ms ± 2% +1.31% (p=0.015 n=10+10) GoParser 285ms ± 1% 285ms ± 1% ~ (p=0.878 n=8+8) Reflect 797ms ± 3% 807ms ± 2% ~ (p=0.065 n=9+10) Tar 334ms ± 0% 335ms ± 4% ~ (p=0.460 n=8+10) XML 419ms ± 0% 423ms ± 1% +0.91% (p=0.001 n=7+9) StdCmd 46.0s ± 0% 46.4s ± 0% +0.85% (p=0.000 n=9+9) name old user-time/op new user-time/op delta Template 337ms ± 3% 346ms ± 5% ~ (p=0.053 n=9+10) Unicode 205ms ±10% 205ms ± 8% ~ (p=1.000 n=10+10) GoTypes 1.22s ± 2% 1.21s ± 3% ~ (p=0.436 n=10+10) Compiler 5.85s ± 1% 5.93s ± 0% +1.46% (p=0.000 n=10+8) SSA 14.9s ± 1% 15.3s ± 1% +2.62% (p=0.000 n=10+10) Flate 229ms ± 4% 228ms ± 6% ~ (p=0.796 n=10+10) GoParser 271ms ± 3% 275ms ± 4% ~ (p=0.165 n=10+10) Reflect 779ms ± 5% 775ms ± 2% ~ (p=0.971 n=10+10) Tar 317ms ± 4% 319ms ± 5% ~ (p=0.853 n=10+10) XML 404ms ± 4% 409ms ± 5% ~ (p=0.436 n=10+10) name old alloc/op new alloc/op delta Template 34.9MB ± 0% 35.0MB ± 0% +0.26% (p=0.000 n=10+10) Unicode 29.3MB ± 0% 29.3MB ± 0% +0.02% (p=0.000 n=10+10) GoTypes 115MB ± 0% 115MB ± 0% +0.30% (p=0.000 n=10+10) Compiler 519MB ± 0% 521MB ± 0% +0.30% (p=0.000 n=10+10) SSA 1.55GB ± 0% 1.57GB ± 0% +1.34% (p=0.000 n=10+9) Flate 24.1MB ± 0% 24.2MB ± 0% +0.10% (p=0.000 n=10+10) GoParser 28.1MB ± 0% 28.1MB ± 0% +0.07% (p=0.000 n=10+10) Reflect 78.7MB ± 0% 78.7MB ± 0% +0.03% (p=0.000 n=8+10) Tar 34.4MB ± 0% 34.5MB ± 0% +0.12% (p=0.000 n=10+10) XML 43.2MB ± 0% 43.2MB ± 0% +0.13% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Template 330k ± 0% 330k ± 0% -0.01% (p=0.017 n=10+10) Unicode 337k ± 0% 337k ± 0% +0.01% (p=0.000 n=9+10) GoTypes 1.15M ± 0% 1.15M ± 0% +0.03% (p=0.000 n=10+10) Compiler 4.77M ± 0% 4.77M ± 0% +0.03% (p=0.000 n=9+10) SSA 12.5M ± 0% 12.6M ± 0% +1.16% (p=0.000 n=10+10) Flate 221k ± 0% 221k ± 0% +0.05% (p=0.000 n=9+10) GoParser 275k ± 0% 275k ± 0% +0.01% (p=0.014 n=10+9) Reflect 944k ± 0% 944k ± 0% -0.02% (p=0.000 n=10+10) Tar 324k ± 0% 323k ± 0% -0.12% (p=0.000 n=10+10) XML 384k ± 0% 384k ± 0% -0.01% (p=0.001 n=10+10) name old object-bytes new object-bytes delta Template 476kB ± 0% 476kB ± 0% -0.04% (p=0.000 n=10+10) Unicode 218kB ± 0% 218kB ± 0% ~ (all equal) GoTypes 1.58MB ± 0% 1.58MB ± 0% -0.04% (p=0.000 n=10+10) Compiler 6.25MB ± 0% 6.24MB ± 0% -0.09% (p=0.000 n=10+10) SSA 15.9MB ± 0% 16.1MB ± 0% +1.22% (p=0.000 n=10+10) Flate 304kB ± 0% 304kB ± 0% -0.13% (p=0.000 n=10+10) GoParser 370kB ± 0% 370kB ± 0% -0.00% (p=0.000 n=10+10) Reflect 1.27MB ± 0% 1.27MB ± 0% -0.12% (p=0.000 n=10+10) Tar 421kB ± 0% 419kB ± 0% -0.64% (p=0.000 n=10+10) XML 518kB ± 0% 517kB ± 0% -0.12% (p=0.000 n=10+10) name old export-bytes new export-bytes delta Template 16.7kB ± 0% 16.7kB ± 0% ~ (all equal) Unicode 6.52kB ± 0% 6.52kB ± 0% ~ (all equal) GoTypes 29.2kB ± 0% 29.2kB ± 0% ~ (all equal) Compiler 88.0kB ± 0% 88.0kB ± 0% ~ (all equal) SSA 109kB ± 0% 109kB ± 0% ~ (all equal) Flate 4.49kB ± 0% 4.49kB ± 0% ~ (all equal) GoParser 8.10kB ± 0% 8.10kB ± 0% ~ (all equal) Reflect 7.71kB ± 0% 7.71kB ± 0% ~ (all equal) Tar 9.15kB ± 0% 9.15kB ± 0% ~ (all equal) XML 12.3kB ± 0% 12.3kB ± 0% ~ (all equal) name old text-bytes new text-bytes delta HelloSize 676kB ± 0% 672kB ± 0% -0.59% (p=0.000 n=10+10) CmdGoSize 7.26MB ± 0% 7.24MB ± 0% -0.18% (p=0.000 n=10+10) name old data-bytes new data-bytes delta HelloSize 10.2kB ± 0% 10.2kB ± 0% ~ (all equal) CmdGoSize 248kB ± 0% 248kB ± 0% ~ (all equal) name old bss-bytes new bss-bytes delta HelloSize 125kB ± 0% 125kB ± 0% ~ (all equal) CmdGoSize 145kB ± 0% 145kB ± 0% ~ (all equal) name old exe-bytes new exe-bytes delta HelloSize 1.46MB ± 0% 1.45MB ± 0% -0.31% (p=0.000 n=10+10) CmdGoSize 14.7MB ± 0% 14.7MB ± 0% -0.17% (p=0.000 n=10+10) Change-Id: Ic72b0c189dd542f391e1c9ab88a76e9148dc4285 Reviewed-on: https://go-review.googlesource.com/106495 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-05-08runtime: add js/wasm architectureRichard Musiol
This commit adds the js/wasm architecture to the runtime package. Currently WebAssembly has no support for threads yet, see https://github.com/WebAssembly/design/issues/1073. Because of that, there is no preemption of goroutines and no sysmon goroutine. Design doc: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4 About WebAssembly assembly files: https://docs.google.com/document/d/1GRmy3rA4DiYtBlX-I1Jr_iHykbX8EixC3Mq0TCYqbKc Updates #18892 Change-Id: I7f12d21b5180500d55ae9fd2f7e926a1731db391 Reviewed-on: https://go-review.googlesource.com/103877 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-05-03runtime/internal/atomic: improve ARM atomicsCherry Zhang
This is a follow-up of CL 93637. There, when we redirect sync/atomic to runtime/internal/atomic, a few good implementations of ARM atomics were lost. This CL brings most of them back, with some improvements. - Change atomic Store to a plain store with memory barrier, as we already changed atomic Load to plain load with memory barrier. - Use native 64-bit atomics on ARMv7, jump to Go implementations on older machines. But drop the kernel helper. In particular, for Load64, just do loads, not using Cas on the address being load from, so it works also for read-only memory (since we have already fixed 32-bit Load). Change-Id: I725cd65cf945ae5200db81a35be3f251c9f7af14 Reviewed-on: https://go-review.googlesource.com/111315 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-05-03runtime: use native CAS and memory barrier on ARMv7Cherry Zhang
This gets us around the kernel helpers on ARMv7. It is slightly faster than using the kernel helper. name old time/op new time/op delta AtomicLoad-4 72.5ns ± 0% 69.5ns ± 0% -4.08% (p=0.000 n=9+9) AtomicStore-4 57.6ns ± 1% 54.4ns ± 0% -5.58% (p=0.000 n=10+9) [Geo mean] 64.6ns 61.5ns -4.83% If performance is really critical, we can even do compiler intrinsics on GOARM=7. Fixes #23792. Change-Id: I36497d880890b26bdf01e048b542bd5fd7b17d23 Reviewed-on: https://go-review.googlesource.com/94076 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-05-03sync/atomic: redirect many functions to runtime/internal/atomicCherry Zhang
The implementation of atomics are inherently tricky. It would be good to have them implemented in a single place, instead of multiple copies. Mostly a simple redirect. On 386, some functions in sync/atomic have better implementations, which are moved to runtime/internal/atomic. On ARM, some functions in sync/atomic have better implementations. They are dropped by this CL, but restored with an improved version in a follow-up CL. On linux/arm, 64-bit CAS kernel helper is dropped, as we're trying to move away from kernel helpers. Fixes #23778. Change-Id: Icb9e1039acc92adbb2a371c34baaf0b79551c3ea Reviewed-on: https://go-review.googlesource.com/93637 Reviewed-by: Austin Clements <austin@google.com>
2018-04-18runtime/internal/atomic: don't use Cas in atomic.Load on ARMYuval Pavel Zholkover
Instead issue a memory barrier on ARMv7 after reading the address. Fixes #23777 Change-Id: I7aff2ab0246af64b437ebe0b31d4b30d351890d8 Reviewed-on: https://go-review.googlesource.com/94275 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-03-27cmd/internal/obj/arm: add DMB instructionYuval Pavel Zholkover
Change-Id: Ib67a61d5b37af210ff15d60d72bd5238b9c2d0ca Reviewed-on: https://go-review.googlesource.com/94815 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-03-22cmd/compile/internal/ppc64, runtime internal/atomic, sync/atomic: implement ↵Carlos Eduardo Seo
faster atomics for ppc64x This change implements faster atomics for ppc64x based on the ISA 2.07B, Appendix B.2 recommendations, replacing SYNC/ISYNC by LWSYNC in some cases. Updates #21348 name old time/op new time/op delta Cond1-16 955ns 856ns -10.33% Cond2-16 2.38µs 2.03µs -14.59% Cond4-16 5.90µs 5.44µs -7.88% Cond8-16 12.1µs 11.1µs -8.42% Cond16-16 27.0µs 25.1µs -7.04% Cond32-16 59.1µs 55.5µs -6.14% LoadMostlyHits/*sync_test.DeepCopyMap-16 22.1ns 24.1ns +9.02% LoadMostlyHits/*sync_test.RWMutexMap-16 252ns 249ns -1.20% LoadMostlyHits/*sync.Map-16 16.2ns 16.3ns ~ LoadMostlyMisses/*sync_test.DeepCopyMap-16 22.3ns 22.6ns ~ LoadMostlyMisses/*sync_test.RWMutexMap-16 249ns 247ns -0.51% LoadMostlyMisses/*sync.Map-16 12.7ns 12.7ns ~ LoadOrStoreBalanced/*sync_test.RWMutexMap-16 1.27µs 1.17µs -7.54% LoadOrStoreBalanced/*sync.Map-16 1.12µs 1.10µs -2.35% LoadOrStoreUnique/*sync_test.RWMutexMap-16 1.75µs 1.68µs -3.84% LoadOrStoreUnique/*sync.Map-16 2.07µs 1.97µs -5.13% LoadOrStoreCollision/*sync_test.DeepCopyMap-16 15.8ns 15.9ns ~ LoadOrStoreCollision/*sync_test.RWMutexMap-16 496ns 424ns -14.48% LoadOrStoreCollision/*sync.Map-16 6.07ns 6.07ns ~ Range/*sync_test.DeepCopyMap-16 1.65µs 1.64µs ~ Range/*sync_test.RWMutexMap-16 278µs 288µs +3.75% Range/*sync.Map-16 2.00µs 2.01µs ~ AdversarialAlloc/*sync_test.DeepCopyMap-16 3.45µs 3.44µs ~ AdversarialAlloc/*sync_test.RWMutexMap-16 226ns 227ns ~ AdversarialAlloc/*sync.Map-16 1.09µs 1.07µs -2.36% AdversarialDelete/*sync_test.DeepCopyMap-16 553ns 550ns -0.57% AdversarialDelete/*sync_test.RWMutexMap-16 273ns 274ns ~ AdversarialDelete/*sync.Map-16 247ns 249ns ~ UncontendedSemaphore-16 79.0ns 65.5ns -17.11% ContendedSemaphore-16 112ns 97ns -13.77% MutexUncontended-16 3.34ns 2.51ns -24.69% Mutex-16 266ns 191ns -28.26% MutexSlack-16 226ns 159ns -29.55% MutexWork-16 377ns 338ns -10.14% MutexWorkSlack-16 335ns 308ns -8.20% MutexNoSpin-16 196ns 184ns -5.91% MutexSpin-16 710ns 666ns -6.21% Once-16 1.29ns 1.29ns ~ Pool-16 8.64ns 8.71ns ~ PoolOverflow-16 1.60µs 1.44µs -10.25% SemaUncontended-16 5.39ns 4.42ns -17.96% SemaSyntNonblock-16 539ns 483ns -10.42% SemaSyntBlock-16 413ns 354ns -14.20% SemaWorkNonblock-16 305ns 258ns -15.36% SemaWorkBlock-16 266ns 229ns -14.06% RWMutexUncontended-16 12.9ns 9.7ns -24.80% RWMutexWrite100-16 203ns 147ns -27.47% RWMutexWrite10-16 177ns 119ns -32.74% RWMutexWorkWrite100-16 435ns 403ns -7.39% RWMutexWorkWrite10-16 642ns 611ns -4.79% WaitGroupUncontended-16 4.67ns 3.70ns -20.92% WaitGroupAddDone-16 402ns 355ns -11.54% WaitGroupAddDoneWork-16 208ns 250ns +20.09% WaitGroupWait-16 1.21ns 1.21ns ~ WaitGroupWaitWork-16 5.91ns 5.87ns -0.81% WaitGroupActuallyWait-16 92.2ns 85.8ns -6.91% Updates #21348 Change-Id: Ibb9b271d11b308264103829e176c6d9fe8f867d3 Reviewed-on: https://go-review.googlesource.com/95175 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
2018-03-21runtime,sync/atomic: replace asm BYTEs with insts for x86isharipo
For each replacement, test case is added to new 386enc.s file with exception of EMMS, SYSENTER, MFENCE and LFENCE as they are already covered in amd64enc.s (same on amd64 and 386). The replacement became less obvious after go vet suggested changes Before: BYTE $0x0f; BYTE $0x7f; BYTE $0x44; BYTE $0x24; BYTE $0x08 Changed to MOVQ (this form is being tested): MOVQ M0, 8(SP) Refactored to FP-relative access (go vet advice): MOVQ M0, val+4(FP) Change-Id: I56b87cf3371b6ad81ad0cd9db2033aee407b5818 Reviewed-on: https://go-review.googlesource.com/101475 Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
2018-02-14runtime/internal/atomic: add early nil check on ARMCherry Zhang
If nil, fault before taking the lock or calling into the kernel. Change-Id: I013d78a5f9233c2a9197660025f679940655d384 Reviewed-on: https://go-review.googlesource.com/93636 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-02-14runtime/internal/atomic: unify sys_*_arm.s on non-linuxCherry Zhang
Updates #23778. Change-Id: I80e57a15b6e3bbc2e25ea186399ff0e360fc5c21 Reviewed-on: https://go-review.googlesource.com/93635 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-02-13runtime: fix typo in commentYasuhiro Matsumoto
GitHub-Last-Rev: d6a6fa39095cac8a9acfeacbbafd636e1aa9b55b GitHub-Pull-Request: golang/go#23809 Change-Id: Ife18ba2f982b5e1c30bda32d13dcd441778b986a Reviewed-on: https://go-review.googlesource.com/93575 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-02-12runtime: use NOFRAME on mips and mips64Austin Clements
This replaces frame size -4/-8 with the NOFRAME flag in mips and mips64 assembly. This was automated with: sed -i -e 's/\(^TEXT.*[A-Z]\),\( *\)\$-[84]/\1|NOFRAME,\2$0/' $(find -name '*_mips*.s') Plus a manual fix to mkduff.go. The go binary is identical on both architectures before and after this change. Change-Id: I0310384d1a584118c41d1cd3a042bb8ea7227efb Reviewed-on: https://go-review.googlesource.com/92044 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-02-12runtime: fix silly frame sizes on arm and arm64Austin Clements
"-8" is not a sensible frame size on arm and we're about to start rejecting it. Replace it with -4. Likewise, "-4" is not a sensible frame size on arm64 and we're about to start rejecting it. Replace it with -8. Finally, clean up some places we're weirdly inconsistent about using 0 versus -8. Change-Id: If85e229993d5f7f1f0cfa9852b4e294d053bd784 Reviewed-on: https://go-review.googlesource.com/92038 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-10-25runtime/internal/sys: use boolean constants for sys.BigEndianTobias Klauser
The BigEndian constant is only used in boolean context so assign it boolean constants. Change-Id: If19d61dd71cdfbffede1d98b401f11e6535fba59 Reviewed-on: https://go-review.googlesource.com/73270 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-05-16cmd/internal/obj/mips: add support of LLV, SCV, NOOP instructionsCherry Zhang
LLV and SCV are 64-bit load-linked and store-conditional. They were used in runtime as #define WORD. Change them to normal instruction form. NOOP is hardware no-op. It was written as WORD $0. Make a name for it for better disassembly output. Fixes #12561. Fixes #18238. Change-Id: I82c667ce756fa83ef37b034b641e8c4366335e83 Reviewed-on: https://go-review.googlesource.com/40297 Reviewed-by: Minux Ma <minux@golang.org> Run-TryBot: Minux Ma <minux@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-22runtime/internal/atomic: Remove unnecessary checks for GOARCH_ppc64Carlos Eduardo Seo
Starting in go1.9, the minimum processor requirement for ppc64 is POWER8. This means the checks for GOARCH_ppc64 in asm_ppc64x.s can be removed, since we can assume LBAR and STBCCC instructions (both from ISA 2.06) will always be available. Updates #19074 Change-Id: Ib4418169cd9fc6f871a5ab126b28ee58a2f349e2 Reviewed-on: https://go-review.googlesource.com/38406 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-22runtime/internal/atomic: crash on unaligned 64-bit ops on 32-bit MIPSCherry Zhang
This check was originally implemented by Vladimir in https://go-review.googlesource.com/c/31489/1/src/runtime/internal/atomic/atomic_mipsx.go#30 but removed due to my comment (Sorry!). This CL adds it back. Fixes #17786. Change-Id: I7ff4c2539fc9e2afd8199964b587a8ccf093b896 Reviewed-on: https://go-review.googlesource.com/33431 Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-21runtime/internal/atomic: crash on unaligned 64-bit ops on 386 and ARMCherry Zhang
Updates #17786. Will fix mips(32) when the port is fully landed. Change-Id: I00d4ff666ec14a38cadbcd52569b347bb5bc8b75 Reviewed-on: https://go-review.googlesource.com/33236 Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-12runtime/internal/atomic: add TestUnaligned64Brad Fitzpatrick
Add a variant of sync/atomic's TestUnaligned64 to runtime/internal/atomic. Skips the test on arm for now where it's currently failing. Updates #17786 Change-Id: If63f9c1243e9db7b243a95205b2d27f7d1dc1e6e Reviewed-on: https://go-review.googlesource.com/33159 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-03sync/atomic: add support for GOARCH=mips{,le}Vladimir Stefanovic
Change-Id: I10f36710dd95b9bd31b3b82a3c32edcadb90ffa9 Reviewed-on: https://go-review.googlesource.com/31510 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-11-03runtime/internal/atomic: add GOARCH=mips{,le} supportVladimir Stefanovic
Change-Id: I99a48f719fd1a8178fc59787084a074e91c89ac6 Reviewed-on: https://go-review.googlesource.com/31489 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 21:11:01 +0000