aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/mbarrier.go
AgeCommit message (Collapse)Author
2024-11-21all: fix some function names and typos in commentcuishuang
Change-Id: I07e7c8eaa5bd4bac0d576b2f2f4cd3f81b0b77a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/630055 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Commit-Queue: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-11-01runtime: fix out-of-date comment docchangwang ma
Change-Id: I352fa0e4e048b896d63427f1c2c519bfed24c702 Reviewed-on: https://go-review.googlesource.com/c/go/+/622017 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-10-08internal/runtime/maps: initial swiss table map implementationMichael Pratt
Add a new package that will contain a new "Swiss Table" (https://abseil.io/about/design/swisstables) map implementation, which is intended to eventually replace the existing runtime map implementation. This implementation is based on the fabulous github.com/cockroachdb/swiss package contributed by Peter Mattis. This CL adds an hash map implementation. It supports all the core operations, but does not have incremental growth. For #54766. Change-Id: I52cf371448c3817d471ddb1f5a78f3513565db41 Reviewed-on: https://go-review.googlesource.com/c/go/+/582415 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-09-17runtime: move getcallerpc to internal/runtime/sysMichael Pratt
Moving these intrinsics to a base package enables other internal/runtime packages to use them. For #54766. Change-Id: I0b3eded3bb45af53e3eb5bab93e3792e6a8beb46 Reviewed-on: https://go-review.googlesource.com/c/go/+/613260 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-05-29all: document legacy //go:linkname for final round of modulesRuss Cox
Add linknames for most modules with ≥50 dependents. Add linknames for a few other modules that we know are important but are below 50. Remove linknames from badlinkname.go that do not merit inclusion (very small number of dependents). We can add them back later if the need arises. Fixes #67401. (For now.) Change-Id: I1e49fec0292265256044d64b1841d366c4106002 Reviewed-on: https://go-review.googlesource.com/c/go/+/587756 Auto-Submit: Russ Cox <rsc@golang.org> TryBot-Bypass: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-05-29all: document legacy //go:linkname for modules with ≥100 dependentsRuss Cox
For #67401. Change-Id: I015408a3f437c1733d97160ef2fb5da6d4efcc5c Reviewed-on: https://go-review.googlesource.com/c/go/+/587598 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org>
2024-05-23all: document legacy //go:linkname for modules with ≥200 dependentsRuss Cox
Ignored these linknames which have not worked for a while: github.com/xtls/xray-core: context.newCancelCtx removed in CL 463999 (Feb 2023) github.com/u-root/u-root: funcPC removed in CL 513837 (Jul 2023) tinygo.org/x/drivers: net.useNetdev never existed For #67401. Change-Id: I9293f4ef197bb5552b431de8939fa94988a060ce Reviewed-on: https://go-review.googlesource.com/c/go/+/587576 Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-05-23all: document legacy //go:linkname for modules with ≥1,000 dependentsRuss Cox
For #67401. Change-Id: If23a2c07e3dd042a3c439da7088437a330b9caa4 Reviewed-on: https://go-review.googlesource.com/c/go/+/587222 Auto-Submit: Russ Cox <rsc@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-05-23all: document legacy //go:linkname for modules with ≥20,000 dependentsRuss Cox
For #67401. Change-Id: Icc10ede72547d8020c0ba45e89d954822a4b2455 Reviewed-on: https://go-review.googlesource.com/c/go/+/587218 Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-05-22all: document legacy //go:linkname for modules with ≥50,000 dependentsRuss Cox
Note that this depends on the revert of CL 581395 to move zeroVal back. For #67401. Change-Id: I507c27c2404ad1348aabf1ffa3740e6b1957495b Reviewed-on: https://go-review.googlesource.com/c/go/+/587217 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-05-22all: document legacy //go:linkname for modules with ≥100,000 dependentsRuss Cox
For #67401. Change-Id: I51f5b561ee11eb242e3b1585d591281d0df4fc24 Reviewed-on: https://go-review.googlesource.com/c/go/+/587215 Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-04runtime: use .Pointers() instead of manual checkingPouriya
Change-Id: Ib78c1513616089f4942297cd17212b1b11871fd5 GitHub-Last-Rev: f97fe5b5bffffe25dc31de7964588640cb70ec41 GitHub-Pull-Request: golang/go#65819 Reviewed-on: https://go-review.googlesource.com/c/go/+/565515 Reviewed-by: Jorropo <jorropo.pgm@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-11-16runtime: optimize bulkBarrierPreWrite with allocheadersMichael Anthony Knyszek
Currently bulkBarrierPreWrite follows a fairly slow path wherein it calls typePointersOf, which ends up calling into fastForward. This does some fairly heavy computation to move the iterator forward without any assumptions about where it lands at all. It needs to be completely general to support splitting at arbitrary boundaries, for example for scanning oblets. This means that copying objects during the GC mark phase is fairly expensive, and is a regression from before allocheaders. However, in almost all cases bulkBarrierPreWrite and bulkBarrierPreWriteSrcOnly have perfect type information. We can do a lot better in these cases because we're starting on a type-size boundary, which is exactly what the iterator is built around. This change adds the typePointersOfType method which produces a typePointers iterator from a pointer and a type. This change significantly improves the performance of these bulk write barriers, eliminating some performance regressions that were noticed on the perf dashboard. There are still just a couple cases where we have to use the more general typePointersOf calls, but they're fairly rare; most bulk barriers have perfect type information. This change is tested by the GCInfo tests in the runtime and the GCBits tests in the reflect package via an additional check in getgcmask. Results for tile38 before and after allocheaders. There was previous a regression in the p90, now it's gone. Also, the overall win has been boosted slightly. tile38 $ benchstat noallocheaders.results allocheaders.results name old time/op new time/op delta Tile38QueryLoad 481µs ± 1% 468µs ± 1% -2.71% (p=0.000 n=10+10) name old average-RSS-bytes new average-RSS-bytes delta Tile38QueryLoad 6.32GB ± 1% 6.23GB ± 0% -1.38% (p=0.000 n=9+8) name old peak-RSS-bytes new peak-RSS-bytes delta Tile38QueryLoad 6.49GB ± 1% 6.40GB ± 1% -1.38% (p=0.002 n=10+10) name old peak-VM-bytes new peak-VM-bytes delta Tile38QueryLoad 7.72GB ± 1% 7.64GB ± 1% -1.07% (p=0.007 n=10+10) name old p50-latency-ns new p50-latency-ns delta Tile38QueryLoad 212k ± 1% 205k ± 0% -3.02% (p=0.000 n=10+9) name old p90-latency-ns new p90-latency-ns delta Tile38QueryLoad 622k ± 1% 616k ± 1% -1.03% (p=0.005 n=10+10) name old p99-latency-ns new p99-latency-ns delta Tile38QueryLoad 4.55M ± 2% 4.39M ± 2% -3.51% (p=0.000 n=10+10) name old ops/s new ops/s delta Tile38QueryLoad 12.5k ± 1% 12.8k ± 1% +2.78% (p=0.000 n=10+10) Change-Id: I0a48f848eae8777d0fd6769c3a1fe449f8d9d0a6 Reviewed-on: https://go-review.googlesource.com/c/go/+/542219 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-07cmd/compile,runtime: dedup writeBarrier neededMauri de Souza Meneguzzo
The writeBarrier "needed" struct member has the exact same value as "enabled", and used interchangeably. I'm not sure if we plan to make a distinction between the two at some point, but today they are effectively the same, so dedup it and keep only "enabled". Change-Id: I65e596f174e1e820dc471a45ff70c0ef4efbc386 GitHub-Last-Rev: f8c805a91606d42c8d5b178ddd7d0bec7aaf9f55 GitHub-Pull-Request: golang/go#63814 Reviewed-on: https://go-review.googlesource.com/c/go/+/538495 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-26runtime: delete a very stale commentAustin Clements
This comment claims mark termination re-scans stacks and uses the write barrier to determine how much of the stack needs to be rescanned. This hasn't been true since we introduced the hybrid write barrier and deleted stack rescanning with CL 31766 in Go 1.8. Updates #17503 I suppose. Change-Id: I5e90f25020c9fa6f146ec6ed0642ba2b4884c2a8 Reviewed-on: https://go-review.googlesource.com/c/go/+/498435 Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-11runtime: redefine _type to abi.Type; add rtype for methods.David Chase
Change-Id: I1c478b704d84811caa209006c657dda82d9c4cf9 Reviewed-on: https://go-review.googlesource.com/c/go/+/488435 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
2023-05-05internal/abi: refactor (basic) type struct into one definitionDavid Chase
This touches a lot of files, which is bad, but it is also good, since there's N copies of this information commoned into 1. The new files in internal/abi are copied from the end of the stack; ultimately this will all end up being used. Change-Id: Ia252c0055aaa72ca569411ef9f9e96e3d610889e Reviewed-on: https://go-review.googlesource.com/c/go/+/462995 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-04-10reflect: remove typedmemmovepartial as it is unusedDaniel Martí
It appears to have been unused since https://go.dev/cl/298670 in April 2021, as that change removed its only use. It is always in the git history if it is needed again. Change-Id: Ie55d059c102dfaa75bd253e09d48a4b30f45e941 Reviewed-on: https://go-review.googlesource.com/c/go/+/483136 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Pratt <mpratt@google.com>
2023-03-20all: gofmtDamien Neil
Change-Id: I926388ee5aeeff11f765cbd4558b66645d1bbc08 Reviewed-on: https://go-review.googlesource.com/c/go/+/477836 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-02-17cmd/compile: move raw writes out of write barrier codeKeith Randall
Previously, the write barrier calls themselves did the actual writes to memory. Instead, move those writes out to a common location that both the wb-enabled and wb-disabled code paths share. This enables us to optimize the write barrier path without having to worry about performing the actual writes. Change-Id: Ia71ab651908ec124cc33141afb52e4ca19733ac6 Reviewed-on: https://go-review.googlesource.com/c/go/+/447780 Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Bypass: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-16runtime: reimplement GODEBUG=cgocheck=2 as a GOEXPERIMENTKeith Randall
Move this knob from a binary-startup thing to a build-time thing. This will enable followon optmizations to the write barrier. Change-Id: Ic3323348621c76a7dc390c09ff55016b19c43018 Reviewed-on: https://go-review.googlesource.com/c/go/+/447778 Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-01-30reflect,runtime: add Value.ClearCuong Manh Le
Fixes #55002 Change-Id: I7d0f14cc54f67f2769b51d2efafc4ae3714f0e3d Reviewed-on: https://go-review.googlesource.com/c/go/+/457895 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-26runtime: fix a few function names on commentscui fliter
Change-Id: I4be0b1e612dcc21ca6bb7d4395f1c0aa52480759 GitHub-Last-Rev: 032480c4c9ddb2bedea26b01bb80b8a079bfdcf3 GitHub-Pull-Request: golang/go#55993 Reviewed-on: https://go-review.googlesource.com/c/go/+/437518 Reviewed-by: hopehook <hopehook@golangcn.org> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: hopehook <hopehook@golangcn.org>
2022-09-19runtime: make mSpanStateBox accessors nosplitKeith Randall
get, at least, is called from typedmemclr which must not be interruptible. These were previously nosplit by accident before CL 424395 (the only call they had was an intrinsic, so they were leaf functions, so they had no prologue). After CL 424395 they contained a call (in noinline builds), thus had a prologue, thus had a suspension point. I have no idea how we might test this. This is another motivating use case for having a nosplitrec directive in the runtime. Fixes #55156 Fixes #54779 Fixes #54906 Fixes #54907 Change-Id: I851d733d71bda7172c4c96e027657e22b499ee00 Reviewed-on: https://go-review.googlesource.com/c/go/+/431919 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-05all: separate doc comment from //go: directivesRuss Cox
A future change to gofmt will rewrite // Doc comment. //go:foo to // Doc comment. // //go:foo Apply that change preemptively to all comments (not necessarily just doc comments). For #51082. Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9 Reviewed-on: https://go-review.googlesource.com/c/go/+/384260 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01runtime: use correct parameter name in commentIan Lance Taylor
Change-Id: If8713aca9d6b1f1c218e85f09c9ee016833b3faa Reviewed-on: https://go-review.googlesource.com/c/go/+/396515 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2022-02-05runtime: change sys.PtrSize to goarch.PtrSize in commentsIan Lance Taylor
The code was updated, the comments were not. Change-Id: If387779f3abd5e8a1b487fe34c33dcf9ce5fa7ff Reviewed-on: https://go-review.googlesource.com/c/go/+/383495 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-11-02runtime, syscall: add calls to asan functionsfanzha02
Add explicit address sanitizer instrumentation to the runtime and syscall packages. The compiler does not instrument the runtime package. It does instrument the syscall package, but we need to add a couple of cases that it can't see. Refer to the implementation of the asan malloc runtime library, this patch also allocates extra memory as the redzone, around the returned memory region, and marks the redzone as unaddressable to detect the overflows or underflows. Updates #44853. Change-Id: I2753d1cc1296935a66bf521e31ce91e35fcdf798 Reviewed-on: https://go-review.googlesource.com/c/go/+/298614 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: fannie zhang <Fannie.Zhang@arm.com>
2021-06-17[dev.typeparams] runtime: replace uses of runtime/internal/sys.PtrSize with ↵Michael Anthony Knyszek
internal/goarch.PtrSize [generated] [git-generate] cd src/runtime/internal/math gofmt -w -r "sys.PtrSize -> goarch.PtrSize" . goimports -w *.go cd ../.. gofmt -w -r "sys.PtrSize -> goarch.PtrSize" . goimports -w *.go Change-Id: I43491cdd54d2e06d4d04152b3d213851b7d6d423 Reviewed-on: https://go-review.googlesource.com/c/go/+/328337 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-05-21[dev.typeparams] runtime: replace funcPC with internal/abi.FuncPCABIInternalCherry Mui
At this point all funcPC references are ABIInternal functions. Replace with the intrinsics. Change-Id: I3ba7e485c83017408749b53f92877d3727a75e27 Reviewed-on: https://go-review.googlesource.com/c/go/+/321954 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-02-16[dev.regabi] reflect: support for register ABI on amd64 for reflect.(Value).CallMichael Anthony Knyszek
This change adds support for the new register ABI on amd64 to reflect.(Value).Call. If internal/abi's register counts are non-zero, reflect will try to set up arguments in registers on the Call path. Note that because the register ABI becomes ABI0 with zero registers available, this should keep working as it did before. This change does not add any tests for the register ABI case because there's no way to do so at the moment. For #40724. Change-Id: I8aa089a5aa5a31b72e56b3d9388dd3f82203985b Reviewed-on: https://go-review.googlesource.com/c/go/+/272568 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-16cmd/compile: unify reflect, string and slice copy runtime functionsMartin Möhrmann
Use a common runtime slicecopy function to copy strings or slices into slices. This deduplicates similar code previously used in reflect.slicecopy and runtime.stringslicecopy. Change-Id: I09572ff0647a9e12bb5c6989689ce1c43f16b7f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/254658 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Martin Möhrmann <moehrmann@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2020-04-08cmd/compile,runtime: pass only ptr and len to some runtime callsJosh Bleecher Snyder
Some runtime calls accept a slice, but only use ptr and len. This change modifies most such routines to accept only ptr and len. After this change, the only runtime calls that accept an unnecessary cap arg are concatstrings and slicerunetostring. Neither is particularly common, and both are complicated to modify. Negligible compiler performance impact. Shrinks binaries a little. There are only a few regressions; the one I investigated was due to register allocation fluctuation. Passes 'go test -race std cmd', modulo #38265 and #38266. Wow, does that take a long time to run. Updates #36890 file before after Δ % compile 19655024 19655152 +128 +0.001% cover 5244840 5236648 -8192 -0.156% dist 3662376 3658280 -4096 -0.112% link 6680056 6675960 -4096 -0.061% pprof 14789844 14777556 -12288 -0.083% test2json 2824744 2820648 -4096 -0.145% trace 11647876 11639684 -8192 -0.070% vet 8260472 8256376 -4096 -0.050% total 115163736 115118808 -44928 -0.039% Change-Id: Idb29fa6a81d6a82bfd3b65740b98cf3275ca0a78 Reviewed-on: https://go-review.googlesource.com/c/go/+/227163 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-04-08runtime: simplify typedmemmovepartialIan Lance Taylor
The offset is always a multiple of the pointer size. Change-Id: I790e087e89a081044a3ec35d99880533a4c929bd Reviewed-on: https://go-review.googlesource.com/c/go/+/227540 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-04-08runtime: add a couple of writeBarrier.needed checksIan Lance Taylor
Make typedmemmove, typedmemclr, typedmemclrpartial look more like other callers of bulkBarrierPreWrite. Change-Id: Ic47030d88bf07d290f91198b7810ffc16d9769e2 Reviewed-on: https://go-review.googlesource.com/c/go/+/227541 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-04-08runtime: only check for pointers up to ptrdata, not sizeIan Lance Taylor
Change-Id: I166cf253b7f2483d652c98d2fba36c380e2f3347 Reviewed-on: https://go-review.googlesource.com/c/go/+/227177 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2019-03-25runtime: remove kindNoPointersKeith Randall
We already have the ptrdata field in a type, which encodes exactly the same information that kindNoPointers does. My problem with kindNoPointers is that it often leads to double-negative code like: t.kind & kindNoPointers != 0 Much clearer is: t.ptrdata == 0 Update #27167 Change-Id: I92307d7f018a6bbe3daca4a4abb4225e359349b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/169157 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-27internal/reflectlite: lite version of reflect packageMarcel van Lohuizen
to be used by errors package for checking assignability and setting error values in As. Updates #29934. Change-Id: I8c1d02a2c6efa0919d54b286cfe8b4edc26da059 Reviewed-on: https://go-review.googlesource.com/c/161759 Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2018-09-26reflect: use correct write barrier operations for method funcsKeith Randall
Fix the code to use write barriers on heap memory, and no write barriers on stack memory. These errors were discoverd as part of fixing #27695. They may have something to do with that issue, but hard to be sure. The core cause is different, so this fix is a separate CL. Update #27695 Change-Id: Ib005f6b3308de340be83c3d07d049d5e316b1e3c Reviewed-on: https://go-review.googlesource.com/137438 Reviewed-by: Austin Clements <austin@google.com>
2018-08-24runtime: replace typedmemmmove with bulkBarrierPreWrite and memmove in growsliceMartin Möhrmann
A bulkBarrierPreWrite together with a memmove as used in typedslicecopy is faster than a typedmemmove for each element of the old slice that needs to be copied to the new slice. typedslicecopy is not used here as runtime functions should not call other instrumented runtime functions and some conditions like dst == src or the destination slice not being large enought that are checked for in typedslicecopy can not happen in growslice. Append 13.5ns ± 6% 13.3ns ± 3% ~ (p=0.304 n=10+10) AppendGrowByte 1.18ms ± 2% 1.19ms ± 1% ~ (p=0.113 n=10+9) AppendGrowString 123ms ± 1% 73ms ± 1% -40.39% (p=0.000 n=9+8) AppendSlice/1Bytes 3.81ns ± 1% 3.78ns ± 1% ~ (p=0.116 n=10+10) AppendSlice/4Bytes 3.71ns ± 1% 3.70ns ± 0% ~ (p=0.095 n=10+9) AppendSlice/7Bytes 3.73ns ± 0% 3.75ns ± 1% ~ (p=0.442 n=10+10) AppendSlice/8Bytes 4.00ns ± 1% 4.01ns ± 1% ~ (p=0.330 n=10+10) AppendSlice/15Bytes 4.29ns ± 1% 4.28ns ± 1% ~ (p=0.536 n=10+10) AppendSlice/16Bytes 4.28ns ± 1% 4.31ns ± 1% +0.75% (p=0.019 n=10+10) AppendSlice/32Bytes 4.57ns ± 2% 4.58ns ± 2% ~ (p=0.236 n=10+10) AppendSliceLarge/1024Bytes 305ns ± 2% 306ns ± 1% ~ (p=0.236 n=10+10) AppendSliceLarge/4096Bytes 1.06µs ± 1% 1.06µs ± 0% ~ (p=1.000 n=9+10) AppendSliceLarge/16384Bytes 3.12µs ± 2% 3.11µs ± 1% ~ (p=0.493 n=10+10) AppendSliceLarge/65536Bytes 5.61µs ± 5% 5.36µs ± 2% -4.58% (p=0.003 n=10+8) AppendSliceLarge/262144Bytes 21.0µs ± 1% 19.5µs ± 1% -7.09% (p=0.000 n=8+10) AppendSliceLarge/1048576Bytes 78.4µs ± 1% 78.7µs ± 2% ~ (p=0.315 n=8+10) AppendStr/1Bytes 3.96ns ± 6% 3.99ns ± 9% ~ (p=0.591 n=10+10) AppendStr/4Bytes 3.98ns ± 1% 3.99ns ± 1% ~ (p=0.515 n=9+9) AppendStr/8Bytes 4.27ns ± 1% 4.27ns ± 1% ~ (p=0.633 n=10+10) AppendStr/16Bytes 4.56ns ± 2% 4.55ns ± 1% ~ (p=0.869 n=10+10) AppendStr/32Bytes 4.85ns ± 1% 4.89ns ± 1% +0.71% (p=0.003 n=10+8) AppendSpecialCase 18.7ns ± 1% 18.7ns ± 1% ~ (p=0.144 n=10+10) AppendInPlace/NoGrow/Byte 438ns ± 1% 439ns ± 1% ~ (p=0.135 n=10+8) AppendInPlace/NoGrow/1Ptr 1.05µs ± 2% 1.05µs ± 1% ~ (p=0.469 n=10+10) AppendInPlace/NoGrow/2Ptr 1.77µs ± 1% 1.78µs ± 2% ~ (p=0.469 n=10+10) AppendInPlace/NoGrow/3Ptr 1.94µs ± 1% 1.93µs ± 2% ~ (p=0.517 n=10+10) AppendInPlace/NoGrow/4Ptr 3.18µs ± 1% 3.17µs ± 0% ~ (p=0.483 n=10+9) AppendInPlace/Grow/Byte 382ns ± 2% 383ns ± 2% ~ (p=0.705 n=9+10) AppendInPlace/Grow/1Ptr 383ns ± 1% 384ns ± 1% ~ (p=0.844 n=10+10) AppendInPlace/Grow/2Ptr 459ns ± 2% 467ns ± 2% +1.74% (p=0.001 n=10+10) AppendInPlace/Grow/3Ptr 593ns ± 1% 597ns ± 2% ~ (p=0.195 n=10+10) AppendInPlace/Grow/4Ptr 583ns ± 2% 589ns ± 2% ~ (p=0.084 n=10+10) Change-Id: I629872f065a22b29267c1adbfc578aaedd36d365 Reviewed-on: https://go-review.googlesource.com/115755 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-02-27runtime: short-circuit typedmemmove when dst==srcJosh Bleecher Snyder
Change-Id: I855268a4c0d07ad602ec90f5da66422d3d87c5f2 Reviewed-on: https://go-review.googlesource.com/94595 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2018-02-15runtime: short-circuit typedslicecopy when dstp == srcpJosh Bleecher Snyder
If copying from a slice to itself, skip the write barriers and actual memory copies. This happens in practice in code like this snippet from the trim pass in the compiler, when k ends up being 0: copy(s.Values[k:], s.Values[:m]) Change-Id: Ie6924acfd56151f874d87f1d7f1f74320b4c4f10 Reviewed-on: https://go-review.googlesource.com/94023 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-02-13runtime: remove legacy eager write barrierAustin Clements
Now that the buffered write barrier is implemented for all architectures, we can remove the old eager write barrier implementation. This CL removes the implementation from the runtime, support in the compiler for calling it, and updates some compiler tests that relied on the old eager barrier support. It also makes sure that all of the useful comments from the old write barrier implementation still have a place to live. Fixes #22460. Updates #21640 since this fixes the layering concerns of the write barrier (but not the other things in that issue). Change-Id: I580f93c152e89607e0a72fe43370237ba97bae74 Reviewed-on: https://go-review.googlesource.com/92705 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Rick Hudson <rlh@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-10-31cmd/compile,runtime: update instrumentation commentsAustin Clements
The compiler's instrumentation pass has some out-of-date comments about the write barrier and some confusing comments about typedslicecopy. Update these comments and add a comment to typedslicecopy explaining why it's manually instrumented while none of the other operations are. Change-Id: I024e5361d53f1c3c122db0c85155368a30cabd6b Reviewed-on: https://go-review.googlesource.com/74430 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-10-30runtime: use buffered write barrier for bulkBarrierPreWriteAustin Clements
This modifies bulkBarrierPreWrite to use the buffered write barrier instead of the eager write barrier. This reduces the number of system stack switches and sanity checks by a factor of the buffer size (currently 256). This affects both typedmemmove and typedmemclr. Since this is purely a runtime change, it applies to all arches (unlike the pointer write barrier). name old time/op new time/op delta BulkWriteBarrier-12 7.33ns ± 6% 4.46ns ± 9% -39.10% (p=0.000 n=20+19) Updates #22460. Change-Id: I6a686a63bbf08be02b9b97250e37163c5a90cdd8 Reviewed-on: https://go-review.googlesource.com/73832 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
2017-10-30runtime: simplify and optimize typedslicecopyAustin Clements
Currently, typedslicecopy meticulously performs a typedmemmove on every element of the slice. This probably used to be necessary because we only had an individual element's type, but now we use the heap bitmap, so we only need to know whether the type has any pointers and how big it is. Hence, this CL rewrites typedslicecopy to simply perform one bulk barrier and one memmove. This also has a side-effect of eliminating two unnecessary write barriers per slice element that were coming from updates to dstp and srcp, which were stored in the parent stack frame. However, most of the win comes from eliminating the loops. name old time/op new time/op delta BulkWriteBarrier-12 7.83ns ±10% 7.33ns ± 6% -6.45% (p=0.000 n=20+20) Updates #22460. Change-Id: Id3450e9f36cc8e0892f268319b136f0d8f5464b8 Reviewed-on: https://go-review.googlesource.com/73831 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
2017-10-29runtime: allow write barriers in startpanic_mAustin Clements
We're about to start tracking nowritebarrierrec through systemstack calls, which will reveal write barriers in startpanic_m prohibited by various callers. We actually can allow write barriers here because the write barrier is a no-op when we're panicking. Let the compiler know. Updates #22384. For #22460. Change-Id: Ifb3a38d3dd9a4125c278c3680f8648f987a5b0b8 Reviewed-on: https://go-review.googlesource.com/72770 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
2017-09-22runtime: remove getcallerpc argumentAustin Clements
Now that getcallerpc is a compiler intrinsic on x86 and non-x86 platforms don't need the argument, we can drop it. Sadly, this doesn't let us remove any dummy arguments since all of those cases also use getcallersp, which still takes the argument pointer, but this is at least an improvement. Change-Id: I9c34a41cf2c18cba57f59938390bf9491efb22d2 Reviewed-on: https://go-review.googlesource.com/65474 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-04-21cmd/compile: experiment which clobbers all dead pointer fieldsKeith Randall
The experiment "clobberdead" clobbers all pointer fields that the compiler thinks are dead, just before and after every safepoint. Useful for debugging the generation of live pointer bitmaps. Helped find the following issues: Update #15936 Update #16026 Update #16095 Update #18860 Change-Id: Id1d12f86845e3d93bae903d968b1eac61fc461f9 Reviewed-on: https://go-review.googlesource.com/23924 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-02-23runtime: new profile buffer implementation supporting label pointersRuss Cox
The existing CPU profiling buffer is a slice of uintptr, but we want to start including profiling label data in the profiles, and those labels need to be pointers in order to let them describe rich information. This CL implements a new profBuf type that holds both a slice of uint64 for data and a slice of unsafe.Pointer for profiling labels (aka tags). Making the runtime use these buffers will happen in followup CLs. Change-Id: I9ff16b532d8edaf4ce0cbba1098229a561834efc Reviewed-on: https://go-review.googlesource.com/36713 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
generated by cgit v1.3 (git 2.53.0) at 2026-04-16 09:31:50 +0000