| Age | Commit message (Collapse) | Author |
|---|
|
Future CLs will remove the invariant that pointers are always put in
the write barrier in pairs.
The behavior of the assembly code changes a bit, where instead of writing
the pointers unconditionally and then checking for overflow, check for
overflow first and then write the pointers.
Also changed the write barrier flush function to not take the src/dst
as arguments.
Change-Id: I2ef708038367b7b82ea67cbaf505a1d5904c775c
Reviewed-on: https://go-review.googlesource.com/c/go/+/447779
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Keith Randall <khr@golang.org>
|
|
Move this knob from a binary-startup thing to a build-time thing.
This will enable followon optmizations to the write barrier.
Change-Id: Ic3323348621c76a7dc390c09ff55016b19c43018
Reviewed-on: https://go-review.googlesource.com/c/go/+/447778
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
When the GC is scanning some memory (possibly conservatively),
finding a pointer, while concurrently another goroutine is
allocating an object at the same address as the found pointer, the
GC may see the pointer before the object and/or the heap bits are
initialized. This may cause the GC to see bad pointers and
possibly crash.
To prevent this, we make it that the scanner can only see the
object as allocated after the object and the heap bits are
initialized. Currently the allocator uses freeindex to find the
next available slot, and that code is coupled with updating the
free index to a new slot past it. The scanner also uses the
freeindex to determine if an object is allocated. This is somewhat
racy. This CL makes the scanner use a different field, which is
only updated after the object initialization (and a memory
barrier).
Fixes #54596.
Change-Id: I2a57a226369926e7192c253dd0d21d3faf22297c
Reviewed-on: https://go-review.googlesource.com/c/go/+/449017
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
This reverts commit bed2b7cf41471e1521af5a83ae28bd643eb3e038.
Reason for revert: I clicked submit by accident on the wrong CL.
Change-Id: Iddf128cb62f289d472510eb30466e515068271b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/449501
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
|
|
When the GC is scanning some memory (possibly conservatively),
finding a pointer, while concurrently another goroutine is
allocating an object at the same address as the found pointer, the
GC may see the pointer before the object and/or the heap bits are
initialized. This may cause the GC to see bad pointers and
possibly crash.
To prevent this, we make it that the scanner can only see the
object as allocated after the object and the heap bits are
initialized. As the scanner uses the freeindex to determine if an
object is allocated, we delay the increment of freeindex after the
initialization.
As currently in some code path finding the next free index and
updating the free index to a new slot past it is coupled, this
needs a small refactoring. In the new code mspan.nextFreeIndex
return the next free index but not update it (although allocCache
is updated). mallocgc will update it at a later time.
Fixes #54596.
Change-Id: I6dd5ccf743f2d2c46a1ed67c6a8237fe09a71260
Reviewed-on: https://go-review.googlesource.com/c/go/+/427619
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Change-Id: I4be0b1e612dcc21ca6bb7d4395f1c0aa52480759
GitHub-Last-Rev: 032480c4c9ddb2bedea26b01bb80b8a079bfdcf3
GitHub-Pull-Request: golang/go#55993
Reviewed-on: https://go-review.googlesource.com/c/go/+/437518
Reviewed-by: hopehook <hopehook@golangcn.org>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: hopehook <hopehook@golangcn.org>
|
|
Replace all uses of Ctz64/32/8 with TrailingZeros64/32/8, because they
are the same and maybe duplicated. Also renamed CtzXX functions in 386
assembly code.
Change-Id: I19290204858083750f4be589bb0923393950ae6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/438935
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
|
|
This change adds an API to the runtime for arenas. A later CL can
potentially export it as an experimental API, but for now, just the
runtime implementation will suffice.
The purpose of arenas is to improve efficiency, primarily by allowing
for an application to manually free memory, thereby delaying garbage
collection. It comes with other potential performance benefits, such as
better locality, a better allocation strategy, and better handling of
interior pointers by the GC.
This implementation is based on one by danscales@google.com with a few
significant differences:
* The implementation lives entirely in the runtime (all layers).
* Arena chunks are the minimum of 8 MiB or the heap arena size. This
choice is made because in practice 64 MiB appears to be way too large
of an area for most real-world use-cases.
* Arena chunks are not unmapped, instead they're placed on an evacuation
list and when there are no pointers left pointing into them, they're
allowed to be reused.
* Reusing partially-used arena chunks no longer tries to find one used
by the same P first; it just takes the first one available.
* In order to ensure worst-case fragmentation is never worse than 25%,
only types and slice backing stores whose sizes are 1/4th the size of
a chunk or less may be used. Previously larger sizes, up to the size
of the chunk, were allowed.
* ASAN, MSAN, and the race detector are fully supported.
* Sets arena chunks to fault that were deferred at the end of mark
termination (a non-public patch once did this; I don't see a reason
not to continue that).
For #51317.
Change-Id: I83b1693a17302554cb36b6daa4e9249a81b1644f
Reviewed-on: https://go-review.googlesource.com/c/go/+/423359
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
|
|
This places getStackMap alongside argBytes and argMapInternal as
another method of stkframe.
For #54466, albeit rather indirectly.
Change-Id: I411dda3605dd7f996983706afcbefddf29a68a85
Reviewed-on: https://go-review.googlesource.com/c/go/+/424515
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Auto-Submit: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
Some code paths in the runtime (cgo, heapdump) request heap bits
without first checking that the span is !noscan. Instead of trying
to find and work around all those cases, just set the pointer bits
of noscan spans correctly. It's somewhat safer than ensuring we
caught all the possible cases.
Fixes #54557
Fixes #54558
Change-Id: Ibd476e6cdea77c962e4d15aad26f29df66fd94e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/425194
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
Updates #46731
Change-Id: Ic2208c8bb639aa1e390be0d62e2bd799ecf20654
Reviewed-on: https://go-review.googlesource.com/c/go/+/421878
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
|
|
Change-Id: I1226ff66fd0c64984939793eb8ef96c08d030fa1
Reviewed-on: https://go-review.googlesource.com/c/go/+/424399
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: hopehook <hopehook@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
[This is a retry of CL 407036 + its revert CL 422394. The only
content change is the 1-line change in cmd/internal/obj/objfile.go.]
Read the bitmaps one uintptr at a time instead of one byte at a time.
Performance so far:
Allocation heavy, no retention: ~30% faster in heapBitsSetType
Scan heavy, ~no allocation: ~even in scanobject
Change-Id: I04d899e1dbd23e989e9f552cdc1880318779c14c
Reviewed-on: https://go-review.googlesource.com/c/go/+/422635
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
[this is a retry of CL 407035 + its revert CL 422395. The content is unchanged]
Use just 1 bit per word to record the ptr/nonptr bitmap.
Use word-sized operations to manipulate the bitmap, so we can operate
on up to 64 ptr/nonptr bits at a time.
Use a separate bitmap, one bit per word of the ptr/nonptr bitmap,
to encode a no-more-pointers signal. Since we can check 64 ptr/nonptr
bits at once, knowing the exact last pointer location is not necessary.
As a followon CL, we should make the gcdata bitmap an array of
uintptr instead of an array of byte, so we can load 64 bits of it at once.
Similarly for the processing of gc programs.
Change-Id: Ica5eb622f5b87e647be64f471d67b02732ef8be6
Reviewed-on: https://go-review.googlesource.com/c/go/+/422634
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
|
|
This reverts commit b589208c8cc6e08239868f47e12c1449cd797bac.
Reason for revert: Bug somewhere in this code, causing wasm and maybe linux/386 to fail.
Change-Id: I5e1e501d839584e0219271bb937e94348f83c11f
Reviewed-on: https://go-review.googlesource.com/c/go/+/422395
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Keith Randall <khr@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
This reverts commit c3833a55433f4b2981253f64444fe5c3d1bc910a.
Reason for revert: Bug somewhere in this code, causing wasm and maybe linux/386 to fail.
Change-Id: I05f7cfa467598ca0c2c84fd4f752cc4ef117cc51
Reviewed-on: https://go-review.googlesource.com/c/go/+/422394
Run-TryBot: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Read the bitmaps one uintptr at a time instead of one byte at a time.
Performance so far:
Allocation heavy, no retention: ~30% faster in heapBitsSetType
Scan heavy, ~no allocation: ~even in scanobject
Change-Id: I40d492b50d7f89d1b4261c2de58f6d255fa5e93e
Reviewed-on: https://go-review.googlesource.com/c/go/+/407036
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Use just 1 bit per word to record the ptr/nonptr bitmap.
Use word-sized operations to manipulate the bitmap, so we can operate
on up to 64 ptr/nonptr bits at a time.
Use a separate bitmap, one bit per word of the ptr/nonptr bitmap,
to encode a no-more-pointers signal. Since we can check 64 ptr/nonptr
bits at once, knowing the exact last pointer location is not necessary.
This cleans up the bitmap implementation significantly, which will
hopefully make it faster. TODO: measure
As a followon CL, we should make the gcdata bitmap an array of
uintptr instead of an array of byte, so we can load 64 bits of it at once.
Similarly for the processing of gc programs.
Change-Id: I18151b1876d9543599800dec51e2a1b19df97d49
Reviewed-on: https://go-review.googlesource.com/c/go/+/407035
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
|
|
Change-Id: Iedf10558d9a1d3b80a151927b99660b688ed9ccb
Reviewed-on: https://go-review.googlesource.com/c/go/+/418585
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
|
|
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]
Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.
For #51082.
Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
A future change to gofmt will rewrite
// Doc comment.
//go:foo
to
// Doc comment.
//
//go:foo
Apply that change preemptively to all comments (not necessarily just doc comments).
For #51082.
Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9
Reviewed-on: https://go-review.googlesource.com/c/go/+/384260
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
The code was updated, the comments were not.
Change-Id: If387779f3abd5e8a1b487fe34c33dcf9ce5fa7ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/383495
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes #49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
This CL fixes the subtle issue that Elem can promote a
not-in-heap pointer, which could be any bit pattern, into an
unsafe.Pointer, which the garbage collector can see. If that
resulting value is bad, it can crash the GC.
Make sure that we don't introduce bad pointers that way. We can
make Elem() panic, because any such bad pointers are in the Go heap,
and not-in-heap pointers are not allowed to point into the Go heap.
Update #48399
Change-Id: Ieaf35a611b16b4dfb5e907e229ed4a2aed30e18c
Reviewed-on: https://go-review.googlesource.com/c/go/+/350153
Trust: Keith Randall <khr@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
fastrandn is ~50% faster than fastrand() % n.
`ack -v 'fastrand\(\)\s?\%'` finds all modulo on fastrand()
name old time/op new time/op delta
Fastrandn/2 2.86ns ± 0% 1.59ns ± 0% -44.35% (p=0.000 n=9+10)
Fastrandn/3 2.87ns ± 1% 1.59ns ± 0% -44.41% (p=0.000 n=10+9)
Fastrandn/4 2.87ns ± 1% 1.58ns ± 1% -45.10% (p=0.000 n=10+10)
Fastrandn/5 2.86ns ± 1% 1.58ns ± 1% -44.84% (p=0.000 n=10+10)
Change-Id: Ic91f5ca9b9e3b65127bc34792b62fd64fbd13b5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/353269
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
[git-generate]
cd src/runtime
goimports -w *.go
Change-Id: I1387af0f2fd1a213dc2f4c122e83a8db0fcb15f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/329189
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
|
|
internal/goarch.PtrSize [generated]
[git-generate]
cd src/runtime/internal/math
gofmt -w -r "sys.PtrSize -> goarch.PtrSize" .
goimports -w *.go
cd ../..
gofmt -w -r "sys.PtrSize -> goarch.PtrSize" .
goimports -w *.go
Change-Id: I43491cdd54d2e06d4d04152b3d213851b7d6d423
Reviewed-on: https://go-review.googlesource.com/c/go/+/328337
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
|
|
Extend CL 310330 to ARM64, which now has clobberdead mode
implemented in the compiler.
Change-Id: I07f6951d81a0797ef7a74e48b79db5cea2bf876b
Reviewed-on: https://go-review.googlesource.com/c/go/+/324766
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
|
|
When -clobberdead compiler flag is set, the compiler inserts
instructions that set dead slots a specific value. If the GC sees
this value as a live pointer, something is probably wrong. Crash.
Only do this on AMD64 for now, as it is the only platform where
compiler's clobberdead mode is implemented. And on AMD64 the
clobberdead address can never be a valid address.
Change-Id: Ica687b132b5d3ba2a062500d13264fa730405d11
Reviewed-on: https://go-review.googlesource.com/c/go/+/310330
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
It's both simpler and faster to just unconditionally do two 32-bit
multiplies rather than a bunch of branching to try to avoid them.
This is safe thanks to the tight bounds derived in [1] and verified
during mksizeclasses.go.
Benchstat results below for compilebench benchmarks on my P920. See
also [2] for micro benchmarks comparing the new functions against the
originals (as well as several basic attempts at optimizing them).
name old time/op new time/op delta
Template 295ms ± 3% 290ms ± 1% -1.95% (p=0.000 n=20+20)
Unicode 113ms ± 3% 110ms ± 2% -2.32% (p=0.000 n=21+17)
GoTypes 1.78s ± 1% 1.76s ± 1% -1.23% (p=0.000 n=21+20)
Compiler 119ms ± 2% 117ms ± 4% -1.53% (p=0.007 n=20+20)
SSA 14.3s ± 1% 13.8s ± 1% -3.12% (p=0.000 n=17+20)
Flate 173ms ± 2% 170ms ± 1% -1.64% (p=0.000 n=20+19)
GoParser 278ms ± 2% 273ms ± 2% -1.92% (p=0.000 n=20+19)
Reflect 686ms ± 3% 671ms ± 3% -2.18% (p=0.000 n=19+20)
Tar 255ms ± 2% 248ms ± 2% -2.90% (p=0.000 n=20+20)
XML 335ms ± 3% 327ms ± 2% -2.34% (p=0.000 n=20+20)
LinkCompiler 799ms ± 1% 799ms ± 1% ~ (p=0.925 n=20+20)
ExternalLinkCompiler 1.90s ± 1% 1.90s ± 0% ~ (p=0.327 n=20+20)
LinkWithoutDebugCompiler 385ms ± 1% 386ms ± 1% ~ (p=0.251 n=18+20)
[Geo mean] 512ms 504ms -1.61%
name old user-time/op new user-time/op delta
Template 286ms ± 4% 282ms ± 4% -1.42% (p=0.025 n=21+20)
Unicode 104ms ± 9% 102ms ±14% ~ (p=0.294 n=21+20)
GoTypes 1.75s ± 3% 1.72s ± 2% -1.36% (p=0.000 n=21+20)
Compiler 109ms ±11% 108ms ± 8% ~ (p=0.187 n=21+19)
SSA 14.0s ± 1% 13.5s ± 2% -3.25% (p=0.000 n=16+20)
Flate 166ms ± 4% 164ms ± 4% -1.34% (p=0.032 n=19+19)
GoParser 268ms ± 4% 263ms ± 4% -1.71% (p=0.011 n=18+20)
Reflect 666ms ± 3% 654ms ± 4% -1.77% (p=0.002 n=18+20)
Tar 245ms ± 5% 236ms ± 6% -3.34% (p=0.000 n=20+20)
XML 320ms ± 4% 314ms ± 3% -2.01% (p=0.001 n=19+18)
LinkCompiler 744ms ± 4% 747ms ± 3% ~ (p=0.627 n=20+19)
ExternalLinkCompiler 1.71s ± 3% 1.72s ± 2% ~ (p=0.149 n=20+20)
LinkWithoutDebugCompiler 345ms ± 6% 342ms ± 8% ~ (p=0.355 n=20+20)
[Geo mean] 484ms 477ms -1.50%
[1] Daniel Lemire, Owen Kaser, Nathan Kurz. 2019. "Faster Remainder by
Direct Computation: Applications to Compilers and Software Libraries."
https://arxiv.org/abs/1902.01961
[2] https://github.com/mdempsky/benchdivmagic
Change-Id: Ie4d214e7a908b0d979c878f2d404bd56bdf374f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/300994
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
This change modifies mheap's span allocation API to have each caller
declare a purpose, defined as a new enum called spanAllocType.
The purpose behind this change is two-fold:
1. Tight control over who gets to allocate heap memory is, generally
speaking, a good thing. Every codepath that allocates heap memory
places additional implicit restrictions on the allocator. A notable
example of a restriction is work bufs coming from heap memory: write
barriers are not allowed in allocation paths because then we could
have a situation where the allocator calls into the allocator.
2. Memory statistic updating is explicit. Instead of passing an opaque
pointer for statistic updating, which places restrictions on how that
statistic may be updated, we use the spanAllocType to determine which
statistic to update and how.
We also take this opportunity to group all the statistic updating code
together, which should make the accounting code a little easier to
follow.
Change-Id: Ic0b0898959ba2a776f67122f0e36c9d7d60e3085
Reviewed-on: https://go-review.googlesource.com/c/go/+/246970
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
This CL introduces a 24 byte allocation size class which
fits 3 pointers on 64 bit and 6 pointers on 32 bit architectures.
Notably this new size class fits a slice header on 64 bit
architectures exactly while previously a 32 byte size class
would have been used for allocating a slice header on the heap.
The main complexity added with this CL is that heapBitsSetType
needs to handle objects that aren't 16-byte aligned but contain
more than a single pointer on 64-bit architectures.
Due to having a non 16 byte aligned size class on 32 bit a
h.shift of 2 is now possible which means a heap bitmap byte might
only be partially written. Due to this already having been
possible on 64 bit before the heap bitmap code only needed
minor adjustments for 32 bit doublecheck code paths.
Note that this CL changes the slice capacity allocated by append
for slice growth to a target capacity of 17 to 24 bytes.
On 64 bit architectures the capacity of the slice returned by
append([]byte{}, make([]byte, 24)...)) is 32 bytes before and
24 bytes after this CL. Depending on allocation patterns of the
specific Go program this can increase the number of total
alloctions as subsequent appends to the slice can trigger slice
growth earlier than before. On the other side if the slice is
never appended to again above its capacity this will lower heap
usage by 8 bytes.
This CL changes the set of size classes reported in the
runtime.MemStats.BySize array due to it being limited to a
total of 61 size classes. The new 24 byte size class is now
included and the 20480 byte size class is not included anymore.
Fixes #8885
name old time/op new time/op delta
Template 196ms ± 3% 194ms ± 2% ~ (p=0.247 n=10+10)
Unicode 85.6ms ±16% 88.1ms ± 1% ~ (p=0.165 n=10+10)
GoTypes 673ms ± 2% 668ms ± 2% ~ (p=0.258 n=9+9)
Compiler 3.14s ± 6% 3.08s ± 1% ~ (p=0.243 n=10+9)
SSA 6.82s ± 1% 6.76s ± 1% -0.87% (p=0.006 n=9+10)
Flate 128ms ± 7% 127ms ± 3% ~ (p=0.739 n=10+10)
GoParser 154ms ± 3% 153ms ± 4% ~ (p=0.730 n=9+9)
Reflect 404ms ± 1% 412ms ± 4% +1.99% (p=0.022 n=9+10)
Tar 172ms ± 4% 170ms ± 4% ~ (p=0.065 n=10+9)
XML 231ms ± 4% 230ms ± 3% ~ (p=0.912 n=10+10)
LinkCompiler 341ms ± 1% 339ms ± 1% ~ (p=0.243 n=9+10)
ExternalLinkCompiler 1.72s ± 1% 1.72s ± 1% ~ (p=0.661 n=9+10)
LinkWithoutDebugCompiler 221ms ± 2% 221ms ± 2% ~ (p=0.529 n=10+10)
StdCmd 18.4s ± 3% 18.2s ± 1% ~ (p=0.515 n=10+8)
name old user-time/op new user-time/op delta
Template 238ms ± 4% 243ms ± 6% ~ (p=0.661 n=9+10)
Unicode 116ms ± 6% 113ms ± 3% -3.37% (p=0.035 n=9+10)
GoTypes 854ms ± 2% 848ms ± 2% ~ (p=0.604 n=9+10)
Compiler 4.10s ± 1% 4.11s ± 1% ~ (p=0.481 n=8+9)
SSA 9.49s ± 1% 9.41s ± 1% -0.92% (p=0.001 n=9+10)
Flate 149ms ± 6% 151ms ± 7% ~ (p=0.481 n=10+10)
GoParser 189ms ± 2% 190ms ± 2% ~ (p=0.497 n=9+10)
Reflect 511ms ± 2% 508ms ± 2% ~ (p=0.211 n=9+10)
Tar 215ms ± 4% 212ms ± 3% ~ (p=0.105 n=10+10)
XML 288ms ± 2% 288ms ± 2% ~ (p=0.971 n=10+10)
LinkCompiler 559ms ± 4% 557ms ± 1% ~ (p=0.968 n=9+10)
ExternalLinkCompiler 1.78s ± 1% 1.77s ± 1% ~ (p=0.055 n=8+10)
LinkWithoutDebugCompiler 245ms ± 3% 245ms ± 2% ~ (p=0.684 n=10+10)
name old alloc/op new alloc/op delta
Template 34.8MB ± 0% 34.4MB ± 0% -0.95% (p=0.000 n=9+10)
Unicode 28.6MB ± 0% 28.3MB ± 0% -0.95% (p=0.000 n=10+10)
GoTypes 115MB ± 0% 114MB ± 0% -1.02% (p=0.000 n=10+9)
Compiler 554MB ± 0% 549MB ± 0% -0.86% (p=0.000 n=9+10)
SSA 1.28GB ± 0% 1.27GB ± 0% -0.83% (p=0.000 n=10+10)
Flate 21.8MB ± 0% 21.6MB ± 0% -0.87% (p=0.000 n=8+10)
GoParser 26.7MB ± 0% 26.4MB ± 0% -0.97% (p=0.000 n=10+9)
Reflect 75.0MB ± 0% 74.1MB ± 0% -1.18% (p=0.000 n=10+10)
Tar 32.6MB ± 0% 32.3MB ± 0% -0.94% (p=0.000 n=10+7)
XML 41.5MB ± 0% 41.2MB ± 0% -0.90% (p=0.000 n=10+8)
LinkCompiler 105MB ± 0% 104MB ± 0% -0.94% (p=0.000 n=10+10)
ExternalLinkCompiler 153MB ± 0% 152MB ± 0% -0.69% (p=0.000 n=10+10)
LinkWithoutDebugCompiler 63.7MB ± 0% 63.6MB ± 0% -0.13% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Template 336k ± 0% 336k ± 0% +0.02% (p=0.002 n=10+10)
Unicode 332k ± 0% 332k ± 0% ~ (p=0.447 n=10+10)
GoTypes 1.16M ± 0% 1.16M ± 0% +0.01% (p=0.001 n=10+10)
Compiler 4.92M ± 0% 4.92M ± 0% +0.01% (p=0.000 n=10+10)
SSA 11.9M ± 0% 11.9M ± 0% +0.02% (p=0.000 n=9+10)
Flate 214k ± 0% 214k ± 0% +0.02% (p=0.032 n=10+8)
GoParser 270k ± 0% 270k ± 0% +0.02% (p=0.004 n=10+9)
Reflect 877k ± 0% 877k ± 0% +0.01% (p=0.000 n=10+10)
Tar 313k ± 0% 313k ± 0% ~ (p=0.075 n=9+10)
XML 387k ± 0% 387k ± 0% +0.02% (p=0.007 n=10+10)
LinkCompiler 455k ± 0% 456k ± 0% +0.08% (p=0.000 n=10+9)
ExternalLinkCompiler 670k ± 0% 671k ± 0% +0.06% (p=0.000 n=10+10)
LinkWithoutDebugCompiler 113k ± 0% 113k ± 0% ~ (p=0.149 n=10+10)
name old maxRSS/op new maxRSS/op delta
Template 34.1M ± 1% 34.1M ± 1% ~ (p=0.853 n=10+10)
Unicode 35.1M ± 1% 34.6M ± 1% -1.43% (p=0.000 n=10+10)
GoTypes 72.8M ± 3% 73.3M ± 2% ~ (p=0.724 n=10+10)
Compiler 288M ± 3% 295M ± 4% ~ (p=0.393 n=10+10)
SSA 630M ± 1% 622M ± 1% -1.18% (p=0.001 n=10+10)
Flate 26.0M ± 1% 26.2M ± 2% ~ (p=0.493 n=10+10)
GoParser 28.6M ± 1% 28.5M ± 2% ~ (p=0.256 n=10+10)
Reflect 55.5M ± 2% 55.4M ± 1% ~ (p=0.436 n=10+10)
Tar 33.0M ± 1% 32.8M ± 2% ~ (p=0.075 n=10+10)
XML 38.7M ± 1% 39.0M ± 1% ~ (p=0.053 n=9+10)
LinkCompiler 164M ± 1% 164M ± 1% -0.27% (p=0.029 n=10+10)
ExternalLinkCompiler 174M ± 0% 173M ± 0% -0.33% (p=0.002 n=9+10)
LinkWithoutDebugCompiler 137M ± 0% 136M ± 2% ~ (p=0.825 n=9+10)
Change-Id: I9ecf2a10024513abef8fbfbe519e44e0b29b6167
Reviewed-on: https://go-review.googlesource.com/c/go/+/242258
Trust: Martin Möhrmann <moehrmann@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
Currently, the GC stores the object marks for checkmarks mode in the
heap bitmap using a rather complex encoding: for one word objects, the
checkmark is stored in the pointer/scalar bit since one word objects
must be pointers; for larger objects, the checkmark is stored in what
would be the scan/dead bit for the second word of the object. This
encoding made more sense when the runtime used the first scan/dead bit
as the regular mark bit, but we moved away from that long ago.
This encoding and overloading of the heap bitmap bits causes a great
deal of complexity in many parts of the allocator and garbage
collector and leads to some subtle bugs like #15903.
This CL moves the checkmarks mark bits into their own per-arena bitmap
and reclaims the second scan/dead bit as a regular scan/dead bit.
I tested this by enabling doubleCheck mode in heapBitsSetType and
running in both regular and GODEBUG=gccheckmark=1 mode.
Fixes #15903.
No performance degradation. (Very slight improvement on a few
benchmarks, but it's probably just noise.)
name old time/op new time/op delta
BiogoIgor 16.6s ± 1% 16.4s ± 1% -0.94% (p=0.000 n=25+24)
BiogoKrishna 19.2s ± 3% 19.2s ± 3% ~ (p=0.638 n=23+25)
BleveIndexBatch100 6.12s ± 5% 6.17s ± 4% ~ (p=0.170 n=25+25)
CompileTemplate 206ms ± 1% 205ms ± 1% -0.43% (p=0.005 n=24+24)
CompileUnicode 82.2ms ± 2% 81.5ms ± 2% -0.95% (p=0.001 n=22+22)
CompileGoTypes 755ms ± 3% 754ms ± 4% ~ (p=0.715 n=25+25)
CompileCompiler 3.73s ± 1% 3.73s ± 1% ~ (p=0.445 n=25+24)
CompileSSA 8.67s ± 1% 8.66s ± 1% ~ (p=0.836 n=24+22)
CompileFlate 134ms ± 2% 133ms ± 1% -0.66% (p=0.001 n=24+23)
CompileGoParser 164ms ± 1% 163ms ± 1% -0.85% (p=0.000 n=24+24)
CompileReflect 466ms ± 5% 466ms ± 3% ~ (p=0.863 n=25+25)
CompileTar 182ms ± 1% 182ms ± 1% -0.31% (p=0.048 n=24+24)
CompileXML 249ms ± 1% 248ms ± 1% -0.32% (p=0.031 n=21+25)
CompileStdCmd 10.3s ± 1% 10.3s ± 1% ~ (p=0.459 n=23+23)
FoglemanFauxGLRenderRotateBoat 8.66s ± 1% 8.62s ± 1% -0.47% (p=0.000 n=23+24)
FoglemanPathTraceRenderGopherIter1 20.3s ± 3% 20.2s ± 2% ~ (p=0.893 n=25+25)
GopherLuaKNucleotide 29.7s ± 1% 29.8s ± 2% ~ (p=0.421 n=24+25)
MarkdownRenderXHTML 246ms ± 1% 247ms ± 1% ~ (p=0.558 n=25+24)
Tile38WithinCircle100kmRequest 779µs ± 4% 779µs ± 3% ~ (p=0.954 n=25+25)
Tile38IntersectsCircle100kmRequest 1.02ms ± 3% 1.01ms ± 4% ~ (p=0.658 n=25+25)
Tile38KNearestLimit100Request 984µs ± 4% 986µs ± 4% ~ (p=0.627 n=24+25)
[Geo mean] 552ms 551ms -0.19%
https://perf.golang.org/search?q=upload:20200723.6
Change-Id: Ic703f26a83fb034941dc6f4788fc997d56890dec
Reviewed-on: https://go-review.googlesource.com/c/go/+/244539
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
|
|
The heapBitsSetType function has a slow doubleCheck debugging mode
that checks the bitmap written out by the rest of the function using
far more obvious logic. But even this has some surprisingly complex
logic in it. Simplify it a bit. This also happens to fix the logic on
32-bit.
Fixes #40335.
Change-Id: I5cee482ad8adbd01cf5b98e35a270fe941ba4940
Reviewed-on: https://go-review.googlesource.com/c/go/+/244538
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
This change modifies the implementation of (*mspan).countAlloc by
using OnesCount64 (which on many systems is intrinsified). It does so by
using an unsafe pointer cast, but in this case we don't care about
endianness because we're just counting bits set.
This change means we no longer need the popcnt table which was redundant
in the runtime anyway. We can also simplify the logic here significantly
by observing that mark bits allocations are always 8-byte aligned, so we
don't need to handle any edge-cases due to the fact that OnesCount64
operates on 64 bits at a time: all irrelevant bits will be zero.
Overall, this implementation is significantly faster than the old one on
amd64, and should be similarly faster (or better!) on other systems
which support the intrinsic. On systems which do not, it should be
roughly the same performance because OnesCount64 is implemented using a
table in the general case.
Results on linux/amd64:
name old time/op new time/op delta
MSpanCountAlloc/bits=64-4 16.8ns ± 0% 12.7ns ± 0% -24.40% (p=0.000 n=5+4)
MSpanCountAlloc/bits=128-4 23.5ns ± 0% 12.8ns ± 0% -45.70% (p=0.000 n=4+5)
MSpanCountAlloc/bits=256-4 43.5ns ± 0% 12.8ns ± 0% -70.67% (p=0.000 n=4+5)
MSpanCountAlloc/bits=512-4 59.5ns ± 0% 15.4ns ± 0% -74.12% (p=0.008 n=5+5)
MSpanCountAlloc/bits=1024-4 116ns ± 1% 23ns ± 0% -79.84% (p=0.000 n=5+4)
Change-Id: Id4c994be22224653af5333683a69b0937130ed04
Reviewed-on: https://go-review.googlesource.com/c/go/+/216558
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
materializeGCProg allocates a temporary buffer for unrolling a GC
program. Unfortunately, when computing the size of the buffer, it
rounds *down* the number of bytes needed to store bitmap before
rounding up the number of pages needed to store those bytes. The fact
that it rounds up to pages usually mitigates the rounding down, but
the type from #37470 exists right on the boundary where this doesn't
work:
type Sequencer struct {
htable [1 << 17]uint32
buf []byte
}
On 64-bit, this GC bitmap is exactly 8 KiB of zeros, followed by three
one bits. Hence, this needs 8193 bytes of storage, but the current
math in materializeGCProg rounds *down* the three one bits to 8192
bytes. Since this is exactly pageSize, the next step of rounding up to
the page size doesn't mitigate this error, and materializeGCProg
allocates a buffer that is one byte too small. runGCProg then writes
one byte past the end of this buffer, causing either a segfault (if
you're lucky!) or memory corruption.
Fixes #37470.
Change-Id: Iad24c463c501cd9b1dc1924bc2ad007991a094a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/221197
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
When everything is working correctly, any pointer the garbage
collector encounters can only point into a fully initialized heap
span, since the span must have been initialized before that pointer
could escape the heap allocator and become visible to the GC.
However, in various cases, we try to be defensive against bad
pointers. In findObject, this is just a sanity check: we never expect
to find a bad pointer, but programming errors can lead to them. In
spanOfHeap, we don't necessarily trust the pointer and we're trying to
check if it really does point to the heap, though it should always
point to something. Conservative scanning takes this to a new level,
since it can only guess that a word may be a pointer and verify this.
In all of these cases, we have a problem that the span lookup and
check can race with span initialization, since the span becomes
visible to lookups before it's fully initialized.
Furthermore, we're about to start initializing the span without the
heap lock held, which is going to introduce races where accesses were
previously protected by the heap lock.
To address this, this CL makes accesses to mspan.state atomic, and
ensures that the span is fully initialized before setting the state to
mSpanInUse. All loads are now atomic, and in any case where we don't
trust the pointer, it first atomically loads the span state and checks
that it's mSpanInUse, after which it will have synchronized with span
initialization and can safely check the other span fields.
For #10958, #24543, but a good fix in general.
Change-Id: I518b7c63555b02064b98aa5f802c92b758fef853
Reviewed-on: https://go-review.googlesource.com/c/go/+/203286
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Currently, several important fields of a heap span are set by
heapBits.initSpan, which happens after the span has already been
published and returned from the locked region of alloc_m. In
particular, allocBits is set very late, which makes mspan.isFree
unsafe even if you were to lock the heap because it tries to access
allocBits.
This CL fixes this by populating these fields in alloc_m. The next CL
builds on this to only publish the span once it is fully initialized.
Together, they'll make it safe to check allocBits even if there is a
race with alloc_m.
For #10958, #24543, but a good fix in general.
Change-Id: I7fde90023af0f497e826b637efa4d19c32840c08
Reviewed-on: https://go-review.googlesource.com/c/go/+/203285
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
findObject takes the pointer argument as uintptr. If the pointer is to
the local stack and calling findObject happens to require the stack to
be reallocated, then spanOf is called for the old pointer.
Marking findObject as nosplit fixes the issue.
Fixes #35068
Change-Id: I029d36f9c23f91812f18f98839edf02e0ba4082e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202798
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
|
|
This helps keeping findObject's frame small.
Updates #35068
Change-Id: I1b8c1fcc5831944c86f1a30ed2f2d867a5f2b242
Reviewed-on: https://go-review.googlesource.com/c/go/+/202797
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
|
|
Factor out case s == nil, make the code cleaner and easier to read.
Change-Id: I63f52e14351c0a0d20a611b1fe10fdc0d4947d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/202498
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
Those print statements are not a good debug helpers
and only clutter the code.
Change-Id: Ifbf450a04e6fa538af68e6352c016728edb4119a
Reviewed-on: https://go-review.googlesource.com/c/go/+/160537
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
|
|
We already have the ptrdata field in a type, which encodes exactly
the same information that kindNoPointers does.
My problem with kindNoPointers is that it often leads to
double-negative code like:
t.kind & kindNoPointers != 0
Much clearer is:
t.ptrdata == 0
Update #27167
Change-Id: I92307d7f018a6bbe3daca4a4abb4225e359349b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/169157
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Follow-up for CL 147037 and after Brad noticed the "returns whether"
pattern during the review of CL 150621.
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns whether")
Created with:
$ perl -i -npe 's/returns whether/reports whether/' $(git grep -l "returns whether" | grep -v vendor)
Change-Id: I15fe9ff99180ad97750cd05a10eceafdb12dc0b4
Reviewed-on: https://go-review.googlesource.com/c/150918
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns true if")
This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.
(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)
Created with:
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)
Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Since atomic.Or8 is now an intrinsic (and has been for some time),
markBits.setMarked is inlinable. Undo the manual inlining of it.
Change-Id: I8e37ccf0851ad1d3088d9c8ae0f6f0c439d7eb2d
Reviewed-on: https://go-review.googlesource.com/c/138659
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
The previous CL introduced stack objects. This CL removes the old
ambiguously live liveness analysis. After this CL we're relying
on stack objects exclusively.
Update a bunch of liveness tests to reflect the new world.
Fixes #22350
Change-Id: I739b26e015882231011ce6bc1a7f426049e59f31
Reviewed-on: https://go-review.googlesource.com/c/134156
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
Rework how the compiler+runtime handles stack-allocated variables
whose address is taken.
Direct references to such variables work as before. References through
pointers, however, use a new mechanism. The new mechanism is more
precise than the old "ambiguously live" mechanism. It computes liveness
at runtime based on the actual references among objects on the stack.
Each function records all of its address-taken objects in a FUNCDATA.
These are called "stack objects". The runtime then uses that
information while scanning a stack to find all of the stack objects on
a stack. It then does a mark phase on the stack objects, using all the
pointers found on the stack (and ancillary structures, like defer
records) as the root set. Only stack objects which are found to be
live during this mark phase will be scanned and thus retain any heap
objects they point to.
A subsequent CL will remove all the "ambiguously live" logic from
the compiler, so that the stack object tracing will be required.
For this CL, the stack tracing is all redundant with the current
ambiguously live logic.
Update #22350
Change-Id: Ide19f1f71a5b6ec8c4d54f8f66f0e9a98344772f
Reviewed-on: https://go-review.googlesource.com/c/134155
Reviewed-by: Austin Clements <austin@google.com>
|
|
We already aliased mSpanInUse to _MSpanInUse. The dual constants are
getting annoying, so fix all of these to use the mSpan* naming
convention.
This was done automatically with:
sed -i -re 's/_?MSpan(Dead|InUse|Manual|Free)/mSpan\1/g' *.go
plus deleting the existing definition of mSpanInUse.
Change-Id: I09979d9d491d06c10689cea625dc57faa9cc6767
Reviewed-on: https://go-review.googlesource.com/137875
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
The new slice created in growslice is cleared during malloc for
element types containing pointers and therefore can only contain
nil pointers. This change avoids executing write barriers for these
nil pointers by adding and using a special bulkBarrierPreWriteSrcOnly
function that does not enqueue pointers to slots in dst to the write
barrier buffer.
Change-Id: If9b18248bfeeb6a874b0132d19520adea593bfc4
Reviewed-on: https://go-review.googlesource.com/115996
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
|
