| Age | Commit message (Collapse) | Author |
|---|
|
When an application calls runtime.GOMAXPROCS(runtime.GOMAXPROCS(0)), the
runtime does not need to change the actual GOMAXPROCS value (via STW).
However, this call must still transition from "automatic" to "custom"
GOMAXPROCS state, thus disabling background updates.
Thus this case shouldn't return quite as early as it currently does.
Change-Id: I6a6a636c42f73996532bd9f7beb95e933256c9e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/683815
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
|
|
sysReserveAlignedSbrk locks memlock at entry, but it is not
unlocked at one of the return path. Add the missing unlock.
Fixes #74339.
Change-Id: Ib641bc348aca41494ec410e2c4eb9857f3362484
Reviewed-on: https://go-review.googlesource.com/c/go/+/683295
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Change-Id: I0c33830b13c8a187ac82504c7653abb8f8cf7530
Reviewed-on: https://go-review.googlesource.com/c/go/+/681655
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sean Liao <sean@liao.dev>
|
|
Currently the mspan limit field is set after allocSpan returns, *after*
the span has already been published to the GC (including the
conservative scanner). But the limit field is load-bearing, because it's
checked to filter out invalid pointers. A stale limit value could cause
a crash by having the conservative scanner access allocBits out of
bounds.
Fix this by setting the mspan limit field before publishing the span.
For large objects and arena chunks, we adjust the limit down after
allocSpan because we don't have access to the true object's size from
allocSpan. However this is safe, since we first initialize the limit to
something definitely safe (the actual span bounds) and only adjust it
down after. Adjusting it down has the benefit of more precise debug
output, but the window in which it's imprecise is also fine because a
single object (logically, with arena chunks) occupies the whole span, so
the 'invalid' part of the memory will just safely point back to that
object. We can't do this for smaller objects because the limit will
include space that does *not* contain any valid objects.
Fixes #74288.
Change-Id: I0a22e5b9bccc1bfdf51d2b73ea7130f1b99c0c7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/682655
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
Almost everywhere we stop the world we casGToWaitingForGC to prevent
mutual deadlock with the GC trying to scan our stack. This historically
was only necessary if we weren't stopping the world to change the GC
phase, because what we were worried about was mutual deadlock with mark
workers' use of suspendG. And, they were the only users of suspendG.
In Go 1.22 this changed. The execution tracer began using suspendG, too.
This leads to the possibility of mutual deadlock between the execution
tracer and a goroutine trying to start or end the GC mark phase. The fix
is simple: make the stop-the-world calls for the GC also call
casGToWaitingForGC. This way, suspendG is guaranteed to make progress in
this circumstance, and once it completes, the stop-the-world can
complete as well.
We can take this a step further, though, and move casGToWaitingForGC
into stopTheWorldWithSema, since there's no longer really a place we can
afford to skip this detail.
While we're here, rename casGToWaitingForGC to casGToWaitingForSuspendG,
since the GC is now not the only potential source of mutual deadlock.
Fixes #72740.
Change-Id: I5e3739a463ef3e8173ad33c531e696e46260692f
Reviewed-on: https://go-review.googlesource.com/c/go/+/681501
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
Issue #74045 describes a scenario in which gopark is inlined into
readTrace, such that there are no preemption points. This is only a
problem because readTrace spins if trace.shutdown is set, through
traceReaderAvailable. However, trace.shutdown is almost certainly
overkill for traceReaderAvailable. The first condition, checking whether
the reader gen and the flushed gen match, should be sufficient to ensure
the reader wakes up and finishes flushing all buffers. The first
condition is also safe because it guarantees progress. In the case of
shutdown, all the trace work that will be flushed has been flushed, and
so the trace reader will exit into a regular goroutine context when
it's finished. If not shutting down, then the trace reader will release
doneSema, increase readerGen, and then the gopark unlockf will let it
block until new work actually comes in.
Fixes #74045.
Change-Id: Id9b15c277cb731618488771bd484577341b68675
Reviewed-on: https://go-review.googlesource.com/c/go/+/680738
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
Change-Id: I118d1ee09dfd6fd0075f9e5eeeb54441328bff4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/681495
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
It was added in CL 650256, and then the use in the unique package
was removed in CL 650697.
Change-Id: Id95f5dff7e11a2dc3eb544fda2586a305d3d91ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/681476
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
We're going to start writing two files, so having a single global file
we're writing will be a problem.
This has no effect on the generated code.
Change-Id: I49897ea0c6500a29eac89b597d75c0eb3e9b6706
Reviewed-on: https://go-review.googlesource.com/c/go/+/680897
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Before CL 650697, there was only one system goroutine that could
dynamically change between being a user goroutine and a system
goroutine, and that was the finalizer/cleanup goroutine. In goroutine
profiles, it was handled explicitly. It's status would be checked during
the first STW, and its stack would be recorded. This let the goroutine
profiler completely ignore system goroutines once the world was started
again.
CL 650697 added dedicated cleanup goroutines (there may be more than
one), and with this, the logic for finalizer goroutines no longer
scaled. In that CL, I let the isSystemGoroutine check be dynamic and
dropped the special case, but this was based on incorrect assumptions.
Namely, it's possible for the scheduler to observe, for example, the
finalizer goroutine as a system goroutine and ignore it, but then later
the goroutine profiler itself sees it as a user goroutine. At that point
it's too late and already running. This violates the invariant of the
goroutine profile that all goroutines are handled by the profiler before
they start executing. In practice, the result is that the goroutine
profiler can crash when it checks this invariant (not checking the
invariant means racily reading goroutine stack memory).
The root cause of the problem is that these system goroutines do not
participate in the goroutine profiler's state machine. Normally, when
profiling, goroutines transition from 'absent' to 'in-progress' to
'satisfied'. However with system goroutines, the state machine is
ignored entirely. They always stay in the 'absent' state. This means
that if a goroutine transitions from system to user, it is eligible for
a profile record when it shouldn't be. That transition shouldn't be
allowed to occur with respect to the goroutine profiler, because the
goroutine profiler is trying to snapshot the state of every goroutine.
The fix to this problem is simple: don't ignore system goroutines. Let
them participate in the goroutine profile state machine. Instead, decide
whether or not to record the stack after the goroutine has been acquired
for goroutine profiling. This means if the scheduler observes the
finalizer goroutine as a system goroutine, it will get promoted in the
goroutine profiler's state machine, and no other part of the goroutine
profiler will observe the goroutine again. Simultaneously, the
stack record for the goroutine will be correctly skipped.
Fixes #74090.
Change-Id: Icb9a164a033be22aaa942d19e828e895f700ca74
Reviewed-on: https://go-review.googlesource.com/c/go/+/680477
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
We associate WaitGroups with synctest bubbles by attaching a
special to the WaitGroup. It is not possible to attach a special
to a linker-allocated value, such as:
var wg sync.WaitGroup
Avoid panicking when accessing a linker-allocated WaitGroup
from a bubble. We have no way to associate these WaitGroups
with a bubble, so just treat them as always unbubbled.
This is probably fine, since the WaitGroup was always
created outside the bubble in this case.
Fixes #74005
Change-Id: Ic71514b0b8d0cecd62e45cc929ffcbeb16f54a55
Reviewed-on: https://go-review.googlesource.com/c/go/+/679695
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
semrelease is safe to call on the system stack (since it just readies
goroutines) except for the fact that it might perform a direct G
handoff and call into the scheduler. If handoff is set to false this is
exceptionally rare, but could happen, and has happened for the trace
reader goroutine which releases a trace.doneSema.
Fixes #73469.
Change-Id: I37ece678bc4721bbb6e5879d74daac762b7d742a
Reviewed-on: https://go-review.googlesource.com/c/go/+/680315
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
Finalizers and cleanup funcs weren't running on the windows-arm64
builder. Put finalizers/cleanups on a small struct containing a pointer
rather than an *int, which fixes the problem.
Also uncomment a synctest.Wait that was accidentally commented out.
Fixes #73977
Change-Id: Ia6f18d74d6fccf2c5a9222317977c7458d67f158
Reviewed-on: https://go-review.googlesource.com/c/go/+/679696
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Use the synctest bubble ID to identify bubbles in traces,
rather than the goroutine ID of the bubble's root goroutine.
Some waitReasons include a "(synctest)" suffix to distinguish
a durably blocking state from a non-durable one. For example,
"chan send" vs. "chan send (synctest)". Change this suffix
to "(durable)".
Always print a "(durable)" sufix for the state of durably
blocked bubbled goroutines. For example, print "sleep (durable)".
Drop the "[not] durably blocked" text from goroutine states,
since this is now entirely redundant with the waitReason.
Old:
goroutine 8 [chan receive (synctest), synctest bubble 7, durably blocked]:
goroutine 9 [select (no cases), synctest bubble 7, durably blocked]:
New:
goroutine 8 [chan receive (durable), synctest bubble 1]:
goroutine 9 [select (no cases) (durable), synctest bubble 1]:
Change-Id: I89112efb25150a98a2954f54d1910ccec52a5824
Reviewed-on: https://go-review.googlesource.com/c/go/+/679376
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
The synctest.Test function waits for all goroutines in a bubble to
exit before returning. If there is ever a point when all goroutines
in a bubble are durably blocked, it panics and reports a deadlock.
Panic with a different message depending on whether the bubble's
main goroutine has returned or not. The main goroutine returning
stops the bubble clock, so knowing whether it is running or not
is useful debugging information.
The new panic messages are:
deadlock: all goroutines in bubble are blocked
deadlock: main bubble goroutine has exited but blocked goroutines remain
Change-Id: I94a69e79121c272d9c86f412c1c9c7de57ef27ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/679375
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This issue has been fixed for amd64, arm64 and other platforms
in CL 643875, but it was missed when the race support was
submitted for loong64.
Fixes #71395.
Change-Id: I678f381e868214f1b3399be43187db49e1660933
Reviewed-on: https://go-review.googlesource.com/c/go/+/679055
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
This CL makes two changes to reduce the predictability
with which bubbled timers fire.
When asynctimerchan=0 (the default), regular timers with an associated
channel are only added to a timer heap when some channel operation
is blocked on that channel. This allows us to garbage collect
unreferenced, unstopped timers. Timers in a synctest bubble, in
contrast, are always added to the bubble's timer heap.
This CL changes bubbled timers with a channel to be handled the
same as unbubbled ones, adding them to the bubble's timer heap only
when some channel operation is blocked on the timer's channel.
This permits unstopped bubbled timers to be garbage collected,
but more importantly it makes all timers past their deadline
behave identically, regardless of whether they are in a bubble.
This CL also changes timer scheduling to execute bubbled timers
immediately when possible rather than adding them to a heap.
Timers in a bubble's heap are executed when the bubble is idle.
Executing timers immediately avoids creating a predictable
order of execution.
For #73850
Fixes #73934
Change-Id: If82e441546408f780f6af6fb7f6e416d3160295d
Reviewed-on: https://go-review.googlesource.com/c/go/+/678075
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
There are two additional sources of memory overhead per P that come from
greenteagc. One is for ptrBuf, but on platforms other than Windows it
doesn't actually cost anything due to demand-paging (Windows also
demand-pages, but the memory is 'committed' so it still counts against
OS RSS metrics). The other is for per-sizeclass scan stats. However when
greenteagc is disabled, most of these scan stats are completely unused.
The worst-case memory overhead from these two sources is relatively
small (about 10 KiB per P), but for programs with a small memory
footprint running on a machine with a lot of cores, this can be
significant (single-digit percent).
This change does two things. First, it puts ptrBuf initialization behind
the greenteagc experiment, so now that memory is never allocated by
default. Second, it abstracts the implementation details of scan stat
collection and emission, such that we can have two different
implementations depending on the build tag. This lets us remove all the
unused stats when the greenteagc experiment is disabled, reducing the
memory overhead of the stats from ~2.6 KiB per P to 536 bytes per P.
This is enough to make the difference no longer noticable in our
benchmark suite.
Fixes #73931.
Change-Id: I4351f1cbb3f6743d8f5922d757d73442c6d6ad3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/678535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
For testing out duffcopy changes.
Change-Id: I93b4a52d75418a6e31aae5ad99f95d1870812b69
Reviewed-on: https://go-review.googlesource.com/c/go/+/678215
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
|
|
In synctest bubbles, fire timers scheduled for the same instant
in a randomized order.
Pending timers are added to a heap ordered by the timer's wakeup time.
Add a per-timer random value, set when the timer is added to a heap,
to break ties between timers scheduled for the same instant.
Only inject this randomness in synctest bubbles. We could do so
for all timers at the cost of one cheaprand call per timer,
but given that it's effectively impossible to create two timers
scheduled for the same instant outside of a fake-time environment,
don't bother.
Fixes #73876
For #73850
Change-Id: Ie96c86a816f548d4c31e4e014bf9293639155bd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/677276
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
When the GC is disabled, the tracer should emit a heap goal of 0. Not
setting the heap goal to 0 causes an inaccurate NextGC value to be
emmited.
Fixes #63864
Change-Id: Iecceaca86c0a43c1cc4d9433f1f9bb736f01ccbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/639417
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
sync.Cond.Wait is durably blocking. Waking a goroutine out of Cond.Wait
from outside its bubble panics.
Make this panic a fatal panic, since it leaves the notifyList in an
inconsistent state. We could do some work to make this a recoverable
panic, but the complexity doesn't seem worth the outcome.
For #67434
Change-Id: I88874c1519c2e5c0063175297a9b120cedabcd07
Reviewed-on: https://go-review.googlesource.com/c/go/+/675617
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
vgetrandomGetState can call malloc, so this is not a leaf lock.
Our staticlockrank builder doesn't support vgetrandom, so it didn't
catch this.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-staticlockranking
Change-Id: I6a6a636c36c9172e4ebf9493c10cb23cac29a13f
Reviewed-on: https://go-review.googlesource.com/c/go/+/677255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
We already guarantee that no automatic updates to GOMAXPROCS occur after
a GOMAXPROCS call returns. This is easily achieved by having the update
goroutine double-check that updates are still allowed during STW before
committing the new value.
However, it is possible for sysmon to concurrently run defaultGOMAXPROCS
to compute a new GOMAXPROCS value after GOMAXPROCS returns. This new
value will be discarded later, but we'll still perform the system calls
necessary to compute the new value.
Normally this distinction doesn't matter, but if you want to sandbox a
Go program, then you may want to disable GOMAXPROCS updates to reduce
the system call footprint. A call to GOMAXPROCS will disable updates,
but without a guarantee on when sysmon will observe the change it is
somewhat fragile.
Add explicit synchronization between GOMAXPROCS and sysmon to guarantee
that sysmon won't run defaultGOMAXPROCS after GOMAXPROCS returns.
The synchronization is a bit complex because we can't hold a mutex
across STW, nor take a semaphore from sysmon, but the result isn't too
bad.
One oddity is that sched.customGOMAXPROCS and gomaxprocs are no longer
updated in lockstep (even though both are protected by sched.lock), but
I don't believe anything should depend on that.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-staticlockranking
Change-Id: I6a6a636cff243a9b69ac1b5d2f98925648e60236
Reviewed-on: https://go-review.googlesource.com/c/go/+/677037
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Add support to internal/synctest for managing associations between
arbitrary pointers and synctest bubbles. (Implemented internally to
the runtime package by attaching a special to the pointer.)
Associate WaitGroups with bubbles.
Since WaitGroups don't have a constructor,
perform the association when Add is called.
All Add calls must be made from within the same bubble,
or outside any bubble.
When a bubbled goroutine calls WaitGroup.Wait,
the wait is durably blocking iff the WaitGroup is associated
with the current bubble.
Change-Id: I77e2701e734ac2fa2b32b28d5b0c853b7b2825c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/676656
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
The updatemaxprocs metric logic is currently backwards. We only
increment the metric when we update GOMAXPROCS, but that only occurs if
updatemaxprocs is enabled.
Instead, the metric is supposed to increment when updatemaxprocs is
disabled and there would be different behavior if it were enabled.
Theoretically we should run the entire update system in a dry run mode,
and only bail out right before committing updates. But that is an awful
lot of effort for a feature that is disabled. Plus some users (like
sandboxes) want to completely disable the update syscalls
(sched_getaffinity and pread64). If we still do dry run updates then we
need an additional GODEBUG for completely disabling functionality.
This CL also avoids starting the update goroutine at all if disabled,
since it isn't needed.
For #73193.
Change-Id: I6a6a636ceec8fced44e36cb27dcb1b4ba51fce33
Reviewed-on: https://go-review.googlesource.com/c/go/+/677036
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
|
|
There are other parts to updating GOMAXPROCS than just the helper
goroutine, so make the naming more specific.
For #73193.
Change-Id: I6a6a636c31ac80c8d76afe90c0bfc29d3086af4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/677035
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Cleanup functions and finalizers must not run in a synctest bubble.
If they did, a function run by the GC at an unpredictable time
could unblock a bubble that synctest believes is durably
blocked.
Add a test verifying that cleanups and finalizers are always
run by non-bubbled goroutines. (This is already the case because
we never add system goroutines to a bubble.)
For #67434
Change-Id: I5a48db2b26f9712c3b0dc1f425d99814031a2fc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/675257
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
Fixes #73817
Change-Id: I0101bdc797237b4c7eb58b414c71b009b0b44447
Reviewed-on: https://go-review.googlesource.com/c/go/+/675176
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
This adds a test which validates the traces generated by the execution
tracer and the flight recorder depending on the order where they are
stopped and started. This test uncovered that under certain
circumstances, the traces which were produced would possibly be
missing the trace header. All traces have the trace headers included
now. Clock snapshot checks have been disabled for Windows and WASM.
Change-Id: I5be719d228300469891fc56817fbce4ba5453fff
Reviewed-on: https://go-review.googlesource.com/c/go/+/675975
Auto-Submit: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
cleanupQueue.Flush is reachable from mallocgc via sweepAssist. Normally
allp will continue all valid Ps, but procresize itself increases the
size of allp and then allocates new Ps to place in allp. If we get
perfectly unlucky, the new(p) allocations will complete sweeping and
cleanupQueue.Flush will dereference a nil pointer from allp. Avoid this
by skipping nil Ps.
I've looked through every other use of allp and none of them appear to
be reachable from procresize.
Change-Id: I6a6a636cab49ef268eb8fcd9ff9a96790d9c5685
Reviewed-on: https://go-review.googlesource.com/c/go/+/676515
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
runtime.traceClockNow returns a (named) uint64. Make the declaration in
runtime/trace match this type.
Change-Id: I6a6a636ce3596cbc6fc5bac3590703b7b4839c4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/675976
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This change fixes the flaky test which expects setting SetMinAge to a
small ammount. It expects two sync events but should realistically
expect up to 3.
Change-Id: Ibd02fe55ebca99eb880025eb968fcebae9cb09c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/675597
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
CL 674655 modified the checkfinalizers test to spin looking for an
appropriate address to trip the detector, but this doesn't work with
ASAN or in race mode, which both disable the tiny allocator.
Fixes #73834.
Change-Id: I27416da1f29cd953271698551e9ce9724484c683
Reviewed-on: https://go-review.googlesource.com/c/go/+/675395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
Currently weak pointers break in sbrk mode. We can just use the immortal
weak handle map for weak pointers in this case, since nothing is ever
freed.
Fixes #69729.
Change-Id: Ie9fa7e203c22776dc9eb3601c6480107d9ad0c99
Reviewed-on: https://go-review.googlesource.com/c/go/+/674656
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
|
|
This change adds the flight recorder to the trace package.
Flight recording is a technique in which trace data is kept
in a circular buffer and can be flushed upon request. The
implementation will be added in follow-up CLs.
The flight recorder has already been implemented inside of the
golang.org/x/exp/trace package. This copies the current implementation
and modifies it to work within the runtime/trace package.
The changes include:
This adds the ability for multiple consumers (both the execution
tracer and the flight recorder) to subscribe to tracing events. This
change allows us to add multiple consumers without making major
modifications to the runtime. Future optimizations are planned
for this functionality.
This removes the use of byte readers from the process that
parses and processes the trace batches.
This modifies the flight recorder to not parse out the trace
clock frequency, since that requires knowledge of the format that's
unfortunate to encode in yet another place. Right now, the trace clock
frequency is considered stable for the lifetime of the program, so just
grab it directly from the runtime.
This change adds an in-band end-of-generation signal to the internal
implementation of runtime.ReadTrace. The internal implementation is
exported via linkname to runtime/trace, so the flight recorder can
identify exactly when a generation has ended. This signal is also useful
for ensuring that subscribers to runtime trace data always see complete
generations, by starting or stopping data streaming only at generation
boundaries.
For #63185
Change-Id: I5c15345981a6bbe9764a3d623448237e983c64ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/673116
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Fixes #71746
Change-Id: I6a6a46568b092933d8ac2039df99ee9f0edf6e56
Reviewed-on: https://go-review.googlesource.com/c/go/+/674477
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This implements RFC 9155 by removing support for SHA-1 algorithms:
- we don't advertise them in ClientHello and CertificateRequest
(where supportedSignatureAlgorithms is used directly)
- we don't select them in our ServerKeyExchange and CertificateVerify
(where supportedSignatureAlgorithms filters signatureSchemesForCertificate)
- we reject them in the peer's ServerKeyExchange and CertificateVerify
(where we check against the algorithms we advertised in ClientHello
and CertificateRequest)
Fixes #72883
Change-Id: I6a6a4656e2aafd2c38cdd32090d3d8a9a8047818
Reviewed-on: https://go-review.googlesource.com/c/go/+/658216
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
|
|
This test asserts there is no external code, but the sanitizer runtimes
are external code.
Fixes #73783.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race
Change-Id: I6a6a636cf93b7950e3ea35e00ec2eaf89911d712
Reviewed-on: https://go-review.googlesource.com/c/go/+/675296
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This is a typo in CL 670497. The test is using the wrong testprog
function.
The testprog also needs to assert that GOMAXPROCS doesn't change, not
that it is equal to NumCPU, for the GOMAXPROCS=4 case.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: I6a6a636cab6936aa8519e3553b70ab6641ca8010
Reviewed-on: https://go-review.googlesource.com/c/go/+/675097
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
These became race instrumented in CL 643897, but race mode uses more
memory, so the test doesn't make much sense.
For #71395.
Change-Id: I6a6a636cf09ba29625aa9a22550314845fb2e611
Reviewed-on: https://go-review.googlesource.com/c/go/+/675077
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This is a regression in CL 643875. Loading gsignal clobbers R8, which
contains the m pointer needed for loading g0.
For #71395.
Change-Id: I6a6a636ca95442767efe0eb1b358f2139d18c5b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/675035
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
This CL adds two related features enabled by default via compatibility
GODEBUGs containermaxprocs and updatemaxprocs.
On Linux, containermaxprocs makes the Go runtime consider cgroup CPU
bandwidth limits (quota/period) when setting GOMAXPROCS. If the cgroup
limit is lower than the number of logical CPUs available, then the
cgroup limit takes precedence.
On all OSes, updatemaxprocs makes the Go runtime periodically
recalculate the default GOMAXPROCS value and update GOMAXPROCS if it has
changed. If GOMAXPROCS is set manually, this update does not occur. This
is intended primarily to detect changes to cgroup limits, but it applies
on all OSes because the CPU affinity mask can change as well.
The runtime only considers the limit in the leaf cgroup (the one that
actually contains the process), caching the CPU limit file
descriptor(s), which are periodically reread for updates. This is a
small departure from the original proposed design. It will not consider
limits of parent cgroups (which may be lower than the leaf), and it will
not detection cgroup migration after process start.
We can consider changing this in the future, but the simpler approach is
less invasive; less risk to packages that have some awareness of runtime
internals. e.g., if the runtime periodically opens new files during
execution, file descriptor leak detection is difficult to implement in a
stable way.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: I6a6a636c631c1ae577fb8254960377ba91c5dc98
Reviewed-on: https://go-review.googlesource.com/c/go/+/670497
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
For #73193.
Change-Id: I6a6a636ca9fa9cba429cf053468c56c2939cb1ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/668638
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Add build tag gated Valgrind annotations to the runtime which let it
understand how the runtime manages memory. This allows for Go binaries
to be run under Valgrind without emitting spurious errors.
Instead of adding the Valgrind headers to the tree, and using cgo to
call the various Valgrind client request macros, we just add an assembly
function which emits the necessary instructions to trigger client
requests.
In particular we add instrumentation of the memory allocator, using a
two-level mempool structure (as described in the Valgrind manual [0]).
We also add annotations which allow Valgrind to track which memory we
use for stacks, which seems necessary to let it properly function.
We describe the memory model to Valgrind as follows: we treat heap
arenas as a "pool" created with VALGRIND_CREATE_MEMPOOL_EXT (so that we
can use VALGRIND_MEMPOOL_METAPOOL and VALGRIND_MEMPOOL_AUTO_FREE).
Within the pool we treat spans as "superblocks", annotated with
VALGRIND_MEMPOOL_ALLOC. We then allocate individual objects within spans
with VALGRIND_MALLOCLIKE_BLOCK.
It should be noted that running binaries under Valgrind can be _quite
slow_, and certain operations, such as running the GC, can be _very
slow_. It is recommended to run programs with GOGC=off. Additionally,
async preemption should be turned off, since it'll cause strange
behavior (GODEBUG=asyncpreemptoff=1).
Running Valgrind with --leak-check=yes will result in some errors
resulting from some things not being marked fully free'd. These likely
need more annotations to rectify, but for now it is recommended to run
with --leak-check=off.
Updates #73602
[0] https://valgrind.org/docs/manual/mc-manual.html#mc-manual.mempools
Change-Id: I71b26c47d7084de71ef1e03947ef6b1cc6d38301
Reviewed-on: https://go-review.googlesource.com/c/go/+/674077
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Currently the checkfinalizers test (TestDetectCleanupOrFinalizerLeak)
only *tries* to ensure the tiny alloc with a cleanup attached shares a
block with other objects. However, what it does is insufficient, because
it could get unlucky and have the last object allocated be the first
object of a new block.
This change changes the test to guarantee that a tiny object is not at
the start of a fresh block by looking at the alignment of the object's
pointer. If the object's pointer is odd, then that's good enough to know
that it shares a block with something else, since the blocks themselves
are aligned to a much higher power of two.
This fixes a failure I've seen on the builders.
Fixes #73810.
Change-Id: Ieafdbb9cccb0d2dc3659a9a5d9d9233718461635
Reviewed-on: https://go-review.googlesource.com/c/go/+/674655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
Replace the per-generation EvEventBatch containing a lone EvFrequency
event with a per-generation EvEventBatch containing a EvSync header
followed by an EvFrequency and EvClockSnapshot event.
The new EvClockSnapshot event contains trace, mono and wall clock
snapshots taken in close time proximity. Ignoring minor resolution
differences, the trace and mono clock are the same on linux, but not on
windows (which still uses a TSC based trace clock).
Emit the new sync batch at the very beginning of every new generation
rather than the end to be in harmony with the internal/trace reader
which emits a sync event at the beginning of every generation as well
and guarantees monotonically increasing event timestamps.
Bump the version of the trace file format to 1.25 since this change is
not backwards compatible.
Update the internal/trace reader implementation to decode the new
events, but do not expose them to the public reader API yet. This is
done in the next CL.
For #69869
Change-Id: I5bfedccdd23dc0adaf2401ec0970cbcc32363393
Reviewed-on: https://go-review.googlesource.com/c/go/+/653575
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Should fix some asan build failures.
Change-Id: Ic0a816b56a1a278aa0ad541aea962f9fea7b10fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/674696
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
|
|
Replace the hchan.synctest bool with an hchan.bubble reference
to the synctest bubble that created the chan. I originally used
a bool to avoid increasing the size of hchan, but we have space
in hchan's current size class for another pointer.
This lets us detect one bubble operating on a chan created
in a different bubble.
For #67434
Change-Id: If6cf9ffcb372fe7fb3f8f4ef27b664848578ba5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/674515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
A chan created within a synctest bubble may not be
operated on from outside the bubble.
We panicked on send and receive, but not close.
Panic on close as well.
For #67434
Change-Id: I98d39e0cf7baa1a679aca1fb325453d69c535308
Reviewed-on: https://go-review.googlesource.com/c/go/+/671960
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
|
