| Age | Commit message (Collapse) | Author |
|
The tracefpunwindoff GODEBUG applies to all the places where we use
frame pointer unwinding. Originally that was just the execution tracer,
but now we also use it for the block and mutex profilers.
Change-Id: I6a6a69641fede668d96335582dba6d43e62f6a51
Reviewed-on: https://go-review.googlesource.com/c/go/+/718522
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
pagetrace functionality was removed with CL 583379.
Change-Id: I8e8718e6cf5415e326ec127fb294588866ee4e6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/713260
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
|
|
Fixes #72949.
Change-Id: I114eda73c57bc7d596eb1656e738b80c1cbe5254
Reviewed-on: https://go-review.googlesource.com/c/go/+/662039
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Go 1.22 promised to remove the setting in a future release once the
semantics of runtime-internal lock contention matched that of
sync.Mutex. That work is done, remove the setting.
Previously reviewed as https://go.dev/cl/585639.
For #66999
Change-Id: I9fe62558ba0ac12824874a0bb1b41efeb7c0853f
Reviewed-on: https://go-review.googlesource.com/c/go/+/668995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
This adds a new godebug to control whether the runtime applies the
anonymous memory mapping annotations added in https://go.dev/cl/646095.
It is enabled by default.
This has several effects:
* The feature is only enabled by default when the main go.mod has go >=
1.25.
* This feature can be disabled with GODEBUG=decoratemappings=0, or the
equivalents in go.mod or package main. See https://go.dev/doc/godebug.
* As an opaque setting, this option will not appear in runtime/metrics.
* This setting is non-atomic, so it cannot be changed after startup.
I am not 100% sure about my decision for the last two points.
I've made this an opaque setting because it affects every memory mapping
the runtime performs. Thus every mapping would report "non-default
behavior", which doesn't seem useful.
This setting could trivially be atomic and allow changes at run time,
but those changes would only affect future mappings. That seems
confusing and not helpful. On the other hand, going back to annotate or
unannotate every previous mapping when the setting changes is
unwarranted complexity.
For #71546.
Change-Id: I6a6a636c5ad551d76691cba2a6f668d5cff0e352
Reviewed-on: https://go-review.googlesource.com/c/go/+/655895
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
|
|
Document that Caller and Frame.File always use forward slashes
as path separators, even on Windows.
Fixes #3335
Change-Id: Ic5bbf8a1f14af64277dca4783176cd8f70726b91
Reviewed-on: https://go-review.googlesource.com/c/go/+/603275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
Fixes #51473.
Change-Id: Ic868790f480b21b472c98b39be797bcffb1fa892
Reviewed-on: https://go-review.googlesource.com/c/go/+/564142
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This reverts commit 87e930f7289136fad1310d4b63dd4127e409bac5 (CL 585639)
Reason for revert: This is part of a patch series that changed the
handling of contended lock2/unlock2 calls, reducing the maximum
throughput of contended runtime.mutex values, and causing a performance
regression on applications where that is (or became) the bottleneck.
Updates #66999
Updates #67585
Change-Id: I1e286d2a16d16e4af202cd5dc04b2d9c4ee71b32
Reviewed-on: https://go-review.googlesource.com/c/go/+/589097
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
|
|
Go 1.22 promised to remove the setting in a future release once the
semantics of runtime-internal lock contention matched that of
sync.Mutex. That work is done, remove the setting.
For #66999
Change-Id: I3c4894148385adf2756d8754e44d7317305ad758
Reviewed-on: https://go-review.googlesource.com/c/go/+/585639
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
Allow users to decrease the profiling stack depth back to 32 in case
they experience any problems with the new default of 128.
Users may also use this option to increase the depth up to 1024.
Change-Id: Ieaab2513024915a223239278dd97a6e161dde1cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/581917
Reviewed-by: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
allocfreetrace prints all allocations and frees to stderr. It's not
terribly useful because it has a really huge overhead, making it not
feasible to use except for the most trivial programs. A follow-up CL
will replace it with something that is both more thorough and also lower
overhead.
Change-Id: I1d668fee8b6aaef5251a5aea3054ec2444d75eb6
Reviewed-on: https://go-review.googlesource.com/c/go/+/583376
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
Goids are designed to be big enough that they will never be reused:
a uint64 is enough to generate a new goroutine every nanosecond
for 500+ years before wrapping around, and after 500 years you
should probably stop and pick up some security updates.
This note was added in CL 70993 and appears to have just been
a misunderstanding by the CL author.
Change-Id: Ida7099b5191a4e5dbb1e3e9e44b4b86d7779fd6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/582895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
|
|
rpc was an array prior to CL 152537, so it was necessary to slice
it since callers accepts a slice. Now that rpc is already a slice,
slicing it is no longer required.
Change-Id: Ie646ef5e494323c9fb58f3a24f942e3b1ff639ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/579016
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
|
|
Currently, the execution tracer may attempt to take a stack trace of a
goroutine whose stack it does not own. For example, if the goroutine is
in _Grunnable or _Gwaiting. This is easily fixed in all cases by simply
moving the emission of GoStop and GoBlock events to before the
casgstatus happens. The goroutine status is what is used to signal stack
ownership, and the GC may shrink a goroutine's stack if it can acquire
the scan bit.
Although this is easily fixed, the interaction here is very subtle,
because stack ownership is only implicit in the goroutine's scan status.
To make this invariant more maintainable and less error-prone in the
future, this change adds a GODEBUG setting that checks, at the point of
taking a stack trace, whether the caller owns the goroutine. This check
is not quite perfect because there's no way for the stack tracing code
to know that the _Gscan bit was acquired by the caller, so for
simplicity it assumes that it was the caller that acquired the scan bit.
In all other cases however, we can check for ownership precisely. At the
very least, this check is sufficient to catch the issue this change is
fixing.
To make sure this debug check doesn't bitrot, it's always enabled during
trace testing. This new mode has actually caught a few other issues
already, so this change fixes them.
One issue that this debug mode caught was that it's not safe to take a
stack trace of a _Gwaiting goroutine that's being unparked.
Another much bigger issue this debug mode caught was the fact that the
execution tracer could try to take a stack trace of a G that was in
_Gwaiting solely to avoid a deadlock in the GC. The execution tracer
already has a partial list of these cases since they're modeled as the
goroutine just executing as normal in the tracer, but this change takes
the list and makes it more formal. In this specific case, we now prevent
the GC from shrinking the stacks of goroutines in this state if tracing
is enabled. The stack traces from these scenarios are too useful to
discard, but there is indeed a race here between the tracer and any
attempt to shrink the stack by the GC.
Change-Id: I019850dabc8cede202fd6dcc0a4b1f16764209fb
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race
Reviewed-on: https://go-review.googlesource.com/c/go/+/573155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
Updates #25448
Change-Id: Ia1b7a376f5175f67e14ad4bd065d6e8ad5250d38
Reviewed-on: https://go-review.googlesource.com/c/go/+/536955
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>
|
|
Change-Id: I8fe66326994894b17ce0eda991bba942844d26b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/541475
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
profileruntimelocks is new in CL 544195, but the name is deceptive. Even
with profileruntimelocks=0, runtime-internal locks are still profiled.
The actual difference is that call stacks are not collected. Instead all
contention is reported at runtime._LostContendedLock.
Rename this setting to runtimecontentionstacks to make its name more
aligned with its behavior.
In addition, for this release the default is profileruntimelocks=0,
meaning that users are fairly likely to encounter
runtime._LostContendedLock. Rename it to
runtime._LostContendedRuntimeLock in an attempt to make it more
intuitive that these are runtime locks, not locks in application code.
For #57071.
Change-Id: I38aac28b2c0852db643d53b1eab3f3bc42a43393
Reviewed-on: https://go-review.googlesource.com/c/go/+/547055
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
|
|
Go 1.21.1 and Go 1.22 have ceased working around an issue with Linux
kernel defaults for transparent huge pages that can result in excessive
memory overheads. (https://bugzilla.kernel.org/show_bug.cgi?id=93111)
Many Linux distributions disable huge pages altogether these days, so
this problem isn't quite as far-reaching as it used to be. Also, the
problem only affects Go programs with very particular memory usage
patterns.
That being said, because the runtime used to actively deal with this
problem (but with some unpredictable behavior), it's preventing users
that don't have a lot of control over their execution environment from
upgrading to Go beyond Go 1.20.
This change adds a GODEBUG to smooth over the transition. The GODEBUG
setting disables transparent huge pages for all heap memory on Linux,
which is much more predictable than restoring the old behavior.
Fixes #64332.
Change-Id: I73b1894337f0f0b1a5a17b90da1221e118e0b145
Reviewed-on: https://go-review.googlesource.com/c/go/+/547475
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Add runtime-internal locks to the mutex contention profile.
Store up to one call stack responsible for lock contention on the M,
until it's safe to contribute its value to the mprof table. Try to use
that limited local storage space for a relatively large source of
contention, and attribute any contention in stacks we're not able to
store to a sentinel _LostContendedLock function.
Avoid ballooning lock contention while manipulating the mprof table by
attributing to that sentinel function any lock contention experienced
while reporting lock contention.
Guard collecting real call stacks with GODEBUG=profileruntimelocks=1,
since the available data has mixed semantics; we can easily capture an
M's own wait time, but we'd prefer for the profile entry of each
critical section to describe how long it made the other Ms wait. It's
too late in the Go 1.22 cycle to make the required changes to
futex-based locks. When not enabled, attribute the time to the sentinel
function instead.
Fixes #57071
This is a roll-forward of https://go.dev/cl/528657, which was reverted
in https://go.dev/cl/543660
Reason for revert: de-flakes tests (reduces dependence on fine-grained
timers, correctly identifies contention on big-endian futex locks,
attempts to measure contention in the semaphore implementation but only
uses that secondary measurement to finish the test early, skips tests on
single-processor systems)
Change-Id: I31389f24283d85e46ad9ba8d4f514cb9add8dfb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/544195
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Rhys Hiltner <rhys@justin.tv>
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
|
|
This reverts commit go.dev/cl/528657.
Reason for revert: broke a lot of builders.
Change-Id: I70c33062020e997c4df67b3eaa2e886cf0da961e
Reviewed-on: https://go-review.googlesource.com/c/go/+/543660
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Add runtime-internal locks to the mutex contention profile.
Store up to one call stack responsible for lock contention on the M,
until it's safe to contribute its value to the mprof table. Try to use
that limited local storage space for a relatively large source of
contention, and attribute any contention in stacks we're not able to
store to a sentinel _LostContendedLock function.
Avoid ballooning lock contention while manipulating the mprof table by
attributing to that sentinel function any lock contention experienced
while reporting lock contention.
Guard collecting real call stacks with GODEBUG=profileruntimelocks=1,
since the available data has mixed semantics; we can easily capture an
M's own wait time, but we'd prefer for the profile entry of each
critical section to describe how long it made the other Ms wait. It's
too late in the Go 1.22 cycle to make the required changes to
futex-based locks. When not enabled, attribute the time to the sentinel
function instead.
Fixes #57071
Change-Id: I3eee0ccbfc20f333b56f20d8725dfd7f3a526b41
Reviewed-on: https://go-review.googlesource.com/c/go/+/528657
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Rhys Hiltner <rhys@justin.tv>
Reviewed-by: Than McIntosh <thanm@google.com>
|
|
This change mostly implements the design described in #60773 and
includes a new scalable parser for the new trace format, available in
internal/trace/v2. I'll leave this commit message short because this is
clearly an enormous CL with a lot of detail.
This change does not hook up the new tracer into cmd/trace yet. A
follow-up CL will handle that.
For #60773.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race
Change-Id: I5d2aca2cc07580ed3c76a9813ac48ec96b157de0
Reviewed-on: https://go-review.googlesource.com/c/go/+/494187
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Change-Id: Ifb4844efddcb0369b0302eeab72394eeaf5c8072
Reviewed-on: https://go-review.googlesource.com/c/go/+/540022
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: shuang cui <imcusg@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
There's more I could list here, but the mapping becomes more complicated
and more fragile. I think this is sufficient to start with.
Fixes #46846.
Change-Id: I6803486a64888b2f38e244b8e2175ad064d648ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/504115
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
GODEBUG=dontfreezetheworld=1 allows goroutines to continue execution
during fatal panic. This increases the chance that tracebackothers will
encounter running goroutines that it must skip, which is expected and
fine. However, it also introduces the risk that a goroutine transitions
from stopped to running in the middle of traceback, which is unsafe and
may cause traceback crashes.
Mitigate this by halting M execution if it naturally enters the
scheduler. This ensures that goroutines cannot transition from stopped
to running after freezetheworld. We simply deadlock rather than using
gcstopm to continue keeping disturbance to scheduler state to a minimum.
Change-Id: I9aa8d84abf038ae17142f34f4384e920b1490e81
Reviewed-on: https://go-review.googlesource.com/c/go/+/501255
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
On Unix platforms, the runtime previously did nothing special when a
program was run with either the SUID or SGID bits set. This can be
dangerous in certain cases, such as when dumping memory state, or
assuming the status of standard i/o file descriptors.
Taking cues from glibc, this change implements a set of protections when
a binary is run with SUID or SGID bits set (or is SUID/SGID-like). On
Linux, whether to enable these protections is determined by whether the
AT_SECURE flag is passed in the auxiliary vector. On platforms which
have the issetugid syscall (the BSDs, darwin, and Solaris/Illumos), that
is used. On the remaining platforms (currently only AIX) we check
!(getuid() == geteuid() && getgid == getegid()).
Currently when we determine a binary is "tainted" (using the glibc
terminology), we implement two specific protections:
1. we check if the file descriptors 0, 1, and 2 are open, and if they
are not, we open them, pointing at /dev/null (or fail).
2. we force GOTRACKBACK=none, and generally prevent dumping of
trackbacks and registers when a program panics/aborts.
In the future we may add additional protections.
This change requires implementing issetugid on the platforms which
support it, and implementing getuid, geteuid, getgid, and getegid on
AIX.
Thanks to Vincent Dehors from Synacktiv for reporting this issue.
Fixes #60272
Fixes CVE-2023-29403
Change-Id: I73fc93f2b7a8933c192ce3eabbf1db359db7d5fa
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1878434
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/501223
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
Also, clean up atomics on released-per-cycle while we're here.
For #57069.
Change-Id: I14026e8281f01dea1e8c8de6aa8944712b7b24d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/495916
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
GOTRACEBACK=wer is a new traceback level that acts as "crash" and
also enables WER. The same effect can be achieved using
debug.SetTraceback("wer").
The Go runtime currently crashes using exit(2), which bypasses WER
even if it is enabled. To best way to trigger WER is calling
RaiseFailFastException [1] instead, which internally launches the
WER machinery.
This CL also changes how GOTRACEBACK=crash crashes, so both "wer" and
"crash" crash using RaiseFailFastException, which simplifies the
implementation and resolves a longstanding TODO.
Fixes #57441
Fixes #20498
[1] https://learn.microsoft.com/en-us/windows/win32/api/errhandlingapi/nf-errhandlingapi-raisefailfastexception
Change-Id: I45669d619fbbd2f6413ce5e5f08425ed1d9aeb64
Reviewed-on: https://go-review.googlesource.com/c/go/+/474915
Reviewed-by: Davis Goodin <dagood@microsoft.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
|
|
Change tracer to use frame pointer unwinding by default on amd64. The
expansion of inline frames is delayed until the stack table is dumped at
the end of the trace. This requires storing the skip argument in the
stack table, which now resides in pcBuf[0]. For stacks that are not
produced by traceStackID (e.g. CPU samples), a logicalStackSentinel
value in pcBuf[0] indicates that no inline expansion is needed.
Add new GODEBUG=tracefpunwindoff=1 option to use the old unwinder if
needed.
Benchmarks show a considerable decrease in CPU overhead when using frame
pointer unwinding for trace events:
GODEBUG=tracefpunwindoff=1 ../bin/go test -run '^$' -bench '.+PingPong' -count 20 -v -trace /dev/null ./runtime | tee tracefpunwindoff1.txt
GODEBUG=tracefpunwindoff=0 ../bin/go test -run '^$' -bench '.+PingPong' -count 20 -v -trace /dev/null ./runtime | tee tracefpunwindoff0.txt
goos: linux
goarch: amd64
pkg: runtime
cpu: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
│ tracefpunwindoff1.txt │ tracefpunwindoff0.txt │
│ sec/op │ sec/op vs base │
PingPongHog-32 3782.5n ± 0% 740.7n ± 2% -80.42% (p=0.000 n=20)
For #16638
Change-Id: I2928a2fcd8779a31c45ce0f2fbcc0179641190bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/463835
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
This GODEBUG flag disables the freezetheworld call during fatal panic.
freezetheworld asks the scheduler to stop running goroutines on all Ms.
This is normally useful, as it ensures we can collect a traceback from
every goroutine. However, it can be frustrating when debugging the
scheduler itself, as it significantly changes the scheduler state from
when the panic started.
Setting this flag has some disadvantages. Most notably, running
goroutines will not traceback in the standard output (though they may be
included in the final SIGQUIT loop). Additionally, we may missing
concurrently created goroutines when looping over allgs (CL 270861 made
this safe, but still racy). The final state of all goroutines will also
be further removed from the time of panic, as they continued to run for
a while.
One unfortunate part of this flag is the final SIGQUIT loop in the
runtime leaves every thread in the signal handler at exit. This is a bit
frustrating in gdb, which doesn't understand how to step beyond
sigtramp. The data is still there, but you must manually walk.
Change-Id: Ie6bd3ac521fcababea668196b60cf225a0be1a00
Reviewed-on: https://go-review.googlesource.com/c/go/+/478975
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
|
|
Move this knob from a binary-startup thing to a build-time thing.
This will enable followon optmizations to the write barrier.
Change-Id: Ic3323348621c76a7dc390c09ff55016b19c43018
Reviewed-on: https://go-review.googlesource.com/c/go/+/447778
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
This change adds a new GODEBUG flag called pagetrace that writes a
low-overhead trace of how pages of memory are managed by the Go runtime.
The page tracer is kept behind a GOEXPERIMENT flag due to a potential
security risk for setuid binaries.
Change-Id: I6f4a2447d02693c25214400846a5d2832ad6e5c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/444157
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
Like on Linux, use GODEBUG=madvdontneed=1 to enable madvise with
MADV_DONTNEED instead of MADV_FREE.
Based on previous work by Marcelo Araujo in CL 181498.
Fixes #32519
Change-Id: Ib49faf05c42a65591b90e75f663146f213030529
Reviewed-on: https://go-review.googlesource.com/c/go/+/266937
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
|
|
This change updates the gctrace docs to include stacks and globals in
the format line, and prints lastStackScan for "# MB stacks" instead of
maxStackScan, which is more accurate.
Fixes #54649.
Change-Id: Ibff2c390c9c9bf2b24b5b4e98ca346cc98d7cb2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/425366
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
So far was only documented in doc/diagnostics.
Fixes: #54183
Change-Id: Iab67a1500547f30325453c614a2f2f6621dfadb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/423275
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
|
|
For #48409.
Change-Id: Ia6616a377bc4c871b7ffba6f5a59792a09b64809
Reviewed-on: https://go-review.googlesource.com/c/go/+/410734
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Chris Hines <chris.cs.guy@gmail.com>
Reviewed-by: Russ Cox <rsc@golang.org>
|
|
For #48409.
Change-Id: I056afcdbc417ce633e48184e69336213750aae28
Reviewed-on: https://go-review.googlesource.com/c/go/+/406575
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
|
|
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]
Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.
For #51082.
Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
For #44167.
Change-Id: I2dcd13cbe74e88de00e9fc51f9bd86e604a167df
Reviewed-on: https://go-review.googlesource.com/c/go/+/399300
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
This change adds a new debug flag that makes the runtime map pages
PROT_NONE in sysUnused on Linux, in addition to the usual madvise calls.
This behavior mimics the behavior of decommit on Windows, and is helpful
in debugging the scavenger. sysUsed is also updated to re-map the pages
as PROT_READ|PROT_WRITE, mimicing Windows' explicit commit behavior.
Change-Id: Iaac5fcd0e6920bd1d0e753dd4e7f0c0b128fe842
Reviewed-on: https://go-review.googlesource.com/c/go/+/356612
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
crypto/tls briefly used GODEBUG. That usage was removed in CL 191999.
Change-Id: I759b6f1b02db8160075cba30d73823018e19ad9d
GitHub-Last-Rev: 12d2a4a82b1467e4c2214aa78eb9a0af4938a9de
GitHub-Pull-Request: golang/go#49012
Reviewed-on: https://go-review.googlesource.com/c/go/+/356313
Reviewed-by: Austin Clements <austin@google.com>
Trust: Michael Pratt <mpratt@google.com>
|
|
Refactoring done by rf tool:
rf 'ex . {
import "internal/goos"
import "runtime/internal/sys"
sys.GOOS -> goos.GOOS
}'
Change-Id: I4b4aadff8640731ce4cb9bdad9954c267eb484c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/328339
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
|
|
Refactoring performed by the rf tool:
rf 'ex . {
import "internal/goarch"
import "runtime/internal/sys"
sys.GOARCH -> goarch.GOARCH
}'
Change-Id: I4b0246bf4e734f08313c6fff7b547db362057714
Reviewed-on: https://go-review.googlesource.com/c/go/+/328338
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
|
|
This adds the set of GOEXPERIMENTs to the build version if it differs
from the default set of experiments. This exposes the experiment
settings via runtime.Version() and "go version <binary>".
Change-Id: I143dbbc50f66a4cf175469199974e18848075af6
Reviewed-on: https://go-review.googlesource.com/c/go/+/307820
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
|
|
Change-Id: Ic4c7b5086303c7faa49f4cbf6738e66d5de35c7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/282012
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
|
|
The default GOROOT has nothing to do with system details.
Move it next to its one use in package runtime.
This CL is part of a stack adding windows/arm64
support (#36439), intended to land in the Go 1.17 cycle.
This CL is, however, not windows/arm64-specific.
It is cleanup meant to make the port (and future ports) easier.
Change-Id: I1a601fad6335336b4616b834bb21bd8437ee1313
Reviewed-on: https://go-review.googlesource.com/c/go/+/288796
Trust: Russ Cox <rsc@golang.org>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
In Go 1.12, we changed the runtime to use MADV_FREE when available on
Linux (falling back to MADV_DONTNEED) in CL 135395 to address issue
#23687. While MADV_FREE is somewhat faster than MADV_DONTNEED, it
doesn't affect many of the statistics that MADV_DONTNEED does until
the memory is actually reclaimed under OS memory pressure. This
generally leads to poor user experience, like confusing stats in top
and other monitoring tools; and bad integration with management
systems that respond to memory usage.
We've seen numerous issues about this user experience, including
#41818, #39295, #37585, #33376, and #30904, many questions on Go
mailing lists, and requests for mechanisms to change this behavior at
run-time, such as #40870. There are also issues that may be a result
of this, but root-causing it can be difficult, such as #41444 and
#39174. And there's some evidence it may even be incompatible with
Android's process management in #37569.
This CL changes the default to prefer MADV_DONTNEED over MADV_FREE, to
favor user-friendliness and minimal surprise over performance. I think
it's become clear that Linux's implementation of MADV_FREE ultimately
doesn't meet our needs. We've also made many improvements to the
scavenger since Go 1.12. In particular, it is now far more prompt and
it is self-paced, so it will simply trickle memory back to the system
a little more slowly with this change. This can still be overridden by
setting GODEBUG=madvdontneed=0.
Fixes #42330 (meta-issue).
Fixes #41818, #39295, #37585, #33376, #30904 (many of which were
already closed as "working as intended").
Change-Id: Ib6aa7f2dc8419b32516cc5a5fc402faf576c92e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/267100
Trust: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Setting inittrace=1 causes the runtime to emit a single line to standard error for
each package with init work, summarizing the execution time and memory allocation.
The emitted debug information for init functions can be used to find bottlenecks
or regressions in Go startup performance.
Packages with no init function work (user defined or compiler generated) are omitted.
Tracing plugin inits is not supported as they can execute concurrently. This would
make the implementation of tracing more complex while adding support for a very rare
use case. Plugin inits can be traced separately by testing a main package importing
the plugins package imports explicitly.
$ GODEBUG=inittrace=1 go test
init internal/bytealg @0.008 ms, 0 ms clock, 0 bytes, 0 allocs
init runtime @0.059 ms, 0.026 ms clock, 0 bytes, 0 allocs
init math @0.19 ms, 0.001 ms clock, 0 bytes, 0 allocs
init errors @0.22 ms, 0.004 ms clock, 0 bytes, 0 allocs
init strconv @0.24 ms, 0.002 ms clock, 32 bytes, 2 allocs
init sync @0.28 ms, 0.003 ms clock, 16 bytes, 1 allocs
init unicode @0.44 ms, 0.11 ms clock, 23328 bytes, 24 allocs
...
Inspired by stapelberg@google.com who instrumented doInit
in a prototype to measure init times with GDB.
Fixes #41378
Change-Id: Ic37c6a0cfc95488de9e737f5e346b8dbb39174e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/254659
Trust: Martin Möhrmann <moehrmann@google.com>
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
Change-Id: Ia06b6be262922991bae3528e7b061d1db9e4c3c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/236679
Reviewed-by: Austin Clements <austin@google.com>
|
|
This change removes the concept of s.scavAddr in favor of explicitly
reserving and unreserving address ranges. s.scavAddr has several
problems with raciness that can cause the scavenger to miss updates, or
move it back unnecessarily, forcing future scavenge calls to iterate
over searched address space unnecessarily.
This change achieves this by replacing scavAddr with a second addrRanges
which is cloned from s.inUse at the end of each sweep phase. Ranges from
this second addrRanges are then reserved by scavengers (with the
reservation size proportional to the heap size) who are then able to
safely iterate over those ranges without worry of another scavenger
coming in.
Fixes #35788.
Change-Id: Ief01ae170384174875118742f6c26b2a41cbb66d
Reviewed-on: https://go-review.googlesource.com/c/go/+/208378
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Austin Clements <austin@google.com>
|