aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/pprof
AgeCommit message (Collapse)Author
2022-09-23runtime/pprof: force use of 4-column profiles in pprof memprofile outputRuss Cox
Pprof's converter from legacy text format to protobuf format assumes that if the alloc and inuse stats are equal, then what's really going on is that the program makes no distinction, and it reads them as a two-column profile: objects and bytes. Most of the time, some sampled object has been freed, and alloc != inuse. In that case, pprof reads the profile as a four-column profile, with alloc_objects, alloc_bytes, inuse_objects, inuse_bytes. The 2-column form causes problems in a few ways. One is that if you are reading the proto form and expect samples with the 4-column names, they're not there. Another is that pprof's profile merger insists on having the same number of columns and same names. This means that pprof *.memprofile works most of the time but fails if one of the memory profiles hit the unlikely condition that alloc == inuse, since now its converted form differs from the others. Most programs should simply not be using this output form at all, but cmd/compile and cmd/link still do, because x/tools/cmd/compilebench reads some extra values from the text form that we have not yet added to the proto form. For the programs still writing this form, the easiest way to avoid the column collapse issues is to ensure that the header never reports alloc == inuse. The actual values in the header are ignored by pprof now, except for the equality check (they should sum to the other values in the file, so they are technically redundant). Because the actual values are not used except for the equality check, we could hard-code different values like 0 and 1, but just in case, to break as little as possible, this CL only adjusts the values when they would otherwise be equal. In that case it adds 1 to allocBytes. For most profiles, where alloc != inuse already, there is no effect at all. Change-Id: Ia563e402573d0f6eb81ae496645db27c08f9fe31 Reviewed-on: https://go-review.googlesource.com/c/go/+/432758 Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org>
2022-09-19runtime/pprof: set labelMap lengthSasha Melentyev
Change-Id: If09094e72161f2c5da9102706781524e32f87782 GitHub-Last-Rev: 89949bc6ee8622be31c4e4db45ea6da11fd9053f GitHub-Pull-Request: golang/go#54855 Reviewed-on: https://go-review.googlesource.com/c/go/+/428234 Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-09-16runtime: make the wait reason for a g blocked on a mutex more specificMichael Anthony Knyszek
This change adds 3 new waitReasons that correspond to sync.Mutex.Lock, sync.RWMutex.RLock, and sync.RWMutex.Lock that are plumbed down into semacquire1 by exporting new functions to the sync package from the runtime. Currently these three functions show up as "semacquire" in backtraces which isn't very clear, though the stack trace itself should reveal what's really going on. This represents a minor improvement to backtrace readability, though blocking on an RWMutex.w.Lock will still show up as blocking on a regular mutex (I suppose technically it is). This is a step toward helping the runtime identify when a goroutine is blocked on a mutex of some kind. For #49881. Change-Id: Ia409b4d27e117fe4bfdc25fa541e9c58d6d587b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/427616 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-09-08runtime/pprof: use strings.Buildercuiweixie
Change-Id: I0407d96e2ba1376cc33fe91b52b6a8d7e81f59ae Reviewed-on: https://go-review.googlesource.com/c/go/+/428277 Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-09-03internal/syscall/unix: consolidate kernelVersion implementationsKir Kolyshkin
Currently, there are 3 functions returning Linux kernel version numbers. Two of them are identical: - in net, initially added by commit 0a9dd47dd817904e; - in internal/poll, initially added by commit 1c7650aa93bd53; (both were later fixed by commit 66c02645062561a). The third one is a more complex, regexp-based implementation in runtime/pprof, which is only used for a test. Instead of adding one more, let's consolidate existing ones. Remove the complex implementation, and move the simple one into internal/syscall/unix. Use it from all the three places mentioned above. Change-Id: I4a34d9ca47257743c16def30e4dd634e36056091 Reviewed-on: https://go-review.googlesource.com/c/go/+/424896 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Kirill Kolyshkin <kolyshkin@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-23runtime/pprof: remove round-to-file-start adjustmentKeith Randall
This causes a problem in the test sometimes. With a mapping like: 00400000-00411000 r--p 00000000 fe:01 4459044 /tmp/go-build1710804385/b001/pprof.test 00411000-00645000 r-xp 00011000 fe:01 4459044 /tmp/go-build1710804385/b001/pprof.test The removed code would make the first mapping 0x400000-0x645000. Tests then grab the first few addresses to use as PCs, thinking they are in an executable range. But those addresses are really not in an executable range, causing the tests to fail. Change-Id: I5a69d0259d1fd70ff9745df1cbad4d54c5898e7b Reviewed-on: https://go-review.googlesource.com/c/go/+/424295 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Keith Randall <khr@golang.org>
2022-08-20runtime/pprof: add memory mapping info for WindowsEgon Elbre
Fixes #43296 Change-Id: Ib277c2e82c95f71a7a9b7fe1b22215ead7a54a88 Reviewed-on: https://go-review.googlesource.com/c/go/+/416975 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Than McIntosh <thanm@google.com>
2022-08-19runtime/pprof: check Getrusage return value in addMaxRSSTobias Klauser
Depending on the implementation of the getrusage syscall/function, the value of rusage.Maxrss may be undefined in case of an error. Thus, only report MaxRSS in case of no error. Change-Id: I7572ccc53c49eb460e53bded3eb41736eed8d2ed Reviewed-on: https://go-review.googlesource.com/c/go/+/424815 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Joedian Reid <joedian@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-08-19runtime/pprof: report MaxRSS on windowsTobias Klauser
Use GetProcessMemoryInfo to report MaxRSS in memory profiles on windows. Change-Id: I4ac5fe58961b1d5da8a5c1caa8a6e3d0a3281837 Reviewed-on: https://go-review.googlesource.com/c/go/+/424414 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-12runtime: run "gofmt -s -w"Cuong Manh Le
Change-Id: I7eb3de35d1f1f0237962735450b37d738966f30c Reviewed-on: https://go-review.googlesource.com/c/go/+/423254 Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-05-17all: fix spellingJohn Bampton
Change-Id: I68538a50c22b02cdb5aa2a889f9440fed7b94c54 GitHub-Last-Rev: aaac9e78340ac482e9cd1b506a035f271c29648c GitHub-Pull-Request: golang/go#52944 Reviewed-on: https://go-review.googlesource.com/c/go/+/406835 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>
2022-05-16runtime/pprof: slow new goroutine launches in testRhys Hiltner
The goroutine profiler tests include one that launches a steady stream of goroutines. That creates a scheduler busy loop that can prevent forward progress in the rest of the program. Slow down the launches a bit so other goroutines have a chance to run. Fixes #52916 For #52934 Change-Id: I748557201b94918b1fa4960544a51a48d9cacc6b Reviewed-on: https://go-review.googlesource.com/c/go/+/406654 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-05-16runtime/pprof: eliminate arbitrary deadline in testCPUProfileBryan C. Mills
The testCPUProfile helper function iterates until the profile contains enough samples. However, in general very slow builders may need longer to complete tests, and may have less-responsive schedulers (leading to longer durations required to collect profiles with enough samples). To compensate, slower builders generally run tests with longer timeouts. Since this test helper already dynamically scales the profile duration based on the collected samples, allow it to continue to retry and rescale until it would exceed the test's deadline. Fixes #52656 (hopefully). Change-Id: I4561e721927503f33a6d23336efa979bb9d3221f Reviewed-on: https://go-review.googlesource.com/c/go/+/406614 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>
2022-05-13runtime/pprof: fix allFrames cacheRhys Hiltner
The compiler may choose to inline multiple layers of function call, such that A calling B calling C may end up with all of the instructions for B and C written as part of A's function body. Within that function body, some PCs will represent code from function A. Some will represent code from function B, and for each of those the runtime will have an instruction attributable to A that it can report as its caller. Others will represent code from function C, and for each of those the runtime will have an instruction attributable to B and an instruction attributable to A that it can report as callers. When a profiling signal arrives at an instruction in B (as inlined in A) that the runtime also uses to describe calls to C, the profileBuilder ends up with an incorrect cache of allFrames results. That PC should lead to a location record in the profile that represents the frames B<-A, but the allFrames cache's view should expand the PC only to the B frame. Otherwise, when a profiling signal arrives at an instruction in C (as inlined in B in A), the PC stack C,B,A can get expanded to the frames C,B<-A,A as follows: The inlining deck starts empty. The first tryAdd call proposes PC C and frames C, which the deck accepts. The second tryAdd call proposes PC B and, due to the incorrect caching, frames B,A. (A fresh call to allFrames with PC B would return the frame list B.) The deck accepts that PC and frames. The third tryAdd call proposes PC A and frames A. The deck rejects those because a call from A to A cannot possibly have been inlined. This results in a new location record in the profile representing the frames C<-B<-A (good), as called by A (bad). The bug is the cached expansion of PC B to frames B<-A. That mapping is only appropriate for the resulting protobuf-format profile. The cache needs to reflect the results of a call to allFrames, which expands the PC B to the single frame B. For #50996 For #52693 Fixes #52764 Change-Id: I36d080f3c8a05650cdc13ced262189c33b0083b0 Reviewed-on: https://go-review.googlesource.com/c/go/+/404995 Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-05-03runtime/pprof: stress test goroutine profilerRhys Hiltner
For #33250 Change-Id: Ic7aa74b1bb5da9c4319718bac96316b236cb40b2 Reviewed-on: https://go-review.googlesource.com/c/go/+/387414 Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
2022-05-03runtime/pprof: add race annotations for goroutine profilesRhys Hiltner
The race annotations for goroutine label maps covered the special type of read necessary to create CPU profiles. Extend that to include goroutine profiles. Annotate the copy involved in creating new goroutines. Fixes #50292 Change-Id: I10f69314e4f4eba85c506590fe4781f4d6b8ec2d Reviewed-on: https://go-review.googlesource.com/c/go/+/385660 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Austin Clements <austin@google.com>
2022-04-11all: gofmt main repoRuss Cox
[This CL is part of a sequence implementing the proposal #51082. The design doc is at https://go.dev/s/godocfmt-design.] Run the updated gofmt, which reformats doc comments, on the main repository. Vendored files are excluded. For #51082. Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407 Reviewed-on: https://go-review.googlesource.com/c/go/+/384268 Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-05all: separate doc comment from //go: directivesRuss Cox
A future change to gofmt will rewrite // Doc comment. //go:foo to // Doc comment. // //go:foo Apply that change preemptively to all comments (not necessarily just doc comments). For #51082. Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9 Reviewed-on: https://go-review.googlesource.com/c/go/+/384260 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01all: remove trailing blank doc comment linesRuss Cox
A future change to gofmt will rewrite // Doc comment. // func f() to // Doc comment. func f() Apply that change preemptively to all doc comments. For #51082. Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d Reviewed-on: https://go-review.googlesource.com/c/go/+/384259 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-22runtime/pprof: rerun magnitude test on failureRhys Hiltner
Restructure TestCPUProfileMultithreadMagnitude so it will run again with a longer duration on failure. Log the split between the user vs system CPU time that rusage reports. For #50232 Change-Id: Ice5b38ee7594dbee1eaa5686d32b968c306e3e85 Reviewed-on: https://go-review.googlesource.com/c/go/+/393934 Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Knyszek <mknyszek@google.com>
2022-03-16runtime/pprof: do not require a GOROOT/src prefix in testsBryan C. Mills
When paths are trimmed, the reported file locations begin with the package import path (not GOROOT/src). Updates #51461 Change-Id: Idbd408a02e8d03329d10e30b0b08263e69e66285 Reviewed-on: https://go-review.googlesource.com/c/go/+/391812 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-13runtime/pprof: use syscall.RUSAGE_SELFTobias Klauser
Change-Id: Idc37429de5a48e708eda868ca7fa26b28620bac0 Reviewed-on: https://go-review.googlesource.com/c/go/+/391854 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-11runtime/pprof, syscall: report MaxRSS on all unix platformsTobias Klauser
All unix platforms currently supported by Go provide the getrusage syscall. On aix and solaris the Getrusage syscall wrapper is not available yet, so add and use it to report MaxRSS in memory profiles. Change-Id: Ie880a3058171031fd2e12ccf9adfb85ce18858b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/391434 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-09runtime/pprof: fix pcDeck's frame indexingRhys Hiltner
When building the inlining deck, correctly identify which is the last frame in the deck. Otherwise, when some forms of inlining cause a PC to expand to multiple frames, the length of the deck's two slices will diverge. Fixes #51567 Change-Id: I24e7ba32cb16b167f4307178b3f03c29e5362c4b Reviewed-on: https://go-review.googlesource.com/c/go/+/391134 Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Than McIntosh <thanm@google.com>
2022-03-08runtime/pprof: check if PC is reused for inliningRhys Hiltner
When describing call stacks that include inlined function calls, the runtime uses "fake" PCs to represent the frames that inlining removed. Those PCs correspond to real NOP instructions that the compiler inserts for this purpose. Describing the call stack in a protobuf-formatted profile requires the runtime/pprof package to collapse any sequences of fake call sites back into single PCs, removing the NOPs but retaining their line info. But because the NOP instructions are part of the function, they can appear as leaf nodes in a CPU profile. That results in an address that should sometimes be ignored (when it appears as a call site) and that sometimes should be present in the profile (when it is observed consuming CPU time). When processing a PC address, consider it first as a fake PC to add to the current inlining deck, and then as a previously-seen (real) PC. Fixes #50996 Change-Id: I80802369978bd7ac9969839ecfc9995ea4f84ab4 Reviewed-on: https://go-review.googlesource.com/c/go/+/384239 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-02-10runtime/pprof: remove arbitrary sleeps in TestBlockProfileBryan C. Mills
The "block" helpers in TestBlockProfile previously slept for an arbitrary duration and assumed that that duration was long enough for the parent goroutine to have registered as blocking. However — especially on slow or overloaded builders — the current arbitrary duration is sometimes not quite long enough. Rather than increasing the duration to a different arbitrary value (which would make the test slower but not actually eliminate the possibility of flakes!), we can use the runtime's own accounting to detect when the goroutine is actually blocked: we obtain a goroutine dump from the runtime, and assume that blocking has been registered in the profile only if the runtime shows the test goroutine in the appropriate blocked state. That not only makes the test more reliable, but also makes it significantly lower-latency when run on a fast machine. Fixes #6999 Fixes #37844 Change-Id: I465ed2afd406fd2b621419e1f06925f283525f25 Reviewed-on: https://go-review.googlesource.com/c/go/+/384534 Trust: Bryan Mills <bcmills@google.com> Trust: Benny Siegert <bsiegert@gmail.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-01-21runtime/pprof: TestLabelSystemstack parallelLabelHog.func1 must be labeledMichael Pratt
The closure in parallelLabelHog should be labeled in a addition to parallelLabelHog itself. Generally samples on that goroutine land on labelHog, but there is a small portion of the closure outside of labelHog. Fixes #50740. Change-Id: I363b6d8eec2e6920c215686e2039fce6d5b29a98 Reviewed-on: https://go-review.googlesource.com/c/go/+/380055 Reviewed-by: Bryan Mills <bcmills@google.com> Trust: Michael Pratt <mpratt@google.com>
2022-01-20runtime/pprof: allow labels on racecall in TestLabelSystemstackMichael Pratt
Fixes #50705. Change-Id: I85857f836cbe58447625df6cd56756d3a69880ff Reviewed-on: https://go-review.googlesource.com/c/go/+/379834 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-19runtime/pprof: compare all samples vs rusage in ↵Michael Pratt
TestCPUProfileMultithreadMagnitude TestCPUProfileMultithreadMagnitude compares pprof results vs OS rusage to verify that pprof is capturing all CPU usage. Presently it compares the sum of cpuHog1 samples vs rusage. However, background usage from the scheduler, GC, etc can cause additional CPU usage causing test failures if rusage is too far off from the cpuHog1 samples. That said, this test doesn't actually need to care about cpuHog1 samples. It simply cares that pprof samples match rusage, not what the breakdown of usage was. As a result, we can compare the sum of _all_ pprof samples vs rusage, which should implicitly include any background CPU usage. Fixes #50097. Change-Id: I649a18de5b3dcf58b62be5962fa508d14cd4dc79 Reviewed-on: https://go-review.googlesource.com/c/go/+/379535 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-01-19runtime/pprof: assert that labels never appear on unexpected samplesMichael Pratt
This makes TestLabelSystemstack much more strict, enabling it to detect any misplacement of labels. Unfortunately, there are several edge cases where we may not have an obviously correct stack trace, so we generally except the runtime package, with the exception of background goroutines that we know should not be labeled. For #50007 For #50032 Change-Id: I8dce7e7da04f278ce297422227901efe52782ca0 Reviewed-on: https://go-review.googlesource.com/c/go/+/369984 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2022-01-10runtime/pprof: run TestCPUProfileMultithreadMagnitude subtests separatelyMichael Pratt
Currently TestCPUProfileMultithreadMagnitude runs two CPU consumption functions in a single profile and then analyzes the results as separate subtests. This works fine, but when debugging failures it makes manual analysis of the profile dump a bit annoying. Refactor the test to collect separate profiles for each subtest for easier future analysis. For #50097. For #50232. Change-Id: Ia1c8bb86aaaf652e64c5e660dcc2da47d2194c2b Reviewed-on: https://go-review.googlesource.com/c/go/+/372800 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Bryan Mills <bcmills@google.com>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-08runtime/pprof: increase systemstack calls in TestLabelSystemstackMichael Pratt
TestLabelSystemstack needs to collect samples within runtime.systemstack to complete the test. The current approach uses fmt.Fprintf, which gets into systemstack through the allocator and GC, but also does lots of other work. In my measurements, approximately 2% of samples contain runtime.systemstack. The new approach uses debug.SetGCPercent, which uses systemstack for most of its work, including contention on mheap_.lock, which extends usage even more. In my measurements, approximately 99% of samples contain runtime.systemstack. Fixes #50050 Change-Id: I59e5bb756341b716a12e13d2e3fe0adadd7fe956 Reviewed-on: https://go-review.googlesource.com/c/go/+/370375 Reviewed-by: Bryan Mills <bcmills@google.com> Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-07runtime/pprof: assert that labelHog samples are always labeledMichael Pratt
With https://golang.org/issue/50007 resolved, there are no known issues with pprof labels remaining. Thus, the 10% allowed error in TestLabelSystemstack should not be required. Drop it in favor of an explicit assertion that all samples containing labelHog are properly labeled. This is no flaky in my local testing. It is possible that other bugs will appear at larger testing scale, in which case this CL will be reverted, but then at least we will be aware of additional failure modes. For #50007. Change-Id: I1ef530c303bd9a01af649b8b08d4b35505e8aada Reviewed-on: https://go-review.googlesource.com/c/go/+/369744 Reviewed-by: Bryan Mills <bcmills@google.com> Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-07runtime/pprof: consume tag for first CPU recordMichael Pratt
profBuf.write uses an index in b.tags for each entry, even if that entry has no tag (that slice entry just remains 0). profBuf.read similarly returns a tags slice with exactly as many entries as there are records in data. profileBuilder.addCPUData iterates through the tags in lockstep with the data records. Except in the special case of the first record, where it forgets to increment tags. Thus the first read of profiling data has all tags off-by-one. To help avoid regressions, addCPUData is changed to assert that tags contains exactly the correct number of tags. For #50007. Change-Id: I5f32f93003297be8d6e33ad472c185d924a63256 Reviewed-on: https://go-review.googlesource.com/c/go/+/369741 Reviewed-by: Austin Clements <austin@google.com> Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-03runtime: fix missing pprof labelsFelix Geisendörfer
Use gp.m.curg instead of the gp when recording cpu profiler stack traces. This ensures profiler labels are captured when systemstack or similar is executing on behalf of the current goroutine. After this there are still rare cases of samples containing the labelHog function, so more work might be needed. This patch should fix ~99% of the problem. Also change testCPUProfile interface a little to allow the new test to re-run with a longer duration if it fails during a -short run. Fixes #48577. Change-Id: I3dbc9fd5af3c513544e822acaa43055b2e00dfa9 Reviewed-on: https://go-review.googlesource.com/c/go/+/367200 Trust: Michael Pratt <mpratt@google.com> Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-11-12runtime/pprof: mark TestCPUProfileMultithreadMagnitude as flakyMichael Pratt
The Linux kernel starting in 5.9 and fixed in 5.16 has a bug that can break CPU timer signal delivery on new new threads if the timer interrupt fires during handling of the clone system call. Broken CPU timer signal deliver will skew CPU profile results and cause this test to fail. There is currently no known workaround, so mark the test as flaky on builders with known broken kernels. For #49065 Change-Id: I37ceb9ea244869b0aab5cd9a36b27ca2f7e5d315 Reviewed-on: https://go-review.googlesource.com/c/go/+/363214 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-11-09runtime/pprof: include labels in profile dumpMichael Pratt
For tests of pprof label support having the sample labels in the output is needed for effective debugging. For #48577 Change-Id: Ic7c5bc90cb33e8fb477f7db62d9b56a7a9d6ffa8 Reviewed-on: https://go-review.googlesource.com/c/go/+/362614 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-11-03runtime/pprof: disable GC during calibration testRhys Hiltner
TestCPUProfileMultithreadMagnitude compares Go's CPU profile against the OS's accounting of the process's execution time, expecting them to be near equal. Background work from the runtime (especially in the garbage collector) can add significant noise to that measurement and flakiness to the test. Disable automatic GC cycles during the test. Updates #49065 Change-Id: Ie88895bfea17374278c5187f3a83e9f486bd37fb Reviewed-on: https://go-review.googlesource.com/c/go/+/359934 Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Pratt <mpratt@google.com> Trust: Michael Knyszek <mknyszek@google.com>
2021-11-02Revert "runtime: fix missing pprof labels"Bryan C. Mills
This reverts CL 351751. Reason for revert: new test is failing on many builders. Change-Id: I066211c9f25607ca9eb5299aedea2ecc5069e34f Reviewed-on: https://go-review.googlesource.com/c/go/+/360757 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-11-02runtime: fix missing pprof labelsFelix Geisendörfer
Use gp.m.curg instead of the gp when recording cpu profiler stack traces. This ensures profiler labels are captured when systemstack or similar is executing on behalf of the current goroutine. After this there are still rare cases of samples containing the labelHog function, so more work might be needed. This patch should fix ~99% of the problem. Fixes #48577. Change-Id: I27132110e3d09721ec3b3ef417122bc70d8f3279 Reviewed-on: https://go-review.googlesource.com/c/go/+/351751 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Michael Knyszek <mknyszek@google.com>
2021-10-29runtime: fix sweep termination conditionMichael Anthony Knyszek
Currently, there is a chance that the sweep termination condition could flap, causing e.g. runtime.GC to return before all sweep work has not only been drained, but also completed. CL 307915 and CL 307916 attempted to fix this problem, but it is still possible that mheap_.sweepDrained is marked before any outstanding sweepers are accounted for in mheap_.sweepers, leaving a window in which a thread could observe isSweepDone as true before it actually was (and after some time it would revert to false, then true again, depending on the number of outstanding sweepers at that point). This change fixes the sweep termination condition by merging mheap_.sweepers and mheap_.sweepDrained into a single atomic value. This value is updated such that a new potential sweeper will increment the oustanding sweeper count iff there are still outstanding spans to be swept without an outstanding sweeper to pick them up. This design simplifies the sweep termination condition into a single atomic load and comparison and ensures the condition never flaps. Updates #46500. Fixes #45315. Change-Id: I6d69aff156b8d48428c4cc8cfdbf28be346dbf04 Reviewed-on: https://go-review.googlesource.com/c/go/+/333389 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2021-10-28all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)Russ Cox
When these packages are released as part of Go 1.18, Go 1.16 will no longer be supported, so we can remove the +build tags in these files. Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs as defined in src/cmd/dist/buildtool.go, which need to continue to build with Go 1.4 for now. Also reverted src/vendor and src/cmd/vendor, which will need to be updated in their own repos first. Manual changes in runtime/pprof/mprof_test.go to adjust line numbers. For #41184. Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e Reviewed-on: https://go-review.googlesource.com/c/go/+/344955 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-06all: use bytes.Cut, strings.CutRuss Cox
Many uses of Index/IndexByte/IndexRune/Split/SplitN can be written more clearly using the new Cut functions. Do that. Also rewrite to other functions if that's clearer. For #46336. Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448 Reviewed-on: https://go-review.googlesource.com/c/go/+/351711 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-09-27runtime/pprof: skip TestTimeVDSO on AndroidCherry Mui
The test is flaky on Android. VDSO may not be enabled so it may not have the original problem anyway. Change-Id: I73c2902c682a44d893e0d4e34f006c2377ef8816 Reviewed-on: https://go-review.googlesource.com/c/go/+/352509 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-09-27runtime: profile with per-thread timers on LinuxRhys Hiltner
Using setitimer on Linux to request SIGPROF signal deliveries in proportion to the process's on-CPU time results in under-reporting when the program uses several goroutines in parallel. Linux calculates the process's total CPU spend on a regular basis (often every 4ms); if the process has spent enough CPU time since the last calculation to warrant more than one SIGPROF (usually 10ms for the default sample rate of 100 Hz), the kernel is often able to deliver only one of them. With these common settings, that results in Go CPU profiles being attenuated for programs that use more than 2.5 goroutines in parallel. To avoid in effect overflowing the kernel's process-wide CPU counter, and relying on Linux's typical behavior of having the active thread handle the resulting process-targeted signal, use timer_create to request a timer for each OS thread that the Go runtime manages. Have each timer track the CPU time of a single thread, with the resulting SIGPROF going directly to that thread. To continue tracking CPU time spent on threads that don't interact with the Go runtime (such as those created and used in cgo), keep using setitimer in addition to the new mechanism. When a SIGPROF signal arrives, check whether it's due to setitimer or timer_create and filter as appropriate: If the thread is known to Go (has an M) and has a timer_create timer, ignore SIGPROF signals from setitimer. If the thread is not known to Go (does not have an M), ignore SIGPROF signals that are not from setitimer. Counteract the new bias that per-thread profiling adds against short-lived threads (or those that are only active on occasion for a short time, such as garbage collection workers on mostly-idle systems) by configuring the timers' initial trigger to be from a uniform random distribution between "immediate trigger" and the full requested sample period. Updates #35057 Change-Id: Iab753c4e5101bdc09ef9132eec84a75478e05579 Reviewed-on: https://go-review.googlesource.com/c/go/+/324129 Run-TryBot: Rhys Hiltner <rhys@justin.tv> TryBot-Result: Go Bot <gobot@golang.org> Trust: David Chase <drchase@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-09-24runtime: set vdsoSP to caller's SP consistentlyCherry Mui
m.vdsoSP should be set to the SP of the caller of nanotime1, instead of the SP of nanotime1 itself, which matches m.vdsoPC. Otherwise the unmatched vdsoPC and vdsoSP would make the stack trace look like recursive. We already do it correctly on AMD64, 386, and RISCV64. This CL fixes the rest. Fixes #47324. Change-Id: I98b6fcfbe9fc6bdd28b8fe2a1299b7c505371dd4 Reviewed-on: https://go-review.googlesource.com/c/go/+/337590 Trust: Cherry Mui <cherryyz@google.com> Trust: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2021-07-20[dev.typeparams] all: merge master (c8f4e61) into dev.typeparamsMatthew Dempsky
Conflicts: - src/runtime/internal/sys/zgoarch_386.go - src/runtime/internal/sys/zgoarch_amd64.go - src/runtime/internal/sys/zgoarch_arm.go - src/runtime/internal/sys/zgoarch_arm64.go - src/runtime/internal/sys/zgoarch_arm64be.go - src/runtime/internal/sys/zgoarch_armbe.go - src/runtime/internal/sys/zgoarch_mips.go - src/runtime/internal/sys/zgoarch_mips64.go - src/runtime/internal/sys/zgoarch_mips64le.go - src/runtime/internal/sys/zgoarch_mips64p32.go - src/runtime/internal/sys/zgoarch_mips64p32le.go - src/runtime/internal/sys/zgoarch_mipsle.go - src/runtime/internal/sys/zgoarch_ppc.go - src/runtime/internal/sys/zgoarch_ppc64.go - src/runtime/internal/sys/zgoarch_ppc64le.go - src/runtime/internal/sys/zgoarch_riscv.go - src/runtime/internal/sys/zgoarch_riscv64.go - src/runtime/internal/sys/zgoarch_s390.go - src/runtime/internal/sys/zgoarch_s390x.go - src/runtime/internal/sys/zgoarch_sparc.go - src/runtime/internal/sys/zgoarch_sparc64.go - src/runtime/internal/sys/zgoarch_wasm.go On dev.typeparams, CL 328336 moved these files to internal/goarch; whereas on master, CL 333909 reserved GOARCH=loong64. For this CL, I resolved the conflict by simply running "go generate internal/goarch". Merge List: + 2021-07-19 c8f4e6152d spec: correct example comment in Conversions from slice to array + 2021-07-19 1d91551b73 time: correct typo in documentation for UnixMicro + 2021-07-19 404127c30f cmd/compile: fix off-by-one error in traceback argument counting + 2021-07-19 6298cfe672 cmd/compile: fix typo in fatal message of builtinCall + 2021-07-19 49402bee36 cmd/{compile,link}: fix bug in map.zero handling + 2021-07-18 a66190ecee test/bench/go1: fix size for RegexpMatchMedium_32 + 2021-07-18 650fc2117a text/scanner: use Go convention in Position doc comment + 2021-07-16 aa4e0f528e net/http: correct capitalization in cancelTimeBody comment + 2021-07-15 0941dbca6a testing: clarify in docs that TestMain is advanced + 2021-07-15 69728ead87 cmd/go: update error messages in tests to match CL 332573 + 2021-07-15 c1cc9f9c3d cmd/compile: fix lookup package of redeclared dot import symbol + 2021-07-15 21a04e3335 doc/go1.17: mention GOARCH=loong64 + 2021-07-14 2b00a54baf go/build, runtime/internal/sys: reserve GOARCH=loong64 + 2021-07-14 60ddf42b46 cmd/go: change link in error message from /wiki to /doc. + 2021-07-13 d8f348a589 cmd/go: remove a duplicated word from 'go help mod graph' + 2021-07-12 a98589711d crypto/tls: test key type when casting + 2021-07-12 cfbd73ba33 doc/go1.17: editing pass over the "Compiler" section + 2021-07-09 ab4085ce84 runtime/pprof: call runtime.GC twice in memory profile test Change-Id: I1490a4c7e4c560659c21a4eb67d243f35d1f908e
2021-07-09runtime/pprof: call runtime.GC twice in memory profile testMichael Anthony Knyszek
This change fixes #46500 by working around #45315 which may cause freed objects to get missed in the heap profile published for the test. By calling runtime.GC one more time this change ensures that all freed objects are accounted for. Fixes #46500. Change-Id: Iedcd0b37dbaffa688b0ff8631a8b79f7a1169634 Reviewed-on: https://go-review.googlesource.com/c/go/+/333549 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2021-06-22[dev.typeparams] all: merge master (16e82be) into dev.typeparamsCuong Manh Le
Merge List: + 2021-06-21 16e82be454 runtime: fix crash during VDSO calls on PowerPC + 2021-06-21 2e542c3c06 runtime/pprof: deflake TestMorestack more + 2021-06-21 ced0fdbad0 doc/go1.17: note deprecation of 'go get' for installing commands + 2021-06-21 7a5e7047a4 doc/go1.17: add Go 1.18 pre-announcements + 2021-06-21 85a2e24afd doc/go1.17: add security-related release notes + 2021-06-21 1de332996c doc/go1.17: document go/parser.SkipObjectResolution + 2021-06-21 117ebe0f52 cmd/go: do not require the module cache to exist for 'go mod edit' + 2021-06-20 460900a7b5 os/signal: test with a significantly longer fatal timeout + 2021-06-19 b73cc4b02b database/sql: do not rely on timeout for deadlock test + 2021-06-18 86743e7d86 image: add RGBA64Image interface + 2021-06-18 9401172166 runtime: clarify Frames.Next documentation + 2021-06-18 57aaa19aae runtime: disable CPU profiling before removing the SIGPROF handler + 2021-06-18 6f22d2c682 doc/go1.17: fix typo + 2021-06-17 45f251ad6c cmd/pprof,runtime/pprof: disable test on more broken platforms + 2021-06-17 ed834853ad cmd/go: replace a TODO with an explanatory comment + 2021-06-17 4dede02550 cmd/pprof: make ObjAddr a no-op + 2021-06-17 97cee43c93 testing: drop unusual characters from TempDir directory name + 2021-06-17 b0355a3e72 time: fix receiver for Time.IsDST method + 2021-06-17 881b6ea7ba doc/go1.17: fix redundant space + 2021-06-16 0e67ce3d28 cmd/go: in lazy modules, add transitive imports for 'go get' arguments + 2021-06-16 6ea2af0890 cmd/go: add a regression test for #45979 + 2021-06-16 a294e4e798 math/rand: mention half-open intervals explicitly + 2021-06-16 a6a853f94c cmd/asm: restore supporting of *1 scaling on ARM64 Change-Id: Ifdcb817fd44b4fa9c477042b41da55d1d769b016
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 15:28:04 +0000