aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/metrics.go
AgeCommit message (Collapse)Author
2025-10-30runtime: eliminate _PsyscallMichael Anthony Knyszek
This change eliminates the _Psyscall state by using synchronization on the G status _Gsyscall to make syscalls work instead. This removes an atomic Store and an atomic CAS on the syscall path, which reduces syscall and cgo overheads. It also simplifies the syscall paths quite a bit. The one danger with this change is that we have a new combination of states that was previously impossible. There are brief windows where it's possible to observe a goroutine in _Grunning but without a P. This change is careful to hide this detail from the execution tracer, but it may have unexpected effects in the rest of the runtime, making this change somewhat risky. goos: linux goarch: amd64 pkg: internal/runtime/cgobench cpu: AMD EPYC 7B13 │ before.out │ after.out │ │ sec/op │ sec/op vs base │ CgoCall-64 43.69n ± 1% 35.83n ± 1% -17.99% (p=0.002 n=6) CgoCallParallel-64 5.306n ± 1% 5.338n ± 1% ~ (p=0.132 n=6) Change-Id: I4551afc1eea0c1b67a0b2dd26b0d49aa47bf1fb8 Reviewed-on: https://go-review.googlesource.com/c/go/+/646198 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-10-21all: eliminate unnecessary type conversionsJes Cok
Found by github.com/mdempsky/unconvert Change-Id: I88ce10390a49ba768a4deaa0df9057c93c1164de GitHub-Last-Rev: 3b0f7e8f74f58340637f33287c238765856b2483 GitHub-Pull-Request: golang/go#75974 Reviewed-on: https://go-review.googlesource.com/c/go/+/712940 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>
2025-08-15runtime/metrics: add metric for current Go-owned thread countMichael Anthony Knyszek
Fixes #15490. Change-Id: I6ce9edc46398030ff639e22d4ca4adebccdfe1b7 Reviewed-on: https://go-review.googlesource.com/c/go/+/690399 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-08-15runtime/metrics: add metric for total goroutines createdMichael Anthony Knyszek
For #15490. Change-Id: Ic587dda1f42d613ea131a6b53ce6ba6e6cadf4c7 Reviewed-on: https://go-review.googlesource.com/c/go/+/690398 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-08-15runtime/metrics: add metrics for goroutine sched statesMichael Anthony Knyszek
This is largely a port of CL 38180. For #15490. Change-Id: I2726111e472e81e9f9f0f294df97872c2689f061 Reviewed-on: https://go-review.googlesource.com/c/go/+/690397 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-30runtime/metrics: add cleanup and finalizer queue metricsMichael Anthony Knyszek
These metrics are useful for identifying finalizer and cleanup problems, namely slow finalizers and/or cleanups holding up the queue, which can lead to a memory leak. Fixes #72948. Change-Id: I1bb64a9ca751fcb462c96d986d0346e0c2894c95 Reviewed-on: https://go-review.googlesource.com/c/go/+/690396 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2025-05-08runtime: remove ptr/scalar bitmap metrickhr@golang.org
We don't use this mechanism any more, so the metric will always be zero. Since CL 616255. Update #73628 Change-Id: Ic179927a8bc24e6291876c218d88e8848b057c2a Reviewed-on: https://go-review.googlesource.com/c/go/+/671096 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-04-23runtime: move sizeclass defs to new package internal/runtime/gcMichael Anthony Knyszek
We will want to reference these definitions from new generator programs, and this is a good opportunity to cleanup all these old C-style names. Change-Id: Ifb06f0afc381e2697e7877f038eca786610c96de Reviewed-on: https://go-review.googlesource.com/c/go/+/655275 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2024-04-08runtime: move GC pause time CPU metrics update into the STWMichael Anthony Knyszek
This change fixes a possible race with updating metrics and reading them. The update is intended to be protected by the world being stopped, but here, it clearly isn't. Fixing this lets us lower the thresholds in the metrics tests by an order of magnitude, because the only thing we have to worry about now is floating point error (the tests were previously written assuming the floating point error was much higher than it actually was; that turns out not to be the case, and this bug was the problem instead). However, this still isn't that tight of a bound; we still want to catch any and all problems of exactness. For this purpose, this CL adds a test to check the source-of-truth (in uint64 nanoseconds) that ensures the totals exactly match. This means we unfortunately have to take another time measurement, but for now let's prioritize correctness. A few additional nanoseconds of STW time won't be terribly noticable. Fixes #66212. Change-Id: Id02c66e8a43c13b1f70e9b268b8a84cc72293bfd Reviewed-on: https://go-review.googlesource.com/c/go/+/570257 Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Nicolas Hillegeer <aktau@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21runtime: profile contended lock callsRhys Hiltner
Add runtime-internal locks to the mutex contention profile. Store up to one call stack responsible for lock contention on the M, until it's safe to contribute its value to the mprof table. Try to use that limited local storage space for a relatively large source of contention, and attribute any contention in stacks we're not able to store to a sentinel _LostContendedLock function. Avoid ballooning lock contention while manipulating the mprof table by attributing to that sentinel function any lock contention experienced while reporting lock contention. Guard collecting real call stacks with GODEBUG=profileruntimelocks=1, since the available data has mixed semantics; we can easily capture an M's own wait time, but we'd prefer for the profile entry of each critical section to describe how long it made the other Ms wait. It's too late in the Go 1.22 cycle to make the required changes to futex-based locks. When not enabled, attribute the time to the sentinel function instead. Fixes #57071 This is a roll-forward of https://go.dev/cl/528657, which was reverted in https://go.dev/cl/543660 Reason for revert: de-flakes tests (reduces dependence on fine-grained timers, correctly identifies contention on big-endian futex locks, attempts to measure contention in the semaphore implementation but only uses that secondary measurement to finish the test early, skips tests on single-processor systems) Change-Id: I31389f24283d85e46ad9ba8d4f514cb9add8dfb0 Reviewed-on: https://go-review.googlesource.com/c/go/+/544195 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Auto-Submit: Rhys Hiltner <rhys@justin.tv> Run-TryBot: Rhys Hiltner <rhys@justin.tv>
2023-11-20Revert "runtime: profile contended lock calls"Matthew Dempsky
This reverts commit go.dev/cl/528657. Reason for revert: broke a lot of builders. Change-Id: I70c33062020e997c4df67b3eaa2e886cf0da961e Reviewed-on: https://go-review.googlesource.com/c/go/+/543660 Reviewed-by: Than McIntosh <thanm@google.com> Auto-Submit: Matthew Dempsky <mdempsky@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17runtime: profile contended lock callsRhys Hiltner
Add runtime-internal locks to the mutex contention profile. Store up to one call stack responsible for lock contention on the M, until it's safe to contribute its value to the mprof table. Try to use that limited local storage space for a relatively large source of contention, and attribute any contention in stacks we're not able to store to a sentinel _LostContendedLock function. Avoid ballooning lock contention while manipulating the mprof table by attributing to that sentinel function any lock contention experienced while reporting lock contention. Guard collecting real call stacks with GODEBUG=profileruntimelocks=1, since the available data has mixed semantics; we can easily capture an M's own wait time, but we'd prefer for the profile entry of each critical section to describe how long it made the other Ms wait. It's too late in the Go 1.22 cycle to make the required changes to futex-based locks. When not enabled, attribute the time to the sentinel function instead. Fixes #57071 Change-Id: I3eee0ccbfc20f333b56f20d8725dfd7f3a526b41 Reviewed-on: https://go-review.googlesource.com/c/go/+/528657 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Than McIntosh <thanm@google.com>
2023-11-15runtime/metrics: add STW stopping and total time metricsMichael Pratt
This CL adds four new time histogram metrics: /sched/pauses/stopping/gc:seconds /sched/pauses/stopping/other:seconds /sched/pauses/total/gc:seconds /sched/pauses/total/other:seconds The "stopping" metrics measure the time taken to start a stop-the-world pause. i.e., how long it takes stopTheWorldWithSema to stop all Ps. This can be used to detect STW struggling to preempt Ps. The "total" metrics measure the total duration of a stop-the-world pause, from starting to stop-the-world until the world is started again. This includes the time spent in the "start" phase. The "gc" metrics are used for GC-related STW pauses. The "other" metrics are used for all other STW pauses. All of these metrics start timing in stopTheWorldWithSema only after successfully acquiring sched.lock, thus excluding lock contention on sched.lock. The reasoning behind this is that while waiting on sched.lock the world is not stopped at all (all other Ps can run), so the impact of this contention is primarily limited to the goroutine attempting to stop-the-world. Additionally, we already have some visibility into sched.lock contention via contention profiles (#57071). /sched/pauses/total/gc:seconds is conceptually equivalent to /gc/pauses:seconds, so the latter is marked as deprecated and returns the same histogram as the former. In the implementation, there are a few minor differences: * For both mark and sweep termination stops, /gc/pauses:seconds started timing prior to calling startTheWorldWithSema, thus including lock contention. These details are minor enough, that I do not believe the slight change in reporting will matter. For mark termination stops, moving timing stop into startTheWorldWithSema does have the side effect of requiring moving other GC metric calculations outside of the STW, as they depend on the same end time. Fixes #63340 Change-Id: Iacd0bab11bedab85d3dcfb982361413a7d9c0d05 Reviewed-on: https://go-review.googlesource.com/c/go/+/534161 Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-03runtime: eliminate possible stack movements in ReadMetricsSlowMichael Anthony Knyszek
Currently it's possible (and even probable, with mayMoreStackMove mode) for a stack allocation to occur between readmemstats_m and readMetrics in ReadMetricsSlow. This can cause tests to fail by producing metrics that are inconsistent between the two sources. Fix this by breaking out the critical section of readMetrics and calling that from ReadMetricsSlow on the systemstack. Our main constraint in calling readMetrics on the system stack is the fact that we can't acquire the metrics semaphore from the system stack. But if we break out the critical section, then we can acquire that semaphore before we go on the system stack. While we're here, add another readMetrics call before readmemstats_m. Since we're being paranoid about ways that metrics could get skewed between the two calls, let's eliminate all uncertainty. It's possible for readMetrics to allocate new memory, for example for histograms, and fail while it's reading metrics. I believe we're just getting lucky today with the order in which the metrics are produced. Another call to readMetrics will preallocate this data in the samples slice. One nice thing about this second read is that now we effectively have a way to check if readMetrics really will allocate if called a second time on the same samples slice. Fixes #60607. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest Change-Id: If6ce666530903239ef9f02dbbc3f1cb6be71e425 Reviewed-on: https://go-review.googlesource.com/c/go/+/539117 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-09-18all: clean unnecessary castsJes Cok
Run 'unconvert -safe -apply' (https://github.com/mdempsky/unconvert) Change-Id: I24b7cd7d286cddce86431d8470d15c5f3f0d1106 GitHub-Last-Rev: 022e75384c08bb899a8951ba0daffa0f2e14d5a7 GitHub-Pull-Request: golang/go#62662 Reviewed-on: https://go-review.googlesource.com/c/go/+/528696 Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-09-06runtime/metrics: fix /gc/scan/* metricsNayef Ghattas
In the existing implementation, all /gc/scan/* metrics are always equal to 0 due to the dependency on gcStatDep not being set. This leads to gcStatAggregate always containing zeros, and always reporting 0 for those metrics. Also, add a test to ensure that /gc/scan/* metrics are not empty. Fixes #62477. Change-Id: I67497347d50ed5c3ce1719a18714c062ec938cab Reviewed-on: https://go-review.googlesource.com/c/go/+/525595 Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
2023-07-12all: fix typos and remove repeated wordsDan Kortschak
Change-Id: I5f06a4ef1d827eb0fe32a8d98444142108b0d573 Reviewed-on: https://go-review.googlesource.com/c/go/+/508996 Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Keith Randall <khr@google.com>
2023-06-13all: fix mismatched symbolscui fliter
There are some symbol mismatches in the comments, this commit attempts to fix them Change-Id: I5c9075e5218defe9233c075744d243b26ff68496 Reviewed-on: https://go-review.googlesource.com/c/go/+/492996 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: shuang cui <imcusg@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Pratt <mpratt@google.com>
2023-05-24runtime: ensure consistency of /gc/scan/*Michael Anthony Knyszek
Currently /gc/scan/total:bytes is computed as a separate sum. Compute it using the same inputs so it's always consistent with the sum of everything else in /gc/scan/*. For #56857. Change-Id: I43d9148a23b1d2eb948ae990193dca1da85df8a3 Reviewed-on: https://go-review.googlesource.com/c/go/+/497880 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-23runtime/metrics: refactor CPU stats accumulationMichael Anthony Knyszek
Currently the CPU stats are only updated once every mark termination, but for writing robust tests, it's often useful to force this update. Refactor the CPU stats accumulation out of gcMarkTermination and into its own function. This is also a step toward real-time CPU stats. While we're here, fix some incorrect documentation about dedicated GC CPU time. For #59749. For #60276. Change-Id: I8c1a9aca45fcce6ce7999702ae4e082853a69711 Reviewed-on: https://go-review.googlesource.com/c/go/+/487215 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-05-23runtime/metrics: add /gc/scan/total:bytesFelix Geisendörfer
For #56857 Change-Id: I10dbc5db506c95b7578c2b6baf051a351f68bb2a Reviewed-on: https://go-review.googlesource.com/c/go/+/497576 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-05-23runtime/metrics: add /gc/scan/heap:bytesFelix Geisendörfer
For #56857 Change-Id: If3b962f575c33b2cc29f89e33c7aafb476d98ce9 Reviewed-on: https://go-review.googlesource.com/c/go/+/497575 Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-05-23runtime/metrics: add /gc/scan/globals:bytesFelix Geisendörfer
For #56857 Change-Id: I748fd2a33ee76d9a83ea42f2ebf6d9edda243301 Reviewed-on: https://go-review.googlesource.com/c/go/+/497320 Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-23runtime/metrics: add /gc/scan/stack:bytesFelix Geisendörfer
For #56857 Change-Id: I58187d7c4112b35951014ab14f2969bed7f4c8e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/497319 Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-23runtime/metrics: add /gc/gogc:percentFelix Geisendörfer
For #56857 Change-Id: I7e7d2ea3e6ab59291a4cd867c680605ad75bd21f Reviewed-on: https://go-review.googlesource.com/c/go/+/497317 Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2023-05-23runtime/metrics: add /gc/gomemlimit:bytesFelix Geisendörfer
For #56857 Change-Id: I184d752cc615874ada3d0dbc6ed1bf72c8debd0f Reviewed-on: https://go-review.googlesource.com/c/go/+/497316 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
2023-05-23runtime/metrics: add /gc/heap/live:bytesFelix Geisendörfer
For #56857 Change-Id: I0622af974783ab435e91b9fb3c1ba43f256ee4ac Reviewed-on: https://go-review.googlesource.com/c/go/+/497315 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-04-18cmd/go: add check for unknown godebug settingRuss Cox
A //go:debug line mentioning an unknown or retired setting should be diagnosed as making the program invalid. Do that. We agreed on this in the proposal but I forgot to implement it. Change-Id: Ie69072a1682d4eeb6866c02adbbb426f608567c4 Reviewed-on: https://go-review.googlesource.com/c/go/+/476280 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-04html/template: disallow actions in JS template literalsRoland Shoemaker
ECMAScript 6 introduced template literals[0][1] which are delimited with backticks. These need to be escaped in a similar fashion to the delimiters for other string literals. Additionally template literals can contain special syntax for string interpolation. There is no clear way to allow safe insertion of actions within JS template literals, as handling (JS) string interpolation inside of these literals is rather complex. As such we've chosen to simply disallow template actions within these template literals. A new error code is added for this parsing failure case, errJsTmplLit, but it is unexported as it is not backwards compatible with other minor release versions to introduce an API change in a minor release. We will export this code in the next major release. The previous behavior (with the cavet that backticks are now escaped properly) can be re-enabled with GODEBUG=jstmpllitinterp=1. This change subsumes CL471455. Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue. Fixes CVE-2023-24538 Fixes #59234 [0] https://tc39.es/ecma262/multipage/ecmascript-language-expressions.html#sec-template-literals [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802457 Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Julie Qiu <julieqiu@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> Change-Id: Ia221fefdb273bd0f066dffc2abcf2a616801d2f2 Reviewed-on: https://go-review.googlesource.com/c/go/+/482079 TryBot-Bypass: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-04-04mime/multipart: limit parsed mime message sizesDamien Neil
The parsed forms of MIME headers and multipart forms can consume substantially more memory than the size of the input data. A malicious input containing a very large number of headers or form parts can cause excessively large memory allocations. Set limits on the size of MIME data: Reader.NextPart and Reader.NextRawPart limit the the number of headers in a part to 10000. Reader.ReadForm limits the total number of headers in all FileHeaders to 10000. Both of these limits may be set with with GODEBUG=multipartmaxheaders=<values>. Reader.ReadForm limits the number of parts in a form to 1000. This limit may be set with GODEBUG=multipartmaxparts=<value>. Thanks for Jakob Ackermann (@das7pad) for reporting this issue. For CVE-2023-24536 For #59153 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455 Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-by: Julie Qiu <julieqiu@google.com> Change-Id: I08dd297bd75724aade4b0bd6a7d19aeca5bbf99f Reviewed-on: https://go-review.googlesource.com/c/go/+/482077 Run-TryBot: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2023-01-20all: fix typos in go file commentsMarcel Meyer
This is the second round to look for spelling mistakes. This time the manual sifting of the result list was made easier by filtering out capitalized and camelcase words. grep -r --include '*.go' -E '^// .*$' . | aspell list | grep -E -x '[A-Za-z]{1}[a-z]*' | sort | uniq This PR will be imported into Gerrit with the title and first comment (this text) used to generate the subject and body of the Gerrit change. Change-Id: Ie8a2092aaa7e1f051aa90f03dbaf2b9aaf5664a9 GitHub-Last-Rev: fc2bd6e0c51652f13a7588980f1408af8e6080f5 GitHub-Pull-Request: golang/go#57737 Reviewed-on: https://go-review.googlesource.com/c/go/+/461595 Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
2023-01-19internal/godebug: export non-default-behavior counters in runtime/metricsRuss Cox
Allow GODEBUG users to report how many times a setting resulted in non-default behavior. Record non-default-behaviors for all existing GODEBUGs. Also rework tests to ensure that runtime is in sync with runtime/metrics.All, and generate docs mechanically from metrics.All. For #56986. Change-Id: Iefa1213e2a5c3f19ea16cd53298c487952ef05a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/453618 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-11-11all: fix problematic commentscui fliter
Change-Id: Ib6ea1bd04d9b06542ed2b0f453c718115417c62c Reviewed-on: https://go-review.googlesource.com/c/go/+/449755 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Keith Randall <khr@google.com>
2022-09-16runtime/metrics: add /sync/mutex/wait/total:seconds metricMichael Anthony Knyszek
This change adds a metric to the runtime/metrics package which tracks total mutex wait time for sync.Mutex and sync.RWMutex. The purpose of this metric is to be able to quickly get an idea of the total mutex wait time. The implementation of this metric piggybacks off of the existing G runnable tracking infrastructure, as well as the wait reason set on a G when it goes into _Gwaiting. Fixes #49881. Change-Id: I4691abf64ac3574bec69b4d7d4428b1573130517 Reviewed-on: https://go-review.googlesource.com/c/go/+/427618 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-16runtime: make the wait reason for a g blocked on a mutex more specificMichael Anthony Knyszek
This change adds 3 new waitReasons that correspond to sync.Mutex.Lock, sync.RWMutex.RLock, and sync.RWMutex.Lock that are plumbed down into semacquire1 by exporting new functions to the sync package from the runtime. Currently these three functions show up as "semacquire" in backtraces which isn't very clear, though the stack trace itself should reveal what's really going on. This represents a minor improvement to backtrace readability, though blocking on an RWMutex.w.Lock will still show up as blocking on a regular mutex (I suppose technically it is). This is a step toward helping the runtime identify when a goroutine is blocked on a mutex of some kind. For #49881. Change-Id: Ia409b4d27e117fe4bfdc25fa541e9c58d6d587b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/427616 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-09-16runtime/metrics: add CPU statsMichael Anthony Knyszek
This changes adds a breakdown for estimated CPU usage by time. These estimates are not based on real on-CPU counters, so each metric has a disclaimer explaining so. They can, however, be more reasonably compared to a total CPU time metric that this change also adds. Fixes #47216. Change-Id: I125006526be9f8e0d609200e193da5a78d9935be Reviewed-on: https://go-review.googlesource.com/c/go/+/404307 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Josh MacDonald <jmacd@lightstep.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-16runtime: shrink time histogram bucketsMichael Anthony Knyszek
There are lots of useless buckets with too much precision. Introduce a minimum level of precision with a minimum bucket bit. This cuts down on the size of a time histogram dramatically (~3x). Also, pick a smaller sub bucket count; we don't need 6% precision. Also, rename super-buckets to buckets to more closely line up with HDR histogram literature. Change-Id: I199449650e4b34f2a6dca3cf1d8edb071c6655c0 Reviewed-on: https://go-review.googlesource.com/c/go/+/427615 Run-TryBot: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-08-12runtime: convert timeHistogram to atomic typesMichael Pratt
I've dropped the note that sched.timeToRun is protected by sched.lock, as it does not seem to be true. For #53821. Change-Id: I03f8dc6ca0bcd4ccf3ec113010a0aa39c6f7d6ef Reviewed-on: https://go-review.googlesource.com/c/go/+/419449 Reviewed-by: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com>
2022-06-29runtime: add race annotations to metricsSemaMichael Pratt
metricsSema protects the metrics map. The map implementation is race instrumented regardless of which package is it called from. semacquire/semrelease are not automatically race instrumented, so we can trigger race false positives without manually annotating our lock acquire and release. See similar instrumentation on trace.shutdownSema and reflectOffs.lock. Fixes #53542. Change-Id: Ia3fd239ac860e037d09c7cb9c4ad267391e70705 Reviewed-on: https://go-review.googlesource.com/c/go/+/414517 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-05-13runtime/metrics: add the last GC cycle that had the limiter enabledMichael Anthony Knyszek
This metric exports the the last GC cycle index that the GC limiter was enabled. This metric is useful for debugging and identifying the root cause of OOMs, especially when SetMemoryLimit is in use. For #48409. Change-Id: Ic6383b19e88058366a74f6ede1683b8ffb30a69c Reviewed-on: https://go-review.googlesource.com/c/go/+/403614 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-05-13runtime/metrics: add the number of Go-to-C callsMichael Anthony Knyszek
For #47216. Change-Id: I1c2cd518e6ff510cc3ac8d8f72fd52eadcabc16c Reviewed-on: https://go-review.googlesource.com/c/go/+/404306 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-05-13runtime/metrics: add gomaxprocs metricMichael Anthony Knyszek
For #47216. Change-Id: Ib2d48c4583570a2dae9510a52d4c6ffc20161b31 Reviewed-on: https://go-review.googlesource.com/c/go/+/404305 Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-05-12runtime: measure stack usage; start stacks larger if neededKeith Randall
Measure the average stack size used by goroutines at every GC. When starting a new goroutine, allocate an initial goroutine stack of that average size. Intuition is that we'll waste at most 2x in stack space because only half the goroutines can be below average. In turn, we avoid some of the early stack growth / copying needed in the average case. More details in the design doc at: https://docs.google.com/document/d/1YDlGIdVTPnmUiTAavlZxBI1d9pwGQgZT7IKFKlIXohQ/edit?usp=sharing name old time/op new time/op delta Issue18138 95.3µs ± 0% 67.3µs ±13% -29.35% (p=0.000 n=9+10) Fixes #18138 Change-Id: Iba34d22ed04279da7e718bbd569bbf2734922eaa Reviewed-on: https://go-review.googlesource.com/c/go/+/345889 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
2022-05-03runtime: store consistent total allocation stats as uint64Michael Anthony Knyszek
Currently the consistent total allocation stats are managed as uintptrs, which means they can easily overflow on 32-bit systems. Fix this by storing these stats as uint64s. This will cause some minor performance degradation on 32-bit systems, but there really isn't a way around this, and it affects the correctness of the metrics we export. Fixes #52680. Change-Id: I7e6ca44047d46b4bd91c6f87c2d29f730e0d6191 Reviewed-on: https://go-review.googlesource.com/c/go/+/403758 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com>
2022-05-03runtime: check the heap goal and trigger dynamicallyMichael Anthony Knyszek
As it stands, the heap goal and the trigger are set once by gcController.commit, and then read out of gcController. However with the coming memory limit we need the GC to be able to respond to changes in non-heap memory. The simplest way of achieving this is to compute the heap goal and its associated trigger dynamically. In order to make this easier to implement, the GC trigger is now based on the heap goal, as opposed to the status quo of computing both simultaneously. In many cases we just want the heap goal anyway, not both, but we definitely need the goal to compute the trigger, because the trigger's bounds are entirely based on the goal (the initial runway is not). A consequence of this is that we can't rely on the trigger to enforce a minimum heap size anymore, and we need to lift that up directly to the goal. Specifically, we need to lift up any part of the calculation that *could* put the trigger ahead of the goal. Luckily this is just the heap minimum and minimum sweep distance. In the first case, the pacer may behave slightly differently, as the heap minimum is no longer the minimum trigger, but the actual minimum heap goal. In the second case it should be the same, as we ensure the additional runway for sweeping is added to both the goal *and* the trigger, as before, by computing that in gcControllerState.commit. There's also another place we update the heap goal: if a GC starts and we triggered beyond the goal, we always ensure there's some runway. That calculation uses the current trigger, which violates the rule of keeping the goal based on the trigger. Notice, however, that using the precomputed trigger for this isn't even quite correct: due to a bug, or something else, we might trigger a GC beyond the precomputed trigger. So this change also adds a "triggered" field to gcControllerState that tracks the point at which a GC actually triggered. This is independent of the precomputed trigger, so it's fine for the heap goal calculation to rely on it. It also turns out, there's more than just that one place where we really should be using the actual trigger point, so this change fixes those up too. Also, because the heap minimum is set by the goal and not the trigger, the maximum trigger calculation now happens *after* the goal is set, so the maximum trigger actually does what I originally intended (and what the comment says): at small heaps, the pacer picks 95% of the runway as the maximum trigger. Currently, the pacer picks a small trigger based on a not-yet-rounded-up heap goal, so the trigger gets rounded up to the goal, and as per the "ensure there's some runway" check, the runway ends up at always being 64 KiB. That check is supposed to be for exceptional circumstances, not the status quo. There's a test introduced in the last CL that needs to be updated to accomodate this slight change in behavior. So, this all sounds like a lot that changed, but what we're talking about here are really, really tight corner cases that arise from situations outside of our control, like pathologically bad behavior on the part of an OS or CPU. Even in these corner cases, it's very unlikely that users will notice any difference at all. What's more important, I think, is that the pacer behaves more closely to what all the comments describe, and what the original intent was. Another note: at first, one might think that computing the heap goal and trigger dynamically introduces some raciness, but not in this CL: the heap goal and trigger are completely static. Allocation outside of a GC cycle may now be a bit slower than before, as the GC trigger check is now significantly more complex. However, note that this executes basically just as often as gcController.revise, and that makes up for a vanishingly small part of any CPU profile. The next CL cleans up the floating point multiplications on this path nonetheless, just to be safe. For #48409. Change-Id: I280f5ad607a86756d33fb8449ad08555cbee93f9 Reviewed-on: https://go-review.googlesource.com/c/go/+/397014 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-04-29runtime/metrics: add additional allocation metricsMichael Anthony Knyszek
This change adds four additional metrics to the runtime/metrics package to fill in a few gaps with runtime.MemStats that were overlooked. The biggest one is TotalAlloc, which is impossible to find with the runtime/metrics package, but also add a few others for convenience and clarity. For instance, the total number of objects allocated and freed are technically available via allocs-by-size and frees-by-size, but it's onerous to get them (one needs to sum the sample counts in the histograms). The four additional metrics are: - /gc/heap/allocs:bytes -- total bytes allocated (TotalAlloc) - /gc/heap/allocs:objects -- total objects allocated (Mallocs - [tiny]) - /gc/heap/frees:bytes -- total bytes frees (TotalAlloc-HeapAlloc) - /gc/heap/frees:objects -- total objects freed (Frees - [tiny]) This change also updates the descriptions of allocs-by-size and frees-by-size to be more precise. Change-Id: Iec8c1797a584491e3484b198f2e7f325b68954a7 Reviewed-on: https://go-review.googlesource.com/c/go/+/312431 Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-27runtime/metrics: add tiny allocs metricMichael Anthony Knyszek
Currently tiny allocations are not represented in either MemStats or runtime/metrics, but they're represented in MemStats (indirectly) via Mallocs. Add them to runtime/metrics by first merging memstats.tinyallocs into consistentHeapStats (just for simplicity; it's monotonic so metrics would still be self-consistent if we just read it atomically) and then adding /gc/heap/tiny/allocs:objects to the list of supported metrics. Change-Id: Ie478006ab942a3e877b4a79065ffa43569722f3d Reviewed-on: https://go-review.googlesource.com/c/go/+/312909 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-23runtime,runtime/metrics: add metric to track scheduling latenciesMichael Pratt
This change adds a metric to track scheduling latencies, defined as the cumulative amount of time a goroutine spends being runnable before running again. The metric is an approximations and samples instead of trying to record every goroutine scheduling latency. This change was primarily authored by mknyszek@google.com. Change-Id: Ie0be7e6e7be421572eb2317d3dd8dd6f3d6aa152 Reviewed-on: https://go-review.googlesource.com/c/go/+/308933 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-14runtime: move next_gc and last_next_gc into gcControllerStateMichael Anthony Knyszek
This change moves next_gc and last_next_gc into gcControllerState under the names heapGoal and lastHeapGoal respectively. These are fundamentally GC pacer related values, and so it makes sense for them to live here. Partially generated by rf ' ex . { memstats.next_gc -> gcController.heapGoal memstats.last_next_gc -> gcController.lastHeapGoal } ' except for updates to comments and gcControllerState methods, where they're accessed through the receiver, and trace-related renames of NextGC -> HeapGoal, while we're here. For #44167. Change-Id: I1e871ad78a57b01be8d9f71bd662530c84853bed Reviewed-on: https://go-review.googlesource.com/c/go/+/306603 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-02-24docs: fix spellingJohn Bampton
Change-Id: Ib689e5793d9cb372e759c4f34af71f004010c822 GitHub-Last-Rev: d63798388e5dcccb984689b0ae39b87453b97393 GitHub-Pull-Request: golang/go#44259 Reviewed-on: https://go-review.googlesource.com/c/go/+/291949 Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Matthew Dempsky <mdempsky@google.com> Trust: Robert Griesemer <gri@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-16 09:09:04 +0000