From 0b31a46f1fd89b1ede02be9fae920dca6172ffaf Mon Sep 17 00:00:00 2001 From: Rhys Hiltner Date: Tue, 12 Sep 2023 15:44:48 -0700 Subject: runtime: profile contended lock calls Add runtime-internal locks to the mutex contention profile. Store up to one call stack responsible for lock contention on the M, until it's safe to contribute its value to the mprof table. Try to use that limited local storage space for a relatively large source of contention, and attribute any contention in stacks we're not able to store to a sentinel _LostContendedLock function. Avoid ballooning lock contention while manipulating the mprof table by attributing to that sentinel function any lock contention experienced while reporting lock contention. Guard collecting real call stacks with GODEBUG=profileruntimelocks=1, since the available data has mixed semantics; we can easily capture an M's own wait time, but we'd prefer for the profile entry of each critical section to describe how long it made the other Ms wait. It's too late in the Go 1.22 cycle to make the required changes to futex-based locks. When not enabled, attribute the time to the sentinel function instead. Fixes #57071 Change-Id: I3eee0ccbfc20f333b56f20d8725dfd7f3a526b41 Reviewed-on: https://go-review.googlesource.com/c/go/+/528657 LUCI-TryBot-Result: Go LUCI Reviewed-by: Michael Pratt Auto-Submit: Rhys Hiltner Reviewed-by: Than McIntosh --- src/runtime/runtime2.go | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'src/runtime/runtime2.go') diff --git a/src/runtime/runtime2.go b/src/runtime/runtime2.go index e64c3c5695..6bdd66766d 100644 --- a/src/runtime/runtime2.go +++ b/src/runtime/runtime2.go @@ -593,6 +593,8 @@ type m struct { lockedInt uint32 // tracking for internal lockOSThread nextwaitm muintptr // next m waiting for lock + mLockProfile mLockProfile // fields relating to runtime.lock contention + // wait* are used to carry arguments from gopark into park_m, because // there's no stack to put them on. That is their sole purpose. waitunlockf func(*g, unsafe.Pointer) bool @@ -900,6 +902,12 @@ type schedt struct { // stwTotalTimeOther covers the others. stwTotalTimeGC timeHistogram stwTotalTimeOther timeHistogram + + // totalRuntimeLockWaitTime (plus the value of lockWaitTime on each M in + // allm) is the sum of time goroutines have spent in _Grunnable and with an + // M, but waiting for locks within the runtime. This field stores the value + // for Ms that have exited. + totalRuntimeLockWaitTime atomic.Int64 } // Values for the flags field of a sigTabT. -- cgit v1.3