| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Earlier in the development of the new tracer, m.id was used as a the
canonical ID for threads. Later, we switched to m.procid because it
matches the underlying OS resource. However, in that switch, we missed a
spot.
The tracer catches and emits statuses for goroutines that have remained
in either waiting or syscall across a whole generation, and emits a
thread ID for the latter set. The ID being used here, however, was m.id
instead of m.procid, like the rest of the tracer.
This CL also adds a regression test. In order to make the regression
test actually catch the failure, we also have to make the parser a
little less lenient about GoStatus events with GoSyscall: if this isn't
the first generation, then we should've seen the goroutine bound to an
M already when its status is getting emitted for its context. If we emit
the wrong ID, then we'll catch the issue when we emit the right ID when
the goroutine exits the syscall.
Fixes #65196.
Change-Id: I78b64fbea65308de5e1291c478a082a732a8bf9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/557456
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Change-Id: Ifc472ed4cf0433d06f43559930ac80df23656a6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/555496
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
|
|
This addresses some panics (out of bounds slice accesses and nil pointer
dereferences) when parsing malformed data. These were found via light
fuzzing, not by any rigorous means, and more potential panics probably
exist.
Fixes #64878.
Fixes #64879.
Change-Id: I4085788ba7dc91fec62e4abd88f50777577db42f
Reviewed-on: https://go-review.googlesource.com/c/go/+/552995
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
|
|
To begin with, CL 545515 made the trace parser tolerant of
GoCreateSyscall having a P, but that was wrong. Because dropm trashes
the M's syscalltick, that case should never be possible. So the first
thing this change does is it rewrites the test that CL introduced to
expect a failure instead of a success.
What I'd misinterpreted as a case that should be allowed was actually
the same as the other issues causing #64060, which is that the parser
doesn't correctly implement what happens to Ps when a thread calls back
into Go on non-pthread platforms, and what happens when a thread dies
on pthread platorms (or more succinctly, what the runtime does when it
calls dropm).
Specifically, the GoDestroySyscall event implies that if any P is still
running on that M when it's called, that the P stops running. This is
what is intended by the runtime trashing the M's syscalltick; when it
calls back into Go, the tracer models that thread as obtaining a new P
from scratch.
Handling this incorrectly manifests in one of two ways.
On pthread platforms, GoDestroySyscall is only emitted when a C thread
that previously called into Go is destroyed. However, that thread ID can
be reused. Because we have no thread events, whether it's the same
thread or not is totally ambiguous to the tracer. Therefore, the tracer
may observe a thread that previously died try to start running with a
new P under the same identity. The association to the old P is still
intact because the ID is the same, and the tracer gets confused -- it
appears as if two Ps are running on the same M!
On non-pthread platforms, GoDestroySyscall is emitted on every return to
C from Go code. In this case, the same thread with the same identity is
naturally going to keep calling back into Go. But again, since the
runtime trashes syscalltick in dropm, it's always going to acquire a P
from the tracer's perspective. But if this is a different P than before,
just like the pthread case, the parser is going to get confused, since
it looks like two Ps are running on the same M!
The case that CL 545515 actually handled was actually the non-pthread
case, specifically where the same P is reacquired by an M calling back
into Go. In this case, if we tolerate having a P, then what we'll
observe is the M stealing its own P from itself, then running with it.
Now that we know what the problem is, how do we fix it? This change
addresses the problem by emitting an extra event when encountering a
GoDestroySyscall with an active P in its context. In this case, it emits
an additional ProcSteal event to steal from itself, indicating that the
P stopped running. This removes any association between that M and that
P, resolving any ambiguity in the tracer.
There's one other minor detail that needs to be worked out, and that's
what happens to any *real* ProcSteal event that stole the P we're now
emitting an extra ProcSteal event for. Since, this event is going to
look for an M that may have moved on already and the P at this point is
already idle. Luckily, we have *exactly* the right fix for this. The
handler for GoDestroySyscall now moves any active P it has to the
ProcSyscallAbandoned state, indicating that we've lost information about
the P and that it should be treated as already idle. Conceptually this
all makes sense: this is a P in _Psyscall that has been abandoned by the
M it was previously bound to.
It's unfortunate how complicated this has all ended up being, but we can
take a closer look at that in the future.
Fixes #64060.
Change-Id: Ie9e6eb9cf738607617446e3487392643656069a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/546096
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
On non-pthread platforms, it's totally possible for the same M to
GoCreateSyscall/GoDestroySyscall on the same thread multiple times. That
same thread may hold onto its P through all those calls.
For #64060.
Change-Id: Ib968bfd439ecd5bc24fc98d78c06145b0d4b7802
Reviewed-on: https://go-review.googlesource.com/c/go/+/545515
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Currently we dump text traces to the build log on failure
unconditionally, but this may cause the old infrastructure's builds'
logs to get truncated. Avoid that by setting a threshold on the maximum
size of the text trace we're willing to dump.
We don't need this workaround on the new infrastructure -- logs don't
get truncated there.
Change-Id: I0f50f50bb4b90f87250b673fbe56f48235325610
Reviewed-on: https://go-review.googlesource.com/c/go/+/544216
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
Currently entersyscall_gcwait always emits a ProcStop event. Most of the
time, this is correct, since the thread that just put the P into
_Psyscall is the same one that is putting it into _Pgcstop. However it's
possible for another thread to steal the P, start running a goroutine,
and then enter another syscall, putting the P back into _Psyscall. In
this case ProcStop is incorrect; the P is getting stolen. This leads to
broken traces.
Fix this by always emitting a ProcSteal event from entersyscall_gcwait.
This means that most of the time a thread will be 'stealing' the proc
from itself when it enters this function, but that's theoretically fine.
A ProcSteal is really just a fancy ProcStop.
Well, it would be if the parser correctly handled a self-steal. This is
a minor bug that just never came up before, but it's an update order
error (the mState is looked up and modified, but then it's modified
again at the end of the function to match newCtx). There's really no
reason a self-steal shouldn't be allowed, so fix that up and add a test.
Change-Id: Iec3d7639d331e3f2d127f92ce50c2c4a7818fcd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/544215
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
This change emits regions in the goroutine-oriented task view (the
/trace endpoint with the taskid query variable set) in the same way the
old cmd/trace does.
For #60773.
Fixes #63960.
Change-Id: If6c3e7072c694c84a7d2d6c34df668f48d3acc2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/543995
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This is a nice-to-have that's now straightforward to do with the new
trace format. This change adds a new query variable passed to the
/trace endpoint called "view," which indicates the type of view to
use. It is orthogonal with task-related views.
Unfortunately a goroutine-based view isn't included because it's too
likely to cause the browser tab to crash.
For #60773.
For #63960.
Change-Id: Ifbcb8f2d58ffd425819bdb09c586819cb786478d
Reviewed-on: https://go-review.googlesource.com/c/go/+/543695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
This change implements support for the trace?focustask=<taskid> endpoint
in the trace tool for v2 traces.
Note: the one missing feature in v2 vs. v1 is that the "irrelevant" (but
still rendered) events are not grayed out. This basically includes
events that overlapped with events that overlapped with other events
that were in the task time period, but aren't themselves directly
associated. This is probably fine -- the UI already puts a very obvious
focus on the period of time the selected task was running.
For #60773.
For #63960.
Change-Id: I5c78a220ae816e331b74cb67c01c5cd98be40dd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/543596
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
This change adds support for the trace?goid=<goid> endpoint to the trace
tool for v2 traces.
In effect, this change actually implements a per-goroutine view. I tried
to add a link to the main page to enable a "view by goroutines" view
without filtering, but the web trace viewer broke the browser tab when
there were a few hundred goroutines. The risk of a browser hang probably
isn't worth the cases where this is nice, especially since filtering by
goroutine already works. Unfortunate, but c'est l'vie. Might be worth
revisiting if we change out the web viewer in the future.
For #60773.
For #63960.
Change-Id: I8e29f4ab8346af6708fd8824505c30f2c43db796
Reviewed-on: https://go-review.googlesource.com/c/go/+/543595
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
This change fills out the last of cmd/trace's subpages for v2 traces by
adding support for task and region endpoints.
For #60773.
For #63960.
Change-Id: Ifc4c660514b3904788785a1b20e3abc3bb9e55f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/542077
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This code will be useful for the new tracer, and there's no need to
duplicate it. This change copies it to internal/trace/traceviewer, adds
some comments, and renames it to TimeHistogram.
While we're here, let's get rid of the unused String method which has a
comment talking about how awful the rendering is.
Also, let's get rid of uses of niceDuration. We'd have to bring it
with us in the move and I don't think it's worth it. The difference
between the default time.Duration rendering and the niceDuration
rendering is usually a few extra digits of precision. Yes, it's noisier,
but AFAICT it's not substantially worse. It doesn't seem worth the new
API, even if it's just internal. We can also always bring it back later.
For #60773.
For #63960.
Change-Id: I795f58f579f1d503c540c3a40bed12e52bce38e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/542001
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
For v1 traces, cmd/trace contains code for analyzing tasks separately
from the goroutine analysis code present in internal/trace. As I started
to look into porting that code to v2 traces, I noticed that it wouldn't
be too hard to just generalize the existing v2 goroutine summary code to
generate exactly the same information.
This change does exactly that.
For #60773.
For #63960.
Change-Id: I0cdd9bf9ba11fb292a9ffc37dbf18c2a6a2483b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/542076
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
The v2 trace parser currently handles task inheritance and region task
association incorrectly. It assumes that a TaskID of 0 means that there
is no task. However, this is only true for task events. A TaskID of 0
means that a region gets assigned to the "background task." The parser
currently has no concept of a "background task."
Fix this by defining the background task as task ID 0 and redefining
NoTask to ^uint64(0). This aligns the TaskID values more closely with
other IDs in the parser and also enables disambiguating these two cases.
For #60773.
For #63960.
Change-Id: I09c8217b33b87c8f8f8ea3b0203ed83fd3b61e11
Reviewed-on: https://go-review.googlesource.com/c/go/+/543019
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
|
|
This change adds support for the pprof endpoints to cmd/trace/v2.
In the process, I realized we need to pass the goroutine summaries to
more places, and previous CLs had already done the goroutine analysis
during cmd/trace startup. This change thus refactors the goroutine
analysis API once again to operate in a streaming manner, and to run
at the same time as the initial trace parsing. Now we can include it in
the parsedTrace type and pass that around as the de-facto global trace
context.
Note: for simplicity, this change redefines "syscall" profiles to
capture *all* syscalls, not just syscalls that block. IIUC, this choice
was partly the result of a limitation in the previous trace format that
syscalls don't all have complete durations and many short syscalls are
treated as instant. To this end, this change modifies the text on the
main trace webpage to reflect this change.
For #60773.
For #63960.
Change-Id: I601d9250ab0849a0bfaef233fd9b1e81aca9a22a
Reviewed-on: https://go-review.googlesource.com/c/go/+/541999
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
For #60773.
For #63960.
Change-Id: Id97380f19267ec765b25a703ea3e2f284396ad75
Reviewed-on: https://go-review.googlesource.com/c/go/+/541998
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
The last change made the MMU rendering code common and introduced a new
API, but it was kind of messy. Part of the problem was that some of the
Javascript in the template for the main page referred to specific
endpoints on the server.
Fix this by having the Javascript access the same endpoint but with a
different query variable. Now the Javascript code doesn't depend on
specific endpoints, just on query variables for the current endpoint.
For #60773.
For #63960.
Change-Id: I1c559d9859c3a0d62e2094c9d4ab117890b63b31
Reviewed-on: https://go-review.googlesource.com/c/go/+/541259
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This change moves the MMU HTTP handlers and functionality into the
traceviewer package, since unlike the goroutine pages the vast majority
of that functionality is identical between v1 and v2. This change
involves some refactoring so that callers can plug in their own mutator
utilization computation functions (which is the only point of difference
between v1 and v2). The new interface isn't especially nice, but part of
the problem is the MMU handlers depend on specific endpoints to exist. A
follow-up CL will clean this up a bit.
Like the previous CL did for goroutine analysis, modify the v2 mutator
utilization API to accept a slice of trace events. Again, we might as
well reuse what was already parsed and will be needed for other
purposes. It also simplifies the API slightly.
For #60773.
For #63960.
Change-Id: I6c21ec8d1bf7e95eff5363d0e0005c9217fa00e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/541258
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
This is a complete fork and most of a rewrite of the goroutine analysis
pages for v2 traces. It fixes an issue with the old page where GC time
didn't really make any sense, generalizes the page and breaks things
down further, and adds clarifying text.
This change also modifies the SummarizeGoroutines API to not stream the
trace. This is unfortunate, but we're already reading and holding the
entire trace in memory for the trace viewer. We can revisit this
decision in the future. Also, we want to do this now because the
GoroutineSummary holds on to pointers to events, and these events will
be used by the user region and user task analyses. While tracev2 events
are values and they should be equivalent no matter how many times we
parse a trace, this lets us reference the event in the slice directly.
For #60773.
For #63960.
Fixes #62443.
Change-Id: I1c5ab68141869378843f4f2826686038e4533090
Reviewed-on: https://go-review.googlesource.com/c/go/+/541257
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Currently goroutine names are determined (for v2 traces) by
internal/tracev/2.Event.Stack, but this is wrong in general. For
example, if we end up seeing a transition from GoNotExist->GoRunnable
(goroutine creation) then we're taking the stack from the creator, not
the created goroutine (which is what we're naming at that point).
Use the StateTransition.Stack instead. This is always the correct one to
use because we're always naming the goroutine that the state transition
is for.
Change-Id: I3fc7c8e4f85dfee3802d666c0c091b6953c7d6cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/544317
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
Currently StateTransition.Stack is only set for the GoCreate case,
because there are two stacks and we need to distinguish them. But the
docs for StateTransition.Stack say that that stack always references the
resource that is transitioning. There are quite a few cases where
Event.Stack is actually the appropriate stack to for
StateTransition.Stack, but in these cases it's left empty, and the
caller just needs to understand which one to look at. This isn't great.
Forward Event.Stack to StateTransition.Stack whenever Event.Stack also
refers to the resource experiencing the state transition.
Change-Id: Ie43fc6036f2712c7982174d5739d95765312dfcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/544316
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This change refactors the cmd/trace package and adds most of the support
for v2 traces.
The following features of note are missing in this CL and will be
implemented in follow-up CLs:
- The focustask filter for the trace viewer
- The taskid filter for the trace viewer
- The goid filter for the trace viewer
- Pprof profiles
- The MMU graph
- The goroutine analysis pages
- The task analysis pages
- The region analysis pages
This CL makes one notable change to the trace CLI: it makes the -d flag
accept an integer to set the debug mode. For old traces -d != 0 works
just like -d. For new traces -d=1 means the high-level events and -d=2
means the low-level events.
Thanks to Felix Geisendörfer (felix.geisendoerfer@datadoghq.com) for
doing a lot of work on this CL; I picked this up from him and got a
massive headstart as a result.
For #60773.
For #63960.
Change-Id: I3626e22473227c5980134a85f1bb6a845f567b1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/542218
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
|
|
Change-Id: Ic61fb181923159e80a86a41582e83ec466ab9bc4
GitHub-Last-Rev: 92469845665fa1f864d257c8bc175201a43b4d43
GitHub-Pull-Request: golang/go#64080
Reviewed-on: https://go-review.googlesource.com/c/go/+/541741
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Jes Cok <xigua67damn@gmail.com>
|
|
After landing the new execution tracer, the Windows builders failed with
some new errors.
Currently the GoSyscallBegin event has no indicator that its the target
of a ProcSteal event. This can lead to an ambiguous situation that is
unresolvable if timestamps are broken. For instance, if the tracer sees
the ProcSteal event while a goroutine has been observed to be in a
syscall (one that, for instance, did not actually lose its P), it will
proceed with the ProcSteal incorrectly.
This is a little abstract. For a more concrete example, see the
go122-syscall-steal-proc-ambiguous test.
This change resolves this ambiguity by interleaving GoSyscallBegin
events into how Ps are sequenced. Because a ProcSteal has a sequence
number (it has to, it's stopping a P from a distance) it necessarily
has to synchronize with a precise ProcStart event. This change basically
just extends this synchronization to GoSyscallBegin, so the ProcSteal
can't advance until _exactly the right_ syscall has been entered.
This change removes the test skip, since it and CL 541695 fix the two
main issues observed on Windows platforms.
For #60773.
Fixes #64061.
Change-Id: I069389cd7fe1ea903edf42d79912f6e2bcc23f62
Reviewed-on: https://go-review.googlesource.com/c/go/+/541696
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
An out-of-memory error in this test has been observed on 32-bit
platforms, so halve the memory footprint of the test. Also halve the
size of steady-state allocation rate in bytes. The end result should be
approximately the same GC CPU load but at half the memory usage.
Change-Id: I2c2d335da7dc4c5c58cb9d92b6e5a4ece55d24a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/542215
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
Currently the trace parser enforces that the timestamps for a series of
a batches on the same M come in order. We cannot actually assume this in
general because we don't trust timestamps. The source of truth on the
batch order is the order in which they were emitted. If that's wrong, it
should quickly become evident in the trace.
For #60773.
For #64061.
Change-Id: I7d5a407c9568dd1ce0b79d51b2b538ed6072b26d
Reviewed-on: https://go-review.googlesource.com/c/go/+/541695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
There are a couple known issues here. Disable the tests for now so it's
not blocking anyone.
For #64061.
Change-Id: Iaaa9007b93ea78739cb7d2b59b2a1715de29d72b
Reviewed-on: https://go-review.googlesource.com/c/go/+/541197
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
For #63960.
Change-Id: I3d8d1567c4ee213e2ffd2bd91d0ffae9c4c42b92
Reviewed-on: https://go-review.googlesource.com/c/go/+/541535
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
|
|
For #63960.
Change-Id: I1efe35435e32623aba894a915114e394570ebc56
Reviewed-on: https://go-review.googlesource.com/c/go/+/540259
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This change adds a new MutatorUtilization for traces for Go 1.22+.
To facilitate testing, it also generates a short trace with the
gc-stress.go test program (shortening its duration to 10ms) and adds it
to the tests for the internal/trace/v2 package. Notably, we make sure
this trace has a GCMarkAssistActive event to test that codepath.
For #63960.
For #60773.
Change-Id: I2e61f545988677be716818e2a08641c54c4c201f
Reviewed-on: https://go-review.googlesource.com/c/go/+/540256
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
This change mostly implements the design described in #60773 and
includes a new scalable parser for the new trace format, available in
internal/trace/v2. I'll leave this commit message short because this is
clearly an enormous CL with a lot of detail.
This change does not hook up the new tracer into cmd/trace yet. A
follow-up CL will handle that.
For #60773.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race
Change-Id: I5d2aca2cc07580ed3c76a9813ac48ec96b157de0
Reviewed-on: https://go-review.googlesource.com/c/go/+/494187
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Currently STW events are only emitted for GC STWs. There's little reason
why the trace can't contain events for every STW: they're rare so don't
take up much space in the trace, yet being able to see when the world
was stopped is often critical to debugging certain latency issues,
especially when they stem from user-level APIs.
This change adds new "kinds" to the EvGCSTWStart event, renames the
GCSTW events to just "STW," and lets the parser deal with unknown STW
kinds for future backwards compatibility.
But, this change must break trace compatibility, so it bumps the trace
version to Go 1.21.
This change also includes a small cleanup in the trace command, which
previously checked for STW events when deciding whether user tasks
overlapped with a GC. Looking at the source, I don't see a way for STW
events to ever enter the stream that that code looks at, so that
condition has been deleted.
Change-Id: I9a5dc144092c53e92eb6950e9a5504a790ac00cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/494495
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
|
|
A previous iteration of the tracer's user annotation API had different
names for tasks and regions, and used to return functions for ending
them rather than types with End methods. This CL updates the doc
comments to reflect those changes, and also fixes up the internal
documentation of the events (similar to go.dev/cl/465335, the stack
argument was in the wrong place in the list).
The User Log event internal documentation might also look wrong since
the value argument follows the stack argument. However, the User Log
event is a special case where the log message is appended immediately
following the normal event, including the stack argument. There isn't
much room to clarify this next to the event type definitions, so this CL
clarifies the comment where the event is encoded.
Change-Id: I846c709f6026ef01c0a272557d6390b2c17074e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/472955
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Nick Ripley <nick.ripley@datadoghq.com>
|
|
Fixes #58327
Change-Id: I15593e6ee42e04f2de13804ef26e0e66a2307db0
GitHub-Last-Rev: 7e0d04b503d2ba217b76b1ac668a67cd72ec6d7c
GitHub-Pull-Request: golang/go#58338
Reviewed-on: https://go-review.googlesource.com/c/go/+/465335
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
Extra Ms may lead to the "no consistent ordering of events possible" error when parsing trace file with cgo enabled, since:
1. The gs in the extra Ms may be in `_Gdead` status while starting trace by invoking `runtime.StartTrace`,
2. and these gs will trigger `traceEvGoSysExit` events in `runtime.exitsyscall` when invoking go functions from c,
3. then, the events of those gs are under non-consistent ordering, due to missing the previous events.
Add two events, `traceEvGoCreate` and `traceEvGoInSyscall`, in `runtime.StartTrace`, will make the trace parser happy.
Fixes #29707
Change-Id: I2fd9d1713cda22f0ddb36efe1ab351f88da10881
GitHub-Last-Rev: 7bbfddb81b70041250e3c59ce53bea44f7afd2c3
GitHub-Pull-Request: golang/go#54974
Reviewed-on: https://go-review.googlesource.com/c/go/+/429858
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: xie cui <523516579@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
|
|
Change-Id: Ic7f827b8ee758c991ed76be65026b0330207deea
Reviewed-on: https://go-review.googlesource.com/c/go/+/428262
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
assosciated -> associated
Change-Id: Id1cbbdea12f0239a9e173ece934c18cc9ffcf0f4
GitHub-Last-Rev: 7fea9de6c27b95f91591db4b5bd9023eb80dc522
GitHub-Pull-Request: golang/go#54739
Reviewed-on: https://go-review.googlesource.com/c/go/+/425596
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
|
|
This reverts commit ea9c3fd42d94182ce6f87104b68a51ea92f1a571.
Reason for revert: break linux/ricsv64, openbsd/arm, illumos/amd64 builders
Change-Id: I98479a8f63e76eed89a0e8846acf2c73e8441377
Reviewed-on: https://go-review.googlesource.com/c/go/+/423437
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
|
|
Extra Ms may lead to the "no consistent ordering of events possible" error when parsing trace file with cgo enabled, since:
1. The gs in the extra Ms may be in `_Gdead` status while starting trace by invoking `runtime.StartTrace`,
2. and these gs will trigger `traceEvGoSysExit` events in `runtime.exitsyscall` when invoking go functions from c,
3. then, the events of those gs are under non-consistent ordering, due to missing the previous events.
Add two events, `traceEvGoCreate` and `traceEvGoInSyscall`, in `runtime.StartTrace`, will make the trace parser happy.
Fixes #29707
Change-Id: I7cc4b80822d2c46591304a59c9da2c9fc470f1d0
GitHub-Last-Rev: 445de8eaf3fb54e12795ac31e26650f821c5efbc
GitHub-Pull-Request: golang/go#53284
Reviewed-on: https://go-review.googlesource.com/c/go/+/411034
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
If a goroutine is started within a user region, internal/trace assigns
the child goroutine a nameless region for its entire lifetime which is
assosciated the same task as the parent's region.
This is not strictly necessary: a child goroutine is not necessarily
related to the task unless it performs some task operation (in which
case it will be associated with the task through the standard means).
However, it can be quite handy to see child goroutines within a region,
which may be child worker goroutines that you simply didn't perform task
operations on.
If the first GC occurs during a region, the GC worker goroutines will
also inherit a child region. We know for sure that these aren't related
to the task, so filter them out from the region list.
Note that we can't exclude system goroutines from setting activeRegions
in EvGoCreate handling, because we don't know the goroutine start
function name until the first EvGoStart.
Fixes #53784.
Change-Id: Ic83d84e23858a8400a76d1ae2f1418ef49951178
Reviewed-on: https://go-review.googlesource.com/c/go/+/416858
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Update instructions to match what seems to be the historical practice:
to generate canned traces when a version is finalized, rather than
waiting until it is superseded.
Follow rename of trace-internal tests from "Span" to "Region". Update
the net/http test invocation to match the apparent intent and the actual
http_1_5_good behavior (about 7ms of total run time and trace file size
under 50kB).
Change-Id: Ifd4c85882159478852e0b8f0d771b6f16b8f3c1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/413776
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
When the CPU profiler and execution tracer are both active, report the
CPU profile samples in the execution trace data stream.
Include only samples that arrive on the threads known to the runtime,
but include them even when running g0 (such as near the scheduler) or if
there's no P (such as near syscalls).
Render them in "go tool trace" as instantaneous events.
For #16895
Change-Id: I0aa501a7b450c971e510961c0290838729033f7f
Reviewed-on: https://go-review.googlesource.com/c/go/+/400795
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes #49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
This change moves next_gc and last_next_gc into gcControllerState under
the names heapGoal and lastHeapGoal respectively. These are
fundamentally GC pacer related values, and so it makes sense for them to
live here.
Partially generated by
rf '
ex . {
memstats.next_gc -> gcController.heapGoal
memstats.last_next_gc -> gcController.lastHeapGoal
}
'
except for updates to comments and gcControllerState methods, where
they're accessed through the receiver, and trace-related renames of
NextGC -> HeapGoal, while we're here.
For #44167.
Change-Id: I1e871ad78a57b01be8d9f71bd662530c84853bed
Reviewed-on: https://go-review.googlesource.com/c/go/+/306603
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
os.ReadDir is a replacement for ioutil.ReadDir that returns
a slice of fs.DirEntry instead of fs.FileInfo, meaning it is the
more efficient form.
This CL updates call sites throughout the Go source tree
wherever possible. As usual, code built using the Go 1.4
bootstrap toolchain is not included. There is also a use in
go/build that appears in the public API and can't be changed,
at least not without additional changes.
Fixes #42026.
Change-Id: Icfc9dd52c6045020f6830e22c72128499462d561
Reviewed-on: https://go-review.googlesource.com/c/go/+/266366
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)
Update the Go tree to use the preferred names.
As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.
ReadDir changes are in a separate CL, because they are not a
simple search and replace.
For #42026.
Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Change-Id: I9f23490a565b81b155642fe301f897c1d0af03fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/232810
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
|
|
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.
After this change, all.bash is under 5 minutes again on my laptop.
For #26473.
Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
