aboutsummaryrefslogtreecommitdiff
path: root/src/internal
AgeCommit message (Collapse)Author
2024-01-25all: prealloc slice with possible minimum capabilitiesShulhan
2024-01-24runtime: short path for equal pointers in arm64 memequalMauri de Souza Meneguzzo
If memequal is invoked with the same pointers as arguments it ends up comparing the whole memory contents, instead of just comparing the pointers. This effectively makes an operation that could be O(1) into O(n). All the other architectures already have this optimization in place. For instance, arm64 also have it, in memequal_varlen. Such optimization is very specific, one case that it will probably benefit is programs that rely heavily on interning of strings. goos: darwin goarch: arm64 pkg: bytes │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ Equal/same/1-8 2.678n ± ∞ ¹ 2.400n ± ∞ ¹ -10.38% (p=0.008 n=5) Equal/same/6-8 3.267n ± ∞ ¹ 2.431n ± ∞ ¹ -25.59% (p=0.008 n=5) Equal/same/9-8 2.981n ± ∞ ¹ 2.385n ± ∞ ¹ -19.99% (p=0.008 n=5) Equal/same/15-8 2.974n ± ∞ ¹ 2.390n ± ∞ ¹ -19.64% (p=0.008 n=5) Equal/same/16-8 2.983n ± ∞ ¹ 2.380n ± ∞ ¹ -20.21% (p=0.008 n=5) Equal/same/20-8 3.567n ± ∞ ¹ 2.384n ± ∞ ¹ -33.17% (p=0.008 n=5) Equal/same/32-8 3.568n ± ∞ ¹ 2.385n ± ∞ ¹ -33.16% (p=0.008 n=5) Equal/same/4K-8 78.040n ± ∞ ¹ 2.378n ± ∞ ¹ -96.95% (p=0.008 n=5) Equal/same/4M-8 78713.000n ± ∞ ¹ 2.385n ± ∞ ¹ -100.00% (p=0.008 n=5) Equal/same/64M-8 1348095.000n ± ∞ ¹ 2.381n ± ∞ ¹ -100.00% (p=0.008 n=5) geomean 43.52n 2.390n -94.51% ¹ need >= 6 samples for confidence interval at level 0.95 │ old.txt │ new.txt │ │ B/s │ B/s vs base │ Equal/same/1-8 356.1Mi ± ∞ ¹ 397.3Mi ± ∞ ¹ +11.57% (p=0.008 n=5) Equal/same/6-8 1.711Gi ± ∞ ¹ 2.298Gi ± ∞ ¹ +34.35% (p=0.008 n=5) Equal/same/9-8 2.812Gi ± ∞ ¹ 3.515Gi ± ∞ ¹ +24.99% (p=0.008 n=5) Equal/same/15-8 4.698Gi ± ∞ ¹ 5.844Gi ± ∞ ¹ +24.41% (p=0.008 n=5) Equal/same/16-8 4.995Gi ± ∞ ¹ 6.260Gi ± ∞ ¹ +25.34% (p=0.008 n=5) Equal/same/20-8 5.222Gi ± ∞ ¹ 7.814Gi ± ∞ ¹ +49.63% (p=0.008 n=5) Equal/same/32-8 8.353Gi ± ∞ ¹ 12.496Gi ± ∞ ¹ +49.59% (p=0.008 n=5) Equal/same/4K-8 48.88Gi ± ∞ ¹ 1603.96Gi ± ∞ ¹ +3181.17% (p=0.008 n=5) Equal/same/4M-8 49.63Gi ± ∞ ¹ 1637911.85Gi ± ∞ ¹ +3300381.91% (p=0.008 n=5) Equal/same/64M-8 46.36Gi ± ∞ ¹ 26253069.97Gi ± ∞ ¹ +56626517.99% (p=0.008 n=5) geomean 6.737Gi 122.7Gi +1721.01% ¹ need >= 6 samples for confidence interval at level 0.95 Fixes #64381 Change-Id: I7d423930a688edd88c4ba60d45e097296d9be852 GitHub-Last-Rev: ae8189fafb1cba87b5394f09f971746ae9299273 GitHub-Pull-Request: golang/go#64419 Reviewed-on: https://go-review.googlesource.com/c/go/+/545416 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com>
2024-01-24cmd/go: add GORISCV64 environment variableMark Ryan
The variable represents the RISC-V user-mode application profile for which to compile. Valid values are rva20u64 (the default) and rva22u64. Setting GORISCV64=rva20u64 defines the riscv64.rva20u64 build tag, sets the internal variable buildcfg.GORISCV64 to 20 and defines the macro GORISCV64_rva20u64 for use in assembly language code. Setting GORISCV64=rva22u64 defines the riscv64.rva20u64 and riscv64.rva22u64 build tags, sets the internal variable buildcfg.GORISCV64 to 22 and defines the macro GORISCV64_rva22u64 for use in assembly language code. This patch only provides a mechanism for the compiler and hand-coded assembly language functions to take advantage of the RISC-V extensions mandated by the application profiles. Further patches will be required to get the compiler/assembler and assembly language functions to actually generate and use these extensions. Fixes #61476 Change-Id: I9195ae6ee71703cd2112160e89157ab63b8391af Reviewed-on: https://go-review.googlesource.com/c/go/+/541135 Reviewed-by: M Zhuo <mengzhuo1203@gmail.com> Reviewed-by: Joel Sing <joel@sing.id.au> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Wang Yaduo <wangyaduo@linux.alibaba.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: M Zhuo <mengzhuo1203@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2024-01-23net,internal/poll: load ws2_32.dll only when net is importedqmuntal
On Windows, ws2_32.dll is loaded and WSA initialized even if websockets are not used. This CL delays loading of ws2_32.dll and starting WSA until net is initialized. Change-Id: I07ea8241d79709cd4e80d29ba0d792c7444bbfe9 Reviewed-on: https://go-review.googlesource.com/c/go/+/557015 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-23internal/reflectlite: delete TODO pass safe to packEface don't need to copy ↵qiulaidongfeng
if safe==true reflect on the https://go-review.googlesource.com/c/go/+/548436 delete TODO the same. Change-Id: I5b278cbfcb4108e5ffb332ba82dafb1eaa2bd6b2 GitHub-Last-Rev: cfc39509085477e9cba8e8ba1698653837a12301 GitHub-Pull-Request: golang/go#64628 Reviewed-on: https://go-review.googlesource.com/c/go/+/548615 Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2024-01-23runtime,internal/poll: move websocket handling out of the runtime on Windowsqmuntal
On Windows, the netpoll is currently coupled with the websocket usage in the internal/poll package. This CL moves the websocket handling out of the runtime and puts it into the internal/poll package, which already contains most of the async I/O logic for websockets. This is a good refactor per se, as the Go runtime shouldn't know about websockets. In addition, it will make it easier (in a future CL) to only load ws2_32.dll when the Go program actually uses websockets. Change-Id: Ic820872cf9bdbbf092505ed7f7504edb6687735e Reviewed-on: https://go-review.googlesource.com/c/go/+/556936 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-01-23cmd/compile/internal/ssagen,runtime: merge trace consts into internal/abiqiulaidongfeng
For #59670 Change-Id: Iec85ee7312bb566b3f1224424f7d27bf4e408b13 GitHub-Last-Rev: c620abf9673e166505821d75717e820776abc302 GitHub-Pull-Request: golang/go#64905 Reviewed-on: https://go-review.googlesource.com/c/go/+/553295 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2024-01-22Revert "cmd/preprofile: Add preprocess tool to pre-parse the profile file."Michael Pratt
This reverts CL 529738. Reason for revert: Breaking longtest builders For #58102. Fixes #65220. Change-Id: Id295e3249da9d82f6a9e4fc571760302a1362def Reviewed-on: https://go-review.googlesource.com/c/go/+/557460 Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22runtime: use the correct M ID for syscalling goroutines in tracesMichael Anthony Knyszek
Earlier in the development of the new tracer, m.id was used as a the canonical ID for threads. Later, we switched to m.procid because it matches the underlying OS resource. However, in that switch, we missed a spot. The tracer catches and emits statuses for goroutines that have remained in either waiting or syscall across a whole generation, and emits a thread ID for the latter set. The ID being used here, however, was m.id instead of m.procid, like the rest of the tracer. This CL also adds a regression test. In order to make the regression test actually catch the failure, we also have to make the parser a little less lenient about GoStatus events with GoSyscall: if this isn't the first generation, then we should've seen the goroutine bound to an M already when its status is getting emitted for its context. If we emit the wrong ID, then we'll catch the issue when we emit the right ID when the goroutine exits the syscall. Fixes #65196. Change-Id: I78b64fbea65308de5e1291c478a082a732a8bf9f Reviewed-on: https://go-review.googlesource.com/c/go/+/557456 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22cmd/preprofile: Add preprocess tool to pre-parse the profile file.Jin Lin
The pgo compilation time is very long if the profile file is large. We added a preprocess tool to pre-parse profile file in order to expedite the compile time. Change-Id: I6f50bbd01f242448e2463607a9b63483c6ca9a12 Reviewed-on: https://go-review.googlesource.com/c/go/+/529738 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-01-22cmd/link,runtime: merge minfunc const info internal/abiqiulaidongfeng
For #59670 Change-Id: If2b05b1ba30b607b518577b0e11ba5a0b07999c5 GitHub-Last-Rev: a664aa18b5ef674dc2d05c1f7533e1974d265894 GitHub-Pull-Request: golang/go#64906 Reviewed-on: https://go-review.googlesource.com/c/go/+/553276 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2024-01-19internal/goversion: update Version to 1.23Carlos Amedee
(The corresponding update for the last release cycle was CL 510735.) For #40705 For #64340 Change-Id: I123ce68131a6c7b0344cab54cd29402cabb57225 Reviewed-on: https://go-review.googlesource.com/c/go/+/557155 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-18go/types, types2: fix range clause checks for constant range expressionsRobert Griesemer
Add missing checks for the case where the range expression is a (possibly untyped) constant integer expression. Add context parameter to assignVar for better error message where the expression is part of a range clause. Also, rename s/expr/Expr/ where it denotes an AST expression, for clarity. Fixes #65133. For #65137. Change-Id: I72962d76741abe79f613e251f7b060e99261d3ae Reviewed-on: https://go-review.googlesource.com/c/go/+/556398 Run-TryBot: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
2024-01-13internal/trace/v2: reject events for goroutines with unknown statesDominik Honnef
Change-Id: Ifc472ed4cf0433d06f43559930ac80df23656a6e Reviewed-on: https://go-review.googlesource.com/c/go/+/555496 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-12internal/trace/v2: avoid several panics for malformed tracesDominik Honnef
This addresses some panics (out of bounds slice accesses and nil pointer dereferences) when parsing malformed data. These were found via light fuzzing, not by any rigorous means, and more potential panics probably exist. Fixes #64878. Fixes #64879. Change-Id: I4085788ba7dc91fec62e4abd88f50777577db42f Reviewed-on: https://go-review.googlesource.com/c/go/+/552995 Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-11runtime: ensure we free unrolled GC bitmapsKeith Randall
CL 555355 has a bug in it - the GC program flag was also used to decide when to free the unrolled bitmap. After that CL, we just don't free any unrolled bitmaps, leading to a memory leak. Use a separate flag to track types that need to be freed when their corresponding object is freed. Change-Id: I841b65492561f5b5e1853875fbd8e8a872205a84 Reviewed-on: https://go-review.googlesource.com/c/go/+/555416 Auto-Submit: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-18internal/syscall/windows: fix the signature of SetFileInformationByHandleBryan C. Mills
Also fix its call site in internal/poll to pass the length of the actual buffer instead of an unrelated variable, and update the definition of FILE_BASIC_INFO to match the documented field types and add padding that is empirically needed on the 386 architecture. Passing a pointer to a Go-allocated buffer as type uintptr violates the unsafe.Pointer conversion rules, which allow such a conversion only in the call expression itself for a call to syscall.Syscall or equivalent. That can allow the buffer to be corrupted arbitrarily if the Go runtime happens to garbage-collect it while the call to SetFileInformationByHandle is in progress. The Microsoft documentation for SetFileInformationByHandle specifies its third argument type as LPVOID, which corresponds to Go's unsafe.Pointer, not uintptr. Fixes #58933 (maybe). Change-Id: If577b57adea9922f5fcca55e46030c703d8f035c Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/549256 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Than McIntosh <thanm@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2023-12-15runtime: add race annotations in IncNonDefaultDavid Chase
Also use CompareAndSwap to make the code actually less racy. Added a test which will be meaningful when run under the race detector (tested it -race with broken fix in runtime, it failed). Fixes #64649 Change-Id: I5972e08901d1adc8ba74858edad7eba91be1b0ce Reviewed-on: https://go-review.googlesource.com/c/go/+/549796 Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-14all: remove newline characters after return statementsDanil Timerbulatov
This commit is aimed at improving the readability and consistency of the code base. Extraneous newline characters were present after some return statements, creating unnecessary separation in the code. Fixes #64610 Change-Id: Ic1b05bf11761c4dff22691c2f1c3755f66d341f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/548316 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-12-13go/types, types2: report error for range over int if Go version < 1.22Robert Griesemer
Fixes #64704. Change-Id: Ied3af46ab534343cdafba5ee27680b9c6ef3d37a Reviewed-on: https://go-review.googlesource.com/c/go/+/549459 Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Alan Donovan <adonovan@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-13go/types, types2: avoid confusing error message "have m(T), want m(T)"Robert Griesemer
This is a partial fix for situations where a method lookup leads to an error due to non-matching signatures, but where the signatures print exactly the same. This can happen if both signatures contain type parameters (after instantiation) and the type parameters have the same name (such as "T"). For now, rather than printing a confusing error message in this case, leave away the confusing part of the error message (at the cost of providing slightly less information). In the long run, we need to find a better solution for this problem; but this seems better than what we had before. For #61685. Change-Id: I259183f08b9db400ffc8e1cf447967c640a0f444 Reviewed-on: https://go-review.googlesource.com/c/go/+/549296 Reviewed-by: Robert Griesemer <gri@google.com> Run-TryBot: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Findley <rfindley@google.com> Auto-Submit: Robert Griesemer <gri@google.com>
2023-12-12go/types, types2: flip message contents for reverse type inference errorsRobert Griesemer
Add a new flag 'reverse' to control the formatting of type inference error messages. This change only impacts error messages. Fixes #60747. Change-Id: I81e13075e3157252ccc09f358bd29bd676c34499 Reviewed-on: https://go-review.googlesource.com/c/go/+/549055 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Run-TryBot: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-12go/types, types2: better error message for cases of reverse type inferenceRobert Griesemer
Introduce a new type "target" to pass around target types together with a suitable description (typically a variable name) for a better error message. As a side effect, using a specific type (target), rather than just Type avoids accidental confusion with other types. Use the target type description for a better error message in some cases. The error message can be further improved by flipping the order of the sentence (for another CL to keep this one small and simple). Also, and unrelated to this fix, remove the first argument to errorf in infer.go: the argument is always "type" (there's only one call). For #60747. Change-Id: I2118d0fe9e2b4aac959371941064e0e9ca7b3b6e Reviewed-on: https://go-review.googlesource.com/c/go/+/548995 Auto-Submit: Robert Griesemer <gri@google.com> Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-11crypto/x509: gate Policies marshaling with GODEBUGRoland Shoemaker
Use a GODEBUG to choose which certificate policy field to use. If x509usepolicies=1 is set, use the Policies field, otherwise use the PolicyIdentifiers field. Fixes #64248 Change-Id: I3f0b56102e0bac4ebe800497717c61c58ef3f092 Reviewed-on: https://go-review.googlesource.com/c/go/+/546916 Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-08cmd/compile: fix error message for mismatch between the number of type ↵Raghvender
params and arguments Fixes #64276 Change-Id: Ib6651669904e6ea0daf275d85d8bd008b8b21cc6 Reviewed-on: https://go-review.googlesource.com/c/go/+/544018 Reviewed-by: raghvender sundarjee <raghvenders@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Run-TryBot: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-07internal/profile: fully decode proto even if there are no samplesMichael Pratt
This is a partial revert of CL 483137. CL 483137 started checking errors in postDecode, which is good. Now we can catch more malformed pprof protos. However this made TestEmptyProfile fail, so an early return was added when the profile was "empty" (no samples). Unfortunately, this was problematic. Profiles with no samples can still be valid, but skipping postDecode meant that the resulting Profile was missing values from the string table. In particular, net/http/pprof needs to parse empty profiles in order to pass through the sample and period types to a final output proto. CL 483137 broke this behavior. internal/profile.Parse is only used in two places: in cmd/compile to parse PGO pprof profiles, and in net/http/pprof to parse before/after pprof profiles for delta profiles. In both cases, the input is never literally empty (0 bytes). Even a pprof proto with no samples still contains some header fields, such as sample and period type. Upstream github.com/google/pprof/profile even has an explicit error on 0 byte input, so `go tool pprof` will not support such an input. Thus TestEmptyProfile was misleading; this profile doesn't need to support empty input at all. Resolve this by removing TestEmptyProfile and replacing it with an explicit error on empty input, as upstream github.com/google/pprof/profile has. For non-empty input, always run postDecode to ensure the string table is processed. TestConvertCPUProfileEmpty is reverted back to assert the values from before CL 483137. Note that in this case "Empty" means no samples, not a 0 byte input. Continue to allow empty files for PGO in order to minimize the chance of last minute breakage if some users have empty files. Fixes #64566. Change-Id: I83a1f0200ae225ac6da0009d4b2431fe215b283f Reviewed-on: https://go-review.googlesource.com/c/go/+/547996 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-12-06iter, runtime: add coroutine supportRuss Cox
The exported API is only available with GOEXPERIMENT=rangefunc. This will let Go 1.22 users who want to experiment with rangefuncs access an efficient implementation of iter.Pull and iter.Pull2. For #61897. Change-Id: I6ef5fa8f117567efe4029b7b8b0f4d9b85697fb7 Reviewed-on: https://go-review.googlesource.com/c/go/+/543319 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05math/rand, math/rand/v2: use ChaCha8 for global randRuss Cox
Move ChaCha8 code into internal/chacha8rand and use it to implement runtime.rand, which is used for the unseeded global source for both math/rand and math/rand/v2. This also affects the calculation of the start point for iteration over very very large maps (when the 32-bit fastrand is not big enough). The benefit is that misuse of the global random number generators in math/rand and math/rand/v2 in contexts where non-predictable randomness is important for security reasons is no longer a security problem, removing a common mistake among programmers who are unaware of the different kinds of randomness. The cost is an extra 304 bytes per thread stored in the m struct plus 2-3ns more per random uint64 due to the more sophisticated algorithm. Using PCG looks like it would cost about the same, although I haven't benchmarked that. Before this, the math/rand and math/rand/v2 global generator was wyrand (https://github.com/wangyi-fudan/wyhash). For math/rand, using wyrand instead of the Mitchell/Reeds/Thompson ALFG was justifiable, since the latter was not any better. But for math/rand/v2, the global generator really should be at least as good as one of the well-studied, specific algorithms provided directly by the package, and it's not. (Wyrand is still reasonable for scheduling and cache decisions.) Good randomness does have a cost: about twice wyrand. Also rationalize the various runtime rand references. goos: linux goarch: amd64 pkg: math/rand/v2 cpu: AMD Ryzen 9 7950X 16-Core Processor │ bbb48afeb7.amd64 │ 5cf807d1ea.amd64 │ │ sec/op │ sec/op vs base │ ChaCha8-32 1.862n ± 2% 1.861n ± 2% ~ (p=0.825 n=20) PCG_DXSM-32 1.471n ± 1% 1.460n ± 2% ~ (p=0.153 n=20) SourceUint64-32 1.636n ± 2% 1.582n ± 1% -3.30% (p=0.000 n=20) GlobalInt64-32 2.087n ± 1% 3.663n ± 1% +75.54% (p=0.000 n=20) GlobalInt64Parallel-32 0.1042n ± 1% 0.2026n ± 1% +94.48% (p=0.000 n=20) GlobalUint64-32 2.263n ± 2% 3.724n ± 1% +64.57% (p=0.000 n=20) GlobalUint64Parallel-32 0.1019n ± 1% 0.1973n ± 1% +93.67% (p=0.000 n=20) Int64-32 1.771n ± 1% 1.774n ± 1% ~ (p=0.449 n=20) Uint64-32 1.863n ± 2% 1.866n ± 1% ~ (p=0.364 n=20) GlobalIntN1000-32 3.134n ± 3% 4.730n ± 2% +50.95% (p=0.000 n=20) IntN1000-32 2.489n ± 1% 2.489n ± 1% ~ (p=0.683 n=20) Int64N1000-32 2.521n ± 1% 2.516n ± 1% ~ (p=0.394 n=20) Int64N1e8-32 2.479n ± 1% 2.478n ± 2% ~ (p=0.743 n=20) Int64N1e9-32 2.530n ± 2% 2.514n ± 2% ~ (p=0.193 n=20) Int64N2e9-32 2.501n ± 1% 2.494n ± 1% ~ (p=0.616 n=20) Int64N1e18-32 3.227n ± 1% 3.205n ± 1% ~ (p=0.101 n=20) Int64N2e18-32 3.647n ± 1% 3.599n ± 1% ~ (p=0.019 n=20) Int64N4e18-32 5.135n ± 1% 5.069n ± 2% ~ (p=0.034 n=20) Int32N1000-32 2.657n ± 1% 2.637n ± 1% ~ (p=0.180 n=20) Int32N1e8-32 2.636n ± 1% 2.636n ± 1% ~ (p=0.763 n=20) Int32N1e9-32 2.660n ± 2% 2.638n ± 1% ~ (p=0.358 n=20) Int32N2e9-32 2.662n ± 2% 2.618n ± 2% ~ (p=0.064 n=20) Float32-32 2.272n ± 2% 2.239n ± 2% ~ (p=0.194 n=20) Float64-32 2.272n ± 1% 2.286n ± 2% ~ (p=0.763 n=20) ExpFloat64-32 3.762n ± 1% 3.744n ± 1% ~ (p=0.171 n=20) NormFloat64-32 3.706n ± 1% 3.655n ± 2% ~ (p=0.066 n=20) Perm3-32 32.93n ± 3% 34.62n ± 1% +5.13% (p=0.000 n=20) Perm30-32 202.9n ± 1% 204.0n ± 1% ~ (p=0.482 n=20) Perm30ViaShuffle-32 115.0n ± 1% 114.9n ± 1% ~ (p=0.358 n=20) ShuffleOverhead-32 112.8n ± 1% 112.7n ± 1% ~ (p=0.692 n=20) Concurrent-32 2.107n ± 0% 3.725n ± 1% +76.75% (p=0.000 n=20) goos: darwin goarch: arm64 pkg: math/rand/v2 │ bbb48afeb7.arm64 │ 5cf807d1ea.arm64 │ │ sec/op │ sec/op vs base │ ChaCha8-8 2.480n ± 0% 2.429n ± 0% -2.04% (p=0.000 n=20) PCG_DXSM-8 2.531n ± 0% 2.530n ± 0% ~ (p=0.877 n=20) SourceUint64-8 2.534n ± 0% 2.533n ± 0% ~ (p=0.732 n=20) GlobalInt64-8 2.172n ± 1% 4.794n ± 0% +120.67% (p=0.000 n=20) GlobalInt64Parallel-8 0.4320n ± 0% 0.9605n ± 0% +122.32% (p=0.000 n=20) GlobalUint64-8 2.182n ± 0% 4.770n ± 0% +118.58% (p=0.000 n=20) GlobalUint64Parallel-8 0.4307n ± 0% 0.9583n ± 0% +122.51% (p=0.000 n=20) Int64-8 4.107n ± 0% 4.104n ± 0% ~ (p=0.416 n=20) Uint64-8 4.080n ± 0% 4.080n ± 0% ~ (p=0.052 n=20) GlobalIntN1000-8 2.814n ± 2% 5.643n ± 0% +100.50% (p=0.000 n=20) IntN1000-8 4.141n ± 0% 4.139n ± 0% ~ (p=0.140 n=20) Int64N1000-8 4.140n ± 0% 4.140n ± 0% ~ (p=0.313 n=20) Int64N1e8-8 4.140n ± 0% 4.139n ± 0% ~ (p=0.103 n=20) Int64N1e9-8 4.139n ± 0% 4.140n ± 0% ~ (p=0.761 n=20) Int64N2e9-8 4.140n ± 0% 4.140n ± 0% ~ (p=0.636 n=20) Int64N1e18-8 5.266n ± 0% 5.326n ± 1% +1.14% (p=0.001 n=20) Int64N2e18-8 6.052n ± 0% 6.167n ± 0% +1.90% (p=0.000 n=20) Int64N4e18-8 8.826n ± 0% 9.051n ± 0% +2.55% (p=0.000 n=20) Int32N1000-8 4.127n ± 0% 4.132n ± 0% +0.12% (p=0.000 n=20) Int32N1e8-8 4.126n ± 0% 4.131n ± 0% +0.12% (p=0.000 n=20) Int32N1e9-8 4.127n ± 0% 4.132n ± 0% +0.12% (p=0.000 n=20) Int32N2e9-8 4.132n ± 0% 4.131n ± 0% ~ (p=0.017 n=20) Float32-8 4.109n ± 0% 4.105n ± 0% ~ (p=0.379 n=20) Float64-8 4.107n ± 0% 4.106n ± 0% ~ (p=0.867 n=20) ExpFloat64-8 5.339n ± 0% 5.383n ± 0% +0.82% (p=0.000 n=20) NormFloat64-8 5.735n ± 0% 5.737n ± 1% ~ (p=0.856 n=20) Perm3-8 26.65n ± 0% 26.80n ± 1% +0.58% (p=0.000 n=20) Perm30-8 194.8n ± 1% 197.0n ± 0% +1.18% (p=0.000 n=20) Perm30ViaShuffle-8 156.6n ± 0% 157.6n ± 1% +0.61% (p=0.000 n=20) ShuffleOverhead-8 124.9n ± 0% 125.5n ± 0% +0.52% (p=0.000 n=20) Concurrent-8 2.434n ± 3% 5.066n ± 0% +108.09% (p=0.000 n=20) goos: linux goarch: 386 pkg: math/rand/v2 cpu: AMD Ryzen 9 7950X 16-Core Processor │ bbb48afeb7.386 │ 5cf807d1ea.386 │ │ sec/op │ sec/op vs base │ ChaCha8-32 11.295n ± 1% 4.748n ± 2% -57.96% (p=0.000 n=20) PCG_DXSM-32 7.693n ± 1% 7.738n ± 2% ~ (p=0.542 n=20) SourceUint64-32 7.658n ± 2% 7.622n ± 2% ~ (p=0.344 n=20) GlobalInt64-32 3.473n ± 2% 7.526n ± 2% +116.73% (p=0.000 n=20) GlobalInt64Parallel-32 0.3198n ± 0% 0.5444n ± 0% +70.22% (p=0.000 n=20) GlobalUint64-32 3.612n ± 0% 7.575n ± 1% +109.69% (p=0.000 n=20) GlobalUint64Parallel-32 0.3168n ± 0% 0.5403n ± 0% +70.51% (p=0.000 n=20) Int64-32 7.673n ± 2% 7.789n ± 1% ~ (p=0.122 n=20) Uint64-32 7.773n ± 1% 7.827n ± 2% ~ (p=0.920 n=20) GlobalIntN1000-32 6.268n ± 1% 9.581n ± 1% +52.87% (p=0.000 n=20) IntN1000-32 10.33n ± 2% 10.45n ± 1% ~ (p=0.233 n=20) Int64N1000-32 10.98n ± 2% 11.01n ± 1% ~ (p=0.401 n=20) Int64N1e8-32 11.19n ± 2% 10.97n ± 1% ~ (p=0.033 n=20) Int64N1e9-32 11.06n ± 1% 11.08n ± 1% ~ (p=0.498 n=20) Int64N2e9-32 11.10n ± 1% 11.01n ± 2% ~ (p=0.995 n=20) Int64N1e18-32 15.23n ± 2% 15.04n ± 1% ~ (p=0.973 n=20) Int64N2e18-32 15.89n ± 1% 15.85n ± 1% ~ (p=0.409 n=20) Int64N4e18-32 18.96n ± 2% 19.34n ± 2% ~ (p=0.048 n=20) Int32N1000-32 10.46n ± 2% 10.44n ± 2% ~ (p=0.480 n=20) Int32N1e8-32 10.46n ± 2% 10.49n ± 2% ~ (p=0.951 n=20) Int32N1e9-32 10.28n ± 2% 10.26n ± 1% ~ (p=0.431 n=20) Int32N2e9-32 10.50n ± 2% 10.44n ± 2% ~ (p=0.249 n=20) Float32-32 13.80n ± 2% 13.80n ± 2% ~ (p=0.751 n=20) Float64-32 23.55n ± 2% 23.87n ± 0% ~ (p=0.408 n=20) ExpFloat64-32 15.36n ± 1% 15.29n ± 2% ~ (p=0.316 n=20) NormFloat64-32 13.57n ± 1% 13.79n ± 1% +1.66% (p=0.005 n=20) Perm3-32 45.70n ± 2% 46.99n ± 2% +2.81% (p=0.001 n=20) Perm30-32 399.0n ± 1% 403.8n ± 1% +1.19% (p=0.006 n=20) Perm30ViaShuffle-32 349.0n ± 1% 350.4n ± 1% ~ (p=0.909 n=20) ShuffleOverhead-32 322.3n ± 1% 323.8n ± 1% ~ (p=0.410 n=20) Concurrent-32 3.331n ± 1% 7.312n ± 1% +119.50% (p=0.000 n=20) For #61716. Change-Id: Ibdddeed85c34d9ae397289dc899e04d4845f9ed2 Reviewed-on: https://go-review.googlesource.com/c/go/+/516860 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Filippo Valsorda <filippo@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05math/rand/v2: add ChaCha8Russ Cox
This is a replay of CL 516859, after its rollback in CL 543895, with big-endian systems fixed and the tests disabled on RISC-V since the compiler is broken there (#64285). ChaCha8 provides a cryptographically strong generator alongside PCG, so that people who want stronger randomness have access to that. On systems with 128-bit vector math assembly (amd64 and arm64), ChaCha8 runs at about the same speed as PCG (25% slower on amd64, 2% faster on arm64). Fixes #64284. Change-Id: I6290bb8ace28e1aff9a61f805dbe380ccdf25b94 Reviewed-on: https://go-review.googlesource.com/c/go/+/546020 Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05internal/trace/v2: handle thread ID reuse correctlyMichael Anthony Knyszek
To begin with, CL 545515 made the trace parser tolerant of GoCreateSyscall having a P, but that was wrong. Because dropm trashes the M's syscalltick, that case should never be possible. So the first thing this change does is it rewrites the test that CL introduced to expect a failure instead of a success. What I'd misinterpreted as a case that should be allowed was actually the same as the other issues causing #64060, which is that the parser doesn't correctly implement what happens to Ps when a thread calls back into Go on non-pthread platforms, and what happens when a thread dies on pthread platorms (or more succinctly, what the runtime does when it calls dropm). Specifically, the GoDestroySyscall event implies that if any P is still running on that M when it's called, that the P stops running. This is what is intended by the runtime trashing the M's syscalltick; when it calls back into Go, the tracer models that thread as obtaining a new P from scratch. Handling this incorrectly manifests in one of two ways. On pthread platforms, GoDestroySyscall is only emitted when a C thread that previously called into Go is destroyed. However, that thread ID can be reused. Because we have no thread events, whether it's the same thread or not is totally ambiguous to the tracer. Therefore, the tracer may observe a thread that previously died try to start running with a new P under the same identity. The association to the old P is still intact because the ID is the same, and the tracer gets confused -- it appears as if two Ps are running on the same M! On non-pthread platforms, GoDestroySyscall is emitted on every return to C from Go code. In this case, the same thread with the same identity is naturally going to keep calling back into Go. But again, since the runtime trashes syscalltick in dropm, it's always going to acquire a P from the tracer's perspective. But if this is a different P than before, just like the pthread case, the parser is going to get confused, since it looks like two Ps are running on the same M! The case that CL 545515 actually handled was actually the non-pthread case, specifically where the same P is reacquired by an M calling back into Go. In this case, if we tolerate having a P, then what we'll observe is the M stealing its own P from itself, then running with it. Now that we know what the problem is, how do we fix it? This change addresses the problem by emitting an extra event when encountering a GoDestroySyscall with an active P in its context. In this case, it emits an additional ProcSteal event to steal from itself, indicating that the P stopped running. This removes any association between that M and that P, resolving any ambiguity in the tracer. There's one other minor detail that needs to be worked out, and that's what happens to any *real* ProcSteal event that stole the P we're now emitting an extra ProcSteal event for. Since, this event is going to look for an M that may have moved on already and the P at this point is already idle. Luckily, we have *exactly* the right fix for this. The handler for GoDestroySyscall now moves any active P it has to the ProcSyscallAbandoned state, indicating that we've lost information about the P and that it should be treated as already idle. Conceptually this all makes sense: this is a P in _Psyscall that has been abandoned by the M it was previously bound to. It's unfortunate how complicated this has all ended up being, but we can take a closer look at that in the future. Fixes #64060. Change-Id: Ie9e6eb9cf738607617446e3487392643656069a2 Reviewed-on: https://go-review.googlesource.com/c/go/+/546096 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-12-01go/types, types2: add testcase for alias issueRobert Griesemer
Using the new type Alias node, this issue is now fixed. Add a test case. Fixes #50729. Change-Id: I22a4cf31b83de497e052989ca2054227e65e9937 Reviewed-on: https://go-review.googlesource.com/c/go/+/546455 Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com> Auto-Submit: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
2023-11-29types2, go/types: fix type checking of ~[]E passing to unsafe builtinsCuong Manh Le
Fixes #64406 Change-Id: I58002ad722a229fe6db0be08d745fbad86048c6d Reviewed-on: https://go-review.googlesource.com/c/go/+/545395 Reviewed-by: Robert Griesemer <gri@google.com> Run-TryBot: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Auto-Submit: Robert Griesemer <gri@google.com>
2023-11-28internal/trace/v2: tolerate having a P in GoCreateSyscallMichael Anthony Knyszek
On non-pthread platforms, it's totally possible for the same M to GoCreateSyscall/GoDestroySyscall on the same thread multiple times. That same thread may hold onto its P through all those calls. For #64060. Change-Id: Ib968bfd439ecd5bc24fc98d78c06145b0d4b7802 Reviewed-on: https://go-review.googlesource.com/c/go/+/545515 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-22internal/abi, runtime, reflect, cmd: merge maxZero const into internal/abiqiulaidongfeng
For #59670 Change-Id: If38a74ad067a3ea3ff551c0c25c8ef41abec114b GitHub-Last-Rev: fb1f2f3c9f320017627bc3342b061e1e7f6f7fad GitHub-Pull-Request: golang/go#64268 Reviewed-on: https://go-review.googlesource.com/c/go/+/543655 Run-TryBot: qiulaidongfeng <2645477756@qq.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-11-22cmd/asm: fix the KMCTR instruction encoding and argument passingSrinivas Pokala
KMCTR encoding arguments incorrect way, which leading illegal instruction wherver we call KMCTR instruction.IBM z13 machine test's TestAESGCM test using gcmASM implementation, which uses KMCTR instruction to encrypt using AES in counter mode and the KIMD instruction for GHASH. z14+ machines onwards uses gcmKMA implementation for the same. Fixes #63387 Change-Id: I86aeb99573c3f636a71908c99e06a9530655aa5d Reviewed-on: https://go-review.googlesource.com/c/go/+/535675 Reviewed-by: Vishwanatha HD <vishwanatha.hd@ibm.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-11-22internal/bytealg: optimize Count with PCALIGN in riscv64Meng Zhuo
For #63678 Benchmark on Milk-V Mars CM eMMC (Starfive/JH7110 SoC) goos: linux goarch: riscv64 pkg: bytes │ /root/bytes.old.bench │ /root/bytes.pc16.bench │ │ sec/op │ sec/op vs base │ Count/10 223.9n ± 1% 220.8n ± 1% -1.36% (p=0.001 n=10) Count/32 571.6n ± 0% 571.3n ± 0% ~ (p=0.054 n=10) Count/4K 38.56µ ± 0% 38.55µ ± 0% -0.01% (p=0.010 n=10) Count/4M 40.13m ± 0% 39.21m ± 0% -2.28% (p=0.000 n=10) Count/64M 627.5m ± 0% 627.4m ± 0% -0.01% (p=0.019 n=10) CountEasy/10 101.3n ± 0% 101.3n ± 0% ~ (p=1.000 n=10) ¹ CountEasy/32 139.3n ± 0% 139.3n ± 0% ~ (p=1.000 n=10) ¹ CountEasy/4K 5.565µ ± 0% 5.564µ ± 0% -0.02% (p=0.001 n=10) CountEasy/4M 5.619m ± 0% 5.619m ± 0% ~ (p=0.190 n=10) CountEasy/64M 89.94m ± 0% 89.93m ± 0% ~ (p=0.436 n=10) CountSingle/10 53.80n ± 0% 46.06n ± 0% -14.39% (p=0.000 n=10) CountSingle/32 104.30n ± 0% 79.64n ± 0% -23.64% (p=0.000 n=10) CountSingle/4K 10.413µ ± 0% 7.247µ ± 0% -30.40% (p=0.000 n=10) CountSingle/4M 11.603m ± 0% 8.388m ± 0% -27.71% (p=0.000 n=10) CountSingle/64M 230.9m ± 0% 172.3m ± 0% -25.40% (p=0.000 n=10) CountHard1 9.981m ± 0% 9.981m ± 0% ~ (p=0.810 n=10) CountHard2 9.981m ± 0% 9.981m ± 0% ~ (p=0.315 n=10) CountHard3 9.981m ± 0% 9.981m ± 0% ~ (p=0.159 n=10) geomean 144.6µ 133.5µ -7.70% ¹ all samples are equal │ /root/bytes.old.bench │ /root/bytes.pc16.bench │ │ B/s │ B/s vs base │ Count/10 42.60Mi ± 1% 43.19Mi ± 1% +1.39% (p=0.001 n=10) Count/32 53.38Mi ± 0% 53.42Mi ± 0% +0.06% (p=0.049 n=10) Count/4K 101.3Mi ± 0% 101.3Mi ± 0% ~ (p=0.077 n=10) Count/4M 99.68Mi ± 0% 102.01Mi ± 0% +2.34% (p=0.000 n=10) Count/64M 102.0Mi ± 0% 102.0Mi ± 0% ~ (p=0.076 n=10) CountEasy/10 94.18Mi ± 0% 94.18Mi ± 0% ~ (p=0.054 n=10) CountEasy/32 219.1Mi ± 0% 219.1Mi ± 0% +0.01% (p=0.016 n=10) CountEasy/4K 702.0Mi ± 0% 702.0Mi ± 0% +0.00% (p=0.000 n=10) CountEasy/4M 711.9Mi ± 0% 711.9Mi ± 0% ~ (p=0.133 n=10) CountEasy/64M 711.6Mi ± 0% 711.7Mi ± 0% ~ (p=0.447 n=10) CountSingle/10 177.2Mi ± 0% 207.0Mi ± 0% +16.81% (p=0.000 n=10) CountSingle/32 292.7Mi ± 0% 383.2Mi ± 0% +30.91% (p=0.000 n=10) CountSingle/4K 375.1Mi ± 0% 539.0Mi ± 0% +43.70% (p=0.000 n=10) CountSingle/4M 344.7Mi ± 0% 476.9Mi ± 0% +38.33% (p=0.000 n=10) CountSingle/64M 277.2Mi ± 0% 371.5Mi ± 0% +34.05% (p=0.000 n=10) geomean 199.7Mi 219.8Mi +10.10% Change-Id: I1abf6b220b9802028f8ad5eebc8d3b7cfa3e89ea Reviewed-on: https://go-review.googlesource.com/c/go/+/541756 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> Run-TryBot: M Zhuo <mzh@golangcn.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Wang Yaduo <wangyaduo@linux.alibaba.com> Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
2023-11-21internal/trace/v2: dump text trace on failure only if it fits in the logMichael Anthony Knyszek
Currently we dump text traces to the build log on failure unconditionally, but this may cause the old infrastructure's builds' logs to get truncated. Avoid that by setting a threshold on the maximum size of the text trace we're willing to dump. We don't need this workaround on the new infrastructure -- logs don't get truncated there. Change-Id: I0f50f50bb4b90f87250b673fbe56f48235325610 Reviewed-on: https://go-review.googlesource.com/c/go/+/544216 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21runtime: emit a ProcSteal from entersyscall_gcwaitMichael Anthony Knyszek
Currently entersyscall_gcwait always emits a ProcStop event. Most of the time, this is correct, since the thread that just put the P into _Psyscall is the same one that is putting it into _Pgcstop. However it's possible for another thread to steal the P, start running a goroutine, and then enter another syscall, putting the P back into _Psyscall. In this case ProcStop is incorrect; the P is getting stolen. This leads to broken traces. Fix this by always emitting a ProcSteal event from entersyscall_gcwait. This means that most of the time a thread will be 'stealing' the proc from itself when it enters this function, but that's theoretically fine. A ProcSteal is really just a fancy ProcStop. Well, it would be if the parser correctly handled a self-steal. This is a minor bug that just never came up before, but it's an update order error (the mState is looked up and modified, but then it's modified again at the end of the function to match newCtx). There's really no reason a self-steal shouldn't be allowed, so fix that up and add a test. Change-Id: Iec3d7639d331e3f2d127f92ce50c2c4a7818fcd3 Reviewed-on: https://go-review.googlesource.com/c/go/+/544215 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21internal/syscall/unix: add PidFDSendSignal for LinuxKir Kolyshkin
CL 520266 added pidfd_send_signal linux syscall numbers to the syscall package for the sake of a unit test. As pidfd_send_signal will be used from the os package, let's revert the changes to syscall package, add the pidfd_send_signal syscall numbers and the implementation to internal/syscall/unix, and change the above test to use it. Updates #51246. For #62654. Change-Id: I862174c3c1a64baf1080792bdb3a1c1d1b417bb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/528436 Run-TryBot: Kirill Kolyshkin <kolyshkin@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21runtime: enable the exectracer2 experiment by defaultMichael Anthony Knyszek
Fixes #60773. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race Change-Id: Ice3fd4577d58ce593611144242f2cba99c9d2ecb Reviewed-on: https://go-review.googlesource.com/c/go/+/540778 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-11-21cmd/trace/v2: emit regions in the goroutine-oriented task viewMichael Anthony Knyszek
This change emits regions in the goroutine-oriented task view (the /trace endpoint with the taskid query variable set) in the same way the old cmd/trace does. For #60773. Fixes #63960. Change-Id: If6c3e7072c694c84a7d2d6c34df668f48d3acc2a Reviewed-on: https://go-review.googlesource.com/c/go/+/543995 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21cmd/trace/v2: add thread-oriented mode for v2 tracesMichael Anthony Knyszek
This is a nice-to-have that's now straightforward to do with the new trace format. This change adds a new query variable passed to the /trace endpoint called "view," which indicates the type of view to use. It is orthogonal with task-related views. Unfortunately a goroutine-based view isn't included because it's too likely to cause the browser tab to crash. For #60773. For #63960. Change-Id: Ifbcb8f2d58ffd425819bdb09c586819cb786478d Reviewed-on: https://go-review.googlesource.com/c/go/+/543695 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-11-21cmd/trace/v2: add support for a task-oriented procs-based viewMichael Anthony Knyszek
This change implements support for the trace?focustask=<taskid> endpoint in the trace tool for v2 traces. Note: the one missing feature in v2 vs. v1 is that the "irrelevant" (but still rendered) events are not grayed out. This basically includes events that overlapped with events that overlapped with other events that were in the task time period, but aren't themselves directly associated. This is probably fine -- the UI already puts a very obvious focus on the period of time the selected task was running. For #60773. For #63960. Change-Id: I5c78a220ae816e331b74cb67c01c5cd98be40dd4 Reviewed-on: https://go-review.googlesource.com/c/go/+/543596 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21cmd/trace/v2: add support for goroutine filteringMichael Anthony Knyszek
This change adds support for the trace?goid=<goid> endpoint to the trace tool for v2 traces. In effect, this change actually implements a per-goroutine view. I tried to add a link to the main page to enable a "view by goroutines" view without filtering, but the web trace viewer broke the browser tab when there were a few hundred goroutines. The risk of a browser hang probably isn't worth the cases where this is nice, especially since filtering by goroutine already works. Unfortunate, but c'est l'vie. Might be worth revisiting if we change out the web viewer in the future. For #60773. For #63960. Change-Id: I8e29f4ab8346af6708fd8824505c30f2c43db796 Reviewed-on: https://go-review.googlesource.com/c/go/+/543595 TryBot-Bypass: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-11-21cmd/trace/v2: add support for task and region endpointsMichael Anthony Knyszek
This change fills out the last of cmd/trace's subpages for v2 traces by adding support for task and region endpoints. For #60773. For #63960. Change-Id: Ifc4c660514b3904788785a1b20e3abc3bb9e55f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/542077 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21cmd/trace: factor out durationHistogramMichael Anthony Knyszek
This code will be useful for the new tracer, and there's no need to duplicate it. This change copies it to internal/trace/traceviewer, adds some comments, and renames it to TimeHistogram. While we're here, let's get rid of the unused String method which has a comment talking about how awful the rendering is. Also, let's get rid of uses of niceDuration. We'd have to bring it with us in the move and I don't think it's worth it. The difference between the default time.Duration rendering and the niceDuration rendering is usually a few extra digits of precision. Yes, it's noisier, but AFAICT it's not substantially worse. It doesn't seem worth the new API, even if it's just internal. We can also always bring it back later. For #60773. For #63960. Change-Id: I795f58f579f1d503c540c3a40bed12e52bce38e2 Reviewed-on: https://go-review.googlesource.com/c/go/+/542001 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21internal/trace: add task analysis for v2 tracesMichael Anthony Knyszek
For v1 traces, cmd/trace contains code for analyzing tasks separately from the goroutine analysis code present in internal/trace. As I started to look into porting that code to v2 traces, I noticed that it wouldn't be too hard to just generalize the existing v2 goroutine summary code to generate exactly the same information. This change does exactly that. For #60773. For #63960. Change-Id: I0cdd9bf9ba11fb292a9ffc37dbf18c2a6a2483b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/542076 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21internal/trace/v2: redefine NoTask and add BackgroundTaskMichael Anthony Knyszek
The v2 trace parser currently handles task inheritance and region task association incorrectly. It assumes that a TaskID of 0 means that there is no task. However, this is only true for task events. A TaskID of 0 means that a region gets assigned to the "background task." The parser currently has no concept of a "background task." Fix this by defining the background task as task ID 0 and redefining NoTask to ^uint64(0). This aligns the TaskID values more closely with other IDs in the parser and also enables disambiguating these two cases. For #60773. For #63960. Change-Id: I09c8217b33b87c8f8f8ea3b0203ed83fd3b61e11 Reviewed-on: https://go-review.googlesource.com/c/go/+/543019 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
2023-11-21cmd/trace/v2: add support for pprof endpointsMichael Anthony Knyszek
This change adds support for the pprof endpoints to cmd/trace/v2. In the process, I realized we need to pass the goroutine summaries to more places, and previous CLs had already done the goroutine analysis during cmd/trace startup. This change thus refactors the goroutine analysis API once again to operate in a streaming manner, and to run at the same time as the initial trace parsing. Now we can include it in the parsedTrace type and pass that around as the de-facto global trace context. Note: for simplicity, this change redefines "syscall" profiles to capture *all* syscalls, not just syscalls that block. IIUC, this choice was partly the result of a limitation in the previous trace format that syscalls don't all have complete durations and many short syscalls are treated as instant. To this end, this change modifies the text on the main trace webpage to reflect this change. For #60773. For #63960. Change-Id: I601d9250ab0849a0bfaef233fd9b1e81aca9a22a Reviewed-on: https://go-review.googlesource.com/c/go/+/541999 Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21cmd/trace: refactor pprof HTTP SVG serving into traceviewerMichael Anthony Knyszek
For #60773. For #63960. Change-Id: Id97380f19267ec765b25a703ea3e2f284396ad75 Reviewed-on: https://go-review.googlesource.com/c/go/+/541998 Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>