aboutsummaryrefslogtreecommitdiff
path: root/src/cmd/dist
AgeCommit message (Collapse)Author
2023-01-24all: fix some commentscui fliter
Change-Id: I3e9f05d221990b1ae464545d6d8b2e22c35bca21 Reviewed-on: https://go-review.googlesource.com/c/go/+/463077 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-23internal/abi,runtime: refactor map constants into one placeDavid Chase
Previously TryBot-tested with bucket bits = 4. Also tested locally with bucket bits = 5. This makes it much easier to change the size of map buckets, and hopefully provides pointers to all the code that in some way depends on details of map layout. Change-Id: I9f6669d1eadd02f182d0bc3f959dc5f385fa1683 Reviewed-on: https://go-review.googlesource.com/c/go/+/462115 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Austin Clements <austin@google.com>
2023-01-20cmd/dist: mark linux/sparc64 as a broken port, remove incomplete mapDmitri Shuralyov
The linux/sparc64 port is incomplete—it doesn't work, and it doesn't have a builder. Now that dist supports broken ports, mark it as such. The incomplete map was created to hide ports that aren't functional from dist list output. Now that we have the broken port concept, it seems largely redundant, so remove it for now. For #56679. Updates #28944. Change-Id: I34bd23e913ed6d786a4d0aa8d2852f2b926fe4b6 Reviewed-on: https://go-review.googlesource.com/c/go/+/458516 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-01-20cmd/dist: add map of broken ports and -force flagDmitri Shuralyov
It's empty so far. The next CL adds linux/sparc64. Also add -force flag to the bootstrap.bash script so that it's possible to use it with broken ports. For #56679. Change-Id: I09c733d0df0a68df34fb808eae29be010a6da461 Reviewed-on: https://go-review.googlesource.com/c/go/+/458515 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-20cmd/dist: remove GOTESTONLY environment variableDmitri Shuralyov
This hook was added for the Go build system (x/build) to be able to set the run flag value, but it's no longer used anywhere. Remove it for now. Updates #46054. Change-Id: I64e7d68d2b270303f3bd54f73079600f209e350a Reviewed-on: https://go-review.googlesource.com/c/go/+/455519 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Austin Clements <austin@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-20cmd/dist: remove hardcoded timeout scale for arm and mips{,le,64,64le}Dmitri Shuralyov
Fixes #57117. Change-Id: If36da5b138a5e92e27688719ffc1c4dafd590957 Reviewed-on: https://go-review.googlesource.com/c/go/+/455518 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-17cmd/dist: make toolchain build reproducibleRuss Cox
- Build cmd with CGO_ENABLED=0. Doing so removes the C compiler toolchain from the reproducibility perimeter and also results in cmd/go and cmd/pprof binaries that are statically linked, so that they will run on a wider variety of systems. In particular the Linux versions will run on Alpine and NixOS without needing a simulation of libc.so.6. The potential downside of disabling cgo is that cmd/go and cmd/pprof use the pure Go network resolver instead of the host resolver on Unix systems. This means they will not be able to use non-DNS resolver mechanisms that may be specified in /etc/resolv.conf, such as mDNS. Neither program seems likely to need non-DNS names like those, however. macOS and Windows systems still use the host resolver, which they access without cgo. - Build cmd with -trimpath when building a release. Doing so removes $GOPATH from the file name prefixes stored in the binary, so that the build directory does not leak into the final artifacts. - When CC and CXX are empty, do not pick values to hard-code into the source tree and binaries. Instead, emit code that makes the right decision at runtime. In addition to reproducibility, this makes cross-compiled toolchains work better. A macOS toolchain cross-compiled on Linux will now correctly look for clang, instead of looking for gcc because it was built on Linux. - Convert \ to / in file names stored in .a files. These are converted to / in the final binaries, but the hashes of the .a files affect the final build ID of the binaries. Without this change, builds of a Windows toolchain on Windows and non-Windows machines produce identical binaries except for the input hash part of the build ID. - Due to the conversion of \ to / in .a files, convert back when reading inline bodies on Windows to preserve output file names in error messages. Combined, these four changes (along with Go 1.20's removal of installed pkg/**.a files and conversion of macOS net away from cgo) make the output of make.bash fully reproducible, even when cross-compiling: a released macOS toolchain built on Linux or Windows will contain exactly the same bits as a released macOS toolchain built on macOS. The word "released" in the previous sentence is important. For the build IDs in the binaries to work out the same on both systems, a VERSION file must exist to provide a consistent compiler build ID (instead of using a content hash of the binary). For #24904. Fixes #57007. Change-Id: I665e1ef4ff207d6ff469452347dca5bfc81050e6 Reviewed-on: https://go-review.googlesource.com/c/go/+/454836 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Russ Cox <rsc@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-17time/tzdata: generate zip constant during cmd/distRuss Cox
We have a make.bash-time generation capability, so use it to generate the embedded zip file for time/tzdata. This is one less file to try to review in CLs like CL 455356. For #22487. Fixes #43350. Change-Id: I2fcd0665fa0b1c830baec5fb4cd714483fea25a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/455357 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-12-06run.bash, cmd/dist: document GO_TEST_SHORT and GO_TEST_TIMEOUT_SCALEDmitri Shuralyov
These environment variables affect cmd/dist, and in turn run.bash. They exist primarily for the Go build system, but are still needed sometimes when investigating problems. Document them in one place. Fixes #46054. Change-Id: I5dea2ac79b0d203d2f3c9ec2980382f62bead5cd Reviewed-on: https://go-review.googlesource.com/c/go/+/455517 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2022-12-01cmd/dist: remove pkg/$GOOS_$GOARCH and pkg/obj at exitMichael Matloob
pkg/obj will be empty, so just remove it. pkg/$GOOS_$GOARCH will be empty unless the user has specified GODEBUG=installgoroot=all, so check if it's empty, and if so, delete it. Also remove xreaddirfiles, which is unused. Also remove the copy of pkg/$GOOS_$GOARCH in the cmd/go test TestNewReleaseRebuildsStalePackagesInGOPATH. The directory is empty so copying it has no effect. For #47257 Change-Id: Ief90b882d157bd16078cd5d2b83a915bfc831f9a Reviewed-on: https://go-review.googlesource.com/c/go/+/453496 Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-11-28cmd/api: rewrite as package testRuss Cox
No one ever runs 'go tool api', because the invocation has gotten unwieldy enough that it's not practical. And we don't support it as a standalone tool for other packages - it's not even in the distribution. Making it an ordinary package test lets us invoke it more easily from cmd/dist (as go test cmd/api -check) and avoids the increasingly baroque code in run.go to build a command line. Left in cmd/api even though it's no longer a command because (1) it uses a package from cmd/vendor and (2) it uses internal/testenv. Otherwise it could be misc/api. Fixes #56845. Change-Id: I00a13d9c19b1e259fa0e6bb93d1a4dca25f0e8c1 Reviewed-on: https://go-review.googlesource.com/c/go/+/453258 Auto-Submit: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Russ Cox <rsc@golang.org>
2022-11-22cmd/dist: revise skip sequence added in CL 452763Than McIntosh
Revise the code added in CL 452763 that skips some of the -race tests on older windows builders. The old-style skip was doing a log.Printf, which wound up being interpreted in "-list" mode. Fix is to pass in a special rtPreFunc when registering the test (thanks Austin for the fix suggestion). Updates #56904. Change-Id: Ia6ea31fb7f011b539173f47357ab3bf7389f256d Reviewed-on: https://go-review.googlesource.com/c/go/+/452769 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Than McIntosh <thanm@google.com>
2022-11-22cmd/dist: skip -race w/ external linkage on windows 2008Than McIntosh
Add a skip for the external-linkage part of the race detector test for elderly versions of windows, until the underlying cause for the problem can be determined. Updates #56904. Change-Id: I3e8650ff66f34efefabcd6bc343d57124539901b Reviewed-on: https://go-review.googlesource.com/c/go/+/452763 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Than McIntosh <thanm@google.com> Auto-Submit: Than McIntosh <thanm@google.com>
2022-11-22cmd/dist: skip the staleness check for 'cmd' when testing on aix-ppc64Bryan C. Mills
The gcc toolchain on the aix-ppc64 builder apparently does not achieve reproducible builds for packages that use cgo, which causes the binaries in cmd that use package "net" (cmd/go, cmd/pprof, and cmd/trace) to appear stale whenever the Go build cache is cleared. For now, we work around the staleness by rebuilding std and simply not checking whether cmd is stale. For #56896. Updates #47257. Change-Id: I15f86e72dee53904b881710d5d5d613872361510 Reviewed-on: https://go-review.googlesource.com/c/go/+/452680 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Ayappan Perumal <ayappanec@gmail.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-21cmd/dist: add serial/small build option for GOMAXPROCS=1David Chase
If GOMAXPROCS=1, force serial execution, which is better for debugging build problems and also minimizes footprint, if that happens to matter. This wasn't good when the bootstrap was 1.4 because there default GOMAXPROCS=1, but that is no longer the bootstrap version. Change-Id: I637e25c8acb4758795fceef63921eda359a7be29 Reviewed-on: https://go-review.googlesource.com/c/go/+/452556 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-21cmd/dist: stop trying to access runtime/internal/sys.a in debug modeCherry Mui
Now that we don't install the .a files, the installed runtime/internal/sys.a no longer exists. Stop trying to access it. Change-Id: Ic437e7ad95be6109d0b2fcfb81e840f5cb476be8 Reviewed-on: https://go-review.googlesource.com/c/go/+/451360 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com>
2022-11-21cmd/dist: update to assume Go 1.17.13 as minimal Go bootstrap versionMartin Möhrmann
Replace explicit Go version names where possible with generic reference to Go bootstrap version. Updates #44505 Change-Id: I4a6439576efd40e72acd26fcc1472a1a8b0b06e7 Reviewed-on: https://go-review.googlesource.com/c/go/+/344330 Auto-Submit: Martin Möhrmann <martin@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Martin Möhrmann <moehrmann@google.com> Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-18all: add missing periods in commentscui fliter
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29 Reviewed-on: https://go-review.googlesource.com/c/go/+/449757 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Joedian Reid <joedian@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-17cmd/dist: skip non-race testsAustin Clements
In -race mode, the dist test command only registers the std, race, osusergo, and amd64ios tests before returning early from (*tester).registerTests. Prior to CL 450018, the osusergo and amd64ios tests weren't even affected by -race mode, so it seems their inclusion was unintentional. CL 450018 lifted the logic to run tests in race mode, which means these tests went from running without -race to running with -race. Unfortunately, amd64ios is not compatible with -race, so it is now failing on the darwin-amd64-race builder. Fix this by omitting the osusergo and amd64ios tests from -race mode, since it seems like they were really intended to be included anyway. This should fix the darwin-amd64-race builder. Updates #37486. Change-Id: I554bb60bc729dbb6f1bc926f1ea329768b0d6d81 Reviewed-on: https://go-review.googlesource.com/c/go/+/451437 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Austin Clements <austin@google.com>
2022-11-16cmd/dist: simplify old codeAustin Clements
Now that all uses of "go test" have been converted over to the new abstraction, we can delete the old helpers for building "go test" commands and simplify some code that's only used by the new abstraction now. For #37486. Change-Id: I770cd457e018160d694abcc0b6ac80f7dc2e8425 Reviewed-on: https://go-review.googlesource.com/c/go/+/450020 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Austin Clements <austin@google.com>
2022-11-16cmd/dist: convert host tests to use goTestAustin Clements
This adds support for host tests to goTest and registerTest and modifies all uses of registerHostTest to use goTest and registerTest. This eliminates the last case where go test command lines are constructed by hand. Next we'll clean up all of the infrastructure support for that. I traced all exec calls from cmd/dist on linux/amd64 and this makes only no-op changes (such as re-arranging the order of flags). Preparation for #37486. Change-Id: Icb7ec8efdac72bdb819ae24b2f585375d9d9d5b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/450019 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-11-16cmd/dist: convert most remaining tests to use goTestAustin Clements
This converts most of the remaining manual "go test" command line construction in cmd/dist to use the goTest abstraction and registerTest. At this point, the only remaining place that directly constructs go test command lines is runHostTest. This fixes a bug in the "nolibgcc:os/user" test. It was clearly supposed to pass "-run=^Test[^CS]", but the logic to override the "-run" flag for "nolibgcc:net" caused "nolibgcc:os/user" to pass *both* "-run=^Test[^CS]" and "-run=". This was then rewritten into just "-run=" by flattenCmdline, which caused all os/user tests to run, and not actually skip the expensive tests as intended. (This is a great example of why the new abstraction is much more robust than command line construction.) I traced all exec calls from cmd/dist on linux/amd64 and, other than the fix to nolibgcc:os/user, this makes only no-op changes (such as re-arranging the order of flags). For #37486. Change-Id: Ie8546bacc56640ea39f2804a87795c14a3fe4c7d Reviewed-on: https://go-review.googlesource.com/c/go/+/450018 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-11-16cmd/dist: restructure cgo_testAustin Clements
Currently, dist test has a single test called "cgo_test" that runs a large number of different "go test"s. This commit restructures cgo_test into several individual tests, each of which runs a single "go test" that can be described by a goTest object and registered with registerTest. Since this lets us raise the abstraction level of constructing these tests and these tests are mostly covering the Cartesian product of a small number of orthogonal dimensions, we pull the common logic for constructing these tests into a helper function. For consistency, we now pass -tags=static to the static testtls and nocgo tests, but this tag doesn't affect the build of these tests at all. I traced all exec calls from cmd/dist on linux/amd64 and this is the only non-trivial change. For #37486. Change-Id: I53c1efa1c38d785dc71968f05e8d7d636b553e96 Reviewed-on: https://go-review.googlesource.com/c/go/+/450017 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-16cmd/dist: make registerTest take a goTestAustin Clements
The overall goal is to make registerTest the primary entry point for adding dist tests and to convert nearly all dist tests to be represented by a goTest, registered via registerTest. This will centralize the logic for creating dist tests corresponding to go tool tests. I traced all exec calls from cmd/dist on linux/amd64 and this makes only no-op changes (such as re-arranging the order of flags). For #37486. Change-Id: I4749e6f3666134d3259b54ee6055d76a4235c60c Reviewed-on: https://go-review.googlesource.com/c/go/+/450016 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-16cmd/dist: use goTest for manual go test invocationsAustin Clements
This CL rewrites everywhere in dist that manually constructs an exec.Cmd to run "go test" to use the goTest abstraction. All remaining invocations of "go test" after this CL construct the command line manually, but ultimately use addCmd to execute it. I traced all exec calls from cmd/dist on linux/amd64 and this makes only no-op changes (such as re-arranging the order of flags). For #37486. Change-Id: Idc7497e39bac04def7ddaf2010881c9623e76fd4 Reviewed-on: https://go-review.googlesource.com/c/go/+/450015 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Austin Clements <austin@google.com>
2022-11-16cmd/dist: introduce "go test" abstractionAustin Clements
This introduces an abstraction for constructing and running "go test" commands. Currently, dist test is basically a shell script written in Go syntax: it mostly just invokes lots of subprocesses, almost all of which are "go test" invocations, and it constructs those command lines directly from strings all over the place. This CL raises the level of abstraction of invoking go test. The current level of abstraction is not serving us very well: it's conveniently terse, but the actual logic for constructing a command line is typically so spread out that it's difficult to predict what command will actually run. For example, the `gotest` function constructs the basic command, but many tests want to override at least some of these flags, so flattenCmdLine has logic specific to `go test` for eliminating duplicate flags that `go test` itself would reject. At the same time, the logic for constructing many common flags is conditional, leading to a bevy of helpers for constructing flags like `-short` and `-timeout` and `-run` that are scattered throughout test.go and very easy to forget to call. This CL centralizes and flattens all of this knowledge into a new `goTest` type. This type gives dist a single, unified point where we can change anything about how it invokes "go test". There's currently some "unnecessary" abstraction in the implementation of the goTest type to separate "build" and "run" flags. This will become important later when we convert host tests and to do separate build and run steps. The following CLs will convert dist test to use this type rather than directly constructing "go test" command lines. Finally, we'll strip out the scattered helper logic for building command lines. For #37486. Change-Id: I9f1633fe6c0921696419ce8127ed2ca7b7a4e01b Reviewed-on: https://go-review.googlesource.com/c/go/+/448802 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-16cmd/go: make testterminal18153 a normal testAustin Clements
Currently, cmd/go's testterminal18153 is implemented as a special test that doesn't run as part of cmd/go's regular tests. Because the test requires stdout and stderr to be a terminal, it is currently run directly by "dist test" so it can inherit the terminal of all.bash. This has a few problems. It's yet another special case in dist test. dist test also has to be careful to not apply its own buffering to this test, so it can't run in parallel and it limits dist test's own scheduler. It doesn't run as part of regular "go test", which means it usually only gets coverage from running all.bash. And since we have to skip it if all.bash wasn't run at a terminal, I'm sure it often gets skipped even when running all.bash. Fix all of this by rewriting this test to create its own PTY and re-exec "go test" to check that PTY passes through go test. This makes the test self-contained, so it can be a regular cmd/go test, we can drop it from dist test, and it's not sensitive to the environment of all.bash. Preparation for #37486. Updates #18153. Change-Id: I6493dbb0143348e299718f6e311ac8a63f5d69c5 Reviewed-on: https://go-review.googlesource.com/c/go/+/449503 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-16cmd/dist: eliminate "time" special caseAustin Clements
The registerTest function has a special case for commands that start with "time", but we don't use this case anywhere. Delete this special case and its support code. Preparation for #37486. Change-Id: Ica180417e7aa4e4fc260cb97467942bae972fdb6 Reviewed-on: https://go-review.googlesource.com/c/go/+/448801 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-16cmd/dist: eliminate registerSeqTestAustin Clements
This is unused, and eliminating it lets us simplify the whole registerTest mechanism. Preparation for #37486. Change-Id: Ia6221e48192cd17775a5d662bdb389d67a9265bc Reviewed-on: https://go-review.googlesource.com/c/go/+/448800 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-16cmd/dist: add doc comments to some functionsAustin Clements
I keep having to reconstruct how dist test's command system works. This CL documents the main functions to help with this. Change-Id: Ic1102496d3c7f356e73f3fa2ea76498251f525d4 Reviewed-on: https://go-review.googlesource.com/c/go/+/448799 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-11-16cmd/dist: add curiously missing GOOS/GOARCH combinationsAustin Clements
The cmd/dist cgo_test enumerates a large number of platforms in various special cases. Some combinations are suspiciously absent. This CL completes the combinations. I've confirmed using trybots that the newly-enabled tests pass on android/* (this is not surprising because the gohostos is never "android" anyway), windows/arm64, linux/ppc64 (no cgo), linux/loong64 (except for one test, filed #56623), linux/mips*, netbsd/arm (except for one test, filed #56629), and netbsd/arm64. The windows/arm builder is out to lunch, so I'm assuming that works. Since netbsd/arm and arm64 mostly passed these tests, I've also enabled them on netbsd/386 and netbsd/amd64, where they seem to work fine as well. Preparation for #37486. Change-Id: I04c3348e4f422d74d51e714647ca3db379e6e919 Reviewed-on: https://go-review.googlesource.com/c/go/+/448016 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-10cmd/dist: define GOPPC64_{cpu} for PPC64 targetsPaul E. Murphy
This can be used to provide better instruction selection for assembly implementations without having to implement two variants and dynamic runtime selections when a newer GOPPC64 value is used. Change-Id: I4331037d57b128137280aa7904d08d362391f81e Reviewed-on: https://go-review.googlesource.com/c/go/+/449115 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-09all: add missing copyright headercui fliter
Change-Id: Ia5a090953d324f0f8aa9c1808c88125ad5eb6f98 Reviewed-on: https://go-review.googlesource.com/c/go/+/448955 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>
2022-11-04cmd/dist: simplify cgo_testAustin Clements
Now that misc/cgo/test doesn't invoke any Go builds itself, we don't need to use GOFLAGS to thread build flags down into sub-builds. Preparation for #37486. Change-Id: I9ecdd215da5c21339cbf76765f8be2190d19539c Reviewed-on: https://go-review.googlesource.com/c/go/+/447356 Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Austin Clements <austin@google.com>
2022-11-01cmd/dist: fix a variable scope bug:Michael Matloob
We reused p so we were deleting the same directory twice instead of two different directories. Fix that. For #47257 Change-Id: I315ad87d0a9182e00ae4c11b82986227e2b02e17 Reviewed-on: https://go-review.googlesource.com/c/go/+/447115 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-11-01test/bench/go1: eliminate start-up timeAustin Clements
The go1 benchmark suite does a lot of work at package init time, which makes it take quite a while to run even if you're not running any of the benchmarks, or if you're only running a subset of them. This leads to an awkward workaround in dist test to compile but not run the package, unlike roughly all other packages. It also reduces isolation between benchmarks by affecting the starting heap size of all benchmarks. Fix this by initializing all data required by a benchmark when that benchmark runs, and keeping it local so it gets freed by the GC and doesn't leak between benchmarks. Now, none of the benchmarks depend on global state. Re-initializing the data on each benchmark run does add overhead to an actual benchmark run, as each benchmark function is called several times with different values of b.N. A full run of all benchmarks at the default -benchtime=1s now takes ~10% longer; higher -benchtimes would be less. It would be quite difficult to cache this data between invocations of the same benchmark function without leaking between different benchmarks and affecting GC overheads, as the testing package doesn't provide any mechanism for this. This reduces the time to run the binary with no benchmarks from 1.5 seconds to 10 ms, and also reduces the memory required to do this from 342 MiB to 17 MiB. To make sure data was not leaking between different benchmarks, I ran the benchmarks with -shuffle=on. The variance remained low: mostly under 3%. A few benchmarks had higher variance, but in all cases it was similar to the variance between this change. This CL naturally changes the measured performance of several of the benchmarks because it dramatically changes the heap size and hence GC overheads. However, going forward the benchmarks should be much better isolated. For #37486. Change-Id: I252ebea703a9560706cc1990dc5ad22d1927c7a0 Reviewed-on: https://go-review.googlesource.com/c/go/+/443336 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Austin Clements <austin@google.com>
2022-10-28cmd/go: don't install most GOROOT .a files in pkgMichael Matloob
Packages in GOROOT that don't use cgo will not be installed in GOROOT/pkg, and will instead be cached as usual like other Go packages. - add a internal/buildinternal package to hold the identities of the five packages that use cgo - update dist's test code to do a go build std cmd before checking staleness on builders. Because most of those packages no longer have install locations, and have dependencies that don't either, the packages need to be cached to not be stale. - fix index_test to import packages with the path "." when preparing the "want" values to compare the indexed data to. (the module index matches the behavior of build.ImportDir, which always passes in "." as the path. - In both the index and go/build Importers, don't set PkgObj for GOROOT packages which will no longer have install targets. PkgTargetRoot will still be set to compute target paths, which will still be needed in buildmode=shared. - "downgrade" all install actions that don't have a target to build actions. (The target should already not be set for packages that shouldn't be installed). For #47257 Change-Id: Ia5aee6b3b20b58e028119cf0352a4c4a2f10f6b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/432535 Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-28cmd/compile: Enables PGO in Go and performs profile-guided inliningRaj Barik
For #55022 Change-Id: I51f1ba166d5a66dcaf4b280756be4a6bf9545c5e Reviewed-on: https://go-review.googlesource.com/c/go/+/429863 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com>
2022-10-27cmd/dist: produce intermedate .a files in a temporary locationMichael Matloob
Before this change, the .a files for the intermediate go toolchains were produced in the same location as the install target. This change has them produced in a temporary location instead. This change, combined with go install not producing .a files for most stdlib packages by default results in a build of the distribution only producing .a's for the five packages in std that require cgo. (Before this change, the .a's for the final build were not being produced, but stale ones from the intermediate builds were left behind.) For #47257 Change-Id: I91b826cd1ce9aad9c492fb865e36d34dc8bb188e Reviewed-on: https://go-review.googlesource.com/c/go/+/436135 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2022-10-18go,cmd,internal: update to anticipate missing targets and .a filesMichael Matloob
go/build and cmd/go will stop returing Targets for stdlib .a files, and stop producing the .a files is pkg/GOOS_GOARCH. update tests to anticipate that and to pass in importcfgs instead of expecting the compiler can find .a files in their old locations. Adds code to determine locations of .a files to internal/goroot. Also adds internal/goroot to dist's bootstrap directories and changes internal/goroot to build with a bootstrap version of Go. Change-Id: Ie81e51105bddb3f0e374cbf47e81c23edfb67fa5 Reviewed-on: https://go-review.googlesource.com/c/go/+/442303 Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-17all: use Go 1.17.13 for bootstrapRuss Cox
Previously we used Go 1.17, but we realized thanks to tickling a pre-Go1.17.3 bug that if we are going to change the bootstrap toolchain that we should default to the latest available point release at the time we make the switch, not the initial major release, so as to avoid bugs that were fixed in the point releases. This CL updates the default search locations and the release notes. Users who run make.bash and depend on finding $HOME/sdk/go1.17 may need to run go install golang.org/dl/go1.17.13@latest go1.17.13 download to provide a Go 1.17.13 toolchain to their builds. Change-Id: I3a2511f088cf852470a7216a5a41ae775fb561b6 Reviewed-on: https://go-review.googlesource.com/c/go/+/439419 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-17misc/cgo/fortran: convert to Go testAustin Clements
Currently, the entry-point to this test is a Bash script that smoke tests the FORTRAN compiler and then runs a FORTRAN-containing Go test. This CL rearranges things so a pure Go Go test smoke tests the FORTRAN compiler and then runs a non-test FORTRAN-containing Go binary. While we're here, we fix a discrepancy when the host is GOARCH=amd64, but the target is GOARCH=386. Previously, we would pick the wrong libgfortran path because we didn't account for the cross-compilation, causing the link to fail. Except for some reason this was ignored and the test nevertheless "passed". In the new test we're a little more strict, so this build failure will cause the test to fail, so we add a little logic to account for cross-compilation with the host toolchain. For #37486. Change-Id: Ie6f70066885d6fbb4e1b5a2b1e13b85dee5b359b Reviewed-on: https://go-review.googlesource.com/c/go/+/443069 Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Austin Clements <austin@google.com>
2022-10-17misc/cgo/testsigfwd: move to runtime/testprog/testprogcgoAustin Clements
This migrates testsigfwd, which uses some one-off build infrastructure, to be part of the runtime's testprogcgo. The test is largely unchanged. Because it's part of a larger binary, this CL renames a few things and gates the constructor-time signal handler registration on an environment variable. This CL also replaces an errant fmt.Errorf with fmt.Fprintf. For #37486, since it eliminates a non-go-test from dist. Change-Id: I0efd146ea0a0a3f0b361431349a419af0f0ecc61 Reviewed-on: https://go-review.googlesource.com/c/go/+/443068 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-14misc/cgo/testasan: drop testAustin Clements
The testasan test was added back in 2013 (CL 10126044), many years before Go added ASAN support in 2021 (CL 298611). So, in fact, testasan does not test Go ASAN support at all, as you might expect (misc/cgo/testsanitizers does that). It's intended to test whether the Go memory allocator works in a mixed C/Go binary where the C code is compiled with ASAN. The test doesn't actually use ASAN in any way; it just simulates where ASAN of 2013 put its shadow mappings. This made sense to test at the time because Go was picky about where its heap landed and ASAN happened to put its mappings exactly where Go wanted to put its heap. These days, Go is totally flexible about its heap placement, and I wouldn't be surprised if ASAN also works differently. Given all of this, this test adds almost no value today. Drop it. For #37486, since it eliminates a non-go-test from dist. Change-Id: I0292f8efbdc0e1e39650715604535c445fbaa87f Reviewed-on: https://go-review.googlesource.com/c/go/+/443067 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-14cmd/dist: drop support for pre-compiled test binariesAustin Clements
We haven't used this in a while and it's going to complicate later changes to dist, so drop support. This was primarily for supporting slow QEMU-based builders, but an alternative and simpler way to do that if we need to in the future is to supply a go_exec wrapper to run tests in QEMU, like we do for other emulated platforms. Simplification for #37486. Change-Id: Idc0383f59c61d8546ea3b4d2eede4acdaf30d9b6 Reviewed-on: https://go-review.googlesource.com/c/go/+/431256 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Austin Clements <austin@google.com>
2022-10-14cmd/dist, misc/cgo/testsanitizers: enable msan tests on freebsd/amd64Dmitri Goutnik
Adjust os/arch checks to enable msan tests on freebsd/amd64. R=go1.20 For #53298 Change-Id: I3d0f5259db73d526d006a12de5ba6284528cf819 Reviewed-on: https://go-review.googlesource.com/c/go/+/411276 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-10-10cmd/compile/internal/types2: use internal/types/errors instead of local ↵Robert Griesemer
error codes Change-Id: If9b5c2c5d1b89146250bcd19965797baab315876 Reviewed-on: https://go-review.googlesource.com/c/go/+/439564 Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-05cmd/compile: add late lower pass for last rules to runeric fang
Usually optimization rules have corresponding priorities, some need to be run first, some run next, and some run last, which produces the best code. But currently our optimization rules have no priority, this CL adds a late lower pass that runs those rules that need to be run at last, such as split unreasonable constant folding. This pass can be seen as the second round of the lower pass. For example: func foo(a, b uint64) uint64 { d := a+0x1234568 d1 := b+0x1234568 return d&d1 } The code generated by the master branch: 0x0004 00004 ADD $19088744, R0, R2 // movz+movk+add 0x0010 00016 ADD $19088744, R1, R1 // movz+movk+add 0x001c 00028 AND R1, R2, R0 This is because the current constant folding optimization rules do not take into account the range of constants, causing the constant to be loaded repeatedly. This CL splits these unreasonable constants folding in the late lower pass. With this CL the generated code: 0x0004 00004 MOVD $19088744, R2 // movz+movk 0x000c 00012 ADD R0, R2, R3 0x0010 00016 ADD R1, R2, R1 0x0014 00020 AND R1, R3, R0 This CL also adds constant folding optimization for ADDS instruction. In addition, in order not to introduce the codegen regression, an optimization rule is added to change the addition of a negative number into a subtraction of a positive number. go1 benchmarks: name old time/op new time/op delta BinaryTree17-8 1.22s ± 1% 1.24s ± 0% +1.56% (p=0.008 n=5+5) Fannkuch11-8 1.54s ± 0% 1.53s ± 0% -0.69% (p=0.016 n=4+5) FmtFprintfEmpty-8 14.1ns ± 0% 14.1ns ± 0% ~ (p=0.079 n=4+5) FmtFprintfString-8 26.0ns ± 0% 26.1ns ± 0% +0.23% (p=0.008 n=5+5) FmtFprintfInt-8 32.3ns ± 0% 32.9ns ± 1% +1.72% (p=0.008 n=5+5) FmtFprintfIntInt-8 54.5ns ± 0% 55.5ns ± 0% +1.83% (p=0.008 n=5+5) FmtFprintfPrefixedInt-8 61.5ns ± 0% 62.0ns ± 0% +0.93% (p=0.008 n=5+5) FmtFprintfFloat-8 72.0ns ± 0% 73.6ns ± 0% +2.24% (p=0.008 n=5+5) FmtManyArgs-8 221ns ± 0% 224ns ± 0% +1.22% (p=0.008 n=5+5) GobDecode-8 1.91ms ± 0% 1.93ms ± 0% +0.98% (p=0.008 n=5+5) GobEncode-8 1.40ms ± 1% 1.39ms ± 0% -0.79% (p=0.032 n=5+5) Gzip-8 115ms ± 0% 117ms ± 1% +1.17% (p=0.008 n=5+5) Gunzip-8 19.4ms ± 1% 19.3ms ± 0% -0.71% (p=0.016 n=5+4) HTTPClientServer-8 27.0µs ± 0% 27.3µs ± 0% +0.80% (p=0.008 n=5+5) JSONEncode-8 3.36ms ± 1% 3.33ms ± 0% ~ (p=0.056 n=5+5) JSONDecode-8 17.5ms ± 2% 17.8ms ± 0% +1.71% (p=0.016 n=5+4) Mandelbrot200-8 2.29ms ± 0% 2.29ms ± 0% ~ (p=0.151 n=5+5) GoParse-8 1.35ms ± 1% 1.36ms ± 1% ~ (p=0.056 n=5+5) RegexpMatchEasy0_32-8 24.5ns ± 0% 24.5ns ± 0% ~ (p=0.444 n=4+5) RegexpMatchEasy0_1K-8 131ns ±11% 118ns ± 6% ~ (p=0.056 n=5+5) RegexpMatchEasy1_32-8 22.9ns ± 0% 22.9ns ± 0% ~ (p=0.905 n=4+5) RegexpMatchEasy1_1K-8 126ns ± 0% 127ns ± 0% ~ (p=0.063 n=4+5) RegexpMatchMedium_32-8 486ns ± 5% 483ns ± 0% ~ (p=0.381 n=5+4) RegexpMatchMedium_1K-8 15.4µs ± 1% 15.5µs ± 0% ~ (p=0.151 n=5+5) RegexpMatchHard_32-8 687ns ± 0% 686ns ± 0% ~ (p=0.103 n=5+5) RegexpMatchHard_1K-8 20.7µs ± 0% 20.7µs ± 1% ~ (p=0.151 n=5+5) Revcomp-8 175ms ± 2% 176ms ± 3% ~ (p=1.000 n=5+5) Template-8 20.4ms ± 6% 20.1ms ± 2% ~ (p=0.151 n=5+5) TimeParse-8 112ns ± 0% 113ns ± 0% +0.97% (p=0.016 n=5+4) TimeFormat-8 156ns ± 0% 145ns ± 0% -7.14% (p=0.029 n=4+4) Change-Id: I3ced26e89041f873ac989586514ccc5ee09f13da Reviewed-on: https://go-review.googlesource.com/c/go/+/425134 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Eric Fang <eric.fang@arm.com>
2022-10-04cmd/internal/sys: migrate support.go functions to new internal pkgThan McIntosh
Separate out the functions from cmd/internal/sys/support.go and migrate them to a new package internal/platform, so that functions such as "RaceDetectorSupported" can be called from tests in std as well as in cmd. This isn't a complete move of everything in cmd/internal/sys; there are still many functions left. The original version of this CL (patch set 1) called the new package "internal/sys", but for packages that needed both "internal/sys" and "cmd/internal/sys" the import of the former had to be done with a different name, which was confusing and also required a hack in cmd/dist. Updates #56006. Change-Id: I866d62e75adbf3a640a06e2c7386a6e9e2a18d91 Reviewed-on: https://go-review.googlesource.com/c/go/+/438475 Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com>
2022-10-01cmd/go: remove the -i build flagDaniel Martí
The flag is now removed from `go build` and `go test`. It has been deprecated since Go 1.16, printing a warning message. The idea was to fully delete it in Go 1.17, but that didn't happen. First, delete the BuildI variable and its flag declarations, as well as all the bits of docs that mentioned the flag. Second, delete or simplify the code paths that used BuildI. Third, adapt the tests to the removed flag. Some of them are removed, like test_relative_import_dash_i.txt and TestGoTestDashIDashOWritesBinary, as they centered around the flag. The rest are modified to not cover or use the flag. Finally, change cmd/dist to no longer use `go install -i`. The purpose of the flag was that, when bootstrapping the toolchain, all of its dependencies would also be installed as object files. When removing the use of the -i flags, the checkNotStale call right after building toolchain3 would fail as expected, because runtime/internal/sys is now only up to date in the build cache. Luckily, that's not a problem: we run `go install std cmd` right after, so all standard library packages will be installed as object files. Move the checkNotStale call after that install command. Fixes #41696. Change-Id: I5d8139f18aaee07da886d483e663f3f2f00d5f3a Reviewed-on: https://go-review.googlesource.com/c/go/+/416094 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gopher Robot <gobot@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 06:32:41 +0000