aboutsummaryrefslogtreecommitdiff
path: root/src/cmd/distpack
AgeCommit message (Collapse)Author
2025-10-15cmd/distpack: add fix tool to inventoryAlan Donovan
(To be consistent with change to build.go in CL 700795.) For #71859 Change-Id: I8caad28b7e5a2657f21b60a72899daecf0b2c324 Reviewed-on: https://go-review.googlesource.com/c/go/+/712180 Auto-Submit: Alan Donovan <adonovan@google.com> Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Bypass: Alan Donovan <adonovan@google.com>
2025-09-15all: replace strings.Split with strings.SplitSeq1911860538
In Go 1.25+, strings.SplitSeq offers better performance. Here are the benchmark results comparing strings.Split and strings.SplitSeq in a for-loop, with the benchmark code located in src/strings/iter_test.go: goos: darwin goarch: amd64 pkg: cmd/go/internal/auth cpu: Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ ParseGitAuth/standard-8 281.4n ± 1% 218.0n ± 11% -22.54% (p=0.000 n=10) ParseGitAuth/with_url-8 549.1n ± 1% 480.5n ± 13% -12.48% (p=0.002 n=10) ParseGitAuth/minimal-8 235.4n ± 1% 197.3n ± 7% -16.20% (p=0.000 n=10) ParseGitAuth/complex-8 797.6n ± 2% 805.2n ± 4% ~ (p=0.481 n=10) ParseGitAuth/empty-8 87.48n ± 3% 63.25n ± 6% -27.71% (p=0.000 n=10) ParseGitAuth/malformed-8 228.8n ± 1% 171.2n ± 3% -25.17% (p=0.000 n=10) geomean 288.9n 237.7n -17.72% │ old.txt │ new.txt │ │ B/op │ B/op vs base │ ParseGitAuth/standard-8 192.00 ± 0% 96.00 ± 0% -50.00% (p=0.000 n=10) ParseGitAuth/with_url-8 400.0 ± 0% 288.0 ± 0% -28.00% (p=0.000 n=10) ParseGitAuth/minimal-8 144.00 ± 0% 80.00 ± 0% -44.44% (p=0.000 n=10) ParseGitAuth/complex-8 528.0 ± 0% 400.0 ± 0% -24.24% (p=0.000 n=10) ParseGitAuth/empty-8 32.00 ± 0% 16.00 ± 0% -50.00% (p=0.000 n=10) ParseGitAuth/malformed-8 176.00 ± 0% 80.00 ± 0% -54.55% (p=0.000 n=10) geomean 179.0 102.1 -42.96% │ old.txt │ new.txt │ │ allocs/op │ allocs/op vs base │ ParseGitAuth/standard-8 3.000 ± 0% 2.000 ± 0% -33.33% (p=0.000 n=10) ParseGitAuth/with_url-8 4.000 ± 0% 3.000 ± 0% -25.00% (p=0.000 n=10) ParseGitAuth/minimal-8 3.000 ± 0% 2.000 ± 0% -33.33% (p=0.000 n=10) ParseGitAuth/complex-8 4.000 ± 0% 3.000 ± 0% -25.00% (p=0.000 n=10) ParseGitAuth/empty-8 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10) ParseGitAuth/malformed-8 3.000 ± 0% 2.000 ± 0% -33.33% (p=0.000 n=10) geomean 3.086 2.040 -33.91% Updates #69315. Change-Id: Id0219edea45d9658d527b863162ebe917e7821d9 GitHub-Last-Rev: 392b315e122f2c9ef8703ca2dbce8f82ec198556 GitHub-Pull-Request: golang/go#75259 Reviewed-on: https://go-review.googlesource.com/c/go/+/701015 Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
2025-06-03cmd/dist: don't install tools that won't be shipped in distributionMichael Matloob
We shouldn't be installing these tools because we will remove them in distpack. Installing the tools will also prevent us from testing what happens when the tools are missing. The changes below this on the stack, CL 677775 (cmd/doc: build cmd/doc directly into the go command) and CL 677636 (cmd/go/internal/cfg: fix GOROOT setting when forcing host config) are needed for this change to pass tests. The doc change is being done so we preserve the properties in the tests that doc can be invoked without doing a build. It's not strictly necessary (we could just remove the tests) but it's nice to have. The GOROOT setting is a significant bug in switching the configuration to host mode: the value of GOROOT wasn't being reset, which caused issues for go commands built with trimpath, because runtime.GOROOT wouldn't have the correct goroot value. For #71867 Change-Id: I4181711ba117066b7d62d7d013ad4b186871cfb7 Reviewed-on: https://go-review.googlesource.com/c/go/+/677558 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
2025-05-30cmd/distpack: add test case for pack tool being excludedMichael Matloob
For #71867 Change-Id: Ic4c6304b9a6b35c45bf35342523930924c68545a Reviewed-on: https://go-review.googlesource.com/c/go/+/677635 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2025-05-30cmd/distpack: don't keep the pack toolMichael Matloob
This was an oversight: the pack tool isn't actually used in builds. For #71867 Change-Id: Ib1f1cce0b574cf1d2c1002b2f2ab9ef9d750d0fb Reviewed-on: https://go-review.googlesource.com/c/go/+/677557 Reviewed-by: Michael Matloob <matloob@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2025-05-19cmd/distpack: use positive list of tools to keepMichael Matloob
Previously, distpack filtered out tools from the packaged distribution using a list of tools to remove. Instead follow mpratt's suggestion on CL 666755 and instead filter out tools that are not on a list of tools to keep. This will make it easier to tell which tools are actually in the distribution. For #71867 Change-Id: I8336465703ac820028c3381a0a743c457997e78a Reviewed-on: https://go-review.googlesource.com/c/go/+/673696 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
2025-05-16cmd/distpack: remove more tools from packaged distributionMichael Matloob
The "doc", "fix", and "covdata" tools invoked by the go command are not needed for builds. Instead of invoking them directly using the installed binary in the tool directory, use "go tool" to run them, building them if needed. We can then stop distributing those tools in the distribution. covdata is used in tests and can form part of a cached test result, but test results don't have the same requirements as build outputs to be completely determined by the action id. We already don't include a toolid for the covdata tool in the action id for a test run. The more principled way to do things would be to load the covdata package, create the actions to build it, and then depend on the output of that action from the the test action and use that as the covdata tool. For now, it's probably not worth the effort, but, in the future, if we wanted to build a tool like cgo as needed, it would be best to build it in the same action graph. That would introduce a whole bunch of complexity because we'd need to build the tool in the host configuration, and all the configuration parameters are global. For #71867 Change-Id: Id9bbbb5c169296f66c072949f9da552424ecfa2f Reviewed-on: https://go-review.googlesource.com/c/go/+/673119 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
2025-05-16cmd/distpack: remove some tools from packaged distributionMichael Matloob
This change removes some tools that are not used for builds, or otherwise invoked by the go command (other than through "go tool" itself) from the packaged distributions produced by distpack. When these tools are missing, "go tool" will build and run them as needed. Also update a case where we print a buildid commandline to specify invoking buildid using "go tool" rather than the binary at it's install location, because it may not exist there in packaged distributions anymore. The tools in this CL are the lowest hanging fruit. There are a few more tools that aren't used by builds, but we'd have to get the go command to run them using "go tool" rather than finding them in the tool install directory. For #71867 Change-Id: I217683bd549962a1add87405bf3fb1225e2333c5 Reviewed-on: https://go-review.googlesource.com/c/go/+/666755 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
2024-08-30misc/wasm: move wasm runtime files to lib/wasmZxilly
Fixes #68024 Change-Id: I730c3ecfb14472f3eea3895c2a4d5e1d4ac146f5 GitHub-Last-Rev: 4242f360103ba1fd1f4085b8257ccce0f07a68e5 GitHub-Pull-Request: golang/go#68840 Reviewed-on: https://go-review.googlesource.com/c/go/+/604696 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2024-07-23runtime,internal: move runtime/internal/sys to internal/runtime/sysDavid Chase
Cleanup and friction reduction For #65355. Change-Id: Ia14c9dc584a529a35b97801dd3e95b9acc99a511 Reviewed-on: https://go-review.googlesource.com/c/go/+/600436 Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
2024-06-21cmd/internal: separate counter package from telemetry packageMichael Matloob
Move the code that opens and increments counters out of the cmd/internal/telemetry package into cmd/internal/telemetry/counter. The telemetry package has dependencies on the upload code, which we do not want to pull into the rest of the go toolchain. For #68109 Change-Id: I463c106819b169177a783de4a7d93377e81f4e3e Reviewed-on: https://go-review.googlesource.com/c/go/+/593976 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2024-06-13cmd/go: call telemetry.MaybeChild at start of go commandMichael Matloob
Call the new telemetry.MaybeChild function at the start of the go command so that the child process logic can be run immediately without running toolchain selection if this is the child process. The Start function in the telemetry shim package has been renamed to OpenCounters to make it clear that that's its only function. The StartWithUpload function in the telemetry shim package has been renamed to MaybeParent because that's its actual effective behavior in cmd/go, the only place it's called: it won't run as the child because MaybeChild has already been called and would have run as the child if the program was the telemetry child, and it won't open counters because telemetry.Start has been called. Checks are added that those functions are always called before so that the function name and comment are accurate. It might make sense to add a true telemetry.MaybeParent function that doesn't try to start the child or open counters to make things a little simpler. Change-Id: Ie81e2418af85cef18ec41f75db66365f6597b8b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/592535 Reviewed-by: Robert Findley <rfindley@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-05-14cmd: add telemetry for commands in cmdMichael Matloob
This change modifies the commands in cmd to open counter files, increment invocations counters and to increment counters for the names of the flags that were passed in. cmd/pprof and cmd/vet are both wrappers around tools defined in other modules which do their own flag processing so we can't directly increment flag counters right after flags are parsed. For those two commands we wait to increment counters until after the programs have returned. cmd/dist is built with the bootstrap go so it can't depend on telemetry yet. We can add telemetry support to it once 1.23 is the minimum bootstrap version. For #58894 Change-Id: Ic7f6009992465e55c56ad4dc6451bcb1ca51374a Reviewed-on: https://go-review.googlesource.com/c/go/+/585235 Reviewed-by: Sam Thanawalla <samthanawalla@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-08-11cmd/distpack: include directory entries in tar filesRuss Cox
Various tools expect tar files to contain entries for directories. I dropped them when writing cmd/distpack because they're not strictly necessary and omitting them saves space, but it also turns out to break some things, so add them back. We will backport this to release-branch.go1.21 so that Go 1.21.1 will include the directory entries. We can't do anything about Go 1.21.0 retroactively. % tar tzvf go1.22rsc1.src.tar.gz | sed 10q drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/ -rw-r--r-- 0 0 0 1337 Aug 10 10:07 go/CONTRIBUTING.md -rw-r--r-- 0 0 0 1479 Aug 10 10:07 go/LICENSE -rw-r--r-- 0 0 0 1303 Aug 10 10:07 go/PATENTS -rw-r--r-- 0 0 0 1455 Aug 10 10:07 go/README.md -rw-r--r-- 0 0 0 419 Aug 10 10:07 go/SECURITY.md -rw-r--r-- 0 0 0 42 Aug 10 10:07 go/VERSION drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/api/ -rw-r--r-- 0 0 0 1142 Aug 10 10:07 go/api/README -rw-r--r-- 0 0 0 35424 Aug 10 10:07 go/api/except.txt % tar tzvf go1.22rsc1.darwin-amd64.tar.gz | sed 10q drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/ -rw-r--r-- 0 0 0 1337 Aug 10 10:07 go/CONTRIBUTING.md -rw-r--r-- 0 0 0 1479 Aug 10 10:07 go/LICENSE -rw-r--r-- 0 0 0 1303 Aug 10 10:07 go/PATENTS -rw-r--r-- 0 0 0 1455 Aug 10 10:07 go/README.md -rw-r--r-- 0 0 0 419 Aug 10 10:07 go/SECURITY.md -rw-r--r-- 0 0 0 42 Aug 10 10:07 go/VERSION drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/api/ -rw-r--r-- 0 0 0 1142 Aug 10 10:07 go/api/README -rw-r--r-- 0 0 0 35424 Aug 10 10:07 go/api/except.txt % Fixes #61862. Change-Id: Iecd9ba893015295e88715b031b79a104236b9ced Reviewed-on: https://go-review.googlesource.com/c/go/+/518335 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Russ Cox <rsc@golang.org>
2023-08-01cmd/distpack: test for .DS_Store files in all directoriesDmitri Shuralyov
macOS may write a .DS_Store file to any directory at any time in the general case, unfortunately. Expand test rules to catch them no matter where they may appear. For #24904. Change-Id: I98a2bb5d0ae2ab3a191ff87b33d6f8048ca39aa8 Reviewed-on: https://go-review.googlesource.com/c/go/+/514898 Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-07-26cmd/distpack: sort files in standard walk orderRuss Cox
The code was sorting files in the archives entirely by path string, but that's not what fs.WalkDir would do. In a directory with subdirectory foo/bar and file foo/bar.go, foo/bar gets visited first, so foo/bar/baz appears before foo/bar.go, even though "foo/bar/baz" > "foo/bar.go". This CL replaces the string comparison with a path-aware comparison that places foo/bar/baz before foo/bar.go, so that if the tar file is extracted and then repacked using fs.WalkDir, the files will remain in the same order. This will make it easier to compare the pristine distpack-produced tgz for darwin against the rebuilt tgz with signed binaries. Before: % tar tzvf /tmp/cmddist.tgz | grep -C1 runtime/cgo.go -rw-r--r-- 0 0 0 11122 Jul 13 15:00 go/src/runtime/callers_test.go -rw-r--r-- 0 0 0 2416 Jul 13 15:00 go/src/runtime/cgo.go -rw-r--r-- 0 0 0 2795 Jul 13 15:00 go/src/runtime/cgo/abi_amd64.h After: % tar tzvf pkg/distpack/go1.21rsc.src.tar.gz | grep -C1 runtime/cgo.go -rw-r--r-- 0 0 0 1848 Dec 31 1969 go/src/runtime/cgo/signal_ios_arm64.s -rw-r--r-- 0 0 0 2416 Dec 31 1969 go/src/runtime/cgo.go -rw-r--r-- 0 0 0 2479 Dec 31 1969 go/src/runtime/cgo_mmap.go For #24904. For #61513. Change-Id: Ib7374bc0d6324377f81c561bef57fd87b2111b98 Reviewed-on: https://go-review.googlesource.com/c/go/+/511977 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-22internal/platform,cmd/dist: export the list of supported platformsBryan C. Mills
Also switch internal/platform to commit the generated code instead of regenerating it in cmd/dist. Nothing in the generated code depends on the target configuration, and committing the source file makes it more amenable to searching and indexing (particularly on https://cs.opensource.google/go/go). For #60939. Change-Id: I9133dfd5129b3c4d7457267589dfac5e7ecbef65 Reviewed-on: https://go-review.googlesource.com/c/go/+/505175 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>
2023-06-20cmd/distpack: rename go.mod to _go.mod in toolchain modulesRuss Cox
Modules cannot contain go.mod files except at the root (and we don't keep one at the root). Rename the other go.mod files to _go.mod. dl2mod, which we used to convert all the old releases, did this renaming, but it was missed when porting that code to distpack. For #57001. Fixes #60847. Change-Id: I4d646b96b5be15df3b79193e254ddc9b11cc8734 Reviewed-on: https://go-review.googlesource.com/c/go/+/503979 Auto-Submit: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-06-16cmd/distpack: make go_$GOOS_$GOARCH_exec programs executableDmitri Shuralyov
The go command recognizes when a program named go_$GOOS_$GOARCH_exec is in PATH. There are two such programs living in GOROOT/misc/wasm. Like GOROOT/bin/{go,gofmt} and GOROOT/pkg/tool/**, these programs need to have the executable bit set to do their job, so set it. Comparing a distpack produced before and after this change shows that the pack.go file is modified, the two go_{js,wasip1}_wasm_exec programs have the new file mode, and there are no other changes, as expected. The mode change is relevant to the binary and source distributions only. No change to the module zip since it doesn't include GOROOT/misc at all, so no effect on previously created toolchain modules whose checksums are already recorded in the Go checksum database and cannot be changed. (Other than by changing their "v0.0.1" version, but that's expensive.) Fixes #60843. Change-Id: I799b6aacff59c0785cb7743cbd17dda5a9ef91be Reviewed-on: https://go-review.googlesource.com/c/go/+/503975 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-06-06cmd/dist: remove deptab and cleanlist in favor of gentabAustin Clements
dist's deptab is a list of changes to the automatically derived set of package dependencies. It's as old as dist itself, and the first version of deptab in CL 5620045 was quite complex. From the beginning, some of the entries in deptab have been for generated files that need to be added to the dependency set because they can't be discovered if they don't exist. gentab is also as old as dist itself, and lists the generated dependency files. The interaction between deptab and gentab is rather odd. gentab contains only base file names, not whole paths. To figure out what files to generate, dist takes a Cartesian product of deptab and gentab and calls the generator wherever the basename of a path in deptab matches an entry in gentab. This perhaps made sense at the time because some of the generated files appeared in more than one package in deptab. These days, deptab consists exclusively of generated files because dist can correctly derive all other dependencies, and all of the generated files have unique paths. This makes the Cartesian product approach needlessly complex (and so confusing!), and means that the only purpose served by deptab is to provide full paths for generated files. Furthermore, in the dist clean command, it also needed to expand the file names in gentab to complete paths, but it did so using a different list, cleanlist, and the same Cartesian product algorithm. This CL drops all of this complexity by putting full paths into gentab, which lets us delete deptab and cleanlist. Change-Id: Ie3993983734f6da3be453bb4c17a64e22dcf3e8f Reviewed-on: https://go-review.googlesource.com/c/go/+/501137 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2023-05-04all: add String for fs.{FileInfo,DirEntry} implementationsIan Lance Taylor
The new String methods use the new FormatFileInfo and FormatDirEntry functions. Fixes #54451 Change-Id: I414cdfc212ec3c316fb2734756d2117842a23631 Reviewed-on: https://go-review.googlesource.com/c/go/+/491175 Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-04-28cmd/distpack: remove internal/platform/zosarch.goIan Lance Taylor
cmd/dist started generating that file in CL 483695. Also rearrange the list of files to remove slightly to explain better where they come from. Fixes #59889 Change-Id: I062c858596d801157e0d943e4ba2761c0547ac3a Reviewed-on: https://go-review.googlesource.com/c/go/+/490655 Auto-Submit: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-04-11cmd/dist: add -distpack flag to build distribution archivesRuss Cox
We want to enable others to reproduce the exact distribution archives we are serving on go.dev/dl. Today the code for building those archives lives in golang.org/x/build, which is fundamentally tied to running on the Go team build infrastructure and not easy for others to run. This CL adds a new flag -distpack to cmd/dist, usually invoked as make.bash -distpack, to build the distribution archives using code in the main repository that anyone can run. Starting in Go 1.21, the Go team build infrastructure will run this instead of its current custom code to build those archives. The current builds are not reproducible even given identical infrastructure, because the archives are stamped with the current time. It is helpful to have a timestamp in the archives indicating when the code is from, but that time needs to be reproducible. To ensure this, the new -distpack flag extends the VERSION file to include a time stamp, which it uses as the modification time for all files in the archive. The new -distpack flag is implemented by a separate program, cmd/distpack, instead of being in cmd/dist, so that it can be compiled by the toolchain being distributed and not the bootstrap toolchain. Otherwise details like the exact compression algorithms might vary from one bootstrap toolchain to another and produce non-reproducible builds. So there is a new 'go tool distpack', but it's omitted from the distributions themselves, just as 'go tool dist' is. make.bash already accepts any flags for cmd/dist, including -distpack. make.bat is less sophisticated and looks for each known flag, so this CL adds an update to look for -distpack. The CL also changes make.bat to accept the idiomatic Go -flagname in addition to the non-idiomatic (for Go) --flagname. Previously it insisted on the --flag form. I have confirmed that using make.bash -distpack produces the identical distribution archives for windows/amd64, linux/amd64, darwin/amd64, and darwin/arm64 whether it is run on windows/amd64, linux/amd64, or darwin/amd64 hosts. For #24904. Change-Id: Ie6d69365ee3d7294d05b4f96ffb9159b41918074 Reviewed-on: https://go-review.googlesource.com/c/go/+/470676 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Carlos Amedee <amedee@google.com>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 21:48:00 +0000