| Age | Commit message (Collapse) | Author |
|---|
|
We shouldn't be installing these tools because we will remove them in
distpack. Installing the tools will also prevent us from testing what
happens when the tools are missing.
The changes below this on the stack, CL 677775 (cmd/doc: build cmd/doc
directly into the go command) and CL 677636 (cmd/go/internal/cfg: fix
GOROOT setting when forcing host config) are needed for this change to
pass tests. The doc change is being done so we preserve the properties
in the tests that doc can be invoked without doing a build. It's not
strictly necessary (we could just remove the tests) but it's nice to
have. The GOROOT setting is a significant bug in switching the
configuration to host mode: the value of GOROOT wasn't being reset,
which caused issues for go commands built with trimpath, because
runtime.GOROOT wouldn't have the correct goroot value.
For #71867
Change-Id: I4181711ba117066b7d62d7d013ad4b186871cfb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/677558
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
|
|
For #71867
Change-Id: Ic4c6304b9a6b35c45bf35342523930924c68545a
Reviewed-on: https://go-review.googlesource.com/c/go/+/677635
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
|
This was an oversight: the pack tool isn't actually used in builds.
For #71867
Change-Id: Ib1f1cce0b574cf1d2c1002b2f2ab9ef9d750d0fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/677557
Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
|
Previously, distpack filtered out tools from the packaged distribution
using a list of tools to remove. Instead follow mpratt's suggestion on
CL 666755 and instead filter out tools that are not on a list of tools
to keep. This will make it easier to tell which tools are actually in
the distribution.
For #71867
Change-Id: I8336465703ac820028c3381a0a743c457997e78a
Reviewed-on: https://go-review.googlesource.com/c/go/+/673696
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
|
|
The "doc", "fix", and "covdata" tools invoked by the go command are not
needed for builds. Instead of invoking them directly using the installed
binary in the tool directory, use "go tool" to run them, building them
if needed. We can then stop distributing those tools in the
distribution.
covdata is used in tests and can form part of a cached test result, but
test results don't have the same requirements as build outputs to be
completely determined by the action id. We already don't include a
toolid for the covdata tool in the action id for a test run. The more
principled way to do things would be to load the covdata package,
create the actions to build it, and then depend on the output of
that action from the the test action and use that as the covdata tool.
For now, it's probably not worth the effort, but, in the future, if we
wanted to build a tool like cgo as needed, it would be best to build it
in the same action graph. That would introduce a whole bunch of complexity
because we'd need to build the tool in the host configuration, and all
the configuration parameters are global.
For #71867
Change-Id: Id9bbbb5c169296f66c072949f9da552424ecfa2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/673119
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
|
|
This change removes some tools that are not used for builds, or
otherwise invoked by the go command (other than through "go tool"
itself) from the packaged distributions produced by distpack. When these
tools are missing, "go tool" will build and run them as needed.
Also update a case where we print a buildid commandline to specify
invoking buildid using "go tool" rather than the binary at it's install
location, because it may not exist there in packaged distributions
anymore.
The tools in this CL are the lowest hanging fruit. There are a few more
tools that aren't used by builds, but we'd have to get the go command to
run them using "go tool" rather than finding them in the tool install
directory.
For #71867
Change-Id: I217683bd549962a1add87405bf3fb1225e2333c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/666755
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
|
|
Fixes #68024
Change-Id: I730c3ecfb14472f3eea3895c2a4d5e1d4ac146f5
GitHub-Last-Rev: 4242f360103ba1fd1f4085b8257ccce0f07a68e5
GitHub-Pull-Request: golang/go#68840
Reviewed-on: https://go-review.googlesource.com/c/go/+/604696
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
|
Cleanup and friction reduction
For #65355.
Change-Id: Ia14c9dc584a529a35b97801dd3e95b9acc99a511
Reviewed-on: https://go-review.googlesource.com/c/go/+/600436
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
Move the code that opens and increments counters out of the
cmd/internal/telemetry package into cmd/internal/telemetry/counter. The
telemetry package has dependencies on the upload code, which we do not
want to pull into the rest of the go toolchain.
For #68109
Change-Id: I463c106819b169177a783de4a7d93377e81f4e3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/593976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
|
|
Call the new telemetry.MaybeChild function at the start of the go
command so that the child process logic can be run immediately without
running toolchain selection if this is the child process.
The Start function in the telemetry shim package has been renamed to
OpenCounters to make it clear that that's its only function.
The StartWithUpload function in the telemetry shim package has been
renamed to MaybeParent because that's its actual effective behavior in
cmd/go, the only place it's called: it won't run as the child because
MaybeChild has already been called and would have run as the child if
the program was the telemetry child, and it won't open counters because
telemetry.Start has been called. Checks are added that those functions
are always called before so that the function name and comment are
accurate.
It might make sense to add a true telemetry.MaybeParent function that
doesn't try to start the child or open counters to make things a little
simpler.
Change-Id: Ie81e2418af85cef18ec41f75db66365f6597b8b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/592535
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This change modifies the commands in cmd to open counter files,
increment invocations counters and to increment counters for the names
of the flags that were passed in.
cmd/pprof and cmd/vet are both wrappers around tools defined in other
modules which do their own flag processing so we can't directly
increment flag counters right after flags are parsed. For those two
commands we wait to increment counters until after the programs have
returned.
cmd/dist is built with the bootstrap go so it can't depend on telemetry
yet. We can add telemetry support to it once 1.23 is the minimum
bootstrap version.
For #58894
Change-Id: Ic7f6009992465e55c56ad4dc6451bcb1ca51374a
Reviewed-on: https://go-review.googlesource.com/c/go/+/585235
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Various tools expect tar files to contain entries for directories.
I dropped them when writing cmd/distpack because they're not
strictly necessary and omitting them saves space, but it also
turns out to break some things, so add them back.
We will backport this to release-branch.go1.21 so that Go 1.21.1
will include the directory entries. We can't do anything about
Go 1.21.0 retroactively.
% tar tzvf go1.22rsc1.src.tar.gz | sed 10q
drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/
-rw-r--r-- 0 0 0 1337 Aug 10 10:07 go/CONTRIBUTING.md
-rw-r--r-- 0 0 0 1479 Aug 10 10:07 go/LICENSE
-rw-r--r-- 0 0 0 1303 Aug 10 10:07 go/PATENTS
-rw-r--r-- 0 0 0 1455 Aug 10 10:07 go/README.md
-rw-r--r-- 0 0 0 419 Aug 10 10:07 go/SECURITY.md
-rw-r--r-- 0 0 0 42 Aug 10 10:07 go/VERSION
drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/api/
-rw-r--r-- 0 0 0 1142 Aug 10 10:07 go/api/README
-rw-r--r-- 0 0 0 35424 Aug 10 10:07 go/api/except.txt
% tar tzvf go1.22rsc1.darwin-amd64.tar.gz | sed 10q
drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/
-rw-r--r-- 0 0 0 1337 Aug 10 10:07 go/CONTRIBUTING.md
-rw-r--r-- 0 0 0 1479 Aug 10 10:07 go/LICENSE
-rw-r--r-- 0 0 0 1303 Aug 10 10:07 go/PATENTS
-rw-r--r-- 0 0 0 1455 Aug 10 10:07 go/README.md
-rw-r--r-- 0 0 0 419 Aug 10 10:07 go/SECURITY.md
-rw-r--r-- 0 0 0 42 Aug 10 10:07 go/VERSION
drwxr-xr-x 0 0 0 0 Aug 10 10:07 go/api/
-rw-r--r-- 0 0 0 1142 Aug 10 10:07 go/api/README
-rw-r--r-- 0 0 0 35424 Aug 10 10:07 go/api/except.txt
%
Fixes #61862.
Change-Id: Iecd9ba893015295e88715b031b79a104236b9ced
Reviewed-on: https://go-review.googlesource.com/c/go/+/518335
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
|
|
macOS may write a .DS_Store file to any directory at any time in
the general case, unfortunately. Expand test rules to catch them
no matter where they may appear.
For #24904.
Change-Id: I98a2bb5d0ae2ab3a191ff87b33d6f8048ca39aa8
Reviewed-on: https://go-review.googlesource.com/c/go/+/514898
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
|
|
The code was sorting files in the archives entirely by path string,
but that's not what fs.WalkDir would do. In a directory with
subdirectory foo/bar and file foo/bar.go, foo/bar gets visited
first, so foo/bar/baz appears before foo/bar.go, even though
"foo/bar/baz" > "foo/bar.go".
This CL replaces the string comparison with a path-aware
comparison that places foo/bar/baz before foo/bar.go,
so that if the tar file is extracted and then repacked using
fs.WalkDir, the files will remain in the same order.
This will make it easier to compare the pristine distpack-produced
tgz for darwin against the rebuilt tgz with signed binaries.
Before:
% tar tzvf /tmp/cmddist.tgz | grep -C1 runtime/cgo.go
-rw-r--r-- 0 0 0 11122 Jul 13 15:00 go/src/runtime/callers_test.go
-rw-r--r-- 0 0 0 2416 Jul 13 15:00 go/src/runtime/cgo.go
-rw-r--r-- 0 0 0 2795 Jul 13 15:00 go/src/runtime/cgo/abi_amd64.h
After:
% tar tzvf pkg/distpack/go1.21rsc.src.tar.gz | grep -C1 runtime/cgo.go
-rw-r--r-- 0 0 0 1848 Dec 31 1969 go/src/runtime/cgo/signal_ios_arm64.s
-rw-r--r-- 0 0 0 2416 Dec 31 1969 go/src/runtime/cgo.go
-rw-r--r-- 0 0 0 2479 Dec 31 1969 go/src/runtime/cgo_mmap.go
For #24904.
For #61513.
Change-Id: Ib7374bc0d6324377f81c561bef57fd87b2111b98
Reviewed-on: https://go-review.googlesource.com/c/go/+/511977
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
|
|
Also switch internal/platform to commit the generated code instead of
regenerating it in cmd/dist. Nothing in the generated code depends on
the target configuration, and committing the source file makes it
more amenable to searching and indexing (particularly on
https://cs.opensource.google/go/go).
For #60939.
Change-Id: I9133dfd5129b3c4d7457267589dfac5e7ecbef65
Reviewed-on: https://go-review.googlesource.com/c/go/+/505175
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
|
|
Modules cannot contain go.mod files except at the root
(and we don't keep one at the root). Rename the other go.mod
files to _go.mod.
dl2mod, which we used to convert all the old releases,
did this renaming, but it was missed when porting that
code to distpack.
For #57001.
Fixes #60847.
Change-Id: I4d646b96b5be15df3b79193e254ddc9b11cc8734
Reviewed-on: https://go-review.googlesource.com/c/go/+/503979
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
|
|
The go command recognizes when a program named go_$GOOS_$GOARCH_exec
is in PATH. There are two such programs living in GOROOT/misc/wasm.
Like GOROOT/bin/{go,gofmt} and GOROOT/pkg/tool/**, these programs
need to have the executable bit set to do their job, so set it.
Comparing a distpack produced before and after this change shows that
the pack.go file is modified, the two go_{js,wasip1}_wasm_exec programs
have the new file mode, and there are no other changes, as expected.
The mode change is relevant to the binary and source distributions only.
No change to the module zip since it doesn't include GOROOT/misc at all,
so no effect on previously created toolchain modules whose checksums
are already recorded in the Go checksum database and cannot be changed.
(Other than by changing their "v0.0.1" version, but that's expensive.)
Fixes #60843.
Change-Id: I799b6aacff59c0785cb7743cbd17dda5a9ef91be
Reviewed-on: https://go-review.googlesource.com/c/go/+/503975
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
|
|
dist's deptab is a list of changes to the automatically derived set of
package dependencies. It's as old as dist itself, and the first
version of deptab in CL 5620045 was quite complex. From the beginning,
some of the entries in deptab have been for generated files that need
to be added to the dependency set because they can't be discovered if
they don't exist. gentab is also as old as dist itself, and lists the
generated dependency files.
The interaction between deptab and gentab is rather odd. gentab
contains only base file names, not whole paths. To figure out what
files to generate, dist takes a Cartesian product of deptab and gentab
and calls the generator wherever the basename of a path in deptab
matches an entry in gentab. This perhaps made sense at the time
because some of the generated files appeared in more than one package
in deptab.
These days, deptab consists exclusively of generated files because
dist can correctly derive all other dependencies, and all of the
generated files have unique paths. This makes the Cartesian product
approach needlessly complex (and so confusing!), and means that the
only purpose served by deptab is to provide full paths for generated
files.
Furthermore, in the dist clean command, it also needed to expand the
file names in gentab to complete paths, but it did so using a
different list, cleanlist, and the same Cartesian product algorithm.
This CL drops all of this complexity by putting full paths into
gentab, which lets us delete deptab and cleanlist.
Change-Id: Ie3993983734f6da3be453bb4c17a64e22dcf3e8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/501137
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
|
|
The new String methods use the new FormatFileInfo and
FormatDirEntry functions.
Fixes #54451
Change-Id: I414cdfc212ec3c316fb2734756d2117842a23631
Reviewed-on: https://go-review.googlesource.com/c/go/+/491175
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
|
|
cmd/dist started generating that file in CL 483695.
Also rearrange the list of files to remove slightly to explain
better where they come from.
Fixes #59889
Change-Id: I062c858596d801157e0d943e4ba2761c0547ac3a
Reviewed-on: https://go-review.googlesource.com/c/go/+/490655
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
|
|
We want to enable others to reproduce the exact distribution archives
we are serving on go.dev/dl. Today the code for building those
archives lives in golang.org/x/build, which is fundamentally tied to
running on the Go team build infrastructure and not easy for others to
run. This CL adds a new flag -distpack to cmd/dist, usually invoked as
make.bash -distpack, to build the distribution archives using code in
the main repository that anyone can run. Starting in Go 1.21,
the Go team build infrastructure will run this instead of its current
custom code to build those archives.
The current builds are not reproducible even given identical
infrastructure, because the archives are stamped with the current
time. It is helpful to have a timestamp in the archives indicating
when the code is from, but that time needs to be reproducible.
To ensure this, the new -distpack flag extends the VERSION file to
include a time stamp, which it uses as the modification time for all
files in the archive.
The new -distpack flag is implemented by a separate program,
cmd/distpack, instead of being in cmd/dist, so that it can be compiled
by the toolchain being distributed and not the bootstrap toolchain.
Otherwise details like the exact compression algorithms might vary
from one bootstrap toolchain to another and produce non-reproducible
builds. So there is a new 'go tool distpack', but it's omitted from
the distributions themselves, just as 'go tool dist' is.
make.bash already accepts any flags for cmd/dist, including -distpack.
make.bat is less sophisticated and looks for each known flag, so this
CL adds an update to look for -distpack. The CL also changes make.bat
to accept the idiomatic Go -flagname in addition to the non-idiomatic
(for Go) --flagname. Previously it insisted on the --flag form.
I have confirmed that using make.bash -distpack produces the
identical distribution archives for windows/amd64, linux/amd64,
darwin/amd64, and darwin/arm64 whether it is run on
windows/amd64, linux/amd64, or darwin/amd64 hosts.
For #24904.
Change-Id: Ie6d69365ee3d7294d05b4f96ffb9159b41918074
Reviewed-on: https://go-review.googlesource.com/c/go/+/470676
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Carlos Amedee <amedee@google.com>
|
