cmd/dist: make toolchain build reproducible

- Build cmd with CGO_ENABLED=0. Doing so removes the C compiler
  toolchain from the reproducibility perimeter and also results in
  cmd/go and cmd/pprof binaries that are statically linked,
  so that they will run on a wider variety of systems.
  In particular the Linux versions will run on Alpine and NixOS
  without needing a simulation of libc.so.6.

  The potential downside of disabling cgo is that cmd/go and cmd/pprof
  use the pure Go network resolver instead of the host resolver on
  Unix systems. This means they will not be able to use non-DNS
  resolver mechanisms that may be specified in /etc/resolv.conf,
  such as mDNS. Neither program seems likely to need non-DNS names
  like those, however.

  macOS and Windows systems still use the host resolver, which they
  access without cgo.

- Build cmd with -trimpath when building a release.
  Doing so removes $GOPATH from the file name prefixes stored in the
  binary, so that the build directory does not leak into the final artifacts.

- When CC and CXX are empty, do not pick values to hard-code into
  the source tree and binaries. Instead, emit code that makes the
  right decision at runtime. In addition to reproducibility, this
  makes cross-compiled toolchains work better. A macOS toolchain
  cross-compiled on Linux will now correctly look for clang,
  instead of looking for gcc because it was built on Linux.

- Convert \ to / in file names stored in .a files.
  These are converted to / in the final binaries, but the hashes of
  the .a files affect the final build ID of the binaries. Without this
  change, builds of a Windows toolchain on Windows and non-Windows
  machines produce identical binaries except for the input hash part
  of the build ID.

- Due to the conversion of \ to / in .a files, convert back when
  reading inline bodies on Windows to preserve output file names
  in error messages.

Combined, these four changes (along with Go 1.20's removal of
installed pkg/**.a files and conversion of macOS net away from cgo)
make the output of make.bash fully reproducible, even when
cross-compiling: a released macOS toolchain built on Linux or Windows
will contain exactly the same bits as a released macOS toolchain
built on macOS.

The word "released" in the previous sentence is important.
For the build IDs in the binaries to work out the same on
both systems, a VERSION file must exist to provide a consistent
compiler build ID (instead of using a content hash of the binary).

For #24904.
Fixes #57007.

Change-Id: I665e1ef4ff207d6ff469452347dca5bfc81050e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/454836
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

1 files changed, 7 insertions, 5 deletions

diff --git a/src/cmd/compile/internal/noder/reader.go b/src/cmd/compile/internal/noder/reader.go
index d03da27a46..bd15729171 100644
--- a/src/cmd/compile/internal/noder/reader.go
+++ b/src/cmd/compile/internal/noder/reader.go
@@ -9,6 +9,7 @@ import (
 	"go/constant"
 	"internal/buildcfg"
 	"internal/pkgbits"
+	"path/filepath"
 	"strings"
 
 	"cmd/compile/internal/base"
@@ -268,13 +269,14 @@ func (pr *pkgReader) posBaseIdx(idx pkgbits.Index) *src.PosBase {
 	// "$GOROOT" to buildcfg.GOROOT is a close-enough approximation to
 	// satisfy this.
 	//
-	// TODO(mdempsky): De-duplicate this logic with similar logic in
-	// cmd/link/internal/ld's expandGoroot. However, this will probably
-	// require being more consistent about when we use native vs UNIX
-	// file paths.
+	// The export data format only ever uses slash paths
+	// (for cross-operating-system reproducible builds),
+	// but error messages need to use native paths (backslash on Windows)
+	// as if they had been specified on the command line.
+	// (The go command always passes native paths to the compiler.)
 	const dollarGOROOT = "$GOROOT"
 	if buildcfg.GOROOT != "" && strings.HasPrefix(filename, dollarGOROOT) {
-		filename = buildcfg.GOROOT + filename[len(dollarGOROOT):]
+		filename = filepath.FromSlash(buildcfg.GOROOT + filename[len(dollarGOROOT):])
 	}
 
 	if r.Bool() {