go, branch go1.21.5

[release-branch.go1.21] go1.21.5

2023-12-05T18:12:56Z

Change-Id: I203463900d2c63bc398c22593d0908c4f33a6990 Reviewed-on: https://go-review.googlesource.com/c/go/+/547416 Reviewed-by: Carlos Amedee LUCI-TryBot-Result: Go LUCI Auto-Submit: Gopher Robot Reviewed-by: Dmitri Shuralyov

[release-branch.go1.21] net/http: limit chunked data overhead

2023-12-05T17:18:23Z

The chunked transfer encoding adds some overhead to the content transferred. When writing one byte per chunk, for example, there are five bytes of overhead per byte of data transferred: "1\r\nX\r\n" to send "X". Chunks may include "chunk extensions", which we skip over and do not use. For example: "1;chunk extension here\r\nX\r\n". A malicious sender can use chunk extensions to add about 4k of overhead per byte of data. (The maximum chunk header line size we will accept.) Track the amount of overhead read in chunked data, and produce an error if it seems excessive. Updates #64433 Fixes #64435 Fixes CVE-2023-39326 Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135 Reviewed-by: Tatiana Bradley Reviewed-by: Roland Shoemaker (cherry picked from commit 3473ae72ee66c60744665a24b2fde143e8964d4f) Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2095408 Run-TryBot: Roland Shoemaker Reviewed-by: Damien Neil Reviewed-on: https://go-review.googlesource.com/c/go/+/547356 Reviewed-by: Dmitri Shuralyov LUCI-TryBot-Result: Go LUCI

[release-branch.go1.21] cmd/compile: fix findIndVar so it does not match disjointed loop headers

2023-11-28T20:12:59Z

Fix #63984 parseIndVar, prove and maybe more are on the assumption that the loop header is a single block. This can be wrong, ensure we don't match theses cases we don't know how to handle. In the future we could update them so that they know how to handle such cases but theses cases seems rare so I don't think the value would be really high. We could also run a loop canonicalization pass first which could handle this. The repro case looks weird because I massaged it so it would crash with the previous compiler. Change-Id: I4aa8afae9e90a17fa1085832250fc1139c97faa6 Reviewed-on: https://go-review.googlesource.com/c/go/+/539977 Reviewed-by: Heschi Kreinick Reviewed-by: Keith Randall Reviewed-by: Keith Randall LUCI-TryBot-Result: Go LUCI (cherry picked from commit 8b4e1259d0e82c8fe38a1456f997a4e9d63573a2) Reviewed-on: https://go-review.googlesource.com/c/go/+/540535 Reviewed-by: Jorropo Reviewed-by: Mauri de Souza Meneguzzo Reviewed-by: Dmitri Shuralyov Reviewed-by: Dmitri Shuralyov Auto-Submit: Dmitri Shuralyov Reviewed-by: Michael Knyszek

[release-branch.go1.21] runtime: call enableMetadataHugePages and its callees on the systemstack

2023-11-28T18:46:34Z

These functions acquire the heap lock. If they're not called on the systemstack, a stack growth could cause a self-deadlock since stack growth may allocate memory from the page heap. This has been a problem for a while. If this is what's plaguing the ppc64 port right now, it's very surprising (and probably just coincidental) that it's showing up now. For #64050. For #64062. For #64067. Fixes #64073. Change-Id: I2b95dc134d17be63b9fe8f7a3370fe5b5438682f Reviewed-on: https://go-review.googlesource.com/c/go/+/541635 LUCI-TryBot-Result: Go LUCI Run-TryBot: Michael Knyszek Auto-Submit: Michael Knyszek TryBot-Result: Gopher Robot Reviewed-by: Michael Pratt Reviewed-by: Paul Murphy (cherry picked from commit 5f08b4479930af266d4a84c1533b320ed75edba7) Reviewed-on: https://go-review.googlesource.com/c/go/+/541955 Reviewed-by: Dmitri Shuralyov Reviewed-by: Dmitri Shuralyov Auto-Submit: Dmitri Shuralyov

[release-branch.go1.21] crypto/rand,runtime: switch RtlGenRandom for ProcessPrng

2023-11-28T17:48:47Z

RtlGenRandom is a semi-undocumented API, also known as SystemFunction036, which we use to generate random data on Windows. It's definition, in cryptbase.dll, is an opaque wrapper for the documented API ProcessPrng. Instead of using RtlGenRandom, switch to using ProcessPrng, since the former is simply a wrapper for the latter, there should be no practical change on the user side, other than a minor change in the DLLs we load. Updates #53192 Fixes #64413 Change-Id: Ie6891bf97b1d47f5368cccbe92f374dba2c2672a Reviewed-on: https://go-review.googlesource.com/c/go/+/536235 LUCI-TryBot-Result: Go LUCI Reviewed-by: Quim Muntal Auto-Submit: Roland Shoemaker Reviewed-by: Dmitri Shuralyov (cherry picked from commit 693def151adff1af707d82d28f55dba81ceb08e1) Reviewed-on: https://go-review.googlesource.com/c/go/+/545355 Auto-Submit: Dmitri Shuralyov

[release-branch.go1.21] path/filepath: consider \\?\c: as a volume on Windows

2023-11-28T16:59:27Z

While fixing several bugs in path handling on Windows, beginning with \\?\. Prior to #540277, VolumeName considered the first path component after the \\?\ prefix to be part of the volume name. After, it considered only the \\? prefix to be the volume name. Restore the previous behavior. For #64028. Fixes #64041. Change-Id: I6523789e61776342800bd607fb3f29d496257e68 Reviewed-on: https://go-review.googlesource.com/c/go/+/541175 LUCI-TryBot-Result: Go LUCI Reviewed-by: Roland Shoemaker (cherry picked from commit eda42f7c60adab26ed1a340414c726c4bf46b1f7) Reviewed-on: https://go-review.googlesource.com/c/go/+/541521 Reviewed-by: Damien Neil Auto-Submit: Dmitri Shuralyov Reviewed-by: Dmitri Shuralyov

[release-branch.go1.21] cmd/go/internal/vcs: error out if the requested repo does not support a secure protocol

2023-11-27T21:12:17Z

Updates #63845. Fixes #63973. Change-Id: If86d6b13d3b55877b35c087112bd76388c9404b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/539321 Reviewed-by: Michael Matloob LUCI-TryBot-Result: Go LUCI Reviewed-by: Roland Shoemaker Auto-Submit: Bryan Mills (cherry picked from commit be26ae18caf7ddffca4073333f80d0d9e76483c3) Reviewed-on: https://go-review.googlesource.com/c/go/+/540257 Auto-Submit: Dmitri Shuralyov Reviewed-by: Dmitri Shuralyov

[release-branch.go1.21] cmd/go: allow 'go mod download' to switch toolchains if called with explicit arguments

2023-11-09T21:37:26Z

Fixes #62055. Updates #62054. Change-Id: I4ea24070f7d9aa4964c2f215836602068058f718 (cherry picked from CL 540779 and CL 537480) Reviewed-on: https://go-review.googlesource.com/c/go/+/539697 Auto-Submit: Heschi Kreinick LUCI-TryBot-Result: Go LUCI Reviewed-by: Michael Matloob

[release-branch.go1.21] os: report IO_REPARSE_TAG_DEDUP files as regular in Stat and Lstat

2023-11-08T18:13:06Z

Prior to CL 460595, Lstat reported most reparse points as regular files. However, reparse points can in general implement unusual behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR), and Windows allows arbitrary user-defined reparse points, so in general we must not assume that an unrecognized reparse tag represents a regular file; in CL 460595, we began marking them as irregular. As it turns out, the Data Deduplication service on Windows Server runs an Optimization job that turns regular files into reparse files with the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less like regular files, in that they have well-defined sizes and support random-access reads and writes, so most programs can treat them as regular files without difficulty. However, they are still reparse files: as a result, on servers with the Data Deduplication service enabled, files could arbitrarily change from “regular” to “irregular” without explicit user intervention. Since dedup files are converted in the background and otherwise behave like regular files, this change adds a special case to report DEDUP reparse points as regular. Fixes #63764. Updates #63429. No test because to my knowledge we don't have any Windows builders that have the deduplication service enabled, nor do we have a way to reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file. (In theory we could add a builder with the service enabled on a specific volume, write a test that encodes knowledge of that volume, and use the GO_BUILDER_NAME environment variable to run that test only on the specially-configured builders. However, I don't currently have the bandwidth to reconfigure the builders in this way, and given the simplicity of the change I think it is unlikely to regress accidentally.) Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1 Cq-Include-Trybots: luci.golang.try:go1.21-windows-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/538218 Reviewed-by: Alex Brainman Reviewed-by: David Chase Auto-Submit: Heschi Kreinick LUCI-TryBot-Result: Go LUCI

[release-branch.go1.21] internal/poll: add SPLICE_F_NONBLOCK flag for splice to avoid inconsistency with O_NONBLOCK

2023-11-08T18:02:42Z

Fixes #63801 Updates #59041 Updates #63795 Details: https://github.com/golang/go/issues/59041#issuecomment-1766610087 Change-Id: Id3fc1df6d86b7c4cc383d09f9465fa8f4cc2a559 Reviewed-on: https://go-review.googlesource.com/c/go/+/536015 Reviewed-by: Bryan Mills Reviewed-by: Ian Lance Taylor LUCI-TryBot-Result: Go LUCI Auto-Submit: Ian Lance Taylor (cherry picked from commit 40cdf69fc9279ab28f84a6e0f965de8382c578fe) Reviewed-on: https://go-review.googlesource.com/c/go/+/538117 Auto-Submit: Heschi Kreinick Reviewed-by: Mauri de Souza Meneguzzo Reviewed-by: Heschi Kreinick