go, branch go1.21.4

[release-branch.go1.21] go1.21.4

2023-11-07T17:38:04Z

Change-Id: I3d607ba9f701a76a46f3ab3223fa83e5c517d285 Reviewed-on: https://go-review.googlesource.com/c/go/+/540517 LUCI-TryBot-Result: Go LUCI Auto-Submit: Gopher Robot Reviewed-by: Heschi Kreinick Reviewed-by: Cherry Mui

[release-branch.go1.21] path/filepath: fix various issues in parsing Windows paths

2023-11-07T16:36:54Z

On Windows, A root local device path is a path which begins with \\?\ or \??\. A root local device path accesses the DosDevices object directory, and permits access to any file or device on the system. For example \??\C:\foo is equivalent to common C:\foo. The Clean, IsAbs, IsLocal, and VolumeName functions did not recognize root local device paths beginning with \??\. Clean could convert a rooted path such as \a\..\??\b into the root local device path \??\b. It will now convert this path into .\??\b. IsAbs now correctly reports paths beginning with \??\ as absolute. IsLocal now correctly reports paths beginning with \??\ as non-local. VolumeName now reports the \??\ prefix as a volume name. Join(`\`, `??`, `b`) could convert a seemingly innocent sequence of path elements into the root local device path \??\b. It will now convert this to \.\??\b. In addition, the IsLocal function did not correctly detect reserved names in some cases: - reserved names followed by spaces, such as "COM1 ". - "COM" or "LPT" followed by a superscript 1, 2, or 3. IsLocal now correctly reports these names as non-local. For #63713 Fixes #63715 Fixes CVE-2023-45283 Fixes CVE-2023-45284 Change-Id: I446674a58977adfa54de7267d716ac23ab496c54 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2040691 Reviewed-by: Roland Shoemaker Reviewed-by: Tatiana Bradley Run-TryBot: Damien Neil Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2072596 Reviewed-by: Cherry Mui Reviewed-on: https://go-review.googlesource.com/c/go/+/540276 Auto-Submit: Heschi Kreinick LUCI-TryBot-Result: Go LUCI

[release-branch.go1.21] net/http: pull http2 underflow fix from x/net/http2

2023-10-30T21:11:25Z

After CL 534295 was merged to fix a CVE it introduced an underflow when we try to decrement sc.curHandlers in handlerDone. Pull in a fix from x/net/http2: http2: fix underflow in http2 server push https://go-review.googlesource.com/c/net/+/535595 For #63511 Fixes #63560 Change-Id: I5c678ce7dcc53635f3ad5e4999857cb120dfc1ab GitHub-Last-Rev: 587ffa3cafbb9da6bc82ba8a5b83313f81e5c89b GitHub-Pull-Request: golang/go#63561 Reviewed-on: https://go-review.googlesource.com/c/go/+/535575 Run-TryBot: Mauri de Souza Meneguzzo Reviewed-by: Dmitri Shuralyov Reviewed-by: Dmitri Shuralyov Reviewed-by: David Chase Auto-Submit: Dmitri Shuralyov TryBot-Result: Gopher Robot (cherry picked from commit 0046c1414c4910dfe54abfcdbe18e565dd5a60f6) Reviewed-on: https://go-review.googlesource.com/c/go/+/537996 Reviewed-by: Cherry Mui LUCI-TryBot-Result: Go LUCI

[release-branch.go1.21] runtime/cgo: avoid taking the address of crosscall2 in code

2023-10-30T20:36:26Z

Currently, set_crosscall2 takes the address of crosscall2 without using the GOT, which, on some architectures, results in a PC-relative relocation (e.g. R_AARCH64_ADR_PREL_PG_HI21 on ARM64) to the crosscall2 symbol. But crosscall2 is dynamically exported, so the C linker thinks it may bind to a symbol from a different DSO. Some C linker may not like a PC-relative relocation to such a symbol. Using a local trampoline to avoid taking the address of a dynamically exported symbol. It may be possible to not dynamically export crosscall2. But this CL is safer for backport. Later we may remove the trampolines after unexport crosscall2, if they are not needed. Fixes #63509. Updates #62556. Change-Id: Id28457f65ef121d3f87d8189803abc65ed453283 Reviewed-on: https://go-review.googlesource.com/c/go/+/533535 LUCI-TryBot-Result: Go LUCI Reviewed-by: Ian Lance Taylor (cherry picked from commit 872d7181f4084461441787c70ffd1354314987af) Reviewed-on: https://go-review.googlesource.com/c/go/+/534915 Reviewed-by: David Chase

[release-branch.go1.21] cmd/compile/internal/typecheck: fix closure field naming

2023-10-13T16:12:31Z

When creating the struct type to hold variables captured by a function literal, we currently reuse the captured variable names as fields. However, there's no particular reason to do this: these struct types aren't visible to users, and it adds extra complexity in making sure fields belong to the correct packages. Further, it turns out we were getting that subtly wrong. If two function literals from different packages capture variables with identical names starting with an uppercase letter (and in the same order and with corresponding identical types) end up in the same function (e.g., due to inlining), then we could end up creating closure struct types that are "different" (i.e., not types.Identical) yet end up with equal LinkString representations (which violates LinkString's contract). The easy fix is to just always use simple, exported, generated field names in the struct. This should allow further struct reuse across packages too, and shrink binary sizes slightly. For #62498. Fixes #62545. Change-Id: I9c973f5087bf228649a8f74f7dc1522d84a26b51 Reviewed-on: https://go-review.googlesource.com/c/go/+/527135 Auto-Submit: Matthew Dempsky Reviewed-by: Cuong Manh Le Reviewed-by: Keith Randall LUCI-TryBot-Result: Go LUCI (cherry picked from commit e3ce3126212115808bc248bdc9ad92c0a46436fe) Reviewed-on: https://go-review.googlesource.com/c/go/+/534916 Reviewed-by: Matthew Dempsky Auto-Submit: Dmitri Shuralyov Reviewed-by: Dmitri Shuralyov

[release-branch.go1.21] go/types, types2: don't use generics

2023-10-13T15:10:45Z

This fixes cherry-pick CL 531998. For #63339. Change-Id: I6dac0909ca85d68684ce36025284d25db32e0b15 Reviewed-on: https://go-review.googlesource.com/c/go/+/535135 Auto-Submit: Dmitri Shuralyov LUCI-TryBot-Result: Go LUCI Reviewed-by: Alan Donovan

[release-branch.go1.21] runtime: don't eagerly collapse hugepages

2023-10-12T23:27:00Z

This has caused performance issues in production environments. MADV_COLLAPSE can go into direct reclaim, but we call it with the heap lock held. This means that the process could end up stalled fairly quickly if just one allocating goroutine ends up in the madvise call, at least until the madvise(MADV_COLLAPSE) call returns. A similar issue occurred with madvise(MADV_HUGEPAGE), because that could go into direct reclaim on any page fault for MADV_HUGEPAGE-marked memory. My understanding was that the calls to madvise(MADV_COLLAPSE) were fairly rare, and it's "best-effort" nature prevented it from going into direct reclaim often, but this was wrong. It tends to be fairly heavyweight even when it doesn't end up in direct reclaim, and it's almost certainly not worth it. Disable it until further notice and let the kernel fully dictate hugepage policy. The updated scavenger policy is still more hugepage friendly by delaying scavening until hugepages are no longer densely packed, so we don't lose all that much. The Sweet benchmarks show a minimal difference. A couple less realistic benchmarks seem to slow down a bit; they might just be getting unlucky with what the kernel decides to back with a huge page. Some benchmarks on the other hand improve. Overall, it's a wash. name old time/op new time/op delta BiogoIgor 13.1s ± 1% 13.2s ± 2% ~ (p=0.182 n=9+10) BiogoKrishna 12.0s ± 1% 12.1s ± 1% +1.23% (p=0.002 n=9+10) BleveIndexBatch100 4.51s ± 4% 4.56s ± 3% ~ (p=0.393 n=10+10) EtcdPut 20.2ms ± 4% 19.8ms ± 2% ~ (p=0.079 n=10+9) EtcdSTM 109ms ± 3% 111ms ± 3% +1.63% (p=0.035 n=10+10) GoBuildKubelet 31.2s ± 1% 31.3s ± 1% ~ (p=0.780 n=9+10) GoBuildKubeletLink 7.77s ± 0% 7.81s ± 2% ~ (p=0.237 n=8+10) GoBuildIstioctl 31.8s ± 1% 31.7s ± 0% ~ (p=0.136 n=9+9) GoBuildIstioctlLink 7.88s ± 1% 7.89s ± 1% ~ (p=0.720 n=9+10) GoBuildFrontend 11.7s ± 1% 11.8s ± 1% ~ (p=0.278 n=10+9) GoBuildFrontendLink 1.15s ± 4% 1.15s ± 5% ~ (p=0.387 n=9+9) GopherLuaKNucleotide 19.7s ± 1% 20.6s ± 0% +4.48% (p=0.000 n=10+10) MarkdownRenderXHTML 194ms ± 3% 196ms ± 3% ~ (p=0.356 n=9+10) Tile38QueryLoad 633µs ± 2% 629µs ± 2% ~ (p=0.075 n=10+10) name old average-RSS-bytes new average-RSS-bytes delta BiogoIgor 69.2MB ± 3% 68.4MB ± 1% ~ (p=0.190 n=10+10) BiogoKrishna 4.40GB ± 0% 4.40GB ± 0% ~ (p=0.605 n=9+9) BleveIndexBatch100 195MB ± 3% 195MB ± 2% ~ (p=0.853 n=10+10) EtcdPut 107MB ± 4% 108MB ± 3% ~ (p=0.190 n=10+10) EtcdSTM 91.6MB ± 5% 92.6MB ± 4% ~ (p=0.481 n=10+10) GoBuildKubelet 2.26GB ± 1% 2.28GB ± 1% +1.22% (p=0.000 n=10+10) GoBuildIstioctl 1.53GB ± 0% 1.53GB ± 0% +0.21% (p=0.017 n=9+10) GoBuildFrontend 556MB ± 1% 554MB ± 2% ~ (p=0.497 n=9+10) GopherLuaKNucleotide 39.0MB ± 3% 39.0MB ± 1% ~ (p=1.000 n=10+8) MarkdownRenderXHTML 21.2MB ± 2% 21.4MB ± 3% ~ (p=0.190 n=10+10) Tile38QueryLoad 5.99GB ± 2% 6.02GB ± 0% ~ (p=0.243 n=10+9) name old peak-RSS-bytes new peak-RSS-bytes delta BiogoIgor 90.2MB ± 4% 89.2MB ± 2% ~ (p=0.143 n=10+10) BiogoKrishna 4.49GB ± 0% 4.49GB ± 0% ~ (p=0.190 n=10+10) BleveIndexBatch100 283MB ± 8% 274MB ± 6% ~ (p=0.075 n=10+10) EtcdPut 147MB ± 4% 149MB ± 2% +1.55% (p=0.034 n=10+8) EtcdSTM 117MB ± 5% 117MB ± 4% ~ (p=0.905 n=9+10) GopherLuaKNucleotide 44.9MB ± 1% 44.6MB ± 1% ~ (p=0.083 n=8+8) MarkdownRenderXHTML 22.0MB ± 8% 22.1MB ± 9% ~ (p=0.436 n=10+10) Tile38QueryLoad 6.24GB ± 2% 6.29GB ± 2% ~ (p=0.218 n=10+10) name old peak-VM-bytes new peak-VM-bytes delta BiogoIgor 1.33GB ± 0% 1.33GB ± 0% ~ (p=0.504 n=10+9) BiogoKrishna 5.77GB ± 0% 5.77GB ± 0% ~ (p=1.000 n=10+9) BleveIndexBatch100 3.53GB ± 0% 3.53GB ± 0% ~ (p=0.642 n=10+10) EtcdPut 12.1GB ± 0% 12.1GB ± 0% ~ (p=0.564 n=10+10) EtcdSTM 12.1GB ± 0% 12.1GB ± 0% ~ (p=0.633 n=10+10) GopherLuaKNucleotide 1.26GB ± 0% 1.26GB ± 0% ~ (p=0.297 n=9+10) MarkdownRenderXHTML 1.26GB ± 0% 1.26GB ± 0% ~ (p=0.069 n=10+10) Tile38QueryLoad 7.47GB ± 2% 7.53GB ± 2% ~ (p=0.280 n=10+10) name old p50-latency-ns new p50-latency-ns delta EtcdPut 19.8M ± 5% 19.3M ± 3% -2.74% (p=0.043 n=10+9) EtcdSTM 81.4M ± 4% 83.4M ± 4% +2.46% (p=0.029 n=10+10) Tile38QueryLoad 241k ± 1% 240k ± 1% ~ (p=0.393 n=10+10) name old p90-latency-ns new p90-latency-ns delta EtcdPut 30.4M ± 5% 30.6M ± 5% ~ (p=0.971 n=10+10) EtcdSTM 222M ± 3% 226M ± 4% ~ (p=0.063 n=10+10) Tile38QueryLoad 687k ± 2% 691k ± 1% ~ (p=0.173 n=10+8) name old p99-latency-ns new p99-latency-ns delta EtcdPut 42.3M ±10% 41.4M ± 7% ~ (p=0.353 n=10+10) EtcdSTM 486M ± 7% 487M ± 4% ~ (p=0.579 n=10+10) Tile38QueryLoad 6.43M ± 2% 6.37M ± 3% ~ (p=0.280 n=10+10) name old ops/s new ops/s delta EtcdPut 48.6k ± 3% 49.5k ± 2% ~ (p=0.065 n=10+9) EtcdSTM 9.09k ± 2% 8.95k ± 3% -1.56% (p=0.045 n=10+10) Tile38QueryLoad 28.4k ± 1% 28.6k ± 1% +0.87% (p=0.016 n=9+10) Fixes #63335. For #63334. Related to #61718 and #59960. Change-Id: If84c5a8685825d43c912a71418f2597e44e867e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/531816 Reviewed-by: Michael Pratt LUCI-TryBot-Result: Go LUCI Auto-Submit: Michael Knyszek (cherry picked from commit 595deec3dda8e81d514389efdbb4ee2bc38dcabe) Reviewed-on: https://go-review.googlesource.com/c/go/+/532255 Auto-Submit: Dmitri Shuralyov

[release-branch.go1.21] go/types, types2: don't implicitly modify an argument function's type

2023-10-12T23:16:08Z

See the comment in the (very small) fix for a detailed description. Use the opportunity to introduce a generic clone function which may be useful elsewhere. Fixes #63339. Change-Id: Ic63c6b8bc443011b1a201908254f7d062e1aec71 Reviewed-on: https://go-review.googlesource.com/c/go/+/532157 Run-TryBot: Robert Griesemer Reviewed-by: Robert Findley Reviewed-by: Robert Griesemer Auto-Submit: Robert Griesemer TryBot-Result: Gopher Robot Reviewed-on: https://go-review.googlesource.com/c/go/+/531998 Auto-Submit: Dmitri Shuralyov

[release-branch.go1.21] cmd/link: split text sections for arm 32-bit

2023-10-12T22:36:36Z

This CL is a roll-forward (tweaked slightly) of CL 467715, which turned on text section splitting for GOARCH=arm. The intent is to avoid recurrent problems with external linking where there is a disagreement between the Go linker and the external linker over whether a given branch will reach. In the past our approach has been to tweak the reachability calculations slightly to try to work around potential linker problems, but this hasn't proven to be very robust; section splitting seems to offer a better long term fix. Updates #58425. Fixes #63317. Change-Id: I7372d41abce84097906a3d0805b6b9c486f345d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/531795 Reviewed-by: Cherry Mui Run-TryBot: Than McIntosh TryBot-Result: Gopher Robot (cherry picked from commit 1e690409206ff97330b5a91517d453fc5129bab2) Reviewed-on: https://go-review.googlesource.com/c/go/+/532096 Auto-Submit: Dmitri Shuralyov

[release-branch.go1.21] all: tidy dependency versioning after release

2023-10-10T18:10:52Z

Done with: go get golang.org/x/net@internal-branch.go1.21-vendor go mod tidy go mod vendor go generate net/http # zero diff since CL 534235 already did this For #63417. For #63427. For CVE-2023-39325. Change-Id: Ib258e0d8165760a1082e02c2f4c5ce7d2a3c3c90 Reviewed-on: https://go-review.googlesource.com/c/go/+/534415 LUCI-TryBot-Result: Go LUCI Reviewed-by: Dmitri Shuralyov Auto-Submit: Dmitri Shuralyov Reviewed-by: Michael Pratt