aboutsummaryrefslogtreecommitdiff
path: root/src/net
AgeCommit message (Collapse)Author
2024-01-25all: prealloc slice with possible minimum capabilitiesShulhan
2024-01-24net: ignore Dial errors in TestAcceptTimeoutBryan C. Mills
Also use DialContext instead of just Dial so that we can ensure the call returns before we close the listener. The Dial in this test is intended to complete before the call to Accept, but there is no synchronization to ensure that and sometimes it doesn't happen. That's ok and mostly immaterial to the test, but it does mean we need to ignore Dial errors (which can happen when the listener is closed), and we need to be a little more careful about not dialing a port that may have already been reused by some other test. Fixes #65240. Updates #17948. Change-Id: Ife1b5c3062939441b58f4c096461bf5d7841889b Reviewed-on: https://go-review.googlesource.com/c/go/+/558175 Reviewed-by: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-23net,internal/poll: load ws2_32.dll only when net is importedqmuntal
On Windows, ws2_32.dll is loaded and WSA initialized even if websockets are not used. This CL delays loading of ws2_32.dll and starting WSA until net is initialized. Change-Id: I07ea8241d79709cd4e80d29ba0d792c7444bbfe9 Reviewed-on: https://go-review.googlesource.com/c/go/+/557015 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-23net: move cgo-using darwin test to separate packageRuss Cox
It is a goal / requirement that nothing in the standard library on darwin require cgo, and this test-only file makes net use cgo on darwin. Move it elsewhere. Change-Id: I6c11a8391d3913f73ce0098ba63b29adf5864f93 Reviewed-on: https://go-review.googlesource.com/c/go/+/549255 Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org>
2024-01-23net: unskip and attempt to deflake TestAcceptTimeoutBryan C. Mills
This test has been unconditionally skipped for over five years. It may be that whatever was causing it to flake has been fixed. And if it hasn't been fixed, it isn't providing any value. Let's unskip it for the Go 1.23 development cycle and see what happens. Let's also use a separate listener for each test case, so that a leaked Dial goroutine from one case won't interfere with the other. Fixes #17948 (maybe). Change-Id: I239f22ca5d5a44388b9aa0ed4d81e451c6342617 Reviewed-on: https://go-review.googlesource.com/c/go/+/548940 Commit-Queue: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-23net: report timeouts more aggressively in Accept in the fake implementationBryan C. Mills
This ensures that if the listener has already timed out when Accept is called, Accept always returns an error instead of instantaneously accepting a connection that was already pending. For #17948. Change-Id: Iabef7121590df3dcc2fe428429d7c2bc2bcb6cd5 Reviewed-on: https://go-review.googlesource.com/c/go/+/557438 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com>
2024-01-22net: delete TestListenCloseListenBryan C. Mills
In CL 557177, I attempted to fix a logical race in this test (#65175). However, I introduced a data race in the process (#65209). The race was reported on the windows-amd64-race builder. When I tried to reproduce it on linux/amd64, I added a time.Sleep in the Accept loop. However, that Sleep causes the test to fail outright with EADDRINUSE, which suggests that my earlier guess about the open Conn preventing reuse of the port was, in fact, incorrect. On some platforms we could instead use SO_REUSEPORT and avoid closing the first Listener entirely, but that wouldn't be even remotely in the spirit of the original test. Since I don't see a way to preserve the test in a way that is not inherently flaky / racy, I suggest that we just delete it. It was originally added as a regression test for a bug in the nacl port, which no longer exists anyway. (Some of that code may live on in the wasm port, but it doesn't seem worth maintaining a flaky port-independent test to maintain a regression test for a bug specific to secondary platforms.) Fixes #65209. Updates #65175. Change-Id: I32f9da779d24f2e133571f0971ec460cebe7820a Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race Reviewed-on: https://go-review.googlesource.com/c/go/+/557536 Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2024-01-22net/mail: properly handle special characters in phrase and obs-phraseRoland Shoemaker
Fixes a couple of misalignments with RFC 5322 which introduce significant diffs between (mostly) conformant parsers. This change reverts the changes made in CL50911, which allowed certain special RFC 5322 characters to appear unquoted in the "phrase" syntax. It is unclear why this change was made in the first place, and created a divergence from comformant parsers. In particular this resulted in treating comments in display names incorrectly. Additionally properly handle trailing malformed comments in the group syntax. Fixes #65083 Change-Id: I00dddc044c6ae3381154e43236632604c390f672 Reviewed-on: https://go-review.googlesource.com/c/go/+/555596 Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22all: update vendored dependenciesCherry Mui
The Go 1.23 development tree has opened. This is a time to update all golang.org/x/... module versions that contribute packages to the std and cmd modules in the standard library to latest master versions. Generated with: go install golang.org/x/build/cmd/updatestd@latest go install golang.org/x/tools/cmd/bundle@latest updatestd -goroot=$(pwd) -branch=master For #36905. Change-Id: I46a68f27a54f1e3f9e1aa5af4de6ee0b26388f3f Reviewed-on: https://go-review.googlesource.com/c/go/+/557457 Reviewed-by: Carlos Amedee <carlos@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22net: attempt to deflake TestListenCloseListenBryan C. Mills
Also make it flakier in longtest mode by burning through more ephemeral ports. (Burning through the ports raised the failure rate for me locally enough to reliably reproduce the failure in #65175 with -count=10.) Fixes #65175 (I hope). Change-Id: I5f5b68b6bf6a6aa92e66f0288078817041656a3e Reviewed-on: https://go-review.googlesource.com/c/go/+/557177 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22net/http/pprof: configure WriteDeadlineAlexander Yastrebov
Configure write deadline according to requested profiling duration. Fixes #62358 Change-Id: I2350110ff20a637c7e90bdda57026b0b0d9c87ba GitHub-Last-Rev: b79ae38defc4d9b58b23a9d13ff86fa863f18b00 GitHub-Pull-Request: golang/go#64360 Reviewed-on: https://go-review.googlesource.com/c/go/+/544756 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22net: remove an unused sync.Map in the fake net implementationBryan C. Mills
I added this map in CL 526117, but it is apparently unused. I assume that I removed all uses of it while revising that change. Updates #59718. Updates #50216. Change-Id: I8cdac39f4764d1fcc31566408304c850cf0f9374 Reviewed-on: https://go-review.googlesource.com/c/go/+/557176 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-19net: work around runtime scheduler starvation on js and wasip1Bryan C. Mills
Fixes #65177. Updates #65178. Updates #64321. Change-Id: I698fd3b688c7dfbde692eb7c29cbdafc89e7ca32 Cq-Include-Trybots: luci.golang.try:gotip-js-wasm,gotip-wasip1-wasm_wasmtime,gotip-wasip1-wasm_wazero Reviewed-on: https://go-review.googlesource.com/c/go/+/557037 Auto-Submit: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
2024-01-10net: add available godoc linkcui fliter
Change-Id: Ib7c4baf0247c421954aedabfbb6a6af8a08a8936 Reviewed-on: https://go-review.googlesource.com/c/go/+/540021 Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-09net/http: clarify the precedence order for Request.FormValueJes Cok
Fixes #64575 Change-Id: I0eaec642a9dc8ae3b273a6d41131cc7cb8332947 GitHub-Last-Rev: 17aa5170cbfe42cb86d56f1804266850d33c3eb5 GitHub-Pull-Request: golang/go#64578 Reviewed-on: https://go-review.googlesource.com/c/go/+/547855 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-08net: clarify maxListenerBacklog windows implementationMauri de Souza Meneguzzo
The previous TODO comments were somewhat ambiguous. This aims to provide a clearer understanding of the behavior on Windows. Windows does not offer a way to peek at the current backlog length, this is explicitly stated in the winapi for `listen`. When set to `syscall.SOMAXCONN`, the OS dynamically adjusts the backlog to a maximum reasonable value. It goes as far as the dotnet runtime itself introducing a new version of `listen` that does not accept a backlog parameter to help eliminate the confusion when comparing the behavior with UNIXes. The docs also mention that `SOMAXCONN_HINT(N)` can be used, and that it clips the final computed value between (200, 65535), which suggests windows might use a `uint16` to back this number. Either way it does not matter since windows will adjust this value anyway, so I removed the wrapping TODO as well. See https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-listen Change-Id: I7b2e7cb547467c4bfc572ef0477a58de8c772521 GitHub-Last-Rev: 34e74abffe8792c8709c73db4d7a5fa05f64b1d0 GitHub-Pull-Request: golang/go#63549 Reviewed-on: https://go-review.googlesource.com/c/go/+/535475 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-04net/http: respond with 400 Bad Request for empty hex number of chunk lengthAndy Pan
Fixes #64517 Change-Id: I78b8a6a83301deee05c3ff052a6adcd1f965aef2 Reviewed-on: https://go-review.googlesource.com/c/go/+/553835 Auto-Submit: Damien Neil <dneil@google.com> Commit-Queue: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2024-01-04net/http/cgi: in TestCopyError, check for a Handler.ServeHTTP goroutine ↵Bryan C. Mills
instead of a running PID Previously, the test could fail spuriously if the CGI process's PID happened to be reused in between checks. That sort of reuse is highly unlikely on platforms that cycle through the PID space sequentially (such as Linux), but plausible on platforms that use randomized PIDs (such as OpenBSD). Also unskip the test on Windows, since it no longer relies on being able to send signal 0 to an arbitrary PID. Also change the expected failure mode of the test to a timeout instead of a call to t.Fatalf, so that on failure we get a useful goroutine dump for debugging instead of a non-actionable failure message. Fixes #57369 (maybe). Change-Id: Ib7e3fff556450b48cb5e6ea120fdf4d53547479b Reviewed-on: https://go-review.googlesource.com/c/go/+/554075 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com>
2024-01-03net/http: make Request.Clone create fresh copies for matches and otherValuesJes Cok
This change fixes Request.Clone to correctly work with SetPathValue by creating fresh copies for matches and otherValues so that SetPathValue for cloned requests doesn't pollute the original request. While here, also added a doc for Request.SetPathValue. Fixes #64911 Change-Id: I2831b38e135935dfaea2b939bb9db554c75b65ef GitHub-Last-Rev: 1981db16475a49fe8d4b874a6bceec64d28a1332 GitHub-Pull-Request: golang/go#64913 Reviewed-on: https://go-review.googlesource.com/c/go/+/553375 Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Run-TryBot: Jes Cok <xigua67damn@gmail.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-12-14net/netip: remove Prefix.Compare for Go 1.22Russ Cox
API questions remain, so we decided to back it out for Go 1.22. Code still lives in the repo, just unexported. For #61642. Change-Id: Iccd91b0da48ae72dec9f660476826a220c7ca4be Reviewed-on: https://go-review.googlesource.com/c/go/+/549615 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Anderson <dave@natulte.net> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2023-12-14all: remove newline characters after return statementsDanil Timerbulatov
This commit is aimed at improving the readability and consistency of the code base. Extraneous newline characters were present after some return statements, creating unnecessary separation in the code. Fixes #64610 Change-Id: Ic1b05bf11761c4dff22691c2f1c3755f66d341f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/548316 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-12-11net: remove an arbitrary timeout in TestAcceptTimeoutMustReturnBryan C. Mills
This probably fixes the failure mode seen in https://build.golang.org/log/e73acfd930cbe82302505cac0041d9883e2360c5. If not, allowing the test to deadlock and dump goroutines should produce better debugging information than the existing "didn't return in an expected time" failure message. For #58901. Change-Id: Ie0bb1887a4329d2f6b0e7348a4820af71385494e Reviewed-on: https://go-review.googlesource.com/c/go/+/548881 Auto-Submit: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
2023-12-07internal/profile: fully decode proto even if there are no samplesMichael Pratt
This is a partial revert of CL 483137. CL 483137 started checking errors in postDecode, which is good. Now we can catch more malformed pprof protos. However this made TestEmptyProfile fail, so an early return was added when the profile was "empty" (no samples). Unfortunately, this was problematic. Profiles with no samples can still be valid, but skipping postDecode meant that the resulting Profile was missing values from the string table. In particular, net/http/pprof needs to parse empty profiles in order to pass through the sample and period types to a final output proto. CL 483137 broke this behavior. internal/profile.Parse is only used in two places: in cmd/compile to parse PGO pprof profiles, and in net/http/pprof to parse before/after pprof profiles for delta profiles. In both cases, the input is never literally empty (0 bytes). Even a pprof proto with no samples still contains some header fields, such as sample and period type. Upstream github.com/google/pprof/profile even has an explicit error on 0 byte input, so `go tool pprof` will not support such an input. Thus TestEmptyProfile was misleading; this profile doesn't need to support empty input at all. Resolve this by removing TestEmptyProfile and replacing it with an explicit error on empty input, as upstream github.com/google/pprof/profile has. For non-empty input, always run postDecode to ensure the string table is processed. TestConvertCPUProfileEmpty is reverted back to assert the values from before CL 483137. Note that in this case "Empty" means no samples, not a 0 byte input. Continue to allow empty files for PGO in order to minimize the chance of last minute breakage if some users have empty files. Fixes #64566. Change-Id: I83a1f0200ae225ac6da0009d4b2431fe215b283f Reviewed-on: https://go-review.googlesource.com/c/go/+/547996 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-12-05math/rand, math/rand/v2: use ChaCha8 for global randRuss Cox
Move ChaCha8 code into internal/chacha8rand and use it to implement runtime.rand, which is used for the unseeded global source for both math/rand and math/rand/v2. This also affects the calculation of the start point for iteration over very very large maps (when the 32-bit fastrand is not big enough). The benefit is that misuse of the global random number generators in math/rand and math/rand/v2 in contexts where non-predictable randomness is important for security reasons is no longer a security problem, removing a common mistake among programmers who are unaware of the different kinds of randomness. The cost is an extra 304 bytes per thread stored in the m struct plus 2-3ns more per random uint64 due to the more sophisticated algorithm. Using PCG looks like it would cost about the same, although I haven't benchmarked that. Before this, the math/rand and math/rand/v2 global generator was wyrand (https://github.com/wangyi-fudan/wyhash). For math/rand, using wyrand instead of the Mitchell/Reeds/Thompson ALFG was justifiable, since the latter was not any better. But for math/rand/v2, the global generator really should be at least as good as one of the well-studied, specific algorithms provided directly by the package, and it's not. (Wyrand is still reasonable for scheduling and cache decisions.) Good randomness does have a cost: about twice wyrand. Also rationalize the various runtime rand references. goos: linux goarch: amd64 pkg: math/rand/v2 cpu: AMD Ryzen 9 7950X 16-Core Processor │ bbb48afeb7.amd64 │ 5cf807d1ea.amd64 │ │ sec/op │ sec/op vs base │ ChaCha8-32 1.862n ± 2% 1.861n ± 2% ~ (p=0.825 n=20) PCG_DXSM-32 1.471n ± 1% 1.460n ± 2% ~ (p=0.153 n=20) SourceUint64-32 1.636n ± 2% 1.582n ± 1% -3.30% (p=0.000 n=20) GlobalInt64-32 2.087n ± 1% 3.663n ± 1% +75.54% (p=0.000 n=20) GlobalInt64Parallel-32 0.1042n ± 1% 0.2026n ± 1% +94.48% (p=0.000 n=20) GlobalUint64-32 2.263n ± 2% 3.724n ± 1% +64.57% (p=0.000 n=20) GlobalUint64Parallel-32 0.1019n ± 1% 0.1973n ± 1% +93.67% (p=0.000 n=20) Int64-32 1.771n ± 1% 1.774n ± 1% ~ (p=0.449 n=20) Uint64-32 1.863n ± 2% 1.866n ± 1% ~ (p=0.364 n=20) GlobalIntN1000-32 3.134n ± 3% 4.730n ± 2% +50.95% (p=0.000 n=20) IntN1000-32 2.489n ± 1% 2.489n ± 1% ~ (p=0.683 n=20) Int64N1000-32 2.521n ± 1% 2.516n ± 1% ~ (p=0.394 n=20) Int64N1e8-32 2.479n ± 1% 2.478n ± 2% ~ (p=0.743 n=20) Int64N1e9-32 2.530n ± 2% 2.514n ± 2% ~ (p=0.193 n=20) Int64N2e9-32 2.501n ± 1% 2.494n ± 1% ~ (p=0.616 n=20) Int64N1e18-32 3.227n ± 1% 3.205n ± 1% ~ (p=0.101 n=20) Int64N2e18-32 3.647n ± 1% 3.599n ± 1% ~ (p=0.019 n=20) Int64N4e18-32 5.135n ± 1% 5.069n ± 2% ~ (p=0.034 n=20) Int32N1000-32 2.657n ± 1% 2.637n ± 1% ~ (p=0.180 n=20) Int32N1e8-32 2.636n ± 1% 2.636n ± 1% ~ (p=0.763 n=20) Int32N1e9-32 2.660n ± 2% 2.638n ± 1% ~ (p=0.358 n=20) Int32N2e9-32 2.662n ± 2% 2.618n ± 2% ~ (p=0.064 n=20) Float32-32 2.272n ± 2% 2.239n ± 2% ~ (p=0.194 n=20) Float64-32 2.272n ± 1% 2.286n ± 2% ~ (p=0.763 n=20) ExpFloat64-32 3.762n ± 1% 3.744n ± 1% ~ (p=0.171 n=20) NormFloat64-32 3.706n ± 1% 3.655n ± 2% ~ (p=0.066 n=20) Perm3-32 32.93n ± 3% 34.62n ± 1% +5.13% (p=0.000 n=20) Perm30-32 202.9n ± 1% 204.0n ± 1% ~ (p=0.482 n=20) Perm30ViaShuffle-32 115.0n ± 1% 114.9n ± 1% ~ (p=0.358 n=20) ShuffleOverhead-32 112.8n ± 1% 112.7n ± 1% ~ (p=0.692 n=20) Concurrent-32 2.107n ± 0% 3.725n ± 1% +76.75% (p=0.000 n=20) goos: darwin goarch: arm64 pkg: math/rand/v2 │ bbb48afeb7.arm64 │ 5cf807d1ea.arm64 │ │ sec/op │ sec/op vs base │ ChaCha8-8 2.480n ± 0% 2.429n ± 0% -2.04% (p=0.000 n=20) PCG_DXSM-8 2.531n ± 0% 2.530n ± 0% ~ (p=0.877 n=20) SourceUint64-8 2.534n ± 0% 2.533n ± 0% ~ (p=0.732 n=20) GlobalInt64-8 2.172n ± 1% 4.794n ± 0% +120.67% (p=0.000 n=20) GlobalInt64Parallel-8 0.4320n ± 0% 0.9605n ± 0% +122.32% (p=0.000 n=20) GlobalUint64-8 2.182n ± 0% 4.770n ± 0% +118.58% (p=0.000 n=20) GlobalUint64Parallel-8 0.4307n ± 0% 0.9583n ± 0% +122.51% (p=0.000 n=20) Int64-8 4.107n ± 0% 4.104n ± 0% ~ (p=0.416 n=20) Uint64-8 4.080n ± 0% 4.080n ± 0% ~ (p=0.052 n=20) GlobalIntN1000-8 2.814n ± 2% 5.643n ± 0% +100.50% (p=0.000 n=20) IntN1000-8 4.141n ± 0% 4.139n ± 0% ~ (p=0.140 n=20) Int64N1000-8 4.140n ± 0% 4.140n ± 0% ~ (p=0.313 n=20) Int64N1e8-8 4.140n ± 0% 4.139n ± 0% ~ (p=0.103 n=20) Int64N1e9-8 4.139n ± 0% 4.140n ± 0% ~ (p=0.761 n=20) Int64N2e9-8 4.140n ± 0% 4.140n ± 0% ~ (p=0.636 n=20) Int64N1e18-8 5.266n ± 0% 5.326n ± 1% +1.14% (p=0.001 n=20) Int64N2e18-8 6.052n ± 0% 6.167n ± 0% +1.90% (p=0.000 n=20) Int64N4e18-8 8.826n ± 0% 9.051n ± 0% +2.55% (p=0.000 n=20) Int32N1000-8 4.127n ± 0% 4.132n ± 0% +0.12% (p=0.000 n=20) Int32N1e8-8 4.126n ± 0% 4.131n ± 0% +0.12% (p=0.000 n=20) Int32N1e9-8 4.127n ± 0% 4.132n ± 0% +0.12% (p=0.000 n=20) Int32N2e9-8 4.132n ± 0% 4.131n ± 0% ~ (p=0.017 n=20) Float32-8 4.109n ± 0% 4.105n ± 0% ~ (p=0.379 n=20) Float64-8 4.107n ± 0% 4.106n ± 0% ~ (p=0.867 n=20) ExpFloat64-8 5.339n ± 0% 5.383n ± 0% +0.82% (p=0.000 n=20) NormFloat64-8 5.735n ± 0% 5.737n ± 1% ~ (p=0.856 n=20) Perm3-8 26.65n ± 0% 26.80n ± 1% +0.58% (p=0.000 n=20) Perm30-8 194.8n ± 1% 197.0n ± 0% +1.18% (p=0.000 n=20) Perm30ViaShuffle-8 156.6n ± 0% 157.6n ± 1% +0.61% (p=0.000 n=20) ShuffleOverhead-8 124.9n ± 0% 125.5n ± 0% +0.52% (p=0.000 n=20) Concurrent-8 2.434n ± 3% 5.066n ± 0% +108.09% (p=0.000 n=20) goos: linux goarch: 386 pkg: math/rand/v2 cpu: AMD Ryzen 9 7950X 16-Core Processor │ bbb48afeb7.386 │ 5cf807d1ea.386 │ │ sec/op │ sec/op vs base │ ChaCha8-32 11.295n ± 1% 4.748n ± 2% -57.96% (p=0.000 n=20) PCG_DXSM-32 7.693n ± 1% 7.738n ± 2% ~ (p=0.542 n=20) SourceUint64-32 7.658n ± 2% 7.622n ± 2% ~ (p=0.344 n=20) GlobalInt64-32 3.473n ± 2% 7.526n ± 2% +116.73% (p=0.000 n=20) GlobalInt64Parallel-32 0.3198n ± 0% 0.5444n ± 0% +70.22% (p=0.000 n=20) GlobalUint64-32 3.612n ± 0% 7.575n ± 1% +109.69% (p=0.000 n=20) GlobalUint64Parallel-32 0.3168n ± 0% 0.5403n ± 0% +70.51% (p=0.000 n=20) Int64-32 7.673n ± 2% 7.789n ± 1% ~ (p=0.122 n=20) Uint64-32 7.773n ± 1% 7.827n ± 2% ~ (p=0.920 n=20) GlobalIntN1000-32 6.268n ± 1% 9.581n ± 1% +52.87% (p=0.000 n=20) IntN1000-32 10.33n ± 2% 10.45n ± 1% ~ (p=0.233 n=20) Int64N1000-32 10.98n ± 2% 11.01n ± 1% ~ (p=0.401 n=20) Int64N1e8-32 11.19n ± 2% 10.97n ± 1% ~ (p=0.033 n=20) Int64N1e9-32 11.06n ± 1% 11.08n ± 1% ~ (p=0.498 n=20) Int64N2e9-32 11.10n ± 1% 11.01n ± 2% ~ (p=0.995 n=20) Int64N1e18-32 15.23n ± 2% 15.04n ± 1% ~ (p=0.973 n=20) Int64N2e18-32 15.89n ± 1% 15.85n ± 1% ~ (p=0.409 n=20) Int64N4e18-32 18.96n ± 2% 19.34n ± 2% ~ (p=0.048 n=20) Int32N1000-32 10.46n ± 2% 10.44n ± 2% ~ (p=0.480 n=20) Int32N1e8-32 10.46n ± 2% 10.49n ± 2% ~ (p=0.951 n=20) Int32N1e9-32 10.28n ± 2% 10.26n ± 1% ~ (p=0.431 n=20) Int32N2e9-32 10.50n ± 2% 10.44n ± 2% ~ (p=0.249 n=20) Float32-32 13.80n ± 2% 13.80n ± 2% ~ (p=0.751 n=20) Float64-32 23.55n ± 2% 23.87n ± 0% ~ (p=0.408 n=20) ExpFloat64-32 15.36n ± 1% 15.29n ± 2% ~ (p=0.316 n=20) NormFloat64-32 13.57n ± 1% 13.79n ± 1% +1.66% (p=0.005 n=20) Perm3-32 45.70n ± 2% 46.99n ± 2% +2.81% (p=0.001 n=20) Perm30-32 399.0n ± 1% 403.8n ± 1% +1.19% (p=0.006 n=20) Perm30ViaShuffle-32 349.0n ± 1% 350.4n ± 1% ~ (p=0.909 n=20) ShuffleOverhead-32 322.3n ± 1% 323.8n ± 1% ~ (p=0.410 n=20) Concurrent-32 3.331n ± 1% 7.312n ± 1% +119.50% (p=0.000 n=20) For #61716. Change-Id: Ibdddeed85c34d9ae397289dc899e04d4845f9ed2 Reviewed-on: https://go-review.googlesource.com/c/go/+/516860 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Filippo Valsorda <filippo@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05net/http: limit chunked data overheadDamien Neil
The chunked transfer encoding adds some overhead to the content transferred. When writing one byte per chunk, for example, there are five bytes of overhead per byte of data transferred: "1\r\nX\r\n" to send "X". Chunks may include "chunk extensions", which we skip over and do not use. For example: "1;chunk extension here\r\nX\r\n". A malicious sender can use chunk extensions to add about 4k of overhead per byte of data. (The maximum chunk header line size we will accept.) Track the amount of overhead read in chunked data, and produce an error if it seems excessive. Fixes #64433 Fixes CVE-2023-39326 Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135 Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/547335 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-01net/http: avoid leaking io.Copy goroutines (and hijacked connections) in ↵Bryan C. Mills
TestTransportNoReuseAfterEarlyResponse Fixes #64252 (maybe). Change-Id: Iba2a403a9347be4206f14acb11591dc2eb7f9fb8 Reviewed-on: https://go-review.googlesource.com/c/go/+/546616 Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Bryan Mills <bcmills@google.com>
2023-11-29all: update vendored dependenciesDmitri Shuralyov
The Go 1.22 code freeze has recently started. This is a time to update all golang.org/x/... module versions that contribute packages to the std and cmd modules in the standard library to latest master versions. Generated with: go install golang.org/x/build/cmd/updatestd@latest go install golang.org/x/tools/cmd/bundle@latest updatestd -goroot=$(pwd) -branch=master For #36905. Change-Id: I76525261b9a954ed21a3bd3cb6c4a12e6c031d80 Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest,gotip-linux-386-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/546055 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-11-21net/http: use pointers to array for copyBufPoolJorropo
This is inspired by CL 539915, I'm only submitting now that CL 456435 has been merged. This divide the number of objects kept alive by the heap by two and remove the slice header allocation in New and in the put back. Change-Id: Ibcd5166bac5a37f365a533e09a28f3b79f81ad58 Reviewed-on: https://go-review.googlesource.com/c/go/+/543515 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-11-20src: a/an grammar fixesVille Skyttä
Change-Id: I179b50ae8e73677d4d408b83424afbbfe6aa17a1 GitHub-Last-Rev: 2e2d9c1e45556155d02db4df381b99f2d1bc5c0e GitHub-Pull-Request: golang/go#63478 Reviewed-on: https://go-review.googlesource.com/c/go/+/534015 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-11-17all: add missing copyright headerJes Cok
Change-Id: Ic61fb181923159e80a86a41582e83ec466ab9bc4 GitHub-Last-Rev: 92469845665fa1f864d257c8bc175201a43b4d43 GitHub-Pull-Request: golang/go#64080 Reviewed-on: https://go-review.googlesource.com/c/go/+/541741 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Run-TryBot: Jes Cok <xigua67damn@gmail.com>
2023-11-17net,os: arrange zero-copy of os.File and net.TCPConn to net.UnixConnAndy Pan
Fixes #58808 goos: linux goarch: amd64 pkg: net cpu: DO-Premium-Intel │ old │ new │ │ sec/op │ sec/op vs base │ Splice/tcp-to-unix/1024-4 3.783µ ± 10% 3.201µ ± 7% -15.40% (p=0.001 n=10) Splice/tcp-to-unix/2048-4 3.967µ ± 13% 3.818µ ± 16% ~ (p=0.971 n=10) Splice/tcp-to-unix/4096-4 4.988µ ± 16% 4.590µ ± 11% ~ (p=0.089 n=10) Splice/tcp-to-unix/8192-4 6.981µ ± 13% 5.236µ ± 9% -25.00% (p=0.000 n=10) Splice/tcp-to-unix/16384-4 10.192µ ± 9% 7.350µ ± 7% -27.89% (p=0.000 n=10) Splice/tcp-to-unix/32768-4 19.65µ ± 13% 10.28µ ± 16% -47.69% (p=0.000 n=10) Splice/tcp-to-unix/65536-4 41.89µ ± 18% 15.70µ ± 13% -62.52% (p=0.000 n=10) Splice/tcp-to-unix/131072-4 90.05µ ± 11% 29.55µ ± 10% -67.18% (p=0.000 n=10) Splice/tcp-to-unix/262144-4 170.24µ ± 15% 52.66µ ± 4% -69.06% (p=0.000 n=10) Splice/tcp-to-unix/524288-4 326.4µ ± 13% 109.3µ ± 11% -66.52% (p=0.000 n=10) Splice/tcp-to-unix/1048576-4 651.4µ ± 9% 228.3µ ± 14% -64.95% (p=0.000 n=10) geomean 29.42µ 15.62µ -46.90% │ old │ new │ │ B/s │ B/s vs base │ Splice/tcp-to-unix/1024-4 258.2Mi ± 11% 305.2Mi ± 8% +18.21% (p=0.001 n=10) Splice/tcp-to-unix/2048-4 492.5Mi ± 15% 511.7Mi ± 13% ~ (p=0.971 n=10) Splice/tcp-to-unix/4096-4 783.5Mi ± 14% 851.2Mi ± 12% ~ (p=0.089 n=10) Splice/tcp-to-unix/8192-4 1.093Gi ± 11% 1.458Gi ± 8% +33.36% (p=0.000 n=10) Splice/tcp-to-unix/16384-4 1.497Gi ± 9% 2.076Gi ± 7% +38.67% (p=0.000 n=10) Splice/tcp-to-unix/32768-4 1.553Gi ± 11% 2.969Gi ± 14% +91.17% (p=0.000 n=10) Splice/tcp-to-unix/65536-4 1.458Gi ± 23% 3.888Gi ± 11% +166.69% (p=0.000 n=10) Splice/tcp-to-unix/131072-4 1.356Gi ± 10% 4.131Gi ± 9% +204.72% (p=0.000 n=10) Splice/tcp-to-unix/262144-4 1.434Gi ± 13% 4.637Gi ± 4% +223.32% (p=0.000 n=10) Splice/tcp-to-unix/524288-4 1.497Gi ± 15% 4.468Gi ± 10% +198.47% (p=0.000 n=10) Splice/tcp-to-unix/1048576-4 1.501Gi ± 10% 4.277Gi ± 16% +184.88% (p=0.000 n=10) geomean 1.038Gi 1.954Gi +88.28% │ old │ new │ │ B/op │ B/op vs base │ Splice/tcp-to-unix/1024-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/2048-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/4096-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/8192-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/16384-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/32768-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/65536-4 1.000 ± ? 0.000 ± 0% -100.00% (p=0.001 n=10) Splice/tcp-to-unix/131072-4 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10) Splice/tcp-to-unix/262144-4 4.000 ± 25% 0.000 ± 0% -100.00% (p=0.000 n=10) Splice/tcp-to-unix/524288-4 7.500 ± 33% 0.000 ± 0% -100.00% (p=0.000 n=10) Splice/tcp-to-unix/1048576-4 17.00 ± 12% 0.00 ± 0% -100.00% (p=0.000 n=10) geomean ² ? ² ³ ¹ all samples are equal ² summaries must be >0 to compute geomean ³ ratios must be >0 to compute geomean │ old │ new │ │ allocs/op │ allocs/op vs base │ Splice/tcp-to-unix/1024-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/2048-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/4096-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/8192-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/16384-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/32768-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/65536-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/131072-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/262144-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/524288-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ Splice/tcp-to-unix/1048576-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ geomean ² +0.00% ² ¹ all samples are equal ² summaries must be >0 to compute geomean Change-Id: I829061b009a0929a8ef1a15c183793c0b9104dde Reviewed-on: https://go-review.googlesource.com/c/go/+/472475 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17net: document IsNotFound fieldMateusz Poliwczak
After CL 534937, the IsNotFound field is set to true on nodata and NXDOMAIN conditions consistently across systems and all lookup methods. Change-Id: I419d1c818f17e915cee4959c3a9ac92c2e4bb92c GitHub-Last-Rev: 59c537fc0626bfa959f43f9e9958f7af0e54ca55 GitHub-Pull-Request: golang/go#63654 Reviewed-on: https://go-review.googlesource.com/c/go/+/536775 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17net/netip: optimize AddrPort.String for IPv4 addressesTobias Klauser
Use Addr.appendTo4 like in Addr.String. name old time/op new time/op delta AddrPortString/v4-8 47.5ns ± 0% 27.7ns ± 1% -41.81% (p=0.000 n=8+10) AddrPortString/v6-8 95.2ns ± 0% 94.9ns ± 1% -0.35% (p=0.034 n=10+10) AddrPortString/v6_ellipsis-8 96.8ns ± 0% 96.6ns ± 0% -0.15% (p=0.008 n=9+9) AddrPortString/v6_v4-8 70.9ns ± 0% 70.9ns ± 0% ~ (p=0.425 n=10+10) AddrPortString/v6_zone-8 97.0ns ± 0% 97.0ns ± 0% ~ (p=0.838 n=10+10) name old alloc/op new alloc/op delta AddrPortString/v4-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) AddrPortString/v6-8 96.0B ± 0% 96.0B ± 0% ~ (all equal) AddrPortString/v6_ellipsis-8 56.0B ± 0% 56.0B ± 0% ~ (all equal) AddrPortString/v6_v4-8 56.0B ± 0% 56.0B ± 0% ~ (all equal) AddrPortString/v6_zone-8 56.0B ± 0% 56.0B ± 0% ~ (all equal) name old allocs/op new allocs/op delta AddrPortString/v4-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) AddrPortString/v6-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) AddrPortString/v6_ellipsis-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) AddrPortString/v6_v4-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) AddrPortString/v6_zone-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) Change-Id: I3d3fcee807ca33d1e8d6dafb03ab844ea0c76bea Reviewed-on: https://go-review.googlesource.com/c/go/+/542396 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
2023-11-17net/netip: optimize AddrPort.String for IPv6 addressesTobias Klauser
Inline the IPv6-case of joinHostPort and avoid the check for a colon in the address. The address is already known to be an IPv6 address. Also use iota.Uitoa to convert the uin16 port. name old time/op new time/op delta AddrPortString/v4-8 45.7ns ± 0% 46.6ns ± 1% +1.97% (p=0.000 n=10+10) AddrPortString/v6-8 106ns ± 3% 95ns ± 0% -10.79% (p=0.000 n=9+10) AddrPortString/v6_ellipsis-8 110ns ± 0% 96ns ± 0% -12.37% (p=0.000 n=9+10) AddrPortString/v6_v4-8 85.1ns ± 0% 70.8ns ± 0% -16.80% (p=0.000 n=9+10) AddrPortString/v6_zone-8 110ns ± 0% 97ns ± 0% -12.29% (p=0.000 n=10+10) name old alloc/op new alloc/op delta AddrPortString/v4-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) AddrPortString/v6-8 101B ± 0% 96B ± 0% -4.95% (p=0.000 n=10+10) AddrPortString/v6_ellipsis-8 61.0B ± 0% 56.0B ± 0% -8.20% (p=0.000 n=10+10) AddrPortString/v6_v4-8 61.0B ± 0% 56.0B ± 0% -8.20% (p=0.000 n=10+10) AddrPortString/v6_zone-8 61.0B ± 0% 56.0B ± 0% -8.20% (p=0.000 n=10+10) name old allocs/op new allocs/op delta AddrPortString/v4-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) AddrPortString/v6-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=10+10) AddrPortString/v6_ellipsis-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=10+10) AddrPortString/v6_v4-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=10+10) AddrPortString/v6_zone-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=10+10) Change-Id: I96a7f01c4e33bfa157a98558c62e2e0a8147726d Reviewed-on: https://go-review.googlesource.com/c/go/+/542395 Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
2023-11-15net/http: don't set length for non-range encoded content requestsDamien Neil
Historically, serveContent has not set Content-Length when the user provides Content-Encoding. This causes broken responses when the user sets both Content-Length and Content-Encoding, and the request is a range request, because the returned data doesn't match the declared length. CL 381956 fixed this case by changing serveContent to always set a Content-Length header. Unfortunately, I've discovered multiple cases in the wild of users setting Content-Encoding: gzip and passing serveContent a ResponseWriter wrapper that gzips the data written to it. This breaks serveContent in a number of ways. In particular, there's no way for it to respond to Range requests properly, because it doesn't know the recipient's view of the content. What the user should be doing in this case is just using io.Copy to send the gzipped data to the response. Or possibly setting Transfer-Encoding: gzip. But whatever they should be doing, what they are doing has mostly worked for non-Range requests, and setting Content-Length makes it stop working because the length of the file being served doesn't match the number of bytes being sent. So in the interests of not breaking users (even if they're misusing serveContent in ways that are already broken), partially revert CL 381956. For non-Range requests, don't set Content-Length when the user has set Content-Encoding. This matches our previous behavior and causes minimal harm in cases where we could have set Content-Length. (We will send using chunked encoding rather than identity, but that's fine.) For Range requests, set Content-Length unconditionally. Either the user isn't mangling the data in the ResponseWriter, in which case the length is correct, or they are, in which case the response isn't going to contain the right bytes anyway. (Note that a Range request for a Content-Length: gzip file is requesting a range of *gzipped* bytes, not a range from the uncompressed file.) Change-Id: I5e788e6756f34cee520aa7c456826f462a59f7eb Reviewed-on: https://go-review.googlesource.com/c/go/+/542595 Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2023-11-14crypto/tls: remove RSA KEX ciphers from the default listRoland Shoemaker
Removes the RSA KEX based ciphers from the default list. This can be reverted using the tlsrsakex GODEBUG. Fixes #63413 Change-Id: Id221be3eb2f6c24b91039d380313f0c87d339f98 Reviewed-on: https://go-review.googlesource.com/c/go/+/541517 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
2023-11-09net/http: use copyBufPool in transferWriter.doBodyCopy()Michał Matczuk
This is a followup to CL 14177. It applies copyBufPool optimization to transferWriter.doBodyCopy(). The function is used every time Request or Response is written. Without this patch for every Request and Response processed, if there is a body, we need to allocate and GC a 32k buffer. This is quickly causing GC pressure. Fixes #57202 Change-Id: I4c30e1737726ac8d9937846106efd02effbae300 GitHub-Last-Rev: 908573cdbe2e8b6f91ce026cf8632ff5f2c41110 GitHub-Pull-Request: golang/go#57205 Reviewed-on: https://go-review.googlesource.com/c/go/+/456435 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com>
2023-11-07net/url: use quick path in URL.Encode() on empty mapMikhail Mazurskiy
Make url.Values.Encode() slightly more efficient when url.Values is an empty but non-nil map. Change-Id: I7f205cc7e67526a1fa0035eab4773cec5e0f2c99 GitHub-Last-Rev: 0530b439db8cea755464c6dc2d268a0bd567889e GitHub-Pull-Request: golang/go#63836 Reviewed-on: https://go-review.googlesource.com/c/go/+/538637 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Damien Neil <dneil@google.com>
2023-11-07net/http/cookiejar: remove unused variablewulianglongrd
The errNoHostname variable is not used, delete it. Change-Id: I62ca6390fd026e6a8cb1e8147f3fbfc3078c2249 Reviewed-on: https://go-review.googlesource.com/c/go/+/538455 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-11-07net/http/cgi: eliminate use of Perl in testsaimuz
Previously, a Perl script was used to test the net/http/cgi package. This sometimes led to hidden failures as these tests were not run on builders without Perl. Also, this approach posed maintenance difficulties for those unfamiliar with Perl. We have now replaced Perl-based tests with a Go handler to simplify maintenance and ensure consistent testing environments. It's part of our ongoing effort to reduce reliance on Perl throughout the Go codebase (see #20032,#25586,#25669,#27779), thus improving reliability and ease of maintenance. Fixes #63800 Fixes #63828 Change-Id: I8d554af93d4070036cf0cc3aaa9c9b256affbd17 GitHub-Last-Rev: a8034083d824da7d68e5995a7997a1199d78de15 GitHub-Pull-Request: golang/go#63869 Reviewed-on: https://go-review.googlesource.com/c/go/+/538861 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: qiulaidongfeng <2645477756@qq.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Commit-Queue: Bryan Mills <bcmills@google.com>
2023-11-06net/http/cgi: the PATH_INFO should be empty or start with a slashaimuz
fixed PATH_INFO not starting with a slash as described in RFC 3875 for PATH_INFO. Fixes #63925 Change-Id: I1ead98dff190c53eb7a50546569ef6ded3199a0a GitHub-Last-Rev: 1c532e330b0d74ee42afc412611a005bc565bb26 GitHub-Pull-Request: golang/go#63926 Reviewed-on: https://go-review.googlesource.com/c/go/+/539615 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-06net/http: remove Content-Encoding header in roundtrip_jscarl.tao
The fetch api will decode the gzip, but Content-Encoding not be deleted. To ensure that the behavior of roundtrip_js is consistent with native. delete the Content-Encoding header when the response body is decompressed by js fetch api. Fixes #63139 Change-Id: Ie35b3aa050786e2ef865f9ffa992e30ab060506e Reviewed-on: https://go-review.googlesource.com/c/go/+/530155 Commit-Queue: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Damien Neil <dneil@google.com>
2023-11-06net/netip: allow only valid prefix digits in ParsePrefixMauri de Souza Meneguzzo
The prefix bits for a call to ParsePrefix are passed raw to strconv.Atoi, this means that it can accept +- signs as well as leading zeroes, which are not allowed prefix values following RFC 4632 Section 3.1 and RFC 4291 Section 2.3. Validate non-digit characters as well as leading zeroes and return an error accordingly. Fixes #63850 Change-Id: I412a7e1cecc6ee9ea1582d4b04cb40d79ee714f1 GitHub-Last-Rev: 462d97fc5f412e18376356dbc10b63711c084144 GitHub-Pull-Request: golang/go#63859 Reviewed-on: https://go-review.googlesource.com/c/go/+/538860 Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-06net/http: set/override Content-Length for encoded range requestsMitar
Currently, http.ServeContent returns invalid Content-Length header if: * Request is a range request. * Content is encoded (e.g., gzip compressed). * Content-Length of the encoded content has been set before calling http.ServeContent, as suggested in https://github.com/golang/go/issues/19420. Example: w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Length", strconv.Itoa(len(compressedJsonBody))) w.Header().Set("Content-Encoding", "gzip") w.Header().Set("Etag", etag) http.ServeContent( w, req, "", time.Time{}, bytes.NewReader(compressedJsonBody), ) The issue is that http.ServeContent currently sees Content-Length as something optional when Content-Encoding is set, but that is a problem with range request which can send a payload of different size. So this reverts https://go.dev/cl/4538111 and makes Content-Length be set always to the number of bytes which will actually be send (both for range and non-range requests). Without this fix, this is an example response: HTTP/1.1 206 Partial Content Accept-Ranges: bytes Content-Encoding: gzip Content-Length: 351 Content-Range: bytes 100-350/351 Content-Type: application/json; charset=UTF-8 Etag: "amCTP_vgT5PQt5OsAEI7NFJ6Hx1UfEpR5nIaYEInfOA" Date: Sat, 29 Jan 2022 14:42:15 GMT As you see, Content-Length is invalid and should be 251. Change-Id: I4d2ea3a8489a115f92ef1f7e98250d555b47a94e GitHub-Last-Rev: 3aff9126f5d62725c7d539df2d0eb2b860a84ca6 GitHub-Pull-Request: golang/go#50904 Reviewed-on: https://go-review.googlesource.com/c/go/+/381956 Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: t hepudds <thepudds1460@gmail.com> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-11-06net/http/httptest: remove unnecessary creation of http.TransportZeke Lu
In (*Server).StartTLS, it's unnecessary to create an http.Client with a Transport, because a new one will be created with the TLSClientConfig later. Change-Id: I086e28717e9739787529006c3f0296c8224cd790 GitHub-Last-Rev: 33724596bd901a05a91654f8c2df233aa6563ea6 GitHub-Pull-Request: golang/go#60124 Reviewed-on: https://go-review.googlesource.com/c/go/+/494355 Run-TryBot: t hepudds <thepudds1460@gmail.com> Reviewed-by: Heschi Kreinick <heschi@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com>
2023-11-02net/http: remove arbitrary timeouts in tests of Server.ErrorLogBryan C. Mills
This also allows us to remove the chanWriter helper from the test, using a simpler strings.Builder instead, relying on clientServerTest.close for synchronization. (I don't think this runs afoul of #38370, because the handler functions themselves in these tests should never be executed, let alone result in an asynchronous write to the error log.) Fixes #57599. Change-Id: I45c6cefca0bb218f6f9a9659de6bde454547f704 Reviewed-on: https://go-review.googlesource.com/c/go/+/539436 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-26net/http: pull http2 underflow fix from x/net/http2Mauri de Souza Meneguzzo
After CL 534295 was merged to fix a CVE it introduced an underflow when we try to decrement sc.curHandlers in handlerDone. Pull in a fix from x/net/http2: http2: fix underflow in http2 server push https://go-review.googlesource.com/c/net/+/535595 Fixes #63511 Change-Id: I5c678ce7dcc53635f3ad5e4999857cb120dfc1ab GitHub-Last-Rev: 587ffa3cafbb9da6bc82ba8a5b83313f81e5c89b GitHub-Pull-Request: golang/go#63561 Reviewed-on: https://go-review.googlesource.com/c/go/+/535575 Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: David Chase <drchase@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-10-23net/http: remove outdated comment about a support of CONNECT methodKeiichi Hirobe
The net/http.Transport already supports CONNECT after https://go-review.googlesource.com/c/go/+/123156 was merged, which deleted comments in transport.go. Change-Id: I784fdb9b044bc8a4a29bf252328c80a11aaf6901 Reviewed-on: https://go-review.googlesource.com/c/go/+/536057 Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2023-10-23net,internal/poll: mark it as handled even if sendfile(2) succeeded with 0 ↵Andy Pan
bytes sent CL 415834 fixed #53658 and somehow it only fixed it on Linux, sendfile can also succeed with 0 bytes sent on other platforms according to their manuals, this CL will finish the work that CL 415834 left out on other platforms. goos: darwin goarch: arm64 pkg: net │ old │ new │ │ sec/op │ sec/op vs base │ SendfileZeroBytes-10 7.563µ ± 5% 7.184µ ± 6% -5.01% (p=0.009 n=10) │ old │ new │ │ B/op │ B/op vs base │ SendfileZeroBytes-10 3562.5 ± 7% 590.0 ± 2% -83.44% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ SendfileZeroBytes-10 0.00 ± 0% 11.00 ± 0% ? (p=0.000 n=10) [1] https://man.freebsd.org/cgi/man.cgi?sendfile(2) [2] https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/sendfile.2.html [3] https://man.dragonflybsd.org/?command=sendfile&section=2 [4] https://docs.oracle.com/cd/E88353_01/html/E37843/sendfile-3c.html Change-Id: I55832487595ee8e0f44f367cf2a3a1d827ba590d Reviewed-on: https://go-review.googlesource.com/c/go/+/536455 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-10-23net: set IsNotFound on windows and plan9 DNS queriesMateusz Poliwczak
Change-Id: I2a12acb3e4f31dd561d49f47a3b1ae3ac47ab894 GitHub-Last-Rev: 4733964f691c567ccad0279fb553442322543024 GitHub-Pull-Request: golang/go#63542 Reviewed-on: https://go-review.googlesource.com/c/go/+/534937 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com> Run-TryBot: Mateusz Poliwczak <mpoliwczak34@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>