aboutsummaryrefslogtreecommitdiff
path: root/src/net/http
AgeCommit message (Collapse)Author
2023-04-07net/http: wait forever for write results in testsDamien Neil
After performing a round trip on a connection, the connection is usually returned to the idle connection pool. If the write of the request did not complete successfully, the connection is not returned. It is possible for the response to be read before the write goroutine has finished signalling that its write has completed. To allow for this, the check to see if the write completed successfully waits for 50ms for the write goroutine to report the result of the write. See comments in persistConn.wroteRequest for more details. On a slow builder, it is possible for the write goroutine to take longer than 50ms to report the status of its write, leading to test flakiness when successive requests unexpectedly use different connections. Set the timeout for waiting for the writer to an effectively infinite duration in tests. Fixes #51147 Fixes #56275 Fixes #56419 Fixes #56577 Fixes #57375 Fixes #57417 Fixes #57476 Fixes #57604 Fixes #57605 Change-Id: I5e92ffd66b676f3f976d8832c0910f27456a6991 Reviewed-on: https://go-review.googlesource.com/c/go/+/483116 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>
2023-04-07net/http: fix a race in TestResponseControllerSetPastReadDeadlineBryan C. Mills
If the Write goroutine is delayed for long enough after its first Write, the handler may have closed both the readc and donec channels by the time it selects over them, and the donec case may be randomly chosen. Handle that case by explicitly checking readc as well. This fixes a race accidentally introduced in CL 482935 and observed in https://build.golang.org/log/fa684750994d1fda409722f144b90c65b4c52cf9. For #59447. Change-Id: I5c87a599910cf8c1d037e5bbce68bf35afd55d61 Reviewed-on: https://go-review.googlesource.com/c/go/+/483036 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>
2023-04-06net/http: improve failure mode for TestResponseControllerSetPastReadDeadlineDamien Neil
A test flake in #59447 seems to indicate that this test got stuck waiting for the test handler to close the readc channel. If the handler returns early due to an unexpected error, it might fail to close this channel. Add a second channel to act as a signal that the handler has given up and the test should stop. This won't fix whatever happened in the flake, but might help us debug it if it happens again. For #59447 Change-Id: I05d84c6176aa938887d93126a6f3bb4dc941c90d Reviewed-on: https://go-review.googlesource.com/c/go/+/482935 Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com>
2023-04-06net/http: add tests covering non-GET methods for file servingDamien Neil
ServeFile and FileServer will respond to methods such as DELETE by serving the file contents. This is surprising, but we don't want to change it without some consideration. Add tests covering the current behavior. For #59470 Change-Id: Ib6a2594c5b2b7f380149fc1628f7204b308161e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/482876 Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Tatiana Bradley <tatianabradley@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-06Revert "net/http: FileServer method check + minimal OPTIONS implementation"Damien Neil
This reverts https://go.dev/cl/413554 Reason for revert: Backwards-incompatible change in behavior. For #53501 For #59375 Change-Id: Ic3f63b378f9c819599b32e5e6e410f6163849317 Reviewed-on: https://go-review.googlesource.com/c/go/+/482635 Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-04net/http: drop client address comparison in ↵Bryan C. Mills
TestTransportRemovesDeadIdleConnections Since the first client connection is explicitly closed before making the second request, we cannot in general assume that the second request uses a different port (it is equally valid to open the new connection on the same port as the old one that was closed). Fixes #59438. Change-Id: I52d5fe493bd8b1b49270d3996d2019d38d375ce9 Reviewed-on: https://go-review.googlesource.com/c/go/+/482175 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-04all: fix misuses of "a" vs "an"cui fliter
Fixes the misuse of "a" vs "an", according to English grammatical expectations and using https://www.a-or-an.com/ Change-Id: I53ac724070e3ff3d33c304483fe72c023c7cda47 Reviewed-on: https://go-review.googlesource.com/c/go/+/480536 Run-TryBot: shuang cui <imcusg@gmail.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-03-31all: replace fmt.Sprintf("%d") with strconv.ItoaPhilippe Antoine
This was found by running `git grep 'fmt.Sprintf("%d",' | grep -v test | grep -v vendor` And this was automatically fixed with gotiti https://github.com/catenacyber/gotiti and using unconvert https://github.com/mdempsky/unconvert to check if there was (tool which fixed another useless cast) Change-Id: I023926bc4aa8d51de45f712ac739a0a80145c28c GitHub-Last-Rev: 1063e32e5b69b6f9bb17673887b8c4ebe5be8fe4 GitHub-Pull-Request: golang/go#59144 Reviewed-on: https://go-review.googlesource.com/c/go/+/477675 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-24net/http: continue using referer header if it's presentPaschalis Tsilias
Currently, net/http replaces the Referer header with the URL of the previous request, regardless of its status. This CL changes this behavior, respecting the Referer header for secure connections, if it is set. Fixes #44160 Change-Id: I2d7fe37dd681549136329e832188294691584870 Reviewed-on: https://go-review.googlesource.com/c/go/+/291636 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Nick Craig-Wood <nickcw@gmail.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-24net/http: fix typoMichał Matczuk
Change-Id: I6e39f14df65685451d3d98c6def1e0665da20590 GitHub-Last-Rev: d9de237bd24ab3deebdbdf1f75a1835169670c1d GitHub-Pull-Request: golang/go#59200 Reviewed-on: https://go-review.googlesource.com/c/go/+/478855 Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-22net/http: improve logging in TestServerSetKeepAlivesEnabledClosesConnsBryan C. Mills
- Log the actual addresses reported, in case that information is relevant. - Keep going after the first error, so that we report more information about the idle connections after they have been used. (Was the first connection dropped completely, or did it later show up as idle?) - Remove the third request at the end of the test. It had been assuming that the address for a new connection would always be different from the address for the just-closed connection; however, that assumption does not hold in general. Removing the third request addresses one of the two failure modes seen in #55195. It may help in investigating the other failure mode, but I do not expect it to fix the failures entirely. (I suspect that the other failure mode is a synchronization bug in returning the idle connection from the first request.) For #55195. Change-Id: If9604ea68db0697268288ce9812dd57633e83fbd Reviewed-on: https://go-review.googlesource.com/c/go/+/478515 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com>
2023-03-22net/http: simplify Conn lifetimes in TestClientTimeoutKillsConn testsBryan C. Mills
This is intended to fix the failure mode observed in https://build.golang.org/log/f153e06ed547517fb2cddb0fa817fea40a6146f7, but I haven't been able to reproduce that failure mode locally so I'm not sure whether it actually does. Change-Id: Ib14378f1299a76b54013419bdc715a9dbdd94667 Reviewed-on: https://go-review.googlesource.com/c/go/+/478235 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-22net/http: in the IdleConnStrsForTesting_h2 helper, omit conns that cannot be ↵Bryan C. Mills
reused In #59155, we observed that the IdleConnStrsForTesting_h2 helper function sometimes reported extra connections after a "client conn not usable" failure and retry. It turns out that that state corresponds exactly to the http2clientConnIdleState.canTakeNewRequest field, so (with a bit of extra nethttpomithttp2 plumbing) we can use that field in the helper to filter out the unusable connections. Fixes #59155. Change-Id: Ief6283c9c8c5ec47dd9f378beb0ddf720832484e Reviewed-on: https://go-review.googlesource.com/c/go/+/477856 Reviewed-by: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>
2023-03-17net/http: eliminate more arbitrary timeouts in testsBryan C. Mills
Change-Id: I5b3158ecd0eb20dc433a53a2b03eb4551cbb3f7d Reviewed-on: https://go-review.googlesource.com/c/go/+/477196 Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com>
2023-03-17net/http/httputil: use response controller in reverse proxyShang Ding
Previously, the reverse proxy is unable to detect the support for hijack or flush if those things are residing in the response writer in a wrapped manner. The reverse proxy now makes use of the new http response controller as the means to discover the underlying flusher and hijacker associated with the response writer, allowing wrapped flusher and hijacker become discoverable. Change-Id: I53acbb12315c3897be068e8c00598ef42fc74649 Reviewed-on: https://go-review.googlesource.com/c/go/+/468755 Run-TryBot: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-15net/http: avoid making a request to a closed server in TestServerShutdownBryan C. Mills
As soon as the test server closes its listener, its port may be reused for another test server. On some platforms port reuse takes a long time, because they cycle through the available ports before reusing an old one. However, other platforms reuse ports much more aggressively. net/http shouldn't know or care which kind of platform it is on — dialing wild connections is risky and can interfere with other tests no matter what platform we do it on. Instead of making the second request after the server has completely shut down, we can start (and finish!) the entire request while we are certain that the listener has been closed but the port is still open serving an existing request. If everything synchronizes as we expect, that should guarantee that the second request fails. Fixes #56421. Change-Id: I56add243bb9f76ee04ead8f643118f9448fd1280 Reviewed-on: https://go-review.googlesource.com/c/go/+/476036 Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2023-03-13net/http: remove more arbitrary timeouts from server testsBryan C. Mills
This change eliminates the easy, arbitrary timouts that should never happen. It leaves in place a couple of more complicated ones that will probably need retry loops for robustness. For #49336. For #36179. Change-Id: I657ef223a66461413a915da5ce9150f49acec04a Reviewed-on: https://go-review.googlesource.com/c/go/+/476035 Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2023-03-09net/http: use Copy in ServeContent if CopyN not neededLeo Antunes
This small PR allows optimizations made in io.Copy (like the use of io.WriterTo) to be used in one possible path of http.ServeContent (in case of a non-Range request). This, in turn, allows us to skip the buffer allocation in io.Copy. Change-Id: Ifa2ece206ecd4556aaaed15d663b65e95e00bb0a GitHub-Last-Rev: 94fc0318145ba1bd48502564f6488aade871c301 GitHub-Pull-Request: golang/go#56480 Reviewed-on: https://go-review.googlesource.com/c/go/+/446276 Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com>
2023-03-08net/http: remove arbitrary timeout in TestServerAllowsBlockingRemoteAddrBryan C. Mills
If the test actually deadlocks, we probably want a goroutine dump to debug it anyway. Otherwise, the arbitrary timeout can only cause spurious failures. Fixes #36179. Change-Id: Ic2037496959a38d3231eefdbc1dd5d45eebdf306 Reviewed-on: https://go-review.googlesource.com/c/go/+/474582 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-07net/http: support full-duplex HTTP/1 responsesDamien Neil
Add support for concurrently reading from an HTTP/1 request body while writing the response. Normally, the HTTP/1 server automatically consumes any remaining request body before starting to write a response, to avoid deadlocking clients which attempt to write a complete request before reading the response. Add a ResponseController.EnableFullDuplex method which disables this behavior. For #15527 For #57786 Change-Id: Ie7ee8267d8333e9b32b82b9b84d4ad28ab8edf01 Reviewed-on: https://go-review.googlesource.com/c/go/+/472636 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
2023-02-28all: fix typospgxiaolianzi
Change-Id: Ica8d5e5799a4de532764ae86cdb623508d3a8e18 GitHub-Last-Rev: 3e97cca9de3885f2fe0d7deb776e59cc1c73146d GitHub-Pull-Request: golang/go#58689 Reviewed-on: https://go-review.googlesource.com/c/go/+/471021 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-02-27net/http: reduce calls to append in hexEscapeNonASCII to gain a slight ↵Andy Pan
performance boost goos: linux goarch: amd64 pkg: net/http cpu: DO-Premium-Intel │ old │ new │ │ sec/op │ sec/op vs base │ HexEscapeNonASCII-4 469.6n ± 20% 371.1n ± 9% -20.98% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ HexEscapeNonASCII-4 192.0 ± 0% 192.0 ± 0% ~ (p=1.000 n=10) ¹ ¹ all samples are equal │ old │ new │ │ allocs/op │ allocs/op vs base │ HexEscapeNonASCII-4 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹ ¹ all samples are equal Change-Id: Ic8d2b3ddcf2cf724dec3f51a2aba205f2c6e4fe6 Reviewed-on: https://go-review.googlesource.com/c/go/+/425786 Reviewed-by: Than McIntosh <thanm@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Run-TryBot: Andy Pan <panjf2000@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-02-22net/http: remove warning when parsing a query containing a semicolonDamien Neil
It's been years since the behavior here was changed, and there's little point in continuing to warn users of it. Fixes #49399 Change-Id: I95f64ca14cacb64ebe78296593b1cc3d837e6b77 Reviewed-on: https://go-review.googlesource.com/c/go/+/470315 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
2023-02-17net/http: remove five second timeout from TestTransportClosesBodyOnErrorDamien Neil
Wait forever and let the test time out with a stack trace if the expected response doesn't happen. Fixes #57990 Change-Id: I835def63db113752cdd06e03e258cb10d63a6a25 Reviewed-on: https://go-review.googlesource.com/c/go/+/463222 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-02-15net/http: support streaming POST content in wasmWill Hawkins
With new releases of Chrome, Opera and Deno it is possible to stream the body of a POST request. Add support for using that interface when it is available. Change-Id: Ib23d63cd3dea634bd9e267abf4e9a9bfa9c525ad Reviewed-on: https://go-review.googlesource.com/c/go/+/458395 Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-02-14mime/multipart: limit memory/inode consumption of ReadFormDamien Neil
Reader.ReadForm is documented as storing "up to maxMemory bytes + 10MB" in memory. Parsed forms can consume substantially more memory than this limit, since ReadForm does not account for map entry overhead and MIME headers. In addition, while the amount of disk memory consumed by ReadForm can be constrained by limiting the size of the parsed input, ReadForm will create one temporary file per form part stored on disk, potentially consuming a large number of inodes. Update ReadForm's memory accounting to include part names, MIME headers, and map entry overhead. Update ReadForm to store all on-disk file parts in a single temporary file. Files returned by FileHeader.Open are documented as having a concrete type of *os.File when a file is stored on disk. The change to use a single temporary file for all parts means that this is no longer the case when a form contains more than a single file part stored on disk. The previous behavior of storing each file part in a separate disk file may be reenabled with GODEBUG=multipartfiles=distinct. Update Reader.NextPart and Reader.NextRawPart to set a 10MiB cap on the size of MIME headers. Thanks to Jakob Ackermann (@das7pad) for reporting this issue. Fixes #58006 Fixes CVE-2022-41725 Change-Id: Ibd780a6c4c83ac8bcfd3cbe344f042e9940f2eab Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1714276 Reviewed-by: Julie Qiu <julieqiu@google.com> TryBot-Result: Security TryBots <security-trybots@go-security-trybots.iam.gserviceaccount.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/468124 Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-14all: fix some commentscui fliter
Change-Id: I16ec916b47de2f417b681c8abff5a1375ddf491b Reviewed-on: https://go-review.googlesource.com/c/go/+/468055 Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-02-13all: update vendored golang.org/x/netDamien Neil
Pull in HTTP/2 fix to deflake builders: 547e7edf38 http2: avoid referencing ResponseWrite.Write parameter after returning For #58446 Change-Id: I7f3666bc1f20ee03a7ccf25f0e091033cbc635d7 Reviewed-on: https://go-review.googlesource.com/c/go/+/467657 Auto-Submit: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-10vendor, cmd/vendor: update standard library dependenciesDavid Chase
Change-Id: I6facfae14e850f6c9bf3bcb53489c8b475bbb860 Reviewed-on: https://go-review.googlesource.com/c/go/+/467297 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: David Chase <drchase@google.com>
2023-02-10Revert "io: allocate copy buffers from a pool"Damien Neil
This reverts CL 456555. Reason for revert: This seems too likely to exercise race conditions in code where a Write call continues to access its buffer after returning. The HTTP/2 ResponseWriter is one such example. Reverting this change while we think about this some more. For #57202 Change-Id: Ic86823f81d7da410ea6b3f17fb5b3f9a979e3340 Reviewed-on: https://go-review.googlesource.com/c/go/+/467095 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-09net/http: improve js fetch errorsJohan Brandhorst-Satzkorn
The Error type used in failed fetch invocations contain more information than we're currently extracting. Optimistically look for the cause field for extra context for errors. Change-Id: I6c8e4b230a21ec684af2107707aa605fc2148fcf Reviewed-on: https://go-review.googlesource.com/c/go/+/463978 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Evan Phoenix <evan@phx.io> Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-02-04net/http/httputil: fix syntax in ReverseProxy example commentEthan Lowman
Change-Id: I7e71626246af94047fbd1abb9bb77f2cd9b281fb Reviewed-on: https://go-review.googlesource.com/c/go/+/465195 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com>
2023-02-01net/http: remove another arbitrary timeout in TestTLSHandshakeTimeoutBryan C. Mills
Updates #37327 Change-Id: I87774be71ed54e9c45a27062122e6177888e890a Reviewed-on: https://go-review.googlesource.com/c/go/+/226137 Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Damien Neil <dneil@google.com>
2023-01-31io: allocate copy buffers from a poolDamien Neil
CopyBuffer allocates a 32k buffer when no buffer is available. Allocate these buffers from a sync.Pool. This removes an optimization where the copy buffer size was reduced when the source is a io.LimitedReader (including the case of CopyN) with a limit less than the default buffer size. This change could cause a program which only uses io.Copy with sources with a small limit to allocate unnecessarily large buffers. Programs which care about the transient buffer allocation can avoid this by providing their own buffer. name old time/op new time/op delta CopyNSmall-10 165ns ± 7% 117ns ± 7% -29.19% (p=0.001 n=7+7) CopyNLarge-10 7.33µs ±34% 4.07µs ± 2% -44.52% (p=0.001 n=7+7) name old alloc/op new alloc/op delta CopyNSmall-10 2.20kB ±12% 1.20kB ± 4% -45.24% (p=0.000 n=8+7) CopyNLarge-10 148kB ± 9% 81kB ± 4% -45.26% (p=0.000 n=8+7) name old allocs/op new allocs/op delta CopyNSmall-10 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8) CopyNLarge-10 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8) For #57202 Change-Id: I2292226da9ba1dc09a2543f5d74fe5da06080d49 Reviewed-on: https://go-review.googlesource.com/c/go/+/456555 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Thomas Austad <thomas.austad@gmail.com> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-01-30net/http: add section headers to package docRuss Cox
Change-Id: I2379cceeb74cb8511058b24cdd100b21649505ce Reviewed-on: https://go-review.googlesource.com/c/go/+/462197 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2023-01-30net/http: disable fetch on NodeJSJohan Brandhorst-Satzkorn
NodeJS 18 introduced support for the fetch API for making HTTP requests. This broke all wasm tests that were relying on NodeJS falling back to the fake network implementation in net_fake.go. Disable the fetch API on NodeJS to get tests passing. Fixes #57613 Change-Id: Icb2cce6d5289d812da798e07366f8ac26b5f82cb Reviewed-on: https://go-review.googlesource.com/c/go/+/463976 Reviewed-by: Evan Phoenix <evan@phx.io> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-27net/http: fix missing word in ServeMux docOleksii Sholik
Change-Id: I324cedfcdfa1d7eac7be43dc89febb584359c8e3 GitHub-Last-Rev: 7724cd84f323b531215db0406f72481394d0b206 GitHub-Pull-Request: golang/go#57799 Reviewed-on: https://go-review.googlesource.com/c/go/+/462039 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com>
2023-01-26net/http: keep sensitive headers on redirects to the same hostGustavo Falco
Preserve sensitive headers on a redirect to a different port of the same host. Fixes #35104 Change-Id: I5ab57c414ce92a70e688ee684b9ff02fb062b3c6 GitHub-Last-Rev: 8d53e71e2243c141d70d27a503d0f7e6dee64c3c GitHub-Pull-Request: golang/go#54539 Reviewed-on: https://go-review.googlesource.com/c/go/+/424935 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com>
2023-01-20net/http: close Request.Body when pconn write loop exits earlyDamien Neil
The pconn write loop closes a request's body after sending the request, but in the case where the write loop exits with an unsent request in writech the body is never closed. Close the request body in this case. Fixes #49621 Change-Id: Id94a92937bbfc0beb1396446f4dee32fd2059c7e Reviewed-on: https://go-review.googlesource.com/c/go/+/461675 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2023-01-19internal/godebug: export non-default-behavior counters in runtime/metricsRuss Cox
Allow GODEBUG users to report how many times a setting resulted in non-default behavior. Record non-default-behaviors for all existing GODEBUGs. Also rework tests to ensure that runtime is in sync with runtime/metrics.All, and generate docs mechanically from metrics.All. For #56986. Change-Id: Iefa1213e2a5c3f19ea16cd53298c487952ef05a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/453618 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-19runtime: replace panic(nil) with panic(new(runtime.PanicNilError))Russ Cox
Long ago we decided that panic(nil) was too unlikely to bother making a special case for purposes of recover. Unfortunately, it has turned out not to be a special case. There are many examples of code in the Go ecosystem where an author has written panic(nil) because they want to panic and don't care about the panic value. Using panic(nil) in this case has the unfortunate behavior of making recover behave as though the goroutine isn't panicking. As a result, code like: func f() { defer func() { if err := recover(); err != nil { log.Fatalf("panicked! %v", err) } }() call1() call2() } looks like it guarantees that call2 has been run any time f returns, but that turns out not to be strictly true. If call1 does panic(nil), then f returns "successfully", having recovered the panic, but without calling call2. Instead you have to write something like: func f() { done := false defer func() { if err := recover(); !done { log.Fatalf("panicked! %v", err) } }() call1() call2() done = true } which defeats nearly the whole point of recover. No one does this, with the result that almost all uses of recover are subtly broken. One specific broken use along these lines is in net/http, which recovers from panics in handlers and sends back an HTTP error. Users discovered in the early days of Go that panic(nil) was a convenient way to jump out of a handler up to the serving loop without sending back an HTTP error. This was a bug, not a feature. Go 1.8 added panic(http.ErrAbortHandler) as a better way to access the feature. Any lingering code that uses panic(nil) to abort an HTTP handler without a failure message should be changed to use http.ErrAbortHandler. Programs that need the old, unintended behavior from net/http or other packages can set GODEBUG=panicnil=1 to stop the run-time error. Uses of recover that want to detect panic(nil) in new programs can check for recover returning a value of type *runtime.PanicNilError. Because the new GODEBUG is used inside the runtime, we can't import internal/godebug, so there is some new machinery to cross-connect those in this CL, to allow a mutable GODEBUG setting. That won't be necessary if we add any other mutable GODEBUG settings in the future. The CL also corrects the handling of defaulted GODEBUG values in the runtime, for #56986. Fixes #25448. Change-Id: I2b39c7e83e4f7aa308777dabf2edae54773e03f5 Reviewed-on: https://go-review.googlesource.com/c/go/+/461956 Reviewed-by: Robert Griesemer <gri@google.com> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org>
2022-12-21net/http/httputil: don't add X-Forwarded-{Host,Proto} after invoking ↵Damien Neil
Director funcs This reverts CL 407414. When forwarding an inbound request that contains an existing X-Forwarded-Host or X-Forwarded-Proto header, a proxy might want to preserve the header from the inbound request, replace it with its own header, or not include any header at all. CL 407414 replaces inbound X-Forwarded-{Host,Proto} headers by default, and allows a Director func to disable sending these headers at all. However, the Director hook API isn't sufficiently flexible to permit the previous behavior of preserving inbound values unchanged. The new Rewrite API does have this flexibility; users of Rewrite can easily pick the exact behavior they want. Revert the change to ReverseProxy when using a Director func. Users who want a convenient way to set X-Forwarded-* headers to reasonable values can migrate to Rewrite at their convenience, and users depending on the current behavior will be unaffected. For #50465. Fixes #57132. Change-Id: Ic42449c1bb525d6c9920bf721efbc519697f4f20 Reviewed-on: https://go-review.googlesource.com/c/go/+/457595 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-12-16net/http: improve errors in TestCancelRequestWhenSharingConnectionDamien Neil
Provide more information about why this test might be hanging waiting for PutIdleConn to be called (#56587): If the round trip that should result in PutIdleConn being invoked completes, report that to the goroutine waiting for PutIdleConn. For #56587 Change-Id: Ie476ea0ce4a48d2bda6b9b109f89d675a10e7e45 Reviewed-on: https://go-review.googlesource.com/c/go/+/457775 Auto-Submit: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-12-07all: update vendored golang.org/x/netDamien Neil
Pull in HTTP/2 security fix: 1e63c2f08a http2: limit canonical header cache by bytes, not entries Fixes #56350 Change-Id: Ib14024ed894ba266f05d4a6e8c454234a45677d2 Reviewed-on: https://go-review.googlesource.com/c/go/+/455717 Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Jenny Rakoczy <jenny@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-12-06os, net/http: avoid escapes from os.DirFS and http.Dir on WindowsDamien Neil
Do not permit access to Windows reserved device names (NUL, COM1, etc.) via os.DirFS and http.Dir filesystems. Avoid escapes from os.DirFS(`\`) on Windows. DirFS would join the the root to the relative path with a path separator, making os.DirFS(`\`).Open(`/foo/bar`) open the path `\\foo\bar`, which is a UNC name. Not only does this not open the intended file, but permits reference to any file on the system rather than only files on the current drive. Make os.DirFS("") invalid, with all file access failing. Previously, a root of "" was interpreted as "/", which is surprising and probably unintentional. Fixes CVE-2022-41720 Fixes #56694 Change-Id: I275b5fa391e6ad7404309ea98ccc97405942e0f0 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1663834 Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Reviewed-by: Julie Qiu <julieqiu@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/455362 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Jenny Rakoczy <jenny@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/455716 Reviewed-by: Jenny Rakoczy <jenny@golang.org> Run-TryBot: Damien Neil <dneil@google.com>
2022-11-30net/http: deflake and fix TestWrappedResponseControllerDamien Neil
Read the full (empty) response body before closing it, to avoid cancelling the request while the server handler is still running. Wrap the ResponseWriter before calling NewResponseController: This test is intended to verify that wrapping the controller works properly, but neglected to actually wrap the controller. Fixes #56961. Change-Id: I00269f897448ab34676338707b7a04d19ff17963 Reviewed-on: https://go-review.googlesource.com/c/go/+/453860 Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-22all: enable disabled HTTP/2 testsDamien Neil
Update net/http to enable tests that pass with the latest update to the vendored x/net. Update a few tests: Windows apparently doesn't guarantee that time.Since(time.Now()) is >=0, so to set a definitely-expired write deadline, use a time firmly in the past rather than now. Put a backoff loop on TestServerReadTimeout to avoid failures when the timeout expires mid-TLS-handshake. (The TLS handshake timeout is set to min(ReadTimeout, WriteTimeout, ReadHeaderTimeout); there's no way to set a long TLS handshake timeout and a short read timeout.) Don't close the http.Server in TestServerWriteTimeout while the handler may still be executing, since this can result in us getting the wrong error. Change the GOOS=js fake net implementation to properly return ErrDeadlineExceeded when a read/write deadline is exceeded, rather than EAGAIN. For #49837 For #54136 Change-Id: Id8a4ff6ac58336ff212dda3c8799b320cd6b9c19 Reviewed-on: https://go-review.googlesource.com/c/go/+/449935 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-22Revert "net/url, net/http/httputil: accept invalid percent encodings"Damien Neil
This reverts CL 450375. Reason for revert: This change causes test failures (and possibly other problems) for users depending on the existing validation behavior. Rolling back the change for now to give us more time to consider its impact. This landed late in the cycle and isn't urgent; it can wait for 1.21 if we do want to make the change. Fixes #56884 For #56732 Change-Id: I082023c67f1bbb933a617453ab92b67abba876ef Reviewed-on: https://go-review.googlesource.com/c/go/+/452795 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-11-22all: update vendored golang.org/x dependencies for Go 1.20 releaseCherry Mui
The Go 1.20 code freeze has recently started. This is a time to update all golang.org/x/... module versions that contribute packages to the std and cmd modules in the standard library to latest master versions. This CL updates them with x/build/cmd/updatestd. For #36905. Change-Id: Ie0ec91daeb848f00f64686003012297161ad02fa Reviewed-on: https://go-review.googlesource.com/c/go/+/452766 Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-11-19net/http: deflake TestIssue4191_InfiniteGetTimeoutDamien Neil
This test exercises the case where a net.Conn error occurs while writing a response body. It injects an error by setting a timeout on the Conn. If this timeout expires before response headers are written, the test fails. The test attempts to recover from this failure by extending the timeout and retrying. Set the timeout after the response headers are removed, and remove the retry loop. Fixes #56274. Change-Id: I293f8bedb7b20a21d14f43ea9bb48fc56b59441c Reviewed-on: https://go-review.googlesource.com/c/go/+/452175 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Damien Neil <dneil@google.com>