aboutsummaryrefslogtreecommitdiff
path: root/src/net/http/transfer.go
AgeCommit message (Collapse)Author
2023-02-14all: fix some commentscui fliter
Change-Id: I16ec916b47de2f417b681c8abff5a1375ddf491b Reviewed-on: https://go-review.googlesource.com/c/go/+/468055 Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-11-18all: add missing periods in commentscui fliter
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29 Reviewed-on: https://go-review.googlesource.com/c/go/+/449757 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Joedian Reid <joedian@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-09-21net/http: accept HEAD requests with a bodyDamien Neil
RFC 7231 permits HEAD requests to contain a body, although it does state there are no defined semantics for payloads of HEAD requests and that some servers may reject HEAD requests with a payload. Accept HEAD requests with a body. Fix a bug where a HEAD request with a chunked body would interpret the body as the headers for the next request on the connection. For #53960. Change-Id: I83f7112fdedabd6d6291cd956151d718ee6942cd Reviewed-on: https://go-review.googlesource.com/c/go/+/418614 Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-29net/http: don't strip whitespace from Transfer-Encoding headersDamien Neil
Do not accept "Transfer-Encoding: \rchunked" as a valid TE header setting chunked encoding. Thanks to Zeyu Zhang (https://www.zeyu2001.com/) for identifying the issue. Fixes #53188 Fixes CVE-2022-1705 Change-Id: I1a16631425159267f2eca68056b057192a7edf6c Reviewed-on: https://go-review.googlesource.com/c/go/+/409874 Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2022-05-17net/http: allow sending 1xx responsesKévin Dunglas
Currently, it's not possible to send informational responses such as 103 Early Hints or 102 Processing. This patch allows calling WriteHeader() multiple times in order to send informational responses before the final one. If the status code is in the 1xx range, the current content of the header map is also sent. Its content is not removed after the call to WriteHeader() because the headers must also be included in the final response. The Chrome and Fastly teams are starting a large-scale experiment to measure the real-life impact of the 103 status code. Using Early Hints is proposed as a (partial) alternative to Server Push, which are going to be removed from Chrome: https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYLvmQUBY/m/21anpFhxAQAJ Being able to send this status code from servers implemented using Go would help to see if implementing it in browsers is worth it. Fixes #26089 Fixes #36734 Updates #26088 Change-Id: Ib7023c1892c35e8915d4305dd7f6373dbd00a19d GitHub-Last-Rev: 06d749d3454aa35c177a50ce4a25715df21fd742 GitHub-Pull-Request: golang/go#42597 Reviewed-on: https://go-review.googlesource.com/c/go/+/269997 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-03io: NopCloser forward WriterTo implementations if the reader supports itJorropo
This patch also include related fixes to net/http. io_test.go don't test reading or WritingTo of the because the logic is simple. NopCloser didn't even had direct tests before. Fixes #51566 Change-Id: I1943ee2c20d0fe749f4d04177342ce6eca443efe GitHub-Last-Rev: a6b9af4e945a6903735a74aa185e2d1c4c2e2cef GitHub-Pull-Request: golang/go#52340 Reviewed-on: https://go-review.googlesource.com/c/go/+/400236 Run-TryBot: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-04-11all: gofmt main repoRuss Cox
[This CL is part of a sequence implementing the proposal #51082. The design doc is at https://go.dev/s/godocfmt-design.] Run the updated gofmt, which reformats doc comments, on the main repository. Vendored files are excluded. For #51082. Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407 Reviewed-on: https://go-review.googlesource.com/c/go/+/384268 Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-11-07net/http: fix spelling in documentationjiahua wang
Change-Id: I8b0924300eafe27de98975512a78a6527a92e446 Reviewed-on: https://go-review.googlesource.com/c/go/+/354729 Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Damien Neil <dneil@google.com>
2021-09-02net/http: fix hang in probing for a zero-length request bodyDamien Neil
Fix a hang that occurs when making a request and all of the following apply: * The request method is one of GET, HEAD, DELETE, OPTIONS, PROPFIND, or SEARCH. * The Request.Body is non-nil. * The content length is not set, or is set to -1. * Transfer-Encoding: chunked is not set. * The request body does not respond to a read within 200ms. In this case, we give up on probing for a zero-length body and send the request while the probe completes in the background. Fix a bug in the io.Reader wrapping the in-flight probe: It should return io.EOF after the probe completes, but does not. Fixes #47568. Change-Id: I7f9188c96e1210055df68424081af927006e4816 Reviewed-on: https://go-review.googlesource.com/c/go/+/340256 Trust: Damien Neil <dneil@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ingo Oeser <nightlyone@googlemail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-05-10net/http: switch HTTP1 to ASCII equivalents of string functionsRoberto Clapis
The current implementation uses UTF-aware functions like strings.EqualFold and strings.ToLower. This could, in some cases, cause http smuggling. Change-Id: I0e76a993470a1e1b1b472f4b2859ea0a2b22ada0 Reviewed-on: https://go-review.googlesource.com/c/go/+/308009 Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Roberto Clapis <roberto@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-16net/http: ensure Request.Body.Close is called once and only onceRoss Light
Makes *Request.write always close the body, so that callers no longer have to close the body on returned errors, which was the trigger for double-close behavior. Fixes #40382 Change-Id: I128f7ec70415f240d82154cfca134b3f692191e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/257819 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Trust: Damien Neil <dneil@google.com> Trust: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-08-24net/http: set Content-Length:0 for empty PATCH requests as with POST, PATCHSegflow
Sets Content-Length:0 for nil bodies in PATCH requests, as we already do for POST and PUT requests. RFC 2616 mentions that unless a method’s Content-Length is forbidden it can send one. In the wild, we’ve found that Microsoft Azure’s DataLake Gen2 storage API https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/update deliberately rejects PATCH requests without a Content-Length, yet there is no workaround for setting that header when trying to flush the content of a file which was uploaded in a previous request. Fixes #40978 Change-Id: Ib0a623b907d827a1c5ee431dca3c41024fa291c5 GitHub-Last-Rev: 12a3903f2bc22bcc4f5f8e2abcc3922b612b8871 GitHub-Pull-Request: golang/go#40991 Reviewed-on: https://go-review.googlesource.com/c/go/+/250039 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-05-31net/http: reject HTTP/1.1 Content-Length with sign in responsePaschalis Tsilias
Enforces section 14.13 of RFC 2616 so that Content-Length header values with a sign such as "+5" will be rejected. Updates #39017 Change-Id: Icce9f00d03c8475fe704b33f9bed9089ff8802f0 Reviewed-on: https://go-review.googlesource.com/c/go/+/234817 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-05-27net/http: handle body rewind in HTTP/2 connection loss betterRuss Cox
In certain cases the HTTP/2 stack needs to resend a request. It obtains a fresh body to send by calling req.GetBody. This call was missing from the path where the HTTP/2 round tripper returns ErrSkipAltProtocol, meaning fall back to HTTP/1.1. The result was that the HTTP/1.1 fallback request was sent with no body at all. This CL changes that code path to rewind the body before falling back to HTTP/1.1. But rewinding the body is easier said than done. Some requests have no GetBody function, meaning the body can't be rewound. If we need to rewind and can't, that's an error. But if we didn't read anything, we don't need to rewind. So we have to track whether we read anything, with a new ReadCloser wrapper. That in turn requires adding to the couple places that unwrap Body values to look at the underlying implementation. This CL adds the new rewinding code in the main retry loop as well. The new rewindBody function also takes care of closing the old body before abandoning it. That was missing in the old rewind code. Thanks to Aleksandr Razumov for CL 210123 and to Jun Chen for CL 234358, both of which informed this CL. Fixes #32441. Change-Id: Id183758526c087c6b179ab73cf3b61ed23a2a46a Reviewed-on: https://go-review.googlesource.com/c/go/+/234894 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-05-06net/http: use ASCII space trimming throughoutFilippo Valsorda
Security hardening against HTTP request smuggling. Thank you to ZeddYu for reporting this issue. Change-Id: I98bd9f8ffe58360fc3bca9dc5d9a106773e55373 Reviewed-on: https://go-review.googlesource.com/c/go/+/231419 Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-05-06net/http: only support "chunked" in inbound Transfer-Encoding headersFilippo Valsorda
This is a security hardening measure against HTTP request smuggling. Thank you to ZeddYu for reporting this issue. We weren't parsing things correctly anyway, allowing "identity" to be combined with "chunked", and ignoring any Transfer-Encoding header past the first. This is a delicate security surface that already broke before, just be strict and don't add complexity to support cases not observed in the wild (nginx removed "identity" support [1] and multiple TE header support [2]) and removed by RFC 7230 (see page 81). It'd probably be good to also drop support for anything other than "chunked" in outbound TE headers, as "identity" is not a thing anymore, and we are probably off-spec for anything other than "chunked", but it should not be a security concern, so leaving it for now. See #38867. [1]: https://hg.nginx.org/nginx/rev/fe5976aae0e3 [2]: https://hg.nginx.org/nginx/rev/aca005d232ff Change-Id: If17d0827f9c6167a0b19a158e2bc5844ec803288 Reviewed-on: https://go-review.googlesource.com/c/go/+/231418 Reviewed-by: Katie Hockman <katie@golang.org>
2020-05-01net/http: remove badStringError, make some unexported structs non-comparableBrad Fitzpatrick
Reduces binary size by 4K, not counting the http2 changes (in CL 231119) that'll be bundled into this package in the future. Updates golang/go#38782 Change-Id: Id360348707e076b8310a8f409e412d68dd2394b2 Reviewed-on: https://go-review.googlesource.com/c/go/+/231118 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-22Revert "net/http: support gzip, x-gzip Transfer-Encodings"Filippo Valsorda
This reverts commit e6c12c3d0296251f1d5a96ebde811dbfd4a914fe. Reason for revert: the assumption that a T-E of "gzip" implies "chunked" seems incorrect. The RFC does state that one "MUST apply chunked as the final transfer coding" but that should be interpreted to mean that a "chunked" encoding must be listed as the last one, not that one should be assumed to be there if not. This is confirmed by the alternative option to chunking on the server side being to "terminate the message by closing the connection". The issue seems confirmed by the fact that the code in the body of #29162 fails with the following error: net/http: HTTP/1.x transport connection broken: http: failed to gunzip body: unexpected EOF This late in the cycle, revert rather than fix, also because we don't apparently have tests for the correct behavior. Change-Id: I920ec928754cd8e96a06fb7ff8a53316c0f959e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/215757 Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-15all: fix a bunch of misspellingsVille Skyttä
Change-Id: I5b909df0fd048cd66c5a27fca1b06466d3bcaac7 GitHub-Last-Rev: 778c5d21311abee09a5fbda2e4005a5fd4cc3f9f GitHub-Pull-Request: golang/go#35624 Reviewed-on: https://go-review.googlesource.com/c/go/+/207421 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-08net/http: support gzip, x-gzip Transfer-EncodingsEmmanuel T Odeke
Support "gzip" aka "x-gzip" as a transfer-encoding for requests and responses as per RFC 7230 Section 3.3.1. "gzip" and "x-gzip" are equivalents as requested by RFC 7230 Section 4.2.3. Transfer-Encoding is an on-fly property of the body that can be applied by proxies, other servers and basically any intermediary to transport the content e.g. across data centers or backends/machine to machine that need compression. For this change, "gzip" is both explicitly and implicitly combined with transfer-encoding "chunked" in an ordering such as: Transfer-Encoding: gzip, chunked and NOT Transfer-Encoding: chunked, gzip Obviously the latter form is counter-intuitive for streaming. Thus "chunked" is the last value to appear in that transfer-encoding header, if explicitly included. When parsing the response, the chunked body is concatenated as "chunked" does, before finally being decompressed as "gzip". A chunked and compressed body would typically look like this: <LENGTH_1>\r\n<CHUNK_1_GZIPPED_BODY>\r\n<LENGTH_2>\r\n<CHUNK_2_GZIPPED_BODY>\0\r\n which when being processed we would contentate <FULL_BODY> := <CHUNK_1_GZIPPED_BODY> + <CHUNK_2_GZIPPED_BODY> + ... and then finally gunzip it <FINAL_BODY> := gunzip(<FULL_BODY>) If a "chunked" transfer-encoding is NOT applied but "gzip" is applied, we implicitly assume that they requested using "chunked" at the end. This is as per the recommendation of RFC 3.3.1. which explicitly says that for: * Request: " If any transfer coding other than chunked is applied to a request payload body, the sender MUST apply chunked as the final transfer coding to ensure that the message is properly framed." * Response: " If any transfer coding other than chunked is applied to a response payload body, the sender MUST either apply chunked as the final transfer coding or terminate the message by closing the connection." RELNOTE=yes Fixes #29162 Change-Id: Icb8b8b838cf4119705605b29725cabb1fe258491 Reviewed-on: https://go-review.googlesource.com/c/go/+/166517 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-30net/http: make Server return 501 for unsupported transfer-encodingsEmmanuel T Odeke
Ensures that our HTTP/1.X Server properly responds with a 501 Unimplemented as mandated by the spec at RFC 7230 Section 3.3.1, which says: A server that receives a request message with a transfer coding it does not understand SHOULD respond with 501 (Unimplemented). Fixes #30710 Change-Id: I096904e6df053cd1e4b551774cc27523ff3d09f6 Reviewed-on: https://go-review.googlesource.com/c/go/+/167017 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-11all: move internal/x to vendor/golang.org/x and revendor using 'go mod vendor'Bryan C. Mills
This also updates the vendored-in versions of several packages: 'go mod vendor' selects a consistent version of each module, but we had previously vendored an ad-hoc selection of packages. Notably, x/crypto/hkdf was previously vendored in at a much newer commit than the rest of x/crypto. Bringing the rest of x/crypto up to that commit introduced an import of golang.org/x/sys/cpu, which broke the js/wasm build, requiring an upgrade of x/sys to pick up CL 165749. Updates #30228 Updates #30241 Updates #25822 Change-Id: I5b3dbc232b7e6a048a158cbd8d36137af1efb711 Reviewed-on: https://go-review.googlesource.com/c/go/+/164623 Reviewed-by: Filippo Valsorda <filippo@golang.org>
2019-03-07net/http: let Transport request body writes use sendfileChris Marchesi
net.TCPConn has the ability to send data out using system calls such as sendfile when the source data comes from an *os.File. However, the way that I/O has been laid out in the transport means that the File is actually wrapped behind two outer io.Readers, and as such the TCP stack cannot properly type-assert the reader, ensuring that it falls back to genericReadFrom. This commit does the following: * Removes transferBodyReader and moves its functionality to a new doBodyCopy helper. This is not an io.Reader implementation, but no functionality is lost this way, and it allows us to unwrap one layer from the body. * The second layer of the body is unwrapped if the original reader was wrapped with ioutil.NopCloser, which is what NewRequest wraps the body in if it's not a ReadCloser on its own. The unwrap operation passes through the existing body if there's no nopCloser. Note that this depends on change https://golang.org/cl/163737 to properly function, as the lack of ReaderFrom implementation otherwise means that this functionality is essentially walled off. Benchmarks between this commit and https://golang.org/cl/163862, incorporating https://golang.org/cl/163737: linux/amd64: name old time/op new time/op delta FileAndServer_1KB/NoTLS-4 53.2µs ± 0% 53.3µs ± 0% ~ (p=0.075 n=10+9) FileAndServer_1KB/TLS-4 61.2µs ± 0% 60.7µs ± 0% -0.77% (p=0.000 n=10+9) FileAndServer_16MB/NoTLS-4 25.3ms ± 5% 3.8ms ± 6% -84.95% (p=0.000 n=10+10) FileAndServer_16MB/TLS-4 33.2ms ± 2% 13.4ms ± 2% -59.57% (p=0.000 n=10+10) FileAndServer_64MB/NoTLS-4 106ms ± 4% 16ms ± 2% -84.45% (p=0.000 n=10+10) FileAndServer_64MB/TLS-4 129ms ± 1% 54ms ± 3% -58.32% (p=0.000 n=8+10) name old speed new speed delta FileAndServer_1KB/NoTLS-4 19.2MB/s ± 0% 19.2MB/s ± 0% ~ (p=0.095 n=10+9) FileAndServer_1KB/TLS-4 16.7MB/s ± 0% 16.9MB/s ± 0% +0.78% (p=0.000 n=10+9) FileAndServer_16MB/NoTLS-4 664MB/s ± 5% 4415MB/s ± 6% +565.27% (p=0.000 n=10+10) FileAndServer_16MB/TLS-4 505MB/s ± 2% 1250MB/s ± 2% +147.32% (p=0.000 n=10+10) FileAndServer_64MB/NoTLS-4 636MB/s ± 4% 4090MB/s ± 2% +542.81% (p=0.000 n=10+10) FileAndServer_64MB/TLS-4 522MB/s ± 1% 1251MB/s ± 3% +139.95% (p=0.000 n=8+10) darwin/amd64: name old time/op new time/op delta FileAndServer_1KB/NoTLS-8 93.0µs ± 5% 96.6µs ±11% ~ (p=0.190 n=10+10) FileAndServer_1KB/TLS-8 105µs ± 7% 100µs ± 5% -5.14% (p=0.002 n=10+9) FileAndServer_16MB/NoTLS-8 87.5ms ±19% 10.0ms ± 6% -88.57% (p=0.000 n=10+10) FileAndServer_16MB/TLS-8 52.7ms ±11% 17.4ms ± 5% -66.92% (p=0.000 n=10+10) FileAndServer_64MB/NoTLS-8 363ms ±54% 39ms ± 7% -89.24% (p=0.000 n=10+10) FileAndServer_64MB/TLS-8 209ms ±13% 73ms ± 5% -65.37% (p=0.000 n=9+10) name old speed new speed delta FileAndServer_1KB/NoTLS-8 11.0MB/s ± 5% 10.6MB/s ±10% ~ (p=0.184 n=10+10) FileAndServer_1KB/TLS-8 9.75MB/s ± 7% 10.27MB/s ± 5% +5.26% (p=0.003 n=10+9) FileAndServer_16MB/NoTLS-8 194MB/s ±16% 1680MB/s ± 6% +767.83% (p=0.000 n=10+10) FileAndServer_16MB/TLS-8 319MB/s ±10% 963MB/s ± 4% +201.36% (p=0.000 n=10+10) FileAndServer_64MB/NoTLS-8 180MB/s ±31% 1719MB/s ± 7% +853.61% (p=0.000 n=9+10) FileAndServer_64MB/TLS-8 321MB/s ±12% 926MB/s ± 5% +188.24% (p=0.000 n=9+10) Updates #30377. Change-Id: I631a73cea75371dfbb418c9cd487c4aa35e73fcd GitHub-Last-Rev: 4a77dd1b80140274bf3ed20ad7465ff3cc06febf GitHub-Pull-Request: golang/go#30378 Reviewed-on: https://go-review.googlesource.com/c/go/+/163599 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-11-29vendor/golang_org/x: move to internal/xBryan C. Mills
Packages in vendor/ directories have a "vendor/" path prefix in GOPATH mode, but intentionally do not in module mode. Since the import path is embedded in the compiled output, changing that path invalidates cache entries and causes cmd/go to try to rebuild (and reinstall) the vendored libraries, which will fail if the directory containing those libraries is read-only. If I understood correctly, this is the approach Russ suggested as an alternative to https://golang.org/cl/136138. Fixes #27285 Fixes #26988 Change-Id: I8a2507fa892b84cde0a803aaa79e460723da572b Reviewed-on: https://go-review.googlesource.com/c/147443 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2018-10-30net/http: in Transport, don't error on non-chunked response with Trailer headerBigMikes
There are cases where HTTP message specifies the Trailer header but not the Transfer-Encoding = chunked. The existing implementation would return an error in those cases, without returning also the message itself. Instead, it would be preferable to let the library user decide when the message is valid or not. This change makes the fixTrailer() function not to return an error and to keep the Trailer value in the Response.Header map but not populate Response.Trailer. Fixes #27197 Change-Id: Ic1e96791fde97f31dc5ecb8de05c8e4f49465c2c Reviewed-on: https://go-review.googlesource.com/c/145398 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12net/http: flesh out Transport's HTTP/1 CONNECT+bidi support to match HTTP/2Brad Fitzpatrick
Fixes #17227 Change-Id: I0f8964593d69623b85d5759f6276063ee62b2915 Reviewed-on: https://go-review.googlesource.com/c/123156 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-08all: fix a bunch of misspellingsIgor Zhilianin
Change-Id: I94cebca86706e072fbe3be782d3edbe0e22b9432 GitHub-Last-Rev: 8e15a40545704fb21b41a8768079f2da19341ef3 GitHub-Pull-Request: golang/go#28067 Reviewed-on: https://go-review.googlesource.com/c/140437 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-06-27net/http/httptrace: expose request headers for http/1.1Meir Fischer
Some headers, which are set or modified by the http library, are not written to the standard http.Request.Header and are not included as part of http.Response.Request.Header. Exposing all headers alleviates this problem. This is not a complete solution to 19761 since it does not have http/2 support. Updates #19761 Change-Id: Ie8d4f702f4f671666b120b332378644f094e288b Reviewed-on: https://go-review.googlesource.com/67430 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-24net/http: conservatively flush Transport request headers by defaultBrad Fitzpatrick
This changes the http.Transport to flush the bufio.Writer between writing the request headers and the body. That wasn't done in the past to minimize the number of TCP packets on the wire, but that's just an optimization, and it causes problems when servers are waiting for the headers and the client is blocked on something before reading the body. Instead, only do the don't-flush optimization if we know we're not going to block, whitelisting a set of common in-memory Request.Body types. (the same set of types special-cased by http.NewRequest) Fixes #22088 Change-Id: I7717750aa6df32dd3eb92d181b45bc7af24b1144 Reviewed-on: https://go-review.googlesource.com/114316 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-07vendor, net/http: update x/net for httplex to httpguts mergeBrad Fitzpatrick
Updates x/net to git rev cbb82b59bc for: lex/httplex, http/httpguts: merge the httplex package into httpguts https://golang.org/cl/111875 http2: set nextStreamID to 3 when AllowHTTP is set https://golang.org/cl/111835 http2: terminate await request cancel goroutine on conn close https://golang.org/cl/108415 Fixes #24776 (CL 111655 didn't actually include it) Change-Id: I0a21e169ebba2ec35219f347f1e31cd4c67bebdf Reviewed-on: https://go-review.googlesource.com/111876 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Kunpei Sakai <namusyaka@gmail.com> Reviewed-by: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-02-20net/http: use RFC 723x as normative reference in docsDavid Url
Replace references to the obsoleted RFC 2616 with references to RFC 7230 through 7235, to avoid unnecessary confusion. Obvious inconsistencies are marked with todo comments. Updates #21974 Change-Id: I8fb4fcdd1333fc5193b93a2f09598f18c45e7a00 Reviewed-on: https://go-review.googlesource.com/94095 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-10-19net/http: ignore response body when forbidden by status codeBen Burkert
A 1XX, 204, or 304 response may not include a response body according to RFC 7230, section 3.3.3. If a buggy server returns a 204 or 304 response with a body that is chunked encoded, the invalid body is currently made readable in the Response. This can lead to data races due to the transport connection's read loop which does not wait for the body EOF when the response status is 204 or 304. The correct behavior is to ignore the body on a 204 or 304 response, and treat the body data as the beginning of the next request on the connection. Updates #22330. Change-Id: I89a457ceb783b6f66136d5bf9be0a9b0a04fa955 Reviewed-on: https://go-review.googlesource.com/71910 Reviewed-by: Tom Bergan <tombergan@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Tom Bergan <tombergan@google.com>
2017-08-14net/http: various small cleanupsDaniel Martí
* Remove an unnecessary type conversion * Make golint happier about consistent receiver names * Make golint happier about a foo_bar var name Change-Id: I5223808109f6f8b69ed4be76de82faf2478c6a2e Reviewed-on: https://go-review.googlesource.com/54530 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tom Bergan <tombergan@google.com>
2017-06-08net/http: remove invalid checks of Request.Proto* for outgoing requestsBrad Fitzpatrick
The net/http package has long documented that Request.ProtoMajor and Request.ProtoMinor are ignored for outgoing requests (HTTP/1.1 or HTTP/2 is always used, never HTTP/1.0). There was one part in the code that was actually checking 1.0 vs 1.1, but it appears to have been harmless. Remove it. Fixes #18407 Change-Id: I362ed6c47ca2de7a2fbca917ed3e866273cfe41f Reviewed-on: https://go-review.googlesource.com/45155 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-05-24net/http: Don't write 'Connection: close' header multiple times.Filip Gruszczyński
When writing the 'Connection: close' header based on response Close attribute we also check if it is already in the headers scheduled to be written and skip if necessary. Fixes #19499 Change-Id: I92357344a37ae385454ec8006114fa4cfa585810 Reviewed-on: https://go-review.googlesource.com/38076 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02net/http: clean up Transport.RoundTrip error handlingBrad Fitzpatrick
If I put a 10 millisecond sleep at testHookWaitResLoop, before the big select in (*persistConn).roundTrip, two flakes immediately started happening, TestTransportBodyReadError (#19231) and TestTransportPersistConnReadLoopEOF. The problem was that there are many ways for a RoundTrip call to fail (errors reading from Request.Body while writing the response, errors writing the response, errors reading the response due to server closes, errors due to servers sending malformed responses, cancelations, timeouts, etc.), and many of those failures then tear down the TCP connection, causing more failures, since there are always at least three goroutines involved (reading, writing, RoundTripping). Because the errors were communicated over buffered channels to a giant select, the error returned to the caller was a function of which random select case was called, which was why a 10ms delay before the select brought out so many bugs. (several fixed in my previous CLs the past few days). Instead, track the error explicitly in the transportRequest, guarded by a mutex. In addition, this CL now: * differentiates between the two ways writing a request can fail: the io.Copy reading from the Request.Body or the io.Copy writing to the network. A new io.Reader type notes read errors from the Request.Body. The read-from-body vs write-to-network errors are now prioritized differently. * unifies the two mapRoundTripErrorFromXXX methods into one mapRoundTripError method since their logic is now the same. * adds a (*Request).WithT(*testing.T) method in export_test.go, usable by tests, to call t.Logf at points during RoundTrip. This is disabled behind a constant except when debugging. * documents and deflakes TestClientRedirectContext I've tested this CL with high -count values, with/without -race, with/without delays before the select, etc. So far it seems robust. Fixes #19231 (TestTransportBodyReadError flake) Updates #14203 (source of errors unclear; they're now tracked more) Updates #15935 (document Transport errors more; at least understood more now) Change-Id: I3cccc3607f369724b5344763e35ad2b7ea415738 Reviewed-on: https://go-review.googlesource.com/37495 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-12-22net/http: restore Transport's Request.Body byte sniff in limited casesBrad Fitzpatrick
In Go 1.8, we'd removed the Transport's Request.Body one-byte-Read-sniffing to disambiguate between non-nil Request.Body with a ContentLength of 0 or -1. Previously, we tried to see whether a ContentLength of 0 meant actually zero, or just an unset by reading a single byte of the Request.Body and then stitching any read byte back together with the original Request.Body. That historically has caused many problems due to either data races, blocking forever (#17480), or losing bytes (#17071). Thus, we removed it in both HTTP/1 and HTTP/2 in Go 1.8. Unfortunately, during the Go 1.8 beta, we've found that a few people have gotten bitten by the behavior change on requests with methods typically not containing request bodies (e.g. GET, HEAD, DELETE). The most popular example is the aws-go SDK, which always set http.Request.Body to a non-nil value, even on such request methods. That was causing Go 1.8 to send such requests with Transfer-Encoding chunked bodies, with zero bytes, confusing popular servers (including but limited to AWS). This CL partially reverts the no-byte-sniffing behavior and restores it only for GET/HEAD/DELETE/etc requests, and only when there's no Transfer-Encoding set, and the Content-Length is 0 or -1. Updates #18257 (aws-go) bug And also private bug reports about non-AWS issues. Updates #18407 also, but haven't yet audited things enough to declare it fixed. Change-Id: Ie5284d3e067c181839b31faf637eee56e5738a6a Reviewed-on: https://go-review.googlesource.com/34668 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-10-22net/http: add NoBody, don't return nil from NewRequest on zero bodiesBrad Fitzpatrick
This is an alternate solution to https://golang.org/cl/31445 Instead of making NewRequest return a request with Request.Body == nil to signal a zero byte body, add a well-known variable that means explicitly zero. Too many tests inside Google (and presumably the outside world) broke. Change-Id: I78f6ecca8e8aa1e12179c234ccfb6bcf0ee29ba8 Reviewed-on: https://go-review.googlesource.com/31726 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2016-10-19net/http: make NewRequest set empty Body nil, don't peek Read Body in TransportBrad Fitzpatrick
This CL makes NewRequest set Body nil for known-zero bodies, and makes the http1 Transport not peek-Read a byte to determine whether there's a body. Background: Many fields of the Request struct have different meanings for whether they're outgoing (via the Transport) or incoming (via the Server). For outgoing requests, ContentLength and Body are documented as: // Body is the request's body. // // For client requests a nil body means the request has no // body, such as a GET request. The HTTP Client's Transport // is responsible for calling the Close method. Body io.ReadCloser // ContentLength records the length of the associated content. // The value -1 indicates that the length is unknown. // Values >= 0 indicate that the given number of bytes may // be read from Body. // For client requests, a value of 0 with a non-nil Body is // also treated as unknown. ContentLength int64 Because of the ambiguity of what ContentLength==0 means, the http1 and http2 Transports previously Read the first byte of a non-nil Body when the ContentLength was 0 to determine whether there was an actual body (with a non-zero length) and ContentLength just wasn't populated, or it was actually empty. That byte-sniff has been problematic and gross (see #17480, #17071) and was removed for http2 in a previous commit. That means, however, that users doing: req, _ := http.NewRequest("POST", url, strings.NewReader("")) ... would not send a Content-Length header in their http2 request, because the size of the reader (even though it was known, being one of the three common recognized types from NewRequest) was zero, and so the HTTP Transport thought it was simply unset. To signal explicitly-zero vs unset-zero, this CL changes NewRequest to signal explicitly-zero by setting the Body to nil, instead of the strings.NewReader("") or other zero-byte reader. This CL also removes the byte sniff from the http1 Transport, like https://golang.org/cl/31326 did for http2. Updates #17480 Updates #17071 Change-Id: I329f02f124659bf7d8bc01e2c9951ebdd236b52a Reviewed-on: https://go-review.googlesource.com/31445 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-10-17net/http: support multiple identical Content-Length headersEmmanuel Odeke
Referencing RFC 7230 Section 3.3.2, this CL deduplicates multiple identical Content-Length headers of a message or rejects the message as invalid if the Content-Length values differ. Fixes #16490 Change-Id: Ia6b0f58ec7d35710b11a36113d2bd9128f693f64 Reviewed-on: https://go-review.googlesource.com/31252 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-09-30net/http: refactor testing of Request.Body on 0 ContentLengthBrad Fitzpatrick
Code movement only, to look more like the equivalent http2 code, and to make an upcoming fix look more obvious. Updates #16002 (to be fixed once this code is in) Change-Id: Iaa4f965be14e98f9996e7c4624afe6e19bed1a80 Reviewed-on: https://go-review.googlesource.com/30087 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2016-08-16net/http: use keyed composite literalJosh Bleecher Snyder
Makes vet happy. Updates #11041 Change-Id: I23ca413c03ff387359440af8114786cd7880a048 Reviewed-on: https://go-review.googlesource.com/27124 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-07-13all: rename vendored golang.org/x/net packages to golang_orgBrad Fitzpatrick
Regression from Go 1.6 to Go 1.7rc1: we had broken the ability for users to vendor "golang.org/x/net/http2" or "golang.org/x/net/route" because we were vendoring them ourselves and cmd/go and cmd/compile do not understand multiple vendor directories across multiple GOPATH workspaces (e.g. user's $GOPATH and default $GOROOT). As a short-term fix, since fixing cmd/go and cmd/compile is too invasive at this point in the cycle, just rename "golang.org" to "golang_org" for the standard library's vendored copy. Fixes #16333 Change-Id: I9bfaed91e9f7d4ca6bab07befe80d71d437a21af Reviewed-on: https://go-review.googlesource.com/24902 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-05-19net/http: update bundled http2Brad Fitzpatrick
Updates x/net/http2 to git rev 5916dcb1 for: * http2, lex/httplex: make Transport reject bogus headers before sending https://golang.org/cl/23229 * http2: reject more trailer values https://golang.org/cl/23230 Fixes #14048 Fixes #14188 Change-Id: Iaa8beca6e005267a3e849a10013eb424a882f2bb Reviewed-on: https://go-review.googlesource.com/23234 Reviewed-by: Andrew Gerrand <adg@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-04-12all: standardize RFC mention formatDan Peterson
Standardize on space between "RFC" and number. Additionally change the couple "a RFC" instances to "an RFC." Fixes #15258 Change-Id: I2b17ecd06be07dfbb4207c690f52a59ea9b04808 Reviewed-on: https://go-review.googlesource.com/21902 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-02all: single space after period.Brad Fitzpatrick
The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-25net/http: refactored internal shouldClose for readabilityEmmanuel Odeke
Change-Id: Ie89c0945a4cc3aebfa9f7ad7f107bc7ab59ab61c Reviewed-on: https://go-review.googlesource.com/19685 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-01-05net/http: relax recently-updated rules and behavior of CloseNotifierBrad Fitzpatrick
The CloseNotifier implementation and documentation was substantially changed in https://golang.org/cl/17750 but it was a bit too aggressive. Issue #13666 highlighted that in addition to breaking external projects, even the standard library (httputil.ReverseProxy) didn't obey the new rules about not using CloseNotifier until the Request.Body is fully consumed. So, instead of fixing httputil.ReverseProxy, dial back the rules a bit. It's now okay to call CloseNotify before consuming the request body. The docs now say CloseNotifier may wait to fire before the request body is fully consumed, but doesn't say that the behavior is undefined anymore. Instead, we just wait until the request body is consumed and start watching for EOF from the client then. This CL also adds a test to ReverseProxy (using a POST request) that would've caught this earlier. Fixes #13666 Change-Id: Ib4e8c29c4bfbe7511f591cf9ffcda23a0f0b1269 Reviewed-on: https://go-review.googlesource.com/18144 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 11:47:59 +0000