aboutsummaryrefslogtreecommitdiff
path: root/src/net/http/transfer.go
AgeCommit message (Collapse)Author
2016-10-22net/http: add NoBody, don't return nil from NewRequest on zero bodiesBrad Fitzpatrick
This is an alternate solution to https://golang.org/cl/31445 Instead of making NewRequest return a request with Request.Body == nil to signal a zero byte body, add a well-known variable that means explicitly zero. Too many tests inside Google (and presumably the outside world) broke. Change-Id: I78f6ecca8e8aa1e12179c234ccfb6bcf0ee29ba8 Reviewed-on: https://go-review.googlesource.com/31726 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2016-10-19net/http: make NewRequest set empty Body nil, don't peek Read Body in TransportBrad Fitzpatrick
This CL makes NewRequest set Body nil for known-zero bodies, and makes the http1 Transport not peek-Read a byte to determine whether there's a body. Background: Many fields of the Request struct have different meanings for whether they're outgoing (via the Transport) or incoming (via the Server). For outgoing requests, ContentLength and Body are documented as: // Body is the request's body. // // For client requests a nil body means the request has no // body, such as a GET request. The HTTP Client's Transport // is responsible for calling the Close method. Body io.ReadCloser // ContentLength records the length of the associated content. // The value -1 indicates that the length is unknown. // Values >= 0 indicate that the given number of bytes may // be read from Body. // For client requests, a value of 0 with a non-nil Body is // also treated as unknown. ContentLength int64 Because of the ambiguity of what ContentLength==0 means, the http1 and http2 Transports previously Read the first byte of a non-nil Body when the ContentLength was 0 to determine whether there was an actual body (with a non-zero length) and ContentLength just wasn't populated, or it was actually empty. That byte-sniff has been problematic and gross (see #17480, #17071) and was removed for http2 in a previous commit. That means, however, that users doing: req, _ := http.NewRequest("POST", url, strings.NewReader("")) ... would not send a Content-Length header in their http2 request, because the size of the reader (even though it was known, being one of the three common recognized types from NewRequest) was zero, and so the HTTP Transport thought it was simply unset. To signal explicitly-zero vs unset-zero, this CL changes NewRequest to signal explicitly-zero by setting the Body to nil, instead of the strings.NewReader("") or other zero-byte reader. This CL also removes the byte sniff from the http1 Transport, like https://golang.org/cl/31326 did for http2. Updates #17480 Updates #17071 Change-Id: I329f02f124659bf7d8bc01e2c9951ebdd236b52a Reviewed-on: https://go-review.googlesource.com/31445 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-10-17net/http: support multiple identical Content-Length headersEmmanuel Odeke
Referencing RFC 7230 Section 3.3.2, this CL deduplicates multiple identical Content-Length headers of a message or rejects the message as invalid if the Content-Length values differ. Fixes #16490 Change-Id: Ia6b0f58ec7d35710b11a36113d2bd9128f693f64 Reviewed-on: https://go-review.googlesource.com/31252 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-09-30net/http: refactor testing of Request.Body on 0 ContentLengthBrad Fitzpatrick
Code movement only, to look more like the equivalent http2 code, and to make an upcoming fix look more obvious. Updates #16002 (to be fixed once this code is in) Change-Id: Iaa4f965be14e98f9996e7c4624afe6e19bed1a80 Reviewed-on: https://go-review.googlesource.com/30087 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2016-08-16net/http: use keyed composite literalJosh Bleecher Snyder
Makes vet happy. Updates #11041 Change-Id: I23ca413c03ff387359440af8114786cd7880a048 Reviewed-on: https://go-review.googlesource.com/27124 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-07-13all: rename vendored golang.org/x/net packages to golang_orgBrad Fitzpatrick
Regression from Go 1.6 to Go 1.7rc1: we had broken the ability for users to vendor "golang.org/x/net/http2" or "golang.org/x/net/route" because we were vendoring them ourselves and cmd/go and cmd/compile do not understand multiple vendor directories across multiple GOPATH workspaces (e.g. user's $GOPATH and default $GOROOT). As a short-term fix, since fixing cmd/go and cmd/compile is too invasive at this point in the cycle, just rename "golang.org" to "golang_org" for the standard library's vendored copy. Fixes #16333 Change-Id: I9bfaed91e9f7d4ca6bab07befe80d71d437a21af Reviewed-on: https://go-review.googlesource.com/24902 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-05-19net/http: update bundled http2Brad Fitzpatrick
Updates x/net/http2 to git rev 5916dcb1 for: * http2, lex/httplex: make Transport reject bogus headers before sending https://golang.org/cl/23229 * http2: reject more trailer values https://golang.org/cl/23230 Fixes #14048 Fixes #14188 Change-Id: Iaa8beca6e005267a3e849a10013eb424a882f2bb Reviewed-on: https://go-review.googlesource.com/23234 Reviewed-by: Andrew Gerrand <adg@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-04-12all: standardize RFC mention formatDan Peterson
Standardize on space between "RFC" and number. Additionally change the couple "a RFC" instances to "an RFC." Fixes #15258 Change-Id: I2b17ecd06be07dfbb4207c690f52a59ea9b04808 Reviewed-on: https://go-review.googlesource.com/21902 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-02all: single space after period.Brad Fitzpatrick
The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-25net/http: refactored internal shouldClose for readabilityEmmanuel Odeke
Change-Id: Ie89c0945a4cc3aebfa9f7ad7f107bc7ab59ab61c Reviewed-on: https://go-review.googlesource.com/19685 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-01-05net/http: relax recently-updated rules and behavior of CloseNotifierBrad Fitzpatrick
The CloseNotifier implementation and documentation was substantially changed in https://golang.org/cl/17750 but it was a bit too aggressive. Issue #13666 highlighted that in addition to breaking external projects, even the standard library (httputil.ReverseProxy) didn't obey the new rules about not using CloseNotifier until the Request.Body is fully consumed. So, instead of fixing httputil.ReverseProxy, dial back the rules a bit. It's now okay to call CloseNotify before consuming the request body. The docs now say CloseNotifier may wait to fire before the request body is fully consumed, but doesn't say that the behavior is undefined anymore. Instead, we just wait until the request body is consumed and start watching for EOF from the client then. This CL also adds a test to ReverseProxy (using a POST request) that would've caught this earlier. Fixes #13666 Change-Id: Ib4e8c29c4bfbe7511f591cf9ffcda23a0f0b1269 Reviewed-on: https://go-review.googlesource.com/18144 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2015-12-17net/http: be more consistent about Request.Method "" vs "GET"Brad Fitzpatrick
Patch from Russ. No bug identified, but I didn't search exhaustively. The new code is easier to read. Fixes #13621 Change-Id: Ifda936e4101116fa254ead950b5fe06adb14e977 Reviewed-on: https://go-review.googlesource.com/17981 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2015-12-16net/http: split Trailers tests into two halvesBrad Fitzpatrick
The old test was in client_test.go but was a mix of four things: - clients writing trailers - servers reading trailers - servers writing trailers - clients reading trailers It definitely wasn't just about clients. This moves it into clientserver_test.go and separates it into two halves: - servers writing trailers + clients reading trailers - clients writing trailers + servers reading trailers Which still isn't ideal, but is much better, and easier to read. Updates #13557 Change-Id: I8c3e58a1f974c1b10bb11ef9b588cfa0f73ff5d9 Reviewed-on: https://go-review.googlesource.com/17895 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Blake Mizerany <blake.mizerany@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-10-13net/http: ignore Transfer-Encoding for HTTP/1.0 responsesBrad Fitzpatrick
Fixes #12785 Change-Id: Iae4383889298c6a78b1ba41bd2cda70b0758fcba Reviewed-on: https://go-review.googlesource.com/15737 Reviewed-by: Andrew Gerrand <adg@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-08-05net/http: close server conn after broken trailersJed Denlea
Prior to this change, broken trailers would be handled by body.Read, and an error would be returned to its caller (likely a Handler), but that error would go completely unnoticed by the rest of the server flow allowing a broken connection to be reused. This is a possible request smuggling vector. Fixes #12027. Change-Id: I077eb0b8dff35c5d5534ee5f6386127c9954bd58 Reviewed-on: https://go-review.googlesource.com/13148 Reviewed-by: Russ Cox <rsc@golang.org>
2015-08-03net/http: fix server/transport data race when sharing the request bodyBrad Fitzpatrick
Introduced in https://go-review.googlesource.com/12865 (git rev c2db5f4c). This fix doesn't add any new lock acquistions: it just moves the existing one taken by the unreadDataSize method and moves it out wider. It became flaky at rev c2db5f4c, but now reliably passes again: $ go test -v -race -run=TestTransportAndServerSharedBodyRace -count=100 net/http Fixes #11985 Change-Id: I6956d62839fd7c37e2f7441b1d425793f4a0db30 Reviewed-on: https://go-review.googlesource.com/12909 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-07-27net/http: pause briefly after closing Server connection when body remainsBrad Fitzpatrick
From https://github.com/golang/go/issues/11745#issuecomment-123555313 this implements option (b), having the server pause slightly after sending the final response on a TCP connection when we're about to close it when we know there's a request body outstanding. This biases the client (which might not be Go) to prefer our response header over the request body write error. Updates #11745 Change-Id: I07cb0b74519d266c8049d9e0eb23a61304eedbf8 Reviewed-on: https://go-review.googlesource.com/12658 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2015-07-07net/http: revert overly-strict part of earlier smuggling defenseBrad Fitzpatrick
The recent https://golang.org/cl/11810 is reportedly a bit too aggressive. Apparently some HTTP requests in the wild do contain both a Transfer-Encoding along with a bogus Content-Length. Instead of returning a 400 Bad Request error, we should just ignore the Content-Length like we did before. Change-Id: I0001be90d09f8293a34f04691f608342875ff5c4 Reviewed-on: https://go-review.googlesource.com/11962 Reviewed-by: Andrew Gerrand <adg@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-06-30net/http: harden Server against request smugglingBrad Fitzpatrick
See RFC 7230. Thanks to Régis Leroy for the report. Change-Id: Ic1779bc2180900430d4d7a4938cac04ed73c304c Reviewed-on: https://go-review.googlesource.com/11810 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2015-06-25net/http: bound the number of bytes read seeking EOF in Handler's Body.CloseBrad Fitzpatrick
If a client sent a POST with a huge request body, calling req.Body.Close in the handler (which is implicit at the end of a request) would end up consuming it all. Put a cap on that, using the same threshold used elsewhere for similar cases. Fixes #9662 Change-Id: I26628413aa5f623a96ef7c2609a8d03c746669e5 Reviewed-on: https://go-review.googlesource.com/11412 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-14net/http: flush request body chunks in TransportBrad Fitzpatrick
The Transport's writer to the remote server is wrapped in a bufio.Writer to suppress many small writes while writing headers and trailers. However, when writing the request body, the buffering may get in the way if the request body is arriving slowly. Because the io.Copy from the Request.Body to the writer is already buffered, the outer bufio.Writer is unnecessary and prevents small Request.Body.Reads from going to the server right away. (and the io.Reader contract does say to return when you've got something, instead of blocking waiting for more). After the body is finished, the Transport's bufio.Writer is still used for any trailers following. A previous attempted fix for this made the chunk writer always flush if the underlying type was a bufio.Writer, but that is not quite correct. This CL instead makes it opt-in by using a private sentinel type (wrapping a *bufio.Writer) to the chunk writer that requests Flushes after each chunk body (the chunk header & chunk body are still buffered together into one write). Fixes #6574 Change-Id: Icefcdf17130c9e285c80b69af295bfd3e72c3a70 Reviewed-on: https://go-review.googlesource.com/10021 Reviewed-by: Andrew Gerrand <adg@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-04-29net/http: handle "close" amongst multiple Connection tokensBrad Fitzpatrick
Fixes #8840 Change-Id: I194d0248734c15336f91a6bcf57ffcc9c0a3a435 Reviewed-on: https://go-review.googlesource.com/9434 Reviewed-by: David Crawshaw <crawshaw@golang.org>
2015-04-29net/http: Don't set Content-Length: -1 when responding to a POSTPeter Waldschmidt
Fixes an issue where Response.Write writes out a Content-Length: -1 header when the corresponding Request is a POST or PUT and the ContentLength was not previously set. This was encountered when using httputil.DumpResponse to write out the response from a server that responded to a PUT request with no Content-Length header. The dumped output is thus invalid. Change-Id: I52c6ae8ef3443f1f9de92aeee9f9581dabb05991 Reviewed-on: https://go-review.googlesource.com/9496 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2015-01-07bufio: add Reader.DiscardBrad Fitzpatrick
Reader.Discard is the complement to Peek. It discards the next n bytes of input. We already have Reader.Buffered to see how many bytes of data are sitting available in memory, and Reader.Peek to get that that buffer directly. But once you're done with the Peek'd data, you can't get rid of it, other than Reading it. Both Read and io.CopyN(ioutil.Discard, bufReader, N) are relatively slow. People instead resort to multiple blind ReadByte calls, just to advance the internal b.r variable. I've wanted this previously, several people have asked for it in the past on golang-nuts/dev, and somebody just asked me for it again in a private email. There are a few places in the standard library we'd use it too. Change-Id: I85dfad47704a58bd42f6867adbc9e4e1792bc3b0 Reviewed-on: https://go-review.googlesource.com/2260 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-30net/http: support for setting trailers from a server HandlerBrad Fitzpatrick
We already had client support for trailers, but no way for a server to set them short of hijacking the connection. Fixes #7759 Change-Id: Ic83976437739ec6c1acad5f209ed45e501dbb93a Reviewed-on: https://go-review.googlesource.com/2157 Reviewed-by: Andrew Gerrand <adg@golang.org>
2014-09-08build: move package sources from src/pkg to srcRuss Cox
Preparation was in CL 134570043. This CL contains only the effect of 'hg mv src/pkg/* src'. For more about the move, see golang.org/s/go14nopkg.
generated by cgit v1.3-5-g9baa (git 2.53.0) at 2026-04-17 13:19:46 +0000