aboutsummaryrefslogtreecommitdiff
path: root/src/encoding
AgeCommit message (Collapse)Author
2020-02-24encoding/json: limit max nesting depthJordan Liggitt
Limit the maximum nesting depth when parsing to protect against stack overflow, permitted by https://tools.ietf.org/html/rfc7159#section-9 A nesting depth limit of 10,000 was chosen to be a conservative balance between avoiding stack overflow and avoiding impacting legitimate JSON documents. 10,000 is less than 1% of the experimental stack depth limit with the default stack size: * On 64-bit systems, the default stack limit is 1GB, which allows ~2,800,000 frames of recursive parsing * On 32-bit systems, the default stack limit is 250MB, which allows ~1,100,000 frames of recursive parsing Fixes #31789 Change-Id: I4f5a90e89dcb4ab1a957ad9d02e1fa0efafaccf6 Reviewed-on: https://go-review.googlesource.com/c/go/+/199837 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2020-02-22encoding/json: fix broken link to json.orgJon Johnson
This updates the link to a newer image. Change-Id: Ibdfe8c57d9217a325bcfde98cb6f952ca63d588a GitHub-Last-Rev: f5970ba395781c0d299dfdaa1ed75e46125b8268 GitHub-Pull-Request: golang/go#36938 Reviewed-on: https://go-review.googlesource.com/c/go/+/217297 Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2019-12-10all: fix a number of misuses of the word "an"Daniel Martí
After golang.org/cl/210124, I wondered if the same error had gone unnoticed elsewhere. I quickly spotted another dozen mistakes after reading through the output of: git grep '\<[Aa]n [bcdfgjklmnpqrtvwyz][a-z]' Many results are false positives for acronyms like "an mtime", since it's pronounced "an em-time". However, the total amount of output isn't that large given how simple the grep pattern is. Change-Id: Iaa2ca69e42f4587a9e3137d6c5ed758887906ca6 Reviewed-on: https://go-review.googlesource.com/c/go/+/210678 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Zach Jones <zachj1@gmail.com> Reviewed-by: Robert Griesemer <gri@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-20encoding/base64: document that Strict mode still ignores newlinesFilippo Valsorda
An application that wants to reject non-canonical encodings is likely to care about other sources of malleability. Change-Id: I1d3a5b281d2631ca78df3f89b957a02687a534d8 Reviewed-on: https://go-review.googlesource.com/c/go/+/188858 Reviewed-by: Katie Hockman <katie@golang.org>
2019-11-11encoding/json: error when encoding a pointer cycleDaniel Martí
Otherwise we'd panic with a stack overflow. Most programs are in control of the data being encoded and can ensure there are no cycles, but sometimes it's not that simple. For example, running a user's html template with script tags can easily result in crashes if the user can find a pointer cycle. Adding the checks via a map to every ptrEncoder.encode call slowed down the benchmarks below by a noticeable 13%. Instead, only start doing the relatively expensive pointer cycle checks if we're many levels of pointers deep in an encode state. A threshold of 1000 is small enough to capture pointer cycles before they're a problem (the goroutine stack limit is currently 1GB, and I needed close to a million levels to reach it). Yet it's large enough that reasonable uses of the json encoder only see a tiny 1% slow-down due to the added ptrLevel field and check. name old time/op new time/op delta CodeEncoder-8 2.34ms ± 1% 2.37ms ± 0% +1.05% (p=0.000 n=10+10) CodeMarshal-8 2.42ms ± 1% 2.44ms ± 0% +1.10% (p=0.000 n=10+10) name old speed new speed delta CodeEncoder-8 829MB/s ± 1% 820MB/s ± 0% -1.04% (p=0.000 n=10+10) CodeMarshal-8 803MB/s ± 1% 795MB/s ± 0% -1.09% (p=0.000 n=10+10) name old alloc/op new alloc/op delta CodeEncoder-8 43.1kB ± 8% 42.5kB ±10% ~ (p=0.989 n=10+10) CodeMarshal-8 1.99MB ± 0% 1.99MB ± 0% ~ (p=0.254 n=9+6) name old allocs/op new allocs/op delta CodeEncoder-8 0.00 0.00 ~ (all equal) CodeMarshal-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) Finally, add a few tests to ensure that the code handles the edge cases properly. Fixes #10769. Change-Id: I73d48e0cf6ea140127ea031f2dbae6e6a55e58b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/187920 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com> Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-11-08Revert "encoding/asn1: fix unmarshalling SEQUENCE OF SET"Than McIntosh
This reverts CL 160819 (commit 4692343cf401a5bbcc29) Reason for revert: causing lots of failures on master Change-Id: I96fd39ae80fe350ba8b3aa310443d41daec38093 Reviewed-on: https://go-review.googlesource.com/c/go/+/206146 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-08encoding/binary: add float support to fast pathMartin Garton
This adds float type support to the main switch blocks in Read and Write, instead of falling back to reflection. This gives a considerable speedup for the float types: ReadFloats-8 129ns ± 9% 70ns ± 8% -46.02% (p=0.001 n=7+7) WriteFloats-8 131ns ± 6% 86ns ±11% -34.59% (p=0.001 n=7+7) ReadSlice1000Float32s-8 14.6µs ±14% 4.8µs ±12% -67.29% (p=0.001 n=7+7) WriteSlice1000Float32s-8 16.4µs ±20% 4.7µs ± 8% -71.01% (p=0.001 n=7+7) Change-Id: I0be99d068b07d10dd6eb1137b45eff6f7c216b87 GitHub-Last-Rev: 4ff326e99ca35977d819f0ba29c10d9efc7e811c GitHub-Pull-Request: golang/go#31803 Reviewed-on: https://go-review.googlesource.com/c/go/+/174959 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-08encoding/asn1: fix unmarshalling SEQUENCE OF SETkaxapi
Fixes #27426 Change-Id: I34d4784658ce7b9e6130bae9717e80d0e9a290a2 GitHub-Last-Rev: 6de610cdcef11832f131b84a0338b68af16b10da GitHub-Pull-Request: golang/go#30059 Reviewed-on: https://go-review.googlesource.com/c/go/+/160819 Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
2019-11-08encoding/binary: make Read return an error when data is not a pointerUdalov Max
Make binary.Read return an error when passed `data` argument is not a pointer to a fixed-size value or a slice of fixed-size values. Fixes #32927 Change-Id: I04f48be55fe9b0cc66c983d152407d0e42cbcd95 Reviewed-on: https://go-review.googlesource.com/c/go/+/184957 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-01encoding/binary: cache struct sizes to speed up Read and WriteLorenz Bauer
A majority of work is spent in dataSize when en/decoding the same struct over and over again. This wastes a lot of work, since the result doesn't change for a given reflect.Value. Cache the result of the function for structs, so that subsequent calls to dataSize can avoid doing work. name old time/op new time/op delta ReadStruct 1.00µs ± 1% 0.37µs ± 1% -62.99% (p=0.029 n=4+4) WriteStruct 1.00µs ± 3% 0.37µs ± 1% -62.69% (p=0.008 n=5+5) name old speed new speed delta ReadStruct 75.1MB/s ± 1% 202.9MB/s ± 1% +170.16% (p=0.029 n=4+4) WriteStruct 74.8MB/s ± 3% 200.4MB/s ± 1% +167.96% (p=0.008 n=5+5) Fixes #34471 Change-Id: Ic5d987ca95f1197415ef93643a0af6fc1224fdf0 Reviewed-on: https://go-review.googlesource.com/c/go/+/199539 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31encoding/json: avoid work when unquoting strings, take 2Daniel Martí
This is a re-submission of CL 151157, since it was reverted in CL 190909 due to an introduced crash found by a fuzzer. The revert CL included regression tests, while this CL includes a fixed version of the original change. In particular, what we forgot in the original optimization was that we still need the length and trailing quote checks at the beginning of unquoteBytes. Without those, we could end up in a crash later on. We can work out how many bytes can be unquoted trivially in rescanLiteral, which already iterates over a string's bytes. Removing the extra loop in unquoteBytes simplifies the function and speeds it up, especially when decoding simple strings, which are common. While at it, we can remove the check that s[0]=='"', since all call sites already meet that condition. name old time/op new time/op delta CodeDecoder-8 10.6ms ± 2% 10.5ms ± 1% -1.01% (p=0.004 n=20+10) name old speed new speed delta CodeDecoder-8 183MB/s ± 2% 185MB/s ± 1% +1.02% (p=0.003 n=20+10) Updates #28923. Change-Id: I8c6b13302bcd86a364bc998d72451332c0809cde Reviewed-on: https://go-review.googlesource.com/c/go/+/190659 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Peter Weinberger <pjw@google.com>
2019-10-30encoding/xml: fix token decoder on early EOFSam Whited
The documentation for TokenReader suggests that implementations of the interface may return a token and io.EOF together, indicating that it is the last token in the stream. This is similar to io.Reader. However, if you wrap such a TokenReader in a Decoder it complained about the EOF. A test was added to ensure this behavior on Decoder's. Change-Id: I9083c91d9626180d3bcf5c069a017050f3c7c4a8 Reviewed-on: https://go-review.googlesource.com/c/go/+/130556 Run-TryBot: Sam Whited <sam@samwhited.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-28encoding/json: remove allocation when using a Marshaler with value receiverPhil Pearl
If we marshal a non-pointer struct field whose type implements Marshaler with a non-pointer receiver, then we avoid an allocation if we take the address of the field before casting it to an interface. name old time/op new time/op delta EncodeMarshaler-8 104ns ± 1% 92ns ± 2% -11.72% (p=0.001 n=7+7) name old alloc/op new alloc/op delta EncodeMarshaler-8 36.0B ± 0% 4.0B ± 0% -88.89% (p=0.000 n=8+8) name old allocs/op new allocs/op delta EncodeMarshaler-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8) Test coverage already looks good enough for this change. TestRefValMarshal already covers all possible combinations of value & pointer receivers on value and pointer struct fields. Change-Id: I6fc7f72396396d98f9a90c3c86e813690f41c099 Reviewed-on: https://go-review.googlesource.com/c/go/+/203608 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-27encoding/json: improve performance of CompactPhil Pearl
This change improves performance of Compact by using a sync.Pool to allow re-use of a scanner. This also has the side-effect of removing an allocation for each field that implements Marshaler when marshalling JSON. name old time/op new time/op delta EncodeMarshaler-8 118ns ± 2% 104ns ± 1% -12.21% (p=0.001 n=7+7) name old alloc/op new alloc/op delta EncodeMarshaler-8 100B ± 0% 36B ± 0% -64.00% (p=0.000 n=8+8) name old allocs/op new allocs/op delta EncodeMarshaler-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=8+8) Change-Id: Ic70c61a0a6354823da5220f5aad04b94c054f233 Reviewed-on: https://go-review.googlesource.com/c/go/+/200864 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-16encoding/json: correct caller's name in encoding errorsEugene Kalinin
1. Change mapencode.encode to use fmt.Error rather than MarshalerError. MarshalerError refer to MarshalJSON, but mapencode.encode does not use that. 2. Add sourceFunc field to MarshalerError to record the name of the function that creates the error, so that the Error method can report it correctly. Fixes #29753 Change-Id: I186c2fac8470ae2f9e300501de3730face642230 Reviewed-on: https://go-review.googlesource.com/c/go/+/184119 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-11encoding/json: add Decoder.InputOffset for offset accessFrancesco Renzi
This makes Decoder.offset public while renaming it to Decoder.InputOffset to match encoding/xml Decoder API Code changes made by Adam Stankiewicz [sheerun@sher.pl] Fixes #29688 Change-Id: I86dbfd2b2da80160846e92bfa580c53d8d45e2db Reviewed-on: https://go-review.googlesource.com/c/go/+/200677 Run-TryBot: Johan Brandhorst <johan.brandhorst@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-10encoding/json: support TextUnmarshaler for map keys with string underlying typesCuong Manh Le
When unmarshaling to a map, the map's key type must either be a string, an integer, or implement encoding.TextUnmarshaler. But for a user defined type, reflect.Kind will not distinguish between the static type and the underlying type. In: var x MyString = "x" t := reflect.TypeOf(x) println(t.Kind() == reflect.String) the Kind of x is still reflect.String, even though the static type of x is MyString. Moreover, checking for the map's key type is a string occurs first, so even if the map key type MyString implements encoding.TextUnmarshaler, it will be ignored. To fix the bug, check for encoding.TextUnmarshaler first. Fixes #34437 Change-Id: I780e0b084575e1dddfbb433fe03857adf71d05fb Reviewed-on: https://go-review.googlesource.com/c/go/+/200237 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2019-10-10encoding/json: stop escaping U+2028 and U+2029 in CompactTom Thorogood
Compact has been inconsistently escaping only some problematic characters (U+2028 and U+2029), but not others (<, > and &). This change addresses this inconsistency by removing the escaping of U+2028 and U+2029. Callers who need to escape the output of Compact should use HTMLEscape which escapes <, >, &, U+2028 and U+2029. Fixes #34070 Fixes #30357 Updates #5836 Change-Id: Icfce7691d2b8b1d9b05ba7b64d2d1e4f3b67871b GitHub-Last-Rev: 38859fe3e2fd586bbd45175c2742f7b123836bf3 GitHub-Pull-Request: golang/go#34804 Reviewed-on: https://go-review.googlesource.com/c/go/+/200217 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-16encoding/json: validate strings when decoding into NumberLucas Bremgartner
Unmarshaling a string into a json.Number should first check that the string is a valid Number. If not, we should fail without decoding it. Fixes #14702 Change-Id: I286178e93df74ad63c0a852c3f3489577072cf47 GitHub-Last-Rev: fe69bb68eed06d056639f440d2daf4bb7c99013b GitHub-Pull-Request: golang/go#34272 Reviewed-on: https://go-review.googlesource.com/c/go/+/195045 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-16encoding/json: make Number with the ,string option marshal with quotesLucas Bremgartner
Add quotes when marshaling a json.Number with the string option set via a struct tag. This ensures that the resulting json can be unmarshaled into the source struct without error. Fixes #34268 Change-Id: Ide167d9dec77019554870b5957b37dc258119d81 GitHub-Last-Rev: dde81b71208be01c253bb87dbb6f81ac6e0785be GitHub-Pull-Request: golang/go#34269 Reviewed-on: https://go-review.googlesource.com/c/go/+/195043 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-11encoding/json: encode nil encoding.TextMarshaler instance as "null"William Poussier
Fixes #34235. Change-Id: Ia3795fd18860530fa6a4b171545f525e784ffdcb GitHub-Last-Rev: 1a319c452857818f7aaf22ef46823b43ca9b2276 GitHub-Pull-Request: golang/go#34238 Reviewed-on: https://go-review.googlesource.com/c/go/+/194642 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-11encoding/json: fix and optimize marshal for quoted stringLucas Bremgartner
Since Go 1.2 every string can be marshaled to JSON without error even if it contains invalid UTF-8 byte sequences. Therefore there is no need to use Marshal again for the only reason of enclosing the string in double quotes. Not using Marshal here also removes the error check as there has not been a way for Marshal to fail anyway. name old time/op new time/op delta Issue34127-4 360ns ± 3% 200ns ± 3% -44.56% (p=0.008 n=5+5) name old alloc/op new alloc/op delta Issue34127-4 56.0B ± 0% 40.0B ± 0% -28.57% (p=0.008 n=5+5) name old allocs/op new allocs/op delta Issue34127-4 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.008 n=5+5) Fixes #34154 Change-Id: Ib60dc11980f9b20d8bef2982de7168943d632263 GitHub-Last-Rev: 9b0ac1d4c5318b6bf9ed7930320f2bd755f9939c GitHub-Pull-Request: golang/go#34127 Reviewed-on: https://go-review.googlesource.com/c/go/+/193604 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-11encoding/base32: increase performance and code reuseSven Taute
Add benchmarks for the Encode/Decode functions operating on []byte and increase decoding performance by removing the calls to strings.Map/bytes.Map and reusing the newline filtering code that is used by NewDecoder. Cut allocations in half for DecodeString. Comparison using the new benchmarks: name old time/op new time/op delta Encode 16.7µs ± 1% 17.0µs ± 2% +2.25% (p=0.000 n=9+9) EncodeToString 21.1µs ± 1% 20.9µs ± 1% -0.96% (p=0.000 n=10+10) Decode 141µs ± 1% 54µs ± 1% -61.51% (p=0.000 n=10+10) DecodeString 81.4µs ± 0% 54.7µs ± 1% -32.79% (p=0.000 n=9+10) name old speed new speed delta Encode 492MB/s ± 1% 481MB/s ± 2% -2.19% (p=0.000 n=9+9) EncodeToString 389MB/s ± 1% 392MB/s ± 1% +0.97% (p=0.000 n=10+10) Decode 93.0MB/s ± 1% 241.6MB/s ± 1% +159.82% (p=0.000 n=10+10) DecodeString 161MB/s ± 0% 240MB/s ± 1% +48.78% (p=0.000 n=9+10) Change-Id: Id53633514a9e14ecd0389d52114b2b8ca64370cb GitHub-Last-Rev: f4be3cf55caf5b89d76d14b7f32422faff39e3c3 GitHub-Pull-Request: golang/go#30376 Reviewed-on: https://go-review.googlesource.com/c/go/+/163598 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-09-10encoding/json: don't reset before returning buffer to poolandig
Reset is already performed when retrieving from pool Change-Id: Ia810dd18d3e55a1565a5ad435a00d1e46724576c GitHub-Last-Rev: d9df74a4aeb86e5d292c9fc33568a3c9a64a967d GitHub-Pull-Request: golang/go#34195 Reviewed-on: https://go-review.googlesource.com/c/go/+/194338 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2019-09-10encoding/json: don't indirect pointers when decoding nullRomain Baugue
The indirect method checked the type of the child when indirecting a pointer. If the current value is a pointer and we are decoding null, we can skip this entirely and return early, avoiding the whole descent. Fixes #31776 Change-Id: Ib8b2a2357572c41f56fceac59b5a858980f3f65e Reviewed-on: https://go-review.googlesource.com/c/go/+/174699 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2019-09-09encoding/asn1: handle ASN1's string type BMPStringsergeilem
This code enables handling of ASN1's string type BMPString, used in some digital signatures. Parsing code taken from golang.org/x/crypto/pkcs12. Change-Id: Ibeae9cf4d8ae7c18f8b5420ad9244a16e117ff6b GitHub-Last-Rev: 694525351411f2ec3982a6bf4ac33be892ce1b12 GitHub-Pull-Request: golang/go#26690 Reviewed-on: https://go-review.googlesource.com/c/go/+/126624 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-09-09src: gofmt -sAinar Garipov
Change-Id: I56d7eeaf777ac30886ee77428ca1ac72b77fbf7d Reviewed-on: https://go-review.googlesource.com/c/go/+/193849 Run-TryBot: Dave Cheney <dave@cheney.net> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-09encoding/json: clarify Unmarshal behavior for map keysEli Bendersky
This is a documentation-only change Fixes #33298 Change-Id: I816058a872b57dc868dff11887214d9de92d9342 Reviewed-on: https://go-review.googlesource.com/c/go/+/188821 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-08all: fix typosAinar Garipov
Use the following (suboptimal) script to obtain a list of possible typos: #!/usr/bin/env sh set -x git ls-files |\ grep -e '\.\(c\|cc\|go\)$' |\ xargs -n 1\ awk\ '/\/\// { gsub(/.*\/\//, ""); print; } /\/\*/, /\*\// { gsub(/.*\/\*/, ""); gsub(/\*\/.*/, ""); }' |\ hunspell -d en_US -l |\ grep '^[[:upper:]]\{0,1\}[[:lower:]]\{1,\}$' |\ grep -v -e '^.\{1,4\}$' -e '^.\{16,\}$' |\ sort -f |\ uniq -c |\ awk '$1 == 1 { print $2; }' Then, go through the results manually and fix the most obvious typos in the non-vendored code. Change-Id: I3cb5830a176850e1a0584b8a40b47bde7b260eae Reviewed-on: https://go-review.googlesource.com/c/go/+/193848 Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-02std: remove unused bits of code all over the placeDaniel Martí
Some were never used, and some haven't been used for years. One exception is net/http's readerAndCloser, which was only used in a test. Move it to a test file. While at it, remove a check in regexp that could never fire; the field is an uint32, so it can never be negative. Change-Id: Ia2200f6afa106bae4034045ea8233b452f38747b Reviewed-on: https://go-review.googlesource.com/c/go/+/192621 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-02encoding/json: fix scanner byte offset on scanEnditchyny
scanEnd is delayed one byte so we decrement the scanner bytes count by 1 to ensure that this value is correct in the next call of Decode. Fixes #32399 Change-Id: I8c8698e7f95bbcf0373aceaa05319819eae9d86f GitHub-Last-Rev: 0ac25d8de23d38c7ac577faddc6983571023f561 GitHub-Pull-Request: golang/go#32598 Reviewed-on: https://go-review.googlesource.com/c/go/+/182117 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-01encoding/json: revert Compact HTML escaping documentationTom Thorogood
This partly reverts CL 173417 as it incorrectly documented that Compact performed HTML escaping and the output was safe to embed inside HTML <script> tags. This has never been true. Although Compact does escape U+2028 and U+2029, it doesn't escape <, > or &. Compact is thus only performing a subset of HTML escaping and it's output is not safe to embed inside HTML <script> tags. A more complete fix would be for Compact to either never perform any HTML escaping, as it was prior to CL 10883045, or to actually perform the same HTML escaping as HTMLEscape. Neither change is likely safe enough for go1.13. Updates #30357 Change-Id: I912f0fe9611097d988048b28228c4a5b985080ba GitHub-Last-Rev: aebababc9233c5705785b225377e80096d4bb8c4 GitHub-Pull-Request: golang/go#33427 Reviewed-on: https://go-review.googlesource.com/c/go/+/188717 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-01encoding/json: fix panic for nil instances of TextMarshaler in map keysWilliam Poussier
This change adds a a check in the encodeWithString.resolve method to ensure that a reflect.Value with kind Ptr is not nil before the type assertion to TextMarshaler. If the value is nil, the method returns a nil error, and the map key encodes to an empty string. Fixes #33675 Change-Id: I0a04cf690ae67006f6a9c5f8cbb4cc99d236bca8 GitHub-Last-Rev: 6c987c90846f854e21814dbfb3a073605ec8a94c GitHub-Pull-Request: golang/go#33700 Reviewed-on: https://go-review.googlesource.com/c/go/+/190697 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2019-08-28encoding/xml: rename fInnerXml to fInnerXMLEddie Scholtz
Per the code review guidelines: "Words in names that are initialisms or acronyms have a consistent case." Change-Id: I347b02d2f48455f2cbbc040191ba197e3e8f23fc Reviewed-on: https://go-review.googlesource.com/c/go/+/191970 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-27encoding/json: remove unnecessary isValidNumber callDaniel Martí
The decoder called this function to check numbers being decoded into a json.Number. However, these can't be quoted as strings, so the tokenizer has already verified they are valid JSON numbers. Verified this by adding a test with such an input. As expected, it produces a syntax error, not the fmt.Errorf - that line could never execute. Since the only remaining non-test caller of isvalidnumber is in encode.go, move the function there. This change should slightly reduce the amount of work when decoding into json.Number, though that isn't very common nor part of any current benchmarks. Change-Id: I67a1723deb3d18d5b542d6dd35f3ae56a43f23eb Reviewed-on: https://go-review.googlesource.com/c/go/+/184817 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-27encoding/json: fix the broken "overwriting of data" testsDaniel Martí
Because TestUnmarshal actually allocates a new value to decode into using ptr's pointer type, any existing data is thrown away. This was harmless in alomst all of the test cases, minus the "overwriting of data" ones added in 2015 in CL 12209. I spotted that nothing covered decoding a JSON array with few elements into a slice which already had many elements. I initially assumed that the code was buggy or that some code could be removed, when in fact there simply wasn't any code covering the edge case. Move those two tests to TestPrefilled, which already served a very similar purpose. Remove the map case, as TestPrefilled already has plenty of prefilled map cases. Moreover, we no longer reset an entire map when decoding, as per the godoc: To unmarshal a JSON object into a map, Unmarshal first establishes a map to use. If the map is nil, Unmarshal allocates a new map. Otherwise Unmarshal reuses the existing map, keeping existing entries. Finally, to ensure that ptr is used correctly in the future, make TestUnmarshal error if it's anything other than a pointer to a zero value. That is, the only correct use should be new(type). Don't rename the ptr field, as that would be extremely noisy and cause unwanted merge conflicts. Change-Id: I41e3ecfeae42d877ac5443a6bd622ac3d6c8120c Reviewed-on: https://go-review.googlesource.com/c/go/+/185738 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-21Revert "encoding/json: avoid work when unquoting strings"Filippo Valsorda
This reverts CL 151157. CL 151157 introduced a crash when decoding into ",string" fields. It came with a moderate speedup, so at this stage of the release cycle let's just revert it, and reapply it in Go 1.14 with the fix in CL 190659. Also applied the test cases from CL 190659. Updates #33728 Change-Id: Ie46e2bc15224b251888580daf6b79d5865f3878e Reviewed-on: https://go-review.googlesource.com/c/go/+/190909 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-08-18encoding/json: fix format string in the Fuzz funcDmitry Vyukov
Currently test build fails with: $ go test -tags=gofuzz encoding/json encoding/json/fuzz.go:36:4: Println call has possible formatting directive %s FAIL encoding/json [build failed] Change-Id: I23aef44a421ed0e7bcf48b74ac5a8c6768a4841b Reviewed-on: https://go-review.googlesource.com/c/go/+/190698 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-31encoding/json: clarify Marshal behavior for string keys of mapsEli Bendersky
This is a documentation-only change. Fixes #28827 Change-Id: Ife9ab997809048784f35872b09905bc209a05eff Reviewed-on: https://go-review.googlesource.com/c/go/+/188417 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-28all: change some function documentation to be more idiomaticDominik Honnef
Change-Id: I932de9bb061a8ba3332ef03207983e8b98d6f1e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/187918 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-06encoding/json: obey SetEscapeHTML in all MarshalJSON casesDaniel Martí
It wasn't obeyed in the case where the MarshalJSON method uses a pointer receiver, and the encoder grabs the address of a value to find that method. addrMarshalerEncoder is the function that does this work, but it ignored opts.escapeHTML. Here's the before and after of the added test case, which was failing before the fix. Now the two cases are correct and consistent. {"NonPtr":"<str>","Ptr":"\u003cstr\u003e"} {"NonPtr":"<str>","Ptr":"<str>"} Fixes #32896. Change-Id: Idc53077ece074973558bd3bb5ad036380db0d02c Reviewed-on: https://go-review.googlesource.com/c/go/+/184757 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Caleb Spare <cespare@gmail.com>
2019-05-22encoding/gob: properly ignore errorsLeon Klingele
Change-Id: I8827cef0f57459384329c50c51795350da0ede4b GitHub-Last-Rev: c9ad9e12b5a0fff47c21a8c299b762b64b8c9c7c GitHub-Pull-Request: golang/go#30010 Reviewed-on: https://go-review.googlesource.com/c/go/+/160434 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-21all: remove PEM-encoded private keys from testsRuss Cox
Gerrit is complaining about pushes that affect these files and forcing people to use -o nokeycheck, which defeats the point of the check. Hide the keys from this kind of scan by marking them explicitly as testing keys. This is a little annoying but better than training everyone who ever edits one of these test files to reflexively override the Gerrit check. The only remaining keys explicitly marked as private instead of testing are in examples, and there's not much to do about those. Hopefully they are not edited as much. Change-Id: I4431592b5266cb39fe6a80b40e742d97da803a0b Reviewed-on: https://go-review.googlesource.com/c/go/+/178178 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09encoding/gob: rename encBuffer.WriteByte to writeByteRuss Cox
Renaming the method makes clear, both to readers and to vet, that this method is not the implementation of io.ByteWriter. Working toward making the tree vet-safe instead of having so many exceptions in cmd/vet/all/whitelist. For #31916. Change-Id: I5b509eb7f0118d5f2d3c6e352ff2849cd5a3071e Reviewed-on: https://go-review.googlesource.com/c/go/+/176110 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-06all: simplify code using "gofmt -s -w"Shulhan
Most changes are removing redundant declaration of type when direct instantiating value of map or slice, e.g. []T{T{}} become []T{{}}. Small changes are removing the high order of subslice if its value is the length of slice itself, e.g. T[:len(T)] become T[:]. The following file is excluded due to incompatibility with go1.4, - src/cmd/compile/internal/gc/ssa.go Change-Id: Id3abb09401795ce1e6da591a89749cba8502fb26 Reviewed-on: https://go-review.googlesource.com/c/go/+/166437 Run-TryBot: Dave Cheney <dave@cheney.net> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-04all: add Unwrap and Is methods to various error typesDamien Neil
Add Unwrap methods to types which wrap an underlying error: "encodinc/csv".ParseError "encoding/json".MarshalerError "net/http".transportReadFromServerError "net".OpError "net".DNSConfigError "net/url".Error "os/exec".Error "signal/internal/pty".PtyError "text/template".ExecError Add os.ErrTemporary. A case could be made for putting this error value in package net, since no exported error types in package os include a Temporary method. However, syscall errors returned from the os package do include this method. Add Is methods to error types with a Timeout or Temporary method, making errors.Is(err, os.Err{Timeout,Temporary}) equivalent to testing the corresponding method: "context".DeadlineExceeded "internal/poll".TimeoutError "net".adrinfoErrno "net".OpError "net".DNSError "net/http".httpError "net/http".tlsHandshakeTimeoutError "net/pipe".timeoutError "net/url".Error Updates #30322 Updates #29934 Change-Id: I409fb20c072ea39116ebfb8c7534d493483870dc Reviewed-on: https://go-review.googlesource.com/c/go/+/170037 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2019-04-30encoding/gob: adding missing fuzz skip to one of the fuzz testsBrad Fitzpatrick
It's slow & often times out randomly on longtest builders. Not useful. Fixes #31517 Change-Id: Icedbb0c94fbe43d04e8b47d5785ac61c5e2d8750 Reviewed-on: https://go-review.googlesource.com/c/go/+/174522 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-30encoding/csv: add a Fuzz functionRomain Baugue
Adds a sample Fuzz test function to package encoding/csv based on https://github.com/dvyukov/go-fuzz-corpus/blob/master/csv/main.go Updates #19109 Updates #31309 Change-Id: Ieb0cb6caa1df72dbb7e29df4bdeed0bfa91187d3 Reviewed-on: https://go-review.googlesource.com/c/go/+/174302 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-30encoding/json: fix Unmarshal hang on recursive pointersLE Manh Cuong
indirect walks down v until it gets to a non-pointer. But it does not handle the case when v is a pointer to itself, like in: var v interface{} v = &v Unmarshal(b, v) So just stop immediately if we see v is a pointer to itself. Fixes #31740 Change-Id: Ie396264119e24d70284cd9bf76dcb2050babb069 Reviewed-on: https://go-review.googlesource.com/c/go/+/174337 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 18:40:28 +0000