aboutsummaryrefslogtreecommitdiff
path: root/src/encoding
AgeCommit message (Collapse)Author
2025-08-11encoding/json: fix Indent trailing whitespace regression in goexperiment.jsonv2Joe Tsai
The Indent function preserves trailing whitespace, while the v1 emulation under v2 implementation accidentally dropped it. There was prior logic that attempted to preserve it, but it did not work correctly since it ran in a defer and accidentally mutated the dst input argument rather than the output argument. Move the logic to the end and avoid a defer. Also, add a test to both v1 and v1in2 to codify this behavior. This only modifies code that is compiled in under goexperiment.jsonv2. Updates #13520 Fixes #74806 Change-Id: I22b1a8da5185eb969e2a8a111b625d3752cfcbe8 Reviewed-on: https://go-review.googlesource.com/c/go/+/692195 Reviewed-by: Sean Liao <sean@liao.dev> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: David Chase <drchase@google.com> Auto-Submit: Sean Liao <sean@liao.dev>
2025-08-11encoding/json/v2: fix UnmarshalDecode regression with EOFJoe Tsai
When EOF is encountered within jsontext.Decoder stream without starting to parse any token, UnmarshalDecode should report EOF, rather than converting it into ErrUnexpectedEOF. This fixes a regression introduced by https://go.dev/cl/689919. This change only affects code compiled under goexperiment.jsonv2. Fixes #74835 Change-Id: I7e8e57ab11b462c422c538503ed8c6b91ead53bd Reviewed-on: https://go-review.googlesource.com/c/go/+/692175 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2025-08-08all: update blog linkjiahua wang
Change-Id: I8631fbc552b85f35b494a8d5a2c0baf68ee66982 Reviewed-on: https://go-review.googlesource.com/c/go/+/690215 Reviewed-by: qiu laidongfeng <2645477756@qq.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Mark Freeman <markfreeman@google.com>
2025-07-25encoding/json: fix truncated Token error regression in goexperiment.jsonv2Joe Tsai
The jsontext.Decoder.ReadToken method reports a non-EOF error, if the token stream is truncated and does not form a valid JSON value. In contrast, the v1 json.Decoder.Token method would report EOF so long as the input was a prefix of some valid JSON value. Modify json.Decoder.Token to preserve historical behavior. This only modifies code that is compiled in under goexperiment.jsonv2. Updates #69449 Fixes #74750 Change-Id: Ifd281c46f118f0e748076013fefc7659f77c56ed Reviewed-on: https://go-review.googlesource.com/c/go/+/689516 Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-07-25encoding/json/jsontext: preserve buffer capacity in Encoder.ResetJoe Tsai
This does the equivalent of CL 681177 for the Encoder. It preserves the internal buffer between resets. Change-Id: I5e9353b6d7755e067d4f9a4d1ea3d8f056253027 Reviewed-on: https://go-review.googlesource.com/c/go/+/690375 Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-07-24encoding/json: fix extra data regression under goexperiment.jsonv2Joe Tsai
When operating under v1 semantics in the v2 implementation, a extra data error should take precedence over any semantic error that could theoretically occur within the value itself. This change only affects code compiled under goexperiment.jsonv2. Fixes #74614 Change-Id: I055a606b053fa66b0c766ae205487b8290109285 Reviewed-on: https://go-review.googlesource.com/c/go/+/689919 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-24encoding/json: reduce error text regressions under goexperiment.jsonv2Joe Tsai
There were minor and unnecessary error text changes when v1 was implemented using v2. Reduce divergences if possible. Of the cases reported in #74713, there are no more differences for: v1: json: cannot unmarshal number into Go value of type chan int v2: json: cannot unmarshal number into Go value of type chan int and v1: json: cannot unmarshal number into Go value of type error v2: json: cannot unmarshal number into Go value of type error However, there is a difference between: v1: json: cannot unmarshal string into Go struct field .F.V of type int v2: json: cannot unmarshal string into Go struct field S.F.V of type int For reasons unclear, the v1 logic was always inconsistent about whether it could properly record the root struct type, while the v1 emulation layer under v2 is always able to. This only modifies code that is compiled in under goexperiment.jsonv2. Fixes #74713 Change-Id: I9e87323b1810130cb929288fdd86aff4be82d5f2 Reviewed-on: https://go-review.googlesource.com/c/go/+/689918 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-07-24encoding/json/jsontext: preserve buffer capacity in Decoder.ResetFilip Petkovski
The Decoder.Reset method is not preserving the internal buffer between resets, causing buffer capacity to be lost and resulting in unnecessary allocations when reusing decoders. This is particularly problematic when decoding many small messages. This commit fixes the Reset method to preserve the internal buffer. It makes sure aliasing is removed if the buffer currently points to an internal byte slice of a bytes.Buffer. It adds a TestDecoderReset test structured into subtests to better validate the different scenarios. Change-Id: Ia685bff47034598224489173bb7f2ffd48e89da5 GitHub-Last-Rev: 462ddc936409ee3da6d75def7d9de59eb3c65f8d GitHub-Pull-Request: golang/go#74120 Reviewed-on: https://go-review.googlesource.com/c/go/+/681177 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2025-07-14encoding/json: decompose legacy optionsJoe Tsai
WARNING: This commit contains breaking changes for those already using GOEXPERIMENT=jsonv2. This decomposes FormatBytesWithLegacySemantics as: * FormatBytesWithLegacySemantics * FormatByteArrayAsArray * ParseBytesWithLooseRFC4648 This decomposes FormatTimeWithLegacySemantics as: * FormatDurationAsNano * ParseTimeWithLooseRFC3339 In particular, it splits out specific behaviors from the option that may need to be specified on a finer-grain level. FormatByteArrayAsArray and FormatDurationAsNano are targeted to just the default representation of a [N]byte or time.Duration type. Both of these are not necessary if the `format` tag is explicitly specified. However, we want to isolate their behavior from other behaviors that used to be part of FormatBytesWithLegacySemantics and FormatTimeWithLegacySemantics. ParseBytesWithLooseRFC4648 and ParseTimeWithLooseRFC3339 are targeted to just historically buggy parsing according to the relevant RFCs, which may need to be enabled by some services for backwards compatibility. While FormatTimeWithLegacySemantics is deleted, we still need FormatBytesWithLegacySemantics to configure highly esoteric aspects of how v1 used to handle byte slices. We rename OmitEmptyWithLegacyDefinition as OmitEmptyWithLegacySemantics to be consistent with other options with the WithLegacySemantics suffix. Updates #71497 Change-Id: Ic660515fb086fe3af237135f195736de99c2bd33 Reviewed-on: https://go-review.googlesource.com/c/go/+/685395 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2025-07-14encoding/json/v2: add security section to docJoe Tsai
This follows up CL 684315 with an expanded section in the v2 doc. Updates #14750 Updates #71845 Change-Id: I1ffa97e030f5f2b709e8142028e3c8e0e38b80ce Reviewed-on: https://go-review.googlesource.com/c/go/+/685195 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2025-07-11encoding/json: remove legacy option to EscapeInvalidUTF8Joe Tsai
In the presence of invalid UTF-8, the AllowInvalidUTF8 option allows such bytes to be present, but silently mangles them using the Unicode replacement character. The v2 default is to emit the replacement character verbatim (which is valid UTF-8 and exactly what it is for). However, the v1 behavior has historically been to emit the escaped form of the replacement character. This behavior was introduced in https://go.dev/cl/11211045 where the documentation says that it is: replacing invalid bytes with the Unicode replacement rune U+FFFD but the implementation actually replaces it with the escaped form of the Unicode replacement rune. Given that the documentation differs from the implementation, the actual behavior is likely an oversight. Given how esoteric of behavior this is, we change the v1in2 behavior to avoid the unnecesary escaping and drop support for EscapeInvalidUTF8. This does not violate the Go compatibility agreement since we do not document what the exact syntactic output is. Also, there has already been prior precedence for changing the output: * [encoding/json: encode \b and \f as '\b' and '\f' in JSON strings](https://go.dev/cl/521675) * [encoding/json: encode \n in strings as "\n", not "\u000A"](https://go.dev/cl/4678046) * [encoding/json: encode \t as \t instead of \u0009](https://go.dev/cl/162340043) * [encoding/json: use standard ES6 formatting for numbers during marshal](https://go.dev/cl/30371) Fixes #74551 Change-Id: Ib59a873c44713d302f1f6ab103ffba2520d63276 Reviewed-on: https://go-review.googlesource.com/c/go/+/687116 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2025-07-11encoding/json/v2: report wrapped io.ErrUnexpectedEOFJoe Tsai
In the event that the input is just JSON whitespace, the underlying jsontext.Decoder treats this as an empty stream and reports io.EOF. The logic in unmarshalFull simply casted io.EOF as io.ErrUnexpectedEOF, which is inconsistent with how all other io.ErrUnexpectedEOF are reported, which are wrapped within a jsontext.SyntacticError. Do the same thing for consistency. We add a v1 test (without goexperiment.jsonv2) to verify that the behavior is identical to how v1 has always behaved. We add a v1in2 test (with goexperiment.jsonv2) to verify that the v1in2 behavior correctly replicates historical v1 behavior. We also fix a faulty check in v1 Decoder.Decode, where it tried to detect errUnexpectedEnd and return an unwrapped io.ErrUnexpectedEOF error. This is the exact semantic that v1 has always done in streaming Decoder.Decode (but not non-streaming Unmarshal). There is a prior bug reported in #25956 about this inconsistency, but we aim to preserve historical v1 behavior to reduce the probability of churn when v1 is re-implemented in terms of v2. Fixes #74548 Change-Id: Ibca52c3699ff3c09141e081c85f853781a86ec8e Reviewed-on: https://go-review.googlesource.com/c/go/+/687115 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Carlos Amedee <carlos@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
2025-06-30encoding/json/jsontext: use bytes.Buffer.AvailableBufferJoe Tsai
This logic was added in October, 2021: https://github.com/go-json-experiment/json/commit/0b3bd4e1ed96587be346b7f964d6bb3fcfed65f4 before the introduction of bytes.Buffer.AvailableBuffer in March, 2023. https://go.dev/cl/474635 Updates #71845 Change-Id: I96800e1ba8fce15cc78316779db4ddcd4fe1d510 Reviewed-on: https://go-review.googlesource.com/c/go/+/685136 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-06-30encoding/json/v2: avoid escaping jsonopts.StructJoe Tsai
The jsonopts.Struct.join method unfortunately escapes the receiver because it is passed to JoinUnknownOption, which is a dynamically implemented function. This affects jsontext.Encoder.reset and jsontext.Decoder.reset, which relied on a local jsonopts.Struct to temporarily store prior options such that it would have to be heap allocated. Adjust the signature of JoinUnknownOption to avoid pointers so that nothing escape. This is a regression from https://github.com/go-json-experiment/json/pull/163 Performance: name old time/op new time/op delta Marshal/Bool-32 72.1ns ± 2% 51.3ns ± 1% -28.77% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Marshal/Bool-32 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10) Updates #71845 Change-Id: Ife500d82d3d2beb13652553a4ffdf882c136f5a0 Reviewed-on: https://go-review.googlesource.com/c/go/+/685135 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-06-27encoding/json: add security section to docRoland Shoemaker
Add a section to the package doc which details the security considerations of using encoding/json, in particular with respect to parser misalignment issues. Additionally, clarify previously ambiguous statement in the Unmarshal doc about how case is used when matching keys in objects, and add a note about how duplicate keys are handled. Fixes #14750 Change-Id: I66f9b845efd98c86a684d7333b3aa8a456564922 Reviewed-on: https://go-review.googlesource.com/c/go/+/684315 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Auto-Submit: Roland Shoemaker <roland@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2025-06-27encoding/json: fix typo in hotlink for jsontext.PreserveRawStringsJoe Tsai
Updates #71845 Change-Id: Ie099e7ac77293696fd9e69559487e27f4b70ab3f Reviewed-on: https://go-review.googlesource.com/c/go/+/684416 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-06-25encoding/json/jsontext: remove Encoder.UnusedBufferJoe Tsai
WARNING: This commit contains a breaking change. This is permissible since jsontext is experimental and not subject to the Go 1 compatibility agreement. Existing callers of UnusedBuffer should use AvailableBuffer instead. Updates #71497 Change-Id: Ib080caf306d545a8fb038e57f0817b18dd0f91cf Reviewed-on: https://go-review.googlesource.com/c/go/+/683897 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2025-06-25encoding/json/jsontext: rename Encoder.UnusedBuffer as Encoder.AvailableBufferJoe Tsai
This follows the precedent set by: bufio.Writer.AvailableBuffer bytes.Buffer.AvailableBuffer both with methods that return a zero-length buffer that is intended to only be used with a following Write call. This keeps the older UnusedBuffer method around so that at least one commit that has both methods for migration purposes. Updates #71497 Change-Id: I3815f593e09f645280ae5ad9cbdd63a6c147123b Reviewed-on: https://go-review.googlesource.com/c/go/+/683896 Reviewed-by: Damien Neil <dneil@google.com> TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-06-24encoding/json: use zstd compressed testdataJoe Tsai
There is a non-public zstd decoder in the stdlib (CL 473356) and also zstd compressed testdata already present. Delete testdata/code.json.gz and instead use internal/jsontest/testdata/golang_source.json.zst, which has exactly the same content: $ cat internal/jsontest/testdata/golang_source.json.zst | zstd -d | sha1sum 3f70b6fd429f4aba3e8e1c3e5a294c8f2e219a6e - $ cat testdata/code.json.gz | zstd -d | sha1sum 3f70b6fd429f4aba3e8e1c3e5a294c8f2e219a6e - This will reduce the size of the final Go release by 118KB. Updates #71845 Change-Id: I6da2df27bd260befc0a44c6bc0255365be0a5b0f Reviewed-on: https://go-review.googlesource.com/c/go/+/525516 Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Bypass: Damien Neil <dneil@google.com>
2025-06-24encoding/json/v2: support ISO 8601 durationsJoe Tsai
Based on the discussion in #71631, it is hotly contested whether the default JSON representation for a Go time.Duration should be the time.Duration.String format or a particular profile of ISO 8601. Regardless of the default, it seems clear that we should at least support ISO 8601 if specified via a format flag. Note that this CL does not alter the default representation. Unfortunately, ISO 8601 is a large and evolving standard with many optional extensions and optional restrictions. Thus, the term "ISO 8601 duration" unfortunately does not resolve to a particular grammar, nor one that is stable. However, there is precedence that we can follow in this matter. JSON finds its heritage in JavaScript and JavaScript is adding a Temporal.Duration type whose default JSON representation is ISO 8601. There is a well-specified grammar for their particular profile of ISO 8601, which is documented at: https://tc39.es/proposal-temporal/#prod-Duration This particular CL adds support for ISO 8601 according to the exact same grammar that JavaScript uses. While Temporal.Duration is technically still a proposal, it is already in stage 3 of the TC39 proposal process (i.e., "no changes to the proposal are expected" and "has been recommended for implementation") and therefore close to final adoption. One major concern with ISO 8601 is that it supports nominal date units like years, months, weeks, and days that do not have an accurate meaning without being anchored to a particular point in time and place on Earth. Fortunately, JavaScript (by default) avoids producing Temporal.Duration values with nominal units unless arithmetic in JavaScript explicitly sets a largestUnits value that is larger than "hours". In the Go implementation, we support syntactically parsing the full ISO 8601 grammar (according to JavaScript), but semantically report an error if nominal units are present. This ensures that ISO 8601 durations remain accurate so long as they only use the accurate units of hours, minutes, or seconds. Updates #71631 Change-Id: I983593662f2150461ebc486a5acfeb72f0286939 Reviewed-on: https://go-review.googlesource.com/c/go/+/682403 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-06-24encoding/json/v2: reject unquoted dash as a JSON field nameJoe Tsai
In this blog: https://blog.trailofbits.com/2025/06/17/unexpected-security-footguns-in-gos-parsers/ the concern was raised that whenever "-" is combined with other options, the "-" is intepreted as as a name, rather than an ignored field, which may go contrary to user expectation. Static analysis demonstrates that there are ~2k instances of `json:"-,omitempty" in the wild, where almost all of them intended for the field to be ignored. To prevent this footgun, reject any tags that has "-," as a prefix and warn the user to choose one of the reasonable alternatives. The documentation of json/v2 already suggests `json:"'-'"` as the recommended way to explicitly specify dash as the name. See Example_fieldNames for example usages of the single-quoted literal. Update the v1 json documentation to suggest the same thing. Updates #71497 Change-Id: I7687b6eecdf82a5d894d057c78a4a90af4f5a6e4 Reviewed-on: https://go-review.googlesource.com/c/go/+/683175 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-06-23encoding/json/v2: report error on time.Duration without explicit formatJoe Tsai
The default representation of a time.Duration is still undecided. In order to keep the future open, report an error on a time.Duration without an explicit format flag provided. Updates #71631 Change-Id: I08248404ff6551723851417c8188a13f53c61937 Reviewed-on: https://go-review.googlesource.com/c/go/+/682455 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
2025-06-23encoding/json/jsontext: consistently use JSON terminologyJoe Tsai
RFC 8259, section 2 uses the term "begin-array" amd "begin-object" rather than "start array" or "start object". Be consistent in our documentation. Change-Id: I172eb354c5e64b84c74bd662b1e581424e719a32 Reviewed-on: https://go-review.googlesource.com/c/go/+/683155 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2025-06-17encoding/json/jsontext: fix spelling errorKevin Burke
Change-Id: Ic1f385afbe35addba8b3c439ccb64c56b1d300c9 Reviewed-on: https://go-review.googlesource.com/c/go/+/681795 Reviewed-by: Sean Liao <sean@liao.dev> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2025-06-11encoding/json/jsontext, encoding/json/v2: document experimental natureDamien Neil
Change-Id: I7b2c391749e0113e006f37b2ac1ebfe3ee0a4e0e Reviewed-on: https://go-review.googlesource.com/c/go/+/680715 TryBot-Bypass: Damien Neil <dneil@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Auto-Submit: Damien Neil <dneil@google.com>
2025-05-20encoding/json: avoid supurious synctest deadlock detectionDamien Neil
Use a sync.OnceValue rather than a sync.WaitGroup to coordinate access to encoderCache entries. The OnceValue better expresses the intent of the code (we want to initialize the cache entry only once). However, the motivation for this change is to avoid testing/synctest incorrectly reporting a deadlock when multiple bubbles call Marshal at the same time. Goroutines blocked on WaitGroup.Wait are "durably blocked", causing confusion when a goroutine in one bubble Waits for a goroutine in a different bubble. Goroutines blocked on OnceValue are not durably blocked, avoiding the problem. Fixes #73733 For #67434 Change-Id: I81cddda80af67cf5c280fd4327620bc37e7a6fe6 Reviewed-on: https://go-review.googlesource.com/c/go/+/673335 Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-04-18encoding/json: add json/v2 with GOEXPERIMENT=jsonv2 guardDamien Neil
This imports the proposed new v2 JSON API implemented in github.com/go-json-experiment/json as of commit d3c622f1b874954c355e60c8e6b6baa5f60d2fed. When GOEXPERIMENT=jsonv2 is set, the encoding/json/v2 and encoding/jsontext packages are visible, the encoding/json package is implemented in terms of encoding/json/v2, and the encoding/json package include various additional APIs. (See #71497 for details.) When GOEXPERIMENT=jsonv2 is not set, the new API is not present and the encoding/json package is unchanged. The experimental API is not bound by the Go compatibility promise and is expected to evolve as updates are made to the json/v2 proposal. The contents of encoding/json/internal/jsontest/testdata are compressed with zstd v1.5.7 with the -19 option. Fixes #71845 For #71497 Change-Id: Ib8c94e5f0586b6aaa22833190b41cf6ef59f4f01 Reviewed-on: https://go-review.googlesource.com/c/go/+/665796 Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-04-15encoding/json: correct method comment to reflect actual argumentAlex S
Change-Id: I0e9040ee5b84463f0391e8e4ae1b64a036867913 GitHub-Last-Rev: 859c82a254f49fa4b5376c0e8fff6f62f5131f62 GitHub-Pull-Request: golang/go#73123 Reviewed-on: https://go-review.googlesource.com/c/go/+/662015 Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sean Liao <sean@liao.dev>
2025-03-14encoding/asn1: make sure implicit fields roundtripRoland Shoemaker
Make sure Marshal and Unmarshal support the same field tags for implicit encoding choices. In particular this adds support for Unmarshalling implicitly tagged GeneralizedTime fields. Also add tests and update the docs. Fixes #72078 Change-Id: I21465ee4bcd73a7db0d0c36b2df53cabfc480185 Reviewed-on: https://go-review.googlesource.com/c/go/+/654275 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-03-13crypto/x509,ecoding/asn1: better handling of weird encodingsRoland Shoemaker
For various cursed reasons we need to support the BMPString and T61String ASN.1 string encodings. These types use the defunct UCS-2 and T.61 character encodings respectively. This change rejects some characters when decoding BMPStrings which are not valid in UCS-2, and properly parses T61Strings instead of treating them as plain UTF-8. While still not perfect, this matches the behavior of most other implementations, particularly BoringSSL. Ideally we'd just remove support for these ASN.1 types (particularly in crypto/x509, where we don't actually expose any API), but doing so is likely to break some deploy certificates which unfortunately still use these types in DNs, despite them being deprecated since 1999/2002. Fixes #71862 Change-Id: Ib8f392656a35171e48eaf71a200be6d7605b2f02 Reviewed-on: https://go-review.googlesource.com/c/go/+/651275 Reviewed-by: Daniel McCarney <daniel@binaryparadox.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
2025-03-05encoding/pem: clarify Decode only works on linesSean Liao
Fixes #53524 Change-Id: I929ee3c055c3ca564cd6cc374124f493aea2fbf6 Reviewed-on: https://go-review.googlesource.com/c/go/+/421636 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2025-02-25encoding/json: use builtin min function in appendStringJes Cok
To make code a bit simpler. Change-Id: I59fca1d5760e304abd53873ecf9ca8b2903e02e8 GitHub-Last-Rev: 1369df6da16121c342a4e678efe3e5b082485b74 GitHub-Pull-Request: golang/go#71873 Reviewed-on: https://go-review.googlesource.com/c/go/+/651355 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-02-06encoding/base64: use internal/byteorderIan Lance Taylor
This permits us to remove the dependency on reflect. Change-Id: I60b1e9fd713f340bfd5eec2edfa58fc724a8e2d8 Reviewed-on: https://go-review.googlesource.com/c/go/+/641936 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Bypass: Ian Lance Taylor <iant@golang.org> Auto-Submit: Ian Lance Taylor <iant@golang.org>
2025-02-03all: replace reflect.Value.Type.Kind with reflect.Value.KindSean Liao
Fixes #46107 Change-Id: I170f3cacda652752cd740e04b565a616a0e43fd1 Reviewed-on: https://go-review.googlesource.com/c/go/+/632635 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Commit-Queue: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2025-02-03all: run gofmtIan Lance Taylor
Change-Id: I0af1903ed1e4f2bf4ea273847b024520c577ef6d Reviewed-on: https://go-review.googlesource.com/c/go/+/642496 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org>
2025-01-14encoding/json: cleanup testsJoe Tsai
Perform minor cleanups in tests to improve printout of diffs and/or follow modern coding style. This reduces the amount of diffs between v1 and the v2 prototype. Change-Id: I019bb9642e2135f2fa3eac6abfa6df91c397aa82 Reviewed-on: https://go-review.googlesource.com/c/go/+/642257 Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-01-13encoding/json: add cases to TestUnmarshal for fatal syntactic errorsJoe Tsai
The presence of a syntax error in the input immediately unmarshaling before unmarshaling into the underlying value. Otherwise, semantic errors are generally lazily reported and allow unmarshaling to continue on. Change-Id: Icf1cfc684e415312d9c8bf739c396ede15299d7d Reviewed-on: https://go-review.googlesource.com/c/go/+/642295 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-01-13encoding/json: always check resulting Go value for unmarshalingJoe Tsai
Even if an error occurs during unmarshal, check the resulting Go value. The documented API specifies no guarantees on how much of a Go value will be populated when an error occurs and the "json" package is technically not bounded by the Go compatibility agreement to ensure this behavior never changes. However, there is still value in running checks for what exactly what is partially mutated in the event of an error even if this is not guaranteed behavior. Change-Id: I6e923a31f77768a14c4adfb0d37dbeee5807a4a2 Reviewed-on: https://go-review.googlesource.com/c/go/+/642275 Auto-Submit: Joseph Tsai <joetsai@digital-static.net> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-01-08encoding/json: improve fidelity of TestUnmarshal for NumbersJoe Tsai
In particular, cover the behavior of unmarshaling a JSON string into a Number type regardless of whether the `string` option is specified or not. Change-Id: Ibc55f16860442240bcfeea1fd51aaa76f7e50f67 Reviewed-on: https://go-review.googlesource.com/c/go/+/641416 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-12-30encoding/binary: add documentation for endian methodswillboland
While the comments are on an unexported type, gopls correctly shows them when using the exported vars LittleEndian and BigEndian. Fixes #68083 Change-Id: I53668c3140ad00f7b58437be74e6df773e2916f1 GitHub-Last-Rev: cef717123cdc632b59683f4ce55aee6c286ca4c7 GitHub-Pull-Request: golang/go#71058 Reviewed-on: https://go-review.googlesource.com/c/go/+/638936 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
2024-12-30encoding/json: remove suggestion on Unmarshaler with JSON nullJoe Tsai
It is not true that Unmarshal always treats a JSON null as being equivalent to a no-op. For bools, ints, uints, floats, strings, arrays, and structs, it treats a JSON null as a no-op. However, for []byte, slice, map, pointer, or interface, it zeros the underlying value. Remove this suggestion as the actual behavior is inconsistent. Note that the proposed behavior in v2 Unmarshal is to consistently zero out the underlying value. Change-Id: I02cef0bf7919f25cfd0aceb04486d37498761181 Reviewed-on: https://go-review.googlesource.com/c/go/+/638416 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
2024-12-27encoding/json: expand and modernize TestInterfaceSetJoe Tsai
Add more test cases to cover a wider range of edge cases. Use a generic addr function to take the address of a value. Even though redudant, explicitly include a cast to the top-level Go type so that it is more readable what the expected input and ouput types are. Change-Id: I3ef68df6f1beb903ae237cd49f3dcb91e5270fe7 Reviewed-on: https://go-review.googlesource.com/c/go/+/638256 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-12-06encoding/base64, encoding/base32: add doc details to DecodeString and ↵Dmytro Yeroshkin
AppendDecode Currently only Encoding.Decode has the information that `\r` and `\n` are ignored. However, this also applies to the other decoding methods. Since this is not intuitive behavior, we should add this information to the other impacted methods. Change-Id: I6f71fe1f4280fe75f2694a3cc1b759652eb8b8c1 Reviewed-on: https://go-review.googlesource.com/c/go/+/634215 Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Chris Ingram <chrisingram@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Truc Le <trucleduc@google.com>
2024-11-21all: fix some function names and typos in commentcuishuang
Change-Id: I07e7c8eaa5bd4bac0d576b2f2f4cd3f81b0b77a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/630055 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Commit-Queue: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-11-19encoding/json: check exact structure of local error types in testsJoe Tsai
During the development of error wrapping (#29934), the tests were modified to stop using reflect.DeepEqual since the prototype for error wrapping at the time included frame information of where the error was created. However, that change diminished the fidelity of the test so that it is no longer as strict, which affects the endeavor to implement v1 in terms of the v2 prototype. For locally declared error types, use reflect.DeepEqual to check that the exact structure of the error value matches. Change-Id: I443d418533866ab8d533bca3785fdc741e2c140e Reviewed-on: https://go-review.googlesource.com/c/go/+/629517 Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-08encoding/json, text/template: use reflect.Value.Equal instead of ==Emmanuel T Odeke
This change applies a fix for a reflect.Value incorrect comparison using "==" or reflect.DeepEqual. This change is a precursor to the change that'll bring in the static analyzer "reflectvaluecompare", by ensuring that all tests pass beforehand. Updates #43993 Change-Id: I6c47eb0a1de6353ac7495cb8cb49b318b7ebba56 Reviewed-on: https://go-review.googlesource.com/c/go/+/626116 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2024-10-28all: skip and fix various tests with -asan and -msanMichael Anthony Knyszek
First, skip all the allocation count tests. In some cases this aligns with existing skips for -race, but in others we've got new issues. These are debug modes, so some performance loss is expected, and this is clearly no worse than today where the tests fail. Next, skip internal linking and static linking tests for msan and asan. With asan we get an explicit failure that neither are supported by the C and/or Go compilers. With msan, we only get the Go compiler telling us internal linking is unavailable. With static linking, we segfault instead. Filed #70080 to track that. Next, skip some malloc tests with asan that don't quite work because of the redzone. This is because of some sizeclass assumptions that get broken with the redzone and the fact that the tiny allocator is effectively disabled (again, due to the redzone). Next, skip some runtime/pprof tests with asan, because of extra allocations. Next, skip some malloc tests with asan that also fail because of extra allocations. Next, fix up memstats accounting for arenas when asan is enabled. There is a bug where more is added to the stats than subtracted. This also simplifies the accounting a little. Next, skip race tests with msan or asan enabled; they're mutually incompatible. Fixes #70054. Fixes #64256. Fixes #64257. For #70079. For #70080. Change-Id: I99c02a0b9d621e44f1f918b307aa4a4944c3ec60 Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15 Reviewed-on: https://go-review.googlesource.com/c/go/+/622855 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
2024-10-23encoding/json: clarify omitempty option for {array,slice,map,string}Jes Cok
This CL is inspired by: https://github.com/golang/go/issues/29310#issuecomment-758768325 When I read omitempty option in encoding/xml package, I find it's a bit different than encoding/json package. I think it's more precise to say: "any array, slice, map, or string of length zero." Update #29310 Change-Id: Ia77167c3155411640224b349d4b34d0bb91ee11e GitHub-Last-Rev: a4cf00dcc75067bd259bc600f288c9de4b5393cf GitHub-Pull-Request: golang/go#69984 Reviewed-on: https://go-review.googlesource.com/c/go/+/621835 Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Michael Pratt <mpratt@google.com>
2024-10-21encoding/xml: document that embedded interfaces look like non-embedded onesSean Liao
Fixes #69941 Change-Id: Icc3c88d57c14fa9ca203c693d67f144686fed8cc Reviewed-on: https://go-review.googlesource.com/c/go/+/621076 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-10-08encoding/asn1: use slices.Equal in ObjectIdentifier.EqualTobias Klauser
Change-Id: I5efe3b9dcee85dfa34b6072c6f85108b6fc7cf99 Reviewed-on: https://go-review.googlesource.com/c/go/+/618515 Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 07:32:09 +0000