go/src/encoding/binary/binary_test.go, branch main

all: skip and fix various tests with -asan and -msan

2024-10-28T21:04:51Z

First, skip all the allocation count tests. In some cases this aligns with existing skips for -race, but in others we've got new issues. These are debug modes, so some performance loss is expected, and this is clearly no worse than today where the tests fail. Next, skip internal linking and static linking tests for msan and asan. With asan we get an explicit failure that neither are supported by the C and/or Go compilers. With msan, we only get the Go compiler telling us internal linking is unavailable. With static linking, we segfault instead. Filed #70080 to track that. Next, skip some malloc tests with asan that don't quite work because of the redzone. This is because of some sizeclass assumptions that get broken with the redzone and the fact that the tiny allocator is effectively disabled (again, due to the redzone). Next, skip some runtime/pprof tests with asan, because of extra allocations. Next, skip some malloc tests with asan that also fail because of extra allocations. Next, fix up memstats accounting for arenas when asan is enabled. There is a bug where more is added to the stats than subtracted. This also simplifies the accounting a little. Next, skip race tests with msan or asan enabled; they're mutually incompatible. Fixes #70054. Fixes #64256. Fixes #64257. For #70079. For #70080. Change-Id: I99c02a0b9d621e44f1f918b307aa4a4944c3ec60 Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15 Reviewed-on: https://go-review.googlesource.com/c/go/+/622855 Reviewed-by: Cherry Mui TryBot-Bypass: Michael Knyszek

encoding/binary: add missing test helper calls

2024-07-23T19:17:29Z

Various functions in binary_test.go were missing some t.Helper() calls, so this adds them in. Change-Id: I0e5894e3c3cf72627203694a32adca51fe1563f5 GitHub-Last-Rev: b28c9255b3b5db3cf18b5406a4e13613d10b8da5 GitHub-Pull-Request: golang/go#63897 Reviewed-on: https://go-review.googlesource.com/c/go/+/539135 Reviewed-by: Dmitri Shuralyov Reviewed-by: Keith Randall LUCI-TryBot-Result: Go LUCI Reviewed-by: Keith Randall Reviewed-by: qiu laidongfeng2 <2645477756@qq.com> Auto-Submit: Keith Randall

encoding/binary: speed up Size

2024-05-20T19:16:18Z

Size() is currently not called from the fast path, since the package handles the buffer sizing for Read and Write internally. This will change when adding Append() because callers can use Size to avoid allocations when writing into bytes.Buffer via AvailableBuffer for example. Add a fast path for simple types and extend the existing struct size cache to arrays of structs. Change-Id: I3af16a2b6c9e2dbe6166a2f8c96bcd2e936719e2 Reviewed-on: https://go-review.googlesource.com/c/go/+/584358 Reviewed-by: Austin Clements Reviewed-by: Keith Randall LUCI-TryBot-Result: Go LUCI Auto-Submit: Austin Clements

encoding/binary: add Append, Encode and Decode

2024-05-20T18:58:26Z

Add a function which appends the binary representation of a value to the end of a slice. This allows users to encode values with zero allocations. Also add Encode and Decode functions which mimic unicode/utf8. goos: darwin goarch: arm64 pkg: encoding/binary cpu: Apple M1 Pro │ base.txt │ append.txt │ │ sec/op │ sec/op vs base │ ReadSlice1000Int32s-10 2.690µ ± 0% 2.532µ ± 3% -5.86% (p=0.002 n=6) ReadStruct-10 205.8n ± 0% 201.4n ± 1% -2.14% (p=0.002 n=6) WriteStruct-10 159.1n ± 0% 153.5n ± 0% -3.55% (p=0.002 n=6) WriteSlice1000Structs-10 129.8µ ± 0% 124.2µ ± 0% -4.34% (p=0.002 n=6) ReadSlice1000Structs-10 161.7µ ± 0% 160.3µ ± 0% -0.89% (p=0.002 n=6) ReadInts-10 156.8n ± 0% 161.6n ± 0% +3.09% (p=0.002 n=6) WriteInts-10 134.5n ± 0% 139.5n ± 2% +3.72% (p=0.002 n=6) WriteSlice1000Int32s-10 2.691µ ± 16% 2.551µ ± 4% -5.20% (p=0.002 n=6) PutUint16-10 0.6448n ± 4% 0.6212n ± 1% ~ (p=0.093 n=6) AppendUint16-10 1.414n ± 0% 1.424n ± 1% ~ (p=0.115 n=6) PutUint32-10 0.6210n ± 0% 0.6211n ± 0% ~ (p=0.833 n=6) AppendUint32-10 1.414n ± 0% 1.426n ± 1% +0.85% (p=0.017 n=6) PutUint64-10 0.6210n ± 0% 0.6394n ± 1% +2.95% (p=0.002 n=6) AppendUint64-10 1.414n ± 0% 1.427n ± 2% ~ (p=0.052 n=6) LittleEndianPutUint16-10 0.6239n ± 0% 0.6271n ± 1% ~ (p=0.063 n=6) LittleEndianAppendUint16-10 1.421n ± 0% 1.432n ± 1% +0.81% (p=0.002 n=6) LittleEndianPutUint32-10 0.6240n ± 0% 0.6240n ± 0% ~ (p=0.766 n=6) LittleEndianAppendUint32-10 1.422n ± 1% 1.425n ± 0% ~ (p=0.673 n=6) LittleEndianPutUint64-10 0.6242n ± 0% 0.6238n ± 0% -0.08% (p=0.030 n=6) LittleEndianAppendUint64-10 1.420n ± 0% 1.449n ± 1% +2.04% (p=0.002 n=6) ReadFloats-10 39.36n ± 0% 42.54n ± 1% +8.08% (p=0.002 n=6) WriteFloats-10 33.65n ± 0% 35.27n ± 1% +4.80% (p=0.002 n=6) ReadSlice1000Float32s-10 2.656µ ± 0% 2.526µ ± 1% -4.91% (p=0.002 n=6) WriteSlice1000Float32s-10 2.765µ ± 0% 2.857µ ± 3% +3.31% (p=0.002 n=6) ReadSlice1000Uint8s-10 129.1n ± 1% 130.4n ± 1% ~ (p=0.126 n=6) WriteSlice1000Uint8s-10 144.90n ± 3% 18.67n ± 2% -87.12% (p=0.002 n=6) PutUvarint32-10 12.11n ± 0% 12.12n ± 0% ~ (p=0.675 n=6) PutUvarint64-10 30.82n ± 0% 30.79n ± 1% ~ (p=0.658 n=6) AppendStruct-10 107.8n ± 0% AppendSlice1000Structs-10 119.0µ ± 0% AppendInts-10 55.29n ± 0% AppendSlice1000Int32s-10 2.211µ ± 1% geomean 33.07n 48.18n -7.03% Fixes #60023 Change-Id: Ife3f217b11d5f3eaa5a53fe8a7e877552f751f94 Reviewed-on: https://go-review.googlesource.com/c/go/+/579157 Reviewed-by: Keith Randall Auto-Submit: Austin Clements Reviewed-by: Ingo Oeser Reviewed-by: Austin Clements LUCI-TryBot-Result: Go LUCI

encoding/binary: cache struct sizes to speed up Read and Write for slice of structs.

2024-03-13T18:32:53Z

A lot of allocations happen in dataSize due to reflection. Cache the result of the function when encoding a slice of structs similar to what is done for struct types so that subsequent calls to dataSize can avoid allocations. │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ WriteSlice1000Structs-2 846.7µ ± 4% 856.4µ ± 3% ~ (p=0.602 n=20) │ old.txt │ new.txt │ │ B/s │ B/s vs base │ WriteSlice1000Structs-2 84.48Mi ± 4% 83.52Mi ± 3% ~ (p=0.602 n=20) │ old.txt │ new.txt │ │ B/op │ B/op vs base │ WriteSlice1000Structs-2 80.18Ki ± 0% 80.06Ki ± 0% -0.15% (p=0.000 n=20) │ old.txt │ new.txt │ │ allocs/op │ allocs/op vs base │ WriteSlice1000Structs-2 16.000 ± 0% 1.000 ± 0% -93.75% (p=0.000 n=2 │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ ReadSlice1000Structs-2 847.4µ ± 4% 821.1µ ± 3% -3.10% (p=0.012 n=20) │ old.txt │ new.txt │ │ B/s │ B/s vs base │ ReadSlice1000Structs-2 84.40Mi ± 4% 87.11Mi ± 3% +3.20% (p=0.012 n=20) │ old.txt │ new.txt │ │ B/op │ B/op vs base │ ReadSlice1000Structs-2 80.12Ki ± 0% 80.00Ki ± 0% -0.15% (p=0.000 n=20) │ old.txt │ new.txt │ │ allocs/op │ allocs/op vs base │ ReadSlice1000Structs-2 16.000 ± 0% 1.000 ± 0% -93.75% (p=0.000 n=20) Fixes #66253 Change-Id: I8227e61306db1fe103489ea4fee2429247c3debc Reviewed-on: https://go-review.googlesource.com/c/go/+/570855 LUCI-TryBot-Result: Go LUCI Reviewed-by: Ian Lance Taylor Reviewed-by: Cherry Mui Auto-Submit: Ian Lance Taylor

encoding/binary: on invalid type return -1 from Size

2023-06-20T18:28:44Z

Size is defined as returning -1 if the type is not fixed-size. Before this CL cases like Size((*[]int)(nil)) would crash. Fixes #60892 Change-Id: Iee8e20a0aee24b542b78cb4160c3b2c5a3eb02c2 Reviewed-on: https://go-review.googlesource.com/c/go/+/504575 Auto-Submit: Ian Lance Taylor Run-TryBot: Ian Lance Taylor Reviewed-by: Robert Griesemer TryBot-Result: Gopher Robot Reviewed-by: Bryan Mills

encoding/binary: add word size to the error message of the failed constraint

2023-04-04T22:12:53Z

Test added. Fixes #22860 Change-Id: I08304834a2b7b10b4ac729bf36761692eb4731da Reviewed-on: https://go-review.googlesource.com/c/go/+/113075 Reviewed-by: Emmanuel Odeke Reviewed-by: Ian Lance Taylor Run-TryBot: Ian Lance Taylor Reviewed-by: Michael Knyszek Auto-Submit: Ian Lance Taylor TryBot-Result: Gopher Robot

encoding/binary: add var NativeEndian

2023-01-27T18:17:20Z

Updates #57237 Change-Id: I149c8b7eeac91b87b5810250f96d48ca87135807 Reviewed-on: https://go-review.googlesource.com/c/go/+/463218 Reviewed-by: Ian Lance Taylor Reviewed-by: Dmitri Shuralyov TryBot-Result: Gopher Robot Run-TryBot: Ian Lance Taylor Run-TryBot: xie cui <523516579@qq.com> Auto-Submit: Ian Lance Taylor

encoding/binary: add AppendByteOrder

2022-03-02T18:21:38Z

AppendByteOrder specifies new methods for LittleEndian and BigEndian for appending an unsigned integer to a byte slice. The performance of AppendXXX methods are slower than PutXXX methods since the former needs to do a few slice operations, while the latter is essentially a single integer store. In practice, existing usages of PutXXX performed slicing operations around the call such that this cost was present, regardless. name time/op PutUint16-24 0.48ns ± 2% AppendUint16-24 1.54ns ± 1% PutUint32-24 0.46ns ± 2% AppendUint32-24 0.89ns ± 1% PutUint64-24 0.46ns ± 2% AppendUint64-24 0.89ns ± 1% LittleEndianPutUint16-24 0.47ns ± 2% LittleEndianAppendUint16-24 1.54ns ± 1% LittleEndianPutUint32-24 0.45ns ± 3% LittleEndianAppendUint32-24 0.92ns ± 2% LittleEndianPutUint64-24 0.46ns ± 3% LittleEndianAppendUint64-24 0.95ns ± 4% Fixes #50601 Change-Id: I33d2bbc93a3ce01a9269feac33a2432bc1166ead Reviewed-on: https://go-review.googlesource.com/c/go/+/386017 Trust: Joseph Tsai Trust: Josh Bleecher Snyder Reviewed-by: Josh Bleecher Snyder Run-TryBot: Joseph Tsai TryBot-Result: Gopher Robot

all: gofmt -w -r 'interface{} -> any' src

2021-12-13T18:45:54Z

And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox Run-TryBot: Russ Cox Reviewed-by: Robert Griesemer TryBot-Result: Gopher Robot