aboutsummaryrefslogtreecommitdiff
path: root/src/compress/flate
AgeCommit message (Collapse)Author
2025-11-25compress/flate: move big non-pointer arrays to end of compressorIan Lance Taylor
The compressor type is fairly large: 656616 bytes on amd64. Before this patch, it had fields of slice and interface type near the end of the struct. As those types always contain pointers, the ptrBytes value in the type descriptor was quite large. That forces the garbage collector to do extra work scanning for pointers, and wastes a bit of executable space recording the gcmask for the type. This patch moves the arrays to the end of the type, fixing those minor issues. Change-Id: I849a75a19cc61137c8797a1ea5a4c97e0f69b4db Reviewed-on: https://go-review.googlesource.com/c/go/+/707596 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Mark Freeman <markfreeman@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2025-10-23all: replace Split in loops with more efficient SplitSeqcuishuang
Find these replacements through https://pkg.go.dev/golang.org/x/tools/gopls/internal/analysis/modernize. Change-Id: If88fe0e109b7c7557bfb3d3ffc15271af1ab5856 Reviewed-on: https://go-review.googlesource.com/c/go/+/668437 Reviewed-by: Sean Liao <sean@liao.dev> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: David Chase <drchase@google.com>
2025-05-19std: pass bytes.Buffer and strings.Builder by pointerAlan Donovan
This CL fixes a number of (all true positive) findings of vet's copylock analyzer patched to treat the Bu{ff,uild}er types as non-copyable after first use. This does require imposing an additional indirection between noder.writer and Encoder since the field is embedded by value but its constructor now returns a pointer. Updates golang/go#25907 Updates golang/go#47276 Change-Id: I0b4d77ac12bcecadf06a91709e695365da10766c Reviewed-on: https://go-review.googlesource.com/c/go/+/635339 Reviewed-by: Robert Findley <rfindley@google.com> Commit-Queue: Alan Donovan <adonovan@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Alan Donovan <adonovan@google.com>
2025-03-07compress/flate,compress/lzw: fix incorrect godoc linksOlivier Mengué
Fix incorrect godoc links related to the use of the name "Reader" for different things in the various compress/* packages: - in compress/flate Reader is the interface describing the underlying reader, not the decompressor as in other packages, so "returned reader" must not be linked to Reader. - in compress/lzw and compress/gzip Reader is the decompressor, not the interface of the underlying reader, so "underlying reader" must not be linked to Reader. With this patch the formatting of "underlying reader" and "returned reader" is consistent accross compress/* packages. Change-Id: Iea315fd5ee5b6c177855693d68841f3709a382cf Reviewed-on: https://go-review.googlesource.com/c/go/+/655335 Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-06compress/lzw,compress/gzip,compress/flate,compress/zlib,compress/bzip2: go ↵Olivier Mengué
doc links Add godoc links to compress/* package doc. Change-Id: I768ca250a39b0bb70eca35ac5b3b77ead73ca5f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/655057 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-02-18all: use a more straightforward return valuecuishuang
Change-Id: I27e86c221da7f541c4823f501801e02942c9a829 Reviewed-on: https://go-review.googlesource.com/c/go/+/649935 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-09-06compress/flate: use built-in clear to simplify the codeapocelipes
The new bootstrap toolchain allows us to use the built-in clear. Updates #64751 Change-Id: Ic363e1059f34c46eaa4267c0b40a4ed8d5b3961b GitHub-Last-Rev: 46ca735bfcd99a9874d7904a705970ed0cadf61c GitHub-Pull-Request: golang/go#69253 Reviewed-on: https://go-review.googlesource.com/c/go/+/610516 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-08-20src: fix typosAlexander Cyon
Fix typos in ~30 files Change-Id: Ie433aea01e7d15944c1e9e103691784876d5c1f9 GitHub-Last-Rev: bbaeb3d1f88a5fa6bbb69607b1bd075f496a7894 GitHub-Pull-Request: golang/go#68964 Reviewed-on: https://go-review.googlesource.com/c/go/+/606955 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-04-26compress: reordering fields to reduce struct sizesapocelipes
Overall, there are 32 bytes reduced. Change-Id: I455bf0874b33fa47719f42618e4800c7ff2a9e88 GitHub-Last-Rev: 7670344c4a643afdec0fdae3d34fdb8ccd81205f GitHub-Pull-Request: golang/go#67010 Reviewed-on: https://go-review.googlesource.com/c/go/+/581355 Reviewed-by: Joedian Reid <joedian@google.com> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-10-13compress: some minor corrections or adjustments for godoc linkcui fliter
Change-Id: Ic1fc777b9a4e8262851e27646d53a0ec4fe55b64 Reviewed-on: https://go-review.googlesource.com/c/go/+/535076 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2023-10-12compress: add available godoc linkcui fliter
Change-Id: Ia6e88aec59cb294e8b303a00fcd69f4cbf0dc09a Reviewed-on: https://go-review.googlesource.com/c/go/+/534759 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: shuang cui <imcusg@gmail.com>
2023-09-01compress: linkify references to io.ByteReaderJoe Tsai
The special treatment of io.ByteReader is somewhat significant. Linkify references to that type from documentation. Change-Id: I77c06e4c6b957d107638e2ff2a4c58148b396ed3 Reviewed-on: https://go-review.googlesource.com/c/go/+/525016 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2023-06-14all: fix spelling errorsAlexander Yastrebov
Fix spelling errors discovered using https://github.com/codespell-project/codespell. Errors in data files and vendored packages are ignored. Change-Id: I83c7818222f2eea69afbd270c15b7897678131dc GitHub-Last-Rev: 3491615b1b82832cc0064f535786546e89aa6184 GitHub-Pull-Request: golang/go#60758 Reviewed-on: https://go-review.googlesource.com/c/go/+/502576 Auto-Submit: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-24compress/flate, archive/zip: reduce memory allocationsMaksim Meshkov
The existing implementation allocates a new 4KB buffer each time it opens flate-encoded file in a zip archive. This commit allows the flate reader to reuse the buffer on call Reset instead of allocating a new one. It is noticeable when a zip archive contains a huge amount of files, e.g. zip archive has 50_000 files, for each file 4KB buffer is allocated, so it is 200MB memory allocations. If files are read sequentially only one buffer is needed. Fixes #59774 Change-Id: Ib16336b101ba58e8f0f30a45dc5fd4eeebc801a1 GitHub-Last-Rev: f3f395b2ad95b7ad7ce9df6f5e49c7b6a0627627 GitHub-Pull-Request: golang/go#59775 Reviewed-on: https://go-review.googlesource.com/c/go/+/487675 Run-TryBot: Ian Lance Taylor <iant@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2022-11-18all: add missing periods in commentscui fliter
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29 Reviewed-on: https://go-review.googlesource.com/c/go/+/449757 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Joedian Reid <joedian@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-09-29compress: remove useless break statementcuiweixie
Change-Id: Ia2b376d134d4fd273924de2e4cdee9eba5a15c57 Reviewed-on: https://go-review.googlesource.com/c/go/+/436707 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-15compress/flate: update NewReader documentationJoe Tsai
Calling close is no longer necessary. It was was originally necessary to cleanup goroutines spawned to decompress the stream. This has not been the case since CL 4548079. Update the documentation to mention how it handles trailing data after the end of the DEFLATE stream. Change-Id: Ieacba264230560713b7b8d604665223fd096f4ec Reviewed-on: https://go-review.googlesource.com/c/go/+/430377 Run-TryBot: Joseph Tsai <joetsai@digital-static.net> Auto-Submit: Joseph Tsai <joetsai@digital-static.net> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2022-05-17all: fix spellingJohn Bampton
Change-Id: I68538a50c22b02cdb5aa2a889f9440fed7b94c54 GitHub-Last-Rev: aaac9e78340ac482e9cd1b506a035f271c29648c GitHub-Pull-Request: golang/go#52944 Reviewed-on: https://go-review.googlesource.com/c/go/+/406835 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>
2022-05-17compress/flate: remove var newHTobias Klauser
After CL 404696 hewH is no longer used outide the loop and val can be used inside the loop instead. This leads to another slight improvement in some benchmarks (only non-zero results reported): name old time/op new time/op delta Decode/Digits/Huffman/1e4-4 125µs ±40% 96µs ± 7% -22.72% (p=0.000 n=10+10) Decode/Digits/Huffman/1e5-4 1.02ms ± 4% 0.97ms ± 3% -4.29% (p=0.000 n=10+10) Decode/Digits/Huffman/1e6-4 10.5ms ± 3% 10.1ms ± 2% -3.34% (p=0.000 n=10+9) Decode/Digits/Speed/1e4-4 130µs ± 5% 119µs ± 3% -8.33% (p=0.000 n=10+10) Decode/Digits/Speed/1e5-4 1.32ms ± 3% 1.26ms ± 4% -4.14% (p=0.001 n=10+10) Decode/Digits/Speed/1e6-4 13.4ms ± 3% 12.8ms ± 3% -4.64% (p=0.000 n=10+10) Decode/Digits/Default/1e4-4 142µs ±19% 124µs ± 3% -12.75% (p=0.000 n=10+9) Decode/Digits/Default/1e5-4 1.33ms ± 4% 1.27ms ± 2% -4.45% (p=0.000 n=10+10) Decode/Digits/Compression/1e4-4 132µs ± 4% 126µs ± 3% -4.59% (p=0.000 n=10+10) Decode/Digits/Compression/1e5-4 1.31ms ± 4% 1.28ms ± 3% -2.12% (p=0.015 n=10+10) Decode/Digits/Compression/1e6-4 13.2ms ± 3% 12.8ms ± 4% -3.36% (p=0.000 n=10+10) Decode/Newton/Huffman/1e4-4 138µs ± 5% 128µs ± 3% -7.55% (p=0.000 n=10+9) Decode/Newton/Huffman/1e5-4 1.25ms ± 1% 1.23ms ± 3% -2.21% (p=0.027 n=8+10) Decode/Newton/Huffman/1e6-4 13.0ms ± 5% 12.2ms ± 5% -6.54% (p=0.000 n=10+10) Decode/Newton/Speed/1e4-4 128µs ± 3% 118µs ± 4% -7.34% (p=0.000 n=9+10) Decode/Newton/Speed/1e5-4 1.06ms ± 3% 1.02ms ± 3% -3.58% (p=0.001 n=9+10) Decode/Newton/Speed/1e6-4 10.4ms ± 4% 10.1ms ± 3% -3.15% (p=0.003 n=10+10) Decode/Newton/Compression/1e4-4 105µs ± 2% 108µs ± 5% +2.82% (p=0.043 n=9+10) Encode/Digits/Speed/1e5-4 1.65ms ± 2% 1.70ms ± 4% +2.77% (p=0.003 n=8+10) Encode/Newton/Default/1e6-4 58.0ms ± 2% 57.0ms ± 1% -1.59% (p=0.001 n=9+9) name old speed new speed delta Decode/Digits/Huffman/1e4-4 82.2MB/s ±30% 103.9MB/s ± 8% +26.38% (p=0.000 n=10+10) Decode/Digits/Huffman/1e5-4 98.5MB/s ± 4% 102.9MB/s ± 3% +4.46% (p=0.000 n=10+10) Decode/Digits/Huffman/1e6-4 95.6MB/s ± 3% 98.9MB/s ± 2% +3.44% (p=0.000 n=10+9) Decode/Digits/Speed/1e4-4 76.9MB/s ± 5% 83.8MB/s ± 3% +9.06% (p=0.000 n=10+10) Decode/Digits/Speed/1e5-4 75.8MB/s ± 3% 79.1MB/s ± 4% +4.34% (p=0.001 n=10+10) Decode/Digits/Speed/1e6-4 74.4MB/s ± 3% 78.0MB/s ± 3% +4.86% (p=0.000 n=10+10) Decode/Digits/Default/1e4-4 70.7MB/s ±17% 80.6MB/s ± 2% +13.93% (p=0.000 n=10+9) Decode/Digits/Default/1e5-4 75.4MB/s ± 4% 78.9MB/s ± 3% +4.60% (p=0.000 n=10+10) Decode/Digits/Compression/1e4-4 75.8MB/s ± 3% 79.4MB/s ± 3% +4.79% (p=0.000 n=10+10) Decode/Digits/Compression/1e5-4 76.5MB/s ± 4% 78.1MB/s ± 3% +2.15% (p=0.015 n=10+10) Decode/Digits/Compression/1e6-4 75.7MB/s ± 3% 78.3MB/s ± 4% +3.49% (p=0.000 n=10+10) Decode/Newton/Huffman/1e4-4 72.4MB/s ± 5% 78.3MB/s ± 3% +8.13% (p=0.000 n=10+9) Decode/Newton/Huffman/1e5-4 79.8MB/s ± 1% 81.6MB/s ± 3% +2.29% (p=0.026 n=8+10) Decode/Newton/Huffman/1e6-4 76.9MB/s ± 5% 82.2MB/s ± 5% +6.96% (p=0.000 n=10+10) Decode/Newton/Speed/1e4-4 78.4MB/s ± 3% 84.6MB/s ± 4% +7.92% (p=0.000 n=9+10) Decode/Newton/Speed/1e5-4 94.2MB/s ± 3% 97.7MB/s ± 3% +3.72% (p=0.001 n=9+10) Decode/Newton/Speed/1e6-4 96.0MB/s ± 4% 99.1MB/s ± 3% +3.24% (p=0.003 n=10+10) Decode/Newton/Compression/1e4-4 95.2MB/s ± 2% 92.6MB/s ± 5% -2.67% (p=0.043 n=9+10) Encode/Digits/Speed/1e5-4 60.6MB/s ± 2% 59.0MB/s ± 4% -2.66% (p=0.002 n=8+10) Encode/Newton/Default/1e6-4 17.3MB/s ± 2% 17.5MB/s ± 1% +1.60% (p=0.001 n=9+9) Change-Id: I833b008fe5b67cd17ffdfbec22a3e4b10fcddece Reviewed-on: https://go-review.googlesource.com/c/go/+/406754 Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-11compress/flate: remove compressor.hash fieldIan Lance Taylor
After CL 20929, the only remaining uses of d.hash are immediately after it is set. Benchmarks where benchstat reports a non-zero delta: name old time/op new time/op delta Decode/Newton/Huffman/1e5-12 982µs ± 6% 922µs ± 4% -6.12% (p=0.000 n=10+10) Decode/Newton/Huffman/1e6-12 8.14ms ± 2% 7.71ms ± 2% -5.19% (p=0.000 n=10+10) Encode/Digits/Huffman/1e5-12 241µs ± 0% 239µs ± 0% -0.62% (p=0.000 n=9+10) Encode/Digits/Speed/1e4-12 113µs ± 0% 114µs ± 0% +1.21% (p=0.000 n=9+8) Encode/Digits/Speed/1e5-12 1.04ms ± 0% 1.05ms ± 0% +0.22% (p=0.027 n=9+8) Encode/Digits/Speed/1e6-12 10.3ms ± 0% 10.3ms ± 0% +0.16% (p=0.036 n=9+8) Encode/Digits/Default/1e4-12 193µs ± 0% 195µs ± 1% +0.77% (p=0.010 n=8+8) Encode/Digits/Default/1e5-12 2.92ms ± 1% 2.93ms ± 0% +0.54% (p=0.002 n=8+8) Encode/Digits/Compression/1e4-12 193µs ± 0% 194µs ± 0% +0.67% (p=0.000 n=8+8) Encode/Digits/Compression/1e5-12 2.92ms ± 0% 2.93ms ± 0% +0.53% (p=0.002 n=8+8) Encode/Digits/Compression/1e6-12 31.0ms ± 1% 31.2ms ± 0% +0.80% (p=0.000 n=10+9) Encode/Newton/Huffman/1e4-12 34.8µs ± 1% 34.3µs ± 0% -1.31% (p=0.000 n=9+10) Encode/Newton/Huffman/1e5-12 294µs ± 0% 291µs ± 0% -1.09% (p=0.000 n=8+9) Encode/Newton/Huffman/1e6-12 2.91ms ± 0% 2.89ms ± 0% -0.89% (p=0.000 n=8+9) Encode/Newton/Speed/1e5-12 838µs ± 0% 836µs ± 0% -0.18% (p=0.015 n=8+8) Encode/Newton/Speed/1e6-12 8.35ms ± 0% 8.28ms ± 1% -0.89% (p=0.000 n=8+10) Encode/Newton/Default/1e5-12 3.55ms ± 0% 3.54ms ± 0% -0.23% (p=0.027 n=8+9) Encode/Newton/Default/1e6-12 37.4ms ± 0% 37.3ms ± 0% -0.18% (p=0.001 n=9+8) Encode/Newton/Compression/1e6-12 53.3ms ± 0% 53.1ms ± 0% -0.25% (p=0.010 n=10+9) name old speed new speed delta Decode/Newton/Huffman/1e5-12 102MB/s ± 6% 109MB/s ± 4% +6.48% (p=0.000 n=10+10) Decode/Newton/Huffman/1e6-12 123MB/s ± 2% 130MB/s ± 2% +5.49% (p=0.000 n=10+10) Encode/Digits/Huffman/1e5-12 416MB/s ± 0% 418MB/s ± 0% +0.62% (p=0.000 n=9+10) Encode/Digits/Speed/1e4-12 88.9MB/s ± 0% 87.8MB/s ± 0% -1.19% (p=0.000 n=9+8) Encode/Digits/Speed/1e5-12 95.7MB/s ± 0% 95.5MB/s ± 0% -0.22% (p=0.025 n=9+8) Encode/Digits/Speed/1e6-12 97.5MB/s ± 0% 97.3MB/s ± 0% -0.16% (p=0.026 n=9+8) Encode/Digits/Default/1e4-12 51.7MB/s ± 0% 51.3MB/s ± 1% -0.76% (p=0.007 n=8+8) Encode/Digits/Default/1e5-12 34.3MB/s ± 1% 34.1MB/s ± 0% -0.54% (p=0.001 n=8+8) Encode/Digits/Compression/1e4-12 51.8MB/s ± 0% 51.5MB/s ± 0% -0.66% (p=0.000 n=8+8) Encode/Digits/Compression/1e5-12 34.3MB/s ± 0% 34.1MB/s ± 0% -0.53% (p=0.002 n=8+8) Encode/Digits/Compression/1e6-12 32.3MB/s ± 1% 32.0MB/s ± 0% -0.79% (p=0.000 n=10+9) Encode/Newton/Huffman/1e4-12 288MB/s ± 1% 291MB/s ± 0% +1.33% (p=0.000 n=9+10) Encode/Newton/Huffman/1e5-12 340MB/s ± 0% 344MB/s ± 0% +1.10% (p=0.000 n=8+9) Encode/Newton/Huffman/1e6-12 343MB/s ± 0% 346MB/s ± 0% +0.90% (p=0.000 n=8+9) Encode/Newton/Speed/1e5-12 119MB/s ± 0% 120MB/s ± 0% +0.18% (p=0.014 n=8+8) Encode/Newton/Speed/1e6-12 120MB/s ± 0% 121MB/s ± 1% +0.90% (p=0.000 n=8+10) Encode/Newton/Default/1e5-12 28.2MB/s ± 0% 28.3MB/s ± 0% +0.23% (p=0.029 n=8+9) Encode/Newton/Default/1e6-12 26.7MB/s ± 0% 26.8MB/s ± 0% +0.20% (p=0.000 n=9+7) Encode/Newton/Compression/1e6-12 18.8MB/s ± 0% 18.8MB/s ± 0% +0.25% (p=0.012 n=10+9) Change-Id: I8ba0efcf5d42595d856082656b45f87bb3d7be0f Reviewed-on: https://go-review.googlesource.com/c/go/+/404696 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-05-08compress/flate: cancel redundant operationsHeisenberg
The assignment operation in the program seems to be redundant, the first judgment will continue to overwrite the previous value. The subsequent slicing operation will cut all the values without frequency. Change-Id: Id59fc36dd5bacfde881edaf0d9c1af5348286611 Reviewed-on: https://go-review.googlesource.com/c/go/+/244157 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Joe Tsai <joetsai@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-05-02compress/flate: move idempotent close logic to compressorJoe Tsai
The compressor methods already have logic for handling a sticky error. Merge the logic from CL 136475 into that. This slightly changes the error message to be more sensible in the situation where it's returned by Flush. Updates #27741 Change-Id: Ie34cf3164d0fa6bd0811175ca467dbbcb3be1395 Reviewed-on: https://go-review.googlesource.com/c/go/+/403514 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-05-02compress/flate: return error on closed stream writeGregory Man
Previously flate.Writer allowed writes after Close, and this behavior could lead to stream corruption. Fixes #27741 Change-Id: Iee1ac69f8199232f693dba77b275f7078257b582 Reviewed-on: https://go-review.googlesource.com/c/go/+/136475 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-04-11all: gofmt main repoRuss Cox
[This CL is part of a sequence implementing the proposal #51082. The design doc is at https://go.dev/s/godocfmt-design.] Run the updated gofmt, which reformats doc comments, on the main repository. Vendored files are excluded. For #51082. Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407 Reviewed-on: https://go-review.googlesource.com/c/go/+/384268 Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-01all: fix various doc comment formatting nitsRuss Cox
A run of lines that are indented with any number of spaces or tabs format as a <pre> block. This commit fixes various doc comments that format badly according to that (standard) rule. For example, consider: // - List item. // Second line. // - Another item. Because the - lines are unindented, this is actually two paragraphs separated by a one-line <pre> block. This CL rewrites it to: // - List item. // Second line. // - Another item. Today, that will format as a single <pre> block. In a future release, we hope to format it as a bulleted list. Various other minor fixes as well, all in preparation for reformatting. For #51082. Change-Id: I95cf06040d4186830e571cd50148be3bf8daf189 Reviewed-on: https://go-review.googlesource.com/c/go/+/384257 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-19compress/flate: fix corrupted outputKlaus Post
The fastest compression mode can pick up a false match for every 2GB of input data resulting in incorrectly decompressed data. Since matches are allowed to be up to and including at maxMatchOffset we must offset the buffer by an additional element to prevent the first 4 bytes to match after an out-of-reach value after shiftOffsets has been called. We offset by `maxMatchOffset + 1` so offset 0 in the table will now fail the `if offset > maxMatchOffset` in all cases. Fixes #41420 Change-Id: If1fbe01728e132b8a207e3f3f439edd832dcc710 GitHub-Last-Rev: 50fabab0da874c37543b139459a810e12e83cee2 GitHub-Pull-Request: golang/go#41477 Reviewed-on: https://go-review.googlesource.com/c/go/+/255879 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Joe Tsai <thebrokentoaster@gmail.com> Trust: Matthew Dempsky <mdempsky@google.com>
2020-10-16compress/flate: revert a goto for-loopDaniel Martí
In https://golang.org/cl/16528, a goto loop was chosen over a regular for loop since that would make the function inlinable. Thanks to the recent https://golang.org/cl/256459, for loops without a label can now be inlined. So we can undo the workaround and simplify the code. Also add the function to TestIntendedInlining, which passes both before and after the change, as expected. For #14768. Change-Id: Ie5df55a6bcb07c538ca331eef2f908807ff0b516 Reviewed-on: https://go-review.googlesource.com/c/go/+/263037 Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-06compress/flate: remove unneeded zeroing of bytes array in ↵Dan Scales
(*huffmanBitWriter).reset There is no correctness reason to zero out the w.bytes array in (w *huffmanBitWriter).reset, since w.nbytes is correctly set to zero. The elements of the bytes array are always written sequentially, with nbytes indicating how many elements have been written, and are only read up to the current value of nybytes. We have a pprof profile of a web server that compresses its request/responses, and the zeroing in reset() is taking up 2.6% of the CPU time of the server (and could be causing more slowdowns elsewhere due to its effects on the cache). This overhead may be showing up especially because there are many request/responses that are all fairly small. I'm not sure if the zeroing of the bytes array was intended as extra protection of data across reset uses in the same program, but no protection is needed as long as the huffman_bit_writer code remains correct. Change-Id: I67f2b2f56cff9dcc38d8fc0aea885bb010aeedbf Reviewed-on: https://go-review.googlesource.com/c/go/+/258577 Run-TryBot: Dan Scales <danscales@google.com> Run-TryBot: Joe Tsai <joetsai@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Joe Tsai <joetsai@google.com> Trust: Joe Tsai <joetsai@google.com> Trust: Dan Scales <danscales@google.com>
2020-07-16compress/flate: fix another deflate Reset inconsistencyRuss Cox
While investigating #34121, fixed by CL 193605, I discovered another case where Reset was not quite resetting enough. This specific case is not a problem in Reset itself but rather that the Huffman bit writer in one code path is using uninitialized memory left over from a previous block, making the compression not choose the optimal compression method. Fixes #34121. Change-Id: I29245b28214d924e382f91e2c56b4b8a9b7da13d Reviewed-on: https://go-review.googlesource.com/c/go/+/243140 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-07-16compress/flate: fix deflate Reset consistencyKlaus Post
Modify the overflow detection logic to shuffle the contents of the table to a lower offset to avoid leaking the effects of a previous use of compress.Writer past Reset calls. Fixes #34121 Change-Id: I9963eadfa5482881e7b7adbad4c2cae146b669ab GitHub-Last-Rev: 8b35798cdd4d5a901d6422647b12984d7e500ba3 GitHub-Pull-Request: golang/go#34128 Reviewed-on: https://go-review.googlesource.com/c/go/+/193605 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-28compress/flate: improve deflate performance by register allocating the indexNao YONASHIRO
Use local index variable to help the go compiler use a register to for the hash index instead of continuous memory read and write operations. compress/flate: Encode/Digits/Huffman/1e4-4 35.3µs ± 1% 32.7µs ± 0% -7.48% (p=0.000 n=17+19) Encode/Digits/Huffman/1e5-4 330µs ± 0% 312µs ± 0% -5.55% (p=0.000 n=17+18) Encode/Digits/Huffman/1e6-4 3.30ms ± 0% 3.12ms ± 0% -5.64% (p=0.000 n=18+19) Encode/Digits/Speed/1e4-4 157µs ± 0% 156µs ± 0% -0.41% (p=0.000 n=17+19) Encode/Digits/Speed/1e5-4 1.46ms ± 0% 1.46ms ± 1% ~ (p=0.478 n=20+19) Encode/Digits/Speed/1e6-4 14.4ms ± 0% 14.4ms ± 0% ~ (p=0.835 n=19+20) Encode/Digits/Default/1e4-4 309µs ± 0% 310µs ± 0% +0.23% (p=0.000 n=19+17) Encode/Digits/Default/1e5-4 4.76ms ± 0% 4.76ms ± 0% ~ (p=0.297 n=19+19) Encode/Digits/Default/1e6-4 51.0ms ± 0% 51.0ms ± 1% ~ (p=0.233 n=18+19) Encode/Digits/Compression/1e4-4 309µs ± 0% 310µs ± 0% +0.21% (p=0.000 n=17+20) Encode/Digits/Compression/1e5-4 4.76ms ± 0% 4.76ms ± 0% ~ (p=0.749 n=20+19) Encode/Digits/Compression/1e6-4 50.9ms ± 0% 50.9ms ± 0% ~ (p=0.499 n=18+19) Encode/Newton/Huffman/1e4-4 51.9µs ± 0% 48.0µs ± 0% -7.61% (p=0.000 n=19+19) Encode/Newton/Huffman/1e5-4 396µs ± 0% 377µs ± 0% -4.79% (p=0.000 n=18+19) Encode/Newton/Huffman/1e6-4 3.95ms ± 0% 3.74ms ± 0% -5.21% (p=0.000 n=20+17) Encode/Newton/Speed/1e4-4 155µs ± 0% 154µs ± 0% -0.67% (p=0.000 n=17+18) Encode/Newton/Speed/1e5-4 1.17ms ± 0% 1.16ms ± 0% -0.64% (p=0.000 n=20+16) Encode/Newton/Speed/1e6-4 11.6ms ± 0% 11.5ms ± 0% -0.63% (p=0.000 n=19+20) Encode/Newton/Default/1e4-4 347µs ± 0% 347µs ± 0% ~ (p=0.744 n=20+19) Encode/Newton/Default/1e5-4 5.06ms ± 0% 5.02ms ± 0% -0.77% (p=0.000 n=20+19) Encode/Newton/Default/1e6-4 53.3ms ± 1% 52.8ms ± 0% -0.91% (p=0.000 n=18+16) Encode/Newton/Compression/1e4-4 351µs ± 0% 351µs ± 0% ~ (p=0.277 n=20+20) Encode/Newton/Compression/1e5-4 6.90ms ± 0% 6.85ms ± 0% -0.61% (p=0.000 n=19+18) Encode/Newton/Compression/1e6-4 73.2ms ± 0% 72.8ms ± 0% -0.52% (p=0.000 n=18+18) name old speed new speed delta Encode/Digits/Huffman/1e4-4 283MB/s ± 1% 306MB/s ± 0% +8.09% (p=0.000 n=17+19) Encode/Digits/Huffman/1e5-4 303MB/s ± 0% 321MB/s ± 0% +5.87% (p=0.000 n=18+18) Encode/Digits/Huffman/1e6-4 303MB/s ± 0% 321MB/s ± 0% +5.98% (p=0.000 n=18+19) Encode/Digits/Speed/1e4-4 63.9MB/s ± 0% 64.2MB/s ± 0% +0.41% (p=0.000 n=17+19) Encode/Digits/Speed/1e5-4 68.5MB/s ± 0% 68.4MB/s ± 1% ~ (p=0.481 n=20+19) Encode/Digits/Speed/1e6-4 69.4MB/s ± 0% 69.3MB/s ± 0% ~ (p=0.712 n=19+20) Encode/Digits/Default/1e4-4 32.3MB/s ± 0% 32.3MB/s ± 0% -0.23% (p=0.000 n=19+17) Encode/Digits/Default/1e5-4 21.0MB/s ± 0% 21.0MB/s ± 0% ~ (p=0.460 n=19+19) Encode/Digits/Default/1e6-4 19.6MB/s ± 0% 19.6MB/s ± 1% ~ (p=0.180 n=18+19) Encode/Digits/Compression/1e4-4 32.3MB/s ± 0% 32.3MB/s ± 0% -0.21% (p=0.000 n=17+20) Encode/Digits/Compression/1e5-4 21.0MB/s ± 0% 21.0MB/s ± 0% ~ (p=0.700 n=20+19) Encode/Digits/Compression/1e6-4 19.6MB/s ± 0% 19.6MB/s ± 0% ~ (p=0.486 n=18+19) Encode/Newton/Huffman/1e4-4 193MB/s ± 0% 208MB/s ± 0% +8.23% (p=0.000 n=19+19) Encode/Newton/Huffman/1e5-4 252MB/s ± 0% 265MB/s ± 0% +5.04% (p=0.000 n=18+19) Encode/Newton/Huffman/1e6-4 253MB/s ± 0% 267MB/s ± 0% +5.49% (p=0.000 n=20+17) Encode/Newton/Speed/1e4-4 64.5MB/s ± 0% 65.0MB/s ± 0% +0.67% (p=0.000 n=17+18) Encode/Newton/Speed/1e5-4 85.7MB/s ± 0% 86.3MB/s ± 0% +0.65% (p=0.000 n=20+16) Encode/Newton/Speed/1e6-4 86.2MB/s ± 0% 86.7MB/s ± 0% +0.63% (p=0.000 n=19+20) Encode/Newton/Default/1e4-4 28.9MB/s ± 0% 28.9MB/s ± 0% ~ (p=0.840 n=20+19) Encode/Newton/Default/1e5-4 19.8MB/s ± 0% 19.9MB/s ± 0% +0.78% (p=0.000 n=20+19) Encode/Newton/Default/1e6-4 18.8MB/s ± 1% 18.9MB/s ± 0% +0.93% (p=0.000 n=18+16) Encode/Newton/Compression/1e4-4 28.5MB/s ± 0% 28.5MB/s ± 0% ~ (p=0.244 n=20+20) Encode/Newton/Compression/1e5-4 14.5MB/s ± 0% 14.6MB/s ± 0% +0.61% (p=0.000 n=19+18) Encode/Newton/Compression/1e6-4 13.7MB/s ± 0% 13.7MB/s ± 0% +0.53% (p=0.000 n=18+18) image/png: name old time/op new time/op delta EncodeGray-4 2.16ms ± 1% 1.85ms ± 1% -14.17% (p=0.000 n=86+91) EncodeGrayWithBufferPool-4 1.99ms ± 0% 1.69ms ± 0% -15.09% (p=0.000 n=97+94) EncodeNRGBOpaque-4 6.51ms ± 1% 5.62ms ± 1% -13.66% (p=0.000 n=90+92) EncodeNRGBA-4 7.33ms ± 1% 6.12ms ± 1% -16.49% (p=0.000 n=89+90) EncodePaletted-4 5.10ms ± 1% 4.96ms ± 1% -2.76% (p=0.000 n=90+87) EncodeRGBOpaque-4 6.51ms ± 1% 5.63ms ± 1% -13.49% (p=0.000 n=94+87) EncodeRGBA-4 24.3ms ± 2% 23.0ms ± 0% -5.23% (p=0.000 n=91+89) name old speed new speed delta EncodeGray-4 142MB/s ± 1% 166MB/s ± 1% +16.50% (p=0.000 n=86+91) EncodeGrayWithBufferPool-4 154MB/s ± 0% 182MB/s ± 0% +17.78% (p=0.000 n=97+94) EncodeNRGBOpaque-4 189MB/s ± 1% 219MB/s ± 1% +15.82% (p=0.000 n=90+93) EncodeNRGBA-4 168MB/s ± 1% 201MB/s ± 1% +19.75% (p=0.000 n=89+90) EncodePaletted-4 60.3MB/s ± 1% 62.0MB/s ± 1% +2.84% (p=0.000 n=90+87) EncodeRGBOpaque-4 189MB/s ± 1% 218MB/s ± 1% +15.60% (p=0.000 n=94+87) EncodeRGBA-4 50.6MB/s ± 2% 53.4MB/s ± 0% +5.51% (p=0.000 n=91+89) Change-Id: Ifed4486a7ba19a26abe5cbf2142f15cc7464e84f Reviewed-on: https://go-review.googlesource.com/c/go/+/187837 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-31cmd/go: further reduce init workDaniel Martí
The first biggest offender was crypto/des.init at ~1%. It's cryptographically broken and the init function is relatively expensive, which is unfortunate as both crypto/tls and crypto/x509 (and by extension, cmd/go) import it. Hide the work behind sync.Once. The second biggest offender was flag.sortFlags at just under 1%, used by the Visit flagset methods. It allocated two slices, which made a difference as cmd/go iterates over multiple flagsets during init. Use a single slice with a direct sort.Interface implementation. Another big offender is initializing global maps. Reducing this work in cmd/go/internal/imports and net/textproto gives us close to another whole 1% in saved work. The former can use map literals, and the latter can hide the work behind sync.Once. Finally, compress/flate used newHuffmanBitWriter as part of init, which allocates many objects and slices. Yet it only used one of the slice fields. Allocating just that slice saves a surprising ~0.3%, since we generated a lot of unnecessary garbage. All in all, these little pieces amount to just over 3% saved CPU time. name old time/op new time/op delta ExecGoEnv-8 3.61ms ± 1% 3.50ms ± 0% -3.02% (p=0.000 n=10+10) Updates #26775. Updates #29382. Change-Id: I915416e88a874c63235ba512617c8aef35c0ca8b Reviewed-on: https://go-review.googlesource.com/c/go/+/166459 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-19src, misc: apply gofmtRobert Griesemer
This applies the new gofmt literal normalizations to the library. Change-Id: I8c1e8ef62eb556fc568872c9f77a31ef236348e7 Reviewed-on: https://go-review.googlesource.com/c/162539 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-12-20compress/flate: fix the old url for the flate algorithmOsamu TONOMORI
Change-Id: I84b74bc96516033bbf4a01f9aa81fe60d5a41355 Reviewed-on: https://go-review.googlesource.com/c/155317 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-06all: fix a bunch of misspellingsIgor Zhilianin
Change-Id: If2954bdfc551515403706b2cd0dde94e45936e08 GitHub-Last-Rev: d4cfc41a5504cf10befefdb881d4c45986a1d1f8 GitHub-Pull-Request: golang/go#28049 Reviewed-on: https://go-review.googlesource.com/c/140299 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-01Revert "compress: move benchmark text from src/testdata to ↵Katie Hockman
src/compress/testdata" This reverts commit 067bb443af6b44cb026ab182a26d157dbd1b2dd6. Reason for revert: Failing Darwin-arm builds because that testing environment does not access testdata from sibling directories. A future change will likely be made to move this testdata out of src/testdata to create a solution that doesn't require the single-file directory. Updates #27151 Change-Id: I8dbf5dd9512c94a605ee749ff4655cb00b0de686 Reviewed-on: https://go-review.googlesource.com/138737 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28compress: move benchmark text from src/testdata to src/compress/testdataKatie Hockman
This text is used mainly for benchmark compression testing, and in one net test. The text was prevoiusly in a src/testdata directory, but since that directory would only include one file, the text is moved to the existing src/compression/testdata directory. This does not cause any change to the benchmark results. Updates #27151 Change-Id: I38ab5089dfe744189a970947d15be50ef1d48517 Reviewed-on: https://go-review.googlesource.com/138495 Run-TryBot: Katie Hockman <katie@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-09-24compress: reduce copies of new text for compression testingKatie Hockman
The previous book was 387 KiB decompressed and 119 KiB compressed, the new book is 567 KiB decompressed and 132 KiB compressed. Overall, this change will reduce the release binary size by 196 KiB. The new book will allow for slightly more extensive compression testing with a larger text. Command to run the benchmark tests used with benchstat: `../bin/go test -run='^$' -count=4 -bench=. compress/bzip2 compress/flate` When running the benchmarks locally, changed "Newton" to "Twain" and filtered the tests with the -bench flag to include only those which were relevant to these changes. benchstat results below: name old time/op new time/op delta DecodeTwain-8 19.6ms ± 2% 24.1ms ± 1% +23.04% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 140µs ± 3% 139µs ± 5% ~ (p=0.886 n=4+4) Decode/Twain/Huffman/1e5-8 1.27ms ± 3% 1.26ms ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e6-8 12.4ms ± 0% 13.2ms ± 1% +6.42% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 133µs ± 1% 123µs ± 1% -7.35% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 1.20ms ± 0% 1.02ms ± 3% -15.32% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 12.0ms ± 2% 10.1ms ± 3% -15.89% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 131µs ± 6% 108µs ± 5% -17.84% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 1.06ms ± 2% 0.80ms ± 1% -24.97% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 10.0ms ± 3% 8.0ms ± 3% -20.06% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 128µs ± 4% 115µs ± 4% -9.70% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 1.04ms ± 2% 0.83ms ± 4% -20.37% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 10.4ms ± 4% 8.1ms ± 5% -22.25% (p=0.029 n=4+4) Encode/Twain/Huffman/1e4-8 55.7µs ± 2% 55.6µs ± 1% ~ (p=1.000 n=4+4) Encode/Twain/Huffman/1e5-8 441µs ± 0% 435µs ± 2% ~ (p=0.343 n=4+4) Encode/Twain/Huffman/1e6-8 4.31ms ± 4% 4.30ms ± 4% ~ (p=0.886 n=4+4) Encode/Twain/Speed/1e4-8 193µs ± 1% 166µs ± 2% -14.09% (p=0.029 n=4+4) Encode/Twain/Speed/1e5-8 1.54ms ± 1% 1.22ms ± 1% -20.53% (p=0.029 n=4+4) Encode/Twain/Speed/1e6-8 15.3ms ± 1% 12.2ms ± 3% -20.62% (p=0.029 n=4+4) Encode/Twain/Default/1e4-8 393µs ± 1% 390µs ± 1% ~ (p=0.114 n=4+4) Encode/Twain/Default/1e5-8 6.12ms ± 4% 6.02ms ± 5% ~ (p=0.486 n=4+4) Encode/Twain/Default/1e6-8 69.4ms ± 5% 59.0ms ± 4% -15.07% (p=0.029 n=4+4) Encode/Twain/Compression/1e4-8 423µs ± 2% 379µs ± 2% -10.34% (p=0.029 n=4+4) Encode/Twain/Compression/1e5-8 7.00ms ± 1% 7.88ms ± 3% +12.49% (p=0.029 n=4+4) Encode/Twain/Compression/1e6-8 76.6ms ± 5% 80.9ms ± 3% ~ (p=0.114 n=4+4) name old speed new speed delta DecodeTwain-8 19.8MB/s ± 2% 23.6MB/s ± 1% +18.84% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 71.7MB/s ± 3% 72.1MB/s ± 6% ~ (p=0.943 n=4+4) Decode/Twain/Huffman/1e5-8 78.8MB/s ± 3% 79.5MB/s ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e6-8 80.5MB/s ± 0% 75.6MB/s ± 1% -6.03% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 75.2MB/s ± 1% 81.2MB/s ± 1% +7.93% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 83.4MB/s ± 0% 98.6MB/s ± 3% +18.16% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 83.6MB/s ± 2% 99.5MB/s ± 3% +18.91% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 76.3MB/s ± 6% 92.8MB/s ± 4% +21.62% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 94.4MB/s ± 3% 125.7MB/s ± 1% +33.24% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 100MB/s ± 3% 125MB/s ± 3% +25.12% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 78.4MB/s ± 4% 86.8MB/s ± 4% +10.73% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 95.7MB/s ± 2% 120.3MB/s ± 4% +25.65% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 96.4MB/s ± 4% 124.0MB/s ± 5% +28.64% (p=0.029 n=4+4) Encode/Twain/Huffman/1e4-8 179MB/s ± 2% 180MB/s ± 1% ~ (p=1.000 n=4+4) Encode/Twain/Huffman/1e5-8 227MB/s ± 0% 230MB/s ± 2% ~ (p=0.343 n=4+4) Encode/Twain/Huffman/1e6-8 232MB/s ± 4% 233MB/s ± 4% ~ (p=0.886 n=4+4) Encode/Twain/Speed/1e4-8 51.8MB/s ± 1% 60.4MB/s ± 2% +16.43% (p=0.029 n=4+4) Encode/Twain/Speed/1e5-8 65.1MB/s ± 1% 81.9MB/s ± 1% +25.83% (p=0.029 n=4+4) Encode/Twain/Speed/1e6-8 65.2MB/s ± 1% 82.2MB/s ± 3% +26.00% (p=0.029 n=4+4) Encode/Twain/Default/1e4-8 25.4MB/s ± 1% 25.6MB/s ± 1% ~ (p=0.114 n=4+4) Encode/Twain/Default/1e5-8 16.4MB/s ± 4% 16.6MB/s ± 5% ~ (p=0.486 n=4+4) Encode/Twain/Default/1e6-8 14.4MB/s ± 6% 17.0MB/s ± 4% +17.67% (p=0.029 n=4+4) Encode/Twain/Compression/1e4-8 23.6MB/s ± 2% 26.4MB/s ± 2% +11.54% (p=0.029 n=4+4) Encode/Twain/Compression/1e5-8 14.3MB/s ± 1% 12.7MB/s ± 3% -11.08% (p=0.029 n=4+4) Encode/Twain/Compression/1e6-8 13.1MB/s ± 4% 12.4MB/s ± 3% ~ (p=0.114 n=4+4) name old alloc/op new alloc/op delta DecodeTwain-8 3.63MB ± 0% 3.63MB ± 0% +0.15% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 42.0kB ± 0% 41.3kB ± 0% -1.62% (p=0.029 n=4+4) Decode/Twain/Huffman/1e5-8 43.5kB ± 0% 45.1kB ± 0% +3.74% (p=0.029 n=4+4) Decode/Twain/Huffman/1e6-8 71.7kB ± 0% 80.0kB ± 0% +11.55% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 41.2kB ± 0% 41.3kB ± 0% ~ (p=0.286 n=4+4) Decode/Twain/Speed/1e5-8 45.1kB ± 0% 43.9kB ± 0% -2.80% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 72.8kB ± 0% 81.3kB ± 0% +11.72% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 41.2kB ± 0% 41.2kB ± 0% -0.22% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 44.4kB ± 0% 43.0kB ± 0% -3.02% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 71.0kB ± 0% 61.8kB ± 0% -13.00% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 41.3kB ± 0% 41.2kB ± 0% -0.29% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 43.3kB ± 0% 43.0kB ± 0% -0.72% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 69.1kB ± 0% 63.7kB ± 0% -7.90% (p=0.029 n=4+4) name old allocs/op new allocs/op delta DecodeTwain-8 51.0 ± 0% 51.2 ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e4-8 15.0 ± 0% 14.0 ± 0% -6.67% (p=0.029 n=4+4) Decode/Twain/Huffman/1e5-8 20.0 ± 0% 23.0 ± 0% +15.00% (p=0.029 n=4+4) Decode/Twain/Huffman/1e6-8 134 ± 0% 161 ± 0% +20.15% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 17.0 ± 0% 18.0 ± 0% +5.88% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 30.0 ± 0% 31.0 ± 0% +3.33% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 193 ± 0% 228 ± 0% +18.13% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 17.0 ± 0% 15.0 ± 0% -11.76% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 28.0 ± 0% 32.0 ± 0% +14.29% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 199 ± 0% 158 ± 0% -20.60% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 17.0 ± 0% 15.0 ± 0% -11.76% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 28.0 ± 0% 32.0 ± 0% +14.29% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 196 ± 0% 150 ± 0% -23.47% (p=0.029 n=4+4) Updates #27151 Change-Id: I6c439694ed16a33bb4c63fbfb8570c7de46b4f2d Reviewed-on: https://go-review.googlesource.com/135495 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-06-01all: update comment URLs from HTTP to HTTPS, where possibleTim Cooper
Each URL was manually verified to ensure it did not serve up incorrect content. Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df Reviewed-on: https://go-review.googlesource.com/115798 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-05-07compress/flate: do not rename math/bits importRuss Cox
Makes compress/flate work better with cmd/dist bootstrap. Change-Id: Ifc7d74027367008e82c1d14ec77141830583ba82 Reviewed-on: https://go-review.googlesource.com/111815 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-17compress/flate: optimize huffSymIlya Tocar
By using local variables and assigning them back to decompressor at the end of huffSym, we allow compiler to keep them in registers and avoid reloading/storing them repeatedly. To archive this, moreBits was inlined and specialized to work with local variables. Also move EOF error conversion to helper function, to make inlined part of moreBits more readable. Together this results in nice speed-up: name old time/op new time/op delta Decode/Digits/Huffman/1e4-6 278µs ± 1% 240µs ± 2% -13.72% (p=0.000 n=10+10) Decode/Digits/Huffman/1e5-6 2.38ms ± 1% 2.05ms ± 1% -14.12% (p=0.000 n=10+10) Decode/Digits/Huffman/1e6-6 23.4ms ± 1% 19.9ms ± 0% -14.69% (p=0.000 n=9+9) Decode/Digits/Speed/1e4-6 280µs ± 2% 254µs ± 1% -9.28% (p=0.000 n=10+9) Decode/Digits/Speed/1e5-6 2.53ms ± 1% 2.35ms ± 1% -7.17% (p=0.000 n=10+10) Decode/Digits/Speed/1e6-6 24.8ms ± 1% 23.0ms ± 1% -7.22% (p=0.000 n=10+10) Decode/Digits/Default/1e4-6 281µs ± 2% 259µs ± 3% -8.03% (p=0.000 n=10+10) Decode/Digits/Default/1e5-6 2.45ms ± 1% 2.30ms ± 1% -6.15% (p=0.000 n=10+10) Decode/Digits/Default/1e6-6 24.1ms ± 1% 22.6ms ± 0% -6.31% (p=0.000 n=9+9) Decode/Digits/Compression/1e4-6 279µs ± 2% 261µs ± 2% -6.53% (p=0.000 n=8+9) Decode/Digits/Compression/1e5-6 2.44ms ± 1% 2.30ms ± 1% -5.72% (p=0.000 n=10+9) Decode/Digits/Compression/1e6-6 24.0ms ± 1% 22.6ms ± 0% -6.10% (p=0.000 n=10+9) Decode/Twain/Huffman/1e4-6 316µs ± 2% 267µs ± 3% -15.30% (p=0.000 n=9+10) Decode/Twain/Huffman/1e5-6 2.62ms ± 0% 2.22ms ± 0% -15.24% (p=0.000 n=10+10) Decode/Twain/Huffman/1e6-6 25.7ms ± 1% 21.8ms ± 0% -15.19% (p=0.000 n=10+10) Decode/Twain/Speed/1e4-6 290µs ± 1% 264µs ± 2% -9.17% (p=0.000 n=9+10) Decode/Twain/Speed/1e5-6 2.35ms ± 1% 2.13ms ± 1% -9.74% (p=0.000 n=9+10) Decode/Twain/Speed/1e6-6 22.9ms ± 0% 20.7ms ± 0% -9.68% (p=0.000 n=10+9) Decode/Twain/Default/1e4-6 270µs ± 2% 252µs ± 2% -6.67% (p=0.000 n=9+10) Decode/Twain/Default/1e5-6 2.02ms ± 1% 1.84ms ± 1% -8.85% (p=0.000 n=10+10) Decode/Twain/Default/1e6-6 19.1ms ± 0% 17.5ms ± 1% -8.73% (p=0.000 n=9+10) Decode/Twain/Compression/1e4-6 272µs ± 1% 250µs ± 4% -8.20% (p=0.000 n=9+10) Decode/Twain/Compression/1e5-6 2.01ms ± 0% 1.84ms ± 1% -8.57% (p=0.000 n=9+10) Decode/Twain/Compression/1e6-6 19.1ms ± 0% 17.4ms ± 1% -8.75% (p=0.000 n=9+10) name old speed new speed delta Decode/Digits/Huffman/1e4-6 35.9MB/s ± 1% 41.7MB/s ± 2% +15.91% (p=0.000 n=10+10) Decode/Digits/Huffman/1e5-6 41.9MB/s ± 1% 48.8MB/s ± 1% +16.44% (p=0.000 n=10+10) Decode/Digits/Huffman/1e6-6 42.8MB/s ± 1% 50.2MB/s ± 0% +17.22% (p=0.000 n=9+9) Decode/Digits/Speed/1e4-6 35.7MB/s ± 2% 39.4MB/s ± 1% +10.22% (p=0.000 n=10+9) Decode/Digits/Speed/1e5-6 39.6MB/s ± 1% 42.6MB/s ± 1% +7.73% (p=0.000 n=10+10) Decode/Digits/Speed/1e6-6 40.3MB/s ± 1% 43.4MB/s ± 1% +7.78% (p=0.000 n=10+10) Decode/Digits/Default/1e4-6 35.6MB/s ± 2% 38.7MB/s ± 2% +8.74% (p=0.000 n=10+10) Decode/Digits/Default/1e5-6 40.9MB/s ± 1% 43.6MB/s ± 1% +6.55% (p=0.000 n=10+10) Decode/Digits/Default/1e6-6 41.5MB/s ± 1% 44.3MB/s ± 0% +6.73% (p=0.000 n=9+9) Decode/Digits/Compression/1e4-6 35.8MB/s ± 2% 38.3MB/s ± 2% +6.99% (p=0.000 n=8+9) Decode/Digits/Compression/1e5-6 40.9MB/s ± 1% 43.4MB/s ± 1% +6.07% (p=0.000 n=10+9) Decode/Digits/Compression/1e6-6 41.6MB/s ± 1% 44.3MB/s ± 0% +6.49% (p=0.000 n=10+9) Decode/Twain/Huffman/1e4-6 31.7MB/s ± 2% 37.4MB/s ± 3% +18.08% (p=0.000 n=9+10) Decode/Twain/Huffman/1e5-6 38.2MB/s ± 0% 45.0MB/s ± 0% +17.97% (p=0.000 n=10+10) Decode/Twain/Huffman/1e6-6 38.9MB/s ± 1% 45.9MB/s ± 0% +17.90% (p=0.000 n=10+10) Decode/Twain/Speed/1e4-6 34.5MB/s ± 1% 38.0MB/s ± 2% +10.11% (p=0.000 n=9+10) Decode/Twain/Speed/1e5-6 42.5MB/s ± 1% 47.0MB/s ± 1% +10.79% (p=0.000 n=9+10) Decode/Twain/Speed/1e6-6 43.7MB/s ± 0% 48.3MB/s ± 0% +10.72% (p=0.000 n=10+9) Decode/Twain/Default/1e4-6 37.1MB/s ± 2% 39.8MB/s ± 2% +7.15% (p=0.000 n=9+10) Decode/Twain/Default/1e5-6 49.5MB/s ± 1% 54.3MB/s ± 1% +9.71% (p=0.000 n=10+10) Decode/Twain/Default/1e6-6 52.3MB/s ± 0% 57.3MB/s ± 1% +9.57% (p=0.000 n=9+10) Decode/Twain/Compression/1e4-6 36.7MB/s ± 1% 40.0MB/s ± 4% +8.96% (p=0.000 n=9+10) Decode/Twain/Compression/1e5-6 49.8MB/s ± 0% 54.5MB/s ± 1% +9.38% (p=0.000 n=9+10) Decode/Twain/Compression/1e6-6 52.3MB/s ± 0% 57.3MB/s ± 1% +9.58% (p=0.000 n=9+10) Change-Id: Iabfd285535ddb210f7f48f33317c6463b5532400 Reviewed-on: https://go-review.googlesource.com/102235 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-08-31compress/flate: remove non-standard extensions to flateJoe Tsai
Some constants were added to flate that seem to be an experimental attempt at increasing the window size. However, according to RFC1951, the largest window size is 32KiB, so these constants are non-standard. Delete them. Fixes #18458 Change-Id: Ia94989637ca031a56bce2548624fa48044caa7b9 Reviewed-on: https://go-review.googlesource.com/60490 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-03-01compress/flate: use math/bits.Reverse8/16 instead of local implementationRobert Griesemer
No measurable impact on performance (specifically, no degradation). Reverse is used in Huffman en/de-coding. For completeness, here are all the speed-related benchmark results: name old time/op new time/op delta Decode/Digits/Huffman/1e4-8 181µs ± 0% 178µs ± 1% ~ (p=0.100 n=3+3) Decode/Digits/Huffman/1e5-8 1.60ms ± 3% 1.56ms ± 3% ~ (p=0.400 n=3+3) Decode/Digits/Huffman/1e6-8 15.7ms ± 1% 15.3ms ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e4-8 179µs ± 0% 180µs ± 0% ~ (p=0.200 n=3+3) Decode/Digits/Speed/1e5-8 1.68ms ± 0% 1.66ms ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e6-8 16.6ms ± 2% 16.6ms ± 5% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e4-8 179µs ± 1% 178µs ± 1% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e5-8 1.62ms ± 3% 1.62ms ± 4% ~ (p=1.000 n=3+3) Decode/Digits/Default/1e6-8 16.0ms ± 2% 16.0ms ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e4-8 179µs ± 1% 179µs ± 0% ~ (p=0.200 n=3+3) Decode/Digits/Compression/1e5-8 1.62ms ± 2% 1.62ms ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e6-8 16.1ms ± 3% 16.0ms ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e4-8 205µs ± 2% 207µs ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e5-8 1.77ms ± 2% 1.77ms ± 4% ~ (p=0.700 n=3+3) Decode/Twain/Huffman/1e6-8 17.4ms ± 2% 17.4ms ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Speed/1e4-8 186µs ± 1% 186µs ± 1% ~ (p=0.400 n=3+3) Decode/Twain/Speed/1e5-8 1.53ms ± 2% 1.52ms ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Speed/1e6-8 14.9ms ± 1% 14.8ms ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Default/1e4-8 176µs ± 1% 174µs ± 0% ~ (p=0.200 n=3+3) Decode/Twain/Default/1e5-8 1.30ms ± 2% 1.31ms ± 1% ~ (p=0.700 n=3+3) Decode/Twain/Default/1e6-8 12.6ms ± 3% 12.5ms ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e4-8 177µs ± 0% 174µs ± 1% ~ (p=0.100 n=3+3) Decode/Twain/Compression/1e5-8 1.30ms ± 1% 1.31ms ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e6-8 12.5ms ± 1% 12.5ms ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Huffman/1e4-8 47.4µs ± 1% 46.5µs ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Huffman/1e5-8 453µs ± 2% 446µs ± 1% ~ (p=0.700 n=3+3) Encode/Digits/Huffman/1e6-8 4.44ms ± 3% 4.39ms ± 0% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e4-8 190µs ± 4% 185µs ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Speed/1e5-8 1.78ms ± 5% 1.75ms ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e6-8 17.9ms ± 7% 17.3ms ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e4-8 366µs ± 1% 361µs ± 0% ~ (p=0.200 n=3+3) Encode/Digits/Default/1e5-8 5.58ms ± 5% 5.44ms ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e6-8 59.0ms ± 3% 58.2ms ± 1% ~ (p=0.700 n=3+3) Encode/Digits/Compression/1e4-8 369µs ± 3% 362µs ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Compression/1e5-8 5.50ms ± 2% 5.47ms ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Compression/1e6-8 59.4ms ± 2% 58.5ms ± 1% ~ (p=0.400 n=3+3) Encode/Twain/Huffman/1e4-8 64.4µs ± 3% 64.7µs ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Huffman/1e5-8 526µs ± 1% 526µs ± 2% ~ (p=1.000 n=3+3) Encode/Twain/Huffman/1e6-8 5.18ms ± 2% 5.17ms ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Speed/1e4-8 206µs ± 1% 204µs ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e5-8 1.73ms ± 2% 1.70ms ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e6-8 16.7ms ± 0% 16.7ms ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e4-8 423µs ± 3% 418µs ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e5-8 6.34ms ± 4% 6.23ms ± 0% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e6-8 68.0ms ± 3% 67.5ms ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e4-8 435µs ± 3% 424µs ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e5-8 7.01ms ± 1% 6.92ms ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Compression/1e6-8 77.1ms ± 4% 75.5ms ± 1% ~ (p=0.400 n=3+3) name old speed new speed delta Decode/Digits/Huffman/1e4-8 55.2MB/s ± 0% 56.2MB/s ± 1% ~ (p=0.100 n=3+3) Decode/Digits/Huffman/1e5-8 62.4MB/s ± 3% 64.1MB/s ± 3% ~ (p=0.400 n=3+3) Decode/Digits/Huffman/1e6-8 63.8MB/s ± 1% 65.3MB/s ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e4-8 55.8MB/s ± 0% 55.4MB/s ± 0% ~ (p=0.200 n=3+3) Decode/Digits/Speed/1e5-8 59.6MB/s ± 0% 60.3MB/s ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e6-8 60.1MB/s ± 2% 60.3MB/s ± 4% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e4-8 55.8MB/s ± 1% 56.1MB/s ± 1% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e5-8 61.8MB/s ± 3% 61.7MB/s ± 4% ~ (p=1.000 n=3+3) Decode/Digits/Default/1e6-8 62.4MB/s ± 2% 62.4MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e4-8 55.7MB/s ± 1% 56.0MB/s ± 0% ~ (p=0.300 n=3+3) Decode/Digits/Compression/1e5-8 61.7MB/s ± 2% 61.9MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e6-8 62.2MB/s ± 3% 62.6MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e4-8 48.8MB/s ± 2% 48.4MB/s ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e5-8 56.4MB/s ± 2% 56.6MB/s ± 4% ~ (p=0.700 n=3+3) Decode/Twain/Huffman/1e6-8 57.6MB/s ± 2% 57.5MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Speed/1e4-8 53.7MB/s ± 1% 53.9MB/s ± 1% ~ (p=0.400 n=3+3) Decode/Twain/Speed/1e5-8 65.5MB/s ± 2% 65.6MB/s ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Speed/1e6-8 66.9MB/s ± 1% 67.4MB/s ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Default/1e4-8 56.9MB/s ± 1% 57.3MB/s ± 0% ~ (p=0.200 n=3+3) Decode/Twain/Default/1e5-8 77.2MB/s ± 2% 76.6MB/s ± 1% ~ (p=0.700 n=3+3) Decode/Twain/Default/1e6-8 79.3MB/s ± 3% 80.0MB/s ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e4-8 56.4MB/s ± 0% 57.5MB/s ± 1% ~ (p=0.100 n=3+3) Decode/Twain/Compression/1e5-8 76.8MB/s ± 1% 76.5MB/s ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e6-8 80.1MB/s ± 1% 79.8MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Huffman/1e4-8 211MB/s ± 1% 215MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Huffman/1e5-8 221MB/s ± 2% 224MB/s ± 1% ~ (p=0.700 n=3+3) Encode/Digits/Huffman/1e6-8 225MB/s ± 3% 228MB/s ± 0% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e4-8 52.8MB/s ± 4% 54.1MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Speed/1e5-8 56.2MB/s ± 5% 57.0MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e6-8 56.0MB/s ± 6% 57.7MB/s ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e4-8 27.3MB/s ± 1% 27.7MB/s ± 0% ~ (p=0.200 n=3+3) Encode/Digits/Default/1e5-8 17.9MB/s ± 4% 18.4MB/s ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e6-8 17.0MB/s ± 3% 17.2MB/s ± 1% ~ (p=0.500 n=3+3) Encode/Digits/Compression/1e4-8 27.1MB/s ± 3% 27.6MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Compression/1e5-8 18.2MB/s ± 2% 18.3MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Compression/1e6-8 16.9MB/s ± 2% 17.1MB/s ± 1% ~ (p=0.400 n=3+3) Encode/Twain/Huffman/1e4-8 155MB/s ± 3% 155MB/s ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Huffman/1e5-8 190MB/s ± 1% 190MB/s ± 2% ~ (p=1.000 n=3+3) Encode/Twain/Huffman/1e6-8 193MB/s ± 2% 193MB/s ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Speed/1e4-8 48.5MB/s ± 1% 49.1MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e5-8 57.7MB/s ± 2% 59.0MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e6-8 59.7MB/s ± 0% 59.7MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e4-8 23.6MB/s ± 3% 23.9MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e5-8 15.8MB/s ± 4% 16.1MB/s ± 0% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e6-8 14.7MB/s ± 3% 14.8MB/s ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e4-8 23.0MB/s ± 3% 23.6MB/s ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e5-8 14.3MB/s ± 1% 14.5MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Compression/1e6-8 13.0MB/s ± 4% 13.2MB/s ± 1% ~ (p=0.400 n=3+3) Measured on a "quiet" (no browser running) 2.3 GHz Intel Core i7, running macOS 10.12.3. See also #19279. Change-Id: Ice759eb34eb37442b543957447c264e0aadc1fa9 Reviewed-on: https://go-review.googlesource.com/37460 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-01-12compress/flate: avoid large stack growth in fillDeflateJoe Tsai
Ranging over an array causes the array to be copied over to the stack, which cause large re-growths. Instead, we should iterate over slices of the array. Also, assigning a large struct literal uses the stack even though the actual fields being populated are small in comparison to the entirety of the struct (see #18636). Fixing the stack growth does not alter CPU-time performance much since the stack-growth and copying was such a tiny portion of the compression work: name old time/op new time/op delta Encode/Digits/Default/1e4-8 332µs ± 1% 332µs ± 1% ~ (p=0.796 n=10+10) Encode/Digits/Default/1e5-8 5.07ms ± 2% 5.05ms ± 1% ~ (p=0.815 n=9+8) Encode/Digits/Default/1e6-8 53.7ms ± 1% 53.9ms ± 1% ~ (p=0.075 n=10+10) Encode/Twain/Default/1e4-8 380µs ± 1% 380µs ± 1% ~ (p=0.684 n=10+10) Encode/Twain/Default/1e5-8 5.79ms ± 2% 5.79ms ± 1% ~ (p=0.497 n=9+10) Encode/Twain/Default/1e6-8 61.5ms ± 1% 61.8ms ± 1% ~ (p=0.247 n=10+10) name old speed new speed delta Encode/Digits/Default/1e4-8 30.1MB/s ± 1% 30.1MB/s ± 1% ~ (p=0.753 n=10+10) Encode/Digits/Default/1e5-8 19.7MB/s ± 2% 19.8MB/s ± 1% ~ (p=0.795 n=9+8) Encode/Digits/Default/1e6-8 18.6MB/s ± 1% 18.5MB/s ± 1% ~ (p=0.072 n=10+10) Encode/Twain/Default/1e4-8 26.3MB/s ± 1% 26.3MB/s ± 1% ~ (p=0.616 n=10+10) Encode/Twain/Default/1e5-8 17.3MB/s ± 2% 17.3MB/s ± 1% ~ (p=0.484 n=9+10) Encode/Twain/Default/1e6-8 16.3MB/s ± 1% 16.2MB/s ± 1% ~ (p=0.238 n=10+10) Updates #18636 Fixes #18625 Change-Id: I471b20339bf675f63dc56d38b3acdd824fe23328 Reviewed-on: https://go-review.googlesource.com/35122 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-12compress/flate: add examplesJoe Tsai
Updates #16360 Change-Id: I66ff23e0501363f58fe891d5e95806422071f93b Reviewed-on: https://go-review.googlesource.com/30162 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-04all: sprinkle t.Parallel on some slow testsBrad Fitzpatrick
I used the slowtests.go tool as described in https://golang.org/cl/32684 on packages that stood out. go test -short std drops from ~56 to ~52 seconds. This isn't a huge win, but it was mostly an exercise. Updates #17751 Change-Id: I9f3402e36a038d71e662d06ce2c1d52f6c4b674d Reviewed-on: https://go-review.googlesource.com/32751 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-10-30compress/flate: tighten the BestSpeed max match offset bound.Nigel Tao
Previously, we were off by one. Also fix a comment typo. Change-Id: Ib94d23acc56d5fccd44144f71655481f98803ac8 Reviewed-on: https://go-review.googlesource.com/32149 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-16 08:01:16 +0000