aboutsummaryrefslogtreecommitdiff
path: root/src/compress/bzip2
AgeCommit message (Collapse)Author
2026-02-02compress/bzip2: use obscuretestdata.ReadFile to avoid creating temporaryIan Lance Taylor
The temporary was not being deleted. Not creating it at all is better. Change-Id: Id9c5498a2f7d2f58770a09a2a76f4ab30943dd3d Reviewed-on: https://go-review.googlesource.com/c/go/+/740780 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-12-15archive/tar, compress/bzip2: base64 some troublesome testdata filesIan Lance Taylor
This avoids complaints from scanners that look for and open tar and bz2 files, and complain if they look weird. In this case, they do look weird, because they are intentionally strange. This kind of thing shouldn't be necessary, but we already have the machinery to do it so it's easy enough. Fixes #76799 Change-Id: Ib302b3aef30108a1325f91fcb2d166f8e1863792 Reviewed-on: https://go-review.googlesource.com/c/go/+/729780 Reviewed-by: David Chase <drchase@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2025-03-06compress/lzw,compress/gzip,compress/flate,compress/zlib,compress/bzip2: go ↵Olivier Mengué
doc links Add godoc links to compress/* package doc. Change-Id: I768ca250a39b0bb70eca35ac5b3b77ead73ca5f7 Reviewed-on: https://go-review.googlesource.com/c/go/+/655057 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2024-04-26compress: reordering fields to reduce struct sizesapocelipes
Overall, there are 32 bytes reduced. Change-Id: I455bf0874b33fa47719f42618e4800c7ff2a9e88 GitHub-Last-Rev: 7670344c4a643afdec0fdae3d34fdb8ccd81205f GitHub-Pull-Request: golang/go#67010 Reviewed-on: https://go-review.googlesource.com/c/go/+/581355 Reviewed-by: Joedian Reid <joedian@google.com> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
2024-04-15compress/bzip2: simplify Huffman tree constructionaimuz
This change simplifies the construction of the Huffman tree in the bzip2 package by replacing custom sort logic with the more concise and idiomatic use of "slices" and "cmp" packages. Change-Id: I2a8aef146b54b9433038b133d2cc8856ba077c72 GitHub-Last-Rev: c031bb56635552205c12b87add5f4337a7def74f GitHub-Pull-Request: golang/go#66817 Reviewed-on: https://go-review.googlesource.com/c/go/+/578438 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-18compress/bzip2,lzw: use built-in clear to simplify codeapocelipes
Change-Id: I16c17e322c757c8c657364065948d7cec66a8346 GitHub-Last-Rev: 9a5104fe9874dd7c604c526be3f082487f2aaf01 GitHub-Pull-Request: golang/go#66377 Reviewed-on: https://go-review.googlesource.com/c/go/+/572199 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Auto-Submit: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com> Reviewed-by: Jorropo <jorropo.pgm@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2023-10-13compress: some minor corrections or adjustments for godoc linkcui fliter
Change-Id: Ic1fc777b9a4e8262851e27646d53a0ec4fe55b64 Reviewed-on: https://go-review.googlesource.com/c/go/+/535076 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2023-10-12compress: add available godoc linkcui fliter
Change-Id: Ia6e88aec59cb294e8b303a00fcd69f4cbf0dc09a Reviewed-on: https://go-review.googlesource.com/c/go/+/534759 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: shuang cui <imcusg@gmail.com>
2023-09-01compress: linkify references to io.ByteReaderJoe Tsai
The special treatment of io.ByteReader is somewhat significant. Linkify references to that type from documentation. Change-Id: I77c06e4c6b957d107638e2ff2a4c58148b396ed3 Reviewed-on: https://go-review.googlesource.com/c/go/+/525016 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2023-06-16compress/bzip2: fix typoJes Cok
Remove redundant "this". Change-Id: Ia845e44119bf5ba1862f62da335466219a37c325 GitHub-Last-Rev: f0cdaf03285da8b053b71cc2911f5ab5beff580b GitHub-Pull-Request: golang/go#60807 Reviewed-on: https://go-review.googlesource.com/c/go/+/503655 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-04-05all: replace `` and '' with “ (U+201C) and ” (U+201D) in doc commentsRuss Cox
go/doc in all its forms applies this replacement when rendering the comments. We are considering formatting doc comments, including doing this replacement as part of the formatting. Apply it to our source files ahead of time. For #51082. Change-Id: Ifcc1f5861abb57c5d14e7d8c2102dfb31b7a3a19 Reviewed-on: https://go-review.googlesource.com/c/go/+/384262 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-06all: fix some typosDan Kortschak
Change-Id: I7dfae0fc91c2d70873ec7ec920be7c0a4888153a Reviewed-on: https://go-review.googlesource.com/c/go/+/390175 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Trust: Daniel Martí <mvdan@mvdan.cc>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-23all: fix quoting for compress/bzip2 and time's godocsubham sarkar
The existing usage of grave accent (`) and apostrophe (') at some places made godoc to ignore them and show it as it is. So, use both of the characters twice (consecutively) so that godoc can convert it to {left,right} double quotation mark. Fixes #41958 Change-Id: I64fd9b5fa34f416ad595009d09f5482e10bd8b4f Reviewed-on: https://go-review.googlesource.com/c/go/+/262397 Reviewed-by: Russ Cox <rsc@golang.org> Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-09-24compress: reduce copies of new text for compression testingKatie Hockman
The previous book was 387 KiB decompressed and 119 KiB compressed, the new book is 567 KiB decompressed and 132 KiB compressed. Overall, this change will reduce the release binary size by 196 KiB. The new book will allow for slightly more extensive compression testing with a larger text. Command to run the benchmark tests used with benchstat: `../bin/go test -run='^$' -count=4 -bench=. compress/bzip2 compress/flate` When running the benchmarks locally, changed "Newton" to "Twain" and filtered the tests with the -bench flag to include only those which were relevant to these changes. benchstat results below: name old time/op new time/op delta DecodeTwain-8 19.6ms ± 2% 24.1ms ± 1% +23.04% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 140µs ± 3% 139µs ± 5% ~ (p=0.886 n=4+4) Decode/Twain/Huffman/1e5-8 1.27ms ± 3% 1.26ms ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e6-8 12.4ms ± 0% 13.2ms ± 1% +6.42% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 133µs ± 1% 123µs ± 1% -7.35% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 1.20ms ± 0% 1.02ms ± 3% -15.32% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 12.0ms ± 2% 10.1ms ± 3% -15.89% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 131µs ± 6% 108µs ± 5% -17.84% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 1.06ms ± 2% 0.80ms ± 1% -24.97% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 10.0ms ± 3% 8.0ms ± 3% -20.06% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 128µs ± 4% 115µs ± 4% -9.70% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 1.04ms ± 2% 0.83ms ± 4% -20.37% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 10.4ms ± 4% 8.1ms ± 5% -22.25% (p=0.029 n=4+4) Encode/Twain/Huffman/1e4-8 55.7µs ± 2% 55.6µs ± 1% ~ (p=1.000 n=4+4) Encode/Twain/Huffman/1e5-8 441µs ± 0% 435µs ± 2% ~ (p=0.343 n=4+4) Encode/Twain/Huffman/1e6-8 4.31ms ± 4% 4.30ms ± 4% ~ (p=0.886 n=4+4) Encode/Twain/Speed/1e4-8 193µs ± 1% 166µs ± 2% -14.09% (p=0.029 n=4+4) Encode/Twain/Speed/1e5-8 1.54ms ± 1% 1.22ms ± 1% -20.53% (p=0.029 n=4+4) Encode/Twain/Speed/1e6-8 15.3ms ± 1% 12.2ms ± 3% -20.62% (p=0.029 n=4+4) Encode/Twain/Default/1e4-8 393µs ± 1% 390µs ± 1% ~ (p=0.114 n=4+4) Encode/Twain/Default/1e5-8 6.12ms ± 4% 6.02ms ± 5% ~ (p=0.486 n=4+4) Encode/Twain/Default/1e6-8 69.4ms ± 5% 59.0ms ± 4% -15.07% (p=0.029 n=4+4) Encode/Twain/Compression/1e4-8 423µs ± 2% 379µs ± 2% -10.34% (p=0.029 n=4+4) Encode/Twain/Compression/1e5-8 7.00ms ± 1% 7.88ms ± 3% +12.49% (p=0.029 n=4+4) Encode/Twain/Compression/1e6-8 76.6ms ± 5% 80.9ms ± 3% ~ (p=0.114 n=4+4) name old speed new speed delta DecodeTwain-8 19.8MB/s ± 2% 23.6MB/s ± 1% +18.84% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 71.7MB/s ± 3% 72.1MB/s ± 6% ~ (p=0.943 n=4+4) Decode/Twain/Huffman/1e5-8 78.8MB/s ± 3% 79.5MB/s ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e6-8 80.5MB/s ± 0% 75.6MB/s ± 1% -6.03% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 75.2MB/s ± 1% 81.2MB/s ± 1% +7.93% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 83.4MB/s ± 0% 98.6MB/s ± 3% +18.16% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 83.6MB/s ± 2% 99.5MB/s ± 3% +18.91% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 76.3MB/s ± 6% 92.8MB/s ± 4% +21.62% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 94.4MB/s ± 3% 125.7MB/s ± 1% +33.24% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 100MB/s ± 3% 125MB/s ± 3% +25.12% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 78.4MB/s ± 4% 86.8MB/s ± 4% +10.73% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 95.7MB/s ± 2% 120.3MB/s ± 4% +25.65% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 96.4MB/s ± 4% 124.0MB/s ± 5% +28.64% (p=0.029 n=4+4) Encode/Twain/Huffman/1e4-8 179MB/s ± 2% 180MB/s ± 1% ~ (p=1.000 n=4+4) Encode/Twain/Huffman/1e5-8 227MB/s ± 0% 230MB/s ± 2% ~ (p=0.343 n=4+4) Encode/Twain/Huffman/1e6-8 232MB/s ± 4% 233MB/s ± 4% ~ (p=0.886 n=4+4) Encode/Twain/Speed/1e4-8 51.8MB/s ± 1% 60.4MB/s ± 2% +16.43% (p=0.029 n=4+4) Encode/Twain/Speed/1e5-8 65.1MB/s ± 1% 81.9MB/s ± 1% +25.83% (p=0.029 n=4+4) Encode/Twain/Speed/1e6-8 65.2MB/s ± 1% 82.2MB/s ± 3% +26.00% (p=0.029 n=4+4) Encode/Twain/Default/1e4-8 25.4MB/s ± 1% 25.6MB/s ± 1% ~ (p=0.114 n=4+4) Encode/Twain/Default/1e5-8 16.4MB/s ± 4% 16.6MB/s ± 5% ~ (p=0.486 n=4+4) Encode/Twain/Default/1e6-8 14.4MB/s ± 6% 17.0MB/s ± 4% +17.67% (p=0.029 n=4+4) Encode/Twain/Compression/1e4-8 23.6MB/s ± 2% 26.4MB/s ± 2% +11.54% (p=0.029 n=4+4) Encode/Twain/Compression/1e5-8 14.3MB/s ± 1% 12.7MB/s ± 3% -11.08% (p=0.029 n=4+4) Encode/Twain/Compression/1e6-8 13.1MB/s ± 4% 12.4MB/s ± 3% ~ (p=0.114 n=4+4) name old alloc/op new alloc/op delta DecodeTwain-8 3.63MB ± 0% 3.63MB ± 0% +0.15% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 42.0kB ± 0% 41.3kB ± 0% -1.62% (p=0.029 n=4+4) Decode/Twain/Huffman/1e5-8 43.5kB ± 0% 45.1kB ± 0% +3.74% (p=0.029 n=4+4) Decode/Twain/Huffman/1e6-8 71.7kB ± 0% 80.0kB ± 0% +11.55% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 41.2kB ± 0% 41.3kB ± 0% ~ (p=0.286 n=4+4) Decode/Twain/Speed/1e5-8 45.1kB ± 0% 43.9kB ± 0% -2.80% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 72.8kB ± 0% 81.3kB ± 0% +11.72% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 41.2kB ± 0% 41.2kB ± 0% -0.22% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 44.4kB ± 0% 43.0kB ± 0% -3.02% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 71.0kB ± 0% 61.8kB ± 0% -13.00% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 41.3kB ± 0% 41.2kB ± 0% -0.29% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 43.3kB ± 0% 43.0kB ± 0% -0.72% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 69.1kB ± 0% 63.7kB ± 0% -7.90% (p=0.029 n=4+4) name old allocs/op new allocs/op delta DecodeTwain-8 51.0 ± 0% 51.2 ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e4-8 15.0 ± 0% 14.0 ± 0% -6.67% (p=0.029 n=4+4) Decode/Twain/Huffman/1e5-8 20.0 ± 0% 23.0 ± 0% +15.00% (p=0.029 n=4+4) Decode/Twain/Huffman/1e6-8 134 ± 0% 161 ± 0% +20.15% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 17.0 ± 0% 18.0 ± 0% +5.88% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 30.0 ± 0% 31.0 ± 0% +3.33% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 193 ± 0% 228 ± 0% +18.13% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 17.0 ± 0% 15.0 ± 0% -11.76% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 28.0 ± 0% 32.0 ± 0% +14.29% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 199 ± 0% 158 ± 0% -20.60% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 17.0 ± 0% 15.0 ± 0% -11.76% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 28.0 ± 0% 32.0 ± 0% +14.29% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 196 ± 0% 150 ± 0% -23.47% (p=0.029 n=4+4) Updates #27151 Change-Id: I6c439694ed16a33bb4c63fbfb8570c7de46b4f2d Reviewed-on: https://go-review.googlesource.com/135495 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-06-01all: update comment URLs from HTTP to HTTPS, where possibleTim Cooper
Each URL was manually verified to ensure it did not serve up incorrect content. Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df Reviewed-on: https://go-review.googlesource.com/115798 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-03-21compress/bzip2: remove bit-tricksIlya Tocar
Since compiler is now able to generate conditional moves, we can replace bit-tricks with simple if/else. This even results in slightly better performance: name old time/op new time/op delta DecodeDigits-6 13.4ms ± 4% 13.0ms ± 2% -2.63% (p=0.003 n=10+10) DecodeTwain-6 37.5ms ± 1% 36.3ms ± 1% -3.03% (p=0.000 n=10+9) DecodeRand-6 4.23ms ± 1% 4.07ms ± 1% -3.67% (p=0.000 n=10+9) name old speed new speed delta DecodeDigits-6 7.47MB/s ± 4% 7.67MB/s ± 2% +2.69% (p=0.002 n=10+10) DecodeTwain-6 10.4MB/s ± 1% 10.7MB/s ± 1% +3.25% (p=0.000 n=10+8) DecodeRand-6 3.87MB/s ± 1% 4.03MB/s ± 2% +4.08% (p=0.000 n=10+10) diff --git a/src/compress/bzip2/huffman.go b/src/compress/bzip2/huffman.go Change-Id: Ie96ef1a9e07013b07e78f22cdccd531f3341caca Reviewed-on: https://go-review.googlesource.com/102015 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> Reviewed-by: Joe Tsai <joetsai@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-02-17compress/bzip2: use sort.Slice in huffman.goJoe Kyo
Change-Id: Ie4d23cdb81473a4c989a977a127479cf825084dc Reviewed-on: https://go-review.googlesource.com/77850 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-09-25compress/bzip2: fix checksum mismatch on empty readsJoe Tsai
Previously, the read method checked whether the current block was fully consumed or not based on whether the buffer could be filled with a non-zero number of bytes. This check is problematic because zero bytes could be read if the provided buffer is empty. We fix this case by simply checking for whether the input buffer provided by the user was empty or not. If empty, we assume that we could not read any bytes because the buffer was too small, rather than indicating that the current block was fully exhausted. This check causes bzip2.Reader to be unable to make progress on the next block unless a non-empty buffer is provided. However, that is an entirely reasonable expectation since a non-empty buffer needs to be provided eventually anyways to read the actual contents of subsequent blocks. Fixes #22028 Change-Id: I2bb1b2d54e78567baf2bf7b490a272c0853d7bfe Reviewed-on: https://go-review.googlesource.com/66110 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-05-24compress/bzip2: remove dead code in huffman.goJoe Tsai
The logic performs a series of shifts, which are useless given that they are followed by an assignment that overrides the value of the previous computation. I suspect (but cannot prove) that this is leftover logic from an original approach that attempted to store both the Huffman code and the length within the same variable instead of using two different variables as it currently does now. Fixes #17949 Change-Id: Ibf6c807c6cef3b28bfdaf2b68d9bc13503ac21b2 Reviewed-on: https://go-review.googlesource.com/44091 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-03-04Revert "all: test adjustments for the iOS builder"Elias Naur
This reverts commit 467109bf56fb560d1fd8a27c6184dbfe4f64ffef. Replaced by a improved strategy later in the CL relation chain. Change-Id: Ib90813b5a6c4716b563c8496013d2d57f9c022b8 Reviewed-on: https://go-review.googlesource.com/36066 Reviewed-by: David Crawshaw <crawshaw@golang.org> Run-TryBot: Elias Naur <elias.naur@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-01-14all: test adjustments for the iOS builderDavid Crawshaw
The working directory is now adjusted to match the typical Go test working directory in main, as the old trick for adjusting earlier stopped working with the latest version of LLDB bugs. That means the small number of places where testdata files are read before main is called no longer work. This CL adjusts those reads to happen after main is called. (This has the bonus effect of not reading some benchmark testdata files in all.bash.) Fixes compress/bzip2, go/doc, go/parser, os, and time package tests on the iOS builder. Change-Id: If60f026aa7848b37511c36ac5e3985469ec25209 Reviewed-on: https://go-review.googlesource.com/35255 Run-TryBot: David Crawshaw <crawshaw@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-04-15all: remove unnecessary type conversionsMatthew Dempsky
cmd and runtime were handled separately, and I'm intentionally skipped syscall. This is the rest of the standard library. CL generated mechanically with github.com/mdempsky/unconvert. Change-Id: I9e0eff886974dedc37adb93f602064b83e469122 Reviewed-on: https://go-review.googlesource.com/22104 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-25all: delete dead non-test codeDominik Honnef
This change removes a lot of dead code. Some of the code has never been used, not even when it was first commited. The rest shouldn't have survived refactors. This change doesn't remove unused routines helpful for debugging, nor does it remove code that's used in commented out blocks of code that are only unused temporarily. Furthermore, unused constants weren't removed when they were part of a set of constants from specifications. One noteworthy omission from this CL are about 1000 lines of unused code in cmd/fix, 700 lines of which are the typechecker, which hasn't been used ever since the pre-Go 1 fixes have been removed. I wasn't sure if this code should stick around for future uses of cmd/fix or be culled as well. Change-Id: Ib714bc7e487edc11ad23ba1c3222d1fd02e4a549 Reviewed-on: https://go-review.googlesource.com/20926 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-03compress/bzip2: refactor unit testsJoe Tsai
Over the years as more bugs were discovered with the bzip2 library, new Tests were appended the unit tests and the tests became gnarly. Clean up the tests to be more consistent with modern Go style in addition to coalescing common tests into a general version that iterates over a list of input/output pairs. This has the advantage that the input, output, and test code are all in the same area, rather than being sprawled around the test file. There is no loss of test coverage. Change-Id: I377ed89378f0b89763d4a56ffc37b22d9c2a369e Reviewed-on: https://go-review.googlesource.com/20133 Run-TryBot: Joe Tsai <joetsai@digital-static.net> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-03-02compress/bzip2: prevent zero-length Huffman codesJoe Tsai
Unlike RFC 1951 (DEFLATE), bzip2 does not use zero-length Huffman codes to indicate that the symbol is missing. Instead, bzip2 uses a sparse bitmap to indicate which symbols are present. Thus, it is undefined what happens when a length of zero is used. Thus, fix the parsing logic so that the length cannot ever go below 1-bit similar to how the C logic does things. To confirm that the C bzip2 utility chokes on this data: $ echo "425a6836314159265359b1f7404b000000400040002000217d184682ee48 a70a12163ee80960" | xxd -r -p | bzip2 -d bzip2: Data integrity error when decompressing For reference see: bzip2-1.0.6/decompress.c:320 Change-Id: Ic1568f8e7f80cdea51d887b4d712cc239c2fe85e Reviewed-on: https://go-review.googlesource.com/20119 Run-TryBot: Joe Tsai <joetsai@digital-static.net> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-02-28compress/bzip2: fix benchmark to actually measure decompression rateJoe Tsai
Motivation: * Previously, the size of the compressed data was used for metrics, rather than the uncompressed size. This causes the library to appear to perform poorly relative to C or other implementation. Switch it to use the uncompressed size so that it matches how decompression benchmarks are usually done (like in compress/flate). This also makes it easier to compare bzip2 rates to other algorithms since they measure performance in this way. * Also, reset the timer after doing initialization work. Change-Id: I32112c2ee8e7391e658c9cf31039f70a689d9b9d Reviewed-on: https://go-review.googlesource.com/17611 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-28compress/bzip2: use correct block sizeJoe Tsai
The bzip2 block size is a multiple of 100*1000 not 100*1024. Thus, the bzip2 decoder would incorrectly decode files with larger block sizes when it should have otherwise failed. Fortunately, we can correct this in a backwards compatible way since Go has no implementation of a bzip2 encoder to produce bad blocks :) To confirm that the C bzip2 utlity chokes on this data: $ echo "425a683131415926535936dc55330063ffc0006000200020a40830008b00 08b8bb9229c28481b6e2a998" | xxd -r -p | bzip2 -d bzip2: Data integrity error when decompressing. Fixes #13941 Change-Id: I2402e8829a8027ef94dd4fac050b200440a3d4e4 Reviewed-on: https://go-review.googlesource.com/20011 Run-TryBot: Joe Tsai <joetsai@digital-static.net> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-12-02compress/bzip2/testdata: make Mark.Twain-Tom.Sawyer.txt freeJoe Tsai
Commit 7a1fb95d50b7a8302445ebc1296010695151ce7b strips non-free license from Mark.Twain-Tom.Sawyer.txt, but forgot to remove it from the compressed version of the file. Update #13216 Change-Id: I60f53275d56ba5baa6898db47b1d41f85e985c00 Reviewed-on: https://go-review.googlesource.com/17264 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-11-24compress: make Mark.Twain-Tom.Sawyer.txt licensed under non-free license ↵Jakub Čajka
free again This change strips non-free license from Mark.Twain-Tom.Sawyer.txt along with all reference to Project Gutenberg in the file and the whole source tree. Making the file public domain again. Fixes #13216 Change-Id: I2f41b0de225f627dde152efe93c006a4c24be668 Reviewed-on: https://go-review.googlesource.com/17196 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-08-28compress/bzip2: make decoding fasterAlberto Donizetti
Issue 6754 reports that Go bzip2 Decode function is much slower (about 2.5x in go1.5) than the Python equivalent (which is actually just a wrapper around the usual C library) on random data. Profiling the code shows that half a dozen of CMP instructions in a tight loop are responsibile for most of the execution time. This patch reduces the number of branches of the loop, greatly improving performance on random data and speeding up decoding of real data. name old time/op new time/op delta DecodeDigits-4 9.28ms ± 1% 8.05ms ± 1% -13.18% (p=0.000 n=15+14) DecodeTwain-4 28.9ms ± 2% 26.4ms ± 1% -8.57% (p=0.000 n=15+14) DecodeRand-4 3.94ms ± 1% 3.06ms ± 1% -22.45% (p=0.000 n=15+14) name old speed new speed delta DecodeDigits-4 4.65MB/s ± 1% 5.36MB/s ± 1% +15.21% (p=0.000 n=13+14) DecodeTwain-4 4.32MB/s ± 2% 4.72MB/s ± 1% +9.36% (p=0.000 n=15+14) DecodeRand-4 4.27MB/s ± 1% 5.51MB/s ± 1% +28.86% (p=0.000 n=15+14) I've run some benchmark comparing Go bzip2 implementation with the usual Linux bzip2 command (which is written in C). On my machine this patch brings go1.5 from ~2.26x to ~1.50x of bzip2 time (on 64MB random data) from ~1.70x to ~1.50x of bzip2 time (on 100MB english text) from ~2.00x to ~1.88x of bzip2 time (on 64MB /dev/zero data) Fixes #6754 Change-Id: I3cb12d2c0c2243c1617edef1edc88f05f91d26d1 Reviewed-on: https://go-review.googlesource.com/13853 Reviewed-by: Nigel Tao <nigeltao@golang.org>
2015-02-06all: don't refer to code.google.com/p/go{,-wiki}/Péter Surányi
Only documentation / comment changes. Update references to point to golang.org permalinks or go.googlesource.com/go. References in historical release notes under doc are left as is. Change-Id: Icfc14e4998723e2c2d48f9877a91c5abef6794ea Reviewed-on: https://go-review.googlesource.com/4060 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2014-12-19compress/bzip2: s/repeat_power/repeatPower/David Crawshaw
Change-Id: I64c8c247acd5d134b2f17ed7aab0a035d7710679 Reviewed-on: https://go-review.googlesource.com/1804 Reviewed-by: Minux Ma <minux@golang.org>
2014-09-30compress/*: note that NewReader may introduce bufferingRuss Cox
Fixes #8309. LGTM=r R=golang-codereviews, r CC=golang-codereviews, iant https://golang.org/cl/147380043
2014-09-08build: move package sources from src/pkg to srcRuss Cox
Preparation was in CL 134570043. This CL contains only the effect of 'hg mv src/pkg/* src'. For more about the move, see golang.org/s/go14nopkg.