aboutsummaryrefslogtreecommitdiff
path: root/src/archive/zip/reader_test.go
AgeCommit message (Collapse)Author
2026-01-15archive/zip: reduce CPU usage in index constructionDamien Neil
Constructing the zip index (which is done once when first opening a file in an archive) can consume large amounts of CPU when processing deeply-nested directory paths. Switch to a less inefficient algorithm. Thanks to Jakub Ciolek for reporting this issue. goos: darwin goarch: arm64 pkg: archive/zip cpu: Apple M4 Pro │ /tmp/bench.0 │ /tmp/bench.1 │ │ sec/op │ sec/op vs base │ ReaderOneDeepDir-14 25983.62m ± 2% 46.01m ± 2% -99.82% (p=0.000 n=8) ReaderManyDeepDirs-14 16.221 ± 1% 2.763 ± 6% -82.96% (p=0.000 n=8) ReaderManyShallowFiles-14 130.3m ± 1% 128.8m ± 2% -1.20% (p=0.003 n=8) geomean 3.801 253.9m -93.32% Fixes #77102 Fixes CVE-2025-61728 Change-Id: I2c9c864be01b2a2769eb67fbab1b250aeb8f6c42 Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3060 Reviewed-by: Nicholas Husin <husin@google.com> Reviewed-by: Neal Patel <nealpatel@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/736713 Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-10-17all: remove unnecessary loop variable copies in testsTobias Klauser
Copying the loop variable is no longer necessary since Go 1.22. Change-Id: Iebb21dac44a20ec200567f1d786f105a4ee4999d Reviewed-on: https://go-review.googlesource.com/c/go/+/711640 Reviewed-by: Florian Lehner <lehner.florian86@gmail.com> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-11archive/zip: error on ReadDir if there are invalid file namesSean Liao
Fixes #50179 Change-Id: I616a6d1279d025e345d2daa6d44b687c8a2d09e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/656495 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-07-25archive: use slices and maps to clean up testsapocelipes
Replace reflect.DeepEqual with slices.Equal/maps.Equal, which is much faster. Clean up some unnecessary helper functions. Change-Id: I9b94bd43886302b9b327539ab065a435ce0d75d9 GitHub-Last-Rev: b9ca21f165bcc5e45733e6a511a2344b1aa4a281 GitHub-Pull-Request: golang/go#67607 Reviewed-on: https://go-review.googlesource.com/c/go/+/587936 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2024-05-16archive/zip: treat truncated EOCDR comment as an errorDamien Neil
When scanning for an end of central directory record, treat an EOCDR signature with a record containing a truncated comment as an error. Previously, we would skip over the invalid record and look for another one. Other implementations do not do this (they either consider this a hard error, or just ignore the truncated comment). This parser misalignment allowed presenting entirely different archive contents to Go programs and other zip decoders. Fixes #66869 Change-Id: I94e5cb028534bb5704588b8af27f1e22ea49c7c6 Reviewed-on: https://go-review.googlesource.com/c/go/+/585397 Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-27all: make use of builtin clearJes Cok
Change-Id: I1df0685c75fc1044ba46003a69ecc7dfc53bbc2b Reviewed-on: https://go-review.googlesource.com/c/go/+/574675 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Than McIntosh <thanm@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-09-25all: use the indefinite article an in commentsJes Cok
Change-Id: I8787458f9ccd3b5cdcdda820d8a45deb4f77eade GitHub-Last-Rev: be865d67ef68815b8c1c2a9ad222fff594620e66 GitHub-Pull-Request: golang/go#63165 Reviewed-on: https://go-review.googlesource.com/c/go/+/530120 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-22all: clean unnecessary casts for test filesJes Cok
This is a follow up of CL 528696. Change-Id: I5b71eabedb12567c4b1b36f7182a3d2b0ed662a5 GitHub-Last-Rev: acaf3ac11c38042ad27b99e1c70a3c9f1a554a15 GitHub-Pull-Request: golang/go#62713 Reviewed-on: https://go-review.googlesource.com/c/go/+/529197 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-04-24archive/zip: reject overflowing directorySize & directoryOffsetRoland Shoemaker
We added a check for incorrect baseOffset in CL 408734, but in doing so we introduced a panic when directoryOffset overflowed a int64. The zip spec uses uint64, but since io.SectionReader requires int64 we convert, and possibly introduce an overflow. If offset < 0 && size-offset < 0, SectionReader will panic when we attempt to read from it. Since it's extremely unlikely we're ever going to process a zip file larger than 1<<63-1 byte, just limit directory size and offset to the max int64. Change-Id: I1aaa755cf4da927a6e12ef59f97dfc83a3426d86 Reviewed-on: https://go-review.googlesource.com/c/go/+/488195 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Roland Shoemaker <roland@golang.org>
2023-04-05archive/zip: return ErrInsecurePath for unsafe paths by OpenReaderImre Rad
zip.NewReader was recently improved to return ErrInsecurePath when insecure entries are encountered. This change adopts the same logic for the OpenReader interface as well. Fixes #58641 Change-Id: I0d8be94d073cc14cf93a914dc250f85b19cec4ab GitHub-Last-Rev: 68391dc51562aebc893ec70fdfbdfb181955983a GitHub-Pull-Request: golang/go#58658 Reviewed-on: https://go-review.googlesource.com/c/go/+/470735 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Damien Neil <dneil@google.com>
2023-01-20archive/zip: use base offset 0 if it has a valid entryIan Lance Taylor
In CL 408734 we introduced a fall back to base offset 0 if reading a directory entry at the computed base offset failed. We have now found a file in the wild for which the computed base offset is incorrect, but happens to refer to a valid directory entry. In this CL, we change the fallback such that if the first directory header relative to base offset 0 is valid, we just use base offset 0. Change-Id: Ia9ace20c1065d1f651035f16f7d91d741ab1dbf4 Reviewed-on: https://go-review.googlesource.com/c/go/+/461598 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Run-TryBot: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-12-06archive/zip: only consider UncompressedSize when checking dirsRoland Shoemaker
CL 454475 switched from checking CompressedSize to UncompressedSize when determining if we should consider an archive malformed because it contains data and added a test for an example of this (a JAR). We should also remove the hasDataDescriptor check, since that is basically an alias for CompressedSize > 0. The test didn't catch this because we didn't actually attempt to read from the returned reader. Change-Id: Ibc4c1aa9c3a733f3ebf4a956d1e2f8f4900a29cd Reviewed-on: https://go-review.googlesource.com/c/go/+/455523 Run-TryBot: Roland Shoemaker <roland@golang.org> Reviewed-by: Julie Qiu <julieqiu@google.com> Auto-Submit: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-12-01archive/zip: tolerate compressed directories with zero uncompressed sizeRoland Shoemaker
In CL 449955 we made reading of directories with associated file data an error, since it is a "must not" in the zip specification. It turns out that a number of implementations make the mistake of not setting the correct compression method on directories (in particular the Java jar tool does this when storing the META-INF directory). If the compression method used is not 0 (stored) then the compressed size of the directory can be > 0, despite the uncompressed size still being 0. Since this mistake is not uncommon, we are forced to tolerate it. We still fail if the recorded uncompressed size is > 0, which should be a significantly harder mistake to make. Change-Id: Ia732b10787f26ab937ac9cf9869ac3042efb8118 Reviewed-on: https://go-review.googlesource.com/c/go/+/454475 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org>
2022-11-21archive/tar, archive/zip: disable insecure file name checks with GODEBUGDamien Neil
Add GODEBUG=tarinsecurepath=1 and GODEBUG=zipinsecurepath=1 settings to disable file name validation. For #55356. Change-Id: Iaacdc629189493e7ea3537a81660215a59dd40a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/452495 Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-11-16archive/tar, archive/zip: return ErrInsecurePath for unsafe pathsDamien Neil
Return a distinguishable error when reading an archive file with a path that is: - absolute - escapes the current directory (../a) - on Windows, a reserved name such as NUL Users may ignore this error and proceed if they do not need name sanitization or intend to perform it themselves. Fixes #25849 Fixes #55356 Change-Id: Ieefa163f00384bc285ab329ea21a6561d39d8096 Reviewed-on: https://go-review.googlesource.com/c/go/+/449937 Reviewed-by: Joseph Tsai <joetsai@digital-static.net> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-11-14archive/zip: don't read directories containing file dataAlexander Yastrebov
Fixes #54801 Change-Id: I3d03516792975ddb09835b2621c57e12e7cbad35 GitHub-Last-Rev: 4faa7e14dcc48c05d707f1e137d915da24133e14 GitHub-Pull-Request: golang/go#56714 Reviewed-on: https://go-review.googlesource.com/c/go/+/449955 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org>
2022-06-02archive/zip: if non-zero base offset fails, fall back to zeroIan Lance Taylor
This permits us to read files that earlier Go releases could read. It is also compatible with other zip programs. Change-Id: I7e2999f1073c4db5ba3f51f92681e0b149d55b3e Reviewed-on: https://go-review.googlesource.com/c/go/+/408734 Run-TryBot: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-10archive/zip: error if using io/fs on zip with duplicate entriesIan Lance Taylor
Fixes #50390 Change-Id: I92787cdb3fa198ff88dcaadeccfcb49a3a6a88cf Reviewed-on: https://go-review.googlesource.com/c/go/+/374954 Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-08archive/zip: permit zip files to have prefixesIan Lance Taylor
A Java jar file is a zip file, but it can have a prefix that is a bash script that unpacks the zip file. Most zip programs ignore such prefixes. This CL changes the archive/zip package to do the same. Fixes #10464 Fixes #51337 Change-Id: I976e9c64684644317bd21077bc5b4a2baf626ee6 Reviewed-on: https://go-review.googlesource.com/c/go/+/387976 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-04-11archive/zip: fail fast if UncompressedSize64 < nreadMeng Zhuo
The zip reader checks that the uncompressed file size is valid after all compressed files read until EOF. However in between reading each file, there could have already been an overflow where nread > UncompressedSize64 hence this change will now return ErrFormat in such situations. Fixes #49791 Change-Id: If3584a57d173de6a97bf35c07d2a99ff6972f820 Reviewed-on: https://go-review.googlesource.com/c/go/+/366854 Trust: mzh <mzh@golangcn.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2022-04-01all: remove trailing blank doc comment linesRuss Cox
A future change to gofmt will rewrite // Doc comment. // func f() to // Doc comment. func f() Apply that change preemptively to all doc comments. For #51082. Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d Reviewed-on: https://go-review.googlesource.com/c/go/+/384259 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-11-07archive/zip: don't read data descriptor earlyArran Walker
Go 1.17 introduced an unnecessary change to when a zip's data descriptor is read for file entries, how it is parsed and how the crc32 field is used. Before Go 1.17, the data descriptor was read immediately after a file entry's content. This continuous read is a pattern existing applications have come to rely upon (for example, where reads at specific offsets might be translated to HTTP range requests). In Go 1.17, all data descriptors are immediately read upon opening the file. This results in scattered and non-continuous reads of the archive, and depending on the underlying reader, might have severe performance implications. In addition, an additional object is now initialized for each entry, but is mostly redundant. Previously, the crc32 field in the data descriptor would return an error if it did not match the central directory's entry. This check has seemingly been unintentionally removed. If the central directory crc32 is invalid and a data descriptor is present, no error is returned. This change reverts to the previous handling of data descriptors, before CL 312310. Fixes #48374 Fixes #49089 Change-Id: I5df2878c4fcc9e500064e7175f3ab9727c82f100 Reviewed-on: https://go-review.googlesource.com/c/go/+/357489 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-02archive/zip: don't panic on (*Reader).OpenJason7602
Previously, opening a zip with (*Reader).Open could result in a panic if the zip contained a file whose name was exclusively made up of slash characters or ".." path elements. Open could also panic if passed the empty string directly as an argument. Now, any files in the zip whose name could not be made valid for fs.FS.Open will be skipped, and no longer added to the fs.FS file list, although they are still accessible through (*Reader).File. Note that it was already the case that a file could be accessible from (*Reader).Open with a name different from the one in (*Reader).File, as the former is the cleaned name, while the latter is the original one. Finally, made the actual panic site robust as a defense-in-depth measure. Fixes CVE-2021-41772 Fixes #48085 Co-authored-by: Filippo Valsorda <filippo@golang.org> Change-Id: I6271a3f2892e7746f52e213b8eba9a1bba974678 Reviewed-on: https://go-review.googlesource.com/c/go/+/349770 Run-TryBot: Filippo Valsorda <filippo@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> Trust: Katie Hockman <katie@golang.org> Trust: Julie Qiu <julie@golang.org>
2021-08-20archive/zip: prevent preallocation check from overflowingRoland Shoemaker
If the indicated directory size in the archive header is so large that subtracting it from the archive size overflows a uint64, the check that the indicated number of files in the archive can be effectively bypassed. Prevent this from happening by checking that the indicated directory size is less than the size of the archive. Thanks to the OSS-Fuzz project for discovering this issue and to Emmanuel Odeke for reporting it. Fixes #47801 Fixes CVE-2021-39293 Change-Id: Ifade26b98a40f3b37398ca86bd5252d12394dd24 Reviewed-on: https://go-review.googlesource.com/c/go/+/343434 Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2021-05-25archive/zip: only preallocate File slice if reasonably sizedRoland Shoemaker
Since the number of files in the EOCD record isn't validated, it isn't safe to preallocate Reader.Files using that field. A malformed archive can indicate it contains up to 1 << 128 - 1 files. We can still safely preallocate the slice by checking if the specified number of files in the archive is reasonable, given the size of the archive. Thanks to the OSS-Fuzz project for discovering this issue and to Emmanuel Odeke for reporting it. Fixes #46242 Fixes CVE-2021-33196 Change-Id: I3c76d8eec178468b380d87fdb4a3f2cb06f0ee76 Reviewed-on: https://go-review.googlesource.com/c/go/+/318909 Trust: Roland Shoemaker <roland@golang.org> Trust: Katie Hockman <katie@golang.org> Trust: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-05-03archive/zip: add File.OpenRaw, Writer.CreateRaw, Writer.CopyEddie Scholtz
These new methods provide support for cases where performance is a primary concern. For example, copying files from an existing zip to a new zip without incurring the decompression and compression overhead. Using an optimized, external compression method and writing the output to a zip archive. And compressing file contents in parallel and then sequentially writing the compressed bytes to a zip archive. TestWriterCopy is copied verbatim from https://github.com/rsc/zipmerge Fixes #34974 Change-Id: Iade5bc245ba34cdbb86364bf59f79f38bb9e2eb6 Reviewed-on: https://go-review.googlesource.com/c/go/+/312310 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-04-29archive/zip: only return directory once via io/fs.FSIan Lance Taylor
While we're here fix the ModTime value for directories. Fixes #43872 Fixes #45345 Change-Id: I155e6517713ef6a9482b9431f1167a44337c6ad2 Reviewed-on: https://go-review.googlesource.com/c/go/+/311530 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org>
2021-04-18archive/zip: fix imports block of biggestZipBytes generatorPaschalis Tsilias
Fixes #45529 Change-Id: I4d64c40aa6733b783dc4066e222f17abeb7ad413 Reviewed-on: https://go-review.googlesource.com/c/go/+/309357 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Trust: Joe Tsai <thebrokentoaster@gmail.com> Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-03-10archive/zip: fix panic in Reader.OpenRoland Shoemaker
When operating on a Zip file that contains a file prefixed with "../", Open(...) would cause a panic in toValidName when attempting to strip the prefixed path components. Fixes CVE-2021-27919 Fixes #44916 Change-Id: Ic755d8126cb0897e2cbbdacf572439c38dde7b35 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004761 Reviewed-by: Filippo Valsorda <valsorda@google.com> Reviewed-by: Russ Cox <rsc@google.com> Reviewed-by: Katie Hockman <katiehockman@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/300489 Trust: Katie Hockman <katie@golang.org> Run-TryBot: Katie Hockman <katie@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20archive/zip: make Reader implement fs.FSRuss Cox
Now a zip.Reader (an open zip file) can be passed to code that accepts a file system, such as (soon) template parsing. For #41190. Change-Id: If51b12e39db3ccc27f643c2453d3300a38035360 Reviewed-on: https://go-review.googlesource.com/c/go/+/243937 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-10-20all: update references to symbols moved from os to io/fsRuss Cox
The old os references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. For #41190. Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd Reviewed-on: https://go-review.googlesource.com/c/go/+/243907 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2019-11-21all: base64-encode binaries that will cause Apple notarization to failAndrew
Starting with macOS 10.15 (Catalina), Apple now requires all software distributed outside of the App Store to be notarized. Any binaries we distribute must abide by a strict set of requirements like code-signing and having a minimum target SDK of 10.9 (amongst others). Apple’s notarization service will recursively inspect archives looking to find notarization candidate binaries. If it finds a binary that does not meet the requirements or is unable to decompress an archive, it will reject the entire distribution. From cursory testing, it seems that the service uses content sniffing to determine file types, so changing the file extension will not work. There are some binaries and archives included in our distribution that are being detected by Apple’s service as potential candidates for notarization or decompression. As these are files used by tests and some are intentionally invalid, we don’t intend to ever make them compliant. As a workaround for this, we base64-encode any binaries or archives that Apple’s notarization service issues a warning for, as these warnings will become errors in January 2020. Updates #34986 Change-Id: I106fbb6227b61eb221755568f047ee11103c1680 Reviewed-on: https://go-review.googlesource.com/c/go/+/208118 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-08-27archive/zip: remove unused special caseAndrew Gerrand
This removes a special case that was added to fix issue #10956, but that was never actually effective. The code in the test case still fails to read, so perhaps the zip64 support added in CL 6463050 inadvertently caught this particular case. It's possible that the original theorized bug still exists, but I'm not convinced it was ever fixed. Update #28700 Change-Id: I4854de616364510f64a6def30b308686563f8dbb Reviewed-on: https://go-review.googlesource.com/c/go/+/179757 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-21archive/zip: return error from NewReader when negative size is passedJeet Parekh
Fixes #26589 Change-Id: I180883a13cec229093654004b42c48d76ee20272 GitHub-Last-Rev: 2d9879de43fbcfb413116d69accdade6bc042c97 GitHub-Pull-Request: golang/go#26667 Reviewed-on: https://go-review.googlesource.com/126617 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-02-23archive/zip: fix handling of Info-ZIP Unix extended timestampsJoe Tsai
The Info-ZIP Unix1 extra field is specified as such: >>> Value Size Description ----- ---- ----------- 0x5855 Short tag for this extra block type ("UX") TSize Short total data size for this block AcTime Long time of last access (GMT/UTC) ModTime Long time of last modification (GMT/UTC) <<< The previous handling was incorrect in that it read the AcTime field instead of the ModTime field. The test-osx.zip test unfortunately locked in the wrong behavior. Manually parsing that ZIP file shows that the encoded MS-DOS date and time are 0x4b5f and 0xa97d, which corresponds with a date of 2017-10-31 21:11:58, which matches the correct mod time (off by 1 second due to MS-DOS timestamp resolution). Fixes #23901 Change-Id: I567824c66e8316b9acd103dbecde366874a4b7ef Reviewed-on: https://go-review.googlesource.com/96895 Run-TryBot: Joe Tsai <joetsai@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-12-01archive/zip: add test for Modified vs ModTime behaviorRuss Cox
Lock in fix for #22738, submitted in CL 78031. Fixes #22738. Change-Id: I6896feb158569e3f12fa7055387cbd7caad29ef4 Reviewed-on: https://go-review.googlesource.com/80635 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2017-11-06archive/zip: add FileHeader.NonUTF8 fieldJoe Tsai
The NonUTF8 field provides users with a way to explictly tell the ZIP writer to avoid setting the UTF-8 flag. This is necessary because many readers: 1) (Still) do not support UTF-8 2) And use the local system encoding instead Thus, even though character encodings other than CP-437 and UTF-8 are not officially supported by the ZIP specification, pragmatically the world has permitted use of them. When a non-standard encoding is used, it is the user's responsibility to ensure that the target system is expecting the encoding used (e.g., producing a ZIP file you know is used on a Chinese version of Windows). We adjust the detectUTF8 function to account for Shift-JIS and EUC-KR not being identical to ASCII for two characters. We don't need an API for users to explicitly specify that they are encoding with UTF-8 since all single byte characters are compatible with all other common encodings (Windows-1256, Windows-1252, Windows-1251, Windows-1250, IEC-8859, EUC-KR, KOI8-R, Latin-1, Shift-JIS, GB-2312, GBK) except for the non-printable characters and the backslash character (all of which are invalid characters in a path name anyways). Fixes #10741 Change-Id: I9004542d1d522c9137973f1b6e2b623fa54dfd66 Reviewed-on: https://go-review.googlesource.com/75592 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-11-06archive/zip: add FileHeader.Modified fieldJoe Tsai
The ModifiedTime and ModifiedDate fields are not expressive enough for many of the time extensions that have since been added to ZIP, nor are they easy to access since they in a legacy MS-DOS format, and must be set and retrieved via the SetModTime and ModTime methods. Instead, we add new field Modified of time.Time type that contains all of the previous information and more. Support for extended timestamps have been attempted before, but the change was reverted because it provided no ability for the user to specify the timezone of the legacy MS-DOS fields. Technically the old API did not either, but users were manually offsetting the timestamp to achieve the same effect. The Writer now writes the legacy timestamps according to the timezone of the FileHeader.Modified field. When the Modified field is set via the SetModTime method, it is in UTC, which preserves the old behavior. The Reader attempts to determine the timezone if both the legacy and extended timestamps are present since it can compute the delta between the two values. Since Modified is a superset of the information in ModifiedTime and ModifiedDate, we mark ModifiedTime, ModifiedDate, ModTime, and SetModTime as deprecated. Fixes #18359 Change-Id: I29c6bc0a62908095d02740df3e6902f50d3152f1 Reviewed-on: https://go-review.googlesource.com/74970 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-12-20Revert: "archive/zip: handle mtime in NTFS/UNIX/ExtendedTS extra fields"Joe Tsai
This change reverts the following CLs: CL/18274: handle mtime in NTFS/UNIX/ExtendedTS extra fields CL/30811: only use Extended Timestamp on non-zero MS-DOS timestamps We are reverting support for extended timestamps since the support was not not complete. CL/18274 added full support for reading extended timestamp fields and minimal support for writing them. CL/18274 is incomplete because it made no changes to the FileHeader struct, so timezone information was lost when reading and/or writing. While CL/18274 was a step in the right direction, we should provide full support for high precision timestamps in both the reader and writer. This will probably require that we add a new field of type time.Time. The complete fix is too involved to add in the time remaining for Go 1.8 and will be completed in Go 1.9. Updates #10242 Updates #17403 Updates #18359 Fixes #18378 Change-Id: Icf6d028047f69379f7979a29bfcb319a02f4783e Reviewed-on: https://go-review.googlesource.com/34651 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-06archive/zip: handle mtime in NTFS/UNIX/ExtendedTS extra fieldsYasuhiro Matsumoto
Handle NTFS timestamp, UNIX timestamp, Extended extra timestamp. Writer supports only Extended extra timestamp field, matching most zip creators. Fixes #10242. Change-Id: Id665db274e63def98659231391fb77392267ac1e Reviewed-on: https://go-review.googlesource.com/18274 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2016-03-15archive/zip: add missing argument to error messageAlberto Donizetti
Silence vet. Change-Id: I987438847389500cf3b5bc545ef918c66917b51a Reviewed-on: https://go-review.googlesource.com/20683 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-02-02archive/zip: handle pre-zip64 zip files containing 2³²-1-byte contentRuss Cox
This corrects a regression from Go 1.5 introduced by CL 18317. Fixes #14185. Change-Id: Ic3215714846d9f28809cd04e3eb3664b599244f4 Reviewed-on: https://go-review.googlesource.com/19151 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-09-11archive/zip: fixes unexpected EOF when reading archivealexander demakin
If comment of the archive contains data which looks like a zip64 directory, the comment is parsed as an actual directory header. Commit adds some additional checks similar to the checks in minizip library. Fixes #12449 Change-Id: Ia0fc950e47b9c39f77d88401b9ca30100ca7c808 Reviewed-on: https://go-review.googlesource.com/14433 Run-TryBot: Andrew Gerrand <adg@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-14archive/zip: fix returned error on truncated data descriptorMichael Gehring
Return io.ErrUnexpectedEOF instead of io.EOF when reading a truncated data descriptor. Fixes #11146. Change-Id: Ia1905955165fd38af3c557d1fa1703ed8be893e2 Reviewed-on: https://go-review.googlesource.com/11070 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-27archive/zip: sanity check the TOC's declared number of filesBrad Fitzpatrick
Fixes #10956 Change-Id: If8517094f04250c4f722e1e899a237eb6e170eb9 Reviewed-on: https://go-review.googlesource.com/10421 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-26archive/zip: verify number of File bytes read at EOFBrad Fitzpatrick
Fixes #10957 Change-Id: I75fe25133dfcebd1682a8058b1c354ec894cc997 Reviewed-on: https://go-review.googlesource.com/10384 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2014-09-08build: move package sources from src/pkg to srcRuss Cox
Preparation was in CL 134570043. This CL contains only the effect of 'hg mv src/pkg/* src'. For more about the move, see golang.org/s/go14nopkg.
generated by cgit v1.3 (git 2.53.0) at 2026-04-16 09:29:13 +0000