aboutsummaryrefslogtreecommitdiff
path: root/src/io
AgeCommit message (Collapse)Author
13 daysio/fs: fix godoc to refer to the 'fsys' argOlivier Mengué
Fix godoc for package io/fs functions where 'fs' is mentioned instead of the 'fsys' argument. This allows to more clearly distinguish references to the methods of that argument from functions of the io/fs package. Change-Id: I18674940e59dcf4501f46ee48f94fc58948df28c Reviewed-on: https://go-review.googlesource.com/c/go/+/755480 Auto-Submit: Alan Donovan <adonovan@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Damien Neil <dneil@google.com>
2026-03-10io/fs: document when ErrClosed should be returnedkovan
Pull ErrClosed out of the grouped var declaration so it can have its own doc comment explaining that file system implementations should return it when operations are attempted on a closed file. Fixes #63918 Change-Id: I1fe2041c594ad6fd37e240e21fe9b3d4104daa46 Reviewed-on: https://go-review.googlesource.com/c/go/+/741446 Reviewed-by: Ian Lance Taylor <iant@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
2026-02-06io/fs: document that Sub does not check for existenceSean Liao
Also recommend os.Root for constraining access. Fixes #77447 Change-Id: Id8d178163438e9264fe19ed37ba03e1ee5d0bd35 Reviewed-on: https://go-review.googlesource.com/c/go/+/742284 Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-11-26io: reduce intermediate allocations in ReadAll and have a smaller final resultthepudds
Currently, io.ReadAll allocates a significant amount of intermediate memory as it grows its result slice to the size of the input data. This CL aims to reduce the allocated memory. Geomean benchstat results comparing existing io.ReadAll to this CL for a variety of input sizes: │ old | new vs base │ sec/op 132.2µ 66.32µ -49.83% B/op 645.4Ki 324.6Ki -49.70% final-capacity 178.3k 151.3k -15.10% excess-ratio 1.216 1.033 -15.10% The corresponding full benchstat results are below. The input data sizes are a blend of random sizes, power-of-2 sizes, and power-of-10 sizes. This CL reduces intermediate bytes allocated in io.ReadAll by reading via a set of slices of exponentially growing size, and then copying into a final perfectly-sized slice at the end. The current memory allocations impact real uses. For example, in #50774 two real-world reports were ~60% more bytes allocated via io.ReadAll compared to an alternate approach, and also a separate report of ~5x more bytes allocated than the input data size of ~5MiB. Separately, bytes.Buffer.ReadFrom uses a 2x growth strategy, which usually can beat the pre-existing io.ReadAll on total bytes allocated but sometimes not (depending on alignment between exact input data size and growth). That said, bytes.Buffer.ReadFrom usually ends up with more excess memory used in a larger final result than the current io.ReadAll (often significantly more). If we compare bytes.Buffer.ReadFrom to this CL, we also see better geomean overall results reported with this CL: │ bytes.Buffer | io.ReadAll (new) │ sec/op 104.6µ 66.32µ -36.60% B/op 466.9Ki 324.6Ki -30.48% final-capacity 247.4k 151.3k -38.84% excess-ratio 1.688 1.033 -38.84% (Full corresponding benchstat results comparing this CL vs. bytes.Buffer are at https://go.dev/play/p/eqwk2BkaSwJ). One challenge with almost any change of growth strategy for something widely used is there can be a subset of users that benefited more from the old growth approach (e.g., based on their data size aligning particularly well with the old growth), even if the majority of users on average benefit from the new growth approach. To help mitigate that, this CL somewhat follows the old read pattern in its early stages. Here are the full benchstat results comparing the existing io.ReadAll vs. this CL. The standard metrics are included, plus the final result capacity and an excess capacity ratio, which is the final capacity of the result divided by the input data size (so 1.0 is no excess memory present in the result, though due to size class rounding the ratio is usually above 1.0 unless the input data size exactly matches a size class). We consider smaller reported excess capacity to be better for most uses given it means the final allocation puts less pressure on the GC (both in cases when it will almost immediately be garbage in user code, or if for example the final result is held for multiple GC cycles). The input data sizes used in the benchmarks: - Six powers of 10. - Six powers of 2. - Ten random sizes between 1KiB and 100MiB (chosen uniformly on a log scale). - size=300 (so that we have something below 512, which is the initial read size). goos: linux goarch: amd64 pkg: io cpu: AMD EPYC 7B13 │ old │ io.ReadAll (new) │ │ sec/op │ sec/op vs base │ ReadAll/size=300-16 113.0n ± 0% 115.4n ± 2% +2.08% (p=0.005 n=20) ReadAll/size=512-16 295.0n ± 2% 288.7n ± 1% -2.14% (p=0.006 n=20) ReadAll/size=1000-16 549.2n ± 1% 492.8n ± 1% -10.28% (p=0.000 n=20) ReadAll/size=4096-16 3.193µ ± 1% 2.277µ ± 1% -28.70% (p=0.000 n=20) ReadAll/size=6648-16 4.318µ ± 1% 3.100µ ± 1% -28.21% (p=0.000 n=20) ReadAll/size=10000-16 7.771µ ± 1% 4.629µ ± 1% -40.43% (p=0.000 n=20) ReadAll/size=12179-16 7.724µ ± 1% 5.066µ ± 1% -34.42% (p=0.000 n=20) ReadAll/size=16384-16 13.664µ ± 1% 7.309µ ± 1% -46.51% (p=0.000 n=20) ReadAll/size=32768-16 24.07µ ± 2% 14.52µ ± 2% -39.67% (p=0.000 n=20) ReadAll/size=65536-16 43.14µ ± 2% 24.00µ ± 2% -44.37% (p=0.000 n=20) ReadAll/size=80000-16 57.12µ ± 2% 31.28µ ± 2% -45.24% (p=0.000 n=20) ReadAll/size=100000-16 75.08µ ± 2% 38.18µ ± 3% -49.15% (p=0.000 n=20) ReadAll/size=118014-16 76.06µ ± 1% 50.03µ ± 3% -34.22% (p=0.000 n=20) ReadAll/size=131072-16 103.99µ ± 1% 52.31µ ± 2% -49.70% (p=0.000 n=20) ReadAll/size=397601-16 518.1µ ± 6% 204.2µ ± 2% -60.58% (p=0.000 n=20) ReadAll/size=626039-16 934.9µ ± 3% 398.7µ ± 7% -57.35% (p=0.000 n=20) ReadAll/size=1000000-16 1800.3µ ± 8% 651.4µ ± 6% -63.82% (p=0.000 n=20) ReadAll/size=1141838-16 2236.3µ ± 5% 710.2µ ± 5% -68.24% (p=0.000 n=20) ReadAll/size=2414329-16 4.517m ± 3% 1.471m ± 3% -67.43% (p=0.000 n=20) ReadAll/size=5136407-16 8.547m ± 3% 2.060m ± 1% -75.90% (p=0.000 n=20) ReadAll/size=10000000-16 13.303m ± 4% 3.767m ± 4% -71.68% (p=0.000 n=20) ReadAll/size=18285584-16 23.414m ± 2% 6.790m ± 5% -71.00% (p=0.000 n=20) ReadAll/size=67379426-16 55.93m ± 4% 24.50m ± 5% -56.20% (p=0.000 n=20) ReadAll/size=100000000-16 84.61m ± 5% 33.84m ± 5% -60.00% (p=0.000 n=20) geomean 132.2µ 66.32µ -49.83% │ old │ io.ReadAll (new) │ │ B/op │ B/op vs base │ ReadAll/size=300-16 512.0 ± 0% 512.0 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=512-16 1.375Ki ± 0% 1.250Ki ± 0% -9.09% (p=0.000 n=20) ReadAll/size=1000-16 2.750Ki ± 0% 2.125Ki ± 0% -22.73% (p=0.000 n=20) ReadAll/size=4096-16 17.00Ki ± 0% 10.12Ki ± 0% -40.44% (p=0.000 n=20) ReadAll/size=6648-16 23.75Ki ± 0% 15.75Ki ± 0% -33.68% (p=0.000 n=20) ReadAll/size=10000-16 45.00Ki ± 0% 23.88Ki ± 0% -46.94% (p=0.000 n=20) ReadAll/size=12179-16 45.00Ki ± 0% 25.88Ki ± 0% -42.50% (p=0.000 n=20) ReadAll/size=16384-16 82.25Ki ± 0% 36.88Ki ± 0% -55.17% (p=0.000 n=20) ReadAll/size=32768-16 150.25Ki ± 0% 78.88Ki ± 0% -47.50% (p=0.000 n=20) ReadAll/size=65536-16 278.3Ki ± 0% 134.9Ki ± 0% -51.53% (p=0.000 n=20) ReadAll/size=80000-16 374.3Ki ± 0% 190.9Ki ± 0% -49.00% (p=0.000 n=20) ReadAll/size=100000-16 502.3Ki ± 0% 214.9Ki ± 0% -57.22% (p=0.000 n=20) ReadAll/size=118014-16 502.3Ki ± 0% 286.9Ki ± 0% -42.88% (p=0.000 n=20) ReadAll/size=131072-16 670.3Ki ± 0% 294.9Ki ± 0% -56.01% (p=0.000 n=20) ReadAll/size=397601-16 1934.3Ki ± 0% 919.8Ki ± 0% -52.45% (p=0.000 n=20) ReadAll/size=626039-16 3.092Mi ± 0% 1.359Mi ± 0% -56.04% (p=0.000 n=20) ReadAll/size=1000000-16 4.998Mi ± 0% 2.086Mi ± 0% -58.27% (p=0.000 n=20) ReadAll/size=1141838-16 6.334Mi ± 0% 2.219Mi ± 0% -64.98% (p=0.000 n=20) ReadAll/size=2414329-16 12.725Mi ± 0% 4.789Mi ± 0% -62.37% (p=0.000 n=20) ReadAll/size=5136407-16 25.28Mi ± 0% 10.44Mi ± 0% -58.71% (p=0.000 n=20) ReadAll/size=10000000-16 49.84Mi ± 0% 21.92Mi ± 0% -56.02% (p=0.000 n=20) ReadAll/size=18285584-16 97.88Mi ± 0% 35.99Mi ± 0% -63.23% (p=0.000 n=20) ReadAll/size=67379426-16 375.2Mi ± 0% 158.0Mi ± 0% -57.91% (p=0.000 n=20) ReadAll/size=100000000-16 586.7Mi ± 0% 235.9Mi ± 0% -59.80% (p=0.000 n=20) geomean 645.4Ki 324.6Ki -49.70% │ old │ io.ReadAll (new) │ │ final-cap │ final-cap vs base │ ReadAll/size=300-16 512.0 ± 0% 512.0 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=512-16 896.0 ± 0% 512.0 ± 0% -42.86% (p=0.000 n=20) ReadAll/size=1000-16 1.408k ± 0% 1.024k ± 0% -27.27% (p=0.000 n=20) ReadAll/size=4096-16 5.376k ± 0% 4.096k ± 0% -23.81% (p=0.000 n=20) ReadAll/size=6648-16 6.912k ± 0% 6.784k ± 0% -1.85% (p=0.000 n=20) ReadAll/size=10000-16 12.29k ± 0% 10.24k ± 0% -16.67% (p=0.000 n=20) ReadAll/size=12179-16 12.29k ± 0% 12.29k ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=16384-16 21.76k ± 0% 16.38k ± 0% -24.71% (p=0.000 n=20) ReadAll/size=32768-16 40.96k ± 0% 32.77k ± 0% -20.00% (p=0.000 n=20) ReadAll/size=65536-16 73.73k ± 0% 65.54k ± 0% -11.11% (p=0.000 n=20) ReadAll/size=80000-16 98.30k ± 0% 81.92k ± 0% -16.67% (p=0.000 n=20) ReadAll/size=100000-16 131.1k ± 0% 106.5k ± 0% -18.75% (p=0.000 n=20) ReadAll/size=118014-16 131.1k ± 0% 122.9k ± 0% -6.25% (p=0.000 n=20) ReadAll/size=131072-16 172.0k ± 0% 131.1k ± 0% -23.81% (p=0.000 n=20) ReadAll/size=397601-16 442.4k ± 0% 401.4k ± 0% -9.26% (p=0.000 n=20) ReadAll/size=626039-16 704.5k ± 0% 630.8k ± 0% -10.47% (p=0.000 n=20) ReadAll/size=1000000-16 1.114M ± 0% 1.008M ± 0% -9.56% (p=0.000 n=20) ReadAll/size=1141838-16 1.401M ± 0% 1.147M ± 0% -18.13% (p=0.000 n=20) ReadAll/size=2414329-16 2.753M ± 0% 2.417M ± 0% -12.20% (p=0.000 n=20) ReadAll/size=5136407-16 5.399M ± 0% 5.145M ± 0% -4.70% (p=0.000 n=20) ReadAll/size=10000000-16 10.56M ± 0% 10.00M ± 0% -5.28% (p=0.000 n=20) ReadAll/size=18285584-16 20.65M ± 0% 18.29M ± 0% -11.42% (p=0.000 n=20) ReadAll/size=67379426-16 78.84M ± 0% 67.39M ± 0% -14.53% (p=0.000 n=20) ReadAll/size=100000000-16 123.2M ± 0% 100.0M ± 0% -18.82% (p=0.000 n=20) geomean 178.3k 151.3k -15.10% │ old │ io.ReadAll (new) │ │ excess-ratio │ excess-ratio vs base │ ReadAll/size=300-16 1.707 ± 0% 1.707 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=512-16 1.750 ± 0% 1.000 ± 0% -42.86% (p=0.000 n=20) ReadAll/size=1000-16 1.408 ± 0% 1.024 ± 0% -27.27% (p=0.000 n=20) ReadAll/size=4096-16 1.312 ± 0% 1.000 ± 0% -23.78% (p=0.000 n=20) ReadAll/size=6648-16 1.040 ± 0% 1.020 ± 0% -1.92% (p=0.000 n=20) ReadAll/size=10000-16 1.229 ± 0% 1.024 ± 0% -16.68% (p=0.000 n=20) ReadAll/size=12179-16 1.009 ± 0% 1.009 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=16384-16 1.328 ± 0% 1.000 ± 0% -24.70% (p=0.000 n=20) ReadAll/size=32768-16 1.250 ± 0% 1.000 ± 0% -20.00% (p=0.000 n=20) ReadAll/size=65536-16 1.125 ± 0% 1.000 ± 0% -11.11% (p=0.000 n=20) ReadAll/size=80000-16 1.229 ± 0% 1.024 ± 0% -16.68% (p=0.000 n=20) ReadAll/size=100000-16 1.311 ± 0% 1.065 ± 0% -18.76% (p=0.000 n=20) ReadAll/size=118014-16 1.111 ± 0% 1.041 ± 0% -6.30% (p=0.000 n=20) ReadAll/size=131072-16 1.312 ± 0% 1.000 ± 0% -23.78% (p=0.000 n=20) ReadAll/size=397601-16 1.113 ± 0% 1.010 ± 0% -9.25% (p=0.000 n=20) ReadAll/size=626039-16 1.125 ± 0% 1.008 ± 0% -10.40% (p=0.000 n=20) ReadAll/size=1000000-16 1.114 ± 0% 1.008 ± 0% -9.52% (p=0.000 n=20) ReadAll/size=1141838-16 1.227 ± 0% 1.004 ± 0% -18.17% (p=0.000 n=20) ReadAll/size=2414329-16 1.140 ± 0% 1.001 ± 0% -12.19% (p=0.000 n=20) ReadAll/size=5136407-16 1.051 ± 0% 1.002 ± 0% -4.66% (p=0.000 n=20) ReadAll/size=10000000-16 1.056 ± 0% 1.000 ± 0% -5.30% (p=0.000 n=20) ReadAll/size=18285584-16 1.129 ± 0% 1.000 ± 0% -11.43% (p=0.000 n=20) ReadAll/size=67379426-16 1.170 ± 0% 1.000 ± 0% -14.53% (p=0.000 n=20) ReadAll/size=100000000-16 1.232 ± 0% 1.000 ± 0% -18.83% (p=0.000 n=20) geomean 1.216 1.033 -15.10% │ io.ReadAll │ io.ReadAll (new) │ │ allocs/op │ allocs/op vs base │ ReadAll/size=300-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=512-16 2.000 ± 0% 3.000 ± 0% +50.00% (p=0.000 n=20) ReadAll/size=1000-16 3.000 ± 0% 4.000 ± 0% +33.33% (p=0.000 n=20) ReadAll/size=4096-16 7.000 ± 0% 9.000 ± 0% +28.57% (p=0.000 n=20) ReadAll/size=6648-16 8.000 ± 0% 10.000 ± 0% +25.00% (p=0.000 n=20) ReadAll/size=10000-16 10.00 ± 0% 11.00 ± 0% +10.00% (p=0.000 n=20) ReadAll/size=12179-16 10.00 ± 0% 11.00 ± 0% +10.00% (p=0.000 n=20) ReadAll/size=16384-16 12.00 ± 0% 13.00 ± 0% +8.33% (p=0.000 n=20) ReadAll/size=32768-16 14.00 ± 0% 15.00 ± 0% +7.14% (p=0.000 n=20) ReadAll/size=65536-16 16.00 ± 0% 16.00 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=80000-16 17.00 ± 0% 17.00 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=100000-16 18.00 ± 0% 17.00 ± 0% -5.56% (p=0.000 n=20) ReadAll/size=118014-16 18.00 ± 0% 18.00 ± 0% ~ (p=1.000 n=20) ¹ ReadAll/size=131072-16 19.00 ± 0% 18.00 ± 0% -5.26% (p=0.000 n=20) ReadAll/size=397601-16 23.00 ± 0% 22.00 ± 0% -4.35% (p=0.000 n=20) ReadAll/size=626039-16 25.00 ± 0% 23.00 ± 0% -8.00% (p=0.000 n=20) ReadAll/size=1000000-16 27.00 ± 0% 24.00 ± 0% -11.11% (p=0.000 n=20) ReadAll/size=1141838-16 28.00 ± 0% 24.00 ± 0% -14.29% (p=0.000 n=20) ReadAll/size=2414329-16 31.00 ± 0% 26.00 ± 0% -16.13% (p=0.000 n=20) ReadAll/size=5136407-16 34.00 ± 0% 28.00 ± 0% -17.65% (p=0.000 n=20) ReadAll/size=10000000-16 37.00 ± 0% 30.00 ± 0% -18.92% (p=0.000 n=20) ReadAll/size=18285584-16 40.00 ± 0% 31.00 ± 0% -22.50% (p=0.000 n=20) ReadAll/size=67379426-16 46.00 ± 0% 35.00 ± 0% -23.91% (p=0.000 n=20) ReadAll/size=100000000-16 48.00 ± 0% 36.00 ± 0% -25.00% (p=0.000 n=20) geomean 14.89 14.65 -1.65% Finally, the read size in this CL currently grows exponentially at a 1.5x growth rate. The old approach had its read size grow at a ~1.25x growth rate once the reads are larger. We could consider for example using a 1.25x read size growth rate here as well. There are perhaps some ~mild trade-offs. One benefit might be something like a ~5% smaller peak live heap contribution (at the end, when copying into the final result) if we used a 1.25x read growth instead of 1.5x read growth. That said, for some systems, larger read sizes can trigger higher throughput behavior further down the stack or elsewhere in a system, such as via better read-ahead behavior, larger transfer sizes, etc. I've observed this effect in various real-world systems, including distributed systems as well as for example with spinning platters (which are still widely used, including backing various "Internet scale" systems). When the effect exists, it is usually substantial. Therefore, my guess is it is better to get to larger read sizes faster, which is one reason the CL is using 1.5x read size growth rate instead of 1.25x. Also, for something like peak live heap contribution, we are already getting substantial wins in this CL for total heap bytes allocated, so maybe that is OK. (I have the actual benchmark in a separate commit, which I can send later, or I can update this CL if preferred). Fixes #50774 Updates #74299 Change-Id: I65eabf1d83a00fbdbe42e4c697116955f8251740 Reviewed-on: https://go-review.googlesource.com/c/go/+/722500 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-10-13all: replace calls to errors.As with errors.AsTypeJulien Cretel
This change replaces most occurrences (in code as well as in comments) of errors.As with errors.AsType. It leaves the errors package and vendored code untouched. Change-Id: I3bde73f318a0b408bdb8f5a251494af15a13118a GitHub-Last-Rev: 8aaaa36a5a12d2a6a90c6d51680464e1a3115139 GitHub-Pull-Request: golang/go#75698 Reviewed-on: https://go-review.googlesource.com/c/go/+/708495 Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-10-10all: omit unnecessary reassignmentcuishuang
The new version of Go has been optimized, and variables do not need to be reassigned. Change-Id: I0374b049271e53510f2b162f6821fb3595f2c8ff Reviewed-on: https://go-review.googlesource.com/c/go/+/710835 Auto-Submit: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2025-10-02io/fs: move path name documentation up to the package doc commentDamien Neil
Perhaps surprisingly to users, io/fs path names are slash-separated. Move the documentation for path names up to the top of the package rather than burying it in the ValidPath documentation. Change-Id: Id338df07c74a16be74c687ac4c45e0513ee40a8c Reviewed-on: https://go-review.googlesource.com/c/go/+/708616 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com> Auto-Submit: Damien Neil <dneil@google.com>
2025-05-27io/fs: add examples for Glob,ReadFile and ValidPathcuishuang
Change-Id: I8451179bc0fa88b7e60afbc6fd9e06a22a94f3aa Reviewed-on: https://go-review.googlesource.com/c/go/+/673835 Reviewed-by: Sean Liao <sean@liao.dev> Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
2025-04-19fs: clarify documentation for ReadDir methodMatthew Burton
The fs.ReadDir method behaves the same way as os.ReadDir, in that when n <= 0, ReadDir returns all DirEntry values remaining in the dictionary. Update the comment to reflect that only remaining DirEntry values are returned (not all entries), for subsequent calls. Fixes #69301 Change-Id: I41ef7ef1c8e3fe7d64586f5297512697dc60dd40 Reviewed-on: https://go-review.googlesource.com/c/go/+/663215 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Sean Liao <sean@liao.dev> Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Jorropo <jorropo.pgm@gmail.com> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-15std: add //go:fix inline directives to some deprecated functionsAlan Donovan
In particular, we apply it only to functions where it is always a code improvement to inline the call. We also apply it to some constants. In a few cases this may introduce a panic statement at the caller, which is debatable, but making the potential for panic evident is the purpose of the deprecation. The gofix analyzer in gopls v0.18 will show a diagnostic for calls to the annotated functions, and will offer to inline the call. The new //go:fix annotation needs a special exemption in the pragma check in the compiler. Updates #32816 Change-Id: I43bf15648ac12251734109eb7102394f8a76d55e Reviewed-on: https://go-review.googlesource.com/c/go/+/648995 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Commit-Queue: Alan Donovan <adonovan@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Auto-Submit: Alan Donovan <adonovan@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-13io: use runtime.AddCleanup instead of runtime.SetFinalizerCarlos Amedee
Replace the usage of runtime.SetFinalizer with runtime.AddCleanup. Updates #70907 Change-Id: Id604ca44ea67dcf8f87797e27347c6f4e9ad0b86 Reviewed-on: https://go-review.googlesource.com/c/go/+/638556 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Carlos Amedee <carlos@golang.org> TryBot-Bypass: Carlos Amedee <carlos@golang.org>
2025-02-03io/fs: add ReadLinkFS interfaceRoxy Light
Added implementations for *io/fs.subFS, os.DirFS, and testing/fstest.MapFS. Amended testing/fstest.TestFS to check behavior. Addressed TODOs in archive/tar and os.CopyFS around symbolic links. I am deliberately not changing archive/zip in this CL, since it currently does not resolve symlinks as part of its filesystem implementation. I am unsure of the compatibility restrictions on doing so, so figured it would be better to address independently. testing/fstest.MapFS now includes resolution of symlinks, with MapFile.Data storing the symlink data. The behavior change there seemed less intrusive, especially given its intended usage in tests, and it is especially helpful in testing the io/fs function implementations. Fixes #49580 Change-Id: I58ec6915e8cc97341cdbfd9c24c67d1b60139447 Reviewed-on: https://go-review.googlesource.com/c/go/+/385534 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Funda Secgin <fundasecgin33@gmail.com>
2024-11-05io: simplify tests by removing redundant statementsOleksandr Redko
Change-Id: I4bcaa6b42571626c88e3374c328bbfe993476242 Reviewed-on: https://go-review.googlesource.com/c/go/+/625295 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Commit-Queue: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-05io/fs: clarify that "." may only be used for rootIan Lance Taylor
For #70155 Change-Id: I648791c484e19bb12c6e4f84e2dc42eaaa4db546 Reviewed-on: https://go-review.googlesource.com/c/go/+/624595 Reviewed-by: Damien Neil <dneil@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Chase <drchase@google.com>
2024-10-07io/fs: document need to call File.CloseSean Liao
Fixes #69723 Change-Id: Id081f68cd30abfa77a971a2370f4a0ebf330eb8d Reviewed-on: https://go-review.googlesource.com/c/go/+/618095 Reviewed-by: Alan Donovan <adonovan@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Commit-Queue: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-09-04all: use t.Chdir in testsKir Kolyshkin
Change-Id: I5bc514bedeb1155e6db52e37736fd6101774aea0 Reviewed-on: https://go-review.googlesource.com/c/go/+/529896 Auto-Submit: Ian Lance Taylor <iant@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Commit-Queue: Ian Lance Taylor <iant@golang.org>
2024-07-25go,internal,io,mime: use slices and maps to clean testsapocelipes
Replace reflect.DeepEqual with slices.Equal/maps.Equal, which is much faster. Change-Id: Id9cb550884da817da96befdeccfecb3325fb4414 GitHub-Last-Rev: 7d64d78feb86e3ea1af6c24ea6782cb85731bb52 GitHub-Pull-Request: golang/go#67612 Reviewed-on: https://go-review.googlesource.com/c/go/+/587819 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-07-07io: add test for Pipe constructor allocationsCuong Manh Le
Updates #67633 Change-Id: If3da9317ba36cb8a7868db94b45c402e1793e018 Reviewed-on: https://go-review.googlesource.com/c/go/+/588219 Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Mateusz Poliwczak <mpoliwczak34@gmail.com> Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Than McIntosh <thanm@google.com>
2024-05-23all: change from sort functions to slices functions where feasibleIan Lance Taylor
Doing this because the slices functions are slightly faster and slightly easier to use. It also removes one dependency layer. This CL does not change packages that are used during bootstrap, as the bootstrap compiler does not have the required slices functions. It does not change the go/scanner package because the ErrorList Len, Swap, and Less methods are part of the Go 1 API. Change-Id: If52899be791c829198e11d2408727720b91ebe8a Reviewed-on: https://go-review.googlesource.com/c/go/+/587655 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Commit-Queue: Ian Lance Taylor <iant@google.com> Reviewed-by: Damien Neil <dneil@google.com>
2024-05-07io/fs: use slices.Containsqiulaidongfeng
Change-Id: Ifd91722fd63af89af96a90dd69c73488f7fab5d3 GitHub-Last-Rev: da03963a07201ffca6ae9d50afdab121be8ad208 GitHub-Pull-Request: golang/go#67179 Reviewed-on: https://go-review.googlesource.com/c/go/+/583296 Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-04-15io: fix typo inside copyBufferChemaclass
The "rt" seems to be caused after copy-pasting the previous "wt" block which make sense as WriterTo, but for ReaderFrom it makes more sense thinking of rf instead of rt. Change-Id: I873699c27211bea6cdba3e199f36eb3c38188d70 GitHub-Last-Rev: 1795600a9b29946d824ba645c137da216bdf6302 GitHub-Pull-Request: golang/go#66811 Reviewed-on: https://go-review.googlesource.com/c/go/+/578635 Reviewed-by: Ian Lance Taylor <iant@google.com> Commit-Queue: Robert Griesemer <gri@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
2024-03-11io: close PipeReader in testguoguangwu
Change-Id: I33858efc00dff02432f28f1e5a94aeea261a5bad GitHub-Last-Rev: 98861f8d6e187a03330a0947ff651826024fcad2 GitHub-Pull-Request: golang/go#66230 Reviewed-on: https://go-review.googlesource.com/c/go/+/570357 Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-02-10io/fs: set ErrInvalid for FS.Open from SubFS when it fails ValidPathAndy Pan
Fixes #65419 Change-Id: I8f9f82ab0387d8bb39aaca4f9e60e36ee15c587d Reviewed-on: https://go-review.googlesource.com/c/go/+/560137 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-01-15io/fs: godoc links to testing/fstestOlivier Mengué
Add godoc links from io/fs to testing/fstest for discoverability. Change-Id: I6550b4b703d2214faa732987ec8630ac903705b5 Reviewed-on: https://go-review.googlesource.com/c/go/+/534095 Reviewed-by: Michael Knyszek <mknyszek@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2024-01-09io/fs,path/filepath: fix typo in SkipAll/SkipDir docRoger Peppe
Also make the reference into a doc link. Change-Id: Ib112307a65b65c8f963abf60aa92cb1942de940c Reviewed-on: https://go-review.googlesource.com/c/go/+/554295 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-27io/fs, os: unify PathError.Path for dirFS.{ReadFile, ReadDir}Jes Cok
Fixes #64366 Change-Id: Ie78ab2cb9e11c0766665cd37fd7a26d36a1c24fa GitHub-Last-Rev: 3cb3bb84a0ccccc88ba30258f52f699ac1364a7b GitHub-Pull-Request: golang/go#64372 Reviewed-on: https://go-review.googlesource.com/c/go/+/544835 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Jes Cok <xigua67damn@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-10-18io: add available godoc linkcui fliter
Change-Id: I5973a352edb73e02a274d939d6d0573788640dc9 Reviewed-on: https://go-review.googlesource.com/c/go/+/535435 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2023-10-11io/fs: add godoc linksOlivier Mengué
Change-Id: Icde42bd33d58f75acdede439f7525f9d06554140 Reviewed-on: https://go-review.googlesource.com/c/go/+/534096 Reviewed-by: qiulaidongfeng <2645477756@qq.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-10-09io: add (*SectionReader).Outer()Carlo Alberto Ferraris
Fixes #61870 Updates #61727 Change-Id: Iaef9b59c402d68f6bf64be212db2b6746abe8900 Reviewed-on: https://go-review.googlesource.com/c/go/+/526855 Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-09-12io/fs, path/filepath, cmd/gofmt: replace statDirEntry with fs.FileInfoToDirEntrymstmdev
Change-Id: Ie914367314ca72fab34c4d4529755dea853cf325 GitHub-Last-Rev: bc61665d33d7f15da371dcfb90c132fab25fc233 GitHub-Pull-Request: golang/go#62342 Reviewed-on: https://go-review.googlesource.com/c/go/+/523876 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Commit-Queue: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2023-08-17io: reduce allocations in Pipe constructorJoe Tsai
Rather than having PipeWriter and PipeReader a wrapper type on pipe, make them have the same underlying memory representation and rely instead of simply casting the same *pipe pointer as either a *PipeReader or *PipeWriter to control the set of methods. This reduces the number of allocations by 2, going from a total of 6 down to 4 allocations. Change-Id: I09207a00c4b7afb44c7773d752c5628a07e24fda Reviewed-on: https://go-review.googlesource.com/c/go/+/473535 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-08-07all: add a few more godoc linksDaniel Martí
Over the past few months as I read the standard library's documentation I kept finding spots where godoc links would have helped me. I kept adding to a stash of changes to fix them up bit by bit. The stash has grown big enough by now, and we're nearing a release, so I think it's time to merge to avoid git conflicts or bit rot. Note that a few sentences are slightly reworded, since "implements the Fooer interface" can just be "implements [Fooer]" now that the link provides all the context needed to the user. Change-Id: I01c31d3d3ff066d06aeb44f545f8dd0fb9a8d998 Reviewed-on: https://go-review.googlesource.com/c/go/+/508395 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-08-05io: remove manual SectionReader.Size in SectionReader.ReadAtJorropo
Change-Id: Ib3e8953dbdefa2b78c31b1bcbf0909bce248e423 Reviewed-on: https://go-review.googlesource.com/c/go/+/500475 Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-05-26io: clarify that Read(nil) can return 0, EOFIan Lance Taylor
Fixes #40385 Change-Id: I965b5db985fd4418a992e883073cbc8309b2cb88 Reviewed-on: https://go-review.googlesource.com/c/go/+/498355 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Rob Pike <r@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-05-20io: prevent seeking to position prior to offsetwrite.baseJabar Asadi
We don't want to permit writing before the start of an OffsetWriter. One of the goals of OffsetWriter is to restrict where data can be written. However, this rule can be violated by WriteAt() method of OffsetWriter as the following code shows: f, _ := os.Create("file.txt") owr := io.NewOffsetWriter(f, 10) owr.Write([]byte("world")) owr.WriteAt([]byte("hello"), -10) Change-Id: I6c7519fea68daefa641f25130cdd9803dc8aae22 GitHub-Last-Rev: a29d890d6f32fd5a1ecef84d012b8447b406e2e2 GitHub-Pull-Request: golang/go#60222 Reviewed-on: https://go-review.googlesource.com/c/go/+/495155 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Jabar Asadi <jasadi@d2iq.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-05-04all: add String for fs.{FileInfo,DirEntry} implementationsIan Lance Taylor
The new String methods use the new FormatFileInfo and FormatDirEntry functions. Fixes #54451 Change-Id: I414cdfc212ec3c316fb2734756d2117842a23631 Reviewed-on: https://go-review.googlesource.com/c/go/+/491175 Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-02io/fs: add FormatFileInfo and FormatDirEntry functionsIan Lance Taylor
For #54451 Change-Id: I3214066f77b1398ac1f2786ea035c83f32f0a826 Reviewed-on: https://go-review.googlesource.com/c/go/+/489555 Run-TryBot: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-04-20io: ReadAll: do not check for realloc in first roundOlivier Mengué
Refactor io.ReadAll to check for realloc of the buffer only after the first read. Fixes: #59702 Change-Id: I93b99139e6756f21738d47e7d9ad08e1d167258e Reviewed-on: https://go-review.googlesource.com/c/go/+/486236 Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2023-04-11all: add wasip1 supportJohan Brandhorst-Satzkorn
Fixes #58141 Co-authored-by: Richard Musiol <neelance@gmail.com> Co-authored-by: Achille Roussel <achille.roussel@gmail.com> Co-authored-by: Julien Fabre <ju.pryz@gmail.com> Co-authored-by: Evan Phoenix <evan@phx.io> Change-Id: I49b66946acc90fdf09ed9223096bfec9a1e5b923 Reviewed-on: https://go-review.googlesource.com/c/go/+/479627 Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
2023-02-28io/ioutil: add doc links for deprecated functionShengyu Zhang
Go 1.19 introduced doc links (https://go.dev/doc/comment#doclinks). It will be convenient when we can directly jump to the suggested function when the original function is deprecated. Change-Id: I6172a5265f3b47aefec53179bca60f9904606b3f GitHub-Last-Rev: b2aa85bf1bb67a39b5b26a047ba90a4b6a61dba4 GitHub-Pull-Request: golang/go#58779 Reviewed-on: https://go-review.googlesource.com/c/go/+/471915 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-02-14Revert "io: detect Writers that access io.Copy's buffer after returning"Damien Neil
This reverts CL 466865. Reason for revert: Causing trybot flakiness due to net/http race, roll back until net/http is fixed. For #58168 Change-Id: I3129deb996abe6466eccf933fe93cbbaf72ae217 Reviewed-on: https://go-review.googlesource.com/c/go/+/467895 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Damien Neil <dneil@google.com>
2023-02-10io: detect Writers that access io.Copy's buffer after returningDamien Neil
When the race detector is enabled, scribble over copy buffers with garbage after Write returns. For #58452 Change-Id: I25547684bcbef7d302d76736cb02e59c89a640ee Reviewed-on: https://go-review.googlesource.com/c/go/+/466865 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Damien Neil <dneil@google.com>
2023-02-10Revert "io: allocate copy buffers from a pool"Damien Neil
This reverts CL 456555. Reason for revert: This seems too likely to exercise race conditions in code where a Write call continues to access its buffer after returning. The HTTP/2 ResponseWriter is one such example. Reverting this change while we think about this some more. For #57202 Change-Id: Ic86823f81d7da410ea6b3f17fb5b3f9a979e3340 Reviewed-on: https://go-review.googlesource.com/c/go/+/467095 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-31io: allocate copy buffers from a poolDamien Neil
CopyBuffer allocates a 32k buffer when no buffer is available. Allocate these buffers from a sync.Pool. This removes an optimization where the copy buffer size was reduced when the source is a io.LimitedReader (including the case of CopyN) with a limit less than the default buffer size. This change could cause a program which only uses io.Copy with sources with a small limit to allocate unnecessarily large buffers. Programs which care about the transient buffer allocation can avoid this by providing their own buffer. name old time/op new time/op delta CopyNSmall-10 165ns ± 7% 117ns ± 7% -29.19% (p=0.001 n=7+7) CopyNLarge-10 7.33µs ±34% 4.07µs ± 2% -44.52% (p=0.001 n=7+7) name old alloc/op new alloc/op delta CopyNSmall-10 2.20kB ±12% 1.20kB ± 4% -45.24% (p=0.000 n=8+7) CopyNLarge-10 148kB ± 9% 81kB ± 4% -45.26% (p=0.000 n=8+7) name old allocs/op new allocs/op delta CopyNSmall-10 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8) CopyNLarge-10 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8) For #57202 Change-Id: I2292226da9ba1dc09a2543f5d74fe5da06080d49 Reviewed-on: https://go-review.googlesource.com/c/go/+/456555 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Thomas Austad <thomas.austad@gmail.com> Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-18io/fs: clean up test helper functionsTobias Klauser
Inline the only use of checkMarks which also allows to drop the always-true report argument. This also ensures the correct line gets reported in case of an error. Also remove the unused markTree function and drop the unused testing.T argument from makeTree. Change-Id: I4033d3e5ecd929d08ce03c563aa99444e102d931 Reviewed-on: https://go-review.googlesource.com/c/go/+/451615 Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-06io: use strings.Buildercuiweixie
Change-Id: Ibab20627ccd1f79e77e3972e2bc6ca42c2c7de13 Reviewed-on: https://go-review.googlesource.com/c/go/+/428263 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-08-25path/filepath, io/fs: add SkipAllPaschalis Tsilias
Fixes #47209 Change-Id: If75b0dd38f2c30a23517205d80c7a6683a5c921c Reviewed-on: https://go-review.googlesource.com/c/go/+/363814 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-08-19io: add OffsetWriter, NewOffsetWriterhopehook
Offsetwriter refers to the design of SectionReader and removes the section parameter n. Since the size of the written data is determined by the user, we cannot know where the end offset of the original data is. The offset of SeekEnd is not valid in Seek method. Fixes #45899. Change-Id: I9d9445aecfa0dd4fc5168f2f65e1e3055c201b45 Reviewed-on: https://go-review.googlesource.com/c/go/+/406776 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-07-12io/fs: fix stack exhaustion in GlobJulie Qiu
A limit is added to the number of path separators allowed by an input to Glob, to prevent stack exhaustion issues. Thanks to Juho Nurminen of Mattermost who reported a similar issue in path/filepath. Fixes CVE-2022-30630 Fixes golang/go#53415 Change-Id: I5a9d02591fed90cd3d52627f5945f1301e53465d Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1497588 Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/417065 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-06-25io: clarify SeekEnd offset valueHowJMay
fixes #53474 Change-Id: I14c3dc800dc27233630a54592328bb0df1bbaa5d GitHub-Last-Rev: 46f93cfbd41c1b3274b570744f18d08e7759eb1e GitHub-Pull-Request: golang/go#53505 Reviewed-on: https://go-review.googlesource.com/c/go/+/413614 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 22:51:40 +0000