aboutsummaryrefslogtreecommitdiff
path: root/src/internal/poll/copy_file_range_linux.go
AgeCommit message (Collapse)Author
2025-09-15internal/syscall/unix: add KernelVersionGEKir Kolyshkin
There are a few places in the code which checks that the running kernel is greater than or equal to x.y. The check takes a few lines and the checking code is somewhat distracting. Let's abstract this check into a simple function, KernelVersionGE, and convert the users accordingly. Add a test case (I'm not sure it has much value, can be dropped). Change-Id: I8ec91dcc7452363361f95e46794701c0ae57d956 Reviewed-on: https://go-review.googlesource.com/c/go/+/700796 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Mark Freeman <markfreeman@google.com>
2025-02-19internal/poll: note that copy_file_range bug is fixedMichael Pratt
We shouldn't drop the workaround in case we're running on an unpatched kernel. For #44272. Change-Id: I6a6a636cb81c31856ac9b682e7d02fa1d8efa5d9 Reviewed-on: https://go-review.googlesource.com/c/go/+/644878 Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Ian Lance Taylor <iant@golang.org>
2024-08-20internal/poll: raise Linux's maxCopyFileRangeRound to MAX_RW_COUNTAndy Pan
Change-Id: I23c0c850ba57e7a49b78159d9293d1d25e1d2340 Reviewed-on: https://go-review.googlesource.com/c/go/+/606637 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Commit-Queue: Ian Lance Taylor <iant@golang.org>
2024-08-15internal,os: employ copy_file_range(2) for file-to-file copying on FreeBSDAndy Pan
FreeBSD 13.0 introduced the Linux-compatible copy_file_range(2) system call, we should make use of it. Ref: https://www.gnu.org/software/gnulib/manual/html_node/copy_005ffile_005frange.html https://reviews.freebsd.org/D20584?id=60021 https://man.freebsd.org/cgi/man.cgi?copy_file_range(2) Change-Id: I75edb5629717289c8887be436613d3a8b3820bdc Reviewed-on: https://go-review.googlesource.com/c/go/+/604655 Run-TryBot: Andy Pan <panjf2000@gmail.com> Reviewed-by: Carlos Amedee <carlos@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-04-04internal/poll: eliminate the redundant conditional branch for ↵Andy Pan
isKernelVersionGE53 Follow up CL 573755 Change-Id: I27c7571d3ef1274cf2c6892e678f946f9b65de33 Reviewed-on: https://go-review.googlesource.com/c/go/+/576416 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2024-03-25internal/poll: use sync.OnceValue to determine kernel version ≥ 5.3 in ↵Tobias Klauser
CopyFileRange Change-Id: I13fdf86c3f46bf3c83cb116e9dd3bc4ab1a949d7 Reviewed-on: https://go-review.googlesource.com/c/go/+/573755 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: qiulaidongfeng <2645477756@qq.com>
2023-02-22Revert "internal/poll: drop redundant ENOSYS in CopyFileRange"Ian Lance Taylor
This reverts CL 428555. Reason for revert: It appears that even a newer kernel can get ENOSYS from copy_file_range. Fixes #58592 Change-Id: Ib8dd1be61544f54bf652a99dc0b449109f8f50ed Reviewed-on: https://go-review.googlesource.com/c/go/+/470016 Reviewed-by: Andy Pan <panjf2000@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2022-09-09internal/poll: drop redundant ENOSYS in CopyFileRangeAndy Pan
Update CL 425881 and CL 428396 I browsed the source code related to copy_file_range in the kernel and found that the latest kernel may still return EXDEV errors in copy_file_range(2) due to certain cases, for details see: https://elixir.bootlin.com/linux/v5.19.7/source/fs/read_write.c#L1559, https://elixir.bootlin.com/linux/v5.19.7/source/fs/read_write.c#L1479, and https://elixir.bootlin.com/linux/v5.19.7/source/fs/read_write.c#L1439. Therefore, the EXDEV still needs to be kept, but the ENOSYS error can be safely removed. Change-Id: I47026b8dd33f7ffc4de1306af6b67c7b4d2062d2 Reviewed-on: https://go-review.googlesource.com/c/go/+/428555 Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Andy Pan <panjf2000@gmail.com>
2022-09-06Revert "internal/poll: drop redundant ENOSYS and EXDEV error checks in ↵Daniel Martí
CopyFileRange()" This reverts https://go.dev/cl/425881. Reason for revert: broke make.bash on linux/amd64 with Linux 5.19.6. [...] Building Go toolchain2 using go_bootstrap and Go toolchain1. go install internal/unsafeheader: copying /tmp/go-build4206185186/b007/_pkg_.a to /home/mvdan/tip/pkg/linux_amd64/internal/unsafeheader.a: write /home/mvdan/tip/pkg/linux_amd64/internal/unsafeheader.a: copy_file_range: invalid cross-device link go install internal/goarch: copying /tmp/go-build4206185186/b006/_pkg_.a to /home/mvdan/tip/pkg/linux_amd64/internal/goarch.a: write /home/mvdan/tip/pkg/linux_amd64/internal/goarch.a: copy_file_range: invalid cross-device link [...] Change-Id: I793856935d4315a870c2d31da46be00cc342b5f8 Reviewed-on: https://go-review.googlesource.com/c/go/+/428396 Reviewed-by: Wayne Zuo <wdvxdr@golangcn.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: hopehook <hopehook@golangcn.org> Reviewed-by: Andy Pan <panjf2000@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-09-05internal/poll: drop redundant ENOSYS and EXDEV error checks in CopyFileRange()Andy Pan
The initial CL 229101 didn't limit the kernel version, but relies on error checking to ensure the kernel version >= 4.5 or >= 5.3 when it's calling copy_file_range(2) to copy data across file systems. Since we have now put the kernel version checking at the beginning of the function, introduced by CL 268338, which returns early instead of going forward to the code behind when the kernel verion is older than 5.3, therefore, those subsequent related error checks are no longer needed. Change-Id: Ifc4a530723e21f0bde91d6420cde9cb676081922 Reviewed-on: https://go-review.googlesource.com/c/go/+/425881 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: hopehook <hopehook@golangcn.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-09-03internal/syscall/unix: consolidate kernelVersion implementationsKir Kolyshkin
Currently, there are 3 functions returning Linux kernel version numbers. Two of them are identical: - in net, initially added by commit 0a9dd47dd817904e; - in internal/poll, initially added by commit 1c7650aa93bd53; (both were later fixed by commit 66c02645062561a). The third one is a more complex, regexp-based implementation in runtime/pprof, which is only used for a test. Instead of adding one more, let's consolidate existing ones. Remove the complex implementation, and move the simple one into internal/syscall/unix. Use it from all the three places mentioned above. Change-Id: I4a34d9ca47257743c16def30e4dd634e36056091 Reviewed-on: https://go-review.googlesource.com/c/go/+/424896 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Kirill Kolyshkin <kolyshkin@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-29internal/poll: use sync.Once instead to guard CopyFileRange() with kernel 5.3Andy Pan
The existing implementation creates more branches with more states: -1, 0, 1, which makes it not very intuitive to understand, let's use sync.Once and boolean instead to make it more straightforward. Change-Id: I05766e5fdf7dba37d6565f84d3db4373f9342fe5 Reviewed-on: https://go-review.googlesource.com/c/go/+/425880 Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2021-02-24docs: fix spellingJohn Bampton
Change-Id: Ib689e5793d9cb372e759c4f34af71f004010c822 GitHub-Last-Rev: d63798388e5dcccb984689b0ae39b87453b97393 GitHub-Pull-Request: golang/go#44259 Reviewed-on: https://go-review.googlesource.com/c/go/+/291949 Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Matthew Dempsky <mdempsky@google.com> Trust: Robert Griesemer <gri@golang.org>
2021-02-16internal/poll: if copy_file_range returns 0, assume it failedIan Lance Taylor
On current Linux kernels copy_file_range does not correctly handle files in certain special file systems, such as /proc. For those file systems it fails to copy any data and returns zero. This breaks Go's io.Copy for those files. Fix the problem by assuming that if copy_file_range returns 0 the first time it is called on a file, that that file is not supported. In that case fall back to just using read. This will force an extra system call when using io.Copy to copy a zero-sized normal file, but at least it will work correctly. For #36817 Fixes #44272 Change-Id: I02e81872cb70fda0ce5485e2ea712f219132e614 Reviewed-on: https://go-review.googlesource.com/c/go/+/291989 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2020-11-20net, internal/poll: reset value before adding in minor kernel versionIan Lance Taylor
Fixes #42733 Change-Id: I5446aeb5de13cd70212755fb12c9bc484f343c74 Reviewed-on: https://go-review.googlesource.com/c/go/+/271846 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-11-10internal/poll: use copy_file_range only on Linux kernel >= 5.3Tobias Klauser
https://man7.org/linux/man-pages/man2/copy_file_range.2.html#VERSIONS states: A major rework of the kernel implementation occurred in 5.3. Areas of the API that weren't clearly defined were clarified and the API bounds are much more strictly checked than on earlier kernels. Applications should target the behaviour and requirements of 5.3 kernels. Rather than attempting to detect the file system for source and destination files (which means two additional statfs syscalls) and skip copy_file_range in case of known defects (e.g. CIFS -> CIFS), just assume copy_file_range to be broken on kernels < 5.3. Fixes #42400 Change-Id: I3a531296182c1d6e341772cc9d2be5bf83e52575 Reviewed-on: https://go-review.googlesource.com/c/go/+/268338 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-04internal/poll: treat copy_file_range EIO as not-handledTobias Klauser
Fixes #42334 Change-Id: Ife51df4e7d2539a04393abfdec45e3f902975fca Reviewed-on: https://go-review.googlesource.com/c/go/+/266940 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-19internal/poll: treat copy_file_range EPERM as not-handledTobias Klauser
Fixes #40893 Change-Id: I938ea4796c1e1d1e136117fe78b06ad6da8e40de Reviewed-on: https://go-review.googlesource.com/c/go/+/249257 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Antonio Troina <thoeni@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-19internal/poll: treat copy_file_range EOPNOTSUPP as not-handledIan Lance Taylor
Fixes #40731 Change-Id: I3e29878d597318acf5edcc38497aa2624f72be35 Reviewed-on: https://go-review.googlesource.com/c/go/+/248258 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-05-11internal/poll, os: loop on EINTRIan Lance Taylor
Historically we've assumed that we can install all signal handlers with the SA_RESTART flag set, and let the system restart slow functions if a signal is received. Therefore, we don't have to worry about EINTR. This is only partially true, and we've added EINTR checks already for connect, and open/read on Darwin, and sendfile on Solaris. Other cases have turned up in #36644, #38033, and #38836. Also, #20400 points out that when Go code is included in a C program, the C program may install its own signal handlers without SA_RESTART. In that case, Go code will see EINTR no matter what it does. So, go ahead and check for EINTR. We don't check in the syscall package; people using syscalls directly may want to check for EINTR themselves. But we do check for EINTR in the higher level APIs in os and net, and retry the system call if we see it. This change looks safe, but of course we may be missing some cases where we need to check for EINTR. As such cases turn up, we can add tests to runtime/testdata/testprogcgo/eintr.go, and fix the code. If there are any such cases, their handling after this change will be no worse than it is today. For #22838 Fixes #20400 Fixes #36644 Fixes #38033 Fixes #38836 Change-Id: I7e46ca8cafed0429c7a2386cc9edc9d9d47a6896 Reviewed-on: https://go-review.googlesource.com/c/go/+/232862 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-28os, internal/poll, internal/syscall/unix: use copy_file_range on LinuxAndrei Tudor Călin
Linux 4.5 introduced (and Linux 5.3 refined) the copy_file_range system call, which allows file systems the opportunity to implement copy acceleration techniques. This commit adds support for copy_file_range(2) to the os package. Introduce a new ReadFrom method on *os.File, which makes *os.File implement the io.ReaderFrom interface. If dst and src are both files, this enables io.Copy(dst, src) to call dst.ReadFrom(src), which, in turn, will call copy_file_range(2) if possible. If copy_file_range(2) is not supported by the host kernel, or if either of dst or src refers to a non-regular file, ReadFrom falls back to the regular io.Copy code path. Add internal/poll.CopyFileRange, which acquires locks on the appropriate poll.FDs and performs the actual work, as well as internal/syscall/unix.CopyFileRange, which wraps the copy_file_range system call itself at the lowest level. Rework file layout in internal/syscall/unix to accomodate the additional system call numbers needed for copy_file_range. Merge these definitions with the ones used by getrandom(2) into sysnum_linux_$GOARCH.go files. A note on additional optimizations: if dst and src both refer to pipes in the invocation dst.ReadFrom(src), we could, in theory, use the existing splice(2) code in package internal/poll to splice directly from src to dst. Attempting this runs into trouble with the poller, however. If we call splice(src, dst) and see EAGAIN, we cannot know if it came from src not being ready for reading or dst not being ready for writing. The write end of src and the read end of dst are not under our control, so we cannot reliably use the poller to wait for readiness. Therefore, it seems infeasible to use the new ReadFrom method to splice between pipes directly. In conclusion, for now, the only optimization enabled by the new ReadFrom method on *os.File is the copy_file_range optimization. Fixes #36817. Change-Id: I696372639fa0cdf704e3f65414f7321fc7d30adb Reviewed-on: https://go-review.googlesource.com/c/go/+/229101 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-16 05:56:34 +0000