| Age | Commit message (Collapse) | Author |
|
The openbsd/mips64 port is dead, remove the remaining code specific to
that port and clean up build tags.
Fixes #61546
Change-Id: I0328b7b76ce1ddacd3a526b3f4ae29eaa1254c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/746480
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
Windows requires these objects to remain valid during the whole
overlapped operation. This currently works because we have a non-moving
GC and these objects are taken from a sync.Pool.
For correctness, and to ensure a possible moving GC in the future does
not break it in a similar manner, pin these objects in execIO.
Updates #77975.
Change-Id: Iff07009d40e4a439026a961a6dca9f6843cbd61d
Reviewed-on: https://go-review.googlesource.com/c/go/+/753560
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
The destination of SendFile is a socket, which doesn't support file
offsets. There is no need to keep track of the file offset, and doing
so causes a race between SendFile and Read.
While here, make sure that SendFile tests do call poll.SendFile.
Fixes #78015
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race,gotip-linux-amd64-race
Change-Id: I8cce45c0c110e848d9bdbc5ba340b92ca041f0a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/752860
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Mark Freeman <markfreeman@google.com>
|
|
According to https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsarecvfrom,
the memory pointed to by lpFromlen must remain available during the
overlapped I/O, and therefore cannot be allocated on the stack.
CL 685417 moved the rsan field out of the operation struct and placed
it on stack, which violates the above requirement and causes stack
corruption.
Unfortunately, it is no longer possible to cleanly revert CL 685417.
Instead of attempting to revert it, this CL bundles rsan together
with rsa in the same sync.Pool. The new wsaRsa struct is still in the
same size class, so no additional overhead is introduced by this
change.
Fixes #77975.
Change-Id: I5ffbccb332515116ddc03fb7c40ffc9293cad2ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/753040
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Commit-Queue: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
Fixes #77875.
Change-Id: I0ed91be7ed10c04ddea3af55548a8dbf0be5f3a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/750420
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
When reading the contents of a directory using
File.ReadDir or File.Readdir, the os.FileInfo was
populated on Unix platforms using lstat.
This lstat call is vulnerable to a TOCTOU race
and could escape the root.
For example:
- Open the directory "dir" within a Root.
This directory contains a file named "file".
- Use File.ReadDir to list the contents of "dir",
receiving a os.DirEntry for "dir/file".
- Replace "dir" with a symlink to "/etc".
- Use DirEntry.Info to retrieve the FileInfo for "dir/file".
This FileInfo contains information on "/etc/file" instead.
This escape permits identifying the presence or absence of
files outside a Root, as well as retreiving stat metadata
(size, mode, modification time, etc.) for files outside a Root.
This escape does not permit reading or writing to files
outside a Root.
Fixes #77827
Fixes CVE-2026-27139
Change-Id: I40004f830c588e516aff8ee593d630d36a6a6964
Reviewed-on: https://go-review.googlesource.com/c/go/+/749480
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-by: Nicholas Husin <nsh@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
references to it
The read lock in readWriteLock might be holding the last reference to
the fd. If the fd is closed while the read lock is still held, then the
fd will not be destroyed until the read lock is released and fd.destroy
is called.
The race windows is small and difficult to trigger on real workloads,
but it has been observed in CI.
Fixes #77609
Fixes #74754 (tentatively)
Change-Id: I17be2d23dced1258e1a986c7359e27a81fc51c53
Reviewed-on: https://go-review.googlesource.com/c/go/+/745601
Reviewed-by: Mark Freeman <markfreeman@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This is a step towards deferring adding the handle to IOCP until the
first IO operation.
FD.pollable() obscures the fact that it is really checking if the handle
is associated with the IOCP. This doesn't need to be a function that
checks multiple conditions. It can be a simple boolean field that tracks
whether the handle is associated with the IOCP or not.
For #76391
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: I3ee6532f8a387fb5cfae8ae3d20ea9569f585e71
Reviewed-on: https://go-review.googlesource.com/c/go/+/742282
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
This is a step towards deferring adding the handle to IOCP until the
first IO operation.
There is a small race windows between execIO and DisassociateIOCP where
execIO checks if the fd is disassociated before passing the operation to
the OS. DisassociateIOCP can set the disassociated flag right after that
check but before Windows started processing the IO operation. Once
Windows takes over, the race doesn't matter anymore because Windows
doesn't allow disassociating a handle that has pending IO operations.
If that still hasn't happened, an overlapped IO operation will start
assuming the they can be waited using the Go runtime IOCP, which is
wrong due to the disassociation, leading to undefined behavior.
Fix that race by trying to take a write/read lock in DisassociateIOCP
before setting the disassociated flag, but failing if there is an
ongoing execIO operation so that DisassociateIOCP doesn't block
indefinitely waiting for execIO to finish.
For #76391
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: Iec265fa1900383aace50051d2be750bc76aa0944
Reviewed-on: https://go-review.googlesource.com/c/go/+/741020
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
|
|
This is a step towards deferring adding the handle to IOCP until the
first IO operation.
Having all isBlocking checks inside execIO will make it easier to delay
getting that information until the first IO operation.
It also makes the code simpler, as Pread and Pwrite now only modify
the overlapped object offset, not the one in the FD struct, so they
don't need to reverse the offset change after the IO operation.
For #76391
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: Iea680c502c9417b9569a0b1a9da6f7e6bf916f6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/742283
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
FILE_SKIP_SET_EVENT_ON_HANDLE is always safe to use.
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS is safe as long as the socket
is provided by an IFS provider.
While here, stop using the kindFileNet type, it doesn't provide any
value.
Fixes #77448
Change-Id: Ib3dc0d68c7ff57b6a1f15f017e60a092e4b87f46
Reviewed-on: https://go-review.googlesource.com/c/go/+/742281
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This is a step towards deferring adding the handle to IOCP until the
first IO operation.
waitIO should use the overlapped event if it is provided, regardless of
whether FD is pollable or not. This simplifies reasoning about the code
and makes it more robust to race conditions.
While here, remove the panic in waitIO that was triggered when called
with a blocking handle. That shouldn't happen, but if it does,
fd.pd.wait will return an error that will already be promoted to
a panic.
For #76391
Change-Id: I8e84592568a3ef66e71161eb2c5f515dde638117
Reviewed-on: https://go-review.googlesource.com/c/go/+/742280
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
If the write lock acquisition fails, the read lock acquired earlier is
not released, leading to a potential deadlock.
The deadlock shouldn't occur because when the write lock fails, it
indicates that the FD is closing, and no other goroutine should be
holding the read lock. However, better to be safe and release the read
lock in such cases.
Change-Id: If593c36040a97357f835b42bb3133ff1dc55a638
Reviewed-on: https://go-review.googlesource.com/c/go/+/740560
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
|
|
Fixes #77404
Change-Id: I0402becb94855baf942d6ba3815cc2a3c1526d6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/740921
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
|
|
This is a step towards deferring adding the handle to IOCP until the
first IO operation.
The goal of this CL is to avoid the fd.isBlocking check in fd.pin,
which was happening outside execIO, and making buffer pinning less
error-prone.
This also fixes an issue where buffer used in Pwrite and WriteTo
were unpinned too early when the write buffer was larger than the
maximum chunk size.
For #76391
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: Ia181dcb57a559ae466a4341c36a307ad6678aac0
Reviewed-on: https://go-review.googlesource.com/c/go/+/740561
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
This is a step towards deferring adding the handle to IOCP until the
first IO operation.
The main goal of this CL is to remove the fd.pollable() check in
cancelIO.
For #76391
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: I76263ce12980297d88a5f6c514e4074dfee428cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/740540
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
windows.IsNonblock can block for synchronous handles that have an
outstanding I/O operation. Console handles are always synchronous, so
we should not call IsNonblock for them. Stdin is often a pipe, and
almost always a synchronous handle, so we should not call IsNonblock for
it either. This avoids potential deadlocks during os package
initialization, which calls NewFile(syscall.Stdin).
Fixes #75949
Updates #76391
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race,gotip-windows-arm64
Change-Id: I1603932b0a99823019aa0cad960f94cee9996505
Reviewed-on: https://go-review.googlesource.com/c/go/+/724640
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
Change-Id: Ia383d4d322008901cd1e57b05fb522db44076fa2
GitHub-Last-Rev: 5c7cbeb7371c6afa47c280e164087e45ca89f3d2
GitHub-Pull-Request: golang/go#66375
Reviewed-on: https://go-review.googlesource.com/c/go/+/572178
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Sean Liao <sean@liao.dev>
|
|
Replaced by internal/strconv.
Change-Id: I0656a9ad5075e60339e963fbae7d194d2f3e16be
Reviewed-on: https://go-review.googlesource.com/c/go/+/716001
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
Use a sync.Pool to reuse the overlapped object passed to the different
Windows syscalls instead of keeping two of them in the FD struct.
This reduces the size of the FD struct from 248 to 152 bytes.
While here, pin the overlapped object for the duration of the overlapped
IO operation to comply with the memory safety rules.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: I0161d163f681fe94b822c0c885aaa42c449e5342
Reviewed-on: https://go-review.googlesource.com/c/go/+/704235
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
|
|
execIO callers should be agnostic to the fact that it uses an overlapped
object. This will unlock future optimizations and simplifications.
Change-Id: I0a58d992101fa74ac75e3538af04cbc44156f0d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/704175
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
|
|
WSASend and WSARecv functions capture the WSABuf structure before
returning, so there is no need to keep a copy of it in the
operation structure.
Write and Read functions don't need it, they can operate directly
on the byte slice.
To be on the safe side, pin the input byte slice so that stack-allocated
slices don't get moved while overlapped I/O is in progress.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: I474bed94e11acafa0bdd8392b5dcf8993d8e1ed5
Reviewed-on: https://go-review.googlesource.com/c/go/+/704155
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
Let newWSAMsg retrieve the socket from the sync pool for unconnected
sockets instead of forcing the caller to do it.
Change-Id: I9b3d30bf3f5be187cbde5735d519b3b14f7b3008
Reviewed-on: https://go-review.googlesource.com/c/go/+/704116
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
|
|
Overlapped handles don't have the file pointer updated when performing
I/O operations, so there is no need to call syscall.SetFilePointerEx in
FD.Seek. Updating the in-memory offset is sufficient.
Updates #74951 (provides a more complete fix)
Change-Id: Ibede6625cdbd501fc92cfdf8ce2782ec291af2b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/698035
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
There are a few places in the code which checks that the running kernel
is greater than or equal to x.y. The check takes a few lines and the
checking code is somewhat distracting.
Let's abstract this check into a simple function, KernelVersionGE,
and convert the users accordingly.
Add a test case (I'm not sure it has much value, can be dropped).
Change-Id: I8ec91dcc7452363361f95e46794701c0ae57d956
Reviewed-on: https://go-review.googlesource.com/c/go/+/700796
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Mark Freeman <markfreeman@google.com>
|
|
The Windows implementation of several network protocols make use of
sync.Pool, which randomly drops cached items when race is enabled.
While here, zero out the control buffer to allow it to be garbage
collected.
Fixes #75341
Change-Id: Ie20e21adef2edc02ca7b4a78012dd5f3a9f03bee
Reviewed-on: https://go-review.googlesource.com/c/go/+/703195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
The bufs field is used to avoid allocating it every time it is needed.
We can do better by using a sync.Pool to reuse allocations across
operations and FDs instead of the field.
A side benefit is that FD is now 16 bytes smaller and operation more
stateless.
Change-Id: I5d686d1526f6c63e7ca1ae84da1fbf2044b24703
Reviewed-on: https://go-review.googlesource.com/c/go/+/698798
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
|
|
The rsa field was added to the operation structure to avoid allocating
it every time it is needed. We can do better by using a sync.Pool to
reuse allocations across operations and FDs instead of the field.
A side benefit is that FD is now 16 bytes smaller and operation more
stateless.
Change-Id: I3b69a59e36b27f2cdd076cebd8d27a2a350b9c43
Reviewed-on: https://go-review.googlesource.com/c/go/+/698875
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
WSAMsg parameters should be passed to Windows as heap pointers instead
of stack pointers. This is because Windows might access the memory
after the syscall returned in case of a non-blocking operation (which
is the common case), and if the WSAMsg is on the stack, the Go
runtime might have moved it around.
Use a sync.Pool to cache WSAMsg structures to avoid a heap allocation
every time a WSAMsg is needed.
Fixes #74933
Cq-Include-Trybots: luci.golang.try:x_net-gotip-windows-amd64
Change-Id: I075e2ceb25cd545224ab3a10d404340faf19fc01
Reviewed-on: https://go-review.googlesource.com/c/go/+/698797
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
On Windows it is not possible to do concurrent I/O on file handles due
to the way FD.Pread and FD.Pwrite are implemented. This serialization is
achieved by having a dedicated mutex locked in the affected FD methods.
This makes the code difficult to reason about, as there is another
layer of locking introduced by the fdMutex. For example, it is not
obvious that concurrent I/O operations are serialized.
This CL removed the dedicated mutex and uses the fdMutex to provide
read/write locking.
Change-Id: I00389662728ce29428a587c3189bab90a0399215
Reviewed-on: https://go-review.googlesource.com/c/go/+/698096
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
|
|
CL 692436 changed WriteMsgInet{4,6} on windows to pass a zero namelen
when the sockaddr is nil. Turns out Windows also requires name to be
nil when namelen is 0.
With this commit, WriteMsgInet4 and WriteMsgInet6 now nicely align with
WriteMsg.
For #74841
Change-Id: Ifadee2d12d9bce2411f11a0e12b9fa2b3d71990e
Reviewed-on: https://go-review.googlesource.com/c/go/+/698395
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
|
|
For #74841
Change-Id: If2ea23b1eb23e32680bd576f54a0020d7e115797
Reviewed-on: https://go-review.googlesource.com/c/go/+/692436
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
Overlapped handles don't have the file pointer updated when performing
I/O operations, so there is no need to call FD.Seek to reset the file
pointer.
Also, some overlapped file handles don't support seeking. See #74951.
Fixes #74951.
Change-Id: I0edd53beed7d3862730f3b2ed5fe9ba490e66c06
Reviewed-on: https://go-review.googlesource.com/c/go/+/697295
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
|
|
handles
Windows doesn't keep the file pointer for overlapped file handles.
To work around this, we keep track of the current offset ourselves
and use it on every Read/Write operation.
When the user calls File.Seek with whence == io.SeekCurrent, it expects
that the offset we keep track of is also accounted for, else the
the seek'ed value won't match the file pointer seen by the user.
Updates #74951.
Fixes #75081.
Change-Id: Ieca7c3779e5349292883ffc293a8474088a4dec7
Reviewed-on: https://go-review.googlesource.com/c/go/+/697275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
|
|
There is no need to keep the msg field in the poll.operation struct.
This skims down the size of os.File by 112 bytes.
Change-Id: I5c7b1f3989f9bb5f1748df2cba8128d9c479b35d
Reviewed-on: https://go-review.googlesource.com/c/go/+/685418
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Mark Freeman <mark@golang.org>
|
|
There is no need to keep the rsan field in the poll.operation struct.
This skims down the size of os.File by 16 bytes.
Change-Id: I5e99e0e87178b63a19f0b9883b7b3d25abfd9ec3
Reviewed-on: https://go-review.googlesource.com/c/go/+/685417
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Mark Freeman <mark@golang.org>
|
|
There is no need to keep the sa field in the poll.operation struct.
This skims down the size of os.File by 32 bytes.
Change-Id: I6b021a76f582ead5dccb29b001e7a5b068a2c2ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/685416
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Mark Freeman <mark@golang.org>
|
|
There is no need to keep the qty and flags fields in the poll.operation
struct.
This skims down the size of os.File by 16 bytes and makes poll.operation
harder to misuse.
Change-Id: I8943d88f29ed3c7eefbb83114b0d31052abbe646
Reviewed-on: https://go-review.googlesource.com/c/go/+/685436
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Mark Freeman <mark@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
The handle field can be accessed directly wherever needed, there is
no need to store it in the operation struct.
This skims down the size of os.File by 16 bytes.
Change-Id: I87c94cb773437891127b6c36dc7f8883622ffed3
Reviewed-on: https://go-review.googlesource.com/c/go/+/685435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
There is no need to keep the fd in the poll.operation struct,
given that all usages of this field have direct access to the fd struct.
This skims down the size of os.File by 16 bytes.
Change-Id: I837e345250387f62e294cc1772d752865a04ef6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/685415
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
|
|
The WSASocket documentation states that the returned socket must be
closed by calling closesocket instead of CloseHandle. The different
File methods on the net package return an os.File that is not aware
that it should use closesocket. Ideally, os.NewFile should detect that
the passed handle is a socket and use the appropriate close function,
but there is no reliable way to detect that a handle is a socket on
Windows (see CL 671455).
To work around this, we add a hidden function to the os package that
can be used to return an os.File that uses closesocket. This approach
is the same as used on Unix, which also uses a hidden function for other
purposes.
While here, fix a potential issue with FileConn, which was using File.Fd
rather than File.SyscallConn to get the handle. This could result in the
File being closed and garbage collected before the syscall was made.
Fixes #73683.
Change-Id: I179405f34c63cbbd555d8119e0f77157c670eb3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/672195
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Replace the use of SetFinalizer with AddCleanup.
For #70907
Change-Id: I0cb2c2985eb9285e5f92be9dbcb9d77acc0f59c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/671441
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
|
|
Go 1.25 will gain support for overlapped IO on handles passed to
os.NewFile thanks to CL 662236. It was previously not possible to add
an overlapped handle to the Go runtime's IO completion port (IOCP),
and now happens on the first call the an IO method.
This means that there is code that relies on the fact that File.Fd
returns a handle that can always be associated with a custom IOCP.
That wouldn't be the case anymore, as a handle can only be associated
with one IOCP at a time and it must be explicitly disassociated.
To fix this breaking change, File.Fd will disassociate the handle
from the Go runtime IOCP before returning it. It is then not necessary
to defer the association until the first IO method is called, which
was recently added in CL 661955 to support this same use case, but
in a more complex and unreliable way.
Updates #19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race,gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Id8a7e04d35057047c61d1733bad5bf45494b2c28
Reviewed-on: https://go-review.googlesource.com/c/go/+/664455
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
|
TestFileFdsAreInitialised and TestSerialFdsAreInitialised were added
to ensure handles passed to os.NewFile were not added to the runtime
poller. This used to be problematic because the poller could crash
if an external I/O event was received (see #21172).
This is not an issue anymore since CL 482495 and #19098.
Change-Id: I292ceae27724fefe6f438a398ebfe351dd5231d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/665315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
This fixes the support for I/O on overlapped files that are not added to
the poller. Note that CL 661795 already added support for that, but it
really only worked for pipes, not for plain files.
Additionally, this CL also makes this kind of I/O operations to not
notify the external poller to avoid confusing it.
Updates #15388.
Change-Id: I15c6ea74f3a87960aef0986598077b6eab9b9c99
Reviewed-on: https://go-review.googlesource.com/c/go/+/664415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
|
|
Change-Id: Ie76ebb556d635068342747f3f91dd7dc423df531
GitHub-Last-Rev: aea61fb3a054e6bd24f4684f90fb353d5682cd0b
GitHub-Pull-Request: golang/go#73340
Reviewed-on: https://go-review.googlesource.com/c/go/+/664677
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
|
|
The sendfile implementation for platforms supporting it is now in
net/sendfile.go, rather than being duplicated in separate files for
each platform.
The only difference between the implementations was the poll.SendFile
parameters, which have been harmonized, and also linux strictly
asserting for os.File, which now have been relaxed to allow any
type implementing syscall.Conn.
Change-Id: Ia1a2d5ee7380710a36fc555dbf681f7e996ea2ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/664075
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
|
|
Windows sendfile optimization is skipped since CL 472475, which started
passing an os.fileWithoutWriteTo instead of an os.File to sendfile,
and that function was only implemented for os.File.
This CL fixes the issue by asserting against an interface rather than
a concrete type.
Some tests have been reenabled, triggering bugs in poll.SendFile which
have been fixed in this CL.
Fixes #67042.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: Id6f7a0e1e0f34a72216fa9d00c5bf36f5a994219
Reviewed-on: https://go-review.googlesource.com/c/go/+/664055
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
|
|
There is a potential race between a concurrent call to FD.initIO, which
calls FD.pd.init, and a call to FD.Close, which calls FD.pd.evict.
This is solved by calling FD.initIO in FD.Close, as that will block
until the concurrent FD.initIO has completed. Note that FD.initIO is
no-op if first called from here.
The race window is so small that it is not possible to write a test
that triggers it.
Change-Id: Ie2f2818e746b9d626fe3b9eb6b8ff967c81ef863
Reviewed-on: https://go-review.googlesource.com/c/go/+/663815
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
|
|
The runtime/poll package has just gained support for overlapped IO,
see CL 660595 and CL 661955. The only remaining piece was making it
visible to user code via os.NewFile.
Some of the poll.FD.Init responsibility has been moved to os.NewFile
to avoid unnecessary syscalls for the common case of using os.Open,
os.Create, os.OpenFile, and os.Pipe, where we know that the file
is not opened for overlapped IO.
Some internal/poll tests have been moved to the os package to exercise
public APIs rather than internal ones.
The os.NewFile function definition has been moved into an OS-agnostic
file to avoid having duplicated documentation and ensure that the
caller is aware of its behavior across all platforms.
Closes #19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race,gotip-windows-arm64
Change-Id: If043f8b34d588cd4b481777203107ed92d660fd9
Reviewed-on: https://go-review.googlesource.com/c/go/+/662236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
|