aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/proc.go
AgeCommit message (Collapse)Author
2020-01-10runtime: don't let P's timer heap get clogged with deleted timersIan Lance Taylor
Whenever more than 1/4 of the timers on a P's heap are deleted, remove them from the heap. Change-Id: Iff63ed3d04e6f33ffc5c834f77f645c52c007e52 Reviewed-on: https://go-review.googlesource.com/c/go/+/214299 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-01-02runtime: emit trace event in direct semaphore handoffRhys Hiltner
When a goroutine yields the remainder of its time to another goroutine during direct semaphore handoff (as in an Unlock of a sync.Mutex in starvation mode), it needs to signal that change to the execution tracer. The discussion in CL 200577 didn't reach consensus on how best to describe that, but pointed out that "traceEvGoSched / goroutine calls Gosched" could be confusing. Emit a "traceEvGoPreempt / goroutine is preempted" event in this case, to allow the execution tracer to find a consistent event ordering without being both specific and inaccurate about why the active goroutine has changed. Fixes #36186 Change-Id: Ic4ade19325126db2599aff6aba7cba028bb0bee9 Reviewed-on: https://go-review.googlesource.com/c/go/+/211797 Run-TryBot: Dan Scales <danscales@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-11-19runtime, time: remove old timer codeIan Lance Taylor
Updates #6239 Updates #27707 Change-Id: I65e6471829c9de4677d3ac78ef6cd7aa0a1fc4cb Reviewed-on: https://go-review.googlesource.com/c/go/+/171884 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-19runtime: release timersLock while running timerIan Lance Taylor
Dan Scales pointed out a theoretical deadlock in the runtime. The timer code runs timer functions while holding the timers lock for a P. The scavenger queues up a timer function that calls wakeScavenger, which acquires the scavenger lock. The scavengeSleep function acquires the scavenger lock, then calls resetTimer which can call addInitializedTimer which acquires the timers lock for the current P. So there is a potential deadlock, in that the scavenger lock and the timers lock for some P may both be acquired in different order. It's not clear to me whether this deadlock can ever actually occur. Issue 35532 describes another possible deadlock. The pollSetDeadline function acquires pd.lock for some poll descriptor, and in some cases calls resettimer which can in some cases acquire the timers lock for the current P. The timer code runs timer functions while holding the timers lock for a P. The timer function for poll descriptors winds up in netpolldeadlineimpl which acquires pd.lock. So again there is a potential deadlock, in that the pd lock for some poll descriptor and the timers lock for some P may both be acquired in different order. I think this can happen if we change the deadline for a network connection exactly as the former deadline expires. Looking at the code, I don't see any reason why we have to hold the timers lock while running a timer function. This CL implements that change. Updates #6239 Updates #27707 Fixes #35532 Change-Id: I17792f5a0120e01ea07cf1b2de8434d5c10704dd Reviewed-on: https://go-review.googlesource.com/c/go/+/207348 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-13runtime: acquire timersLocks around moveTimersIan Lance Taylor
In the discussion of CL 171828 we decided that it was not necessary to acquire timersLock around the call to moveTimers, because the world is stopped. However, that is not correct, as sysmon runs even when the world is stopped, and it calls timeSleepUntil which looks through the timers. timeSleepUntil acquires timersLock, but that doesn't help if moveTimers is running at the same time. Updates #6239 Updates #27707 Updates #35462 Change-Id: I346c5bde594c4aff9955ae430b37c2b6fc71567f Reviewed-on: https://go-review.googlesource.com/c/go/+/206938 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-12runtime: consistently seed fastrand state across archsCarlo Alberto Ferraris
Some, but not all, architectures mix in OS-provided random seeds when initializing the fastrand state. The others have TODOs saying we need to do the same. Lift that logic up in the architecture-independent part, and use memhash to mix the seed instead of a simple addition. Previously, dumping the fastrand state at initialization would yield something like the following on linux-amd64, where the values in the first column do not change between runs (as thread IDs are sequential and always start at 0), and the values in the second column, while changing every run, are pretty correlated: first run: 0x0 0x44d82f1c 0x5f356495 0x44f339de 0xbe6ac92a 0x44f91cd8 0x1da02dbf 0x44fd91bc 0x7cd59254 0x44fee8a4 0xdc0af6e9 0x4547a1e0 0x3b405b7e 0x474c76fc 0x9a75c013 0x475309dc 0xf9ab24a8 0x4bffd075 second run: 0x0 0xa63fc3eb 0x5f356495 0xa6648dc2 0xbe6ac92a 0xa66c1c59 0x1da02dbf 0xa671bce8 0x7cd59254 0xa70e8287 0xdc0af6e9 0xa7129d2e 0x3b405b7e 0xa7379e2d 0x9a75c013 0xa7e4c64c 0xf9ab24a8 0xa7ecce07 With this change, we get initial states that appear to be much more unpredictable, both within the same run as well as between runs: 0x11bddad7 0x97241c63 0x553dacc6 0x2bcd8523 0x62c01085 0x16413d92 0x6f40e9e6 0x7a138de6 0xa4898053 0x70d816f0 0x5ca5b433 0x188a395b 0x62778ca9 0xd462c3b5 0xd6e160e4 0xac9b4bd 0xb9571d65 0x597a981d Change-Id: Ib22c530157d74200df0083f830e0408fd4aaea58 Reviewed-on: https://go-review.googlesource.com/c/go/+/203439 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-09sync: yield to the waiter when unlocking a starving mutexRhys Hiltner
When we have already assigned the semaphore ticket to a specific waiter, we want to get the waiter running as fast as possible since no other G waiting on the semaphore can acquire it optimistically. The net effect is that, when a sync.Mutex is contended, the code in the critical section guarded by the Mutex gets a priority boost. Fixes #33747 The original work was done in CL 200577 by Carlo Alberto Ferraris. The change was reverted in CL 205817 because it broke the linux-arm64-packet and solaris-amd64-oraclerel builders. Change-Id: I76d79b1d63fd206ed1c57fe6900cb7ae9e4d46cb Reviewed-on: https://go-review.googlesource.com/c/go/+/206180 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-08runtime: add per-p page allocation cacheMichael Anthony Knyszek
This change adds a per-p free page cache which the page allocator may allocate out of without a lock. The change also introduces a completely lockless page allocator fast path. Although the cache contains at most 64 pages (and usually less), the vast majority (85%+) of page allocations are exactly 1 page in size. Updates #35112. Change-Id: I170bf0a9375873e7e3230845eb1df7e5cf741b78 Reviewed-on: https://go-review.googlesource.com/c/go/+/195701 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com>
2019-11-08runtime: add per-p mspan cacheMichael Anthony Knyszek
This change adds a per-p mspan object cache similar to the sudog cache. Unfortunately this cache can't quite operate like the sudog cache, since it is used in contexts where write barriers are disallowed (i.e. allocation codepaths), so rather than managing an array and a slice, it's just an array and a length. A little bit more unsafe, but avoids any write barriers. The purpose of this change is to reduce the number of operations which require the heap lock in allocation, paving the way for a lockless fast path. Updates #35112. Change-Id: I32cfdcd8528fb7be985640e4f3a13cb98ffb7865 Reviewed-on: https://go-review.googlesource.com/c/go/+/196642 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-11-07Revert "sync: yield to the waiter when unlocking a starving mutex"Bryan C. Mills
This reverts CL 200577. Reason for revert: broke linux-arm64-packet and solaris-amd64-oraclerel builders Fixes #35424 Updates #33747 Change-Id: I2575fd84d37995d458183caae54704f15d8b8426 Reviewed-on: https://go-review.googlesource.com/c/go/+/205817 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-07sync: yield to the waiter when unlocking a starving mutexCarlo Alberto Ferraris
When we have already assigned the semaphore ticket to a specific waiter, we want to get the waiter running as fast as possible since no other G waiting on the semaphore can acquire it optimistically. The net effect is that, when a sync.Mutex is contented, the code in the critical section guarded by the Mutex gets a priority boost. Fixes #33747 Change-Id: I9967f0f763c25504010651bdd7f944ee0189cd45 Reviewed-on: https://go-review.googlesource.com/c/go/+/200577 Reviewed-by: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-06runtime: don't hold scheduler lock when calling timeSleepUntilIan Lance Taylor
Otherwise, we can get into a deadlock: sysmon takes the scheduler lock and calls timeSleepUntil which takes each P's timer lock. Simultaneously, some P calls runtimer (holding the P's own timer lock) which wakes up the scavenger, calling goready, calling wakep, calling startm, getting the scheduler lock. Now the sysmon thread is holding the scheduler lock and trying to get a P's timer lock, while some other thread running on that P is holding the P's timer lock and trying to get the scheduler lock. So change sysmon to call timeSleepUntil without holding the scheduler lock, and change timeSleepUntil to use allpLock, which is only held for limited periods of time and should never compete with timer locks. This hopefully Fixes #35375 At least it should fix the linux-arm64-packet builder problems, which occurred more reliably as that system has GOMAXPROCS == 96, giving a lot more scope for this deadlock. Change-Id: I7a7917daf7a4882e0b27ca416e4f6300cfaaa774 Reviewed-on: https://go-review.googlesource.com/c/go/+/205558 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-04runtime: wake netpoller when dropping P, don't sleep too long in sysmonIan Lance Taylor
When dropping a P, if it has any timers, and if some thread is sleeping in the netpoller, wake the netpoller to run the P's timers. This mitigates races between the netpoller deciding how long to sleep and a new timer being added. In sysmon, if all P's are idle, check the timers to decide how long to sleep. This avoids oversleeping if no thread is using the netpoller. This can happen in particular if some threads use runtime.LockOSThread, as those threads do not block in the netpoller. Also, print the number of timers per P for GODEBUG=scheddetail=1. Before this CL, TestLockedDeadlock2 would fail about 1% of the time. With this CL, I ran it 150,000 times with no failures. Updates #6239 Updates #27707 Fixes #35274 Fixes #35288 Change-Id: I7e5193e6c885e567f0b1ee023664aa3e2902fcd1 Reviewed-on: https://go-review.googlesource.com/c/go/+/204800 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-04cmd/compile: add test for skipping empty init functionsCuong Manh Le
CL 200958 adds skipping empty init function feature without any tests for it. A codegen test sounds ideal, but it's unlikely that we can make one for now, so use a program to manipulate runtime/proc.go:initTask directly. Updates #34869 Change-Id: I2683b9a1ace36af6861af02a3a9fb18b3110b282 Reviewed-on: https://go-review.googlesource.com/c/go/+/204217 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-02runtime: implement async scheduler preemptionAustin Clements
This adds signal-based preemption to preemptone. Since STW and forEachP ultimately use preemptone, this also makes these work with async preemption. This also makes freezetheworld more robust so tracebacks from fatal panics should be far less likely to report "goroutine running on other thread; stack unavailable". For #10958, #24543. (This doesn't fix it yet because asynchronous preemption only works on POSIX platforms on 386 and amd64 right now.) Change-Id: If776181dd5a9b3026a7b89a1b5266521b95a5f61 Reviewed-on: https://go-review.googlesource.com/c/go/+/201762 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02runtime: clear preemptStop in dropmIan Lance Taylor
Updates #10958 Updates #24543 Fixes #35294 Change-Id: I60f024d08451565df6d9751dab9832b50cbf637a Reviewed-on: https://go-review.googlesource.com/c/go/+/204957 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-01runtime: unlock sched lock when checkdead throws due to a deadlockIan Lance Taylor
I was doing some testing with GODEBUG=schedtrace=1,scheddetail=1 and I noticed that the program hung after a throw with "all goroutines are asleep". This is because when doing a throw or fatal panic with schedtrace the panic code does a final schedtrace, which needs to acquire the scheduler lock. The checkdead function is always called with the scheduler lock held. So checkdead would throw with the scheduler lock held, then the panic code would call schedtrace, which would block trying to acquire the scheduler lock. This problem will only happen for people debugging the runtime, but it's easy to avoid by having checkdead unlock the scheduler lock before it throws. I only did this for the throws that can happen for a normal program, not for throws that indicate some corruption in the scheduler data. Change-Id: Ic62277b3ca6bee6f0fca8d5eb516c59cb67855cb Reviewed-on: https://go-review.googlesource.com/c/go/+/204778 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-29runtime: clear m.gsignal when the M exitsCherry Zhang
On some platforms (currently ARM and ARM64), when calling into VDSO we store the G to the gsignal stack, if there is one, so if we receive a signal during VDSO we can find the G. When an M exits, it frees the gsignal stack. But m.gsignal.stack still points to that stack. When we call nanotime on this M, we will write to the already freed gsignal stack, which is bad. Prevent this by unlinking the freed stack from the M. Should fix #35235. Change-Id: I338b1fc8ec62aae036f38afaca3484687e11a40d Reviewed-on: https://go-review.googlesource.com/c/go/+/204158 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-25runtime: abstract M preemption check into a functionAustin Clements
We check whether an M is preemptible in a surprising number of places. Put it in one function. For #10958, #24543. Change-Id: I305090fdb1ea7f7a55ffe25851c1e35012d0d06c Reviewed-on: https://go-review.googlesource.com/c/go/+/201439 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25runtime: remove g.gcscanvalidAustin Clements
Currently, gcscanvalid is used to resolve a race between attempts to scan a stack. Now that there's a clear owner of the stack scan operation, there's no longer any danger of racing or attempting to scan a stack more than once, so this CL eliminates gcscanvalid. I double-checked my reasoning by first adding a throw if gcscanvalid was set in scanstack and verifying that all.bash still passed. For #10958, #24543. Fixes #24363. Change-Id: I76794a5fcda325ed7cfc2b545e2a839b8b3bc713 Reviewed-on: https://go-review.googlesource.com/c/go/+/201139 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25runtime: remove old stack scanning codeAustin Clements
This removes scang and preemptscan, since the stack scanning code now uses suspendG. For #10958, #24543. Change-Id: Ic868bf5d6dcce40662a82cb27bb996cb74d0720e Reviewed-on: https://go-review.googlesource.com/c/go/+/201138 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25runtime: add general suspendG/resumeGAustin Clements
Currently, the process of suspending a goroutine is tied to stack scanning. In preparation for non-cooperative preemption, this CL abstracts this into general purpose suspendG/resumeG functions. suspendG and resumeG closely follow the existing scang and restartg functions with one exception: the addition of a _Gpreempted status. Currently, preemption tasks (stack scanning) are carried out by the target goroutine if it's in _Grunning. In this new approach, the task is always carried out by the goroutine that called suspendG. Thus, we need a reliable way to drive the target goroutine out of _Grunning until the requesting goroutine is ready to resume it. The new _Gpreempted state provides the handshake: when a runnable goroutine responds to a preemption request, it now parks itself and enters _Gpreempted. The requesting goroutine races to put it in _Gwaiting, which gives it ownership, but also the responsibility to start it again. This CL adds several TODOs about improving the synchronization on the G status. The existing code already has these problems; we're just taking note of them. The next CL will remove the now-dead scang and preemptscan. For #10958, #24543. Change-Id: I16dbf87bea9d50399cc86719c156f48e67198f16 Reviewed-on: https://go-review.googlesource.com/c/go/+/201137 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25runtime: ensure _Grunning Gs have a valid g.m and g.m.pAustin Clements
We already claim on the documentation for _Grunning that this is case, but execute transitions to _Grunning before assigning g.m. Fix this and make the documentation even more explicit. For #10958, #24543, but also a good cleanup. Change-Id: I1eb0108e7762f55cfb0282aca624af1c0a15fe56 Reviewed-on: https://go-review.googlesource.com/c/go/+/201440 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-23runtime: save/fetch g register during VDSO on ARM and ARM64Cherry Zhang
On ARM and ARM64, during a VDSO call, the g register may be temporarily clobbered by the VDSO code. If a signal is received during the execution of VDSO code, we may not find a valid g reading the g register. In CL 192937, we conservatively assume g is nil. But this approach has a problem: we cannot handle the signal in this case. Further, if the signal is not a profiling signal, we'll call badsignal, which calls needm, which wants to get an extra m, but we don't have one in a non-cgo binary, which cuases the program to hang. This is even more of a problem with async preemption, where we will receive more signals than before. I ran into this problem while working on async preemption support on ARM64. In this CL, before making a VDSO call, we save the g on the gsignal stack. When we receive a signal, we will be running on the gsignal stack, so we can fetch the g from there and move on. We probably want to do the same for PPC64. Currently we rely on that the VDSO code doesn't actually clobber the g register, but this is not guaranteed and we don't have control with. Idea from discussion with Dan Cross and Austin. Should fix #34391. Change-Id: Idbefc5e4c2f4373192c2be797be0140ae08b26e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/202759 Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Austin Clements <austin@google.com>
2019-10-23runtime: add race detector support for new timersIan Lance Taylor
Since the new timers run on g0, which does not have a race context, we add a race context field to the P, and use that for timer functions. This works since all timer functions are in the standard library. Updates #27707 Change-Id: I8a5b727b4ddc8ca6fc60eb6d6f5e9819245e395b Reviewed-on: https://go-review.googlesource.com/c/go/+/171882 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-10-23runtime: update timejump function for new timersIan Lance Taylor
Since timers are now on a P, rather than having a G running timerproc, timejump changes to return a P rather than a G. Updates #27707 Change-Id: I3d05af2d664409a0fd906e709fdecbbcbe00b9a7 Reviewed-on: https://go-review.googlesource.com/c/go/+/171880 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-22runtime: add new modtimer functionIan Lance Taylor
This adds a new field to P, adjustTimers, that tells the P that one of its existing timers was modified to be earlier, and that it therefore needs to resort them. Updates #27707 Change-Id: I4c5f5b51ed116f1d898d3f87cdddfa1b552337f8 Reviewed-on: https://go-review.googlesource.com/c/go/+/171832 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-21runtime: handle timers on P's in procresize/(*pp).destroyIan Lance Taylor
Updates #6239 Updates #27707 Change-Id: I52cab8bf3dc8c552463725fc1d9e4e6b12230b03 Reviewed-on: https://go-review.googlesource.com/c/go/+/171828 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-21runtime: add wasm support for timers on P'sIan Lance Taylor
When we put timers on P's, the wasm code will not be able to rely on the timer goroutine. Use the beforeIdle hook to schedule a wakeup. Updates #6239 Updates #27707 Change-Id: Idf6309944778b8c3d7178f5d09431940843ea233 Reviewed-on: https://go-review.googlesource.com/c/go/+/171827 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-10-21cmd/compile: add fma intrinsic for armsmasher164
This change introduces an arm intrinsic that generates the FMULAD instruction for the fused-multiply-add operation on systems that support it. System support is detected via cpu.ARM.HasVFPv4. A rewrite rule translates the generic intrinsic to FMULAD. Updates #25819. Change-Id: I8459e5dd1cdbdca35f88a78dbeb7d387f1e20efa Reviewed-on: https://go-review.googlesource.com/c/go/+/142117 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-10-21runtime: initial scheduler changes for timers on P'sIan Lance Taylor
Add support to the main scheduler loop for handling timers on P's. This is not used yet, as timers are not yet put on P's. Updates #6239 Updates #27707 Change-Id: I6a359df408629f333a9232142ce19e8be8496dae Reviewed-on: https://go-review.googlesource.com/c/go/+/171826 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-21cmd/compile: add fma intrinsic for amd64smasher164
To permit ssa-level optimization, this change introduces an amd64 intrinsic that generates the VFMADD231SD instruction for the fused-multiply-add operation on systems that support it. System support is detected via cpu.X86.HasFMA. A rewrite rule can then translate the generic ssa intrinsic ("Fma") to VFMADD231SD. The benchmark compares the software implementation (old) with the intrinsic (new). name old time/op new time/op delta Fma-4 27.2ns ± 1% 1.0ns ± 9% -96.48% (p=0.008 n=5+5) Updates #25819. Change-Id: I966655e5f96817a5d06dff5942418a3915b09584 Reviewed-on: https://go-review.googlesource.com/c/go/+/137156 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-10-15runtime: change netpoll to take an amount of time to blockIan Lance Taylor
This new facility will be used by future CLs in this series. Change the only blocking call to netpoll to do the right thing when netpoll returns an empty list. Updates #6239 Updates #27707 Change-Id: I58b3c2903eda61a3698b1a4729ed0e81382bb1ed Reviewed-on: https://go-review.googlesource.com/c/go/+/171821 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-09-26runtime: detail the method commentTardis Xu
Change the comment to make more conformable to the function implementation. Change-Id: I8461e2f09824c50e16223a27d0f61070f04bd21b GitHub-Last-Rev: c25a8493d3938b38e2c318f7a7b94c9f2eb11bb4 GitHub-Pull-Request: golang/go#27404 Reviewed-on: https://go-review.googlesource.com/c/go/+/132477 Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-09-04runtime: don't hold worldsema across mark phaseMichael Anthony Knyszek
This change makes it so that worldsema isn't held across the mark phase. This means that various operations like ReadMemStats may now stop the world during the mark phase, reducing latency on such operations. Only three such operations are still no longer allowed to occur during marking: GOMAXPROCS, StartTrace, and StopTrace. For the former it's because any change to GOMAXPROCS impacts GC mark background worker scheduling and the details there are tricky. For the latter two it's because tracing needs to observe consistent GC start and GC end events, and if StartTrace or StopTrace may stop the world during marking, then it's possible for it to see a GC end event without a start or GC start event without an end, respectively. To ensure that GOMAXPROCS and StartTrace/StopTrace cannot proceed until marking is complete, the runtime now holds a new semaphore, gcsema, across the mark phase just like it used to with worldsema. Fixes #19812. Change-Id: I15d43ed184f711b3d104e8f267fb86e335f86bf9 Reviewed-on: https://go-review.googlesource.com/c/go/+/182657 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-08-28runtime: simplify GOOS detection in mstart()Ayan George
The existing condition is long and repetitive. Using select/case with multiple values in the expression list is more concise and clearer. Change-Id: I43f8abcf958e433468728f1d89ff1436332b29da Reviewed-on: https://go-review.googlesource.com/c/go/+/188519 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-27runtime: simplify some pointer conversionsMatthew Dempsky
Use efaceOf to safely convert from *interface{} to *_eface, and to make it clearer what the pointer arithmetic is computing. Incidentally, remove a spurious unsafe.Pointer->*uint8->unsafe.Pointer round trip conversion in newproc. No behavior change. Change-Id: I2ad9d791d35d8bd008ef43b03dad1589713c5fd4 Reviewed-on: https://go-review.googlesource.com/c/go/+/190457 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-06-28runtime: fix pprof cpu profile corruption on arm/mips/mipsleRuss Cox
CL 42652 changed the profile handler for mips/mipsle to avoid recording a profile when in atomic functions, for fear of interrupting the 32-bit simulation of a 64-bit atomic with a lock. The profile logger itself uses 64-bit atomics and might deadlock (#20146). The change was to accumulate a count of dropped profile events and then send the count when the next ordinary event was sent: if prof.hz != 0 { + if (GOARCH == "mips" || GOARCH == "mipsle") && lostAtomic64Count > 0 { + cpuprof.addLostAtomic64(lostAtomic64Count) + lostAtomic64Count = 0 + } cpuprof.add(gp, stk[:n]) } CL 117057 extended this behavior to include GOARCH == "arm". Unfortunately, the inserted cpuprof.addLostAtomic64 differs from the original cpuprof.add in that it neglects to acquire the lock protecting the profile buffer. This has caused a steady stream of flakes on the arm builders for the past 12 months, ever since CL 117057 landed. This CL moves the lostAtomic count into the profile buffer and then lets the existing addExtra calls take care of it, instead of duplicating the locking logic. Fixes #24991. Change-Id: Ia386c40034fcf46b31f080ce18f2420df4bb8004 Reviewed-on: https://go-review.googlesource.com/c/go/+/184164 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-06-06runtime: mark all Go symbols called from assembly in other packagesAustin Clements
This marks all Go symbols called from assembly in other packages with "go:linkname" directives to ensure they get ABI wrappers. Now that we have this go:linkname convention, this also removes the abi0Syms definition in the runtime, which was used to give morestackc an ABI0 wrapper. Instead, we now just mark morestackc with a go:linkname directive. This was tested with buildall.bash in the default configuration, with -race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't test cgo on non-Linux configurations, I manually grepped for runtime symbols in runtime/cgo. Updates #31230. Change-Id: I6c8aa56be2ca6802dfa2bf159e49c411b9071bf1 Reviewed-on: https://go-review.googlesource.com/c/go/+/179862 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-05-22runtime: revert init order changesKeith Randall
First, remove the randomization of initialization order. Then, revert to source code order instead of sorted package path order. This restores the behavior that was in 1.12. A larger change which will implement the suggestion in #31636 will wait for 1.14. It's too complicated for 1.13 at this point (it has tricky interactions with plugins). Fixes #31636 Change-Id: I35b48e8cc21cf9f93c0973edd9193d2eac197628 Reviewed-on: https://go-review.googlesource.com/c/go/+/178297 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-13runtime: resolve latent function type TODOTamir Duberstein
This was left over from the C->Go transition. Change-Id: I52494af3d49a388dc45b57210ba68292ae01cf84 Reviewed-on: https://go-review.googlesource.com/c/go/+/176897 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-05-09runtime: remove periodic scavengingMichael Anthony Knyszek
This change removes the periodic scavenger which goes over every span in the heap and scavenges it if it hasn't been used for 5 minutes. It should no longer be necessary if we have background scavenging (follow-up). For #30333. Change-Id: Ic3a1a4e85409dc25719ba4593a3b60273a4c71e0 Reviewed-on: https://go-review.googlesource.com/c/go/+/143157 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-05-01runtime: look for idle p to run current goroutine when switching to GC or ↵David Chase
traceReader This repairs one of the several causes of pauses uncovered by a GC microbenchmark. A pause can occur when a goroutine's quantum expires "at the same time" a GC is needed. The current M switches to running a GC worker, which means that the amount of available work has expanded by one. The GC worker, however, does not call ready, and does not itself conditionally wake a P (a "normal" thread would do this). This is also true if M switches to a traceReader. This is problem 4 in this list: https://github.com/golang/go/issues/27732#issuecomment-423301252 Updates #27732. Change-Id: I6905365cac8504cde6faab2420f4421536551f0b Reviewed-on: https://go-review.googlesource.com/c/go/+/146817 Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-05-01runtime: change the span allocation policy to first-fitMichael Anthony Knyszek
This change modifies the treap implementation to be address-ordered instead of size-ordered, and further augments it so it may be used for allocation. It then modifies the find method to implement a first-fit allocation policy. This change to the treap implementation consequently makes it so that spans are scavenged in highest-address-first order without any additional changes to the scavenging code. Because the treap itself is now address ordered, and the scavenging code iterates over it in reverse, the highest address is now chosen instead of the largest span. This change also renames the now wrongly-named "scavengeLargest" method on mheap to just "scavengeLocked" and also fixes up logic in that method which made assumptions about size. For #30333. Change-Id: I94b6f3209211cc1bfdc8cdaea04152a232cfbbb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/164101 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-04-30all: add new GOOS=illumos, split out of GOOS=solarisJoshua M. Clulow
Like GOOS=android which implies the "linux" build tag, GOOS=illumos implies the "solaris" build tag. This lets the existing ecosystem of packages still work on illumos, but still permits packages to start differentiating between solaris and illumos. Fixes #20603 Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622 Reviewed-on: https://go-review.googlesource.com/c/go/+/174457 Run-TryBot: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-29runtime: account for callbacks in checkdead on WindowsIan Lance Taylor
When a callback runs on a different thread in Windows, as in the runtime package test TestCallbackInAnotherThread, it will use the extra M. That can cause the test in checkdead to fail incorrectly. Check whether there actually is an extra M before expecting it. I think this is a general problem unrelated to timers. I think the test was passing previously because the timer goroutine was using an M. But I haven't proved that. This change seems correct, and it avoids the test failure when using the new timers on Windows. Updates #27707 Change-Id: Ieb31c04ff0354d6fae7e173b59bcfadb8b0464cd Reviewed-on: https://go-review.googlesource.com/c/go/+/174037 Reviewed-by: Keith Randall <khr@golang.org>
2019-04-25runtime: switch to P 0 before destroying current PAustin Clements
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize happens to be running on a P that's being destroyed, it moves itself to P 0. However, currently procresize destroys the unused Ps *before* moving itself to P 0. This means it may briefly run on a destroyed P. This is basically harmless, but has at least one very confusing consequence: since destroying a P has write barriers, it may enqueue pointers to a destroyed write barrier buffer. As far as I can tell, there are no negative consequences of this, but this seems really fragile. This CL swaps the order of things, so now procresize moves itself to P 0 if necessary before destroying Ps. This ensures it always has a valid P. This is part of refactoring for #10958 and #24543, but is a good cleanup regardless. Change-Id: I91a23dd6ed27e372f8d6291feec9bc991bcf9812 Reviewed-on: https://go-review.googlesource.com/c/go/+/173941 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-04-23runtime: randomize package initialization order in race modeKeith Randall
This is one small step to force people to not depend on the order of initialization of packages which are not explicitly ordered by import directives. Similar to randomizing map iteration order, this makes sure people aren't depending on the behavior of the current release, so that we can change the order in future releases without breaking everyone. Maybe one day we can randomize always, but for now we do it just in race mode. (We would need to measure the impact on startup time before we enabled it always.) RELNOTE=yes Change-Id: I99026394796125974c5f2c3660a88becb92c9df3 Reviewed-on: https://go-review.googlesource.com/c/go/+/170318 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2019-04-23cmd/go: move runtime/debug.modinfo to runtime.modinfoRuss Cox
It is easier to ensure that the symbol is always present if we move it to package runtime. Avoids init-time work. Also moves it next to buildVersion, the other similar symbol. Setting up for "go version <binary>". For #31624. Change-Id: I943724469ce6992153e701257eb6f12da88c8e4e Reviewed-on: https://go-review.googlesource.com/c/go/+/173341 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-22runtime: improve mstart commentsAustin Clements
Some of the comments were unclear or outdated. Change-Id: I02e01bf60def0074c1fa760e94aa992e9e4969b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/172987 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
generated by cgit v1.3-5-g9baa (git 2.53.0) at 2026-04-17 10:36:19 +0000