aboutsummaryrefslogtreecommitdiff
path: root/src/testing/loop_test.go
AgeCommit message (Collapse)Author
2025-03-26[release-branch.go1.24] testing: detect a stopped timer in B.LoopAustin Clements
Currently, if the user stops the timer in a B.Loop benchmark loop, the benchmark will run until it hits the timeout and fails. Fix this by detecting that the timer is stopped and failing the benchmark right away. We avoid making the fast path more expensive for this check by "poisoning" the B.Loop iteration counter when the timer is stopped so that it falls back to the slow path, which can check the timer. This causes b to escape from B.Loop, which is totally harmless because it was already definitely heap-allocated. But it causes the test/inline_testingbloop.go errorcheck test to fail. I don't think the escape messages actually mattered to that test, they just had to be matched. To fix this, we drop the debug level to -m=1, since -m=2 prints a lot of extra information for escaping parameters that we don't want to deal with, and change one error check to allow b to escape. Fixes #72974. Change-Id: I7d4abbb1ec1e096685514536f91ba0d581cca6b7 Reviewed-on: https://go-review.googlesource.com/c/go/+/659657 Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/660558 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2025-03-26[release-branch.go1.24] testing: detect early return from B.LoopAustin Clements
Currently, if a benchmark function returns prior to B.Loop() returning false, we'll report a bogus result. While there was no way to detect this with b.N-style benchmarks, one way b.Loop()-style benchmarks are more robust is that we *can* detect it. This CL adds a flag to B that tracks if B.Loop() has finished and checks it after the benchmark completes. If there was an early exit (not caused by another error), it reports a B.Error. For #72974. Change-Id: I731c1350e6df938c0ffa08fcedc11dc147e78854 Reviewed-on: https://go-review.googlesource.com/c/go/+/659656 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/660557 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2025-03-26[release-branch.go1.24] testing: separate b.Loop counter from b.NAustin Clements
Currently, b.Loop uses b.N as the iteration count target. However, since it updates the target as it goes, the behavior is quite different from a b.N-style benchmark. To avoid user confusion, this CL gives b.Loop a separate, unexported iteration count target. It ensures b.N is 0 within the b.Loop loop to help catch misuses, and commits the final iteration count to b.N only once the loop is done (as the documentation states "After Loop returns false, b.N contains the total number of iterations that ran, so the benchmark may use b.N to compute other average metrics.") Since there are now two variables used by b.Loop, we put them in an unnamed struct. Also, we rename b.loopN to b.loop.i because this variable tracks the current iteration index (conventionally "i"), not the target (conventionally "n"). Unfortunately, a simple renaming causes B.Loop to be too large for the inliner. Thus, we make one simplification to B.Loop to keep it under the threshold. We're about to lean into that simplification anyway in a follow-up CL, so this is just temporary. For #72974. Change-Id: Ide1c4f1b9ca37f300f3beb0e60ba6202331b183e Reviewed-on: https://go-review.googlesource.com/c/go/+/659655 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> Auto-Submit: Austin Clements <austin@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/660556 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2025-03-25[release-branch.go1.24] testing: allow manual timer control in testing.B.LoopJunyang Shao
Fixes #72934 Change-Id: I56610d2d11d151a8f95b6434bbedbfcd5c11c317 Reviewed-on: https://go-review.googlesource.com/c/go/+/658975 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Commit-Queue: Junyang Shao <shaojunyang@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/660555 Reviewed-by: Junyang Shao <shaojunyang@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-12-15testing: don't measure cleanup time after B.LoopAustin Clements
B.Loop resets the timer on the first iteration so that setup code isn't measured, but it currently leaves the timer running after the last iteration, meaning that cleanup code will still be measured. Fix this by stopping the timer when B.Loop returns false to indicate the end of the benchmark. Updates #61515 Change-Id: I0e0502cb2ce3c24cf872682b88d74e8be2c4529b Reviewed-on: https://go-review.googlesource.com/c/go/+/635898 Reviewed-by: Junyang Shao <shaojunyang@google.com> Auto-Submit: Austin Clements <austin@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-12-15testing: improve B.Loop testAustin Clements
This moves the B.Loop test from package testing_test to package testing, where it can check on more of the internals of the benchmark state. Updates #61515. Change-Id: Ia32d7104526125c5e8a1e35dab7660008afcbf80 Reviewed-on: https://go-review.googlesource.com/c/go/+/635897 Auto-Submit: Austin Clements <austin@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>