aboutsummaryrefslogtreecommitdiff
path: root/src/testing/benchmark.go
AgeCommit message (Collapse)Author
2020-10-20testing: print cpu type as label for benchmarksMartin Möhrmann
Supports 386 and amd64 architectures on all operating systems. Example output: $ go test -bench=.* goos: darwin goarch: amd64 pkg: strconv cpu: Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz BenchmarkAtof64Decimal-4 24431032 46.8 ns/op ... As the displayed CPU information is only used for information purposes it is lazily initialized when needed using the new internal/sysinfo package. This allows internal/cpu to stay without dependencies and avoid initialization costs when the CPU information is not needed as the new code to query the CPU name in internal/cpu can be dead code eliminated if not used. Fixes #39214 Change-Id: I77ae5c5d2fed6b28fa78dd45075f9f0a6a7f1bfd Reviewed-on: https://go-review.googlesource.com/c/go/+/263804 Trust: Martin Möhrmann <moehrmann@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2020-09-02testing: flush test summaries to stdout atomically when streaming outputBryan C. Mills
While debugging #40771, I realized that the chatty printer should only ever print to a single io.Writer (normally os.Stdout). The other Writer implementations in the chain write to local buffers, but if we wrote a test's output to a local buffer, then we did *not* write it to stdout and we should not store it as the most recently logged test. Because the chatty printer should only ever print to one place, it shouldn't receive an io.Writer as an argument — rather, it shouldn't be used at all for destinations other than the main output stream. On the other hand, when we flush the output buffer to stdout in the top-level flushToParent call, it is important that we not allow some other test's output to intrude between the test summary header and the remainder of the test's output. cmd/test2json doesn't know how to parse such an intrusion, and it's confusing to humans too. No test because I couldn't reproduce the user-reported error without modifying the testing package. (This behavior seems to be very sensitive to output size and/or goroutine scheduling.) Fixes #40771 Updates #38458 Change-Id: Ic19bf1d535672b096ba1c8583a3b74aab6d6d766 Reviewed-on: https://go-review.googlesource.com/c/go/+/249026 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-05-21testing: reformat test chatty outputJean de Klerk
In #24929, we decided to stream chatty test output. It looks like, foo_test.go:138: TestFoo/sub-1: hello from subtest 1 foo_test.go:138: TestFoo/sub-2: hello from subtest 2 In this CL, we refactor the output to be grouped by === CONT lines, preserving the old test-file-before-log-line behavior: === CONT TestFoo/sub-1 foo_test.go:138 hello from subtest 1 === CONT TestFoo/sub-2 foo_test.go:138 hello from subtest 2 This should remove a layer of verbosity from tests, and make it easier to group together related lines. It also returns to a more familiar format (the pre-streaming format), whilst still preserving the streaming feature. Fixes #38458 Change-Id: Iaef94c580d69cdd541b2ef055aa004f50d72d078 Reviewed-on: https://go-review.googlesource.com/c/go/+/229085 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Andrew Bonventre <andybons@golang.org>
2020-02-06testing: make Cleanup work for benchmarks too.Roger Peppe
Fixes #37073. Change-Id: I6fb24a3f9d7b7adf3213ac6a8bcbf5fb43975b7e Reviewed-on: https://go-review.googlesource.com/c/go/+/218117 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-23testing: document that benchmark logs are dumped to stdoutAlberto Donizetti
The doc for testing.T and testing.B said that both test and benchmark logs are printed to stderr, but in reality that wasn't the case. CL 24311 fixed the doc for T, this change fixes it for B. Fixes #36257 Change-Id: I0ff77ff44608f60320a1565b371c81e96039e71c Reviewed-on: https://go-review.googlesource.com/c/go/+/212457 Reviewed-by: Gabriel Aszalos <gabriel.aszalos@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Gabriel Aszalos <gabriel.aszalos@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-10testing: use conventional comments for exported internal funcs & structsvovapi
Change-Id: I2dd5ddc22bfff143b81d5945992d8c5fccf387f4 GitHub-Last-Rev: aa637756e772f5ee9094b802df3be9945c8466c4 GitHub-Pull-Request: golang/go#36054 Reviewed-on: https://go-review.googlesource.com/c/go/+/210497 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-22testing: stream log output in verbose modeJean de Klerk
Fixes #24929 Change-Id: Icc426068cd73b75b78001f55e1e5d81ccebbe854 Reviewed-on: https://go-review.googlesource.com/c/go/+/127120 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-04-29testing: delay flag registration; move to an Init functionCaleb Spare
Any code that imports the testing package forces the testing flags to be defined, even in non-test binaries. People work around this today by defining a copy of the testing.TB interface just to avoid importing testing. Fix this by moving flag registration into a new function, testing.Init. Delay calling Init until the testing binary begins to run, in testing.MainStart. Init is exported for cases where users need the testing flags to be defined outside of a "go test" context. In particular, this may be needed where testing.Benchmark is called outside of a test. Fixes #21051 Change-Id: Ib7e02459e693c26ae1ba71bbae7d455a91118ee3 Reviewed-on: https://go-review.googlesource.com/c/go/+/173722 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-25testing: correct two spelling/grammar issuesCaleb Spare
Change-Id: Ic7f7a34e12cc2845e0385a5a872e694d5dca7372 Reviewed-on: https://go-review.googlesource.com/c/go/+/169158 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-22testing: fix fractional ns/op printingAustin Clements
CL 166717 changed the way ns/op benchmark results were printed and inadvertently rounded all ns/op results down to an integer, even if they were small enough to print with digits after the decimal place. For example, prior to this change, we got output like BenchmarkFast-8 380491575 3.12 ns/op CL 166717 changed this to BenchmarkFast-8 380491575 3.00 ns/op This had the further side-effect that ns/op values between 0 and 1 would not be printed at all because they would be rounded down to 0. This CL fixes this by always recomputing the float64 value of ns/op instead of using the int64 truncation from BenchmarkResult.NsPerOp. Fixes #30997. Fixes #31005. Change-Id: I21f73b9d5cc5ad41e7ff535675d07ca00051ecd7 Reviewed-on: https://go-review.googlesource.com/c/go/+/168937 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2019-03-20testing: stop rounding b.NJosh Bleecher Snyder
The original goal of rounding to readable b.N was to make it easier to eyeball times. However, proper analysis requires tooling (such as benchstat) anyway. Instead, take b.N as it comes. This will reduce the impact of external noise such as GC on benchmarks. This requires reworking our iteration estimates. We used to calculate the estimated ns/op and then divide our target ns by that estimate. However, this order of operations was destructive when the ns/op was very small; rounding could hide almost an order of magnitude of variation. Instead, multiply first, then divide. Also, make n an int64 to avoid overflow. Prior to this change, we attempted to cap b.N at 1e9. Due to rounding up, it was possible to get b.N as high as 2e9. This change consistently enforces the 1e9 cap. This change also reduces the wall time required to run benchmarks. Here's the impact of this change on the wall time to run all benchmarks once with benchtime=1s on some std packages: name old time/op new time/op delta bytes 306s ± 1% 238s ± 1% -22.24% (p=0.000 n=10+10) encoding/json 112s ± 8% 99s ± 7% -11.64% (p=0.000 n=10+10) net/http 54.7s ± 7% 44.9s ± 4% -17.94% (p=0.000 n=10+9) runtime 957s ± 1% 714s ± 0% -25.38% (p=0.000 n=10+9) strings 262s ± 1% 201s ± 1% -23.27% (p=0.000 n=10+10) [Geo mean] 216s 172s -20.23% Updates #24735 Change-Id: I7e38efb8e23c804046bf4fc065b3f5f3991d0a15 Reviewed-on: https://go-review.googlesource.com/c/go/+/112155 Reviewed-by: Austin Clements <austin@google.com>
2019-03-19testing: add B.ReportMetric for custom benchmark metricsAustin Clements
This adds a ReportMetric method to testing.B that lets the user report custom benchmark metrics and override built-in metrics. Fixes #26037. Change-Id: I8236fbde3683fc27bbe45cbbedfd377b435edf64 Reviewed-on: https://go-review.googlesource.com/c/go/+/166717 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2019-03-04testing: fix missing verb in StartTimer docEmmanuel T Odeke
Fixes StartTimer's doc with the verb 'be' that was previously missing in 'can also used'. Change-Id: I4b3e6103fbf62d676056d32fcce4618536b7c05c Reviewed-on: https://go-review.googlesource.com/c/go/+/165117 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-02all: use "reports whether" consistently instead of "returns whether"Tobias Klauser
Follow-up for CL 147037 and after Brad noticed the "returns whether" pattern during the review of CL 150621. Go documentation style for boolean funcs is to say: // Foo reports whether ... func Foo() bool (rather than "returns whether") Created with: $ perl -i -npe 's/returns whether/reports whether/' $(git grep -l "returns whether" | grep -v vendor) Change-Id: I15fe9ff99180ad97750cd05a10eceafdb12dc0b4 Reviewed-on: https://go-review.googlesource.com/c/150918 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12testing: implement -benchtime=100xRuss Cox
When running benchmarks with profilers and trying to compare one run against another, it is very useful to be able to force each run to execute exactly the same number of iterations. Discussion on the proposal issue #24735 led to the decision to overload -benchtime, so that instead of saying -benchtime 10s to run a benchmark for 10 seconds, you say -benchtime 100x to run a benchmark 100 times. Fixes #24735. Change-Id: Id17c5bd18bd09987bb48ed12420d61ae9e200fd7 Reviewed-on: https://go-review.googlesource.com/c/139258 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-31Revert "testing: only compute b.N once when passed -count > 1"Alberto Donizetti
This reverts golang.org/cl/110775 Reason for revert: this is causing huge slow-dows on every run after the 1st, on various benchmarks, on multiple architectures (see Issue 25622 for details). It's just a nice-to-have little optimization, and we're near the 1st go1.11 beta release, so I'm reverting it. Fixes #25622 Change-Id: I758ade4af4abf764abd8336d404396992d11a0c6 Reviewed-on: https://go-review.googlesource.com/115535 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2018-05-14testing: allow marking subtest and subbenchmark functions as HelpersDiogo Pinela
Since subtests and subbenchmarks run in a separate goroutine, and thus a separate stack, this entails capturing the stack trace at the point tb.Run is called. The work of getting the file and line information from this stack is only done when needed, however. Continuing the search into the parent test also requires temporarily holding its mutex. Since Run does not hold it while waiting for the subtest to complete, there should be no risk of a deadlock due to this. Fixes #24128 Change-Id: If0bb169f3ac96bd48794624e619ade7edb599f83 Reviewed-on: https://go-review.googlesource.com/108658 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2018-05-14testing: only compute b.N once when passed -count > 1Alberto Donizetti
When running a benchmark multiple times, instead of re-computing the value of b.N each time, use the value found by the first run. For go test -bench=. -benchtime 3s -count 2 p_test.go on the benchmark in the linked issue; before: BenchmarkBenchmark-4 500 10180593 ns/op --- BENCH: BenchmarkBenchmark-4 p_test.go:13: single call took 10.111079ms p_test.go:13: single call took 1.017298685s p_test.go:13: single call took 5.090096124s BenchmarkBenchmark-4 500 10182164 ns/op --- BENCH: BenchmarkBenchmark-4 p_test.go:13: single call took 10.098169ms p_test.go:13: single call took 1.017712905s p_test.go:13: single call took 5.090898517s PASS ok command-line-arguments 12.244s and after: BenchmarkBenchmark-4 500 10177076 ns/op --- BENCH: BenchmarkBenchmark-4 p_test.go:13: single call took 10.091301ms p_test.go:13: single call took 1.016943125s p_test.go:13: single call took 5.088376028s BenchmarkBenchmark-4 500 10171497 ns/op --- BENCH: BenchmarkBenchmark-4 p_test.go:13: single call took 10.140245ms p_test.go:13: single call took 5.085605921s PASS ok command-line-arguments 11.218s Fixes #23423 Change-Id: Ie66a8c5ac43881eb8741e14105db28745b4d56d3 Reviewed-on: https://go-review.googlesource.com/110775 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-01testing: remove claim that b.Run is safe for concurrent useRuss Cox
It's not safe (it crashes), and it's also useless: if you run multiple benchmarks in parallel you will not get reliable timing results from any of them. Fixes #18603. Change-Id: I00e5a72f7c98151543cf7d5573c38383276e391a Reviewed-on: https://go-review.googlesource.com/80841 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-09-06testing: use time.Since instead of time.Now().SubDaniel Martí
Change-Id: Ia0479f65a6ef033a65acf9ac8692777839a53b8c Reviewed-on: https://go-review.googlesource.com/61990 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-28all: remove some unused result paramsDaniel Martí
Most of these are return values that were part of a receiving parameter, so they're still accessible. A few others are not, but those have never had a use. Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that the tool should also warn about unused result parameters. Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b Reviewed-on: https://go-review.googlesource.com/55910 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-08-24testing: parallelize tests over countDmitry Vyukov
Currently all package tests are executed once with Parallel tests executed in parallel. Then this process is repeated count*cpu times. Tests are not parallelized over count*cpu. Parallelizing over cpu is not possible as GOMAXPROCS is a global setting. But it is possible for count. Parallelize over count. Brings down testing of my package with -count=100 form 10s to 0.3s. Change-Id: I76d8322adeb8c5c6e70b99af690291fd69d6402a Reviewed-on: https://go-review.googlesource.com/44830 Run-TryBot: Dmitry Vyukov <dvyukov@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-06-30testing: revert CL 36791's conditional ReadMemStatsBrad Fitzpatrick
Now that ReadMemStats is fast (CL 34937), CL 36791 is not so necessary, and causes confusion. See #20863 This was already partially reverted in CL 46612 but missed two of the spots. Fixes #20863 Change-Id: I1307a0f7b1f9e86e8b6ceaa6a677f24f13431110 Reviewed-on: https://go-review.googlesource.com/47350 Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-06-29testing: clarify rules for concurrent t.Run callsAlberto Donizetti
Apparently, "all such calls must happen" means that the t.Run call must *return* before the outer test function returns, or the calls will cause a data race on t.ran. Clarify the docs. Fixes #20339 Change-Id: I191a9af2a9095be1e0aaf10b79c30e00a9c495cb Reviewed-on: https://go-review.googlesource.com/47150 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-06-28testing: always ReadMemStats before first benchmark runMeir Fischer
If the only way the user indicates they want alloc stats shown is via ReportAllocs, we don't know that until benchFunc is run. Therefore, StopTimer's ReadMemStats will return incorrect data for single cycle runs since there's no counterpart ReadMemStats from StartTimer that initializes alloc stats. It appears that this bug was introduced by CL 46612, "testing: only call ReadMemStats if necessary when benchmarking" Fixes #20590 Change-Id: I3b5ef91677823f4b98011880a3be15423baf7e33 Reviewed-on: https://go-review.googlesource.com/46612 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-06-20testing: harmonize handling of prefix-matched benchmarksRuss Cox
If you have BenchmarkX1 with sub-benchmark Y and you have BenchmarkX2 with no sub-benchmarks, then go test -bench=X/Y runs BenchmarkX1 once with b.N=1 (to find out about Y) and then not again, because it has sub-benchmarks, but arguably also because we're interested in Y. In contrast, it runs BenchmarkX2 in full, even though clearly that is not relevant to the match X/Y. We do have to run X2 once with b.N=1 to probe for having X2/Y, but we should not run it with larger b.N. Fixes #20589. Change-Id: Ib86907e844f34dcaac6cd05757f57db1019201d0 Reviewed-on: https://go-review.googlesource.com/46031 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2017-03-01testing: fix Benchmark() to start at 1 iteration, not 100Heschi Kreinick
The run1 call removed in golang.org/cl/36990 was necessary to initialize the duration of the benchmark. With it gone, the math in launch() starts from 100. This doesn't work out well for second-long benchmark methods. Put it back. Updates #18815 Change-Id: I461f3466c805d0c61124a2974662f7ad45335794 Reviewed-on: https://go-review.googlesource.com/37530 Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2017-02-15testing: fix stats bug for sub benchmarksMarcel van Lohuizen
Fixes golang/go#18815. Change-Id: Ic9d5cb640a555c58baedd597ed4ca5dd9f275c97 Reviewed-on: https://go-review.googlesource.com/36990 Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-02-11testing: fix copy/paste in docsJosh Bleecher Snyder
Follow-up to CL 36791. Change-Id: I1c4831e5dfe90c205782e970ada7faff8a009daa Reviewed-on: https://go-review.googlesource.com/36890 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-11testing: only call ReadMemStats if necessary when benchmarkingJosh Bleecher Snyder
When running benchmarks with -cpuprofile, the entire process gets profiled, and ReadMemStats is surprisingly expensive. Running the sort benchmarks right now with -cpuprofile shows almost half of all execution time in ReadMemStats. Since ReadMemStats is not required if the benchmark does not need allocation stats, simply skip it. This will make cpu profiles nicer to read and significantly speed up the process of running benchmarks. It might also make sense to toggle cpu profiling on/off as we begin/end individual benchmarks, but that wouldn't get us the time savings of skipping ReadMemStats, so this CL is useful in itself. Change-Id: I425197b1ee11be4bc91d22b929e2caf648ebd7c5 Reviewed-on: https://go-review.googlesource.com/36791 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-02-07testing: print extra labels on benchmarksQuentin Smith
When running benchmarks, print "goos", "goarch", and "pkg" labels. This makes it easier to refer to benchmark logs and understand how they were generated. "pkg" is printed only for benchmarks located in GOPATH. Change-Id: I397cbdd57b9fe8cbabbb354ec7bfba59f5625c42 Reviewed-on: https://go-review.googlesource.com/36356 Run-TryBot: Quentin Smith <quentin@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2017-01-18testing: make parallel t.Run safe againRuss Cox
Fixes #18603. Change-Id: I5760c0a9f862200b7e943058a672eb559ac1b9d9 Reviewed-on: https://go-review.googlesource.com/35354 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-09Revert "testing: add T.Context method"Joe Tsai
This reverts commit 26827bc2fe4c80dc68b3793631d24975425c9467. Fixes #18199 Change-Id: I42e292cb4e3d740a4fbb5d0380c6ee15ac742092 Reviewed-on: https://go-review.googlesource.com/34141 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-03testing: add T.Context methodBrad Fitzpatrick
From the doc comment: Context returns the context for the current test or benchmark. The context is cancelled when the test or benchmark finishes. A goroutine started during a test or benchmark can wait for the context's Done channel to become readable as a signal that the test or benchmark is over, so that the goroutine can exit. Fixes #16221. Fixes #17552. Change-Id: I657df946be2c90048cc74615436c77c7d9d1226c Reviewed-on: https://go-review.googlesource.com/31724 Reviewed-by: Rob Pike <r@golang.org>
2016-11-03testing: mark tests and benchmarks failed if a race occurs during executionRuss Cox
Before: $ go test -race -v -run TestRace === RUN TestRace ================== WARNING: DATA RACE Write at 0x00c420076420 by goroutine 7: _/Users/rsc/go/src/cmd/go/testdata/src/testrace.TestRace.func1() /Users/rsc/go/src/cmd/go/testdata/src/testrace/race_test.go:10 +0x3b Previous write at 0x00c420076420 by goroutine 6: _/Users/rsc/go/src/cmd/go/testdata/src/testrace.TestRace() /Users/rsc/go/src/cmd/go/testdata/src/testrace/race_test.go:13 +0xcc testing.tRunner() /Users/rsc/go/src/testing/testing.go:656 +0x104 Goroutine 7 (running) created at: _/Users/rsc/go/src/cmd/go/testdata/src/testrace.TestRace() /Users/rsc/go/src/cmd/go/testdata/src/testrace/race_test.go:12 +0xbb testing.tRunner() /Users/rsc/go/src/testing/testing.go:656 +0x104 Goroutine 6 (running) created at: testing.(*T).Run() /Users/rsc/go/src/testing/testing.go:693 +0x536 testing.runTests.func1() /Users/rsc/go/src/testing/testing.go:877 +0xaa testing.tRunner() /Users/rsc/go/src/testing/testing.go:656 +0x104 testing.runTests() /Users/rsc/go/src/testing/testing.go:883 +0x4ac testing.(*M).Run() /Users/rsc/go/src/testing/testing.go:818 +0x1c3 main.main() _/Users/rsc/go/src/cmd/go/testdata/src/testrace/_test/_testmain.go:42 +0x20f ================== --- PASS: TestRace (0.00s) PASS Found 1 data race(s) FAIL _/Users/rsc/go/src/cmd/go/testdata/src/testrace 1.026s $ After: $ go test -race -v -run TestRace === RUN TestRace ================== WARNING: DATA RACE Write at 0x00c420076420 by goroutine 7: _/Users/rsc/go/src/cmd/go/testdata/src/testrace.TestRace.func1() /Users/rsc/go/src/cmd/go/testdata/src/testrace/race_test.go:10 +0x3b Previous write at 0x00c420076420 by goroutine 6: _/Users/rsc/go/src/cmd/go/testdata/src/testrace.TestRace() /Users/rsc/go/src/cmd/go/testdata/src/testrace/race_test.go:13 +0xcc testing.tRunner() /Users/rsc/go/src/testing/testing.go:656 +0x104 Goroutine 7 (running) created at: _/Users/rsc/go/src/cmd/go/testdata/src/testrace.TestRace() /Users/rsc/go/src/cmd/go/testdata/src/testrace/race_test.go:12 +0xbb testing.tRunner() /Users/rsc/go/src/testing/testing.go:656 +0x104 Goroutine 6 (running) created at: testing.(*T).Run() /Users/rsc/go/src/testing/testing.go:693 +0x536 testing.runTests.func1() /Users/rsc/go/src/testing/testing.go:877 +0xaa testing.tRunner() /Users/rsc/go/src/testing/testing.go:656 +0x104 testing.runTests() /Users/rsc/go/src/testing/testing.go:883 +0x4ac testing.(*M).Run() /Users/rsc/go/src/testing/testing.go:818 +0x1c3 main.main() _/Users/rsc/go/src/cmd/go/testdata/src/testrace/_test/_testmain.go:42 +0x20f ================== --- FAIL: TestRace (0.00s) testing.go:609: race detected during execution of test FAIL FAIL _/Users/rsc/go/src/cmd/go/testdata/src/testrace 0.022s $ Fixes #15972. Change-Id: Idb15b8ab81d65637bb535c7e275595ca4a6e450e Reviewed-on: https://go-review.googlesource.com/32615 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-10-19cmd/go, testing: indicate when no tests are runCaio Marcelo de Oliveira Filho
For example, testing the current directory: $ go test -run XXX testing: warning: no tests to run PASS ok testing 0.013s $ And in a summary: $ go test -run XXX testing ok testing 0.013s [no tests to run] $ These make it easy to spot when the -run regexp hasn't matched anything or there are no tests. Previously the message was printed in the "current directory" case when there were no tests at all, but not for no matches, and either way was not surfaced in the directory list summary form. Fixes #15211. Change-Id: I1c82a423d6bd429fb991c9ca964c9d26c96fd3c5 Reviewed-on: https://go-review.googlesource.com/22341 Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2016-10-18testing: fix flag usage messagesRuss Cox
Fixes #16404. Change-Id: Iabaeeef3eff2fff6e5ed2d6bc9ef9c2f6d1cb5e7 Reviewed-on: https://go-review.googlesource.com/31332 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-08-10testing: respect benchtime on very fast benchmarksJosh Bleecher Snyder
When ns/op dropped below 1, the old code ignored benchtime and reverted to 1s. Change-Id: I59752cef88d8d73bfd5b085f5400ae657f78504e Reviewed-on: https://go-review.googlesource.com/26664 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2016-04-05testing: improve outputMarcel van Lohuizen
This introduces a few changes - Skipped benchmarks now print a SKIP line, also if there was no output - The benchmark name is only printed if there the benchmark was not skipped or did not fail in the probe phase. It also fixes a bug of doubling a skip message in chatty mode in absense of a failure. The chatty flag is now passed in the common struct to allow for testing of the printed messages. Fixes #14799 Change-Id: Ia8eb140c2e5bb467e66b8ef20a2f98f5d95415d5 Reviewed-on: https://go-review.googlesource.com/21504 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-26cmd/go: fix proc-count accumulation in benchmark nameAlberto Donizetti
Fixes #14964 Change-Id: I5f772426081efaa9315c4ecaf60de850af324f1d Reviewed-on: https://go-review.googlesource.com/21139 Reviewed-by: Ahmed Waheed <oneofone@gmail.com> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org> Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-25testing: probe with N=1Marcel van Lohuizen
Change control flow to probe with N=1. This calls benchFunc the same number of times as the old implementation in the absence of subbenchmarks. To be compatible with existing tools, benchmarking only prints a line for "leaf" benchmarks. This means, though, that the name of a benchmark can only be printed after the first iteration. Issue #14863 Change-Id: Ic7b9b89b058f8ebb5287755f24f9e47df8c9537c Reviewed-on: https://go-review.googlesource.com/21043 Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2016-03-22testing: expose subtest and subbenchmark functionalityMarcel van Lohuizen
Fixes #12166 Change-Id: Ie62cba2c39beb5732447ba3688c93c08ef12abb5 Reviewed-on: https://go-review.googlesource.com/18898 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
2016-03-22testing: added name matcher and sanitizerMarcel van Lohuizen
The matcher is responsible for sanitizing and uniquing the test and benchmark names and thus needs to be included before the API can be exposed. Matching currently uses the regexp to only match the top-level tests/benchmarks. Support for subtest matching is for another CL. Change-Id: I7c8464068faef7ebc179b03a7fe3d01122cc4f0b Reviewed-on: https://go-review.googlesource.com/18897 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-18testing: always ignore RunParallel in probe phaseMarcel van Lohuizen
Change-Id: If45410a2d7e48d1c9e6800cd98f81dd89024832c Reviewed-on: https://go-review.googlesource.com/20852 Reviewed-by: Russ Cox <rsc@golang.org>
2016-03-18testing: implementation of subbenchmarksMarcel van Lohuizen
API is not exposed yet. Change-Id: I729360ef2be1d8ea683ca93cdb1763897cc8657c Reviewed-on: https://go-review.googlesource.com/18895 Reviewed-by: Russ Cox <rsc@golang.org>
2016-03-18testing: hoisted chunks of code to prepare for Run methodMarcel van Lohuizen
testing.go: - run method will evolve into the Run method. - added level field in common benchmark.go: - benchContext will be central to distinguish handling of benchmarks between normal Run methods and ones called from within Benchmark function. - expandCPU will evolve into the processing hook for Run methods called within normal processing. - runBench will evolve into the Run method. Change-Id: I1816f9985d5ba94deb0ad062302ea9aee0bb5338 Reviewed-on: https://go-review.googlesource.com/18894 Reviewed-by: Russ Cox <rsc@golang.org>
2016-03-18testing: prepare for the introduction of Run methodsMarcel van Lohuizen
The biggest change is that each test is now responsible for managing the starting and stopping of its parallel subtests. The "Main" test could be run as a tRunner as well. This shows that the introduction of subtests is merely a generalization of and consistent with the current semantics. Change-Id: Ibf8388c08f85d4b2c0df69c069326762ed36a72e Reviewed-on: https://go-review.googlesource.com/18893 Reviewed-by: Russ Cox <rsc@golang.org>
2016-03-02all: single space after period.Brad Fitzpatrick
The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-28testing: make failure in benchmark cause non-zero exit statusCaio Marcelo de Oliveira Filho
Moves the implementation of RunBenchmarks to a non-exported function that returns whether the execution was OK, and uses that to identify failure in benchmarks.The exported function is kept for compatibility. Like before, benchmarks will only be executed if tests and examples pass. The PASS message will not be printed if there was a failure in a benchmark. Example output BenchmarkThatCallsFatal-8 --- FAIL: BenchmarkThatCallsFatal-8 x_test.go:6: called by benchmark FAIL exit status 1 FAIL _/.../src/cmd/go/testdata/src/benchfatal 0.009s Fixes #14307. Change-Id: I6f3ddadc7da8a250763168cc099ae8b325a79602 Reviewed-on: https://go-review.googlesource.com/19889 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-12-01testing: document that T and B are safe for concurrent callsRuss Cox
Fixes #13108. Change-Id: I474cc2a3b7ced1c9eb978fc815f9c6bae9fb3ecc Reviewed-on: https://go-review.googlesource.com/17235 Reviewed-by: Ian Lance Taylor <iant@golang.org>
generated by cgit v1.3-5-g9baa (git 2.53.0) at 2026-04-17 06:34:24 +0000