diff options
| author | Austin Clements <austin@google.com> | 2021-10-20 11:12:41 -0400 |
|---|---|---|
| committer | Austin Clements <austin@google.com> | 2024-10-29 18:07:59 +0000 |
| commit | 1dd567da00eec3ccfbcbafb7a04f4297e7a69b48 (patch) | |
| tree | 221fd608ab892db7c8a8f590efe484d8e754ccba /design/14313-benchmark-format.md | |
| parent | 0ecc9c77faf22aeb0a8385b539d5ba827e351e4b (diff) | |
| download | go-x-proposal-1dd567da00eec3ccfbcbafb7a04f4297e7a69b48.tar.xz | |
14313-benchmark-format: minor refreshes
This updates some links and some stale information in this design doc.
This is slightly odd because we use this design doc also as a
specification, which creates some tension between whether it should
continue to capture the state of the world at the time or be kept
fully up-to-date. I tried to take a light touch in this CL.
Change-Id: Ifa3cf7d70e0f6a65cd9184f76f449d5618211f63
Reviewed-on: https://go-review.googlesource.com/c/proposal/+/357529
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Diffstat (limited to 'design/14313-benchmark-format.md')
| -rw-r--r-- | design/14313-benchmark-format.md | 22 |
1 files changed, 12 insertions, 10 deletions
diff --git a/design/14313-benchmark-format.md b/design/14313-benchmark-format.md index 908ed8d..3fb3ef7 100644 --- a/design/14313-benchmark-format.md +++ b/design/14313-benchmark-format.md @@ -56,31 +56,33 @@ For example, running compress/flate's benchmarks produces this output: BenchmarkEncodeDigitsCompress1e5-8 1 12301511 ns/op 8.13 MB/s BenchmarkEncodeDigitsCompress1e6-8 1 137962041 ns/op 7.25 MB/s -The testing package always reports ns/op, and each benchmark can request the addition of MB/s (throughput) and also B/op and allocs/op (allocation rates). +The testing package always reports ns/op, and directly supports the +addition of MB/s (throughput) and also B/op and allocs/op (allocation +rates). +Benchmarks can report additional metrics with any custom unit using +[`B.ReportMetric`](https://pkg.go.dev/testing#B.ReportMetric). ### Benchmark processors Multiple tools have been written that process this format, -most notably [benchcmp](https://godoc.org/golang.org/x/tools/cmd/benchcmp) -and its more statistically valid successor [benchstat](https://godoc.org/rsc.io/benchstat). -There is also [benchmany](https://godoc.org/github.com/aclements/go-misc/benchmany)'s plot subcommand +most notably [benchcmp](https://pkg.go.dev/golang.org/x/tools/cmd/benchcmp) +and its more statistically valid successor [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat). +There is also [benchmany](https://pkg.go.dev/github.com/aclements/go-misc/benchmany)'s plot subcommand and likely more unpublished programs. ### Benchmark runners Multiple tools have also been written that generate this format. In addition to the standard Go testing package, -[compilebench](https://godoc.org/rsc.io/compilebench) +[compilebench](https://pkg.go.dev/golang.org/x/tools/cmd/compilebench) generates this data format based on runs of the Go compiler, and Austin's unpublished shellbench generates this data format after running an arbitrary shell command. -The [golang.org/x/benchmarks/bench](https://golang.org/x/benchmarks/bench) benchmarks -are notable for _not_ generating this format, -which has made all analysis of those results +The [golang.org/x/benchmarks](https://golang.org/x/benchmarks) benchmarks +are notable for _not_ originally generating this format, +which made all analysis of those results more complex than we believe it should be. -We intend to update those benchmarks to generate the standard format, -once a standard format is defined. Part of the motivation for the proposal is to avoid the need to process custom output formats in future benchmarks. |