[dev.fuzz] internal/fuzz: use scratch []byte for mutations

The mutator will now use a scratch []byte when mutating
[]byte and string types.

I ran the following target locally:

    func FuzzBytesFromStringCmp(f *testing.F) {
        f.Fuzz(func(t *testing.T, a, b string) {
            bytes.Compare([]byte(a), []byte(b))
        })
    }

Before the change, execs were <400/sec:

    === FUZZ  FuzzBytesFromStringCmp
    fuzzing, elapsed: 3.0s, execs: 1090 (363/sec), workers: 8 ...
    fuzzing, elapsed: 6.0s, execs: 2290 (382/sec), workers: 8 ...
    fuzzing, elapsed: 9.0s, execs: 3491 (388/sec), workers: 8 ...
    fuzzing, elapsed: 12.0s, execs: 4691 (391/sec), workers: 8 ...
    fuzzing, elapsed: 15.0s, execs: 5869 (391/sec), workers: 8 ...
    fuzzing, elapsed: 18.0s, execs: 7056 (392/sec), workers: 8 ...

After the change, the execs are ~6000/sec

    === FUZZ  FuzzBytesFromStringCmp
    fuzzing, elapsed: 3.0s, execs: 155129 (51687/sec), workers: 8 ...
    fuzzing, elapsed: 6.0s, execs: 303710 (50606/sec), workers: 8 ...
    fuzzing, elapsed: 9.0s, execs: 454314 (50470/sec), workers: 8 ...
    fuzzing, elapsed: 12.0s, execs: 603212 (50262/sec), workers: 8 ...
    fuzzing, elapsed: 15.0s, execs: 756165 (50401/sec), workers: 8 ...
    fuzzing, elapsed: 18.0s, execs: 899293 (49955/sec), workers: 8 ...

Which is comparable to the same target with two []byte as input:

    === FUZZ  FuzzBytesCmp
    fuzzing, elapsed: 3.0s, execs: 152348 (50757/sec), workers: 8 ...
    fuzzing, elapsed: 6.0s, execs: 314386 (52387/sec), workers: 8 ...
    fuzzing, elapsed: 9.0s, execs: 487413 (54148/sec), workers: 8 ...
    fuzzing, elapsed: 12.0s, execs: 646886 (53901/sec), workers: 8 ...
    fuzzing, elapsed: 15.0s, execs: 814257 (54266/sec), workers: 8 ...
    fuzzing, elapsed: 18.0s, execs: 983214 (54619/sec), workers: 8 ...

Benchark results:
name                      old time/op    new time/op    delta
MutatorBytes/1-8          7.70ms ± 3%    0.00ms ± 3%   -99.99%  (p=0.029 n=4+4)
MutatorBytes/10-8         7.88ms ± 2%    0.00ms ± 6%   -99.99%  (p=0.029 n=4+4)
MutatorBytes/100-8        7.87ms ± 1%    0.00ms ± 2%   -99.99%  (p=0.029 n=4+4)
MutatorBytes/1000-8       8.11ms ± 5%    0.00ms ± 2%   -99.99%  (p=0.029 n=4+4)
MutatorBytes/10000-8      8.11ms ± 4%    0.00ms ± 2%   -99.99%  (p=0.029 n=4+4)
MutatorBytes/100000-8     8.28ms ±10%    0.00ms ± 4%   -99.96%  (p=0.029 n=4+4)
MutatorString/1-8         7.89ms ± 5%    0.00ms ±17%   -99.99%  (p=0.029 n=4+4)
MutatorString/10-8        7.91ms ± 4%    0.00ms ± 7%   -99.99%  (p=0.029 n=4+4)
MutatorString/100-8       8.08ms ± 4%    0.00ms ± 7%   -99.99%  (p=0.029 n=4+4)
MutatorString/1000-8      8.11ms ± 6%    0.00ms ±11%   -99.99%  (p=0.029 n=4+4)
MutatorString/10000-8     8.04ms ± 7%    0.00ms ± 8%   -99.98%  (p=0.029 n=4+4)
MutatorString/100000-8    8.24ms ± 7%    0.01ms ±13%   -99.82%  (p=0.029 n=4+4)

Fixes #46543

Change-Id: I8b078ed3adc1bb6310c33afc49bb6cd78e7e976c
Reviewed-on: https://go-review.googlesource.com/c/go/+/324849
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

1 files changed, 11 insertions, 2 deletions

diff --git a/src/testing/fuzz.go b/src/testing/fuzz.go
index 55e5397193..d62eb55dec 100644
--- a/src/testing/fuzz.go
+++ b/src/testing/fuzz.go
@@ -258,9 +258,18 @@ var supportedTypes = map[reflect.Type]bool{
 // Fuzz runs the fuzz function, ff, for fuzz testing. If ff fails for a set of
 // arguments, those arguments will be added to the seed corpus.
 //
+// ff must be a function with no return value whose first argument is *T and
+// whose remaining arguments are the types to be fuzzed.
+// For example:
+//
+// f.Fuzz(func(t *testing.T, b []byte, i int) { ... })
+//
+// This function should be fast, deterministic, and stateless.
+// None of the pointers to any input data should be retained between executions.
+//
 // This is a terminal function which will terminate the currently running fuzz
-// target by calling runtime.Goexit. To run any code after this function, use
-// Cleanup.
+// target by calling runtime.Goexit.
+// To run any code after fuzzing stops, use (*F).Cleanup.
 func (f *F) Fuzz(ff interface{}) {
 	if f.fuzzCalled {
 		panic("testing: F.Fuzz called more than once")