From 1c72ee7f13831b215b8744f6b35bc4fd53aba5e2 Mon Sep 17 00:00:00 2001 From: Russ Cox Date: Thu, 13 Oct 2022 16:13:46 -0400 Subject: testing: fix many test2json inaccuracies Test2json is parsing the output stream from the test, which includes package testing's own framing lines intermingled with other output, in particular any output printed via fmt.Printf, println, and so on. We have had recurring problems with unexpected partial output lines causing a framing line to be missed. A recent talk at GopherCon gave an example of an integration test involving Docker that happened to print \r-terminated lines instead of \n-terminated lines in some configurations, which in turn broke test2json badly. (https://www.gophercon.com/agenda/session/944259) There are also a variety of open reported issues with similar problems, which this CL also addresses. The general approach is to add a new testing flag -test.v=json that means to print additional output to help test2json. And then test2json takes advantage of that output. Among the fixes: - Identify testing framing more reliably, using ^V (#23036, #26325, #43683, GopherCon talk) - Test that output with \r\n endings is handled correctly (#43683, #34286) - Use === RUN in fuzz tests (#52636, #48132) - Add === RUN lines to note benchmark starts (#27764, #49505) - Print subtest --- PASS/FAIL lines as they happen (#29811) - Add === NAME lines to emit more test change events, such as when a subtest stops and the parent continues running. - Fix event shown in overall test failure (#27568) - Avoid interleaving of writes to os.Stdout and os.Stderr (#33419) Fixes #23036. Fixes #26325. Fixes #27568. Fixes #27764. Fixes #29811. Fixes #33419. Fixes #34286. Fixes #43683. Fixes #49505. Fixes #52636. Change-Id: Id4207b746a20693f92e52d68c6e4a7f8c41cc7c6 Reviewed-on: https://go-review.googlesource.com/c/go/+/443596 Auto-Submit: Russ Cox Run-TryBot: Russ Cox Reviewed-by: Bryan Mills TryBot-Result: Gopher Robot --- src/cmd/internal/test2json/testdata/timeout.json | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 src/cmd/internal/test2json/testdata/timeout.json (limited to 'src/cmd/internal/test2json/testdata/timeout.json') diff --git a/src/cmd/internal/test2json/testdata/timeout.json b/src/cmd/internal/test2json/testdata/timeout.json new file mode 100644 index 0000000000..162a5bde44 --- /dev/null +++ b/src/cmd/internal/test2json/testdata/timeout.json @@ -0,0 +1,7 @@ +{"Action":"run","Test":"Test"} +{"Action":"output","Test":"Test","Output":"=== RUN Test\n"} +{"Action":"output","Test":"Test","Output":"panic: test timed out after 1s\n"} +{"Action":"output","Test":"Test","Output":"\n"} +{"Action":"output","Output":"FAIL\tp\t1.111s\n"} +{"Action":"output","Output":"FAIL\n"} +{"Action":"fail"} -- cgit v1.3