go/src/unicode/utf8/utf8.go, branch fix-runtime-test-GOMAXPROCS

internal/testenv: exclude GOMAXPROCS when building test program

2025-07-20T06:09:31Z

In the environment where GOMAXPROCS set explicitly, for example to 3 in shell profile, the runtime tests will fail with the following error, ---- ok regexp/syntax 0.428s --- FAIL: TestCgroupGOMAXPROCS (0.81s) crash_test.go:186: running /home/ms/src/go/bin/go build -o /tmp/go-build1753772192/testprog.exe crash_test.go:208: built testprog in 796.664277ms --- FAIL: TestCgroupGOMAXPROCS/containermaxprocs=0 (0.00s) cgroup_linux_test.go:60: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (907.06µs): ok cgroup_linux_test.go:63: output got "3\n" want "4\n" --- FAIL: TestCgroupGOMAXPROCSNoLimit (0.00s) cgroup_linux_test.go:82: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (879.194µs): ok cgroup_linux_test.go:85: output got "3\n" want "4\n" --- FAIL: TestCgroupGOMAXPROCSHigherThanNumCPU (0.00s) cgroup_linux_test.go:102: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (852.396µs): ok cgroup_linux_test.go:105: output got "3\n" want "4\n" --- FAIL: TestCgroupGOMAXPROCSRound (0.01s) --- FAIL: TestCgroupGOMAXPROCSRound/50000 (0.00s) cgroup_linux_test.go:156: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (852.099µs): ok cgroup_linux_test.go:159: output got "3\n" want "2\n" --- FAIL: TestCgroupGOMAXPROCSRound/100000 (0.00s) cgroup_linux_test.go:156: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (894.001µs): ok cgroup_linux_test.go:159: output got "3\n" want "2\n" --- FAIL: TestCgroupGOMAXPROCSRound/150000 (0.00s) cgroup_linux_test.go:156: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (850.897µs): ok cgroup_linux_test.go:159: output got "3\n" want "2\n" --- FAIL: TestCgroupGOMAXPROCSSchedAffinity (0.00s) cgroup_linux_test.go:229: /tmp/go-build1753772192/testprog.exe PrintGOMAXPROCS (867.987µs): ok cgroup_linux_test.go:232: output got "3\n" want "2\n" FAIL FAIL runtime 23.088s ---- This changes exclude the GOMAXPROCS when building program for testing so it does not affect the tests. Change-Id: I590d9eca57026539413cf4c93b37f624f179d534

unicode/utf8: use builtin max function to simplify code

2025-02-25T18:16:36Z

Change-Id: I6a73b645d074baaa4d09480bdf4192816a8c2450 GitHub-Last-Rev: 202d498eb019c18b9ba30bccc2cb169c9eb79366 GitHub-Pull-Request: golang/go#71945 Reviewed-on: https://go-review.googlesource.com/c/go/+/652177 Auto-Submit: Keith Randall Auto-Submit: Ian Lance Taylor Reviewed-by: Keith Randall Reviewed-by: Keith Randall LUCI-TryBot-Result: Go LUCI Reviewed-by: Ian Lance Taylor

unicode/utf8: speedup RuneCount

2024-09-15T02:05:37Z

CL 612617 did speedup RuneCountInString, thus we can now use it to speedup RuneCount, too. name old time/op new time/op delta RuneCountTenASCIIChars-8 8.69ns ± 1% 3.59ns ± 2% -58.66% (p=0.000 n=9+9) RuneCountTenJapaneseChars-8 49.8ns ± 2% 40.9ns ± 0% -17.94% (p=0.000 n=10+8) Change-Id: I311750c00efc79af35fb0ca3b482a5d94e0a7977 Reviewed-on: https://go-review.googlesource.com/c/go/+/612955 Reviewed-by: Dmitri Shuralyov Auto-Submit: Cuong Manh Le Reviewed-by: Tim King LUCI-TryBot-Result: Go LUCI

unicode/utf8: use range loop in RuneCountInString

2024-09-12T19:10:46Z

CL 28490 speeded up non-ASCII rune decoding, and ASCII rune is also decoded faster now. Benchmark using: perflock -governor 70% go test -run=NONE -bench=BenchmarkRuneCountInString -count=10 Result: name old time/op new time/op delta RuneCountInStringTenASCIIChars-8 10.2ns ± 0% 7.1ns ± 1% -30.53% (p=0.000 n=8+9) RuneCountInStringTenJapaneseChars-8 49.3ns ± 2% 38.5ns ± 2% -21.84% (p=0.000 n=8+8) Fixes #13162 Change-Id: Ifb01f3799c5c93e7f7c7af13a95becfde85ae807 Reviewed-on: https://go-review.googlesource.com/c/go/+/612617 Reviewed-by: Tim King Reviewed-by: Ian Lance Taylor Auto-Submit: Cuong Manh Le LUCI-TryBot-Result: Go LUCI Auto-Submit: Tim King

unicode/utf8: AppendRune and EncodeRune performance improvement

2024-07-22T21:20:30Z

- Prefer the evaluation of the valid higher byte-width runes branches over the one for invalid ones - Avoid the evaluation of the bytes of the RuneError constant, and instead hard code its byte values - EncodeRune only: inline for fast handling of ASCII goos: linux goarch: amd64 pkg: unicode/utf8 cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz │ baseline.test-append.txt │ append-rune-invalid-case-last.test.txt │ │ sec/op │ sec/op vs base │ AppendASCIIRune-8 0.2135n ± 0% 0.2135n ± 0% ~ (p=0.578 n=20) AppendSpanishRune-8 1.645n ± 1% 1.509n ± 2% -8.30% (p=0.000 n=20) AppendJapaneseRune-8 2.196n ± 1% 2.004n ± 1% -8.74% (p=0.000 n=20) AppendMaxRune-8 2.670n ± 1% 2.349n ± 3% -12.01% (p=0.000 n=20) AppendInvalidRuneMaxPlusOne-8 2.214n ± 2% 1.798n ± 3% -18.77% (p=0.000 n=20) AppendInvalidRuneSurrogate-8 2.258n ± 1% 1.793n ± 2% -20.59% (p=0.000 n=20) AppendInvalidRuneNegative-8 2.171n ± 2% 1.767n ± 2% -18.61% (p=0.000 n=20) geomean 1.559n 1.361n -12.69% goos: linux goarch: amd64 pkg: unicode/utf8 cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz │ baseline.test-encode.txt │ encode-rune-invalid-last.txt │ │ sec/op │ sec/op vs base │ EncodeASCIIRune-8 1.0950n ± 1% 0.2140n ± 0% -80.46% (p=0.000 n=20) EncodeSpanishRune-8 1.499n ± 0% 1.414n ± 2% -5.64% (p=0.000 n=20) EncodeJapaneseRune-8 1.960n ± 2% 1.716n ± 4% -12.43% (p=0.000 n=20) EncodeMaxRune-8 2.145n ± 2% 2.227n ± 1% +3.78% (p=0.000 n=20) EncodeInvalidRuneMaxPlusOne-8 1.955n ± 2% 1.802n ± 2% -7.80% (p=0.000 n=20) EncodeInvalidRuneSurrogate-8 1.946n ± 3% 1.777n ± 2% -8.68% (p=0.000 n=20) EncodeInvalidRuneNegative-8 1.968n ± 2% 1.766n ± 2% -10.29% (p=0.000 n=20) geomean 1.757n 1.308n -25.57% Fixes #68131 Change-Id: Ibcafa75d63cca07a2e78cd06f6f1e382cb8c716e Reviewed-on: https://go-review.googlesource.com/c/go/+/594115 LUCI-TryBot-Result: Go LUCI Auto-Submit: Ian Lance Taylor Reviewed-by: Dmitri Shuralyov Reviewed-by: Ian Lance Taylor

unicode/utf8: update doc for RuneLen

2024-03-08T02:00:36Z

As CL 569755 did, for consistency, this CL slightly improves the documentation for RuneLen. Change-Id: Ic9776648baf2809af36cd16a94d1313938bb0e52 Reviewed-on: https://go-review.googlesource.com/c/go/+/569816 Auto-Submit: Ian Lance Taylor Reviewed-by: Michael Knyszek LUCI-TryBot-Result: Go LUCI Reviewed-by: Ian Lance Taylor Commit-Queue: Ian Lance Taylor

unicode: add available godoc link

2023-11-06T20:02:46Z

Change-Id: I2273274249f05b0492950c27dc5a654422cefc79 Reviewed-on: https://go-review.googlesource.com/c/go/+/539856 TryBot-Result: Gopher Robot Reviewed-by: Rob Pike Run-TryBot: shuang cui Reviewed-by: Cherry Mui Reviewed-by: Heschi Kreinick

unicode/utf8: optimize Valid to parity with ValidString

2022-03-02T11:33:18Z

The benchmarks added in this change revealed that ValidString runs ~17% faster than Valid([]byte) on the ASCII prefix of the input. Inspection of the assembly revealed that the code generated for p[8:] required recomputing the slice capacity to handle the cap=0 special case, which added an ADD -8 instruction. By making len=cap, the capacity becomes a common subexpression with the length, saving the ADD instruction. (Thanks to khr for the tip.) Incidentally, I tried a number of other optimizations but was unable to make consistent gains across all benchmarks. The most promising was to retain the bitmask of non-ASCII bytes from the fast loop; the slow loop would shift it, and when it becomes zero, return to the fast loop. This made the MostlyASCII benchmark 4x faster, but made the other cases slower by up to 10%. cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz benchmark old ns/op new ns/op delta BenchmarkValidTenASCIIChars-16 4.09 4.06 -0.85% BenchmarkValid100KASCIIChars-16 9325 7747 -16.92% BenchmarkValidTenJapaneseChars-16 27.0 27.2 +0.85% BenchmarkValidLongMostlyASCII-16 57277 58361 +1.89% BenchmarkValidLongJapanese-16 94002 93131 -0.93% BenchmarkValidStringTenASCIIChars-16 4.15 4.07 -1.74% BenchmarkValidString100KASCIIChars-16 7980 8019 +0.49% BenchmarkValidStringTenJapaneseChars-16 26.0 25.9 -0.38% BenchmarkValidStringLongMostlyASCII-16 58550 58006 -0.93% BenchmarkValidStringLongJapanese-16 97964 100038 +2.12% Change-Id: Ic9d585dedd9af83c27dd791ecd805150ac949f15 Reviewed-on: https://go-review.googlesource.com/c/go/+/375594 Reviewed-by: Keith Randall Run-TryBot: Keith Randall TryBot-Result: Gopher Robot Trust: Alex Rakoczy

unicode/utf8: add AppendRune

2021-08-28T01:49:50Z

AppendRune appends the UTF-8 encoding of a rune to a []byte. It is a generally more user friendly than EncodeRune. EncodeASCIIRune-4 2.35ns ± 2% EncodeJapaneseRune-4 4.60ns ± 2% AppendASCIIRune-4 0.30ns ± 3% AppendJapaneseRune-4 4.70ns ± 2% The ASCII case is written to be inlineable. Fixes #47609 Change-Id: If4f71eedffd2bd4ef0d7f960cb55b41c637eec54 Reviewed-on: https://go-review.googlesource.com/c/go/+/345571 Trust: Joe Tsai Reviewed-by: Rob Pike Run-TryBot: Rob Pike TryBot-Result: Go Bot

unicode/utf8: document the handling of runes out of range in EncodeRune

2020-09-19T09:43:15Z

Document the way EncodeRune currently handles runes which are out of range. Also add an example showing that behaviour. Change-Id: I0f8e7645ae053474ec319085a2bb6d7f73bc137c Reviewed-on: https://go-review.googlesource.com/c/go/+/255998 Reviewed-by: Rob Pike Reviewed-by: Giovanni Bajo Trust: Giovanni Bajo Run-TryBot: Giovanni Bajo TryBot-Result: Go Bot