From 5e939b3a9ce7864551e86c04a05db887c25e2759 Mon Sep 17 00:00:00 2001 From: Roland Shoemaker Date: Mon, 11 Dec 2023 11:44:23 -0800 Subject: doc: add crypto/tls and crypto/x509 release notes Updates #61422 Change-Id: If561f701882396f8e28e2fc3fa9c76c7169f752e Reviewed-on: https://go-review.googlesource.com/c/go/+/548975 Reviewed-by: Filippo Valsorda Reviewed-by: Damien Neil Auto-Submit: Roland Shoemaker LUCI-TryBot-Result: Go LUCI --- doc/go1.22.html | 60 +++++++++++++++++++++++++-------------------------------- 1 file changed, 26 insertions(+), 34 deletions(-) diff --git a/doc/go1.22.html b/doc/go1.22.html index bf01cffce0..792ea655ac 100644 --- a/doc/go1.22.html +++ b/doc/go1.22.html @@ -363,32 +363,22 @@ defer func() {
crypto/tls
-

- TODO: https://go.dev/issue/43922: implement RFC7627 +

+ ConnectionState.ExportKeyingMaterial will now + return an error unless TLS 1.3 is in use, or the extended_master_secret extension is supported by both the server and + client. crypto/tls has supported this extension since Go 1.20. This can be disabled with the + tlsunsafeekm=1 GODEBUG setting.

-

- TODO: https://go.dev/issue/62459: make default minimum version for servers TLS 1.2 +

+ By default, the minimum version offered by crypto/tls servers is now TLS 1.2 if not specified with + config.MinimumVersion, matching the behavior of crypto/tls + clients. This change can be reverted with the tls10server=1 GODEBUG setting.

-

- TODO: https://go.dev/issue/63413: disable RSA key exchange cipher suites by default -

- -

- TODO: https://go.dev/cl/514997: crypto/tls: change SendSessionTicket to take an options struct; modified api/go1.21.txt -

- -

- TODO: https://go.dev/cl/541516: crypto/tls: change default minimum version to 1.2 -

- -

- TODO: https://go.dev/cl/541517: crypto/tls: remove RSA KEX ciphers from the default list -

- -

- TODO: https://go.dev/cl/544155: crypto/tls: disable ExportKeyingMaterial without EMS +

+ By default, cipher suites without ECDHE support are no longer offered by either clients or servers during pre-TLS 1.3 + handshakes. This change can be reverted with the tlsrsakex=1 GODEBUG setting.

@@ -396,23 +386,25 @@ defer func() {
crypto/x509

- TODO: https://go.dev/issue/57178: support code-constrained roots + The new CertPool.AddCertWithConstraint + method can be used to add customized constraints to root certificates to be applied during chain building.

-

- TODO: https://go.dev/issue/58922: add android user trusted CA folder as a possible source for certificate retrieval +

+ On Android, root certificates will now be loaded from /data/misc/keychain/certs-added as well as /system/etc/security/cacerts.

-

- TODO: https://go.dev/issue/60665: introduce new robust OID type & use it for certificate policies -

- -

- TODO: https://go.dev/cl/519315: crypto/x509: implement AddCertWithConstraint; modified api/next/57178.txt -

+

+ A new type, OID, supports ASN.1 Object Identifiers with individual + components larger than 31 bits. A new field which uses this type, Policies, + is added to the Certificate struct, and is now populated during parsing. Any OIDs which cannot be represented + using a asn1.ObjectIdentifier will appear in Policies, + but not in the old PolicyIdentifiers field. -

- TODO: https://go.dev/cl/520535: crypto/x509: add new OID type and use it in Certificate; modified api/next/60665.txt + When calling CreateCertificate, the Policies field is ignored, and + policies are taken from the PolicyIdentifiers field. Using the x509usepolicies=1 GODEBUG setting inverts this, + populating certificate policies from the Policies field, and ignoring the PolicyIdentifiers field. We may change the + default value of x509usepolicies in Go 1.23, making Policies the default field for marshaling.

-- cgit v1.3 From b60bf8f8e131b026eb2691e736d3df9dce852297 Mon Sep 17 00:00:00 2001 From: Keith Randall Date: Thu, 14 Dec 2023 13:07:50 -0800 Subject: cmd/asm: fix encoding for arm right shift by constant 0 Right shifts, for some odd reasons, can encode shifts of constant 1-32 instead of 0-31. Left shifts, however, can encode shifts 0-31. When the shift amount is 0, arm recommends encoding right shifts using left shifts. Fixes #64715 Change-Id: Id3825349aa7195028037893dfe01fa0e405eaa51 Reviewed-on: https://go-review.googlesource.com/c/go/+/549955 Reviewed-by: Cherry Mui Reviewed-by: Cuong Manh Le LUCI-TryBot-Result: Go LUCI Reviewed-by: Keith Randall --- src/cmd/asm/internal/asm/testdata/arm.s | 8 ++++++++ src/cmd/internal/obj/arm/asm5.go | 8 ++++++++ test/fixedbugs/issue64715.go | 25 +++++++++++++++++++++++++ test/fixedbugs/issue64715.out | 1 + 4 files changed, 42 insertions(+) create mode 100644 test/fixedbugs/issue64715.go create mode 100644 test/fixedbugs/issue64715.out diff --git a/src/cmd/asm/internal/asm/testdata/arm.s b/src/cmd/asm/internal/asm/testdata/arm.s index 2ba22c71de..2b8cadbed8 100644 --- a/src/cmd/asm/internal/asm/testdata/arm.s +++ b/src/cmd/asm/internal/asm/testdata/arm.s @@ -870,10 +870,13 @@ jmp_label_3: BIC.S R0@>R1, R2 // 7021d2e1 // SRL + SRL $0, R5, R6 // 0560a0e1 + SRL $1, R5, R6 // a560a0e1 SRL $14, R5, R6 // 2567a0e1 SRL $15, R5, R6 // a567a0e1 SRL $30, R5, R6 // 256fa0e1 SRL $31, R5, R6 // a56fa0e1 + SRL $32, R5, R6 // 2560a0e1 SRL.S $14, R5, R6 // 2567b0e1 SRL.S $15, R5, R6 // a567b0e1 SRL.S $30, R5, R6 // 256fb0e1 @@ -892,10 +895,13 @@ jmp_label_3: SRL.S R5, R7 // 3775b0e1 // SRA + SRA $0, R5, R6 // 0560a0e1 + SRA $1, R5, R6 // c560a0e1 SRA $14, R5, R6 // 4567a0e1 SRA $15, R5, R6 // c567a0e1 SRA $30, R5, R6 // 456fa0e1 SRA $31, R5, R6 // c56fa0e1 + SRA $32, R5, R6 // 4560a0e1 SRA.S $14, R5, R6 // 4567b0e1 SRA.S $15, R5, R6 // c567b0e1 SRA.S $30, R5, R6 // 456fb0e1 @@ -914,6 +920,8 @@ jmp_label_3: SRA.S R5, R7 // 5775b0e1 // SLL + SLL $0, R5, R6 // 0560a0e1 + SLL $1, R5, R6 // 8560a0e1 SLL $14, R5, R6 // 0567a0e1 SLL $15, R5, R6 // 8567a0e1 SLL $30, R5, R6 // 056fa0e1 diff --git a/src/cmd/internal/obj/arm/asm5.go b/src/cmd/internal/obj/arm/asm5.go index 24b9bdd980..9731bd4151 100644 --- a/src/cmd/internal/obj/arm/asm5.go +++ b/src/cmd/internal/obj/arm/asm5.go @@ -1099,6 +1099,14 @@ func (c *ctxt5) oplook(p *obj.Prog) *Optab { fmt.Printf("\t\t%d %d\n", p.From.Type, p.To.Type) } + if (p.As == ASRL || p.As == ASRA) && p.From.Type == obj.TYPE_CONST && p.From.Offset == 0 { + // Right shifts are weird - a shift that looks like "shift by constant 0" actually + // means "shift by constant 32". Use left shift in this situation instead. + // See issue 64715. + // TODO: rotate by 0? Not currently supported, but if we ever do then include it here. + p.As = ASLL + } + ops := oprange[p.As&obj.AMask] c1 := &xcmp[a1] c3 := &xcmp[a3] diff --git a/test/fixedbugs/issue64715.go b/test/fixedbugs/issue64715.go new file mode 100644 index 0000000000..bf117165b7 --- /dev/null +++ b/test/fixedbugs/issue64715.go @@ -0,0 +1,25 @@ +// run + +// Copyright 2023 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package main + +func boolInt32(b bool) int32 { + if b { + return 1 + } + + return 0 +} + +func f(left uint16, right int32) (r uint16) { + return left >> right +} + +var n = uint16(65535) + +func main() { + println(f(n, boolInt32(int64(n^n) > 1))) +} diff --git a/test/fixedbugs/issue64715.out b/test/fixedbugs/issue64715.out new file mode 100644 index 0000000000..7a53b35687 --- /dev/null +++ b/test/fixedbugs/issue64715.out @@ -0,0 +1 @@ +65535 -- cgit v1.3 From 3313bbb4055f38f53cd43c6c5782a229f445f230 Mon Sep 17 00:00:00 2001 From: David Chase Date: Thu, 14 Dec 2023 14:20:12 -0500 Subject: runtime: add race annotations in IncNonDefault Also use CompareAndSwap to make the code actually less racy. Added a test which will be meaningful when run under the race detector (tested it -race with broken fix in runtime, it failed). Fixes #64649 Change-Id: I5972e08901d1adc8ba74858edad7eba91be1b0ce Reviewed-on: https://go-review.googlesource.com/c/go/+/549796 Run-TryBot: David Chase Reviewed-by: Mauri de Souza Meneguzzo Reviewed-by: Cherry Mui TryBot-Result: Gopher Robot --- src/internal/godebug/godebug_test.go | 31 +++++++++++++++++++++++++++++++ src/runtime/runtime.go | 10 +++++++++- 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/src/internal/godebug/godebug_test.go b/src/internal/godebug/godebug_test.go index ed8e93d453..1ed0a365ab 100644 --- a/src/internal/godebug/godebug_test.go +++ b/src/internal/godebug/godebug_test.go @@ -7,6 +7,7 @@ package godebug_test import ( "fmt" . "internal/godebug" + "internal/race" "internal/testenv" "os" "os/exec" @@ -70,6 +71,36 @@ func TestMetrics(t *testing.T) { } } +// TestPanicNilRace checks for a race in the runtime caused by use of runtime +// atomics (not visible to usual race detection) to install the counter for +// non-default panic(nil) semantics. For #64649. +func TestPanicNilRace(t *testing.T) { + if !race.Enabled { + t.Skip("Skipping test intended for use with -race.") + } + if os.Getenv("GODEBUG") != "panicnil=1" { + cmd := testenv.CleanCmdEnv(testenv.Command(t, os.Args[0], "-test.run=^TestPanicNilRace$", "-test.v", "-test.parallel=2", "-test.count=1")) + cmd.Env = append(cmd.Env, "GODEBUG=panicnil=1") + out, err := cmd.CombinedOutput() + t.Logf("output:\n%s", out) + + if err != nil { + t.Errorf("Was not expecting a crash") + } + return + } + + test := func(t *testing.T) { + t.Parallel() + defer func() { + recover() + }() + panic(nil) + } + t.Run("One", test) + t.Run("Two", test) +} + func TestCmdBisect(t *testing.T) { testenv.MustHaveGoBuild(t) out, err := exec.Command("go", "run", "cmd/vendor/golang.org/x/tools/cmd/bisect", "GODEBUG=buggy=1#PATTERN", os.Args[0], "-test.run=^TestBisectTestCase$").CombinedOutput() diff --git a/src/runtime/runtime.go b/src/runtime/runtime.go index 0829a84e43..92cdfc310e 100644 --- a/src/runtime/runtime.go +++ b/src/runtime/runtime.go @@ -172,7 +172,15 @@ func (g *godebugInc) IncNonDefault() { // *godebug.Setting. inc = new(func()) *inc = (*newInc)(g.name) - g.inc.Store(inc) + if raceenabled { + racerelease(unsafe.Pointer(&g.inc)) + } + if !g.inc.CompareAndSwap(nil, inc) { + inc = g.inc.Load() + } + } + if raceenabled { + raceacquire(unsafe.Pointer(&g.inc)) } (*inc)() } -- cgit v1.3 From f8170cc01722a562eec7cf00836ee008b490c534 Mon Sep 17 00:00:00 2001 From: Keith Randall Date: Thu, 14 Dec 2023 16:29:50 -0800 Subject: cmd/asm: for arm, rewrite argument shifted right by 0 to left by 0. Right shift by 0 has bad semantics. Make sure if we try to right shift by 0, do a left shift by 0 instead. CL 549955 handled full instructions with this strange no-op encoding. This CL handles the shift done to instruction register inputs. (The former is implemented using the latter, but not until deep inside the assembler.) Update #64715 Change-Id: Ibfabb4b13e2595551e58b977162fe005aaaa0ad1 Reviewed-on: https://go-review.googlesource.com/c/go/+/550335 Run-TryBot: Cherry Mui TryBot-Result: Gopher Robot Reviewed-by: Cherry Mui LUCI-TryBot-Result: Go LUCI Reviewed-by: Keith Randall --- src/cmd/asm/internal/asm/testdata/arm.s | 14 ++++++++++++++ src/cmd/internal/obj/arm/asm5.go | 18 ++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/src/cmd/asm/internal/asm/testdata/arm.s b/src/cmd/asm/internal/asm/testdata/arm.s index 2b8cadbed8..93edc8854e 100644 --- a/src/cmd/asm/internal/asm/testdata/arm.s +++ b/src/cmd/asm/internal/asm/testdata/arm.s @@ -943,6 +943,20 @@ jmp_label_3: SLL R5, R7 // 1775a0e1 SLL.S R5, R7 // 1775b0e1 +// Ops with zero shifts should encode as left shifts + ADD R0<<0, R1, R2 // 002081e0 + ADD R0>>0, R1, R2 // 002081e0 + ADD R0->0, R1, R2 // 002081e0 + ADD R0@>0, R1, R2 // 002081e0 + MOVW R0<<0(R1), R2 // 002091e7 + MOVW R0>>0(R1), R2 // 002091e7 + MOVW R0->0(R1), R2 // 002091e7 + MOVW R0@>0(R1), R2 // 002091e7 + MOVW R0, R1<<0(R2) // 010082e7 + MOVW R0, R1>>0(R2) // 010082e7 + MOVW R0, R1->0(R2) // 010082e7 + MOVW R0, R1@>0(R2) // 010082e7 + // MULA / MULS MULAWT R1, R2, R3, R4 // c23124e1 MULAWB R1, R2, R3, R4 // 823124e1 diff --git a/src/cmd/internal/obj/arm/asm5.go b/src/cmd/internal/obj/arm/asm5.go index 9731bd4151..4e6eff9e17 100644 --- a/src/cmd/internal/obj/arm/asm5.go +++ b/src/cmd/internal/obj/arm/asm5.go @@ -1106,6 +1106,24 @@ func (c *ctxt5) oplook(p *obj.Prog) *Optab { // TODO: rotate by 0? Not currently supported, but if we ever do then include it here. p.As = ASLL } + if p.As != AMOVB && p.As != AMOVBS && p.As != AMOVBU && p.As != AMOVH && p.As != AMOVHS && p.As != AMOVHU && p.As != AXTAB && p.As != AXTABU && p.As != AXTAH && p.As != AXTAHU { + // Same here, but for shifts encoded in Addrs. + // Don't do it for the extension ops, which + // need to keep their RR shifts. + fixShift := func(a *obj.Addr) { + if a.Type == obj.TYPE_SHIFT { + typ := a.Offset & SHIFT_RR + isConst := a.Offset&(1<<4) == 0 + amount := a.Offset >> 7 & 0x1f + if isConst && amount == 0 && (typ == SHIFT_LR || typ == SHIFT_AR || typ == SHIFT_RR) { + a.Offset -= typ + a.Offset += SHIFT_LL + } + } + } + fixShift(&p.From) + fixShift(&p.To) + } ops := oprange[p.As&obj.AMask] c1 := &xcmp[a1] -- cgit v1.3 From 793097161b3a263b2b0912265cf99d4d9c6218fa Mon Sep 17 00:00:00 2001 From: Jes Cok Date: Fri, 15 Dec 2023 04:33:18 +0000 Subject: all: fix copyright headers Change-Id: Ib16986fb088030b93140e48c17e3d7bb2c9312cf GitHub-Last-Rev: 41572879f8f45c963c0e3ab9862bc0de59781a13 GitHub-Pull-Request: golang/go#64731 Reviewed-on: https://go-review.googlesource.com/c/go/+/549956 Run-TryBot: Jes Cok TryBot-Result: Gopher Robot Reviewed-by: Dmitri Shuralyov Reviewed-by: Bryan Mills Reviewed-by: Dmitri Shuralyov Auto-Submit: Dmitri Shuralyov --- src/cmd/cgo/internal/test/issue9400/asm_mips64x.s | 2 +- src/cmd/cgo/internal/test/issue9400/asm_riscv64.s | 2 +- src/cmd/cgo/internal/testgodefs/testdata/fieldtypedef.go | 2 +- src/cmd/compile/internal/types2/errorcalls_test.go | 2 +- src/cmd/go/internal/mmap/mmap.go | 2 +- src/cmd/go/internal/mmap/mmap_other.go | 2 +- src/cmd/go/internal/mmap/mmap_unix.go | 2 +- src/cmd/go/internal/mmap/mmap_windows.go | 2 +- src/go/types/errorcalls_test.go | 2 +- src/runtime/asan0.go | 2 +- src/runtime/profbuf.go | 2 +- src/runtime/signal_aix_ppc64.go | 2 +- 12 files changed, 12 insertions(+), 12 deletions(-) diff --git a/src/cmd/cgo/internal/test/issue9400/asm_mips64x.s b/src/cmd/cgo/internal/test/issue9400/asm_mips64x.s index 1f492eafe9..3edba3dd82 100644 --- a/src/cmd/cgo/internal/test/issue9400/asm_mips64x.s +++ b/src/cmd/cgo/internal/test/issue9400/asm_mips64x.s @@ -1,4 +1,4 @@ -// Copyright 2016 The Go Authors. All rights reserved. +// Copyright 2016 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/cmd/cgo/internal/test/issue9400/asm_riscv64.s b/src/cmd/cgo/internal/test/issue9400/asm_riscv64.s index fa34f6bd37..0f10e3a326 100644 --- a/src/cmd/cgo/internal/test/issue9400/asm_riscv64.s +++ b/src/cmd/cgo/internal/test/issue9400/asm_riscv64.s @@ -1,4 +1,4 @@ -// Copyright 2020 The Go Authors. All rights reserved. +// Copyright 2020 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/cmd/cgo/internal/testgodefs/testdata/fieldtypedef.go b/src/cmd/cgo/internal/testgodefs/testdata/fieldtypedef.go index b0c507477f..d3ab1902c1 100644 --- a/src/cmd/cgo/internal/testgodefs/testdata/fieldtypedef.go +++ b/src/cmd/cgo/internal/testgodefs/testdata/fieldtypedef.go @@ -1,4 +1,4 @@ -// Copyright 2018 The Go Authors. All rights reserve d. +// Copyright 2018 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/cmd/compile/internal/types2/errorcalls_test.go b/src/cmd/compile/internal/types2/errorcalls_test.go index 6153b42a34..ba4dc87b6a 100644 --- a/src/cmd/compile/internal/types2/errorcalls_test.go +++ b/src/cmd/compile/internal/types2/errorcalls_test.go @@ -1,6 +1,6 @@ // Copyright 2021 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE ast. +// license that can be found in the LICENSE file. package types2_test diff --git a/src/cmd/go/internal/mmap/mmap.go b/src/cmd/go/internal/mmap/mmap.go index 0cad9caf27..fcbd3e08c1 100644 --- a/src/cmd/go/internal/mmap/mmap.go +++ b/src/cmd/go/internal/mmap/mmap.go @@ -1,4 +1,4 @@ -// Copyright 2011 The Go Authors. All rights reserved. +// Copyright 2011 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/cmd/go/internal/mmap/mmap_other.go b/src/cmd/go/internal/mmap/mmap_other.go index 22e9395b21..4d2844fc37 100644 --- a/src/cmd/go/internal/mmap/mmap_other.go +++ b/src/cmd/go/internal/mmap/mmap_other.go @@ -1,4 +1,4 @@ -// Copyright 2022 The Go Authors. All rights reserved. +// Copyright 2022 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/cmd/go/internal/mmap/mmap_unix.go b/src/cmd/go/internal/mmap/mmap_unix.go index 53bcbb92a8..5dce872368 100644 --- a/src/cmd/go/internal/mmap/mmap_unix.go +++ b/src/cmd/go/internal/mmap/mmap_unix.go @@ -1,4 +1,4 @@ -// Copyright 2011 The Go Authors. All rights reserved. +// Copyright 2011 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/cmd/go/internal/mmap/mmap_windows.go b/src/cmd/go/internal/mmap/mmap_windows.go index 1cf62feca3..d00bef71e5 100644 --- a/src/cmd/go/internal/mmap/mmap_windows.go +++ b/src/cmd/go/internal/mmap/mmap_windows.go @@ -1,4 +1,4 @@ -// Copyright 2011 The Go Authors. All rights reserved. +// Copyright 2011 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/go/types/errorcalls_test.go b/src/go/types/errorcalls_test.go index d76c06da30..0238909282 100644 --- a/src/go/types/errorcalls_test.go +++ b/src/go/types/errorcalls_test.go @@ -1,6 +1,6 @@ // Copyright 2021 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE ast. +// license that can be found in the LICENSE file. package types_test diff --git a/src/runtime/asan0.go b/src/runtime/asan0.go index 0948786200..bcfd96f1ab 100644 --- a/src/runtime/asan0.go +++ b/src/runtime/asan0.go @@ -1,4 +1,4 @@ -// Copyright 2021 The Go Authors. All rights reserved. +// Copyright 2021 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/runtime/profbuf.go b/src/runtime/profbuf.go index 5772a8020c..d3afbcd8c7 100644 --- a/src/runtime/profbuf.go +++ b/src/runtime/profbuf.go @@ -1,4 +1,4 @@ -// Copyright 2017 The Go Authors. All rights reserved. +// Copyright 2017 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. diff --git a/src/runtime/signal_aix_ppc64.go b/src/runtime/signal_aix_ppc64.go index c6cb91a0a2..8ae0f749ed 100644 --- a/src/runtime/signal_aix_ppc64.go +++ b/src/runtime/signal_aix_ppc64.go @@ -1,4 +1,4 @@ -/// Copyright 2018 The Go Authors. All rights reserved. +// Copyright 2018 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. -- cgit v1.3 From 9b4b3e5acca2dabe107fa2c3ed963097d78a4562 Mon Sep 17 00:00:00 2001 From: Michael Pratt Date: Wed, 13 Dec 2023 15:07:28 -0500 Subject: runtime: properly model rwmutex in lock ranking Currently, lock ranking doesn't really try to model rwmutex. It records the internal locks rLock and wLock, but in a subpar fashion: 1. wLock is held from lock to unlock, so it works OK, but it conflates write locks of all rwmutexes as rwmutexW, rather than allowing different rwmutexes to have different rankings. 2. rLock is an internal implementation detail that is only taken when there is contention in rlock. As as result, the reader lock path is almost never checked. Add proper modeling. rwmutexR and rwmutexW remain as the ranks of the internal locks, which have their own ordering. The new init method is passed the ranks of the higher level lock that this represents, just like lockInit for mutex. execW ordered before MALLOC captures the case from #64722. i.e., there can be allocation between BeforeFork and AfterFork. For #64722. Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-staticlockranking Change-Id: I23335b28faa42fb04f1bc9da02fdf54d1616cd28 Reviewed-on: https://go-review.googlesource.com/c/go/+/549536 Reviewed-by: Michael Knyszek LUCI-TryBot-Result: Go LUCI --- src/runtime/export_test.go | 4 +++ src/runtime/lockrank.go | 87 +++++++++++++++++++++++++++------------------ src/runtime/mklockrank.go | 39 ++++++++++++++++---- src/runtime/proc.go | 2 ++ src/runtime/rwmutex.go | 48 ++++++++++++++++++++++--- src/runtime/rwmutex_test.go | 4 +++ 6 files changed, 138 insertions(+), 46 deletions(-) diff --git a/src/runtime/export_test.go b/src/runtime/export_test.go index 2e707b96e2..11e4bd26c5 100644 --- a/src/runtime/export_test.go +++ b/src/runtime/export_test.go @@ -586,6 +586,10 @@ type RWMutex struct { rw rwmutex } +func (rw *RWMutex) Init() { + rw.rw.init(lockRankTestR, lockRankTestW) +} + func (rw *RWMutex) RLock() { rw.rw.rlock() } diff --git a/src/runtime/lockrank.go b/src/runtime/lockrank.go index c56991acb8..43b3239f1e 100644 --- a/src/runtime/lockrank.go +++ b/src/runtime/lockrank.go @@ -18,9 +18,16 @@ const ( lockRankSweepWaiters lockRankAssistQueue lockRankSweep - lockRankPollDesc + lockRankTestR + lockRankTestW + lockRankAllocmW + lockRankExecW lockRankCpuprof + lockRankPollDesc lockRankWakeableSleep + // SCHED + lockRankAllocmR + lockRankExecR lockRankSched lockRankAllg lockRankAllp @@ -29,8 +36,6 @@ const ( lockRankHchan lockRankNotifyList lockRankSudog - lockRankRwmutexW - lockRankRwmutexR lockRankRoot lockRankItab lockRankReflectOffs @@ -64,6 +69,8 @@ const ( lockRankPanic lockRankDeadlock lockRankRaceFini + lockRankRwmutexW + lockRankRwmutexR ) // lockRankLeafRank is the rank of lock that does not have a declared rank, @@ -79,9 +86,15 @@ var lockNames = []string{ lockRankSweepWaiters: "sweepWaiters", lockRankAssistQueue: "assistQueue", lockRankSweep: "sweep", - lockRankPollDesc: "pollDesc", + lockRankTestR: "testR", + lockRankTestW: "testW", + lockRankAllocmW: "allocmW", + lockRankExecW: "execW", lockRankCpuprof: "cpuprof", + lockRankPollDesc: "pollDesc", lockRankWakeableSleep: "wakeableSleep", + lockRankAllocmR: "allocmR", + lockRankExecR: "execR", lockRankSched: "sched", lockRankAllg: "allg", lockRankAllp: "allp", @@ -90,8 +103,6 @@ var lockNames = []string{ lockRankHchan: "hchan", lockRankNotifyList: "notifyList", lockRankSudog: "sudog", - lockRankRwmutexW: "rwmutexW", - lockRankRwmutexR: "rwmutexR", lockRankRoot: "root", lockRankItab: "itab", lockRankReflectOffs: "reflectOffs", @@ -119,6 +130,8 @@ var lockNames = []string{ lockRankPanic: "panic", lockRankDeadlock: "deadlock", lockRankRaceFini: "raceFini", + lockRankRwmutexW: "rwmutexW", + lockRankRwmutexR: "rwmutexR", } func (rank lockRank) String() string { @@ -147,44 +160,50 @@ var lockPartialOrder [][]lockRank = [][]lockRank{ lockRankSweepWaiters: {}, lockRankAssistQueue: {}, lockRankSweep: {}, - lockRankPollDesc: {}, + lockRankTestR: {}, + lockRankTestW: {}, + lockRankAllocmW: {}, + lockRankExecW: {}, lockRankCpuprof: {}, + lockRankPollDesc: {}, lockRankWakeableSleep: {}, - lockRankSched: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep}, - lockRankAllg: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched}, - lockRankAllp: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched}, - lockRankTimers: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllp, lockRankTimers}, - lockRankNetpollInit: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllp, lockRankTimers}, - lockRankHchan: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankWakeableSleep, lockRankHchan}, + lockRankAllocmR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep}, + lockRankExecR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep}, + lockRankSched: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR}, + lockRankAllg: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched}, + lockRankAllp: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched}, + lockRankTimers: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllp, lockRankTimers}, + lockRankNetpollInit: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllp, lockRankTimers}, + lockRankHchan: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankWakeableSleep, lockRankHchan}, lockRankNotifyList: {}, - lockRankSudog: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankWakeableSleep, lockRankHchan, lockRankNotifyList}, - lockRankRwmutexW: {}, - lockRankRwmutexR: {lockRankSysmon, lockRankRwmutexW}, + lockRankSudog: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankWakeableSleep, lockRankHchan, lockRankNotifyList}, lockRankRoot: {}, lockRankItab: {}, lockRankReflectOffs: {lockRankItab}, lockRankUserArenaState: {}, lockRankTraceBuf: {lockRankSysmon, lockRankScavenge}, lockRankTraceStrings: {lockRankSysmon, lockRankScavenge, lockRankTraceBuf}, - lockRankFin: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, - lockRankSpanSetSpine: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, - lockRankMspanSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, - lockRankGcBitsArenas: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankMspanSpecial}, - lockRankProfInsert: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, - lockRankProfBlock: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, - lockRankProfMemActive: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, - lockRankProfMemFuture: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankProfMemActive}, - lockRankGscan: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture}, - lockRankStackpool: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRwmutexW, lockRankRwmutexR, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan}, - lockRankStackLarge: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan}, - lockRankHchanLeaf: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankHchanLeaf}, - lockRankWbufSpans: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan}, - lockRankMheap: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRwmutexW, lockRankRwmutexR, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans}, - lockRankMheapSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRwmutexW, lockRankRwmutexR, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap}, - lockRankGlobalAlloc: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRwmutexW, lockRankRwmutexR, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankMheapSpecial}, - lockRankTrace: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRwmutexW, lockRankRwmutexR, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap}, - lockRankTraceStackTab: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankPollDesc, lockRankCpuprof, lockRankWakeableSleep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRwmutexW, lockRankRwmutexR, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankTrace}, + lockRankFin: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, + lockRankSpanSetSpine: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, + lockRankMspanSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, + lockRankGcBitsArenas: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankMspanSpecial}, + lockRankProfInsert: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, + lockRankProfBlock: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, + lockRankProfMemActive: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings}, + lockRankProfMemFuture: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankHchan, lockRankNotifyList, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankProfMemActive}, + lockRankGscan: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture}, + lockRankStackpool: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan}, + lockRankStackLarge: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan}, + lockRankHchanLeaf: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankHchanLeaf}, + lockRankWbufSpans: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan}, + lockRankMheap: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans}, + lockRankMheapSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap}, + lockRankGlobalAlloc: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankMheapSpecial}, + lockRankTrace: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap}, + lockRankTraceStackTab: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankNetpollInit, lockRankHchan, lockRankNotifyList, lockRankSudog, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankTrace}, lockRankPanic: {}, lockRankDeadlock: {lockRankPanic, lockRankDeadlock}, lockRankRaceFini: {lockRankPanic}, + lockRankRwmutexW: {lockRankTestW, lockRankAllocmW, lockRankExecW}, + lockRankRwmutexR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTestW, lockRankAllocmW, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankAllocmR, lockRankExecR, lockRankRwmutexW}, } diff --git a/src/runtime/mklockrank.go b/src/runtime/mklockrank.go index 241439bdfb..d9ffcbef8d 100644 --- a/src/runtime/mklockrank.go +++ b/src/runtime/mklockrank.go @@ -52,8 +52,16 @@ NONE < assistQueue, sweep; +# Test only +NONE < testR, testW; + # Scheduler, timers, netpoll -NONE < pollDesc, cpuprof, wakeableSleep; +NONE < + allocmW, + execW, + cpuprof, + pollDesc, + wakeableSleep; assistQueue, cpuprof, forcegc, @@ -61,21 +69,23 @@ assistQueue, scavenge, sweep, sweepWaiters, + testR, wakeableSleep +# Above SCHED are things that can call into the scheduler. +< SCHED +# Below SCHED is the scheduler implementation. +< allocmR, + execR < sched; sched < allg, allp; allp, wakeableSleep < timers; timers < netpollInit; # Channels -scavenge, sweep, wakeableSleep < hchan; +scavenge, sweep, testR, wakeableSleep < hchan; NONE < notifyList; hchan, notifyList < sudog; -# RWMutex -NONE < rwmutexW; -rwmutexW, sysmon < rwmutexR; - # Semaphores NONE < root; @@ -100,6 +110,9 @@ traceBuf < traceStrings; # Malloc allg, + allocmR, + execR, # May grow stack + execW, # May allocate after BeforeFork hchan, notifyList, reflectOffs, @@ -136,7 +149,7 @@ gcBitsArenas, < STACKGROW # Below STACKGROW is the stack allocator/copying implementation. < gscan; -gscan, rwmutexR < stackpool; +gscan < stackpool; gscan < stackLarge; # Generally, hchan must be acquired before gscan. But in one case, # where we suspend a G and then shrink its stack, syncadjustsudogs @@ -189,6 +202,18 @@ NONE < panic; panic < deadlock; # raceFini is only held while exiting. panic < raceFini; + +# RWMutex +allocmW, + execW, + testW +< rwmutexW; + +rwmutexW, + allocmR, + execR, + testR +< rwmutexR; ` // cyclicRanks lists lock ranks that allow multiple locks of the same diff --git a/src/runtime/proc.go b/src/runtime/proc.go index aae30dc2a8..b9715f267e 100644 --- a/src/runtime/proc.go +++ b/src/runtime/proc.go @@ -759,6 +759,8 @@ func schedinit() { lockInit(&reflectOffs.lock, lockRankReflectOffs) lockInit(&finlock, lockRankFin) lockInit(&cpuprof.lock, lockRankCpuprof) + allocmLock.init(lockRankAllocmR, lockRankAllocmW) + execLock.init(lockRankExecR, lockRankExecW) traceLockInit() // Enforce that this lock is always a leaf lock. // All of this lock's critical sections should be diff --git a/src/runtime/rwmutex.go b/src/runtime/rwmutex.go index ede3d13599..89fe16c10d 100644 --- a/src/runtime/rwmutex.go +++ b/src/runtime/rwmutex.go @@ -25,6 +25,37 @@ type rwmutex struct { readerCount atomic.Int32 // number of pending readers readerWait atomic.Int32 // number of departing readers + + readRank lockRank // semantic lock rank for read locking + writeRank lockRank // semantic lock rank for write locking +} + +// Lock ranking an rwmutex has two aspects: +// +// Semantic ranking: this rwmutex represents some higher level lock that +// protects some resource (e.g., allocmLock protects creation of new Ms). The +// read and write locks of that resource need to be represented in the lock +// rank. +// +// Internal ranking: as an implementation detail, rwmutex uses two mutexes: +// rLock and wLock. These have lock order requirements: wLock must be locked +// before rLock. This also needs to be represented in the lock rank. +// +// Internal ranking is represented by assigning ranks rwmutexR and rwmutexW to +// rLock and wLock, respectively. +// +// Semantic ranking is represented by acquiring readRank during read lock and +// writeRank during write lock. +// +// readRank is always taken before rwmutexR and writeRank is always taken +// before rwmutexW, so each unique rwmutex must record this order in the lock +// ranking. +func (rw *rwmutex) init(readRank, writeRank lockRank) { + rw.readRank = readRank + rw.writeRank = writeRank + + lockInit(&rw.rLock, lockRankRwmutexR) + lockInit(&rw.wLock, lockRankRwmutexW) } const rwmutexMaxReaders = 1 << 30 @@ -36,10 +67,14 @@ func (rw *rwmutex) rlock() { // deadlock (issue #20903). Alternatively, we could drop the P // while sleeping. acquirem() + + acquireLockRank(rw.readRank) + lockWithRankMayAcquire(&rw.rLock, getLockRank(&rw.rLock)) + if rw.readerCount.Add(1) < 0 { // A writer is pending. Park on the reader queue. systemstack(func() { - lockWithRank(&rw.rLock, lockRankRwmutexR) + lock(&rw.rLock) if rw.readerPass > 0 { // Writer finished. rw.readerPass -= 1 @@ -67,7 +102,7 @@ func (rw *rwmutex) runlock() { // A writer is pending. if rw.readerWait.Add(-1) == 0 { // The last reader unblocks the writer. - lockWithRank(&rw.rLock, lockRankRwmutexR) + lock(&rw.rLock) w := rw.writer.ptr() if w != nil { notewakeup(&w.park) @@ -75,18 +110,20 @@ func (rw *rwmutex) runlock() { unlock(&rw.rLock) } } + releaseLockRank(rw.readRank) releasem(getg().m) } // lock locks rw for writing. func (rw *rwmutex) lock() { // Resolve competition with other writers and stick to our P. - lockWithRank(&rw.wLock, lockRankRwmutexW) + acquireLockRank(rw.writeRank) + lock(&rw.wLock) m := getg().m // Announce that there is a pending writer. r := rw.readerCount.Add(-rwmutexMaxReaders) + rwmutexMaxReaders // Wait for any active readers to complete. - lockWithRank(&rw.rLock, lockRankRwmutexR) + lock(&rw.rLock) if r != 0 && rw.readerWait.Add(r) != 0 { // Wait for reader to wake us up. systemstack(func() { @@ -108,7 +145,7 @@ func (rw *rwmutex) unlock() { throw("unlock of unlocked rwmutex") } // Unblock blocked readers. - lockWithRank(&rw.rLock, lockRankRwmutexR) + lock(&rw.rLock) for rw.readers.ptr() != nil { reader := rw.readers.ptr() rw.readers = reader.schedlink @@ -122,4 +159,5 @@ func (rw *rwmutex) unlock() { unlock(&rw.rLock) // Allow other writers to proceed. unlock(&rw.wLock) + releaseLockRank(rw.writeRank) } diff --git a/src/runtime/rwmutex_test.go b/src/runtime/rwmutex_test.go index ddb16aead4..bdeb9c4901 100644 --- a/src/runtime/rwmutex_test.go +++ b/src/runtime/rwmutex_test.go @@ -29,6 +29,7 @@ func parallelReader(m *RWMutex, clocked chan bool, cunlock *atomic.Bool, cdone c func doTestParallelReaders(numReaders int) { GOMAXPROCS(numReaders + 1) var m RWMutex + m.Init() clocked := make(chan bool, numReaders) var cunlock atomic.Bool cdone := make(chan bool) @@ -100,6 +101,7 @@ func HammerRWMutex(gomaxprocs, numReaders, num_iterations int) { // Number of active readers + 10000 * number of active writers. var activity int32 var rwm RWMutex + rwm.Init() cdone := make(chan bool) go writer(&rwm, num_iterations, &activity, cdone) var i int @@ -141,6 +143,7 @@ func BenchmarkRWMutexUncontended(b *testing.B) { } b.RunParallel(func(pb *testing.PB) { var rwm PaddedRWMutex + rwm.Init() for pb.Next() { rwm.RLock() rwm.RLock() @@ -154,6 +157,7 @@ func BenchmarkRWMutexUncontended(b *testing.B) { func benchmarkRWMutex(b *testing.B, localWork, writeRatio int) { var rwm RWMutex + rwm.Init() b.RunParallel(func(pb *testing.PB) { foo := 0 for pb.Next() { -- cgit v1.3 From 8e3930f2588a3f6d8fcaac7ef8add9c2d5dedc0e Mon Sep 17 00:00:00 2001 From: Michael Anthony Knyszek Date: Mon, 18 Dec 2023 17:03:19 +0000 Subject: runtime: skip TestRuntimeLockMetricsAndProfile for flakiness This test was added to check new mutex profile functionality. Specifically, it checks to make sure that the functionality behind GODEBUG=runtimecontentionstacks works. The runtime currently tracks contention from runtime-internal mutexes in mutex profiles, but it does not record stack traces for them, attributing the time to a dummy symbol. This GODEBUG enables collecting stacks. Just disable the test. Even if this functionality breaks, it won't affect Go users and it'll help keep the builders green. It's fine to leave the test because this will be revisited in the next dev cycle. For #64253. Change-Id: I7938fe0f036fc4e4a0764f030e691e312ec2c9b5 Reviewed-on: https://go-review.googlesource.com/c/go/+/550775 LUCI-TryBot-Result: Go LUCI Reviewed-by: Eli Bendersky Reviewed-by: Cherry Mui Auto-Submit: Michael Knyszek --- src/runtime/metrics_test.go | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/runtime/metrics_test.go b/src/runtime/metrics_test.go index fc0321e4c5..d7f41334cd 100644 --- a/src/runtime/metrics_test.go +++ b/src/runtime/metrics_test.go @@ -9,6 +9,7 @@ import ( "fmt" "internal/goexperiment" "internal/profile" + "internal/testenv" "os" "reflect" "runtime" @@ -947,6 +948,8 @@ func TestSchedPauseMetrics(t *testing.T) { } func TestRuntimeLockMetricsAndProfile(t *testing.T) { + testenv.SkipFlaky(t, 64253) + old := runtime.SetMutexProfileFraction(0) // enabled during sub-tests defer runtime.SetMutexProfileFraction(old) if old != 0 { -- cgit v1.3 From a7097243e462802f3055da9f843013e450a75092 Mon Sep 17 00:00:00 2001 From: "Bryan C. Mills" Date: Tue, 12 Dec 2023 16:26:45 -0500 Subject: internal/syscall/windows: fix the signature of SetFileInformationByHandle Also fix its call site in internal/poll to pass the length of the actual buffer instead of an unrelated variable, and update the definition of FILE_BASIC_INFO to match the documented field types and add padding that is empirically needed on the 386 architecture. Passing a pointer to a Go-allocated buffer as type uintptr violates the unsafe.Pointer conversion rules, which allow such a conversion only in the call expression itself for a call to syscall.Syscall or equivalent. That can allow the buffer to be corrupted arbitrarily if the Go runtime happens to garbage-collect it while the call to SetFileInformationByHandle is in progress. The Microsoft documentation for SetFileInformationByHandle specifies its third argument type as LPVOID, which corresponds to Go's unsafe.Pointer, not uintptr. Fixes #58933 (maybe). Change-Id: If577b57adea9922f5fcca55e46030c703d8f035c Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/549256 LUCI-TryBot-Result: Go LUCI Reviewed-by: Than McIntosh Auto-Submit: Bryan Mills Reviewed-by: Quim Muntal Reviewed-by: Alex Brainman --- src/internal/poll/fd_windows.go | 3 +-- src/internal/syscall/windows/syscall_windows.go | 21 ++++++++++++++++----- src/internal/syscall/windows/zsyscall_windows.go | 2 +- 3 files changed, 18 insertions(+), 8 deletions(-) diff --git a/src/internal/poll/fd_windows.go b/src/internal/poll/fd_windows.go index 9df39edced..2095a6aa29 100644 --- a/src/internal/poll/fd_windows.go +++ b/src/internal/poll/fd_windows.go @@ -1037,8 +1037,7 @@ func (fd *FD) Fchmod(mode uint32) error { var du windows.FILE_BASIC_INFO du.FileAttributes = attrs - l := uint32(unsafe.Sizeof(d)) - return windows.SetFileInformationByHandle(fd.Sysfd, windows.FileBasicInfo, uintptr(unsafe.Pointer(&du)), l) + return windows.SetFileInformationByHandle(fd.Sysfd, windows.FileBasicInfo, unsafe.Pointer(&du), uint32(unsafe.Sizeof(du))) } // Fchdir wraps syscall.Fchdir. diff --git a/src/internal/syscall/windows/syscall_windows.go b/src/internal/syscall/windows/syscall_windows.go index 5854ca60b5..d10e30cb68 100644 --- a/src/internal/syscall/windows/syscall_windows.go +++ b/src/internal/syscall/windows/syscall_windows.go @@ -129,11 +129,22 @@ type SecurityAttributes struct { } type FILE_BASIC_INFO struct { - CreationTime syscall.Filetime - LastAccessTime syscall.Filetime - LastWriteTime syscall.Filetime - ChangedTime syscall.Filetime + CreationTime int64 + LastAccessTime int64 + LastWriteTime int64 + ChangedTime int64 FileAttributes uint32 + + // Pad out to 8-byte alignment. + // + // Without this padding, TestChmod fails due to an argument validation error + // in SetFileInformationByHandle on windows/386. + // + // https://learn.microsoft.com/en-us/cpp/build/reference/zp-struct-member-alignment?view=msvc-170 + // says that “The C/C++ headers in the Windows SDK assume the platform's + // default alignment is used.” What we see here is padding rather than + // alignment, but maybe it is related. + _ uint32 } const ( @@ -150,7 +161,7 @@ const ( //sys GetComputerNameEx(nameformat uint32, buf *uint16, n *uint32) (err error) = GetComputerNameExW //sys MoveFileEx(from *uint16, to *uint16, flags uint32) (err error) = MoveFileExW //sys GetModuleFileName(module syscall.Handle, fn *uint16, len uint32) (n uint32, err error) = kernel32.GetModuleFileNameW -//sys SetFileInformationByHandle(handle syscall.Handle, fileInformationClass uint32, buf uintptr, bufsize uint32) (err error) = kernel32.SetFileInformationByHandle +//sys SetFileInformationByHandle(handle syscall.Handle, fileInformationClass uint32, buf unsafe.Pointer, bufsize uint32) (err error) = kernel32.SetFileInformationByHandle //sys VirtualQuery(address uintptr, buffer *MemoryBasicInformation, length uintptr) (err error) = kernel32.VirtualQuery //sys GetTempPath2(buflen uint32, buf *uint16) (n uint32, err error) = GetTempPath2W diff --git a/src/internal/syscall/windows/zsyscall_windows.go b/src/internal/syscall/windows/zsyscall_windows.go index 5a587ad4f1..931f157cf1 100644 --- a/src/internal/syscall/windows/zsyscall_windows.go +++ b/src/internal/syscall/windows/zsyscall_windows.go @@ -342,7 +342,7 @@ func RtlVirtualUnwind(handlerType uint32, baseAddress uintptr, pc uintptr, entry return } -func SetFileInformationByHandle(handle syscall.Handle, fileInformationClass uint32, buf uintptr, bufsize uint32) (err error) { +func SetFileInformationByHandle(handle syscall.Handle, fileInformationClass uint32, buf unsafe.Pointer, bufsize uint32) (err error) { r1, _, e1 := syscall.Syscall6(procSetFileInformationByHandle.Addr(), 4, uintptr(handle), uintptr(fileInformationClass), uintptr(buf), uintptr(bufsize), 0, 0) if r1 == 0 { err = errnoErr(e1) -- cgit v1.3 From 2acbdd086d4660ae5e4adc429858df4758d675e3 Mon Sep 17 00:00:00 2001 From: Mauri de Souza Meneguzzo Date: Wed, 13 Dec 2023 19:58:23 +0000 Subject: cmd/cgo/internal/testsanitizers: fix msan test failing with clang >= 16 Clang 16 introduced a more aggressive behavior regarding uninitialized memory in the memory sanitizer. The new option -fsanitize-memory-param-retval is enabled by default and makes the test msan8 fail, since it uses an uninitialized variable on purpose. Disable this behavior if we are running with clang 16+. Fixes #64616 Change-Id: If366f978bef984ea73f6ae958f24c8fce99b59fe GitHub-Last-Rev: 60bd64a8fb24a552fce23fb2b43a75e92443e039 GitHub-Pull-Request: golang/go#64691 Reviewed-on: https://go-review.googlesource.com/c/go/+/549297 Auto-Submit: Bryan Mills Reviewed-by: Bryan Mills LUCI-TryBot-Result: Go LUCI Reviewed-by: Than McIntosh --- src/cmd/cgo/internal/testsanitizers/msan_test.go | 5 ++++- src/cmd/cgo/internal/testsanitizers/testdata/msan8.go | 7 +++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/src/cmd/cgo/internal/testsanitizers/msan_test.go b/src/cmd/cgo/internal/testsanitizers/msan_test.go index 83d66f6660..c534b72442 100644 --- a/src/cmd/cgo/internal/testsanitizers/msan_test.go +++ b/src/cmd/cgo/internal/testsanitizers/msan_test.go @@ -71,7 +71,10 @@ func TestMSAN(t *testing.T) { defer dir.RemoveAll(t) outPath := dir.Join(name) - mustRun(t, config.goCmdWithExperiments("build", []string{"-o", outPath, srcPath(tc.src)}, tc.experiments)) + buildcmd := config.goCmdWithExperiments("build", []string{"-o", outPath, srcPath(tc.src)}, tc.experiments) + // allow tests to define -f flags in CGO_CFLAGS + replaceEnv(buildcmd, "CGO_CFLAGS_ALLOW", "-f.*") + mustRun(t, buildcmd) cmd := hangProneCmd(outPath) if tc.wantErr { diff --git a/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go b/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go index 1cb5c5677f..e79d343cc7 100644 --- a/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go +++ b/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go @@ -5,6 +5,13 @@ package main /* +// For clang >= 16, uninitialized memory is more aggressively reported. +// Restore the old behavior for this particular test as it relies on +// uninitialized variables. See #64616 +#if __clang_major__ >= 16 +#cgo CFLAGS: -fno-sanitize-memory-param-retval +#endif + #include #include #include -- cgit v1.3 From 08bec0db3970319ef868b3c11d6cc077cab408b1 Mon Sep 17 00:00:00 2001 From: Irakli Safareli Date: Mon, 18 Dec 2023 08:53:03 +0000 Subject: builtin: mention PanicNilError in comments of recover As of CL 461956 `recover` will not return `nil` if `panic` is called with `nil`. I have updated comments of `recover` to account for this change. Change-Id: Ibd0b27fe9b89fb29349b62ad34e762239a1d165b GitHub-Last-Rev: c773abb75c8cd8e08c3470f064a3205573156fea GitHub-Pull-Request: golang/go#64393 Reviewed-on: https://go-review.googlesource.com/c/go/+/544975 Reviewed-by: Mauri de Souza Meneguzzo Reviewed-by: Than McIntosh Reviewed-by: Keith Randall LUCI-TryBot-Result: Go LUCI Reviewed-by: Keith Randall --- src/builtin/builtin.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/builtin/builtin.go b/src/builtin/builtin.go index da0ace1498..668c799ca7 100644 --- a/src/builtin/builtin.go +++ b/src/builtin/builtin.go @@ -284,9 +284,10 @@ func panic(v any) // by restoring normal execution and retrieves the error value passed to the // call of panic. If recover is called outside the deferred function it will // not stop a panicking sequence. In this case, or when the goroutine is not -// panicking, or if the argument supplied to panic was nil, recover returns -// nil. Thus the return value from recover reports whether the goroutine is -// panicking. +// panicking, recover returns nil. +// +// Prior to Go 1.21, recover would also return nil if panic is called with +// a nil argument. See [panic] for details. func recover() any // The print built-in function formats its arguments in an -- cgit v1.3 From 450f5d90c2c85cb2b031bcf4a65c3b1467231977 Mon Sep 17 00:00:00 2001 From: Russ Cox Date: Mon, 18 Dec 2023 13:34:36 -0500 Subject: doc: add math/rand/v2 release notes Change-Id: If1922413ff948f9b8d8cebec6756b6870f38c162 Reviewed-on: https://go-review.googlesource.com/c/go/+/550777 Reviewed-by: David Chase LUCI-TryBot-Result: Go LUCI --- doc/go1.22.html | 109 ++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 79 insertions(+), 30 deletions(-) diff --git a/doc/go1.22.html b/doc/go1.22.html index 792ea655ac..3d63e99eeb 100644 --- a/doc/go1.22.html +++ b/doc/go1.22.html @@ -274,40 +274,89 @@ defer func() {

New math/rand/v2 package

-

- TODO: https://go.dev/cl/502495: math/rand/v2: start of new API; modified api/next/61716.txt -

- -

- TODO: https://go.dev/cl/502497: math/rand/v2: remove Read; modified api/next/61716.txt -

- -

- TODO: https://go.dev/cl/502498: math/rand/v2: remove Rand.Seed; modified api/next/61716.txt -

- -

- TODO: https://go.dev/cl/502499: math/rand/v2: change Source to use uint64; modified api/next/61716.txt -

- -

- TODO: https://go.dev/cl/502500: math/rand/v2: add, optimize N, UintN, Uint32N, Uint64N; modified api/next/61716.txt -

- -

- TODO: https://go.dev/cl/502505: math/rand/v2: add PCG-DXSM; modified api/next/61716.txt -

+ + + + + + + + + -

- TODO: https://go.dev/cl/502506: math/rand/v2: delete Mitchell/Reeds source; modified api/next/61716.txt +

+ Go 1.22 includes the first “v2” package in the standard library, + math/rand/v2. + The changes compared to math/rand are + detailed in proposal #61716. The most important changes are:

-

- TODO: https://go.dev/cl/516857: math/rand/v2: rename various functions; modified api/next/61716.txt -

+
    +
  • The Read method, deprecated in math/rand, +was not carried forward for math/rand/v2. +(It remains available in math/rand.) +The vast majority of calls to Read should use +crypto/rand’s Read instead. +Otherwise a custom Read can be constructed using the Uint64 method. + +
  • The global generator accessed by top-level functions is unconditionally randomly seeded. +Because the API guarantees no fixed sequence of results, +optimizations like per-thread random generator states are now possible. + +
  • The Source +interface now has a single Uint64 method; +there is no Source64 interface. + +
  • Many methods now use faster algorithms that were not possible to adopt in math/rand +because they changed the output streams. + +
  • The +Intn, +Int31, +Int31n, +Int63, +and +Int64n +top-level functions and methods from math/rand +are spelled more idiomatically in math/rand/v2: +IntN, +Int32, +Int32N, +Int64, +and +Int64N. +There are also new top-level functions and methods +Uint32, +Uint32N, +Uint64, +Uint64N, +Uint, +and +UintN. + +
  • The +new generic function N +is like +Int64N or +Uint64N +but works for any integer type. +For example a random duration from 0 up to 5 minutes is +rand.N(5*time.Minute). + +
  • The Mitchell & Reeds LFSR generator provided by +math/rand’s Source +has been replaced by two more modern pseudo-random generator sources: +ChaCha8 +PCG. +ChaCha8 is a new, cryptographically strong random number generator +roughly similar to PCG in efficiency. +ChaCha8 is the algorithm used for the top-level functions in math/rand/v2. +As of Go 1.22, math/rand's top-level functions (when not explicitly seeded) +and the Go runtime also use ChaCha8 for randomness. +
-

- TODO: https://go.dev/cl/516859: math/rand/v2: add ChaCha8; modified api/next/61716.txt +

+We plan to include an API migration tool in a future release, likely Go 1.23.

Minor changes to the library

-- cgit v1.3 From 761e10be8805bf0b3afd015cf9f09856a6066ad2 Mon Sep 17 00:00:00 2001 From: qmuntal Date: Sun, 17 Dec 2023 17:51:25 +0100 Subject: cmd/link/internal/loadpe: update comment about @feat.00 symbol handling The @feat.00 comment is outdated, as Go does support SEH on windows/amd64. While here, simplify it and add a link to the relevant documentation. Change-Id: I6e3c786ca4f4809baf46015edf569d06b12f1239 Reviewed-on: https://go-review.googlesource.com/c/go/+/550635 Reviewed-by: Than McIntosh TryBot-Result: Gopher Robot Reviewed-by: Dmitri Shuralyov Run-TryBot: Quim Muntal LUCI-TryBot-Result: Go LUCI --- src/cmd/link/internal/loadpe/ldpe.go | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/src/cmd/link/internal/loadpe/ldpe.go b/src/cmd/link/internal/loadpe/ldpe.go index e4134607c4..1ba6debb4e 100644 --- a/src/cmd/link/internal/loadpe/ldpe.go +++ b/src/cmd/link/internal/loadpe/ldpe.go @@ -493,17 +493,10 @@ func Load(l *loader.Loader, arch *sys.Arch, localSymVersion int, input *bio.Read continue } if pesym.SectionNumber == IMAGE_SYM_ABSOLUTE && bytes.Equal(pesym.Name[:], []byte("@feat.00")) { - // Microsoft's linker looks at whether all input objects have an empty - // section called @feat.00. If all of them do, then it enables SEH; - // otherwise it doesn't enable that feature. So, since around the Windows - // XP SP2 era, most tools that make PE objects just tack on that section, - // so that it won't gimp Microsoft's linker logic. Go doesn't support SEH, - // so in theory, none of this really matters to us. But actually, if the - // linker tries to ingest an object with @feat.00 -- which are produced by - // LLVM's resource compiler, for example -- it chokes because of the - // IMAGE_SYM_ABSOLUTE section that it doesn't know how to deal with. Since - // @feat.00 is just a marking anyway, skip IMAGE_SYM_ABSOLUTE sections that - // are called @feat.00. + // The PE documentation says that, on x86 platforms, the absolute symbol named @feat.00 + // is used to indicate that the COFF object supports SEH. + // Go doesn't support SEH on windows/386, so we can ignore this symbol. + // See https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#the-sxdata-section continue } var sect *pe.Section -- cgit v1.3 From 7058f09a8b2570d4a7bfdd64017300206c22ebfe Mon Sep 17 00:00:00 2001 From: Tim King Date: Mon, 18 Dec 2023 12:01:43 -0800 Subject: cmd: go get golang.org/x/tools@83bceaf2 and revendor go get golang.org/x/tools@83bceaf2 # CL 550395 go mod tidy go mod vendor Fixes #64786 Change-Id: Ia9879975eb3e8e4130ded7b2d8ba1277b5811eec Reviewed-on: https://go-review.googlesource.com/c/go/+/550895 TryBot-Result: Gopher Robot Run-TryBot: Tim King Reviewed-by: Alan Donovan LUCI-TryBot-Result: Go LUCI --- src/cmd/go.mod | 2 +- src/cmd/go.sum | 4 +- .../go/analysis/passes/composite/composite.go | 4 +- .../tools/go/analysis/passes/copylock/copylock.go | 2 +- .../analysis/passes/ifaceassert/parameterized.go | 4 +- .../passes/internal/analysisutil/extractdoc.go | 113 ------ .../analysis/passes/internal/analysisutil/util.go | 4 + .../x/tools/go/analysis/passes/nilfunc/nilfunc.go | 2 +- .../x/tools/go/analysis/passes/printf/types.go | 2 +- .../x/tools/go/analysis/passes/shift/shift.go | 2 +- .../go/analysis/passes/stringintconv/string.go | 2 +- .../go/analysis/passes/testinggoroutine/util.go | 2 +- .../x/tools/go/analysis/passes/tests/tests.go | 5 +- .../go/analysis/passes/unmarshal/unmarshal.go | 3 +- .../x/tools/go/analysis/unitchecker/unitchecker.go | 3 +- .../golang.org/x/tools/go/ast/astutil/enclosing.go | 8 +- .../golang.org/x/tools/go/ast/astutil/rewrite.go | 8 +- .../golang.org/x/tools/go/ast/inspector/typeof.go | 4 +- .../x/tools/go/types/objectpath/objectpath.go | 14 +- .../golang.org/x/tools/go/types/typeutil/callee.go | 2 +- .../golang.org/x/tools/go/types/typeutil/map.go | 20 +- .../x/tools/internal/analysisinternal/analysis.go | 386 +++++++++++++++++++++ .../tools/internal/analysisinternal/extractdoc.go | 113 ++++++ .../golang.org/x/tools/internal/facts/imports.go | 14 +- .../x/tools/internal/typeparams/common.go | 24 +- .../x/tools/internal/typeparams/coretype.go | 8 +- .../x/tools/internal/typeparams/enabled_go117.go | 12 - .../x/tools/internal/typeparams/enabled_go118.go | 15 - .../x/tools/internal/typeparams/normalize.go | 20 +- .../tools/internal/typeparams/typeparams_go117.go | 197 ----------- .../tools/internal/typeparams/typeparams_go118.go | 151 -------- .../x/tools/internal/versions/versions.go | 52 +++ .../x/tools/internal/versions/versions_go121.go | 49 --- .../x/tools/internal/versions/versions_go122.go | 38 -- src/cmd/vendor/modules.txt | 3 +- 35 files changed, 631 insertions(+), 661 deletions(-) delete mode 100644 src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/extractdoc.go create mode 100644 src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/analysis.go create mode 100644 src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/extractdoc.go delete mode 100644 src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go117.go delete mode 100644 src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go118.go delete mode 100644 src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go117.go delete mode 100644 src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go118.go create mode 100644 src/cmd/vendor/golang.org/x/tools/internal/versions/versions.go delete mode 100644 src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go121.go delete mode 100644 src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go122.go diff --git a/src/cmd/go.mod b/src/cmd/go.mod index 68ded94ba8..7a426887b4 100644 --- a/src/cmd/go.mod +++ b/src/cmd/go.mod @@ -9,7 +9,7 @@ require ( golang.org/x/sync v0.5.0 golang.org/x/sys v0.15.0 golang.org/x/term v0.15.0 - golang.org/x/tools v0.16.1-0.20231129202111-1b1e4dafd51a + golang.org/x/tools v0.16.2-0.20231218185909-83bceaf2424d ) require github.com/ianlancetaylor/demangle v0.0.0-20230524184225-eabc099b10ab // indirect diff --git a/src/cmd/go.sum b/src/cmd/go.sum index 680616da07..8ea3d75bd1 100644 --- a/src/cmd/go.sum +++ b/src/cmd/go.sum @@ -12,5 +12,5 @@ golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.15.0 h1:y/Oo/a/q3IXu26lQgl04j/gjuBDOBlx7X6Om1j2CPW4= golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0= -golang.org/x/tools v0.16.1-0.20231129202111-1b1e4dafd51a h1:0+Cdrtl1VNF5TeDTr0mLLPN4dMJFHnstbqMo/o1aReI= -golang.org/x/tools v0.16.1-0.20231129202111-1b1e4dafd51a/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0= +golang.org/x/tools v0.16.2-0.20231218185909-83bceaf2424d h1:9YOyUBubvYqtjjtZBnI62JT9/QB9jfPwOQ7xLeyuOIU= +golang.org/x/tools v0.16.2-0.20231218185909-83bceaf2424d/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0= diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/composite/composite.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/composite/composite.go index c7a49776fe..847063bb32 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/composite/composite.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/composite/composite.go @@ -72,7 +72,7 @@ func run(pass *analysis.Pass) (interface{}, error) { } var structuralTypes []types.Type switch typ := typ.(type) { - case *typeparams.TypeParam: + case *types.TypeParam: terms, err := typeparams.StructuralTerms(typ) if err != nil { return // invalid type @@ -163,7 +163,7 @@ func isLocalType(pass *analysis.Pass, typ types.Type) bool { case *types.Named: // names in package foo are local to foo_test too return strings.TrimSuffix(x.Obj().Pkg().Path(), "_test") == strings.TrimSuffix(pass.Pkg.Path(), "_test") - case *typeparams.TypeParam: + case *types.TypeParam: return strings.TrimSuffix(x.Obj().Pkg().Path(), "_test") == strings.TrimSuffix(pass.Pkg.Path(), "_test") } return false diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/copylock/copylock.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/copylock/copylock.go index 2eeb0a330a..6cbbc7e814 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/copylock/copylock.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/copylock/copylock.go @@ -255,7 +255,7 @@ func lockPath(tpkg *types.Package, typ types.Type, seen map[types.Type]bool) typ } seen[typ] = true - if tpar, ok := typ.(*typeparams.TypeParam); ok { + if tpar, ok := typ.(*types.TypeParam); ok { terms, err := typeparams.StructuralTerms(tpar) if err != nil { return nil // invalid type diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/ifaceassert/parameterized.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/ifaceassert/parameterized.go index b84577fcf8..12507f9967 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/ifaceassert/parameterized.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/ifaceassert/parameterized.go @@ -95,14 +95,14 @@ func (w *tpWalker) isParameterized(typ types.Type) (res bool) { return w.isParameterized(t.Elem()) case *types.Named: - list := typeparams.NamedTypeArgs(t) + list := t.TypeArgs() for i, n := 0, list.Len(); i < n; i++ { if w.isParameterized(list.At(i)) { return true } } - case *typeparams.TypeParam: + case *types.TypeParam: return true default: diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/extractdoc.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/extractdoc.go deleted file mode 100644 index 0e175ca06f..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/extractdoc.go +++ /dev/null @@ -1,113 +0,0 @@ -// Copyright 2023 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -package analysisutil - -import ( - "fmt" - "go/parser" - "go/token" - "strings" -) - -// MustExtractDoc is like [ExtractDoc] but it panics on error. -// -// To use, define a doc.go file such as: -// -// // Package halting defines an analyzer of program termination. -// // -// // # Analyzer halting -// // -// // halting: reports whether execution will halt. -// // -// // The halting analyzer reports a diagnostic for functions -// // that run forever. To suppress the diagnostics, try inserting -// // a 'break' statement into each loop. -// package halting -// -// import _ "embed" -// -// //go:embed doc.go -// var doc string -// -// And declare your analyzer as: -// -// var Analyzer = &analysis.Analyzer{ -// Name: "halting", -// Doc: analysisutil.MustExtractDoc(doc, "halting"), -// ... -// } -func MustExtractDoc(content, name string) string { - doc, err := ExtractDoc(content, name) - if err != nil { - panic(err) - } - return doc -} - -// ExtractDoc extracts a section of a package doc comment from the -// provided contents of an analyzer package's doc.go file. -// -// A section is a portion of the comment between one heading and -// the next, using this form: -// -// # Analyzer NAME -// -// NAME: SUMMARY -// -// Full description... -// -// where NAME matches the name argument, and SUMMARY is a brief -// verb-phrase that describes the analyzer. The following lines, up -// until the next heading or the end of the comment, contain the full -// description. ExtractDoc returns the portion following the colon, -// which is the form expected by Analyzer.Doc. -// -// Example: -// -// # Analyzer printf -// -// printf: checks consistency of calls to printf -// -// The printf analyzer checks consistency of calls to printf. -// Here is the complete description... -// -// This notation allows a single doc comment to provide documentation -// for multiple analyzers, each in its own section. -// The HTML anchors generated for each heading are predictable. -// -// It returns an error if the content was not a valid Go source file -// containing a package doc comment with a heading of the required -// form. -// -// This machinery enables the package documentation (typically -// accessible via the web at https://pkg.go.dev/) and the command -// documentation (typically printed to a terminal) to be derived from -// the same source and formatted appropriately. -func ExtractDoc(content, name string) (string, error) { - if content == "" { - return "", fmt.Errorf("empty Go source file") - } - fset := token.NewFileSet() - f, err := parser.ParseFile(fset, "", content, parser.ParseComments|parser.PackageClauseOnly) - if err != nil { - return "", fmt.Errorf("not a Go source file") - } - if f.Doc == nil { - return "", fmt.Errorf("Go source file has no package doc comment") - } - for _, section := range strings.Split(f.Doc.Text(), "\n# ") { - if body := strings.TrimPrefix(section, "Analyzer "+name); body != section && - body != "" && - body[0] == '\r' || body[0] == '\n' { - body = strings.TrimSpace(body) - rest := strings.TrimPrefix(body, name+":") - if rest == body { - return "", fmt.Errorf("'Analyzer %s' heading not followed by '%s: summary...' line", name, name) - } - return strings.TrimSpace(rest), nil - } - } - return "", fmt.Errorf("package doc comment contains no 'Analyzer %s' heading", name) -} diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/util.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/util.go index c0060753f9..3f01b3b55d 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/util.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil/util.go @@ -13,6 +13,8 @@ import ( "go/token" "go/types" "os" + + "golang.org/x/tools/internal/analysisinternal" ) // Format returns a string representation of the expression. @@ -150,3 +152,5 @@ func IsFunctionNamed(f *types.Func, pkgPath string, names ...string) bool { } return false } + +var MustExtractDoc = analysisinternal.MustExtractDoc diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/nilfunc/nilfunc.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/nilfunc/nilfunc.go index 6df134399a..778f7f1f8f 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/nilfunc/nilfunc.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/nilfunc/nilfunc.go @@ -62,7 +62,7 @@ func run(pass *analysis.Pass) (interface{}, error) { obj = pass.TypesInfo.Uses[v] case *ast.SelectorExpr: obj = pass.TypesInfo.Uses[v.Sel] - case *ast.IndexExpr, *typeparams.IndexListExpr: + case *ast.IndexExpr, *ast.IndexListExpr: // Check generic functions such as "f[T1,T2]". x, _, _, _ := typeparams.UnpackIndexExpr(v) if id, ok := x.(*ast.Ident); ok { diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/printf/types.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/printf/types.go index 7cbb0bdbf5..ab98e56998 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/printf/types.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/printf/types.go @@ -72,7 +72,7 @@ func (m *argMatcher) match(typ types.Type, topLevel bool) bool { return true } - if typ, _ := typ.(*typeparams.TypeParam); typ != nil { + if typ, _ := typ.(*types.TypeParam); typ != nil { // Avoid infinite recursion through type parameters. if m.seen[typ] { return true diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/shift/shift.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/shift/shift.go index bafb9112e1..e272df709f 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/shift/shift.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/shift/shift.go @@ -99,7 +99,7 @@ func checkLongShift(pass *analysis.Pass, node ast.Node, x, y ast.Expr) { } var structuralTypes []types.Type switch t := t.(type) { - case *typeparams.TypeParam: + case *types.TypeParam: terms, err := typeparams.StructuralTerms(t) if err != nil { return // invalid type diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/stringintconv/string.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/stringintconv/string.go index bb04dae626..b2591ccff5 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/stringintconv/string.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/stringintconv/string.go @@ -195,7 +195,7 @@ func run(pass *analysis.Pass) (interface{}, error) { func structuralTypes(t types.Type) ([]types.Type, error) { var structuralTypes []types.Type switch t := t.(type) { - case *typeparams.TypeParam: + case *types.TypeParam: terms, err := typeparams.StructuralTerms(t) if err != nil { return nil, err diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/testinggoroutine/util.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/testinggoroutine/util.go index 805ccf49e4..d156851db1 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/testinggoroutine/util.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/testinggoroutine/util.go @@ -57,7 +57,7 @@ func isMethodNamed(f *types.Func, pkgPath string, names ...string) bool { func funcIdent(fun ast.Expr) *ast.Ident { switch fun := astutil.Unparen(fun).(type) { - case *ast.IndexExpr, *typeparams.IndexListExpr: + case *ast.IndexExpr, *ast.IndexListExpr: x, _, _, _ := typeparams.UnpackIndexExpr(fun) // necessary? id, _ := x.(*ast.Ident) return id diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/tests/tests.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/tests/tests.go index d0b0ebb101..6db12f3cb9 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/tests/tests.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/tests/tests.go @@ -17,7 +17,6 @@ import ( "golang.org/x/tools/go/analysis" "golang.org/x/tools/go/analysis/passes/internal/analysisutil" - "golang.org/x/tools/internal/typeparams" ) //go:embed doc.go @@ -391,7 +390,7 @@ func checkExampleName(pass *analysis.Pass, fn *ast.FuncDecl) { if results := fn.Type.Results; results != nil && len(results.List) != 0 { pass.Reportf(fn.Pos(), "%s should return nothing", fnName) } - if tparams := typeparams.ForFuncType(fn.Type); tparams != nil && len(tparams.List) > 0 { + if tparams := fn.Type.TypeParams; tparams != nil && len(tparams.List) > 0 { pass.Reportf(fn.Pos(), "%s should not have type params", fnName) } @@ -460,7 +459,7 @@ func checkTest(pass *analysis.Pass, fn *ast.FuncDecl, prefix string) { return } - if tparams := typeparams.ForFuncType(fn.Type); tparams != nil && len(tparams.List) > 0 { + if tparams := fn.Type.TypeParams; tparams != nil && len(tparams.List) > 0 { // Note: cmd/go/internal/load also errors about TestXXX and BenchmarkXXX functions with type parameters. // We have currently decided to also warn before compilation/package loading. This can help users in IDEs. // TODO(adonovan): use ReportRangef(tparams). diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/unmarshal/unmarshal.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/unmarshal/unmarshal.go index 7043baa899..f4e73528b4 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/unmarshal/unmarshal.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/unmarshal/unmarshal.go @@ -14,7 +14,6 @@ import ( "golang.org/x/tools/go/analysis/passes/internal/analysisutil" "golang.org/x/tools/go/ast/inspector" "golang.org/x/tools/go/types/typeutil" - "golang.org/x/tools/internal/typeparams" ) //go:embed doc.go @@ -92,7 +91,7 @@ func run(pass *analysis.Pass) (interface{}, error) { t := pass.TypesInfo.Types[call.Args[argidx]].Type switch t.Underlying().(type) { - case *types.Pointer, *types.Interface, *typeparams.TypeParam: + case *types.Pointer, *types.Interface, *types.TypeParam: return } diff --git a/src/cmd/vendor/golang.org/x/tools/go/analysis/unitchecker/unitchecker.go b/src/cmd/vendor/golang.org/x/tools/go/analysis/unitchecker/unitchecker.go index 36eed808d8..1fa0d1f68f 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/analysis/unitchecker/unitchecker.go +++ b/src/cmd/vendor/golang.org/x/tools/go/analysis/unitchecker/unitchecker.go @@ -50,7 +50,6 @@ import ( "golang.org/x/tools/go/analysis" "golang.org/x/tools/go/analysis/internal/analysisflags" "golang.org/x/tools/internal/facts" - "golang.org/x/tools/internal/typeparams" "golang.org/x/tools/internal/versions" ) @@ -259,10 +258,10 @@ func run(fset *token.FileSet, cfg *Config, analyzers []*analysis.Analyzer) ([]re Defs: make(map[*ast.Ident]types.Object), Uses: make(map[*ast.Ident]types.Object), Implicits: make(map[ast.Node]types.Object), + Instances: make(map[*ast.Ident]types.Instance), Scopes: make(map[ast.Node]*types.Scope), Selections: make(map[*ast.SelectorExpr]*types.Selection), } - typeparams.InitInstanceInfo(info) versions.InitFileVersions(info) pkg, err := tc.Check(cfg.ImportPath, fset, files, info) diff --git a/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/enclosing.go b/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/enclosing.go index 9fa5aa192c..2c4c4e2328 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/enclosing.go +++ b/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/enclosing.go @@ -11,8 +11,6 @@ import ( "go/ast" "go/token" "sort" - - "golang.org/x/tools/internal/typeparams" ) // PathEnclosingInterval returns the node that encloses the source @@ -322,7 +320,7 @@ func childrenOf(n ast.Node) []ast.Node { children = append(children, n.Recv) } children = append(children, n.Name) - if tparams := typeparams.ForFuncType(n.Type); tparams != nil { + if tparams := n.Type.TypeParams; tparams != nil { children = append(children, tparams) } if n.Type.Params != nil { @@ -377,7 +375,7 @@ func childrenOf(n ast.Node) []ast.Node { tok(n.Lbrack, len("[")), tok(n.Rbrack, len("]"))) - case *typeparams.IndexListExpr: + case *ast.IndexListExpr: children = append(children, tok(n.Lbrack, len("[")), tok(n.Rbrack, len("]"))) @@ -588,7 +586,7 @@ func NodeDescription(n ast.Node) string { return "decrement statement" case *ast.IndexExpr: return "index expression" - case *typeparams.IndexListExpr: + case *ast.IndexListExpr: return "index list expression" case *ast.InterfaceType: return "interface type" diff --git a/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/rewrite.go b/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/rewrite.go index f430b21b9b..58934f7663 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/rewrite.go +++ b/src/cmd/vendor/golang.org/x/tools/go/ast/astutil/rewrite.go @@ -9,8 +9,6 @@ import ( "go/ast" "reflect" "sort" - - "golang.org/x/tools/internal/typeparams" ) // An ApplyFunc is invoked by Apply for each node n, even if n is nil, @@ -252,7 +250,7 @@ func (a *application) apply(parent ast.Node, name string, iter *iterator, n ast. a.apply(n, "X", nil, n.X) a.apply(n, "Index", nil, n.Index) - case *typeparams.IndexListExpr: + case *ast.IndexListExpr: a.apply(n, "X", nil, n.X) a.applyList(n, "Indices") @@ -293,7 +291,7 @@ func (a *application) apply(parent ast.Node, name string, iter *iterator, n ast. a.apply(n, "Fields", nil, n.Fields) case *ast.FuncType: - if tparams := typeparams.ForFuncType(n); tparams != nil { + if tparams := n.TypeParams; tparams != nil { a.apply(n, "TypeParams", nil, tparams) } a.apply(n, "Params", nil, n.Params) @@ -408,7 +406,7 @@ func (a *application) apply(parent ast.Node, name string, iter *iterator, n ast. case *ast.TypeSpec: a.apply(n, "Doc", nil, n.Doc) a.apply(n, "Name", nil, n.Name) - if tparams := typeparams.ForTypeSpec(n); tparams != nil { + if tparams := n.TypeParams; tparams != nil { a.apply(n, "TypeParams", nil, tparams) } a.apply(n, "Type", nil, n.Type) diff --git a/src/cmd/vendor/golang.org/x/tools/go/ast/inspector/typeof.go b/src/cmd/vendor/golang.org/x/tools/go/ast/inspector/typeof.go index 703c813954..2a872f89d4 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/ast/inspector/typeof.go +++ b/src/cmd/vendor/golang.org/x/tools/go/ast/inspector/typeof.go @@ -12,8 +12,6 @@ package inspector import ( "go/ast" "math" - - "golang.org/x/tools/internal/typeparams" ) const ( @@ -171,7 +169,7 @@ func typeOf(n ast.Node) uint64 { return 1 << nIncDecStmt case *ast.IndexExpr: return 1 << nIndexExpr - case *typeparams.IndexListExpr: + case *ast.IndexListExpr: return 1 << nIndexListExpr case *ast.InterfaceType: return 1 << nInterfaceType diff --git a/src/cmd/vendor/golang.org/x/tools/go/types/objectpath/objectpath.go b/src/cmd/vendor/golang.org/x/tools/go/types/objectpath/objectpath.go index e742ecc464..11d5c8c3ad 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/types/objectpath/objectpath.go +++ b/src/cmd/vendor/golang.org/x/tools/go/types/objectpath/objectpath.go @@ -223,7 +223,7 @@ func (enc *Encoder) For(obj types.Object) (Path, error) { // Reject obviously non-viable cases. switch obj := obj.(type) { case *types.TypeName: - if _, ok := obj.Type().(*typeparams.TypeParam); !ok { + if _, ok := obj.Type().(*types.TypeParam); !ok { // With the exception of type parameters, only package-level type names // have a path. return "", fmt.Errorf("no path for %v", obj) @@ -283,7 +283,7 @@ func (enc *Encoder) For(obj types.Object) (Path, error) { } } else { if named, _ := T.(*types.Named); named != nil { - if r := findTypeParam(obj, typeparams.ForNamed(named), path, nil); r != nil { + if r := findTypeParam(obj, named.TypeParams(), path, nil); r != nil { // generic named type return Path(r), nil } @@ -462,7 +462,7 @@ func find(obj types.Object, T types.Type, path []byte, seen map[*types.TypeName] } return find(obj, T.Elem(), append(path, opElem), seen) case *types.Signature: - if r := findTypeParam(obj, typeparams.ForSignature(T), path, seen); r != nil { + if r := findTypeParam(obj, T.TypeParams(), path, seen); r != nil { return r } if r := find(obj, T.Params(), append(path, opParams), seen); r != nil { @@ -505,7 +505,7 @@ func find(obj types.Object, T types.Type, path []byte, seen map[*types.TypeName] } } return nil - case *typeparams.TypeParam: + case *types.TypeParam: name := T.Obj() if name == obj { return append(path, opObj) @@ -525,7 +525,7 @@ func find(obj types.Object, T types.Type, path []byte, seen map[*types.TypeName] panic(T) } -func findTypeParam(obj types.Object, list *typeparams.TypeParamList, path []byte, seen map[*types.TypeName]bool) []byte { +func findTypeParam(obj types.Object, list *types.TypeParamList, path []byte, seen map[*types.TypeName]bool) []byte { for i := 0; i < list.Len(); i++ { tparam := list.At(i) path2 := appendOpArg(path, opTypeParam, i) @@ -562,7 +562,7 @@ func Object(pkg *types.Package, p Path) (types.Object, error) { } // abstraction of *types.{Named,Signature} type hasTypeParams interface { - TypeParams() *typeparams.TypeParamList + TypeParams() *types.TypeParamList } // abstraction of *types.{Named,TypeParam} type hasObj interface { @@ -664,7 +664,7 @@ func Object(pkg *types.Package, p Path) (types.Object, error) { t = tparams.At(index) case opConstraint: - tparam, ok := t.(*typeparams.TypeParam) + tparam, ok := t.(*types.TypeParam) if !ok { return nil, fmt.Errorf("cannot apply %q to %s (got %T, want type parameter)", code, t, t) } diff --git a/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/callee.go b/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/callee.go index 90b3ab0e21..90dc541adf 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/callee.go +++ b/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/callee.go @@ -22,7 +22,7 @@ func Callee(info *types.Info, call *ast.CallExpr) types.Object { // Look through type instantiation if necessary. isInstance := false switch fun.(type) { - case *ast.IndexExpr, *typeparams.IndexListExpr: + case *ast.IndexExpr, *ast.IndexListExpr: // When extracting the callee from an *IndexExpr, we need to check that // it is a *types.Func and not a *types.Var. // Example: Don't match a slice m within the expression `m[0]()`. diff --git a/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/map.go b/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/map.go index 7bd2fdb38b..544246dac1 100644 --- a/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/map.go +++ b/src/cmd/vendor/golang.org/x/tools/go/types/typeutil/map.go @@ -219,7 +219,7 @@ type Hasher struct { // generic types or functions, and instantiated signatures do not have type // parameter lists, we should never encounter a second non-empty type // parameter list when hashing a generic signature. - sigTParams *typeparams.TypeParamList + sigTParams *types.TypeParamList } // MakeHasher returns a new Hasher instance. @@ -297,7 +297,7 @@ func (h Hasher) hashFor(t types.Type) uint32 { // We should never encounter a generic signature while hashing another // generic signature, but defensively set sigTParams only if h.mask is // unset. - tparams := typeparams.ForSignature(t) + tparams := t.TypeParams() if h.sigTParams == nil && tparams.Len() != 0 { h = Hasher{ // There may be something more efficient than discarding the existing @@ -318,7 +318,7 @@ func (h Hasher) hashFor(t types.Type) uint32 { return hash + 3*h.hashTuple(t.Params()) + 5*h.hashTuple(t.Results()) - case *typeparams.Union: + case *types.Union: return h.hashUnion(t) case *types.Interface: @@ -354,14 +354,14 @@ func (h Hasher) hashFor(t types.Type) uint32 { case *types.Named: hash := h.hashPtr(t.Obj()) - targs := typeparams.NamedTypeArgs(t) + targs := t.TypeArgs() for i := 0; i < targs.Len(); i++ { targ := targs.At(i) hash += 2 * h.Hash(targ) } return hash - case *typeparams.TypeParam: + case *types.TypeParam: return h.hashTypeParam(t) case *types.Tuple: @@ -381,7 +381,7 @@ func (h Hasher) hashTuple(tuple *types.Tuple) uint32 { return hash } -func (h Hasher) hashUnion(t *typeparams.Union) uint32 { +func (h Hasher) hashUnion(t *types.Union) uint32 { // Hash type restrictions. terms, err := typeparams.UnionTermSet(t) // if err != nil t has invalid type restrictions. Fall back on a non-zero @@ -392,7 +392,7 @@ func (h Hasher) hashUnion(t *typeparams.Union) uint32 { return h.hashTermSet(terms) } -func (h Hasher) hashTermSet(terms []*typeparams.Term) uint32 { +func (h Hasher) hashTermSet(terms []*types.Term) uint32 { hash := 9157 + 2*uint32(len(terms)) for _, term := range terms { // term order is not significant. @@ -416,7 +416,7 @@ func (h Hasher) hashTermSet(terms []*typeparams.Term) uint32 { // are not identical. // // Otherwise the hash of t depends only on t's pointer identity. -func (h Hasher) hashTypeParam(t *typeparams.TypeParam) uint32 { +func (h Hasher) hashTypeParam(t *types.TypeParam) uint32 { if h.sigTParams != nil { i := t.Index() if i >= 0 && i < h.sigTParams.Len() && t == h.sigTParams.At(i) { @@ -489,7 +489,7 @@ func (h Hasher) shallowHash(t types.Type) uint32 { case *types.Pointer: return 4393139 - case *typeparams.Union: + case *types.Union: return 562448657 case *types.Interface: @@ -504,7 +504,7 @@ func (h Hasher) shallowHash(t types.Type) uint32 { case *types.Named: return h.hashPtr(t.Obj()) - case *typeparams.TypeParam: + case *types.TypeParam: return h.hashPtr(t.Obj()) } panic(fmt.Sprintf("shallowHash: %T: %v", t, t)) diff --git a/src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/analysis.go b/src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/analysis.go new file mode 100644 index 0000000000..2b29168047 --- /dev/null +++ b/src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/analysis.go @@ -0,0 +1,386 @@ +// Copyright 2020 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// Package analysisinternal provides gopls' internal analyses with a +// number of helper functions that operate on typed syntax trees. +package analysisinternal + +import ( + "bytes" + "fmt" + "go/ast" + "go/token" + "go/types" + "strconv" +) + +func TypeErrorEndPos(fset *token.FileSet, src []byte, start token.Pos) token.Pos { + // Get the end position for the type error. + offset, end := fset.PositionFor(start, false).Offset, start + if offset >= len(src) { + return end + } + if width := bytes.IndexAny(src[offset:], " \n,():;[]+-*"); width > 0 { + end = start + token.Pos(width) + } + return end +} + +func ZeroValue(f *ast.File, pkg *types.Package, typ types.Type) ast.Expr { + under := typ + if n, ok := typ.(*types.Named); ok { + under = n.Underlying() + } + switch u := under.(type) { + case *types.Basic: + switch { + case u.Info()&types.IsNumeric != 0: + return &ast.BasicLit{Kind: token.INT, Value: "0"} + case u.Info()&types.IsBoolean != 0: + return &ast.Ident{Name: "false"} + case u.Info()&types.IsString != 0: + return &ast.BasicLit{Kind: token.STRING, Value: `""`} + default: + panic(fmt.Sprintf("unknown basic type %v", u)) + } + case *types.Chan, *types.Interface, *types.Map, *types.Pointer, *types.Signature, *types.Slice, *types.Array: + return ast.NewIdent("nil") + case *types.Struct: + texpr := TypeExpr(f, pkg, typ) // typ because we want the name here. + if texpr == nil { + return nil + } + return &ast.CompositeLit{ + Type: texpr, + } + } + return nil +} + +// IsZeroValue checks whether the given expression is a 'zero value' (as determined by output of +// analysisinternal.ZeroValue) +func IsZeroValue(expr ast.Expr) bool { + switch e := expr.(type) { + case *ast.BasicLit: + return e.Value == "0" || e.Value == `""` + case *ast.Ident: + return e.Name == "nil" || e.Name == "false" + default: + return false + } +} + +// TypeExpr returns syntax for the specified type. References to +// named types from packages other than pkg are qualified by an appropriate +// package name, as defined by the import environment of file. +func TypeExpr(f *ast.File, pkg *types.Package, typ types.Type) ast.Expr { + switch t := typ.(type) { + case *types.Basic: + switch t.Kind() { + case types.UnsafePointer: + return &ast.SelectorExpr{X: ast.NewIdent("unsafe"), Sel: ast.NewIdent("Pointer")} + default: + return ast.NewIdent(t.Name()) + } + case *types.Pointer: + x := TypeExpr(f, pkg, t.Elem()) + if x == nil { + return nil + } + return &ast.UnaryExpr{ + Op: token.MUL, + X: x, + } + case *types.Array: + elt := TypeExpr(f, pkg, t.Elem()) + if elt == nil { + return nil + } + return &ast.ArrayType{ + Len: &ast.BasicLit{ + Kind: token.INT, + Value: fmt.Sprintf("%d", t.Len()), + }, + Elt: elt, + } + case *types.Slice: + elt := TypeExpr(f, pkg, t.Elem()) + if elt == nil { + return nil + } + return &ast.ArrayType{ + Elt: elt, + } + case *types.Map: + key := TypeExpr(f, pkg, t.Key()) + value := TypeExpr(f, pkg, t.Elem()) + if key == nil || value == nil { + return nil + } + return &ast.MapType{ + Key: key, + Value: value, + } + case *types.Chan: + dir := ast.ChanDir(t.Dir()) + if t.Dir() == types.SendRecv { + dir = ast.SEND | ast.RECV + } + value := TypeExpr(f, pkg, t.Elem()) + if value == nil { + return nil + } + return &ast.ChanType{ + Dir: dir, + Value: value, + } + case *types.Signature: + var params []*ast.Field + for i := 0; i < t.Params().Len(); i++ { + p := TypeExpr(f, pkg, t.Params().At(i).Type()) + if p == nil { + return nil + } + params = append(params, &ast.Field{ + Type: p, + Names: []*ast.Ident{ + { + Name: t.Params().At(i).Name(), + }, + }, + }) + } + var returns []*ast.Field + for i := 0; i < t.Results().Len(); i++ { + r := TypeExpr(f, pkg, t.Results().At(i).Type()) + if r == nil { + return nil + } + returns = append(returns, &ast.Field{ + Type: r, + }) + } + return &ast.FuncType{ + Params: &ast.FieldList{ + List: params, + }, + Results: &ast.FieldList{ + List: returns, + }, + } + case *types.Named: + if t.Obj().Pkg() == nil { + return ast.NewIdent(t.Obj().Name()) + } + if t.Obj().Pkg() == pkg { + return ast.NewIdent(t.Obj().Name()) + } + pkgName := t.Obj().Pkg().Name() + + // If the file already imports the package under another name, use that. + for _, cand := range f.Imports { + if path, _ := strconv.Unquote(cand.Path.Value); path == t.Obj().Pkg().Path() { + if cand.Name != nil && cand.Name.Name != "" { + pkgName = cand.Name.Name + } + } + } + if pkgName == "." { + return ast.NewIdent(t.Obj().Name()) + } + return &ast.SelectorExpr{ + X: ast.NewIdent(pkgName), + Sel: ast.NewIdent(t.Obj().Name()), + } + case *types.Struct: + return ast.NewIdent(t.String()) + case *types.Interface: + return ast.NewIdent(t.String()) + default: + return nil + } +} + +// StmtToInsertVarBefore returns the ast.Stmt before which we can safely insert a new variable. +// Some examples: +// +// Basic Example: +// z := 1 +// y := z + x +// If x is undeclared, then this function would return `y := z + x`, so that we +// can insert `x := ` on the line before `y := z + x`. +// +// If stmt example: +// if z == 1 { +// } else if z == y {} +// If y is undeclared, then this function would return `if z == 1 {`, because we cannot +// insert a statement between an if and an else if statement. As a result, we need to find +// the top of the if chain to insert `y := ` before. +func StmtToInsertVarBefore(path []ast.Node) ast.Stmt { + enclosingIndex := -1 + for i, p := range path { + if _, ok := p.(ast.Stmt); ok { + enclosingIndex = i + break + } + } + if enclosingIndex == -1 { + return nil + } + enclosingStmt := path[enclosingIndex] + switch enclosingStmt.(type) { + case *ast.IfStmt: + // The enclosingStmt is inside of the if declaration, + // We need to check if we are in an else-if stmt and + // get the base if statement. + return baseIfStmt(path, enclosingIndex) + case *ast.CaseClause: + // Get the enclosing switch stmt if the enclosingStmt is + // inside of the case statement. + for i := enclosingIndex + 1; i < len(path); i++ { + if node, ok := path[i].(*ast.SwitchStmt); ok { + return node + } else if node, ok := path[i].(*ast.TypeSwitchStmt); ok { + return node + } + } + } + if len(path) <= enclosingIndex+1 { + return enclosingStmt.(ast.Stmt) + } + // Check if the enclosing statement is inside another node. + switch expr := path[enclosingIndex+1].(type) { + case *ast.IfStmt: + // Get the base if statement. + return baseIfStmt(path, enclosingIndex+1) + case *ast.ForStmt: + if expr.Init == enclosingStmt || expr.Post == enclosingStmt { + return expr + } + } + return enclosingStmt.(ast.Stmt) +} + +// baseIfStmt walks up the if/else-if chain until we get to +// the top of the current if chain. +func baseIfStmt(path []ast.Node, index int) ast.Stmt { + stmt := path[index] + for i := index + 1; i < len(path); i++ { + if node, ok := path[i].(*ast.IfStmt); ok && node.Else == stmt { + stmt = node + continue + } + break + } + return stmt.(ast.Stmt) +} + +// WalkASTWithParent walks the AST rooted at n. The semantics are +// similar to ast.Inspect except it does not call f(nil). +func WalkASTWithParent(n ast.Node, f func(n ast.Node, parent ast.Node) bool) { + var ancestors []ast.Node + ast.Inspect(n, func(n ast.Node) (recurse bool) { + if n == nil { + ancestors = ancestors[:len(ancestors)-1] + return false + } + + var parent ast.Node + if len(ancestors) > 0 { + parent = ancestors[len(ancestors)-1] + } + ancestors = append(ancestors, n) + return f(n, parent) + }) +} + +// MatchingIdents finds the names of all identifiers in 'node' that match any of the given types. +// 'pos' represents the position at which the identifiers may be inserted. 'pos' must be within +// the scope of each of identifier we select. Otherwise, we will insert a variable at 'pos' that +// is unrecognized. +func MatchingIdents(typs []types.Type, node ast.Node, pos token.Pos, info *types.Info, pkg *types.Package) map[types.Type][]string { + + // Initialize matches to contain the variable types we are searching for. + matches := make(map[types.Type][]string) + for _, typ := range typs { + if typ == nil { + continue // TODO(adonovan): is this reachable? + } + matches[typ] = nil // create entry + } + + seen := map[types.Object]struct{}{} + ast.Inspect(node, func(n ast.Node) bool { + if n == nil { + return false + } + // Prevent circular definitions. If 'pos' is within an assignment statement, do not + // allow any identifiers in that assignment statement to be selected. Otherwise, + // we could do the following, where 'x' satisfies the type of 'f0': + // + // x := fakeStruct{f0: x} + // + if assign, ok := n.(*ast.AssignStmt); ok && pos > assign.Pos() && pos <= assign.End() { + return false + } + if n.End() > pos { + return n.Pos() <= pos + } + ident, ok := n.(*ast.Ident) + if !ok || ident.Name == "_" { + return true + } + obj := info.Defs[ident] + if obj == nil || obj.Type() == nil { + return true + } + if _, ok := obj.(*types.TypeName); ok { + return true + } + // Prevent duplicates in matches' values. + if _, ok = seen[obj]; ok { + return true + } + seen[obj] = struct{}{} + // Find the scope for the given position. Then, check whether the object + // exists within the scope. + innerScope := pkg.Scope().Innermost(pos) + if innerScope == nil { + return true + } + _, foundObj := innerScope.LookupParent(ident.Name, pos) + if foundObj != obj { + return true + } + // The object must match one of the types that we are searching for. + // TODO(adonovan): opt: use typeutil.Map? + if names, ok := matches[obj.Type()]; ok { + matches[obj.Type()] = append(names, ident.Name) + } else { + // If the object type does not exactly match + // any of the target types, greedily find the first + // target type that the object type can satisfy. + for typ := range matches { + if equivalentTypes(obj.Type(), typ) { + matches[typ] = append(matches[typ], ident.Name) + } + } + } + return true + }) + return matches +} + +func equivalentTypes(want, got types.Type) bool { + if types.Identical(want, got) { + return true + } + // Code segment to help check for untyped equality from (golang/go#32146). + if rhs, ok := want.(*types.Basic); ok && rhs.Info()&types.IsUntyped > 0 { + if lhs, ok := got.Underlying().(*types.Basic); ok { + return rhs.Info()&types.IsConstType == lhs.Info()&types.IsConstType + } + } + return types.AssignableTo(want, got) +} diff --git a/src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/extractdoc.go b/src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/extractdoc.go new file mode 100644 index 0000000000..39507723d3 --- /dev/null +++ b/src/cmd/vendor/golang.org/x/tools/internal/analysisinternal/extractdoc.go @@ -0,0 +1,113 @@ +// Copyright 2023 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package analysisinternal + +import ( + "fmt" + "go/parser" + "go/token" + "strings" +) + +// MustExtractDoc is like [ExtractDoc] but it panics on error. +// +// To use, define a doc.go file such as: +// +// // Package halting defines an analyzer of program termination. +// // +// // # Analyzer halting +// // +// // halting: reports whether execution will halt. +// // +// // The halting analyzer reports a diagnostic for functions +// // that run forever. To suppress the diagnostics, try inserting +// // a 'break' statement into each loop. +// package halting +// +// import _ "embed" +// +// //go:embed doc.go +// var doc string +// +// And declare your analyzer as: +// +// var Analyzer = &analysis.Analyzer{ +// Name: "halting", +// Doc: analysisutil.MustExtractDoc(doc, "halting"), +// ... +// } +func MustExtractDoc(content, name string) string { + doc, err := ExtractDoc(content, name) + if err != nil { + panic(err) + } + return doc +} + +// ExtractDoc extracts a section of a package doc comment from the +// provided contents of an analyzer package's doc.go file. +// +// A section is a portion of the comment between one heading and +// the next, using this form: +// +// # Analyzer NAME +// +// NAME: SUMMARY +// +// Full description... +// +// where NAME matches the name argument, and SUMMARY is a brief +// verb-phrase that describes the analyzer. The following lines, up +// until the next heading or the end of the comment, contain the full +// description. ExtractDoc returns the portion following the colon, +// which is the form expected by Analyzer.Doc. +// +// Example: +// +// # Analyzer printf +// +// printf: checks consistency of calls to printf +// +// The printf analyzer checks consistency of calls to printf. +// Here is the complete description... +// +// This notation allows a single doc comment to provide documentation +// for multiple analyzers, each in its own section. +// The HTML anchors generated for each heading are predictable. +// +// It returns an error if the content was not a valid Go source file +// containing a package doc comment with a heading of the required +// form. +// +// This machinery enables the package documentation (typically +// accessible via the web at https://pkg.go.dev/) and the command +// documentation (typically printed to a terminal) to be derived from +// the same source and formatted appropriately. +func ExtractDoc(content, name string) (string, error) { + if content == "" { + return "", fmt.Errorf("empty Go source file") + } + fset := token.NewFileSet() + f, err := parser.ParseFile(fset, "", content, parser.ParseComments|parser.PackageClauseOnly) + if err != nil { + return "", fmt.Errorf("not a Go source file") + } + if f.Doc == nil { + return "", fmt.Errorf("Go source file has no package doc comment") + } + for _, section := range strings.Split(f.Doc.Text(), "\n# ") { + if body := strings.TrimPrefix(section, "Analyzer "+name); body != section && + body != "" && + body[0] == '\r' || body[0] == '\n' { + body = strings.TrimSpace(body) + rest := strings.TrimPrefix(body, name+":") + if rest == body { + return "", fmt.Errorf("'Analyzer %s' heading not followed by '%s: summary...' line", name, name) + } + return strings.TrimSpace(rest), nil + } + } + return "", fmt.Errorf("package doc comment contains no 'Analyzer %s' heading", name) +} diff --git a/src/cmd/vendor/golang.org/x/tools/internal/facts/imports.go b/src/cmd/vendor/golang.org/x/tools/internal/facts/imports.go index f64695ea52..1fe63ca6b5 100644 --- a/src/cmd/vendor/golang.org/x/tools/internal/facts/imports.go +++ b/src/cmd/vendor/golang.org/x/tools/internal/facts/imports.go @@ -6,8 +6,6 @@ package facts import ( "go/types" - - "golang.org/x/tools/internal/typeparams" ) // importMap computes the import map for a package by traversing the @@ -55,7 +53,7 @@ func importMap(imports []*types.Package) map[string]*types.Package { // infinite expansions: // type N[T any] struct { F *N[N[T]] } // importMap() is called on such types when Analyzer.RunDespiteErrors is true. - T = typeparams.NamedTypeOrigin(T) + T = T.Origin() if !typs[T] { typs[T] = true addObj(T.Obj()) @@ -63,12 +61,12 @@ func importMap(imports []*types.Package) map[string]*types.Package { for i := 0; i < T.NumMethods(); i++ { addObj(T.Method(i)) } - if tparams := typeparams.ForNamed(T); tparams != nil { + if tparams := T.TypeParams(); tparams != nil { for i := 0; i < tparams.Len(); i++ { addType(tparams.At(i)) } } - if targs := typeparams.NamedTypeArgs(T); targs != nil { + if targs := T.TypeArgs(); targs != nil { for i := 0; i < targs.Len(); i++ { addType(targs.At(i)) } @@ -88,7 +86,7 @@ func importMap(imports []*types.Package) map[string]*types.Package { case *types.Signature: addType(T.Params()) addType(T.Results()) - if tparams := typeparams.ForSignature(T); tparams != nil { + if tparams := T.TypeParams(); tparams != nil { for i := 0; i < tparams.Len(); i++ { addType(tparams.At(i)) } @@ -108,11 +106,11 @@ func importMap(imports []*types.Package) map[string]*types.Package { for i := 0; i < T.NumEmbeddeds(); i++ { addType(T.EmbeddedType(i)) // walk Embedded for implicits } - case *typeparams.Union: + case *types.Union: for i := 0; i < T.Len(); i++ { addType(T.Term(i).Type()) } - case *typeparams.TypeParam: + case *types.TypeParam: if !typs[T] { typs[T] = true addObj(T.Obj()) diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/common.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/common.go index d0d0649fe2..cdab988531 100644 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/common.go +++ b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/common.go @@ -42,7 +42,7 @@ func UnpackIndexExpr(n ast.Node) (x ast.Expr, lbrack token.Pos, indices []ast.Ex switch e := n.(type) { case *ast.IndexExpr: return e.X, e.Lbrack, []ast.Expr{e.Index}, e.Rbrack - case *IndexListExpr: + case *ast.IndexListExpr: return e.X, e.Lbrack, e.Indices, e.Rbrack } return nil, token.NoPos, nil, token.NoPos @@ -63,7 +63,7 @@ func PackIndexExpr(x ast.Expr, lbrack token.Pos, indices []ast.Expr, rbrack toke Rbrack: rbrack, } default: - return &IndexListExpr{ + return &ast.IndexListExpr{ X: x, Lbrack: lbrack, Indices: indices, @@ -74,7 +74,7 @@ func PackIndexExpr(x ast.Expr, lbrack token.Pos, indices []ast.Expr, rbrack toke // IsTypeParam reports whether t is a type parameter. func IsTypeParam(t types.Type) bool { - _, ok := t.(*TypeParam) + _, ok := t.(*types.TypeParam) return ok } @@ -100,11 +100,11 @@ func OriginMethod(fn *types.Func) *types.Func { // Receiver is a *types.Interface. return fn } - if ForNamed(named).Len() == 0 { + if named.TypeParams().Len() == 0 { // Receiver base has no type parameters, so we can avoid the lookup below. return fn } - orig := NamedTypeOrigin(named) + orig := named.Origin() gfn, _, _ := types.LookupFieldOrMethod(orig, true, fn.Pkg(), fn.Name()) // This is a fix for a gopls crash (#60628) due to a go/types bug (#60634). In: @@ -157,7 +157,7 @@ func OriginMethod(fn *types.Func) *types.Func { // // In this case, GenericAssignableTo reports that instantiations of Container // are assignable to the corresponding instantiation of Interface. -func GenericAssignableTo(ctxt *Context, V, T types.Type) bool { +func GenericAssignableTo(ctxt *types.Context, V, T types.Type) bool { // If V and T are not both named, or do not have matching non-empty type // parameter lists, fall back on types.AssignableTo. @@ -167,9 +167,9 @@ func GenericAssignableTo(ctxt *Context, V, T types.Type) bool { return types.AssignableTo(V, T) } - vtparams := ForNamed(VN) - ttparams := ForNamed(TN) - if vtparams.Len() == 0 || vtparams.Len() != ttparams.Len() || NamedTypeArgs(VN).Len() != 0 || NamedTypeArgs(TN).Len() != 0 { + vtparams := VN.TypeParams() + ttparams := TN.TypeParams() + if vtparams.Len() == 0 || vtparams.Len() != ttparams.Len() || VN.TypeArgs().Len() != 0 || TN.TypeArgs().Len() != 0 { return types.AssignableTo(V, T) } @@ -182,7 +182,7 @@ func GenericAssignableTo(ctxt *Context, V, T types.Type) bool { // Minor optimization: ensure we share a context across the two // instantiations below. if ctxt == nil { - ctxt = NewContext() + ctxt = types.NewContext() } var targs []types.Type @@ -190,12 +190,12 @@ func GenericAssignableTo(ctxt *Context, V, T types.Type) bool { targs = append(targs, vtparams.At(i)) } - vinst, err := Instantiate(ctxt, V, targs, true) + vinst, err := types.Instantiate(ctxt, V, targs, true) if err != nil { panic("type parameters should satisfy their own constraints") } - tinst, err := Instantiate(ctxt, T, targs, true) + tinst, err := types.Instantiate(ctxt, T, targs, true) if err != nil { return false } diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/coretype.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/coretype.go index 71248209ee..7ea8840eab 100644 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/coretype.go +++ b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/coretype.go @@ -108,15 +108,15 @@ func CoreType(T types.Type) types.Type { // // _NormalTerms makes no guarantees about the order of terms, except that it // is deterministic. -func _NormalTerms(typ types.Type) ([]*Term, error) { +func _NormalTerms(typ types.Type) ([]*types.Term, error) { switch typ := typ.(type) { - case *TypeParam: + case *types.TypeParam: return StructuralTerms(typ) - case *Union: + case *types.Union: return UnionTermSet(typ) case *types.Interface: return InterfaceTermSet(typ) default: - return []*Term{NewTerm(false, typ)}, nil + return []*types.Term{types.NewTerm(false, typ)}, nil } } diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go117.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go117.go deleted file mode 100644 index 18212390e1..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go117.go +++ /dev/null @@ -1,12 +0,0 @@ -// Copyright 2021 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build !go1.18 -// +build !go1.18 - -package typeparams - -// Enabled reports whether type parameters are enabled in the current build -// environment. -const Enabled = false diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go118.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go118.go deleted file mode 100644 index d67148823c..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/enabled_go118.go +++ /dev/null @@ -1,15 +0,0 @@ -// Copyright 2021 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build go1.18 -// +build go1.18 - -package typeparams - -// Note: this constant is in a separate file as this is the only acceptable -// diff between the <1.18 API of this package and the 1.18 API. - -// Enabled reports whether type parameters are enabled in the current build -// environment. -const Enabled = true diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/normalize.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/normalize.go index 9c631b6512..93c80fdc96 100644 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/normalize.go +++ b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/normalize.go @@ -60,7 +60,7 @@ var ErrEmptyTypeSet = errors.New("empty type set") // // StructuralTerms makes no guarantees about the order of terms, except that it // is deterministic. -func StructuralTerms(tparam *TypeParam) ([]*Term, error) { +func StructuralTerms(tparam *types.TypeParam) ([]*types.Term, error) { constraint := tparam.Constraint() if constraint == nil { return nil, fmt.Errorf("%s has nil constraint", tparam) @@ -78,7 +78,7 @@ func StructuralTerms(tparam *TypeParam) ([]*Term, error) { // // See the documentation of StructuralTerms for more information on // normalization. -func InterfaceTermSet(iface *types.Interface) ([]*Term, error) { +func InterfaceTermSet(iface *types.Interface) ([]*types.Term, error) { return computeTermSet(iface) } @@ -88,11 +88,11 @@ func InterfaceTermSet(iface *types.Interface) ([]*Term, error) { // // See the documentation of StructuralTerms for more information on // normalization. -func UnionTermSet(union *Union) ([]*Term, error) { +func UnionTermSet(union *types.Union) ([]*types.Term, error) { return computeTermSet(union) } -func computeTermSet(typ types.Type) ([]*Term, error) { +func computeTermSet(typ types.Type) ([]*types.Term, error) { tset, err := computeTermSetInternal(typ, make(map[types.Type]*termSet), 0) if err != nil { return nil, err @@ -103,9 +103,9 @@ func computeTermSet(typ types.Type) ([]*Term, error) { if tset.terms.isAll() { return nil, nil } - var terms []*Term + var terms []*types.Term for _, term := range tset.terms { - terms = append(terms, NewTerm(term.tilde, term.typ)) + terms = append(terms, types.NewTerm(term.tilde, term.typ)) } return terms, nil } @@ -162,7 +162,7 @@ func computeTermSetInternal(t types.Type, seen map[types.Type]*termSet, depth in tset.terms = allTermlist for i := 0; i < u.NumEmbeddeds(); i++ { embedded := u.EmbeddedType(i) - if _, ok := embedded.Underlying().(*TypeParam); ok { + if _, ok := embedded.Underlying().(*types.TypeParam); ok { return nil, fmt.Errorf("invalid embedded type %T", embedded) } tset2, err := computeTermSetInternal(embedded, seen, depth+1) @@ -171,7 +171,7 @@ func computeTermSetInternal(t types.Type, seen map[types.Type]*termSet, depth in } tset.terms = tset.terms.intersect(tset2.terms) } - case *Union: + case *types.Union: // The term set of a union is the union of term sets of its terms. tset.terms = nil for i := 0; i < u.Len(); i++ { @@ -184,7 +184,7 @@ func computeTermSetInternal(t types.Type, seen map[types.Type]*termSet, depth in return nil, err } terms = tset2.terms - case *TypeParam, *Union: + case *types.TypeParam, *types.Union: // A stand-alone type parameter or union is not permitted as union // term. return nil, fmt.Errorf("invalid union term %T", t) @@ -199,7 +199,7 @@ func computeTermSetInternal(t types.Type, seen map[types.Type]*termSet, depth in return nil, fmt.Errorf("exceeded max term count %d", maxTermCount) } } - case *TypeParam: + case *types.TypeParam: panic("unreachable") default: // For all other types, the term set is just a single non-tilde term diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go117.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go117.go deleted file mode 100644 index 7ed86e1711..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go117.go +++ /dev/null @@ -1,197 +0,0 @@ -// Copyright 2021 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build !go1.18 -// +build !go1.18 - -package typeparams - -import ( - "go/ast" - "go/token" - "go/types" -) - -func unsupported() { - panic("type parameters are unsupported at this go version") -} - -// IndexListExpr is a placeholder type, as type parameters are not supported at -// this Go version. Its methods panic on use. -type IndexListExpr struct { - ast.Expr - X ast.Expr // expression - Lbrack token.Pos // position of "[" - Indices []ast.Expr // index expressions - Rbrack token.Pos // position of "]" -} - -// ForTypeSpec returns an empty field list, as type parameters on not supported -// at this Go version. -func ForTypeSpec(*ast.TypeSpec) *ast.FieldList { - return nil -} - -// ForFuncType returns an empty field list, as type parameters are not -// supported at this Go version. -func ForFuncType(*ast.FuncType) *ast.FieldList { - return nil -} - -// TypeParam is a placeholder type, as type parameters are not supported at -// this Go version. Its methods panic on use. -type TypeParam struct{ types.Type } - -func (*TypeParam) Index() int { unsupported(); return 0 } -func (*TypeParam) Constraint() types.Type { unsupported(); return nil } -func (*TypeParam) Obj() *types.TypeName { unsupported(); return nil } - -// TypeParamList is a placeholder for an empty type parameter list. -type TypeParamList struct{} - -func (*TypeParamList) Len() int { return 0 } -func (*TypeParamList) At(int) *TypeParam { unsupported(); return nil } - -// TypeList is a placeholder for an empty type list. -type TypeList struct{} - -func (*TypeList) Len() int { return 0 } -func (*TypeList) At(int) types.Type { unsupported(); return nil } - -// NewTypeParam is unsupported at this Go version, and panics. -func NewTypeParam(name *types.TypeName, constraint types.Type) *TypeParam { - unsupported() - return nil -} - -// SetTypeParamConstraint is unsupported at this Go version, and panics. -func SetTypeParamConstraint(tparam *TypeParam, constraint types.Type) { - unsupported() -} - -// NewSignatureType calls types.NewSignature, panicking if recvTypeParams or -// typeParams is non-empty. -func NewSignatureType(recv *types.Var, recvTypeParams, typeParams []*TypeParam, params, results *types.Tuple, variadic bool) *types.Signature { - if len(recvTypeParams) != 0 || len(typeParams) != 0 { - panic("signatures cannot have type parameters at this Go version") - } - return types.NewSignature(recv, params, results, variadic) -} - -// ForSignature returns an empty slice. -func ForSignature(*types.Signature) *TypeParamList { - return nil -} - -// RecvTypeParams returns a nil slice. -func RecvTypeParams(sig *types.Signature) *TypeParamList { - return nil -} - -// IsComparable returns false, as no interfaces are type-restricted at this Go -// version. -func IsComparable(*types.Interface) bool { - return false -} - -// IsMethodSet returns true, as no interfaces are type-restricted at this Go -// version. -func IsMethodSet(*types.Interface) bool { - return true -} - -// IsImplicit returns false, as no interfaces are implicit at this Go version. -func IsImplicit(*types.Interface) bool { - return false -} - -// MarkImplicit does nothing, because this Go version does not have implicit -// interfaces. -func MarkImplicit(*types.Interface) {} - -// ForNamed returns an empty type parameter list, as type parameters are not -// supported at this Go version. -func ForNamed(*types.Named) *TypeParamList { - return nil -} - -// SetForNamed panics if tparams is non-empty. -func SetForNamed(_ *types.Named, tparams []*TypeParam) { - if len(tparams) > 0 { - unsupported() - } -} - -// NamedTypeArgs returns nil. -func NamedTypeArgs(*types.Named) *TypeList { - return nil -} - -// NamedTypeOrigin is the identity method at this Go version. -func NamedTypeOrigin(named *types.Named) *types.Named { - return named -} - -// Term holds information about a structural type restriction. -type Term struct { - tilde bool - typ types.Type -} - -func (m *Term) Tilde() bool { return m.tilde } -func (m *Term) Type() types.Type { return m.typ } -func (m *Term) String() string { - pre := "" - if m.tilde { - pre = "~" - } - return pre + m.typ.String() -} - -// NewTerm is unsupported at this Go version, and panics. -func NewTerm(tilde bool, typ types.Type) *Term { - return &Term{tilde, typ} -} - -// Union is a placeholder type, as type parameters are not supported at this Go -// version. Its methods panic on use. -type Union struct{ types.Type } - -func (*Union) Len() int { return 0 } -func (*Union) Term(i int) *Term { unsupported(); return nil } - -// NewUnion is unsupported at this Go version, and panics. -func NewUnion(terms []*Term) *Union { - unsupported() - return nil -} - -// InitInstanceInfo is a noop at this Go version. -func InitInstanceInfo(*types.Info) {} - -// Instance is a placeholder type, as type parameters are not supported at this -// Go version. -type Instance struct { - TypeArgs *TypeList - Type types.Type -} - -// GetInstances returns a nil map, as type parameters are not supported at this -// Go version. -func GetInstances(info *types.Info) map[*ast.Ident]Instance { return nil } - -// Context is a placeholder type, as type parameters are not supported at -// this Go version. -type Context struct{} - -// NewContext returns a placeholder Context instance. -func NewContext() *Context { - return &Context{} -} - -// Instantiate is unsupported on this Go version, and panics. -func Instantiate(ctxt *Context, typ types.Type, targs []types.Type, validate bool) (types.Type, error) { - unsupported() - return nil, nil -} diff --git a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go118.go b/src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go118.go deleted file mode 100644 index cf301af1db..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/internal/typeparams/typeparams_go118.go +++ /dev/null @@ -1,151 +0,0 @@ -// Copyright 2021 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build go1.18 -// +build go1.18 - -package typeparams - -import ( - "go/ast" - "go/types" -) - -// IndexListExpr is an alias for ast.IndexListExpr. -type IndexListExpr = ast.IndexListExpr - -// ForTypeSpec returns n.TypeParams. -func ForTypeSpec(n *ast.TypeSpec) *ast.FieldList { - if n == nil { - return nil - } - return n.TypeParams -} - -// ForFuncType returns n.TypeParams. -func ForFuncType(n *ast.FuncType) *ast.FieldList { - if n == nil { - return nil - } - return n.TypeParams -} - -// TypeParam is an alias for types.TypeParam -type TypeParam = types.TypeParam - -// TypeParamList is an alias for types.TypeParamList -type TypeParamList = types.TypeParamList - -// TypeList is an alias for types.TypeList -type TypeList = types.TypeList - -// NewTypeParam calls types.NewTypeParam. -func NewTypeParam(name *types.TypeName, constraint types.Type) *TypeParam { - return types.NewTypeParam(name, constraint) -} - -// SetTypeParamConstraint calls tparam.SetConstraint(constraint). -func SetTypeParamConstraint(tparam *TypeParam, constraint types.Type) { - tparam.SetConstraint(constraint) -} - -// NewSignatureType calls types.NewSignatureType. -func NewSignatureType(recv *types.Var, recvTypeParams, typeParams []*TypeParam, params, results *types.Tuple, variadic bool) *types.Signature { - return types.NewSignatureType(recv, recvTypeParams, typeParams, params, results, variadic) -} - -// ForSignature returns sig.TypeParams() -func ForSignature(sig *types.Signature) *TypeParamList { - return sig.TypeParams() -} - -// RecvTypeParams returns sig.RecvTypeParams(). -func RecvTypeParams(sig *types.Signature) *TypeParamList { - return sig.RecvTypeParams() -} - -// IsComparable calls iface.IsComparable(). -func IsComparable(iface *types.Interface) bool { - return iface.IsComparable() -} - -// IsMethodSet calls iface.IsMethodSet(). -func IsMethodSet(iface *types.Interface) bool { - return iface.IsMethodSet() -} - -// IsImplicit calls iface.IsImplicit(). -func IsImplicit(iface *types.Interface) bool { - return iface.IsImplicit() -} - -// MarkImplicit calls iface.MarkImplicit(). -func MarkImplicit(iface *types.Interface) { - iface.MarkImplicit() -} - -// ForNamed extracts the (possibly empty) type parameter object list from -// named. -func ForNamed(named *types.Named) *TypeParamList { - return named.TypeParams() -} - -// SetForNamed sets the type params tparams on n. Each tparam must be of -// dynamic type *types.TypeParam. -func SetForNamed(n *types.Named, tparams []*TypeParam) { - n.SetTypeParams(tparams) -} - -// NamedTypeArgs returns named.TypeArgs(). -func NamedTypeArgs(named *types.Named) *TypeList { - return named.TypeArgs() -} - -// NamedTypeOrigin returns named.Orig(). -func NamedTypeOrigin(named *types.Named) *types.Named { - return named.Origin() -} - -// Term is an alias for types.Term. -type Term = types.Term - -// NewTerm calls types.NewTerm. -func NewTerm(tilde bool, typ types.Type) *Term { - return types.NewTerm(tilde, typ) -} - -// Union is an alias for types.Union -type Union = types.Union - -// NewUnion calls types.NewUnion. -func NewUnion(terms []*Term) *Union { - return types.NewUnion(terms) -} - -// InitInstanceInfo initializes info to record information about type and -// function instances. -func InitInstanceInfo(info *types.Info) { - info.Instances = make(map[*ast.Ident]types.Instance) -} - -// Instance is an alias for types.Instance. -type Instance = types.Instance - -// GetInstances returns info.Instances. -func GetInstances(info *types.Info) map[*ast.Ident]Instance { - return info.Instances -} - -// Context is an alias for types.Context. -type Context = types.Context - -// NewContext calls types.NewContext. -func NewContext() *Context { - return types.NewContext() -} - -// Instantiate calls types.Instantiate. -func Instantiate(ctxt *Context, typ types.Type, targs []types.Type, validate bool) (types.Type, error) { - return types.Instantiate(ctxt, typ, targs, validate) -} diff --git a/src/cmd/vendor/golang.org/x/tools/internal/versions/versions.go b/src/cmd/vendor/golang.org/x/tools/internal/versions/versions.go new file mode 100644 index 0000000000..e16f6c33a5 --- /dev/null +++ b/src/cmd/vendor/golang.org/x/tools/internal/versions/versions.go @@ -0,0 +1,52 @@ +// Copyright 2023 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package versions + +// Note: If we use build tags to use go/versions when go >=1.22, +// we run into go.dev/issue/53737. Under some operations users would see an +// import of "go/versions" even if they would not compile the file. +// For example, during `go get -u ./...` (go.dev/issue/64490) we do not try to include +// For this reason, this library just a clone of go/versions for the moment. + +// Lang returns the Go language version for version x. +// If x is not a valid version, Lang returns the empty string. +// For example: +// +// Lang("go1.21rc2") = "go1.21" +// Lang("go1.21.2") = "go1.21" +// Lang("go1.21") = "go1.21" +// Lang("go1") = "go1" +// Lang("bad") = "" +// Lang("1.21") = "" +func Lang(x string) string { + v := lang(stripGo(x)) + if v == "" { + return "" + } + return x[:2+len(v)] // "go"+v without allocation +} + +// Compare returns -1, 0, or +1 depending on whether +// x < y, x == y, or x > y, interpreted as Go versions. +// The versions x and y must begin with a "go" prefix: "go1.21" not "1.21". +// Invalid versions, including the empty string, compare less than +// valid versions and equal to each other. +// The language version "go1.21" compares less than the +// release candidate and eventual releases "go1.21rc1" and "go1.21.0". +// Custom toolchain suffixes are ignored during comparison: +// "go1.21.0" and "go1.21.0-bigcorp" are equal. +func Compare(x, y string) int { return compare(stripGo(x), stripGo(y)) } + +// IsValid reports whether the version x is valid. +func IsValid(x string) bool { return isValid(stripGo(x)) } + +// stripGo converts from a "go1.21" version to a "1.21" version. +// If v does not start with "go", stripGo returns the empty string (a known invalid version). +func stripGo(v string) string { + if len(v) < 2 || v[:2] != "go" { + return "" + } + return v[2:] +} diff --git a/src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go121.go b/src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go121.go deleted file mode 100644 index cf4a7d0360..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go121.go +++ /dev/null @@ -1,49 +0,0 @@ -// Copyright 2023 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build !go1.22 -// +build !go1.22 - -package versions - -// Lang returns the Go language version for version x. -// If x is not a valid version, Lang returns the empty string. -// For example: -// -// Lang("go1.21rc2") = "go1.21" -// Lang("go1.21.2") = "go1.21" -// Lang("go1.21") = "go1.21" -// Lang("go1") = "go1" -// Lang("bad") = "" -// Lang("1.21") = "" -func Lang(x string) string { - v := lang(stripGo(x)) - if v == "" { - return "" - } - return x[:2+len(v)] // "go"+v without allocation -} - -// Compare returns -1, 0, or +1 depending on whether -// x < y, x == y, or x > y, interpreted as Go versions. -// The versions x and y must begin with a "go" prefix: "go1.21" not "1.21". -// Invalid versions, including the empty string, compare less than -// valid versions and equal to each other. -// The language version "go1.21" compares less than the -// release candidate and eventual releases "go1.21rc1" and "go1.21.0". -// Custom toolchain suffixes are ignored during comparison: -// "go1.21.0" and "go1.21.0-bigcorp" are equal. -func Compare(x, y string) int { return compare(stripGo(x), stripGo(y)) } - -// IsValid reports whether the version x is valid. -func IsValid(x string) bool { return isValid(stripGo(x)) } - -// stripGo converts from a "go1.21" version to a "1.21" version. -// If v does not start with "go", stripGo returns the empty string (a known invalid version). -func stripGo(v string) string { - if len(v) < 2 || v[:2] != "go" { - return "" - } - return v[2:] -} diff --git a/src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go122.go b/src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go122.go deleted file mode 100644 index c1c1814b28..0000000000 --- a/src/cmd/vendor/golang.org/x/tools/internal/versions/versions_go122.go +++ /dev/null @@ -1,38 +0,0 @@ -// Copyright 2023 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build go1.22 -// +build go1.22 - -package versions - -import ( - "go/version" -) - -// Lang returns the Go language version for version x. -// If x is not a valid version, Lang returns the empty string. -// For example: -// -// Lang("go1.21rc2") = "go1.21" -// Lang("go1.21.2") = "go1.21" -// Lang("go1.21") = "go1.21" -// Lang("go1") = "go1" -// Lang("bad") = "" -// Lang("1.21") = "" -func Lang(x string) string { return version.Lang(x) } - -// Compare returns -1, 0, or +1 depending on whether -// x < y, x == y, or x > y, interpreted as Go versions. -// The versions x and y must begin with a "go" prefix: "go1.21" not "1.21". -// Invalid versions, including the empty string, compare less than -// valid versions and equal to each other. -// The language version "go1.21" compares less than the -// release candidate and eventual releases "go1.21rc1" and "go1.21.0". -// Custom toolchain suffixes are ignored during comparison: -// "go1.21.0" and "go1.21.0-bigcorp" are equal. -func Compare(x, y string) int { return version.Compare(x, y) } - -// IsValid reports whether the version x is valid. -func IsValid(x string) bool { return version.IsValid(x) } diff --git a/src/cmd/vendor/modules.txt b/src/cmd/vendor/modules.txt index fa05367a8a..d2caf1ffb0 100644 --- a/src/cmd/vendor/modules.txt +++ b/src/cmd/vendor/modules.txt @@ -45,7 +45,7 @@ golang.org/x/sys/windows # golang.org/x/term v0.15.0 ## explicit; go 1.18 golang.org/x/term -# golang.org/x/tools v0.16.1-0.20231129202111-1b1e4dafd51a +# golang.org/x/tools v0.16.2-0.20231218185909-83bceaf2424d ## explicit; go 1.18 golang.org/x/tools/cmd/bisect golang.org/x/tools/cover @@ -92,6 +92,7 @@ golang.org/x/tools/go/ast/inspector golang.org/x/tools/go/cfg golang.org/x/tools/go/types/objectpath golang.org/x/tools/go/types/typeutil +golang.org/x/tools/internal/analysisinternal golang.org/x/tools/internal/bisect golang.org/x/tools/internal/facts golang.org/x/tools/internal/typeparams -- cgit v1.3 From c564d4ae0876895b2a9bd941f1f90022e8318f29 Mon Sep 17 00:00:00 2001 From: Than McIntosh Date: Mon, 18 Dec 2023 21:11:13 +0000 Subject: Revert "cmd/cgo/internal/testsanitizers: fix msan test failing with clang >= 16" This reverts commit https://go.dev/cl/c/go/+/549297 Reason for revert: breaks clang builder Change-Id: I2321dec9bc1fc20dfafa8a984303b0b5710f8aac Reviewed-on: https://go-review.googlesource.com/c/go/+/550779 Auto-Submit: Than McIntosh LUCI-TryBot-Result: Go LUCI Reviewed-by: Mauri de Souza Meneguzzo Reviewed-by: Matthew Dempsky --- src/cmd/cgo/internal/testsanitizers/msan_test.go | 5 +---- src/cmd/cgo/internal/testsanitizers/testdata/msan8.go | 7 ------- 2 files changed, 1 insertion(+), 11 deletions(-) diff --git a/src/cmd/cgo/internal/testsanitizers/msan_test.go b/src/cmd/cgo/internal/testsanitizers/msan_test.go index c534b72442..83d66f6660 100644 --- a/src/cmd/cgo/internal/testsanitizers/msan_test.go +++ b/src/cmd/cgo/internal/testsanitizers/msan_test.go @@ -71,10 +71,7 @@ func TestMSAN(t *testing.T) { defer dir.RemoveAll(t) outPath := dir.Join(name) - buildcmd := config.goCmdWithExperiments("build", []string{"-o", outPath, srcPath(tc.src)}, tc.experiments) - // allow tests to define -f flags in CGO_CFLAGS - replaceEnv(buildcmd, "CGO_CFLAGS_ALLOW", "-f.*") - mustRun(t, buildcmd) + mustRun(t, config.goCmdWithExperiments("build", []string{"-o", outPath, srcPath(tc.src)}, tc.experiments)) cmd := hangProneCmd(outPath) if tc.wantErr { diff --git a/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go b/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go index e79d343cc7..1cb5c5677f 100644 --- a/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go +++ b/src/cmd/cgo/internal/testsanitizers/testdata/msan8.go @@ -5,13 +5,6 @@ package main /* -// For clang >= 16, uninitialized memory is more aggressively reported. -// Restore the old behavior for this particular test as it relies on -// uninitialized variables. See #64616 -#if __clang_major__ >= 16 -#cgo CFLAGS: -fno-sanitize-memory-param-retval -#endif - #include #include #include -- cgit v1.3 From 7383b2a4db5dc93c9b875b42d5add73d27cc4b9f Mon Sep 17 00:00:00 2001 From: Filippo Valsorda Date: Thu, 14 Dec 2023 17:07:19 +0100 Subject: crypto/internal/boring: upgrade module to fips-20220613 Also, add EVP_aead_aes_*_gcm_tls13 to the build, which we will need in a following CL, to avoid rebuilding the syso twice. Updates #64717 Updates #62372 Change-Id: Ie4d853ad9b914c1095cad60694a1ae6f77dc22ce Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-boringcrypto Reviewed-on: https://go-review.googlesource.com/c/go/+/549695 Reviewed-by: Than McIntosh LUCI-TryBot-Result: Go LUCI Reviewed-by: Roland Shoemaker --- src/crypto/internal/boring/Dockerfile | 28 +++++++++++++-------- src/crypto/internal/boring/LICENSE | 2 +- src/crypto/internal/boring/README.md | 7 +++--- src/crypto/internal/boring/build-goboring.sh | 2 +- src/crypto/internal/boring/build.sh | 6 +++++ src/crypto/internal/boring/goboringcrypto.h | 2 ++ .../boring/syso/goboringcrypto_linux_amd64.syso | Bin 2555664 -> 2427120 bytes .../boring/syso/goboringcrypto_linux_arm64.syso | Bin 1980296 -> 2105896 bytes 8 files changed, 31 insertions(+), 16 deletions(-) diff --git a/src/crypto/internal/boring/Dockerfile b/src/crypto/internal/boring/Dockerfile index 58eb028e8a..8fde5c0018 100644 --- a/src/crypto/internal/boring/Dockerfile +++ b/src/crypto/internal/boring/Dockerfile @@ -13,15 +13,21 @@ WORKDIR /boring ENV LANG=C ENV LANGUAGE= -# Following NIST submission draft dated July 3, 2021. -# This corresponds to boringssl.googlesource.com/boringssl tag fips-20210429. -ENV ClangV=12 +# Following NIST submission draft for In Progress module validation. +# This corresponds to boringssl.googlesource.com/boringssl tag fips-20220613. RUN apt-get update && \ - apt-get install --no-install-recommends -y cmake xz-utils wget unzip ca-certificates clang-$ClangV python + apt-get install --no-install-recommends -y cmake xz-utils wget unzip ca-certificates python lsb-release software-properties-common gnupg + +# Install Clang. +ENV ClangV=14 +RUN \ + wget https://apt.llvm.org/llvm.sh && \ + chmod +x llvm.sh && \ + ./llvm.sh $ClangV # Download, validate, unpack, build, and install Ninja. -ENV NinjaV=1.10.2 -ENV NinjaH=ce35865411f0490368a8fc383f29071de6690cbadc27704734978221f25e2bed +ENV NinjaV=1.10.1 +ENV NinjaH=a6b6f7ac360d4aabd54e299cc1d8fa7b234cd81b9401693da21221c62569a23e RUN \ wget https://github.com/ninja-build/ninja/archive/refs/tags/v$NinjaV.tar.gz && \ echo "$NinjaH v$NinjaV.tar.gz" >sha && sha256sum -c sha && \ @@ -33,9 +39,9 @@ RUN \ # Download, validate, unpack, and install Go. ARG GOARCH -ENV GoV=1.16.5 -ENV GoHamd64=b12c23023b68de22f74c0524f10b753e7b08b1504cb7e417eccebdd3fae49061 -ENV GoHarm64=d5446b46ef6f36fdffa852f73dfbbe78c1ddf010b99fa4964944b9ae8b4d6799 +ENV GoV=1.18.1 +ENV GoHamd64=b3b815f47ababac13810fc6021eb73d65478e0b2db4b09d348eefad9581a2334 +ENV GoHarm64=56a91851c97fb4697077abbca38860f735c32b38993ff79b088dac46e4735633 RUN \ eval GoH=\${GoH$GOARCH} && \ wget https://golang.org/dl/go$GoV.linux-$GOARCH.tar.gz && \ @@ -45,8 +51,8 @@ RUN \ ln -s /usr/local/go/bin/go /usr/local/bin/ # Download, validate, and unpack BoringCrypto. -ENV BoringV=853ca1ea1168dff08011e5d42d94609cc0ca2e27 -ENV BoringH=a4d069ccef6f3c7bc0c68de82b91414f05cb817494cd1ab483dcf3368883c7c2 +ENV BoringV=0c6f40132b828e92ba365c6b7680e32820c63fa7 +ENV BoringH=62f733289f2d677c2723f556aa58034c438f3a7bbca6c12b156538a88e38da8a RUN \ wget https://commondatastorage.googleapis.com/chromium-boringssl-fips/boringssl-$BoringV.tar.xz && \ echo "$BoringH boringssl-$BoringV.tar.xz" >sha && sha256sum -c sha && \ diff --git a/src/crypto/internal/boring/LICENSE b/src/crypto/internal/boring/LICENSE index 38990bdb77..05b0963f5e 100644 --- a/src/crypto/internal/boring/LICENSE +++ b/src/crypto/internal/boring/LICENSE @@ -6,7 +6,7 @@ When building with GOEXPERIMENT=boringcrypto, the following applies. The goboringcrypto_linux_amd64.syso object file is built from BoringSSL source code by build/build.sh and is covered by the BoringSSL license reproduced below and also at -https://boringssl.googlesource.com/boringssl/+/fips-20190808/LICENSE. +https://boringssl.googlesource.com/boringssl/+/fips-20220613/LICENSE. BoringSSL is a fork of OpenSSL. As such, large parts of it fall under OpenSSL licensing. Files that are completely new have a Google copyright and an ISC diff --git a/src/crypto/internal/boring/README.md b/src/crypto/internal/boring/README.md index ec02786d96..62106cdc70 100644 --- a/src/crypto/internal/boring/README.md +++ b/src/crypto/internal/boring/README.md @@ -27,13 +27,14 @@ syso/goboringcrypto_linux_arm64.syso is built with: GOARCH=arm64 ./build.sh -Both run on an x86 Debian Linux system using Docker. +Both run using Docker. + For the arm64 build to run on an x86 system, you need apt-get install qemu-user-static qemu-binfmt-support to allow the x86 kernel to run arm64 binaries via QEMU. -See build.sh for more details about the build. - +For the amd64 build to run on an Apple Silicon macOS, you need Rosetta 2. +See build.sh for more details about the build. diff --git a/src/crypto/internal/boring/build-goboring.sh b/src/crypto/internal/boring/build-goboring.sh index 4938b5eac3..c43fad24e8 100755 --- a/src/crypto/internal/boring/build-goboring.sh +++ b/src/crypto/internal/boring/build-goboring.sh @@ -122,7 +122,7 @@ awk -f boringx.awk goboringcrypto.h # writes goboringcrypto.x awk -f boringh.awk goboringcrypto.h # writes goboringcrypto[01].h ls -l ../boringssl/include -clang++ -std=c++11 -fPIC -I../boringssl/include -O2 -o a.out goboringcrypto.cc +clang++ -fPIC -I../boringssl/include -O2 -o a.out goboringcrypto.cc ./a.out || exit 2 # clang implements u128 % u128 -> u128 by calling __umodti3, diff --git a/src/crypto/internal/boring/build.sh b/src/crypto/internal/boring/build.sh index ec960d729d..e2026018a3 100755 --- a/src/crypto/internal/boring/build.sh +++ b/src/crypto/internal/boring/build.sh @@ -22,6 +22,12 @@ platform="" buildargs="" case "$GOARCH" in amd64) + if ! docker run --rm -t amd64/ubuntu:focal uname -m >/dev/null 2>&1; then + echo "# Docker cannot run amd64 binaries." + exit 1 + fi + platform="--platform linux/amd64" + buildargs="--build-arg ubuntu=amd64/ubuntu" ;; arm64) if ! docker run --rm -t arm64v8/ubuntu:focal uname -m >/dev/null 2>&1; then diff --git a/src/crypto/internal/boring/goboringcrypto.h b/src/crypto/internal/boring/goboringcrypto.h index 2b11049728..3663a1b1c3 100644 --- a/src/crypto/internal/boring/goboringcrypto.h +++ b/src/crypto/internal/boring/goboringcrypto.h @@ -125,7 +125,9 @@ void _goboringcrypto_EVP_AEAD_CTX_cleanup(GO_EVP_AEAD_CTX*); int _goboringcrypto_EVP_AEAD_CTX_seal(const GO_EVP_AEAD_CTX*, uint8_t*, size_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t); int _goboringcrypto_EVP_AEAD_CTX_open(const GO_EVP_AEAD_CTX*, uint8_t*, size_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t); const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_128_gcm_tls12(void); +const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_128_gcm_tls13(void); const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_256_gcm_tls12(void); +const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_256_gcm_tls13(void); enum go_evp_aead_direction_t { go_evp_aead_open = 0, go_evp_aead_seal = 1 diff --git a/src/crypto/internal/boring/syso/goboringcrypto_linux_amd64.syso b/src/crypto/internal/boring/syso/goboringcrypto_linux_amd64.syso index 6cea789355..b99e7f5766 100644 Binary files a/src/crypto/internal/boring/syso/goboringcrypto_linux_amd64.syso and b/src/crypto/internal/boring/syso/goboringcrypto_linux_amd64.syso differ diff --git a/src/crypto/internal/boring/syso/goboringcrypto_linux_arm64.syso b/src/crypto/internal/boring/syso/goboringcrypto_linux_arm64.syso index 9659aa1a5e..143a47a0aa 100644 Binary files a/src/crypto/internal/boring/syso/goboringcrypto_linux_arm64.syso and b/src/crypto/internal/boring/syso/goboringcrypto_linux_arm64.syso differ -- cgit v1.3 From 4106de901a8efe914cda6f6c4e8d45ff8c115da4 Mon Sep 17 00:00:00 2001 From: Filippo Valsorda Date: Thu, 14 Dec 2023 22:13:29 +0100 Subject: crypto/tls: align FIPS-only mode with BoringSSL policy This enables TLS 1.3, disables P-521, and disables non-ECDHE suites. Fixes #64717 Updates #62372 Change-Id: I3a65b239ef0198bbdbe5e55e0810e7128f90a091 Reviewed-on: https://go-review.googlesource.com/c/go/+/549975 Reviewed-by: Roland Shoemaker LUCI-TryBot-Result: Go LUCI Reviewed-by: Than McIntosh --- src/crypto/internal/boring/aes.go | 29 ++++++++++---- src/crypto/internal/boring/notboring.go | 1 + src/crypto/tls/boring.go | 26 +++++++----- src/crypto/tls/boring_test.go | 69 +++++++++++++++++++++++--------- src/crypto/tls/cipher_suites.go | 8 +++- src/crypto/tls/handshake_client.go | 4 +- src/crypto/tls/handshake_client_tls13.go | 4 -- src/crypto/tls/handshake_server_test.go | 28 ++++++++----- src/crypto/tls/handshake_server_tls13.go | 7 ++-- src/crypto/tls/notboring.go | 2 + src/crypto/x509/boring.go | 4 +- 11 files changed, 125 insertions(+), 57 deletions(-) diff --git a/src/crypto/internal/boring/aes.go b/src/crypto/internal/boring/aes.go index 8819f576f4..d18ed5cdc5 100644 --- a/src/crypto/internal/boring/aes.go +++ b/src/crypto/internal/boring/aes.go @@ -228,26 +228,41 @@ func (c *aesCipher) NewGCM(nonceSize, tagSize int) (cipher.AEAD, error) { if tagSize != gcmTagSize { return cipher.NewGCMWithTagSize(&noGCM{c}, tagSize) } - return c.newGCM(false) + return c.newGCM(0) } +const ( + VersionTLS12 = 0x0303 + VersionTLS13 = 0x0304 +) + func NewGCMTLS(c cipher.Block) (cipher.AEAD, error) { - return c.(*aesCipher).newGCM(true) + return c.(*aesCipher).newGCM(VersionTLS12) +} + +func NewGCMTLS13(c cipher.Block) (cipher.AEAD, error) { + return c.(*aesCipher).newGCM(VersionTLS13) } -func (c *aesCipher) newGCM(tls bool) (cipher.AEAD, error) { +func (c *aesCipher) newGCM(tlsVersion uint16) (cipher.AEAD, error) { var aead *C.GO_EVP_AEAD switch len(c.key) * 8 { case 128: - if tls { + switch tlsVersion { + case VersionTLS12: aead = C._goboringcrypto_EVP_aead_aes_128_gcm_tls12() - } else { + case VersionTLS13: + aead = C._goboringcrypto_EVP_aead_aes_128_gcm_tls13() + default: aead = C._goboringcrypto_EVP_aead_aes_128_gcm() } case 256: - if tls { + switch tlsVersion { + case VersionTLS12: aead = C._goboringcrypto_EVP_aead_aes_256_gcm_tls12() - } else { + case VersionTLS13: + aead = C._goboringcrypto_EVP_aead_aes_256_gcm_tls13() + default: aead = C._goboringcrypto_EVP_aead_aes_256_gcm() } default: diff --git a/src/crypto/internal/boring/notboring.go b/src/crypto/internal/boring/notboring.go index 361dec9672..02bc468a0d 100644 --- a/src/crypto/internal/boring/notboring.go +++ b/src/crypto/internal/boring/notboring.go @@ -50,6 +50,7 @@ func NewHMAC(h func() hash.Hash, key []byte) hash.Hash { panic("boringcrypto: no func NewAESCipher(key []byte) (cipher.Block, error) { panic("boringcrypto: not available") } func NewGCMTLS(cipher.Block) (cipher.AEAD, error) { panic("boringcrypto: not available") } +func NewGCMTLS13(cipher.Block) (cipher.AEAD, error) { panic("boringcrypto: not available") } type PublicKeyECDSA struct{ _ int } type PrivateKeyECDSA struct{ _ int } diff --git a/src/crypto/tls/boring.go b/src/crypto/tls/boring.go index 1827f76458..aad96b1c74 100644 --- a/src/crypto/tls/boring.go +++ b/src/crypto/tls/boring.go @@ -6,9 +6,10 @@ package tls -import ( - "crypto/internal/boring/fipstls" -) +import "crypto/internal/boring/fipstls" + +// The FIPS-only policies enforced here currently match BoringSSL's +// ssl_policy_fips_202205. // needFIPS returns fipstls.Required(); it avoids a new import in common.go. func needFIPS() bool { @@ -17,19 +18,19 @@ func needFIPS() bool { // fipsMinVersion replaces c.minVersion in FIPS-only mode. func fipsMinVersion(c *Config) uint16 { - // FIPS requires TLS 1.2. + // FIPS requires TLS 1.2 or TLS 1.3. return VersionTLS12 } // fipsMaxVersion replaces c.maxVersion in FIPS-only mode. func fipsMaxVersion(c *Config) uint16 { - // FIPS requires TLS 1.2. - return VersionTLS12 + // FIPS requires TLS 1.2 or TLS 1.3. + return VersionTLS13 } // default defaultFIPSCurvePreferences is the FIPS-allowed curves, // in preference order (most preferable first). -var defaultFIPSCurvePreferences = []CurveID{CurveP256, CurveP384, CurveP521} +var defaultFIPSCurvePreferences = []CurveID{CurveP256, CurveP384} // fipsCurvePreferences replaces c.curvePreferences in FIPS-only mode. func fipsCurvePreferences(c *Config) []CurveID { @@ -54,8 +55,6 @@ var defaultCipherSuitesFIPS = []uint16{ TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, - TLS_RSA_WITH_AES_128_GCM_SHA256, - TLS_RSA_WITH_AES_256_GCM_SHA384, } // fipsCipherSuites replaces c.cipherSuites in FIPS-only mode. @@ -75,8 +74,14 @@ func fipsCipherSuites(c *Config) []uint16 { return list } +// defaultCipherSuitesTLS13FIPS are the FIPS-allowed cipher suites for TLS 1.3. +var defaultCipherSuitesTLS13FIPS = []uint16{ + TLS_AES_128_GCM_SHA256, + TLS_AES_256_GCM_SHA384, +} + // fipsSupportedSignatureAlgorithms currently are a subset of -// defaultSupportedSignatureAlgorithms without Ed25519 and SHA-1. +// defaultSupportedSignatureAlgorithms without Ed25519, SHA-1, and P-521. var fipsSupportedSignatureAlgorithms = []SignatureScheme{ PSSWithSHA256, PSSWithSHA384, @@ -86,7 +91,6 @@ var fipsSupportedSignatureAlgorithms = []SignatureScheme{ PKCS1WithSHA384, ECDSAWithP384AndSHA384, PKCS1WithSHA512, - ECDSAWithP521AndSHA512, } // supportedSignatureAlgorithms returns the supported signature algorithms. diff --git a/src/crypto/tls/boring_test.go b/src/crypto/tls/boring_test.go index 085ff5713e..a192a657b4 100644 --- a/src/crypto/tls/boring_test.go +++ b/src/crypto/tls/boring_test.go @@ -25,6 +25,31 @@ import ( "time" ) +func allCipherSuitesIncludingTLS13() []uint16 { + s := allCipherSuites() + for _, suite := range cipherSuitesTLS13 { + s = append(s, suite.id) + } + return s +} + +func isTLS13CipherSuite(id uint16) bool { + for _, suite := range cipherSuitesTLS13 { + if id == suite.id { + return true + } + } + return false +} + +func generateKeyShare(group CurveID) keyShare { + key, err := generateECDHEKey(rand.Reader, group) + if err != nil { + panic(err) + } + return keyShare{group: group, data: key.PublicKey().Bytes()} +} + func TestBoringServerProtocolVersion(t *testing.T) { test := func(name string, v uint16, msg string) { t.Run(name, func(t *testing.T) { @@ -33,8 +58,11 @@ func TestBoringServerProtocolVersion(t *testing.T) { clientHello := &clientHelloMsg{ vers: v, random: make([]byte, 32), - cipherSuites: allCipherSuites(), + cipherSuites: allCipherSuitesIncludingTLS13(), compressionMethods: []uint8{compressionNone}, + supportedCurves: defaultCurvePreferences, + keyShares: []keyShare{generateKeyShare(CurveP256)}, + supportedPoints: []uint8{pointFormatUncompressed}, supportedVersions: []uint16{v}, } testClientHelloFailure(t, serverConfig, clientHello, msg) @@ -48,25 +76,25 @@ func TestBoringServerProtocolVersion(t *testing.T) { fipstls.Force() defer fipstls.Abandon() - test("VersionSSL30", VersionSSL30, "client offered only unsupported versions") - test("VersionTLS10", VersionTLS10, "client offered only unsupported versions") - test("VersionTLS11", VersionTLS11, "client offered only unsupported versions") - test("VersionTLS12", VersionTLS12, "") - test("VersionTLS13", VersionTLS13, "client offered only unsupported versions") + test("VersionSSL30/fipstls", VersionSSL30, "client offered only unsupported versions") + test("VersionTLS10/fipstls", VersionTLS10, "client offered only unsupported versions") + test("VersionTLS11/fipstls", VersionTLS11, "client offered only unsupported versions") + test("VersionTLS12/fipstls", VersionTLS12, "") + test("VersionTLS13/fipstls", VersionTLS13, "") } func isBoringVersion(v uint16) bool { - return v == VersionTLS12 + return v == VersionTLS12 || v == VersionTLS13 } func isBoringCipherSuite(id uint16) bool { switch id { - case TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, + case TLS_AES_128_GCM_SHA256, + TLS_AES_256_GCM_SHA384, + TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, - TLS_RSA_WITH_AES_128_GCM_SHA256, - TLS_RSA_WITH_AES_256_GCM_SHA384: + TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384: return true } return false @@ -74,7 +102,7 @@ func isBoringCipherSuite(id uint16) bool { func isBoringCurve(id CurveID) bool { switch id { - case CurveP256, CurveP384, CurveP521: + case CurveP256, CurveP384: return true } return false @@ -86,7 +114,7 @@ func isECDSA(id uint16) bool { return suite.flags&suiteECSign == suiteECSign } } - panic(fmt.Sprintf("unknown cipher suite %#x", id)) + return false // TLS 1.3 cipher suites are not tied to the signature algorithm. } func isBoringSignatureScheme(alg SignatureScheme) bool { @@ -98,7 +126,6 @@ func isBoringSignatureScheme(alg SignatureScheme) bool { PKCS1WithSHA384, ECDSAWithP384AndSHA384, PKCS1WithSHA512, - ECDSAWithP521AndSHA512, PSSWithSHA256, PSSWithSHA384, PSSWithSHA512: @@ -109,10 +136,9 @@ func isBoringSignatureScheme(alg SignatureScheme) bool { func TestBoringServerCipherSuites(t *testing.T) { serverConfig := testConfig.Clone() - serverConfig.CipherSuites = allCipherSuites() serverConfig.Certificates = make([]Certificate, 1) - for _, id := range allCipherSuites() { + for _, id := range allCipherSuitesIncludingTLS13() { if isECDSA(id) { serverConfig.Certificates[0].Certificate = [][]byte{testECDSACertificate} serverConfig.Certificates[0].PrivateKey = testECDSAPrivateKey @@ -121,14 +147,19 @@ func TestBoringServerCipherSuites(t *testing.T) { serverConfig.Certificates[0].PrivateKey = testRSAPrivateKey } serverConfig.BuildNameToCertificate() - t.Run(fmt.Sprintf("suite=%#x", id), func(t *testing.T) { + t.Run(fmt.Sprintf("suite=%s", CipherSuiteName(id)), func(t *testing.T) { clientHello := &clientHelloMsg{ vers: VersionTLS12, random: make([]byte, 32), cipherSuites: []uint16{id}, compressionMethods: []uint8{compressionNone}, supportedCurves: defaultCurvePreferences, + keyShares: []keyShare{generateKeyShare(CurveP256)}, supportedPoints: []uint8{pointFormatUncompressed}, + supportedVersions: []uint16{VersionTLS12}, + } + if isTLS13CipherSuite(id) { + clientHello.supportedVersions = []uint16{VersionTLS13} } testClientHello(t, serverConfig, clientHello) @@ -160,7 +191,9 @@ func TestBoringServerCurves(t *testing.T) { cipherSuites: []uint16{TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256}, compressionMethods: []uint8{compressionNone}, supportedCurves: []CurveID{curveid}, + keyShares: []keyShare{generateKeyShare(curveid)}, supportedPoints: []uint8{pointFormatUncompressed}, + supportedVersions: []uint16{VersionTLS12}, } testClientHello(t, serverConfig, clientHello) @@ -279,7 +312,7 @@ func TestBoringClientHello(t *testing.T) { } if !isBoringVersion(hello.vers) { - t.Errorf("client vers=%#x, want %#x (TLS 1.2)", hello.vers, VersionTLS12) + t.Errorf("client vers=%#x", hello.vers) } for _, v := range hello.supportedVersions { if !isBoringVersion(v) { diff --git a/src/crypto/tls/cipher_suites.go b/src/crypto/tls/cipher_suites.go index 6f5bc37197..636689beb4 100644 --- a/src/crypto/tls/cipher_suites.go +++ b/src/crypto/tls/cipher_suites.go @@ -556,7 +556,13 @@ func aeadAESGCMTLS13(key, nonceMask []byte) aead { if err != nil { panic(err) } - aead, err := cipher.NewGCM(aes) + var aead cipher.AEAD + if boring.Enabled { + aead, err = boring.NewGCMTLS13(aes) + } else { + boring.Unreachable() + aead, err = cipher.NewGCM(aes) + } if err != nil { panic(err) } diff --git a/src/crypto/tls/handshake_client.go b/src/crypto/tls/handshake_client.go index f016e01b4b..89004c2898 100644 --- a/src/crypto/tls/handshake_client.go +++ b/src/crypto/tls/handshake_client.go @@ -139,7 +139,9 @@ func (c *Conn) makeClientHello() (*clientHelloMsg, *ecdh.PrivateKey, error) { if len(hello.supportedVersions) == 1 { hello.cipherSuites = nil } - if hasAESGCMHardwareSupport { + if needFIPS() { + hello.cipherSuites = append(hello.cipherSuites, defaultCipherSuitesTLS13FIPS...) + } else if hasAESGCMHardwareSupport { hello.cipherSuites = append(hello.cipherSuites, defaultCipherSuitesTLS13...) } else { hello.cipherSuites = append(hello.cipherSuites, defaultCipherSuitesTLS13NoAES...) diff --git a/src/crypto/tls/handshake_client_tls13.go b/src/crypto/tls/handshake_client_tls13.go index 2f59f6888c..a84cede1b0 100644 --- a/src/crypto/tls/handshake_client_tls13.go +++ b/src/crypto/tls/handshake_client_tls13.go @@ -41,10 +41,6 @@ type clientHandshakeStateTLS13 struct { func (hs *clientHandshakeStateTLS13) handshake() error { c := hs.c - if needFIPS() { - return errors.New("tls: internal error: TLS 1.3 reached in FIPS mode") - } - // The server must not select TLS 1.3 in a renegotiation. See RFC 8446, // sections 4.1.2 and 4.1.3. if c.handshakes > 0 { diff --git a/src/crypto/tls/handshake_server_test.go b/src/crypto/tls/handshake_server_test.go index 15db760716..c0a86a4984 100644 --- a/src/crypto/tls/handshake_server_test.go +++ b/src/crypto/tls/handshake_server_test.go @@ -27,6 +27,7 @@ import ( ) func testClientHello(t *testing.T, serverConfig *Config, m handshakeMessage) { + t.Helper() testClientHelloFailure(t, serverConfig, m, "") } @@ -52,23 +53,32 @@ func testClientHelloFailure(t *testing.T, serverConfig *Config, m handshakeMessa ctx := context.Background() conn := Server(s, serverConfig) ch, err := conn.readClientHello(ctx) - hs := serverHandshakeState{ - c: conn, - ctx: ctx, - clientHello: ch, - } - if err == nil { + if err == nil && conn.vers == VersionTLS13 { + hs := serverHandshakeStateTLS13{ + c: conn, + ctx: ctx, + clientHello: ch, + } err = hs.processClientHello() - } - if err == nil { - err = hs.pickCipherSuite() + } else if err == nil { + hs := serverHandshakeState{ + c: conn, + ctx: ctx, + clientHello: ch, + } + err = hs.processClientHello() + if err == nil { + err = hs.pickCipherSuite() + } } s.Close() if len(expectedSubStr) == 0 { if err != nil && err != io.EOF { + t.Helper() t.Errorf("Got error: %s; expected to succeed", err) } } else if err == nil || !strings.Contains(err.Error(), expectedSubStr) { + t.Helper() t.Errorf("Got error: %v; expected to match substring '%s'", err, expectedSubStr) } } diff --git a/src/crypto/tls/handshake_server_tls13.go b/src/crypto/tls/handshake_server_tls13.go index 21d798de37..b68ff9db4c 100644 --- a/src/crypto/tls/handshake_server_tls13.go +++ b/src/crypto/tls/handshake_server_tls13.go @@ -45,10 +45,6 @@ type serverHandshakeStateTLS13 struct { func (hs *serverHandshakeStateTLS13) handshake() error { c := hs.c - if needFIPS() { - return errors.New("tls: internal error: TLS 1.3 reached in FIPS mode") - } - // For an overview of the TLS 1.3 handshake, see RFC 8446, Section 2. if err := hs.processClientHello(); err != nil { return err @@ -163,6 +159,9 @@ func (hs *serverHandshakeStateTLS13) processClientHello() error { if !hasAESGCMHardwareSupport || !aesgcmPreferred(hs.clientHello.cipherSuites) { preferenceList = defaultCipherSuitesTLS13NoAES } + if needFIPS() { + preferenceList = defaultCipherSuitesTLS13FIPS + } for _, suiteID := range preferenceList { hs.suite = mutualCipherSuiteTLS13(hs.clientHello.cipherSuites, suiteID) if hs.suite != nil { diff --git a/src/crypto/tls/notboring.go b/src/crypto/tls/notboring.go index 7d85b39c59..edccb44d87 100644 --- a/src/crypto/tls/notboring.go +++ b/src/crypto/tls/notboring.go @@ -18,3 +18,5 @@ func fipsCurvePreferences(c *Config) []CurveID { panic("fipsCurvePreferences") } func fipsCipherSuites(c *Config) []uint16 { panic("fipsCipherSuites") } var fipsSupportedSignatureAlgorithms []SignatureScheme + +var defaultCipherSuitesTLS13FIPS []uint16 diff --git a/src/crypto/x509/boring.go b/src/crypto/x509/boring.go index 095b58c315..e6237e96bb 100644 --- a/src/crypto/x509/boring.go +++ b/src/crypto/x509/boring.go @@ -22,7 +22,7 @@ func boringAllowCert(c *Certificate) bool { } // The key must be RSA 2048, RSA 3072, RSA 4096, - // or ECDSA P-256, P-384, P-521. + // or ECDSA P-256 or P-384. switch k := c.PublicKey.(type) { default: return false @@ -31,7 +31,7 @@ func boringAllowCert(c *Certificate) bool { return false } case *ecdsa.PublicKey: - if k.Curve != elliptic.P256() && k.Curve != elliptic.P384() && k.Curve != elliptic.P521() { + if k.Curve != elliptic.P256() && k.Curve != elliptic.P384() { return false } } -- cgit v1.3 From 5b84d50038ae852e638631b0a7e71c719e41d240 Mon Sep 17 00:00:00 2001 From: Than McIntosh Date: Mon, 18 Dec 2023 16:00:06 -0500 Subject: test: skip rangegen.go on 32-bit platforms Add a skip for this test that effectively disables it for 32-bit platforms, so as to not run into timeouts or OOMs on smaller machines. Fixes #64789. Change-Id: I2d428e1dccae62b8bb1a69c5f95699692a282bbb Reviewed-on: https://go-review.googlesource.com/c/go/+/550975 LUCI-TryBot-Result: Go LUCI Reviewed-by: Carlos Amedee --- test/rangegen.go | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/test/rangegen.go b/test/rangegen.go index bdcf099862..8231c64db7 100644 --- a/test/rangegen.go +++ b/test/rangegen.go @@ -25,6 +25,7 @@ import ( "bytes" "fmt" "log" + "math/bits" "os" "os/exec" "strings" @@ -37,6 +38,13 @@ func main() { log.SetFlags(0) log.SetPrefix("rangegen: ") + if !long && bits.UintSize == 32 { + // Skip this test on 32-bit platforms, where it seems to + // cause timeouts and build problems. + skip() + return + } + b := new(bytes.Buffer) tests := "" flush := func(force bool) { @@ -331,3 +339,12 @@ func run(f func(*output, int)int, i int) *output { } ` + +func skip() { + const code = ` +package main +func main() { +} +` + fmt.Printf("%s\n", code) +} -- cgit v1.3 From 90daaa0576aafd66b3b0fb9e5c4814fbdcc4b5ce Mon Sep 17 00:00:00 2001 From: Guoqi Chen Date: Fri, 15 Dec 2023 17:35:40 +0800 Subject: doc/go1.22: announcing support address sanitizer on Loong64 Change-Id: Ide5eddc6c04347ada468bc8dd4c27d9b492cf0ff Reviewed-on: https://go-review.googlesource.com/c/go/+/550176 LUCI-TryBot-Result: Go LUCI Reviewed-by: Bryan Mills Auto-Submit: Dmitri Shuralyov Reviewed-by: Mauri de Souza Meneguzzo Reviewed-by: David Chase --- doc/go1.22.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/go1.22.html b/doc/go1.22.html index 3d63e99eeb..458e17954d 100644 --- a/doc/go1.22.html +++ b/doc/go1.22.html @@ -965,7 +965,7 @@ We plan to include an API migration tool in a future release, likely Go 1.23. The loong64 port now supports passing function arguments and results using registers.

- The linux/loong64 port now supports the memory sanitizer, new-style linker relocations, and the plugin build mode. + The linux/loong64 port now supports the address sanitizer, memory sanitizer, new-style linker relocations, and the plugin build mode.

OpenBSD

-- cgit v1.3 From 6fe0d3758b35afcc342832e376d8d985a5a29070 Mon Sep 17 00:00:00 2001 From: Robert Griesemer Date: Mon, 18 Dec 2023 18:13:30 -0800 Subject: cmd/compile: remove interfacecycles debug flag Per the discussion on the issue, since no problems related to this appeared since Go 1.20, remove the ability to disable the check for anonymous interface cycles permanently. Adjust various tests accordingly. For #56103. Change-Id: Ica2b28752dca08934bbbc163a9b062ae1eb2a834 Reviewed-on: https://go-review.googlesource.com/c/go/+/550896 Run-TryBot: Robert Griesemer Auto-Submit: Robert Griesemer Reviewed-by: Robert Griesemer TryBot-Result: Gopher Robot Reviewed-by: Matthew Dempsky --- src/cmd/compile/internal/base/debug.go | 1 - src/cmd/compile/internal/noder/irgen.go | 29 +++++++++++++------------- src/cmd/compile/internal/types2/stdlib_test.go | 1 + src/go/types/stdlib_test.go | 1 + test/fixedbugs/bug398.go | 10 ++++----- test/fixedbugs/issue16369.go | 4 ++-- 6 files changed, 23 insertions(+), 23 deletions(-) diff --git a/src/cmd/compile/internal/base/debug.go b/src/cmd/compile/internal/base/debug.go index a85f0139fc..aadd950a0a 100644 --- a/src/cmd/compile/internal/base/debug.go +++ b/src/cmd/compile/internal/base/debug.go @@ -36,7 +36,6 @@ type DebugFlags struct { Gossahash string `help:"hash value for use in debugging the compiler"` InlFuncsWithClosures int `help:"allow functions with closures to be inlined" concurrent:"ok"` InlStaticInit int `help:"allow static initialization of inlined calls" concurrent:"ok"` - InterfaceCycles int `help:"allow anonymous interface cycles"` Libfuzzer int `help:"enable coverage instrumentation for libfuzzer"` LoopVar int `help:"shared (0, default), 1 (private loop variables), 2, private + log"` LoopVarHash string `help:"for debugging changes in loop behavior. Overrides experiment and loopvar flag."` diff --git a/src/cmd/compile/internal/noder/irgen.go b/src/cmd/compile/internal/noder/irgen.go index 46511d1f97..d909f3467b 100644 --- a/src/cmd/compile/internal/noder/irgen.go +++ b/src/cmd/compile/internal/noder/irgen.go @@ -92,23 +92,22 @@ func checkFiles(m posMap, noders []*noder) (*types2.Package, *types2.Info) { } // Check for anonymous interface cycles (#56103). - if base.Debug.InterfaceCycles == 0 { - var f cycleFinder - for _, file := range files { - syntax.Inspect(file, func(n syntax.Node) bool { - if n, ok := n.(*syntax.InterfaceType); ok { - if f.hasCycle(n.GetTypeInfo().Type.(*types2.Interface)) { - base.ErrorfAt(m.makeXPos(n.Pos()), errors.InvalidTypeCycle, "invalid recursive type: anonymous interface refers to itself (see https://go.dev/issue/56103)") - - for typ := range f.cyclic { - f.cyclic[typ] = false // suppress duplicate errors - } + // TODO(gri) move this code into the type checkers (types2 and go/types) + var f cycleFinder + for _, file := range files { + syntax.Inspect(file, func(n syntax.Node) bool { + if n, ok := n.(*syntax.InterfaceType); ok { + if f.hasCycle(n.GetTypeInfo().Type.(*types2.Interface)) { + base.ErrorfAt(m.makeXPos(n.Pos()), errors.InvalidTypeCycle, "invalid recursive type: anonymous interface refers to itself (see https://go.dev/issue/56103)") + + for typ := range f.cyclic { + f.cyclic[typ] = false // suppress duplicate errors } - return false } - return true - }) - } + return false + } + return true + }) } base.ExitIfErrors() diff --git a/src/cmd/compile/internal/types2/stdlib_test.go b/src/cmd/compile/internal/types2/stdlib_test.go index 7c14e3476e..405af78572 100644 --- a/src/cmd/compile/internal/types2/stdlib_test.go +++ b/src/cmd/compile/internal/types2/stdlib_test.go @@ -311,6 +311,7 @@ func TestStdFixed(t *testing.T) { testTestDir(t, filepath.Join(testenv.GOROOT(t), "test", "fixedbugs"), "bug248.go", "bug302.go", "bug369.go", // complex test instructions - ignore + "bug398.go", // types2 doesn't check for anonymous interface cycles (go.dev/issue/56103) "issue6889.go", // gc-specific test "issue11362.go", // canonical import path check "issue16369.go", // types2 handles this correctly - not an issue diff --git a/src/go/types/stdlib_test.go b/src/go/types/stdlib_test.go index f90f9388c2..a89cd858db 100644 --- a/src/go/types/stdlib_test.go +++ b/src/go/types/stdlib_test.go @@ -312,6 +312,7 @@ func TestStdFixed(t *testing.T) { testTestDir(t, filepath.Join(testenv.GOROOT(t), "test", "fixedbugs"), "bug248.go", "bug302.go", "bug369.go", // complex test instructions - ignore + "bug398.go", // go/types doesn't check for anonymous interface cycles (go.dev/issue/56103) "issue6889.go", // gc-specific test "issue11362.go", // canonical import path check "issue16369.go", // go/types handles this correctly - not an issue diff --git a/test/fixedbugs/bug398.go b/test/fixedbugs/bug398.go index db3e43c7f9..2b00f6074d 100644 --- a/test/fixedbugs/bug398.go +++ b/test/fixedbugs/bug398.go @@ -1,4 +1,4 @@ -// compile -d=interfacecycles +// errorcheck // Copyright 2012 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style @@ -11,11 +11,11 @@ package p // exported interfaces -type I1 interface { +type I1 interface { // ERROR "invalid recursive type: anonymous interface refers to itself" F() interface{I1} } -type I2 interface { +type I2 interface { // ERROR "invalid recursive type: anonymous interface refers to itself" F() interface{I2} } @@ -28,11 +28,11 @@ func F() bool { // non-exported interfaces -type i1 interface { +type i1 interface { // ERROR "invalid recursive type: anonymous interface refers to itself" F() interface{i1} } -type i2 interface { +type i2 interface { // ERROR "invalid recursive type: anonymous interface refers to itself" F() interface{i2} } diff --git a/test/fixedbugs/issue16369.go b/test/fixedbugs/issue16369.go index 3a7bb7eaed..86d0ce645d 100644 --- a/test/fixedbugs/issue16369.go +++ b/test/fixedbugs/issue16369.go @@ -1,4 +1,4 @@ -// compile -d=interfacecycles +// errorcheck // Copyright 2016 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style @@ -6,7 +6,7 @@ package p -type T interface { +type T interface { // ERROR "invalid recursive type: anonymous interface refers to itself" M(interface { T }) -- cgit v1.3 From 1d4b0b6236febe0646d8d7b0103da5d169f185cc Mon Sep 17 00:00:00 2001 From: Jonathan Amsterdam Date: Mon, 18 Dec 2023 13:07:07 -0500 Subject: doc/go1.22.html: release notes for slog, testing/slogtest and net/http.ServeMux Updates #61422. Change-Id: I8f2dfe92405f409c46465bbb5714eff662ce9b49 Reviewed-on: https://go-review.googlesource.com/c/go/+/550776 Run-TryBot: Jonathan Amsterdam Reviewed-by: Damien Neil TryBot-Result: Gopher Robot --- doc/go1.22.html | 61 +++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 46 insertions(+), 15 deletions(-) diff --git a/doc/go1.22.html b/doc/go1.22.html index 458e17954d..c2bcef3053 100644 --- a/doc/go1.22.html +++ b/doc/go1.22.html @@ -359,6 +359,45 @@ and the Go runtime also use ChaCha8 for randomness. We plan to include an API migration tool in a future release, likely Go 1.23.

+

+ +

+ HTTP routing in the standard library is now more expressive. + The patterns used by net/http.ServeMux have been enhanced to accept methods and wildcards. +

+ +

+ Registering a handler with a method, like "POST /items/create", restricts + invocations of the handler to requests with the given method. A pattern with a method takes precedence over a matching pattern without one. + As a special case, registering a handler with "GET" also registers it with "HEAD". +

+ +

+ Wildcards in patterns, like /items/{id}, match segments of the URL path. + The actual segment value may be accessed by calling the Request.PathValue method. + A wildcard ending in "...", like /files/{path...}, must occur at the end of a pattern and matches all the remaining segments. +

+ +

+ A pattern that ends in "/" matches all paths that have it as a prefix, as always. + To match the exact pattern including the trailing slash, end it with {$}, + as in /exact/match/{$}. +

+ +

+ If two patterns overlap in the requests that they match, then the more specific pattern takes precedence. + If neither is more specific, the patterns conflict. + This rule generalizes the original precedence rules and maintains the property that the order in which + patterns are registered does not matter. +

+ +

+ This change breaks backwards compatiblity in small ways, some obvious—patterns with "{" and "}" behave differently— + and some less so—treatment of escaped paths has been improved. + The change is controlled by a GODEBUG field named httpmuxgo121. + Set httpmuxgo121=1 to restore the old behavior. +

+

Minor changes to the library

@@ -633,14 +672,13 @@ We plan to include an API migration tool in a future release, likely Go 1.23.

log/slog

- TODO: https://go.dev/issue/62418: enable setting level on default log.Logger -

- -

- TODO: https://go.dev/cl/525096: log/slog: add LogLoggerLevel to enable setting level on the default logger; modified api/next/62418.txt + The new SetLogLoggerLevel function + controls the level for the bridge between the `slog` and `log` packages. It sets the minimum level + for calls to the top-level `slog` logging functions, and it sets the level for calls to `log.Logger` + that go through `slog`.

-
+
math/big
@@ -682,10 +720,6 @@ We plan to include an API migration tool in a future release, likely Go 1.23. operating on an fs.FS.

-

- TODO: https://go.dev/issue/61410: enhanced ServeMux routing -

-

The HTTP server and client now reject requests and responses containing an invalid empty Content-Length header. @@ -925,11 +959,8 @@ We plan to include an API migration tool in a future release, likely Go 1.23.

testing/slogtest

- TODO: https://go.dev/issue/61758: support sub-tests -

- -

- TODO: https://go.dev/cl/516076: testing/slogtest: add Run to run cases as subtests; modified api/next/61758.txt + The new Run function uses sub-tests to run test cases, + providing finer-grained control.

-- cgit v1.3