internal/weak: shade pointer in weak-to-strong conversion

There's a bug in the weak-to-strong conversion in that creating the
*only* strong pointer to some weakly-held object during the mark phase
may result in that object not being properly marked.

The exact mechanism for this is that the new strong pointer will always
point to a white object (because it was only weakly referenced up until
this point) and it can then be stored in a blackened stack, hiding it
from the garbage collector.

This "hide a white pointer in the stack" problem is pretty much exactly
what the Yuasa part of the hybrid write barrier is trying to catch, so
we need to do the same thing the write barrier would do: shade the
pointer.

Added a test and confirmed that it fails with high probability if the
pointer shading is missing.

Fixes #69210.

Change-Id: Iaae64ae95ea7e975c2f2c3d4d1960e74e1bd1c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/610396
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

1 files changed, 82 insertions, 0 deletions

diff --git a/src/internal/weak/pointer_test.go b/src/internal/weak/pointer_test.go
index e143749230..5a861bb9ca 100644
--- a/src/internal/weak/pointer_test.go
+++ b/src/internal/weak/pointer_test.go
@@ -5,9 +5,12 @@
 package weak_test
 
 import (
+	"context"
 	"internal/weak"
 	"runtime"
+	"sync"
 	"testing"
+	"time"
 )
 
 type T struct {
@@ -128,3 +131,82 @@ func TestPointerFinalizer(t *testing.T) {
 		t.Errorf("weak pointer is non-nil even after finalization: %v", wt)
 	}
 }
+
+// Regression test for issue 69210.
+//
+// Weak-to-strong conversions must shade the new strong pointer, otherwise
+// that might be creating the only strong pointer to a white object which
+// is hidden in a blackened stack.
+//
+// Never fails if correct, fails with some high probability if incorrect.
+func TestIssue69210(t *testing.T) {
+	if testing.Short() {
+		t.Skip("this is a stress test that takes seconds to run on its own")
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
+	defer cancel()
+
+	// What we're trying to do is manufacture the conditions under which this
+	// bug happens. Specifically, we want:
+	//
+	// 1. To create a whole bunch of objects that are only weakly-pointed-to,
+	// 2. To call Strong while the GC is in the mark phase,
+	// 3. The new strong pointer to be missed by the GC,
+	// 4. The following GC cycle to mark a free object.
+	//
+	// Unfortunately, (2) and (3) are hard to control, but we can increase
+	// the likelihood by having several goroutines do (1) at once while
+	// another goroutine constantly keeps us in the GC with runtime.GC.
+	// Like throwing darts at a dart board until they land just right.
+	// We can increase the likelihood of (4) by adding some delay after
+	// creating the strong pointer, but only if it's non-nil. If it's nil,
+	// that means it was already collected in which case there's no chance
+	// of triggering the bug, so we want to retry as fast as possible.
+	// Our heap here is tiny, so the GCs will go by fast.
+	//
+	// As of 2024-09-03, removing the line that shades pointers during
+	// the weak-to-strong conversion causes this test to fail about 50%
+	// of the time.
+
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		for {
+			runtime.GC()
+
+			select {
+			case <-ctx.Done():
+				return
+			default:
+			}
+		}
+	}()
+	for range max(runtime.GOMAXPROCS(-1)-1, 1) {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			for {
+				for range 5 {
+					bt := new(T)
+					wt := weak.Make(bt)
+					bt = nil
+					time.Sleep(1 * time.Millisecond)
+					bt = wt.Strong()
+					if bt != nil {
+						time.Sleep(4 * time.Millisecond)
+						bt.t = bt
+						bt.a = 12
+					}
+					runtime.KeepAlive(bt)
+				}
+				select {
+				case <-ctx.Done():
+					return
+				default:
+				}
+			}
+		}()
+	}
+	wg.Wait()
+}