Fix 6 issues surfaced by adversarial agent team (round 35)

Concurrency/lifecycle (P1, highest-value):
- singleton_resources.go: RegisterBackgroundTask now REPLACES a stopped
  task instead of reusing it. BackgroundTask.Start is sync.Once-guarded,
  so a task stopped on instance close and re-registered (Traefik
  close+recreate for the same provider) was permanently dead: IsTaskRunning
  returned false (started && !stopped) so callers retried a start that
  was a no-op. Metadata refresh, token cleanup, and health checks now
  resume across a recreate cycle.
- error_recovery.go: the shared graceful-degradation health task was
  bound to the FIRST instance's performHealthChecks (later instances'
  checks never ran) and any instance's Close stopped it for all. Added a
  global live-instance registry; the shared task runs every instance's
  checks and is stopped only when the last instance closes (mirrors the
  singleton-token-cleanup gate).

Storage/cache:
- hybrid.go recordL2Error: Store(now) then immediately compared
  time.Since(t)<1s against its own write => ALWAYS fired, so a single
  transient L2 error flipped the whole cache into fallback (full L2
  bypass -> upstream stampede). Now enters fallback only on a PRIOR
  error within the window (true "multiple errors"). Updated two tests
  that had encoded the old single-error-fallback bug.
- redis_pool.go: ConnectionPool.Put checked closed but sent on
  p.connections without holding p.mu; Close closed it under p.mu.
  Window => 'send on closed channel' panic at shutdown. Put now holds
  p.mu (the lock only Close used).

HTTP/transport:
- jwk.go fetchJWKS: added an explicit >1 MiB guard (was silent partial
  read, could yield a truncated keyset and spurious missing-kid).
- http_client_pool.go configKey: now includes every transport-behavior
  field it actually consumes (compression, HTTP2, keep-alives, timeouts,
  buffer sizes, dial) so behaviorally different configs no longer share
  one transport built with the first config's settings.

Each fix has a regression test. Verified: full source-order suite green,
golangci-lint 0 issues, 3x full -race -count=1 -shuffle=on ./... all exit 0.
This commit is contained in:
2026-08-12 23:55:55 +01:00
parent 01ad33c879
commit 35d26078f0
9 changed files with 361 additions and 27 deletions
+44
View File
@@ -688,3 +688,47 @@ func TestRedisConn_RejectOversizedArgumentBytes(t *testing.T) {
require.Error(t, err)
assert.Contains(t, err.Error(), "arguments too large")
}
// TestConnectionPool_PutDuringClose_NoPanic hammers Put/Get concurrently with
// Close. Before the fix, Put checked closed then sent on p.connections
// without holding p.mu; if Close closed the channel in between, the send
// panicked ("send on closed channel") at shutdown. Run under -race.
func TestConnectionPool_PutDuringClose_NoPanic(t *testing.T) {
mr := NewMiniredisServer(t)
config := &PoolConfig{
Address: mr.GetAddr(),
MaxConnections: 5,
ConnectTimeout: 5 * time.Second,
ReadTimeout: 3 * time.Second,
WriteTimeout: 3 * time.Second,
}
pool, err := NewConnectionPool(config)
require.NoError(t, err)
ctx := context.Background()
stop := make(chan struct{})
var wg sync.WaitGroup
for g := 0; g < 8; g++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-stop:
return
default:
}
if conn, err := pool.Get(ctx); err == nil {
pool.Put(conn) // may race Close
}
}
}()
}
time.Sleep(5 * time.Millisecond)
_ = pool.Close()
close(stop)
wg.Wait()
}