fix(providers): address adversarial review findings F1, F4, F6, F9

F1: TestModelRegistry_MaybeRefresh_Throttled_OnePerInterval mutated a
plain `now` variable while a background refresh goroutine read it
through nowFn, racing under -race in about 14% of full-suite runs once
this branch's timer-heavy tests shared the parallel phase. Guards the
clock with a mutex instead. registry.go is untouched.

F4: a mid-body watchdog timeout never reached provider-health
accounting. limiter.recordProviderAttempt only ran at header-arrival
time, so a provider that reliably stalled mid-body reported
attempts=1 failures=0 forever. watchdogBody.fire now invokes the same
attemptRecorder a second time, as a failure, when it fires. Reuses the
existing recorder rather than adding new plumbing; this does mean two
attempts are recorded for one logical request (one success at header
time, one failure at watchdog-fire time), not one attempt marked
failed outright.

F6: watchdogBody.Read reset the timer on any err == nil read,
regardless of n. A reader returning (0, nil) forever would have held
the watchdog open indefinitely. Reset now only fires when n > 0.

F9 (three items):
- proxyUpstream passed logPrefix as a bare providerName, which
  newWatchdogBody wrapped in a hardcoded `provider %q` — labeling an
  MCP/A2A target as a provider. watchdogBody now takes an
  already-formatted label from the caller instead.
- errUpstream was wrapped twice: once in watchdogBody's own error, once
  again by every caller that already wraps it. watchdogBody now wraps
  only errProviderTimeout; callers' existing wraps still make
  errors.Is(err, errUpstream) hold.
- timeout_test.go read wb.timedOut directly in one place while every
  other access is atomic; switched to atomic.LoadInt32.

Also rewords errProviderTimeout's message from "provider request timed
out" to "upstream progress stalled" (finding F3, documented not fixed
per coordinator ruling): the watchdog measures time since the last
successful upstream read, which can go quiet because of a slow client
write between reads, not only a slow provider. The new wording no
longer asserts which side caused it.

New tests: TestChatCompletion_Stream_StallsMidBody_RecordsProviderFailure,
TestWatchdogBody_ZeroByteRead_DoesNotResetTimer,
TestWatchdogBody_TimeoutError_UsesCallerLabel_NotHardcodedProvider.
Each mutation-verified per its own doc comment.
This commit is contained in:
2026-08-23 02:40:14 +01:00
parent 834238d141
commit 51e675acb0
5 changed files with 332 additions and 43 deletions
+12 -3
View File
@@ -1014,8 +1014,13 @@ func (g *Gateway) proxyUpstream(w http.ResponseWriter, r *http.Request, upstream
// carries a recorder (attemptRecorderFromContext, providers.go) —
// handleTargetProxy (mcp_a2a.go), this function's other caller, never
// wraps r's context this way, so an MCP/A2A target proxy attempt is
// correctly never accounted as provider traffic.
if rec := attemptRecorderFromContext(r.Context()); rec != nil {
// correctly never accounted as provider traffic. rec is kept in scope
// (not just checked inline) so the watchdogBody below can reuse it
// too — a mid-body stall must reach the SAME provider-health
// accounting a build/send failure already does (coordinator
// adversarial review, 2026-08-23, finding F4).
rec := attemptRecorderFromContext(r.Context())
if rec != nil {
rec(resp, err)
}
if err != nil {
@@ -1028,7 +1033,11 @@ func (g *Gateway) proxyUpstream(w http.ResponseWriter, r *http.Request, upstream
writeOAIError(w, http.StatusBadGateway, "server_error", "upstream connection error")
return proxyResult{}, false
}
resp.Body = newWatchdogBody(resp.Body, cancel, timeout, logPrefix)
// logPrefix, not a hardcoded "provider %q": handleTargetProxy's own
// calls here (mcp_a2a.go) pass "mcp target (name ...)"/"a2a target
// (name ...)" — an MCP/A2A target is not a provider, and the watchdog
// error text must not claim it is (finding F9).
resp.Body = newWatchdogBody(resp.Body, cancel, timeout, logPrefix, rec)
defer resp.Body.Close() //nolint:errcheck // read-side close; nothing actionable on failure
copyHeadersExcept(w.Header(), resp.Header, hopByHopHeaders, dangerousResponseHeaders)