fix(providers): deadline failures, cheaper polling, async writes

Review round (v0.22): 1 must-fix + 5 should-fix. This covers the three
backend items.

MUST-1's WCAG fix lives in the webui commit; this is the three provider/
admin-side SHOULD items.

SHOULD-1 (timeout blindness): a hung upstream previously read as 100%
healthy — isTransient (retry.go) deliberately excludes
context.DeadlineExceeded from what it retries (retrying past the
gateway's own deadline is pointless), and recordProviderAttempt reused
that same classification unforked, so a timed-out attempt recorded no
failure. Fixed narrowly in recordProviderAttempt only, per the ruling:
isTransient itself stays untouched and shared. A new isDeadlineExceeded
adds exactly one extra condition — a context.Canceled error (the client
walked away) still counts as an attempt only, from both functions alike.
The sentinel match (matchesSentinel) is a hand-rolled errors.Unwrap walk,
not errors.Is: providerHTTPError's own doc comment already documents that
errors.As panics under Yaegi checking whether an interpreted pointer type
implements error; errors.Is walks the identical internal machinery and
carries the same risk, so this sidesteps it with a bare
`err.(sentinelUnwrapper)` comma-ok assertion instead — plain interface
satisfaction, not reflection over a possibly-interpreted type.

SHOULD-2 (poll fan-out): a (provider, model) scope's minute-window
counters fed nothing but a badge's title text on the Providers tab, so a
dashboard with N configured models paid N wasted minute-window reads on
every 5s poll. providerCounterKeys/providerUsage now read a
kindProviderModel scope's day window only (2 keys, down from 4); a
kindProvider scope is unaffected (still 4 — one scope per provider, not
per model, so the read amplification never applied there).
adminModelRateView drops AttemptsMinute/FailuresMinute entirely, not just
unpopulated fields — the webui side (separate commit) falls back to a
day-window-only title for a per-model badge.

SHOULD-5 (TTFB): recordProviderAttempt's store write is telemetry-only —
nothing in the request path gates on it — so it now runs via a new
limiter.spawn dependency-injection field (mirrors nowFn/waitFn's existing
pattern), defaulting to a real goroutine in production and overridden to
run synchronously in every test that asserts on the resulting counter
immediately afterward. This removes the synchronous Redis round trip
this counter family added to the streaming TTFB path entirely. Loss on
process shutdown is accepted (telemetry only).
This commit is contained in:
2026-08-21 23:59:47 +01:00
parent 080daebb53
commit 1b4516efbb
6 changed files with 343 additions and 49 deletions
+8
View File
@@ -54,6 +54,14 @@ func newAdminGatewayHandle(t *testing.T, cfg *Config) (http.Handler, *Gateway) {
if !ok {
t.Fatal("handler is not *Gateway")
}
// SHOULD-5 (v0.22 review round): production spawns recordProviderAttempt's
// store write in its own goroutine, off a request's TTFB path — every
// admin_test.go test that asserts on a resulting provider counter (GET
// /admin/api/overview's Attempts*/Failures* fields) needs it to have
// already landed, so spawn runs synchronously here rather than per-test
// (the deterministic-test half of that dependency-injection field; see
// limiter.spawn's own doc comment, limits.go).
gw.limiter.spawn = func(f func()) { f() }
return h, gw
}