fix(providers): deadline failures, cheaper polling, async writes

Review round (v0.22): 1 must-fix + 5 should-fix. This covers the three
backend items.

MUST-1's WCAG fix lives in the webui commit; this is the three provider/
admin-side SHOULD items.

SHOULD-1 (timeout blindness): a hung upstream previously read as 100%
healthy — isTransient (retry.go) deliberately excludes
context.DeadlineExceeded from what it retries (retrying past the
gateway's own deadline is pointless), and recordProviderAttempt reused
that same classification unforked, so a timed-out attempt recorded no
failure. Fixed narrowly in recordProviderAttempt only, per the ruling:
isTransient itself stays untouched and shared. A new isDeadlineExceeded
adds exactly one extra condition — a context.Canceled error (the client
walked away) still counts as an attempt only, from both functions alike.
The sentinel match (matchesSentinel) is a hand-rolled errors.Unwrap walk,
not errors.Is: providerHTTPError's own doc comment already documents that
errors.As panics under Yaegi checking whether an interpreted pointer type
implements error; errors.Is walks the identical internal machinery and
carries the same risk, so this sidesteps it with a bare
`err.(sentinelUnwrapper)` comma-ok assertion instead — plain interface
satisfaction, not reflection over a possibly-interpreted type.

SHOULD-2 (poll fan-out): a (provider, model) scope's minute-window
counters fed nothing but a badge's title text on the Providers tab, so a
dashboard with N configured models paid N wasted minute-window reads on
every 5s poll. providerCounterKeys/providerUsage now read a
kindProviderModel scope's day window only (2 keys, down from 4); a
kindProvider scope is unaffected (still 4 — one scope per provider, not
per model, so the read amplification never applied there).
adminModelRateView drops AttemptsMinute/FailuresMinute entirely, not just
unpopulated fields — the webui side (separate commit) falls back to a
day-window-only title for a per-model badge.

SHOULD-5 (TTFB): recordProviderAttempt's store write is telemetry-only —
nothing in the request path gates on it — so it now runs via a new
limiter.spawn dependency-injection field (mirrors nowFn/waitFn's existing
pattern), defaulting to a real goroutine in production and overridden to
run synchronously in every test that asserts on the resulting counter
immediately afterward. This removes the synchronous Redis round trip
this counter family added to the streaming TTFB path entirely. Loss on
process shutdown is accepted (telemetry only).
This commit is contained in:
2026-08-21 23:59:47 +01:00
parent 080daebb53
commit 1b4516efbb
6 changed files with 343 additions and 49 deletions
+13 -11
View File
@@ -308,14 +308,18 @@ type adminProviderView struct {
// adminModelRateView is one upstream model's current-window attempt/
// failure counters within its provider — adminProviderView.ModelRates'
// value type (Feature A, v0.22). Field shape and meaning are identical to
// adminProviderView's own Attempts*/Failures* fields, just scoped to one
// (provider, model) pair instead of the whole provider.
// value type (Feature A, v0.22). Day window only, no AttemptsMinute/
// FailuresMinute (SHOULD-2, v0.22 review round — removed, not merely
// left unpopulated): a dashboard with N configured models was paying N
// wasted minute-window store reads on every 5s poll for a figure that
// only ever fed a badge's title text, never its own displayed tier — see
// limits.go's providerCounterKeys/providerScopeKeyCount for the read-side
// half of this same ruling. The Providers tab's per-model badge falls
// back to a day-window-only title when these are absent; see webui's
// lib/provider-rate.ts (dayRateTitle) and ProviderRateBadge.vue.
type adminModelRateView struct {
AttemptsDay int64 `json:"attemptsDay"`
FailuresDay int64 `json:"failuresDay"`
AttemptsMinute int64 `json:"attemptsMinute"`
FailuresMinute int64 `json:"failuresMinute"`
AttemptsDay int64 `json:"attemptsDay"`
FailuresDay int64 `json:"failuresDay"`
}
// adminRedisView is the redis status line in GET /admin/api/overview.
@@ -423,10 +427,8 @@ func (g *Gateway) buildAdminOverview() adminOverviewResponse {
mc := modelCounters[mi]
mi++
modelRates[model] = adminModelRateView{
AttemptsDay: mc.attemptsDay,
FailuresDay: mc.failuresDay,
AttemptsMinute: mc.attemptsMinute,
FailuresMinute: mc.failuresMinute,
AttemptsDay: mc.attemptsDay,
FailuresDay: mc.failuresDay,
}
}
providers[i] = adminProviderView{