mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-14 05:18:54 +00:00
fix(providers): deadline failures, cheaper polling, async writes
Review round (v0.22): 1 must-fix + 5 should-fix. This covers the three backend items. MUST-1's WCAG fix lives in the webui commit; this is the three provider/ admin-side SHOULD items. SHOULD-1 (timeout blindness): a hung upstream previously read as 100% healthy — isTransient (retry.go) deliberately excludes context.DeadlineExceeded from what it retries (retrying past the gateway's own deadline is pointless), and recordProviderAttempt reused that same classification unforked, so a timed-out attempt recorded no failure. Fixed narrowly in recordProviderAttempt only, per the ruling: isTransient itself stays untouched and shared. A new isDeadlineExceeded adds exactly one extra condition — a context.Canceled error (the client walked away) still counts as an attempt only, from both functions alike. The sentinel match (matchesSentinel) is a hand-rolled errors.Unwrap walk, not errors.Is: providerHTTPError's own doc comment already documents that errors.As panics under Yaegi checking whether an interpreted pointer type implements error; errors.Is walks the identical internal machinery and carries the same risk, so this sidesteps it with a bare `err.(sentinelUnwrapper)` comma-ok assertion instead — plain interface satisfaction, not reflection over a possibly-interpreted type. SHOULD-2 (poll fan-out): a (provider, model) scope's minute-window counters fed nothing but a badge's title text on the Providers tab, so a dashboard with N configured models paid N wasted minute-window reads on every 5s poll. providerCounterKeys/providerUsage now read a kindProviderModel scope's day window only (2 keys, down from 4); a kindProvider scope is unaffected (still 4 — one scope per provider, not per model, so the read amplification never applied there). adminModelRateView drops AttemptsMinute/FailuresMinute entirely, not just unpopulated fields — the webui side (separate commit) falls back to a day-window-only title for a per-model badge. SHOULD-5 (TTFB): recordProviderAttempt's store write is telemetry-only — nothing in the request path gates on it — so it now runs via a new limiter.spawn dependency-injection field (mirrors nowFn/waitFn's existing pattern), defaulting to a real goroutine in production and overridden to run synchronously in every test that asserts on the resulting counter immediately afterward. This removes the synchronous Redis round trip this counter family added to the streaming TTFB path entirely. Loss on process shutdown is accepted (telemetry only).
This commit is contained in:
@@ -54,6 +54,14 @@ func newAdminGatewayHandle(t *testing.T, cfg *Config) (http.Handler, *Gateway) {
|
||||
if !ok {
|
||||
t.Fatal("handler is not *Gateway")
|
||||
}
|
||||
// SHOULD-5 (v0.22 review round): production spawns recordProviderAttempt's
|
||||
// store write in its own goroutine, off a request's TTFB path — every
|
||||
// admin_test.go test that asserts on a resulting provider counter (GET
|
||||
// /admin/api/overview's Attempts*/Failures* fields) needs it to have
|
||||
// already landed, so spawn runs synchronously here rather than per-test
|
||||
// (the deterministic-test half of that dependency-injection field; see
|
||||
// limiter.spawn's own doc comment, limits.go).
|
||||
gw.limiter.spawn = func(f func()) { f() }
|
||||
return h, gw
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user