2 Commits

Include renames

Author	SHA1	Message	Date
lukaszraczylo	d45125ee07	fix(failover): default off, add cost guard, fix adversarial-review findings ... Flip FailoverConfig.Enabled to default false. A version upgrade with no config change must preserve prior behaviour, and failover changes which provider serves a request. Update tests, README, and the yaegi-check probe to opt in explicitly. Add a mandatory cost guard. A candidate is skipped unless its input AND output cost per token are each no greater than the primary candidate's, resolved through the same canonical-then-bare pricing path the real billing code uses. Unknown pricing is never treated as free: a known primary against an unknown candidate skips, an unknown primary against a known non-zero candidate skips, everything else that is equal is allowed. Not configurable. Fix four blocking findings from adversarial review of commit 5d62479. F1: openai-type adapters mutate the request map in place to force stream usage reporting. Reusing one map across candidates let a later candidate read back an earlier candidate's own injection and leak a usage-only SSE chunk to a client that never asked for it. Every candidate now attempts against a fresh copy of the original request. F2: a streaming candidate that fails before writing anything still stages SSE-only response headers. Those headers now get cleared before every candidate attempt, so they cannot survive into a later candidate's plain JSON response. F3: the request-health gate used the retry-shaped isTransient classification, which treats a non-429 4xx as a successful outcome. A provider with a dead API key was never skipped and was retried forever. The gate now reuses failoverEligible's own classification instead. F4: a shared request context that an earlier candidate exhausted was attributed to whichever candidate happened to run next, opening a healthy provider's breaker for a timeout that was not its fault. A candidate is no longer attempted once the shared context is already done; the loop stops and reports the last real error instead. Also fix three should-fix findings from the same review. F5: a failure arriving while the request-health breaker is already open but not yet past its cooldown window is now a no-op, instead of resetting backoff and sliding the open window forward on every such failure. F6: the request-health gate now records at most one outcome per candidate per logical request, not one per same-provider retry attempt, so retry.attempts alone can no longer open the breaker on a single request. F7: every candidate's cache entry is now checked before any candidate is attempted, so a cached failover response no longer costs a round trip to a still-broken primary first. F10: the generic "failing over" log line is now rate-limited to once per 30 seconds during a sustained outage, reusing the existing logGate type. The 404 loud-log line stays unthrottled, as ruled. F9 (optional): skip building the failover candidate pool entirely when failover is disabled. Correct an overstated test comment on TestHandleChat_Failover_DiscoveryUnhealthy_StillTried: it proves alpha is still tried, not that ordering places it second. Document, but do not fix here, that this plugin's HTTP client sets no per-attempt timeout, so a provider that accepts a connection and then hangs will never trigger failover. This is a pre-existing property of the shared HTTP client, not specific to failover. Deferred: F8 (surfacing request-health state on the admin dashboard) touches admin.go, which a sibling branch is also editing concurrently; left for a follow-up to avoid a merge conflict in a file this change does not otherwise need. Every fix above is proven by a dedicated regression test, and each was verified by reverting the fix in place and confirming the corresponding test fails before restoring it.	2026-08-23 01:18:58 +01:00
lukaszraczylo	5d624791fa	feat(failover): add cross-provider failover for overlapping models ... Add failover.go with the full candidate pipeline. resolveWithCandidates resolves a request exactly like resolve does, then adds every other provider sharing the identical bare model id. orderedFailoverCandidates applies a hard skip for request-path-unhealthy providers, deprioritizes discovery-unhealthy providers without dropping them, fails open when every candidate reads unhealthy, and caps the list at maxAttempts. Add requestHealthTracker, a separate per-pod request-path health signal fed by the same isTransient/isDeadlineExceeded classification recordProviderAttempt already uses. It never widens the existing discovery circuit breaker, per that breaker's own documented scope. Rewrite runMeteredCall (routes_unified.go) to loop over candidates instead of resolving once. Each attempt rebinds the attempt recorder to its own provider, recomputes the cache key before the response is cached, and re-injects the alias echo key an adapter may have deleted on a prior failed attempt. Failover never runs once any byte has reached the client, since statusTrackingWriter already tracks that state. A failure falls through on everything except HTTP 400, matching the brief exactly. A 404 failover logs loudly, naming both providers and the model, since it usually signals a real configuration mistake. Add FailoverConfig (Config.Failover): an Enabled pointer defaulting to true, and a MaxAttempts bound between 1 and 10, defaulting to 3. A single-provider deployment, or one with no overlapping bare model ids, behaves exactly as before regardless of this setting. Add a yaegi-check probe (tools/yaegi-check/main.go) driving two providers under the real interpreter, one failing and one serving, proving the candidate loop runs correctly interpreted. Update README.md with a Failover section and correct the Retry section's stale claim that no cross-provider failover exists. Media routes (images, audio speech, audio transcriptions) do not get failover in this change. Each has its own request-shape handling that needs its own scoped follow-up.	2026-08-23 00:32:12 +01:00