fix(failover): default off, add cost guard, fix adversarial-review findings

Flip FailoverConfig.Enabled to default false. A version upgrade with no
config change must preserve prior behaviour, and failover changes which
provider serves a request. Update tests, README, and the yaegi-check
probe to opt in explicitly.

Add a mandatory cost guard. A candidate is skipped unless its input AND
output cost per token are each no greater than the primary candidate's,
resolved through the same canonical-then-bare pricing path the real
billing code uses. Unknown pricing is never treated as free: a known
primary against an unknown candidate skips, an unknown primary against a
known non-zero candidate skips, everything else that is equal is
allowed. Not configurable.

Fix four blocking findings from adversarial review of commit 5d62479.

F1: openai-type adapters mutate the request map in place to force
stream usage reporting. Reusing one map across candidates let a later
candidate read back an earlier candidate's own injection and leak a
usage-only SSE chunk to a client that never asked for it. Every
candidate now attempts against a fresh copy of the original request.

F2: a streaming candidate that fails before writing anything still
stages SSE-only response headers. Those headers now get cleared before
every candidate attempt, so they cannot survive into a later
candidate's plain JSON response.

F3: the request-health gate used the retry-shaped isTransient
classification, which treats a non-429 4xx as a successful outcome. A
provider with a dead API key was never skipped and was retried forever.
The gate now reuses failoverEligible's own classification instead.

F4: a shared request context that an earlier candidate exhausted was
attributed to whichever candidate happened to run next, opening a
healthy provider's breaker for a timeout that was not its fault. A
candidate is no longer attempted once the shared context is already
done; the loop stops and reports the last real error instead.

Also fix three should-fix findings from the same review.

F5: a failure arriving while the request-health breaker is already open
but not yet past its cooldown window is now a no-op, instead of
resetting backoff and sliding the open window forward on every such
failure.

F6: the request-health gate now records at most one outcome per
candidate per logical request, not one per same-provider retry attempt,
so retry.attempts alone can no longer open the breaker on a single
request.

F7: every candidate's cache entry is now checked before any candidate
is attempted, so a cached failover response no longer costs a round
trip to a still-broken primary first.

F10: the generic "failing over" log line is now rate-limited to once
per 30 seconds during a sustained outage, reusing the existing logGate
type. The 404 loud-log line stays unthrottled, as ruled.

F9 (optional): skip building the failover candidate pool entirely when
failover is disabled.

Correct an overstated test comment on
TestHandleChat_Failover_DiscoveryUnhealthy_StillTried: it proves alpha
is still tried, not that ordering places it second.

Document, but do not fix here, that this plugin's HTTP client sets no
per-attempt timeout, so a provider that accepts a connection and then
hangs will never trigger failover. This is a pre-existing property of
the shared HTTP client, not specific to failover.

Deferred: F8 (surfacing request-health state on the admin dashboard)
touches admin.go, which a sibling branch is also editing concurrently;
left for a follow-up to avoid a merge conflict in a file this change
does not otherwise need.

Every fix above is proven by a dedicated regression test, and each was
verified by reverting the fix in place and confirming the corresponding
test fails before restoring it.
This commit is contained in:
2026-08-23 01:18:58 +01:00
parent 5d624791fa
commit d45125ee07
6 changed files with 1174 additions and 126 deletions
+11 -2
View File
@@ -545,6 +545,14 @@ type Gateway struct {
// rather than a nil-pointer panic.
failoverHealth *requestHealthTracker
name string
// failoverLogGate rate-limits runMeteredCall's generic "failing over"
// log line (routes_unified.go) to once per storeErrorLogEvery
// (adversarial-review fix, F10) — reuses auth.go's own logGate type,
// the identical technique shouldLogAuthFailure/logStoreError already
// apply elsewhere in this package. Zero-value-usable: the first call
// on a fresh Gateway always logs. Deliberately does NOT gate the 404
// loud-log line (operator ruling: that one must always log).
failoverLogGate logGate
// failover is Config.Failover, validated and resolved once by
// newGateway (validateFailoverConfig, failover.go).
failover failoverConfig
@@ -657,8 +665,9 @@ func newGateway(ctx context.Context, next http.Handler, config *Config, name str
g := &Gateway{next: next, name: name, cfg: config, auth: auth}
// feat/failover: validated once, here — see FailoverConfig's own doc
// comment (failover.go) for the default (enabled) and what "single-
// provider deployment behaves exactly as before" means in practice.
// comment (failover.go) for the default (disabled, coordinator
// ruling) and what "single-provider deployment behaves exactly as
// before" means in practice.
failoverCfg, err := validateFailoverConfig(config.Failover)
if err != nil {
return nil, err