fix(gateway): close billing, limit, routing and translation gaps

Billing:
- Passthrough SSE streams now bill the usage the upstream reports.
  The gateway falls back to a request-size estimate only on a 2xx
  stream or a truncated JSON body.
- A unified stream that fails after it starts is estimated, not
  billed at zero.
- Billing uses the built-in LiteLLM price table and honours
  modelMeta free: true.
- Pricing entries are validated. Cost arithmetic rounds the total and
  saturates on overflow.
- Gemini thinking tokens count as completion tokens.

Limits and storage:
- A rejected request rolls back its counts on the other scopes, so one
  user cannot lock out a group.
- Redis EXPIRE replies are checked. Keys recreated without a TTL are
  repaired at once.
- A mutating pipeline is not retried after reply bytes arrive.
- History, model totals and target usage report a store outage.

Routing and translation:
- Discovered ids with a provider-like prefix are listed only in
  owner/id form, so listing and routing agree.
- Anthropic and Gemini discovery follows pagination.
- The developer role maps to system.
- stream_options.include_usage is honoured on translated streams.
- tool_choice none keeps tools when history holds tool blocks.
- refusal maps to content_filter. Data URI parameters are stripped.

Streaming, retry and failover:
- SSE accepts CR, LF and CRLF line ends. This closes a CR injection
  path.
- Retry-After is never jittered. A cancelled wait returns the
  context error.
- The watchdog timer stops at end of body.
- Oversized bodies return 413.
- A failed stream ends with an error event.
- Failover restores pre-existing headers. Client cancels do not count
  as provider successes.

Also: "admin" is a reserved provider name. metrics.path must start
with "/". Pricing warnings go to the right gateway instance. The admin
targets view sends access as null or [] so the two states differ.
Upstream organisation headers are stripped. The users-file reload
error is logged once per error. Dead code is removed, and test-only
helpers move to test files.

Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
This commit is contained in:
2026-09-23 09:44:31 +01:00
parent b3b8ae6ad5
commit 4c01c75276
62 changed files with 6064 additions and 1182 deletions
+64
View File
@@ -100,6 +100,70 @@ func resolveModelMeta(provider, model string, cfg map[string]*ModelMetaConfig, d
return out
}
// priceKnown reports whether model — given as both its canonical
// "provider/model" id and its bare upstream id, canonical checked first,
// the same two-id order routes_unified.go's own modelPriceKnown/
// unifiedCostMicrosKnown already check — has a KNOWN price for billing
// purposes: a configured pricing override or the built-in tables
// (lookupPricing, pricing.go — builtinPricing then the generated
// builtinModelMetaTable), OR an operator's own modelMeta entry that marks
// the model explicitly Free (review-auth finding F1, 2026-09 audit:
// modelPriceKnown used to consult only pricing overrides and the built-in
// table, so a model an operator declared modelMeta:{free:true} — the
// common case for a self-hosted/local model with no real cost — was still
// reported "unpriced" and refused by the AllowUnpricedWithCostBudget cost
// guard even though its price, zero, is perfectly well known).
//
// meta is checked by the SAME two key shapes lookupModelMetaConfig itself
// checks (an exact "provider/model" key, then a bare "model" key) — done
// here as plain map lookups rather than a resolveModelMeta call, since
// resolveModelMeta also resolves context-window/discovery data this
// cost-only check has no use for, and canonical here already IS the
// joined "provider/model" string, not a separate (provider, model) pair
// resolveModelMeta could split it back into.
//
// This does not change AllowUnpricedWithCostBudget's own default — it
// only widens what counts as "known", so a model already treated as
// priced stays priced, and a model an operator has explicitly marked free
// now also counts as priced, rather than being refused as though it had
// no price on record at all.
func priceKnown(canonical, bare string, overrides map[string]*ModelPricing, meta map[string]*ModelMetaConfig) bool {
if _, ok := lookupPricing(canonical, overrides); ok {
return true
}
if _, ok := lookupPricing(bare, overrides); ok {
return true
}
return modelMetaFree(canonical, bare, meta)
}
// modelMetaFree reports whether the operator modelMeta entry for
// canonical (or, absent that, bare) marks it Free — the SAME two-key
// lookup priceKnown itself performs (an exact "provider/model" key, then
// a bare "model" key), factored out so billing (unifiedCostMicros, routes_unified.go)
// and the 402 cost-budget guard (priceKnown, above) can never disagree
// about which models are free (verify-core fix, round 4). Before this,
// priceKnown alone treated a modelMeta free:true model as priced, but
// unifiedCostMicros never consulted modelMeta at all, so a self-hosted
// id an operator declared free that also happened to match a bare id in
// the generated LiteLLM table (gpt-oss-120b, llama-3.3-70b, and other
// common local-model names) was billed that table's real price instead
// of 0 — passing the 402 guard while silently charging for it anyway.
//
// Precedence matches the display path (lookupModelMetaConfig): an exact
// "provider/model" entry decides on its own; the bare entry is consulted
// only when no provider/model entry exists, so /v1/models and billing
// never disagree about whether a model is free.
func modelMetaFree(canonical, bare string, meta map[string]*ModelMetaConfig) bool {
if m, ok := meta[canonical]; ok && m != nil {
return m.Free
}
if m, ok := meta[bare]; ok && m != nil {
return m.Free
}
return false
}
// lookupModelMetaConfig resolves cfg's config-override layer for
// (provider, model): an exact "provider/model" key wins over a bare
// "model" key, matching this feature's own spec precedence (a key names