fix(gateway): close billing, limit, routing and translation gaps

Billing:
- Passthrough SSE streams now bill the usage the upstream reports.
  The gateway falls back to a request-size estimate only on a 2xx
  stream or a truncated JSON body.
- A unified stream that fails after it starts is estimated, not
  billed at zero.
- Billing uses the built-in LiteLLM price table and honours
  modelMeta free: true.
- Pricing entries are validated. Cost arithmetic rounds the total and
  saturates on overflow.
- Gemini thinking tokens count as completion tokens.

Limits and storage:
- A rejected request rolls back its counts on the other scopes, so one
  user cannot lock out a group.
- Redis EXPIRE replies are checked. Keys recreated without a TTL are
  repaired at once.
- A mutating pipeline is not retried after reply bytes arrive.
- History, model totals and target usage report a store outage.

Routing and translation:
- Discovered ids with a provider-like prefix are listed only in
  owner/id form, so listing and routing agree.
- Anthropic and Gemini discovery follows pagination.
- The developer role maps to system.
- stream_options.include_usage is honoured on translated streams.
- tool_choice none keeps tools when history holds tool blocks.
- refusal maps to content_filter. Data URI parameters are stripped.

Streaming, retry and failover:
- SSE accepts CR, LF and CRLF line ends. This closes a CR injection
  path.
- Retry-After is never jittered. A cancelled wait returns the
  context error.
- The watchdog timer stops at end of body.
- Oversized bodies return 413.
- A failed stream ends with an error event.
- Failover restores pre-existing headers. Client cancels do not count
  as provider successes.

Also: "admin" is a reserved provider name. metrics.path must start
with "/". Pricing warnings go to the right gateway instance. The admin
targets view sends access as null or [] so the two states differ.
Upstream organisation headers are stripped. The users-file reload
error is logged once per error. Dead code is removed, and test-only
helpers move to test files.

Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
This commit is contained in:
2026-09-23 09:44:31 +01:00
parent b3b8ae6ad5
commit 4c01c75276
62 changed files with 6064 additions and 1182 deletions
+47
View File
@@ -849,6 +849,53 @@ func TestWatchdogBody_ZeroByteRead_DoesNotResetTimer(t *testing.T) {
}
}
// eofReadCloser always returns (0, io.EOF), simulating an upstream body
// that has already been fully, successfully read — used by
// TestWatchdogBody_Read_EOF_StopsTimer_FireNeverRuns (finding 11) to
// prove Read stops the watchdog timer itself, rather than waiting for
// the caller's own, possibly much later, Close call.
type eofReadCloser struct{}
func (eofReadCloser) Read([]byte) (int, error) { return 0, io.EOF }
func (eofReadCloser) Close() error { return nil }
// TestWatchdogBody_Read_EOF_StopsTimer_FireNeverRuns proves finding 11
// (review-routes.md): a Read returning io.EOF must stop the watchdog
// timer immediately, not leave it armed until the caller's eventual
// Close. Without this, a caller that reads the upstream body to
// completion and then spends time WRITING it out to a slow client
// (forwardJSON copies up to 32MiB before its deferred Close runs) races
// the still-armed timer, which can fire and record a spurious provider
// failure for an upstream that, in fact, finished cleanly. Close is
// deliberately NOT called until after the assertion below, mirroring
// TestWatchdogBody_Close_StopsTimer_FireNeverRuns' own "prove it without
// relying on Close" shape.
//
// MUTATION VERIFIED: removing the `if err == io.EOF { ...; wb.timer.
// Stop() }` branch from watchdogBody.Read (timeout.go) made this test
// fail — wb.timedOut read back as 1 after the sleep, since the un-stopped
// timer still fired on schedule.
func TestWatchdogBody_Read_EOF_StopsTimer_FireNeverRuns(t *testing.T) {
const timeout = 30 * time.Millisecond
frc := eofReadCloser{}
_, cancel := context.WithCancel(context.Background())
wb := newWatchdogBody(frc, cancel, timeout, "p1", nil)
if _, err := wb.Read(make([]byte, 16)); err != io.EOF { //nolint:errorlint // io.EOF returned bare by the fixture above
t.Fatalf("Read: err = %v, want io.EOF", err)
}
if got := atomic.LoadInt32(&wb.done); got != 1 {
t.Fatalf("wb.done = %d after a Read returning io.EOF, want 1", got)
}
time.Sleep(3 * timeout) // well past when an un-stopped timer would fire
if got := atomic.LoadInt32(&wb.timedOut); got != 0 {
t.Errorf("wb.timedOut = %d after a Read returning io.EOF, want 0 — the timer must be stopped at EOF, not left armed until Close", got)
}
_ = wb.Close()
}
// TestWatchdogBody_TimeoutError_UsesCallerLabel_NotHardcodedProvider
// proves finding F9 (coordinator adversarial review, 2026-08-23):
// proxyUpstream (routes_passthrough.go) previously passed logPrefix as a