Model discovery used each API's default page size: 20 for Anthropic and
50 for Gemini. A full catalog needed several sequential requests, and at
pod start that overran the 5s warm-fill budget. One replica then
logged "initial discovery ... context deadline exceeded" and went
without that provider's models until the next probe. Every page now
asks for the documented maximum of 1000: Anthropic limit and Gemini
pageSize. Gemini gets pageSize on each page, because it requires the
same parameters across one listing.
Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
Billing:
- Passthrough SSE streams now bill the usage the upstream reports.
The gateway falls back to a request-size estimate only on a 2xx
stream or a truncated JSON body.
- A unified stream that fails after it starts is estimated, not
billed at zero.
- Billing uses the built-in LiteLLM price table and honours
modelMeta free: true.
- Pricing entries are validated. Cost arithmetic rounds the total and
saturates on overflow.
- Gemini thinking tokens count as completion tokens.
Limits and storage:
- A rejected request rolls back its counts on the other scopes, so one
user cannot lock out a group.
- Redis EXPIRE replies are checked. Keys recreated without a TTL are
repaired at once.
- A mutating pipeline is not retried after reply bytes arrive.
- History, model totals and target usage report a store outage.
Routing and translation:
- Discovered ids with a provider-like prefix are listed only in
owner/id form, so listing and routing agree.
- Anthropic and Gemini discovery follows pagination.
- The developer role maps to system.
- stream_options.include_usage is honoured on translated streams.
- tool_choice none keeps tools when history holds tool blocks.
- refusal maps to content_filter. Data URI parameters are stripped.
Streaming, retry and failover:
- SSE accepts CR, LF and CRLF line ends. This closes a CR injection
path.
- Retry-After is never jittered. A cancelled wait returns the
context error.
- The watchdog timer stops at end of body.
- Oversized bodies return 413.
- A failed stream ends with an error event.
- Failover restores pre-existing headers. Client cancels do not count
as provider successes.
Also: "admin" is a reserved provider name. metrics.path must start
with "/". Pricing warnings go to the right gateway instance. The admin
targets view sends access as null or [] so the two states differ.
Upstream organisation headers are stripped. The users-file reload
error is logged once per error. Dead code is removed, and test-only
helpers move to test files.
Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
Lifts all 14 functions the operator flagged as <75% (measured at ed2119c)
to >=75%, using table-driven tests (require for fatals, assert for checks):
- limits.go: failPolicyGet (50.0% -> 100.0%)
- provider_gemini.go: httpClient (0.0% -> 100.0%)
- redis_store.go: get (71.4% -> 78.6%)
- resp.go: dialTimeoutFor, respDeadlineExceededErr.{Error,Timeout,Temporary},
ensureConnLocked, getBatch (66.7-30.0% -> 91.3-100.0%)
- routes_media.go: resolveMediaRequest (50.0% -> 100.0%)
- routes_unified.go: unifiedCostMicros (66.7% -> 100.0%)
- translate_anthropic.go: isTruthy, toFloat64 (50.0/33.3% -> 100.0%)
- translate_gemini.go: geminiToolConfigFromOpenAI (66.7% -> 100.0%)
resp_test.go adds a hand-rolled fakeConn (net.Conn with injectable
Write/SetDeadline errors) to reach two respClient error branches a real
TCP connection cannot be coaxed into deterministically.
Total coverage: 90.7% -> 92.4% (no regression; report has the full
per-file table). Zero production-code changes. Existing tests untouched:
every diff is a pure addition of new test functions and their imports.
Gates: go test -count=1 (pass), -race (pass, 5x incl. an isolated
baseline-vs-branch comparison after one flaky run implicated pre-existing
registry_test.go timing tests this task never touched), gofmt/vet (clean),
golangci-lint incl. gosec and fieldalignment (0 issues), make yaegi-check
(OK), make integration (pass, untouched-green).
lukaszraczylo