mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-16 05:37:28 +00:00
fix(gateway): close billing, limit, routing and translation gaps
Billing: - Passthrough SSE streams now bill the usage the upstream reports. The gateway falls back to a request-size estimate only on a 2xx stream or a truncated JSON body. - A unified stream that fails after it starts is estimated, not billed at zero. - Billing uses the built-in LiteLLM price table and honours modelMeta free: true. - Pricing entries are validated. Cost arithmetic rounds the total and saturates on overflow. - Gemini thinking tokens count as completion tokens. Limits and storage: - A rejected request rolls back its counts on the other scopes, so one user cannot lock out a group. - Redis EXPIRE replies are checked. Keys recreated without a TTL are repaired at once. - A mutating pipeline is not retried after reply bytes arrive. - History, model totals and target usage report a store outage. Routing and translation: - Discovered ids with a provider-like prefix are listed only in owner/id form, so listing and routing agree. - Anthropic and Gemini discovery follows pagination. - The developer role maps to system. - stream_options.include_usage is honoured on translated streams. - tool_choice none keeps tools when history holds tool blocks. - refusal maps to content_filter. Data URI parameters are stripped. Streaming, retry and failover: - SSE accepts CR, LF and CRLF line ends. This closes a CR injection path. - Retry-After is never jittered. A cancelled wait returns the context error. - The watchdog timer stops at end of body. - Oversized bodies return 413. - A failed stream ends with an error event. - Failover restores pre-existing headers. Client cancels do not count as provider successes. Also: "admin" is a reserved provider name. metrics.path must start with "/". Pricing warnings go to the right gateway instance. The admin targets view sends access as null or [] so the two states differ. Upstream organisation headers are stripped. The users-file reload error is logged once per error. Dead code is removed, and test-only helpers move to test files. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
This commit is contained in:
@@ -849,6 +849,53 @@ func TestWatchdogBody_ZeroByteRead_DoesNotResetTimer(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// eofReadCloser always returns (0, io.EOF), simulating an upstream body
|
||||
// that has already been fully, successfully read — used by
|
||||
// TestWatchdogBody_Read_EOF_StopsTimer_FireNeverRuns (finding 11) to
|
||||
// prove Read stops the watchdog timer itself, rather than waiting for
|
||||
// the caller's own, possibly much later, Close call.
|
||||
type eofReadCloser struct{}
|
||||
|
||||
func (eofReadCloser) Read([]byte) (int, error) { return 0, io.EOF }
|
||||
func (eofReadCloser) Close() error { return nil }
|
||||
|
||||
// TestWatchdogBody_Read_EOF_StopsTimer_FireNeverRuns proves finding 11
|
||||
// (review-routes.md): a Read returning io.EOF must stop the watchdog
|
||||
// timer immediately, not leave it armed until the caller's eventual
|
||||
// Close. Without this, a caller that reads the upstream body to
|
||||
// completion and then spends time WRITING it out to a slow client
|
||||
// (forwardJSON copies up to 32MiB before its deferred Close runs) races
|
||||
// the still-armed timer, which can fire and record a spurious provider
|
||||
// failure for an upstream that, in fact, finished cleanly. Close is
|
||||
// deliberately NOT called until after the assertion below, mirroring
|
||||
// TestWatchdogBody_Close_StopsTimer_FireNeverRuns' own "prove it without
|
||||
// relying on Close" shape.
|
||||
//
|
||||
// MUTATION VERIFIED: removing the `if err == io.EOF { ...; wb.timer.
|
||||
// Stop() }` branch from watchdogBody.Read (timeout.go) made this test
|
||||
// fail — wb.timedOut read back as 1 after the sleep, since the un-stopped
|
||||
// timer still fired on schedule.
|
||||
func TestWatchdogBody_Read_EOF_StopsTimer_FireNeverRuns(t *testing.T) {
|
||||
const timeout = 30 * time.Millisecond
|
||||
frc := eofReadCloser{}
|
||||
_, cancel := context.WithCancel(context.Background())
|
||||
wb := newWatchdogBody(frc, cancel, timeout, "p1", nil)
|
||||
|
||||
if _, err := wb.Read(make([]byte, 16)); err != io.EOF { //nolint:errorlint // io.EOF returned bare by the fixture above
|
||||
t.Fatalf("Read: err = %v, want io.EOF", err)
|
||||
}
|
||||
if got := atomic.LoadInt32(&wb.done); got != 1 {
|
||||
t.Fatalf("wb.done = %d after a Read returning io.EOF, want 1", got)
|
||||
}
|
||||
|
||||
time.Sleep(3 * timeout) // well past when an un-stopped timer would fire
|
||||
|
||||
if got := atomic.LoadInt32(&wb.timedOut); got != 0 {
|
||||
t.Errorf("wb.timedOut = %d after a Read returning io.EOF, want 0 — the timer must be stopped at EOF, not left armed until Close", got)
|
||||
}
|
||||
_ = wb.Close()
|
||||
}
|
||||
|
||||
// TestWatchdogBody_TimeoutError_UsesCallerLabel_NotHardcodedProvider
|
||||
// proves finding F9 (coordinator adversarial review, 2026-08-23):
|
||||
// proxyUpstream (routes_passthrough.go) previously passed logPrefix as a
|
||||
|
||||
Reference in New Issue
Block a user