mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
lukaszraczylo
3ed473bdfe
Review fixes for Task 12: 1. (CRITICAL) runUnified now calls g.limiter.account BEFORE handling an adapter error, not after returning on it. All three adapters' forwardStream deliberately return captured usage alongside a mid-stream error, so a client that aborts right after the usage frame arrives was getting tokens delivered but never billed - repeated aborts bypassed every token/cost budget. A non-streaming error path always carries zero usage by contract, so accounting it is a no-op (account skips writes once both total tokens and cost are zero). The zero-usage-stream logf and the zero-usage-non-stream estimate now only fire when callErr is nil, so a genuine error no longer gets a fabricated token estimate. New test: TestHandleChat_MidStreamDropAfterUsageChunk_AccountsPartialUsage (hijack+close after the usage chunk; asserts the limiter's counter shows the captured tokens). Verified this test fails against the pre-fix ordering and the two new alias tests below fail when their guarded delete() call is removed, before restoring and re-running clean. 2. Added TestHandleChat_OpenAI_AliasKeyStrippedFromUpstreamBody and TestHandleEmbeddings_OpenAI_AliasKeyStrippedFromUpstreamBody: the openai-type adapter marshals req verbatim as the upstream wire body, so it is the one path where a leftover gatewayAliasKey entry actually reaches a real provider. The only existing alias-leak assertion covered anthropic, which rebuilds the body field-by-field and was never at risk. 3. Reworded the stale comment on limiter.evaluateScope (limits.go): it described a nil-limits scope's req counters as a normal, expected case. Ruling (e), implemented in routes_unified.go's buildLimitScopes, means the unified route never builds a nil-limits scope in the first place - the nil check remains a defensive no-op, not a path any current caller exercises. Gates: gofmt, go vet, fieldalignment, golangci-lint (incl. gosec) all clean; go test ./... -race -count=1 passes (full suite, including all Task 12 tests).