Implements Task 12: POST /v1/chat/completions and POST /v1/embeddings
through a shared pipeline (routes_unified.go: handleChat, handleEmbeddings,
runUnified) wired into ServeHTTP's dispatch in llmgateway.go.
Pipeline: decode (10MiB cap) -> resolve model -> enforce user/group
limits -> call adapter -> account usage, translating every failure mode
into its HTTP envelope per the task brief's carried rulings:
- ALIAS ECHO (ruling a): the client-requested model id is threaded to the
adapter via the internal gatewayAliasKey convention and echoed back in
anthropic/gemini's translated response model field; every adapter
strips the key before building its upstream wire body so it never
leaks to a real provider.
- Limit violations (ruling b): storeDown -> 503 server_error; otherwise
429 rate_limit_error with Retry-After when the violation names a
window.
- Stream-safe error handling (ruling c): statusTrackingWriter.wroteHeader
gates whether an adapter error still gets an envelope; providerHTTPError
wraps into {error:{message,type,code,upstream}}; translateError maps to
400/501; context.Canceled logs quietly; anything else is a 502.
- Usage/accounting (ruling d): zero non-streaming usage gets a
ceil(len(body)/4) prompt-token estimate; zero streaming usage logs once
and accounts the request only. unifiedCostMicros extends costMicros'
single-id lookup to canonical-then-bare without changing its signature.
- Scopes (ruling e): user scope only when u.limits is set, group scope
only when grp.limits is set, user first.
routes_unified_test.go drives every case end-to-end through
Gateway.ServeHTTP against httptest fake upstreams (openai- and
anthropic-shaped): happy path with counters asserted via the limiter's
own store, incremental stream flushing, 401/403/404/429/503, embeddings,
alias echo through a provider-prefixed anthropic request, wrapped
upstream error envelopes, and a mid-stream drop that must not append a
second envelope to an already-committed response.
Gates: gofmt, go vet, fieldalignment, golangci-lint (incl. gosec) all
clean; go test ./... -race -count=1 passes.
1. refreshProvider (the plugin's only background goroutine) had no
recover(): a panicking adapter's listModels would crash the whole
Traefik process. Restructured around deferred recover + finishRefresh,
so a panic is turned into a logged error, inFlight always releases, and
the next interval retries. Documented as a scoped exception to
limits.go's no-background-goroutine rule.
2. newGateway wrapped ctx in an outer 5s WithTimeout before warmFill,
but warmFill already derives a fresh 5s budget per provider from
whatever ctx it receives. The outer wrap made that a single budget
shared across every discovery-enabled provider instead of a per-provider
one - providers after the first could get a near-expired context and
fail instantly. Dropped the outer wrapper; construction's worst case is
now N * 5s, accepted.
3. group.allowsModel stripped at any first slash, so a glob like
'deepseek-*' could spuriously match a bare model id that merely
contains a slash (e.g. an upstream's own 'uni/deepseek-v4-flash-0731'
naming) even though the leading segment names no configured provider.
allowsModel is now an exact glob with no internal splitting;
resolveAgainst and listFor generate the provider-prefixed and
bare-suffix candidates themselves, only when a prefix legitimately
names a configured provider. Updated the Task-3 TestIdentify assertion
to match.
4. Annotated the discarded json.Encode error in handleModels.
Gate: go test ./... -count=1, -race, gofmt, vet, fieldalignment,
golangci-lint (incl gosec) all clean.
Adds modelRegistry: per-provider explicit+discovered model sets, bare-id
and provider-prefixed resolution with sorted-name collision precedence,
group authorization pairing (allowsModel AND allowsProvider), and a
group-filtered OpenAI-compatible /v1/models catalog.
Discovery is lazy and throttled per provider: newGateway runs a
synchronous 5s-bounded warm fill for discovery-enabled providers
(non-fatal on error), and Gateway.ServeHTTP triggers a throttled,
inFlight-guarded background refresh per provider per interval
(default 1h, per-provider DiscoveryInterval parsed at construction).
A failed refresh keeps the previous discovered set (stale-while-error).
Wires GET /v1/models into ServeHTTP (auth required, 401 envelope
otherwise) and gives Gateway its adapters/registry fields, both built
in newGateway.
newModelRegistry's signature gained an error return beyond the task
brief's literal listing: an invalid DiscoveryInterval must be a
constructor error per the controller ruling, which the brief's bare
*modelRegistry signature could not express.
lukaszraczylo