Spec §5 (v0.2): Config.ModelAliases map[string]string, alias-first
resolution precedence ahead of provider-prefix splitting, alias-OR-target
authorization, lazy target resolution with a 404 naming both ids,
/v1/models listing, admin overview alias table (JSON + dashboard), and
construction-time validation (shape, prefix-shadow, explicit-model
collision, chain, empty target).
routes_unified.go/routes_media.go needed no resolution-call-site changes
- both already pass the client's raw requested id through to
registry.resolve, so aliases inherit the __alias echo and multipart
rewrite machinery for free. cache.go needed no changes either - cacheKey
already hashes requestedModel, so two aliases resolving to the same
target already produce distinct cache entries.
Adds registry_test.go validation table + precedence/authz/lazy-404/
listing coverage, a dedicated cache-key split test, chat/embeddings/
images/audio-transcriptions end-to-end tests, an extended admin overview
test, and an integration test covering the openai-type vs anthropic-type
echo asymmetry under real Traefik+Yaegi.
C1 (quota bypass): a client sending stream_options.include_usage=false
suppressed upstream usage reporting and escaped every token/cost budget,
since the non-streaming T12 estimate never covers a streaming response.
chatCompletion (provider_openai.go) now always forces include_usage=true
on the outgoing request, merged into a copy of the client's own
stream_options map so the caller's map is never mutated in place.
clientAskedUsage is derived from the client's own value via isTruthy, so
the usage-only chunk is still suppressed for a client that never asked
for it.
I1 (vacuous security tests): the X-Api-Key credential-strip set had zero
effective coverage — existing tests either never sent x-api-key, or
targeted anthropic, whose injectAuth overwrites x-api-key regardless of
whether the strip ran. Added one openai-passthrough and one MCP-proxy
test that authenticate via x-api-key itself and assert it arrives empty
upstream. Mutation-checked: both fail when "X-Api-Key": true is removed
from gatewayCredentialHeaders (verified, then restored).
I3 (auth-event logging): added Gateway.logAuthEvent (logger.go), called
at all six authStore.identify call sites (llmgateway.go's ServeHTTP,
routes_unified.go's handleModels). A failed attempt logs method, path,
and remote address; a success logs the resolved user's name and route.
Neither line ever includes the presented API key.
M2 (usage.estimated): runUnified now reads result.estimated and logs the
estimate explicitly when a non-streaming response's usage falls back to
the body-size heuristic, making the field genuinely read — removes the
stale //nolint:unused on limits.go's usage struct.
M8 (gosec consistency): secrets.go's file: secret path read now carries
the same "#nosec G304 -- operator-supplied file path from middleware
config" annotation as users_file.go's equivalent read.
Doc nit: mcp_a2a.go's validateTargetURLs comment now also names the
dot-only ("."/"..") rule validateConfigName enforces, not just character
set and reserved names.
Gates: go test ./... -count=1 (-race) green, gofmt -l . empty, go vet
clean, make yaegi-check OK, golangci-lint (gosec+staticcheck+unused+
errcheck+govet, global config) 0 issues.
Adds integration/ (separate module): docker compose stack with real Traefik
v3.5 + localPlugins, mock openai/anthropic/gemini/tool providers, shared
Redis, and a raw-TCP streaming timing probe. 6/7 tests pass; streaming
incrementality fails on a confirmed external yaegi/Traefik bug (see
task-15-report.md), not a plugin defect.
Fixes 4 real Yaegi-interpretation bugs surfaced by running the plugin under
real Traefik for the first time (none caught by go test or yaegi-check,
which never exercises chat/embeddings/streaming/Redis):
- .traefik.yml: add basePkg (Traefik derives the yaegi binding identifier
from the module path, not the package's own declared name; without this
override the plugin never loads at all).
- registry.go: resolveAgainst's multi-value return panicked yaegi on an
inline string concatenation; assign to a local first.
- routes_unified.go: errors.As panicked yaegi on interpreted pointer error
types; use plain type assertions (both types are always returned bare,
never wrapped).
- sse.go: readSSE's dispatch closure hit the same multi-value-assignment
panic as registry.go; split into separate statements.
- resp.go: respErr as a defined string type was indistinguishable from a
plain string in a yaegi type assertion, misreading every successful
Redis SELECT reply as failed and silently disabling rate limiting;
changed to a struct.
Review fixes for Task 12:
1. (CRITICAL) runUnified now calls g.limiter.account BEFORE handling an
adapter error, not after returning on it. All three adapters'
forwardStream deliberately return captured usage alongside a mid-stream
error, so a client that aborts right after the usage frame arrives was
getting tokens delivered but never billed - repeated aborts bypassed
every token/cost budget. A non-streaming error path always carries zero
usage by contract, so accounting it is a no-op (account skips writes
once both total tokens and cost are zero). The zero-usage-stream logf
and the zero-usage-non-stream estimate now only fire when callErr is
nil, so a genuine error no longer gets a fabricated token estimate.
New test: TestHandleChat_MidStreamDropAfterUsageChunk_AccountsPartialUsage
(hijack+close after the usage chunk; asserts the limiter's counter shows
the captured tokens). Verified this test fails against the pre-fix
ordering and the two new alias tests below fail when their guarded
delete() call is removed, before restoring and re-running clean.
2. Added TestHandleChat_OpenAI_AliasKeyStrippedFromUpstreamBody and
TestHandleEmbeddings_OpenAI_AliasKeyStrippedFromUpstreamBody: the
openai-type adapter marshals req verbatim as the upstream wire body, so
it is the one path where a leftover gatewayAliasKey entry actually
reaches a real provider. The only existing alias-leak assertion covered
anthropic, which rebuilds the body field-by-field and was never at risk.
3. Reworded the stale comment on limiter.evaluateScope (limits.go): it
described a nil-limits scope's req counters as a normal, expected case.
Ruling (e), implemented in routes_unified.go's buildLimitScopes, means
the unified route never builds a nil-limits scope in the first place -
the nil check remains a defensive no-op, not a path any current caller
exercises.
Gates: gofmt, go vet, fieldalignment, golangci-lint (incl. gosec) all
clean; go test ./... -race -count=1 passes (full suite, including all
Task 12 tests).
Implements Task 12: POST /v1/chat/completions and POST /v1/embeddings
through a shared pipeline (routes_unified.go: handleChat, handleEmbeddings,
runUnified) wired into ServeHTTP's dispatch in llmgateway.go.
Pipeline: decode (10MiB cap) -> resolve model -> enforce user/group
limits -> call adapter -> account usage, translating every failure mode
into its HTTP envelope per the task brief's carried rulings:
- ALIAS ECHO (ruling a): the client-requested model id is threaded to the
adapter via the internal gatewayAliasKey convention and echoed back in
anthropic/gemini's translated response model field; every adapter
strips the key before building its upstream wire body so it never
leaks to a real provider.
- Limit violations (ruling b): storeDown -> 503 server_error; otherwise
429 rate_limit_error with Retry-After when the violation names a
window.
- Stream-safe error handling (ruling c): statusTrackingWriter.wroteHeader
gates whether an adapter error still gets an envelope; providerHTTPError
wraps into {error:{message,type,code,upstream}}; translateError maps to
400/501; context.Canceled logs quietly; anything else is a 502.
- Usage/accounting (ruling d): zero non-streaming usage gets a
ceil(len(body)/4) prompt-token estimate; zero streaming usage logs once
and accounts the request only. unifiedCostMicros extends costMicros'
single-id lookup to canonical-then-bare without changing its signature.
- Scopes (ruling e): user scope only when u.limits is set, group scope
only when grp.limits is set, user first.
routes_unified_test.go drives every case end-to-end through
Gateway.ServeHTTP against httptest fake upstreams (openai- and
anthropic-shaped): happy path with counters asserted via the limiter's
own store, incremental stream flushing, 401/403/404/429/503, embeddings,
alias echo through a provider-prefixed anthropic request, wrapped
upstream error envelopes, and a mid-stream drop that must not append a
second envelope to an already-committed response.
Gates: gofmt, go vet, fieldalignment, golangci-lint (incl. gosec) all
clean; go test ./... -race -count=1 passes.
1. refreshProvider (the plugin's only background goroutine) had no
recover(): a panicking adapter's listModels would crash the whole
Traefik process. Restructured around deferred recover + finishRefresh,
so a panic is turned into a logged error, inFlight always releases, and
the next interval retries. Documented as a scoped exception to
limits.go's no-background-goroutine rule.
2. newGateway wrapped ctx in an outer 5s WithTimeout before warmFill,
but warmFill already derives a fresh 5s budget per provider from
whatever ctx it receives. The outer wrap made that a single budget
shared across every discovery-enabled provider instead of a per-provider
one - providers after the first could get a near-expired context and
fail instantly. Dropped the outer wrapper; construction's worst case is
now N * 5s, accepted.
3. group.allowsModel stripped at any first slash, so a glob like
'deepseek-*' could spuriously match a bare model id that merely
contains a slash (e.g. an upstream's own 'uni/deepseek-v4-flash-0731'
naming) even though the leading segment names no configured provider.
allowsModel is now an exact glob with no internal splitting;
resolveAgainst and listFor generate the provider-prefixed and
bare-suffix candidates themselves, only when a prefix legitimately
names a configured provider. Updated the Task-3 TestIdentify assertion
to match.
4. Annotated the discarded json.Encode error in handleModels.
Gate: go test ./... -count=1, -race, gofmt, vet, fieldalignment,
golangci-lint (incl gosec) all clean.
Adds modelRegistry: per-provider explicit+discovered model sets, bare-id
and provider-prefixed resolution with sorted-name collision precedence,
group authorization pairing (allowsModel AND allowsProvider), and a
group-filtered OpenAI-compatible /v1/models catalog.
Discovery is lazy and throttled per provider: newGateway runs a
synchronous 5s-bounded warm fill for discovery-enabled providers
(non-fatal on error), and Gateway.ServeHTTP triggers a throttled,
inFlight-guarded background refresh per provider per interval
(default 1h, per-provider DiscoveryInterval parsed at construction).
A failed refresh keeps the previous discovered set (stale-while-error).
Wires GET /v1/models into ServeHTTP (auth required, 401 envelope
otherwise) and gives Gateway its adapters/registry fields, both built
in newGateway.
newModelRegistry's signature gained an error return beyond the task
brief's literal listing: an invalid DiscoveryInterval must be a
constructor error per the controller ruling, which the brief's bare
*modelRegistry signature could not express.
lukaszraczylo