Close a HIGH-severity OOM path on the unified and media JSON routes
(security review finding 1). Three parts:
- checkAndCount now runs via a new admitRequest helper BEFORE the
request body is read at all, on both routes_unified.go's runUnified
and every routes_media.go handler. The rate-limit scopes need only the
already-resolved user/group, so a caller already over their
requestsPerMinute is refused without paying for the read+decode of a
body that was always going to be discarded. Measured: an encoding/json
decode into map[string]any costs ~11x the wire body in live heap, so
the prior ordering let an already-limited caller burn ~110MiB per
request before ever seeing the 429.
- A new bounded, buffered-channel semaphore (Gateway.bodyAdmission,
mirroring limits.go's spawnTokens shape) caps how many unified/media
requests may concurrently hold a decoded body in memory. Acquired
non-blockingly at each route's entry via acquireBodyAdmission; on
exhaustion, returns 503 with Retry-After rather than blocking. Sized
by defaultBodyAdmissionCap (GOMAXPROCS * 8, clamped to [16, 128]);
Config.MaxInFlightBodyRequests, when set, always overrides the
self-tuned default.
- The unified chat/embeddings routes now decode against a new, narrower
maxUnifiedRequestBytes (4MiB) instead of the shared maxRequestBytes
(10MiB), which the media JSON routes, audio/transcriptions' multipart
cap, and the MCP federation JSON-RPC routes keep unchanged. 1MiB (the
audit's own first suggestion) was evaluated and rejected: this
package documents and exercises inline base64 image_url content parts
in chat messages (translate_anthropic.go, translate_gemini.go), and a
single moderately-sized photo commonly exceeds 1MiB once
base64-encoded. 4MiB preserves that documented single-inline-image
case while still cutting worst-case per-request decode amplification
from ~110MiB to ~44MiB.
routes_media.go's resolveMediaRequest is split: model resolution stays
(renamed resolveMediaModel), the limit check moves into the shared
admitRequest.
Tests: admission-before-decode is asserted directly (a 429'd request's
body is never Read), semaphore exhaustion is asserted deterministically
by filling the channel directly, the self-tune/override contract and its
clamp bounds are pinned, and a zero-config run of ordinary traffic is
asserted to see no new 503s. A dedicated test proves the chosen 4MiB cap
still accepts a realistic ~1.5MiB inline image — the case 1MiB would
have broken.
Track per-provider and per-(provider,upstream-model) upstream-attempt
outcomes using the existing fixed-window counter machinery (limits.go),
following mcp_a2a.go's per-target counter pattern: kind-embedded keys
(kindProvider="prov", kindProviderModel="provmodel") keep a provider
scope collision-safe against a user/group/MCP-target/agent of the same
name.
limiter.recordProviderAttempt(provider, model, resp, err) increments an
"attempts" counter on every call, and a "failures" counter only for a
provider-fault outcome — reusing retry.go's own isTransient(resp, err)
classification exactly (transport/connection error, HTTP 429, or a 5xx),
not forking it. Both counters are written at minute and day granularity
only; provider health is a now-and-today question, not a billing one, so
there is no month window. limiter.providerUsage batches every scope's
read into one storeGetMulti round trip, mirroring targetUsage's own
single-batch discipline.
Wired at every chokepoint an upstream HTTP attempt resolves:
- providers.go's upstreamJSON/upstreamRawBytes path (chat, embeddings,
images, audio) via retry.go's retryPolicy.do, which now looks up an
optional attemptRecorder from ctx and invokes it once per retry
attempt (attempt-accounting: a request retried twice before
succeeding reports three attempts, not one).
- routes_unified.go's runUnified (chat/embeddings, streaming and
non-streaming alike — both funnel through the same upstreamJSON call).
- routes_media.go's three media handlers (images.generations,
audio.speech, audio.transcriptions).
- routes_passthrough.go's proxyUpstream (native provider passthrough) —
a single un-retried attempt, provider-level only: the upstream model
lives in the response body, read only after the attempt resolves.
The attemptRecorder is threaded via context (providers.go's
withAttemptRecorder/attemptRecorderFromContext), not a new function
parameter on every call site — proxyUpstream and retryPolicy.do are
both shared with paths that must NOT record provider accounting
(registry.go's discovery listModels calls, and the MCP/A2A target
proxy, mcp_a2a.go's handleTargetProxy), so scoping via context keeps
their signatures unchanged and those callers correctly opt out by
simply never wrapping their context.
Data layer for the upcoming Vue admin charts (v0.2 data-layer task).
Tokens-in/out split (limits.go): metricTok replaced by metricTokIn
("tokin") and metricTokOut ("tokout"). account() records usage.prompt
and usage.completion independently, skipping a direction only when its
own value is zero. A TokensPerDay/TokensPerMonth limit still enforces
one TOTAL budget: tokenBudgetViolation reads both counters in one
storeGetMulti batch and sums them; the violation message still says
"tokens". Estimated usage (the request-body-size fallback) lands
entirely in tokin, documented at its call site.
Hour window (limits.go): windowHour ("hour"), bucket format
"2006010215" UTC, hourWindowTTL (48h). checkAndCount/account both write
it on every metered route; no LimitsConfig field ever names it, so
evaluateScope never evaluates it. dayWindowTTL/monthWindowTTL bumped to
35d/400d so the usage-history API's largest span always finds a live
key.
Total scope (limits.go, routes_unified.go): a synthetic
{kind:"total", id:"all"} scope, limits always nil. withTotalScope
wraps buildLimitScopes' result at every metered route (unified,
media, passthrough, mcp/a2a) so none can forget it; handleAdminAPI's
own accounting stays on buildLimitScopes alone, so admin traffic never
contributes to it.
History API (admin.go): GET /admin/api/usage/history?scope=&metric=&
window=&span= — admin-gated like the other two JSON routes, but
skips checkAndCount ahead of that same change landing for every
/admin/api/* route in a later task. Validates scope kind/metric/window
(400), span bounds per window (400), and unknown user/group id (404).
Computes every bucket key up front and reads them in ONE storeGetMulti
batch; a storeDown read is 503, never a silently-zero series. Works
against memoryStore too.
Admin usage API (admin.go): scopeUsage/adminUsageEntryView now expose
tokensInPerDay/tokensOutPerDay and their per-month counterparts in
place of the combined tokensPerDay/tokensPerMonth pair; buildAdminUsage
adds a "total" row to the response, still one storeGetMulti round trip
overall. usageKeysPerScope names the per-scope key stride (was a bare
6, now 8), closing a deferred review follow-up while the count changed
anyway.
Tests: TDD throughout — bucket/window-key math incl. UTC and hour
boundaries, the month step-back day-31 AddDate trap, split accounting,
token budget summing both directions, total-scope counting-without-
evaluating on every metered route, and the full history API (shape,
oldest-first, span bounds, validation table, 404, 503, one-batch
assertion via a counting stub). Integration: TestAdminDashboard now
fetches an hour history series for user alice and for total and
asserts a non-zero current-hour bucket.
Migration: none — old "tok" counter keys simply expire on their
existing TTL; nothing is deployed yet.
Controller-ruled small fixes accumulated from T3/T4 reviews:
1. routes_media.go: multipart Content-Type check is now an exact
mediaType != "multipart/form-data" comparison (HasPrefix admitted
"multipart/form-datax"); a malformed-tail multipart body (fails to
parse at all) now reports "malformed multipart body", distinct from
"model is required" (a body that parses fine but carries no model
part) and "duplicate model field".
2. provider_openai.go: audioTranscription's doc comment no longer claims
the body is always "replayed unchanged"/the boundary always
"verbatim" — it now reflects the rewrite path routes_media.go
already takes for an aliased model id.
3. rewriteMultipartModel: buf.Grow(len(body)+512) up front, avoiding
repeated reallocate-and-copy growth while rebuilding an aliased
multipart body.
4. limits.go currentUsage: guards len(vals) == len(keys) before indexing
storeGetMulti's result, so a non-conforming counterStore's short slice
reports storeDown instead of panicking.
5. admin.go dashboard JS: the 5s poll no longer wipes an existing
auth-error message when no key is stored (only a successful load or a
new submission clears it now); clearStoredKey on a 401/403 now only
fires when the failing request's own key still matches what's
currently stored, guarding against a late response for a key the user
has since replaced.
6. Spec §4's duplicated /admin/api/overview bullet and the
nosniff/no-store sentence's placement were already fixed upstream
(docs/superpowers/specs/2026-08-20-llmgateway-v02-design.md, commit
c02010d) — verified identical, nothing further to commit here.
7. Added TestAdmin_JSONSecurityHeaders (nosniff/no-store/CSP on both
admin JSON APIs) and TestHandleAudioTranscriptions_MalformedTail_400
(covers fix 1's new malformed-body branch).
go test ./... -count=1 -race, gofmt, vet, make yaegi-check, golangci-lint
(incl. gosec) all pass.
lukaszraczylo