mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
main
32
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 lukaszraczylo
|
65b2f121c4 |
feat(admin): add fleet statistics and admin read endpoints
Counters (always on when admin is enabled): - last seen per user and group, written asynchronously after admission, throttled to one write per scope per minute per replica - hourly provider and model attempts, failures and timeouts - failovers per provider, rejections per hour, and unpriced (402) refusals - cache hits, misses and cost saved - request counts per target and caller Opt-in counters under admin.stats (both default off): - userModel: usage per user and model - latency: latency histograms per provider and model All counters ride the existing Redis batches. A test measures three round trips per request with defaults and with every statistic on. A cache hit takes two. A model request is no longer counted for failed failover candidates. New admin endpoints: - usage/series and usage/totals - performance, with percentiles from the histograms - catalog, with the price billing actually uses and its source - consumers, with key source, grants and model access - config, with secrets redacted The overview gains features and a pricing summary. Usage gains lastSeen. usage/models gains offset and more detail fields. Reads are chunked at 1600 keys per round trip and capped at 64000 keys per request. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv |
||
|
lukaszraczylo
|
3e05a79c0b |
feat(admin): report per-model detail, including free models
GET /admin/api/usage/models accepts detail=1. Each entry then carries requests, tokensIn, tokensOut, costMicroUsd and a free flag. A model is listed when any of those four totals is non-zero, so a free model with traffic appears with value 0 even when ranked by cost. The free flag is true when modelMeta marks the model free, or its resolved price is zero. Without detail the response is unchanged. The image and audio routes now write the per-model request counter after a successful upstream call, so media models show up in the ranking. They still record no tokens or cost. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv |
||
|
lukaszraczylo
|
6776981675 |
feat(admin): add events feed, fleet-wide rejections and span rankings
GET /admin/api/events returns recent failures, newest first. It covers rate limits, budget and unpriced-model refusals, upstream errors, timeouts and capacity rejections. Events go into an in-process ring of 200 entries and are copied to a Redis list (llmgw:events, LPUSH plus LTRIM to 200). The copy happens only on the error path, is throttled to 50 per second per replica, and never blocks a request. When Redis is absent or down, the feed falls back to this replica's ring. Messages are fixed strings or scrubbed provider errors. Fields are assigned one at a time, because Yaegi drops values from a multi-value assignment that contains a call. Rejections per day are counted per scope and in total, in the same store call that already rolls back a rejected request. /admin/api/usage returns them as rejectionsPerDay. /admin/api/usage/models accepts span, which sums that many buckets, and prefix, which filters model ids before the limit applies. The overview reports the replica id, the instance name and the warnings logged while the middleware was built. The failover and retry fan-out warning is now logged at WARN. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv |
||
|
lukaszraczylo
|
4c01c75276 |
fix(gateway): close billing, limit, routing and translation gaps
Billing: - Passthrough SSE streams now bill the usage the upstream reports. The gateway falls back to a request-size estimate only on a 2xx stream or a truncated JSON body. - A unified stream that fails after it starts is estimated, not billed at zero. - Billing uses the built-in LiteLLM price table and honours modelMeta free: true. - Pricing entries are validated. Cost arithmetic rounds the total and saturates on overflow. - Gemini thinking tokens count as completion tokens. Limits and storage: - A rejected request rolls back its counts on the other scopes, so one user cannot lock out a group. - Redis EXPIRE replies are checked. Keys recreated without a TTL are repaired at once. - A mutating pipeline is not retried after reply bytes arrive. - History, model totals and target usage report a store outage. Routing and translation: - Discovered ids with a provider-like prefix are listed only in owner/id form, so listing and routing agree. - Anthropic and Gemini discovery follows pagination. - The developer role maps to system. - stream_options.include_usage is honoured on translated streams. - tool_choice none keeps tools when history holds tool blocks. - refusal maps to content_filter. Data URI parameters are stripped. Streaming, retry and failover: - SSE accepts CR, LF and CRLF line ends. This closes a CR injection path. - Retry-After is never jittered. A cancelled wait returns the context error. - The watchdog timer stops at end of body. - Oversized bodies return 413. - A failed stream ends with an error event. - Failover restores pre-existing headers. Client cancels do not count as provider successes. Also: "admin" is a reserved provider name. metrics.path must start with "/". Pricing warnings go to the right gateway instance. The admin targets view sends access as null or [] so the two states differ. Upstream organisation headers are stripped. The users-file reload error is logged once per error. Dead code is removed, and test-only helpers move to test files. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv |
||
|
lukaszraczylo
|
b3b8ae6ad5 |
fix(security): harden retry, SSE, Redis and cost-budget paths
The gateway refuses a request for an unpriced model when the caller has a cost budget. An unpriced model cost 0, so the budget counter was never written and the budget could never fire. The new allowUnpricedWithCostBudget setting restores the old behaviour. The retry loop honours an upstream Retry-After of up to 30s and jitters its own computed backoff. SSE parsing bounds the total size of one event. The Redis client can use TLS. Provider base URLs and errors are scrubbed of credentials before they reach logs or the admin view. The gateway warns when failover and retry settings multiply into a large number of outbound calls. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv |
||
|
lukaszraczylo
|
72b0df5b0c |
feat(auth): let a user join several groups and hold a personal grant
A user entry can now list several groups in "groups", next to the existing "group" field. A user can also hold a personal grant with its own "providers" and optional "models" lists. Users with one group and no personal grant behave exactly as before. A request is allowed only when one grant allows the provider, the model and, for passthrough, the path. Grants never merge, so one grant cannot supply the provider while another supplies the model or the path. The personal grant uses the path list of the first group. A personal "models" list without "providers" is a configuration error. Each member group applies its own limits. Caching stays on unless a member group turns it off, and the shortest group TTL wins. The /v1/models cache uses a stable, length-prefixed key, so reloads do not grow it. Tests cover the grant union, cross-grant denials, passthrough path and model coupling, reloads and the cache key. The yaegi-check harness asserts the same cases under the interpreter. Claude-Session: https://claude.ai/code/session_019uJHQKcNLMACkXssb3L7Q3 |
||
|
lukaszraczylo
|
d2152b169c |
fix(target-health): scrub lastError URLs and fix unknown-state gaps
Combines three closely-coupled target_health.go fixes that all touch targetHealthTracker.record/stateForLocked in the same commit, since splitting them would leave an intermediate commit that fails to compile or a test asserting behaviour its own commit does not yet implement. F1: record gains a rawURL parameter and scrubs err's text through sanitizeProviderErr before storing lastError, so a target URL configured with an embedded credential can never leak through GET /admin/api/targets. The scrub runs before truncateRunes so a long embedded URL cannot survive truncation unscrubbed. Every probe, the per-server proxy hook (mcp_a2a.go), and both federation record sites (mcp_federation.go) now pass their target's configured URL through. F3: maybeSweepTargetHealth re-checks the elapsed interval after winning the single-flight CAS, closing a window where a caller's own stale interval read could still win the CAS once a separate, faster sweep had already started and finished within the same interval. F4: targetHealthEntry gains everOK, set on record's success branch. stateForLocked now reads a target that has never once succeeded as unknown rather than healthy, even below failureThreshold — the threshold still protects against a single transient first failure, it just does not promote a target this tracker has never seen answer to healthy. targetHealthSnapshot gains observed so the admin view can still tell apart never-observed (omit every other field) from observed-but-never-succeeded (keep them) despite both reading state unknown. The now-redundant lastOK field is removed. Also included: target_health_test.go's own test-hygiene fixes (F6) — an idempotent, t.Cleanup-registered unblock ahead of srv.Close so a failing assertion cannot leave a released server handler blocked until the test binary's own timeout, and an atomic hit counter in the disabled-sweep test — since they land in the same file as the tests above. Claude-Session: https://claude.ai/code/session_01EebV5jyWvYocByNQ3oss1w |
||
|
lukaszraczylo
|
a31647f652 |
feat(target-health): expose target health on admin API and metrics
Add a health object to each row of GET /admin/api/targets (adminTargetHealthView, admin.go): state, an RFC3339 lastCheck, lastError, source, and latencyMs, every optional field omitted while the target has never been observed. Both serveAdminTargets and the MCP/agent listing routes (handleMCPServers, handleAgents) trigger the active probe sweep before responding, so admin polling and client listings double as the heartbeat that drives it. Add llmgateway_target_healthy to /metrics (metrics.go): a 1/0 gauge labeled by kind (mcp or agent) and target name, with no sample at all for a target the tracker has never observed. serveMetrics triggers the sweep too, so a Prometheus scrape becomes a heartbeat as well. Claude-Session: https://claude.ai/code/session_01EebV5jyWvYocByNQ3oss1w |
||
|
lukaszraczylo
|
4bb49b4fe9 |
feat(admin): add per-model usage statistics to the charts section
The Charts view could break usage down by user, group or total, but not by model. Per-model data existed only as day-window attempt and failure counts, which measure availability rather than delivered usage. A served model now accumulates the same four metrics a user or group does. The scope kind is "model" and its id is the canonical "provider/model", so a request that fails over counts against the provider that answered it, and one bare id served by two providers ranks as two rows. The request counter is written after the response, in the batch that already writes tokens and cost. That is the first point at which the serving model is known, so no extra round trip is needed and admission never double counts. GET /admin/api/usage/models ranks models for one metric and window. It returns only models with non-zero usage, by operator requirement: the catalog runs to hundreds of models and a padded ranking is unreadable. The read stays one storeGetMulti over the catalog. The Charts view gains a Models tab with its own metric picker, and the scope picker gains the models that have traffic. |
||
|
lukaszraczylo
|
ba09c9f732 |
feat(admin): surface usage-provenance summary in the Providers overview
Add adminProviderView.Provenance to GET /admin/api/overview: a compact, per-provider map keyed by provenance kind, populated only for "estimated" and "unbilled" observations. "reported" never appears here, since it is the healthy default and AttemptsDay/FailuresDay already describe it; the full reported/estimated/unbilled breakdown, by request and by token, stays on GET /metrics. buildAdminProvenanceViews mirrors buildAdminLatencyViews' own shape: one batched g.provenance.snapshot() read per response, grouped by provider, so this adds no additional store round trip and keeps the already-large overview response small, matching the existing latency summary's own size budget. A test proves the reported/estimated/unbilled filtering, verified against a real mutation of the exclusion guard before being kept. |
||
|
lukaszraczylo
|
063a05e9a7 |
feat(metrics): instrument upstream latency
Add time-to-first-byte and total-duration histograms for every upstream call, observation only. No throttling, priorities, or control logic reads this data. The operator wants strain-based throttling later and needs real traffic data first. Extend watchdogBody (timeout.go) rather than add a new interception layer: it already wraps every upstream response body for provider adapters, native passthrough, and the MCP/A2A target proxy, and already observes the first successful read. armLatency arms a watchdogBody after construction, so newWatchdogBody keeps its existing signature and behavior; its own pinned tests still pass unchanged. A timestamp plus a few atomics are the only added cost; Read and Close each pay one nil check when latency collection is disabled. Expose llmgateway_upstream_ttfb_seconds and llmgateway_upstream_duration_seconds on the existing Prometheus endpoint (metrics.go). Both are in-process, per-replica histograms, never store-backed, so sum() is always the correct cross-replica aggregation. Buckets run 0.1s to 600s, chosen to resolve fast TTFB and short completions finely, cover the bulk of real chat completions, and bracket the five-minute default request timeout. Labels are provider and stream always, model only when Config.Metrics.ModelLabel is set, matching the existing per-model opt-in convention. Add a compact per-provider, per-stream average summary to GET /admin/api/overview, sourced from the same in-process store, without a full histogram or a per-model breakdown. Every write path is gated on metricsEnabled(cfg), so a deployment with metrics disabled collects nothing and the histogram families never render. No behavior change to request handling: same responses, status codes, headers, and accounting. Fix a Yaegi-only bug this feature's own context-threading change exposed: two distinct empty struct types used as context.WithValue keys compare equal to each other under the interpreter's interface equality, though never under compiled Go. Threading a second recorder (latencyRecorder) alongside the existing attemptRecorder made this observable for the first time, since compiled tests never see it and the plugin previously carried only one such key. Give latencyRecorderCtxKey a field so its size differs from attemptRecorderCtxKey's, confirmed against a minimal Yaegi reproduction before and after the fix. Fix a second bug, reproducible under compiled Go too: io.Reader may legally return its final data together with io.EOF in one call, and Go's real net/http body reader does this for small, fully-buffered responses. The original TTFB capture lived inside the err == nil branch, so it never fired for a typical small non-streaming completion. Move the check ahead of the error branch so it fires on any n > 0, leaving the pinned timer-reset and Close semantics untouched. Tests mutate the real source per case and confirm each one fails first, recorded in each test's own doc comment. |
||
|
lukaszraczylo
|
619931b2b2 |
feat(admin): surface provider health state in overview
Add HealthState and OpenUntil to adminProviderView, GET /admin/api/overview's per-provider row. HealthState mirrors the discovery circuit breaker's state: closed, open, or half-open. OpenUntil is the zero time.Time unless the breaker is open. A provider with discovery disabled, or one that has never failed a refresh, always reads closed. This keeps the dashboard's default view unchanged for a healthy deployment. Both fields come from providerSnapshot, read in the same locked pass as the existing lastRefresh/lastErr fields, so a concurrent finishRefresh can never hand the dashboard a breaker state that disagrees with the rest of the row. |
||
|
lukaszraczylo
|
800f63ac53 |
fix(admin): chunk usage batch reads to bound connection contention
Fixes finding 4 from the 2026-08-22 security audit: buildAdminUsage sent every user's, group's, and the total scope's counters (8 keys each) to limiter.currentUsage in ONE storeGetMulti round trip. With 1,000 users that is ~8,000 GETs pipelined through the single mutex-guarded store connection, during which every concurrent checkAndCount from live LLM traffic queues behind it (up to respCallTimeout) — and the admin dashboard polls this endpoint every 5s. Add chunkedCurrentUsage: splits the scopes slice into batches of at most adminUsageChunkScopes (200) before calling currentUsage, letting live traffic interleave between chunks while each chunk still batches its own scopes in one round trip. At or under 200 scopes (the live cluster's 9 users) this makes exactly one call, identical to today's behavior — the payload shape is unchanged, so no webui coordination is needed. |
||
|
lukaszraczylo
|
eac0862d68 |
feat(admin,webui): metadata exposure
admin.go: adminProviderView gains ModelMeta (map[string]adminModelMetaView,
keyed by upstream model id) and adminAliasView gains its own ModelMeta —
both built via g.registry.resolveMetaFor/resolveMetaForAliasName
(feature v0.23). adminModelMetaView uses pointer fields
(*int/*float64), not plain zero values: a nil pointer means unknown, a
non-nil pointer (even one pointing at 0) means known, so a free model's
real zero cost never gets confused with an absent one over JSON.
resolveMetaForAliasName resolves an alias's metadata for this
unrestricted admin view via an internal &group{} (GroupConfig's own
documented empty-means-all semantics) — buildAdminOverview already
shows every provider/alias unconditionally, with no group filtering
anywhere else in its response either.
webui/src/types/api.ts: AdminModelMetaView mirrors the Go shape
(optional numbers, undefined means unknown); AdminProviderView.modelMeta
and AdminAliasView.modelMeta wire it through.
webui/src/lib/format.ts: formatContextWindow (binary-K "256k"-style),
formatModelCostHover ("in $X / out $Y per MTok", or "free"), and
their shared formatUsdPerMTok building block.
ModelChip.vue (operator scope refinement): every chip — the accordion's
model chips, the alias table's chips, a group's usage-detail chips —
gains optional contextTokens/inputPerMTokUsd/outputPerMTokUsd props.
When known, they append a resolved-metadata phrase to the existing
copy-state hover text, bound to BOTH title and aria-label (the same
mouse-vs-keyboard/screen-reader parity ProviderRateBadge.vue already
applies); when nothing is known, the hover is unchanged from before this
feature. An alias's chip carries its INHERITED metadata, matching the
alias-inheritance rule the backend already applies.
ProvidersView.vue: the accordion's model list gains a visible, muted
context-only chip ("256k") beside each ModelChip — cost stays
hover-only on the chip itself, per the operator's own "chips +
hover would be redundant on price" resolution.
|
||
|
lukaszraczylo
|
1b4516efbb |
fix(providers): deadline failures, cheaper polling, async writes
Review round (v0.22): 1 must-fix + 5 should-fix. This covers the three backend items. MUST-1's WCAG fix lives in the webui commit; this is the three provider/ admin-side SHOULD items. SHOULD-1 (timeout blindness): a hung upstream previously read as 100% healthy — isTransient (retry.go) deliberately excludes context.DeadlineExceeded from what it retries (retrying past the gateway's own deadline is pointless), and recordProviderAttempt reused that same classification unforked, so a timed-out attempt recorded no failure. Fixed narrowly in recordProviderAttempt only, per the ruling: isTransient itself stays untouched and shared. A new isDeadlineExceeded adds exactly one extra condition — a context.Canceled error (the client walked away) still counts as an attempt only, from both functions alike. The sentinel match (matchesSentinel) is a hand-rolled errors.Unwrap walk, not errors.Is: providerHTTPError's own doc comment already documents that errors.As panics under Yaegi checking whether an interpreted pointer type implements error; errors.Is walks the identical internal machinery and carries the same risk, so this sidesteps it with a bare `err.(sentinelUnwrapper)` comma-ok assertion instead — plain interface satisfaction, not reflection over a possibly-interpreted type. SHOULD-2 (poll fan-out): a (provider, model) scope's minute-window counters fed nothing but a badge's title text on the Providers tab, so a dashboard with N configured models paid N wasted minute-window reads on every 5s poll. providerCounterKeys/providerUsage now read a kindProviderModel scope's day window only (2 keys, down from 4); a kindProvider scope is unaffected (still 4 — one scope per provider, not per model, so the read amplification never applied there). adminModelRateView drops AttemptsMinute/FailuresMinute entirely, not just unpopulated fields — the webui side (separate commit) falls back to a day-window-only title for a per-model badge. SHOULD-5 (TTFB): recordProviderAttempt's store write is telemetry-only — nothing in the request path gates on it — so it now runs via a new limiter.spawn dependency-injection field (mirrors nowFn/waitFn's existing pattern), defaulting to a real goroutine in production and overridden to run synchronously in every test that asserts on the resulting counter immediately afterward. This removes the synchronous Redis round trip this counter family added to the streaming TTFB path entirely. Loss on process shutdown is accepted (telemetry only). |
||
|
lukaszraczylo
|
bf23b41568 |
feat(admin): rates + discovery flag in API
Extend GET /admin/api/overview's existing providers payload rather than adding a sibling endpoint: the route already returns one providers array per request on the dashboard's 5s poll, is already admin-gated and stats-excluded, and the new fields are one more batched limiter read sharing that same request — a second endpoint would duplicate all three properties for no benefit. adminProviderView gains AttemptsDay/FailuresDay/AttemptsMinute/ FailuresMinute (the provider's own current-window counters) and ModelRates (a per-model breakdown, keyed by upstream model id, via the new adminModelRateView) — Feature A. buildAdminOverview builds every provider's and every (provider, model) pair's limitScope in two passes and reads them all through limiter.providerUsage in ONE storeGetMulti call, then slices the flat result back by index. adminProviderView also gains DiscoveryEnabled, mirroring a new providerSnapshot.discoveryEnabled field (registry.go) read straight off providerState — it is set once at construction and never mutated afterward, so it needs neither providerState's mutex nor a place in its existing snapshot() method's mutable-field return shape. Feature B: this lets the webui tell "discovery is off" apart from "discovery is on but has not refreshed yet", which previously both showed the same zero lastRefresh as an identical "refreshed never". |
||
|
lukaszraczylo
|
66fa0b56ea |
feat(admin): MCP/agent target visibility
Add per-target request accounting for the MCP/A2A proxy path and a
new admin endpoint to surface it.
- limits.go: countTargetRequest increments one target's req counters
at min/hour/day/month in a single storeIncrMulti call — pure
accounting, no limits, mirroring account()'s own contract.
targetUsage batches the read side the same way currentUsage does
for user/group scopes.
- mcp_a2a.go: handleTargetProxy attributes every admitted proxied
request to its target's own scope, kind "mcp" or "agent"
(scopeKindAgent — deliberately not targetKindAgent's "a2a" routing
convention). kind is embedded in the counter key (windowKey), so a
target can never collide with a user/group/total scope of the same
id, whatever any of them are named.
- admin.go: new GET /admin/api/targets, admin-gated and
stats-excluded exactly like the other three /admin/api/* routes.
Access lists are computed via the SAME matchesGlob call
group.allowsMCP/allowsAgent themselves use, so the panel can never
disagree with enforcement; a target every group can reach collapses
to an omitted access list ("empty meaning all").
No new config surface: target scopes carry no configurable limits
this round.
Tests: TestTargetScopeKind (mapping), TestHandleTargetProxy_
PerTargetCounters_AttributedAndCollisionSafe (table-driven over
mcp/a2a, proves counters land on the right scope and never collide
with a same-named user), TestAdminTargets_* (gating tables extended,
shape/sorting/URL-sanitization/access-list/counter assertions).
|
||
|
lukaszraczylo
|
03bc67a5f0 |
feat(admin): expose group provider/model access
GET /admin/api/usage's group rows now carry the group's configured Providers/Models/MCPServers/Agents (GroupConfig, llmgateway.go), echoed exactly as configured -- never resolved/expanded to the full catalog server-side. Empty/unset stays omitted (omitempty), matching GroupConfig's own JSON convention and group.allowsX's empty-means-all semantics (auth.go): the dashboard can tell 'restricted to these N' apart from 'unrestricted' without the payload growing with catalog size. groupSummary (auth.go) gains providers/models/mcpServers/agents, carried through from authStore.snapshot unchanged. adminUsageEntryView (admin.go) gains the matching JSON fields, populated only for group entries -- a user or the total scope row never carries them, mirroring GroupName's own user-only convention. The Usage view's group accordion (webui/UsageView.vue) renders an Access block above the existing stat grid: Badge chips for explicit providers, ModelChip for explicit models (both shadcn/existing components, no hand-rolled chips), or muted 'All providers'/'All models' text when unrestricted. Display-only this round -- not wired into the user/group search filter. types/api.ts's AdminUsageEntryView mirrors the new optional fields. Tests: admin_test.go's new table-driven TestAdminUsage_GroupAccessLists covers a restricted group's exact lists, an unrestricted group's empty/omitted lists, and that a user/total row never carries them. |
||
|
lukaszraczylo
|
f9b289f1ea |
feat(webui): provider model accordion
Clicking a provider row in the dashboard expands to show its model
list, with a search filter and copy-to-clipboard on each model.
Backend (registry.go/admin.go): providerSnapshot gains models []string
- the sorted explicit union discovered id set, computed in the SAME
providerState.mu critical section as lastRefresh/lastErr (not via a
separate knownIDs() call, which takes the same non-reentrant lock and
would deadlock) so the three values are never assembled from two
separate locked passes a concurrent finishRefresh could interleave
between. adminProviderView exposes it as models (never nil - a real,
if empty, JSON array). New tests: overview shape asserts sorted
models per provider; a new TestModelRegistry_Snapshot_ModelsStaleWhileError
proves a failed refresh after an earlier success keeps the previous
discovered set (existing stale-while-error semantics, no special
handling added); TestAdmin_SecretRedaction unaffected (model ids are
not secrets, still passes as-is).
Frontend: the accordion is NOT shadcn-vue's Accordion or Collapsible -
both wrap trigger+content in one div, which cannot legally sit
between two tr elements inside a table/tbody. Installed
Collapsible via the CLI to check (verified components.json's now-
defused font/iconLibrary omission holds under a real install: main.css
stayed byte-for-byte unchanged) and inspected its source: reka-ui's
CollapsibleContent does support an "as" prop that could force it onto
a tr, but its animation-measurement code (getBoundingClientRect +
inline transition styles) is untested in that shape and not worth
taking on for an instant show/hide with no transition - removed the
installed files, kept a plain reactive Set<string> + v-if second
TableRow, which reuses Accordion's own visual language (rotating
chevron, keyboard-accessible trigger row: role=button, tabindex,
Enter/Space handlers) without misusing a component built for block
content.
Model chips (new ModelChip.vue) show the full ROUTABLE id -
"{providerName}/{modelId}" unconditionally, so a discovered id that
already contains a slash (uni/deepseek-v4-flash-0731 on provider
"real") renders as real/uni/deepseek-v4-flash-0731 and stays
copy-paste-valid - and copy that id to the clipboard on click
(lib/clipboard.ts: navigator.clipboard.writeText with a
select-the-text fallback when the Clipboard API is unavailable, never
the deprecated execCommand('copy')), with a brief checkmark
copied-state affordance. No CSP change: Clipboard access is a browser
permission, not a CSP directive.
Search input (OverviewView.vue) filters the providers table to
providers with a matching model (case-insensitive substring) and the
aliases table to aliases whose own name or target matches, auto-
expanding matching providers while a query is active and filtering
each expanded provider's own chip list down to just its matches (so a
72-model provider does not stay a 72-chip wall for a 1-model search).
Manual expand/collapse state is tracked separately from the search's
own auto-expand set, so clearing the query restores exactly what the
user had open by hand - nothing to explicitly "restore", since the
auto-expand set was never written into the manual one.
Integration: TestAdminDashboard's overview assertion extended
(adminProviderModels helper) to check the openai provider's models
array contains dynamic.yml.tmpl's explicit "gpt-mock" model - stays
API-level (asserts the JSON payload), not the rendered HTML, so the
prefixed-display-form request is left at the frontend only per scope.
Regenerated via make admin-ui; admin_assets_gen.go included.
|
||
|
lukaszraczylo
|
1aa2b43932 |
fix(webui): defuse scaffold config traps, tighten inline guards
Micro-fix from re-review: 1. webui/components.json: deleted the font/iconLibrary keys instead of picking a value. Re-review checked shadcn-vue's actual parse schema (node_modules/shadcn-vue/dist/schema/index.js's rawConfigSchema): both fields are z.string().optional(), not an enum - the five-choice enum lives only on the init CLI's --font/--icon-library flags, not on the components.json schema. The previous commit's "schema forbids a fix" rationale was wrong; corrected here. Verified empirically, not just by reading the schema: ran a real `npx shadcn-vue add badge -o -y` against components.json with both keys removed - src/assets/ main.css came back byte-for-byte unchanged (previously this exact class of call re-added a Google Fonts import and the .dark-class toggle). That same run did still re-add @lucide/vue to package.json's dependencies (some components' registry entries list an icon package regardless of iconLibrary) - reverted before committing, and webui/README.md's warning section rewritten to explain the omission (so nobody "helpfully" restores the keys), fix the schema claim, and note the narrower remaining package.json trap. 2. admin.go:27-28: adminCSP's doc comment now names all three set-sites (GET /admin, every GET /admin/assets/*, the three JSON routes) - previously only mentioned the page and JSON routes, dropped after the asset route gained the same header in the prior fix round. 3. Both inline-script guards (webui/generate.mjs's assertNoInlineAssets, admin_test.go's hasInlineScript) tightened from a bare \bsrc\s*= pattern to (^|\s)src\s*= - \b alone matches inside "data-src=" too (the "-" before "src" is a non-word character, so \b sits right before "src"), which would wrongly treat a data-src/nosrc attribute as the real src= that makes a <script> external. New Go test TestHasInlineScript_DataSrcIsNotExternal: <script data-src="x"> is still flagged as inline. No webui/src/ change this round - components.json isn't bundled, and generate.mjs's edit only touches its own build-time guard, not the Go-source-generation path. admin_assets_gen.go NOT regenerated (confirmed via a throwaway `node generate.mjs` + gofmt -w run: output is byte-identical to what's already committed). |
||
|
lukaszraczylo
|
b98416d1e9 |
fix(webui): version display, CSP guards, config traps, bucket-aligned stacking
Review fix round (4 Important + 2 folded): 1. Version display: OverviewView.vue was missing the muted version line the old vanilla-JS page rendered (types/api.ts already declared overview.version) - added. 2. CSP invariant now guarded twice: webui/generate.mjs's new assertNoInlineAssets fails the build if dist/index.html contains an inline <script> (no src= attribute) or any <style> tag - the exact condition the tightened admin CSP (script-src/style-src 'self', no 'unsafe-inline') depends on. admin_test.go's new TestAdminIndexHTML_NoInlineAssets re-asserts the identical invariant server-side, against the decoded adminIndexHTML admin.go actually serves, so a regression is caught even if admin_assets_gen.go were regenerated with a modified copy of generate.mjs. 3. webui/components.json's iconLibrary:"lucide"/font:"inter" fields cannot be fixed in place - checked shadcn-vue's own schema (PRESET_ICON_LIBRARIES/PRESET_FONTS in the installed package): no "none"/fontawesome/system choice exists for either, and JSON (unlike the tsconfig.*.json files touched in the previous commit) has no comment syntax to warn inline. Documented instead: webui/README.md gained a loud warning section naming the exact regression (already hit twice this branch - a re-added Google Fonts import, a re-added lucide icon import) and the two-step fix to apply after any future shadcn-vue add/init call. 4. Real-CSP verification: the browser pass earlier in this branch ran under vite preview, which sends no CSP header at all. New integration/integration_test.go: TestAdminAssetCSP fetches GET /admin against the real Traefik+Yaegi pipeline, extracts the asset URL from its <script src="..."> tag, fetches that asset, and asserts both responses carry the exact adminCSP string byte-for-byte. admin.go's serveAdminAsset now also sets Content-Security-Policy (previously JSON/page routes only) so this holds for a direct navigation to an asset URL, not just as a same-origin sub-resource of GET /admin; admin_test.go's TestAdminAsset_Serving updated to match. 5. (folded) UsageChart.vue stacked the "tokens" tab's two series (tokin/tokout - two separate HTTP requests, not one atomic call) by positional index. A window rollover landing between the two requests could leave one series one bucket ahead, mis-stacking that refresh. Now aligns both series by their bucket key (a sorted union of every active series' buckets - the fixed-width, zero-padded bucket strings sort chronologically by construction), with a missing bucket in one series reading as 0 rather than shifting the whole series. 6. (folded) AuthGate.vue now triggers an immediate dashboard.refresh() on successful key submit, instead of waiting up to 5s for the next poll tick - the dashboard no longer sits blank right after auth. Rebuilt via make admin-ui; admin_assets_gen.go regenerated. |
||
|
lukaszraczylo
|
9fced97050 |
feat(admin): exclude admin traffic from usage statistics
Operator directive (progress ledger, 2026-08-21): admin requests must
not touch req/min-req/day statistics. handleAdminAPI no longer calls
checkAndCount for GET /admin/api/overview or GET /admin/api/usage (it
already skipped it for GET /admin/api/usage/history) - admin traffic
must never appear in usage statistics, which measure real LLM traffic
only. identify()'s group return is now discarded (u, _, ok :=), since
buildLimitScopes is no longer built here.
Direct consequence, documented in the new doc comment: an admin's own
requestsPerMinute/requestsPerDay limit, if configured, is no longer
enforced against admin-route traffic either - accepted trade-off
(admin-gated, cheap reads; a self-inflicted, not shared, resource
cost).
admin_test.go: TestAdmin_RateLimitApplies (asserted a 2nd request
429s) inverted to TestAdmin_NeverRateLimitedAndCountersUntouched - 5x
requests to both overview and usage against an admin key pinned at
RequestsPerMinute:1, asserting every one is 200 and the admin's own
req/min counter stays exactly 0.
integration/integration_test.go: TestAdmin_Dashboard's admin-user
assertion inverted from requestsPerDay > 0 ("admin API calls count
too") to requestsPerDay == 0 ("admin routes never call
checkAndCount") - the admin key is used throughout the whole suite
exclusively to hit /admin/api/*, so it must show exactly 0 despite
issuing many requests by the time this test runs.
|
||
|
lukaszraczylo
|
7b28f34bb2 |
feat(admin): serve baked vue panel with usage charts
admin.go: GET /admin now serves the generated adminIndexHTML
(admin_assets_gen.go, webui/generate.mjs's output) instead of the old
adminPageHTML Go-string const (removed, ~340 lines, plus its now-dead
io import). New route GET /admin/assets/{hashedname}
(adminAssetsPathPrefix) serves the built app's content-hashed JS/CSS
from the generated adminAssets map - unauthenticated (same rationale as
GET /admin itself), Content-Type per asset, Cache-Control: public,
max-age=31536000, immutable, X-Content-Type-Options: nosniff; an
unknown name is a 404 in the standard OAI error envelope.
CSP tightens: script-src/style-src drop 'unsafe-inline' for 'self' (the
built app has no inline script or style of any kind - vite.config.ts's
build.modulePreload.polyfill:false sees to that), img-src gains 'self'
data:' for the shell's inlined-SVG favicon.
Makefile: new admin-ui target (npm ci && vite build, then
generate.mjs, then gofmt -w) - the only place Node is invoked; CI and
every other make target still need nothing beyond Go.
admin_test.go: TestAdminPage_CSPHeaderAndFetchURLs rewritten as a shell
test against the generated file (new CSP string, body references
/admin/assets/ and the <title>'s "LLM Gateway", no external
src=/href= reference - the favicon's xmlns attribute is excluded from
that check, it names an XML namespace URI, not a fetched resource);
TestAdmin_JSONSecurityHeaders' expected CSP updated; new
TestAdminAsset_Serving (Content-Type/nosniff/immutable-cache/byte-exact
body), TestAdminAsset_UnknownReturns404, and
TestAdminAssets_GeneratedFilePresent (non-empty index.html/assets, no
leftover {{/}}/generator-marker text leaked into the decoded HTML -
proves the base64 round-trip actually worked).
|
||
|
lukaszraczylo
|
5cc681496c |
perf(limits): single-batch counter writes, bounded fallback retention
Review fix round (2 Important + 5 folded) on 08cfdb4/b3fb705.
1. counterStore gains incrMulti(entries []counterIncr) ([]int64, error).
redisStore.incrMulti sends every entry's INCRBY+EXPIRE pair as ONE
pipeline (ttlToSeconds extracted from incrBy, now shared);
memoryStore.incrMulti loops incrBy for interface conformance.
checkAndCount batches every scope's req:min/req:day/req:hour into one
storeIncrMulti call (down from up to 9 round trips for a 3-scope
request) and reads req:min/req:day back from the reply, no extra
GETs. account batches every scope's nonzero tok-in/tok-out/cost
entries into one call (down from up to 27). Violation math and
fail-open/fail-closed/latched semantics are unchanged — only the
round-trip count moves.
2. memoryStoreMaxTTL (48h) caps every TTL memoryStore actually applies,
clamped inside incrBy so incrMulti's loop inherits it for free. The
in-process fallback is continuity, not history: applying the real
hour/day/month TTLs there was growing live-key count ~30x, multiplying
sweepEvery's own documented "167x incrBy cliff" by the same factor.
Sweep cadence is unchanged (still 30s-gated); the clamp shrinks what
there is to sweep instead.
Folded: historyMaxSpan's margin comment corrected (~4 days, not a
month); checkAndCount's godoc names req:hour; the integration
current-hour assertion sums every point instead of reading only the
last one (an hour-rollover race the suite cannot control); a new
exact-boundary test proves tokin+tokout landing exactly on a
TokensPerDay limit still violates; README documents the history route
as unthrottled (admin-gate is its only trust boundary) and the
fallback's 48h retention ceiling.
Tests: countingIncrStore proves exactly one incrMulti call per
checkAndCount and per account, with correct per-scope/metric/window
values and unchanged violation math; a fake-RESP-server test pins
incrMulti's wire format (one pipeline, entry-ordered replies); TTL
clamp tests cover both the ceiling and the pass-through case at the
boundary.
Gates: go test ./... -count=1, -race, gofmt, vet, golangci-lint,
make yaegi-check all green. make integration: every test passes
except the pre-existing TestUsersFileHotReload flake, which is NOT
caused by this change — verified via an isolated worktree at the
prior commit (
|
||
|
lukaszraczylo
|
08cfdb4d99 |
feat: split token counters and hour-resolution usage history
Data layer for the upcoming Vue admin charts (v0.2 data-layer task).
Tokens-in/out split (limits.go): metricTok replaced by metricTokIn
("tokin") and metricTokOut ("tokout"). account() records usage.prompt
and usage.completion independently, skipping a direction only when its
own value is zero. A TokensPerDay/TokensPerMonth limit still enforces
one TOTAL budget: tokenBudgetViolation reads both counters in one
storeGetMulti batch and sums them; the violation message still says
"tokens". Estimated usage (the request-body-size fallback) lands
entirely in tokin, documented at its call site.
Hour window (limits.go): windowHour ("hour"), bucket format
"2006010215" UTC, hourWindowTTL (48h). checkAndCount/account both write
it on every metered route; no LimitsConfig field ever names it, so
evaluateScope never evaluates it. dayWindowTTL/monthWindowTTL bumped to
35d/400d so the usage-history API's largest span always finds a live
key.
Total scope (limits.go, routes_unified.go): a synthetic
{kind:"total", id:"all"} scope, limits always nil. withTotalScope
wraps buildLimitScopes' result at every metered route (unified,
media, passthrough, mcp/a2a) so none can forget it; handleAdminAPI's
own accounting stays on buildLimitScopes alone, so admin traffic never
contributes to it.
History API (admin.go): GET /admin/api/usage/history?scope=&metric=&
window=&span= — admin-gated like the other two JSON routes, but
skips checkAndCount ahead of that same change landing for every
/admin/api/* route in a later task. Validates scope kind/metric/window
(400), span bounds per window (400), and unknown user/group id (404).
Computes every bucket key up front and reads them in ONE storeGetMulti
batch; a storeDown read is 503, never a silently-zero series. Works
against memoryStore too.
Admin usage API (admin.go): scopeUsage/adminUsageEntryView now expose
tokensInPerDay/tokensOutPerDay and their per-month counterparts in
place of the combined tokensPerDay/tokensPerMonth pair; buildAdminUsage
adds a "total" row to the response, still one storeGetMulti round trip
overall. usageKeysPerScope names the per-scope key stride (was a bare
6, now 8), closing a deferred review follow-up while the count changed
anyway.
Tests: TDD throughout — bucket/window-key math incl. UTC and hour
boundaries, the month step-back day-31 AddDate trap, split accounting,
token budget summing both directions, total-scope counting-without-
evaluating on every metered route, and the full history API (shape,
oldest-first, span bounds, validation table, 404, 503, one-batch
assertion via a counting stub). Integration: TestAdminDashboard now
fetches an hour history series for user alice and for total and
asserts a non-zero current-hour bucket.
Migration: none — old "tok" counter keys simply expire on their
existing TTL; nothing is deployed yet.
|
||
|
lukaszraczylo
|
657480348b |
fix: v0.2 final review wave
Transcriptions Yaegi coverage via integration mock, flattened admin usage batch, hardened CSP, sanitized provider lastErr, source-archive prefix template, vendor export-ignore warning, release paths. |
||
|
lukaszraczylo
|
64eef947a7 | ci: release chain — shared workflows, semver wording, source-only goreleaser, version stamping | ||
|
lukaszraczylo
|
2090b31ae2 |
feat: operator-defined model aliases
Spec §5 (v0.2): Config.ModelAliases map[string]string, alias-first resolution precedence ahead of provider-prefix splitting, alias-OR-target authorization, lazy target resolution with a 404 naming both ids, /v1/models listing, admin overview alias table (JSON + dashboard), and construction-time validation (shape, prefix-shadow, explicit-model collision, chain, empty target). routes_unified.go/routes_media.go needed no resolution-call-site changes - both already pass the client's raw requested id through to registry.resolve, so aliases inherit the __alias echo and multipart rewrite machinery for free. cache.go needed no changes either - cacheKey already hashes requestedModel, so two aliases resolving to the same target already produce distinct cache entries. Adds registry_test.go validation table + precedence/authz/lazy-404/ listing coverage, a dedicated cache-key split test, chat/embeddings/ images/audio-transcriptions end-to-end tests, an extended admin overview test, and an integration test covering the openai-type vs anthropic-type echo asymmetry under real Traefik+Yaegi. |
||
|
lukaszraczylo
|
2fe71c25eb |
docs: v0.2 features
README, examples, and a comment update covering v0.2's four features:
- README: new Caching section (key semantics incl. requestedModel +
endpoint, hit/miss header, count-requests-free-tokens, group override,
redis-required) and Admin section (enable flag, admin:true users,
browser key-entry flow, sessionStorage, what's exposed/redacted,
poll-counts-against-quota, unauthenticated-shell). Retry and Image/audio
sections already existed from T3/T4; left as-is. Config reference gains
cache/admin rows at the top level, GroupConfig.cache, UserConfig.admin,
and full CacheConfig/AdminConfig tables. Known limitations gains
images/audio-not-cost-accounted and cache-shares-entries-across-users.
- examples/kubernetes.yaml: adds cache{enabled:true,ttl:5m} (this example
already configures redis) and admin{enabled:false} with a comment on
enabling it, plus an inline admin:true user gated behind that flag.
Round-trips through New() cleanly (verified with a throwaway test).
- examples/docker-compose.yml: adds retry, cache (left disabled — this
minimal demo runs no Redis), and admin (left disabled, with a comment
on enabling it). Round-trips through New() cleanly (verified the same
way).
- admin.go: pluginVersion's doc comment now says to keep it in sync with
the release tag.
|
||
|
lukaszraczylo
|
63459b0c24 |
fix: v0.2 review sweep
Controller-ruled small fixes accumulated from T3/T4 reviews: 1. routes_media.go: multipart Content-Type check is now an exact mediaType != "multipart/form-data" comparison (HasPrefix admitted "multipart/form-datax"); a malformed-tail multipart body (fails to parse at all) now reports "malformed multipart body", distinct from "model is required" (a body that parses fine but carries no model part) and "duplicate model field". 2. provider_openai.go: audioTranscription's doc comment no longer claims the body is always "replayed unchanged"/the boundary always "verbatim" — it now reflects the rewrite path routes_media.go already takes for an aliased model id. 3. rewriteMultipartModel: buf.Grow(len(body)+512) up front, avoiding repeated reallocate-and-copy growth while rebuilding an aliased multipart body. 4. limits.go currentUsage: guards len(vals) == len(keys) before indexing storeGetMulti's result, so a non-conforming counterStore's short slice reports storeDown instead of panicking. 5. admin.go dashboard JS: the 5s poll no longer wipes an existing auth-error message when no key is stored (only a successful load or a new submission clears it now); clearStoredKey on a 401/403 now only fires when the failing request's own key still matches what's currently stored, guarding against a late response for a key the user has since replaced. 6. Spec §4's duplicated /admin/api/overview bullet and the nosniff/no-store sentence's placement were already fixed upstream (docs/superpowers/specs/2026-08-20-llmgateway-v02-design.md, commit c02010d) — verified identical, nothing further to commit here. 7. Added TestAdmin_JSONSecurityHeaders (nosniff/no-store/CSP on both admin JSON APIs) and TestHandleAudioTranscriptions_MalformedTail_400 (covers fix 1's new malformed-body branch). go test ./... -count=1 -race, gofmt, vet, make yaegi-check, golangci-lint (incl. gosec) all pass. |
||
|
lukaszraczylo
|
33bd8bc2cc |
fix(admin): browser key entry, batched usage reads, hardened responses
Review fixes (2 Important + 4 folded): 1. GET /admin now serves the HTML shell with no authentication - a browser navigating directly to the URL cannot attach a custom header. The page reads an admin key from sessionStorage and sends it as x-api-key on /admin/api/* fetches, falling back to an inline key-entry form on a missing key or any 401/403. Both JSON routes keep their original 401/403/200 gate. Spec ss4 amended in place. 2. GET /admin/api/usage no longer serializes 6 GETs per user/group scope on the shared Redis connection. respClient gains getBatch (one pipelined round trip); counterStore gains getMulti; limiter gains storeGetMulti mirroring storeGet's fail-open/fail-closed policy. currentUsage now issues one getMulti per scope. 3. Provider baseUrl is stripped of userinfo and query before it is ever echoed in the overview JSON (sanitizeBaseURL), stronger than net/url's own Redacted (which leaves a bare username in place). 4. limiter.redisStatus now also returns the last failure's timestamp, so the dashboard can render "last error (Ns ago)" instead of a bare message that reads as currently-broken indefinitely. 5. Both JSON admin responses now set X-Content-Type-Options: nosniff, Cache-Control: no-store, and the same CSP the HTML page uses. 6. Added a test proving a file-sourced (replaceFileUsers) admin user gets the same access an inline admin user does. go test ./... -count=1 -race, gofmt, vet, make yaegi-check all pass. |
||
|
lukaszraczylo
|
547b61f5a9 |
feat: read-only admin dashboard and JSON API
Adds GET /admin (single-page dashboard), GET /admin/api/overview, and GET /admin/api/usage per spec §4: providers/redis/cache/groups summary and per-user/group current-window usage+limits. Gated by Config.Admin (default off, not registered when disabled) and UserConfig.Admin. Gate order: disabled -> 404 fall-through; unauthenticated -> 401; authenticated non-admin -> 403; admin -> serve, counted against the admin's own request limits like any other route. Adds small, read-only, mutex-guarded hooks: modelRegistry.snapshot(), limiter.currentUsage()/redisStatus(), authStore.snapshot(). No secret field (API key, digest, redis password) is ever read into any admin response type; verified by a canary-literal redaction test. HTML/CSS/JS ship as a single Go raw-string const (Yaegi forbids go:embed); confirmed interpretable via make yaegi-check. |