mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
0c61296c73cfa1dbe0678ac047773a2869cc172e
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 lukaszraczylo
|
0c61296c73 |
feat(registry): model metadata capture and resolution
Config gets two new surfaces (llmgateway.go): Config.ModelMeta (map[string]*ModelMetaConfig, keyed by exact "provider/model" or a bare model id) for operator overrides, and ProviderConfig.MetadataPath for openai-type providers whose discovery should also capture per-model context (LM Studio's native /api/v0/models shape). modelmeta.go adds the layering itself: resolveModelMeta applies config override > discovered context > builtin table > absent for context, and config override/free > builtin > absent for cost (discovery never reports cost). resolveAliasModelMeta implements alias inheritance: an alias inherits its target's fully resolved metadata unless the alias itself has its own modelMeta entry, applied per field. validateModelMeta rejects negative values and a free:true/non-zero-cost contradiction at construction. registry.go wires this into the request path: providerState gains a discoveredContext map (stale-while-error, mirroring discovered's own convention) with contextFor/setDiscoveredContext; a new optional modelMetadataFetcher interface, type-asserted (interpreted-to-interpreted, documented as yaegi-safe against the Trap-4 restriction) rather than added to providerAdapter, lets warmFill/refreshProvider capture metadata alongside listModels without touching anthropic/gemini adapters at all. modelObject/listFor now attach resolved context_window/pricing to every GET /v1/models entry, including aliases (resolveMetaFor/ resolveMetaForAlias/resolveMetaForAliasName). provider_openai.go implements modelMetadataFetcher: fetchModelMetadata reads ProviderConfig.MetadataPath, preferring loaded_context_length over max_context_length when the LM Studio response reports both. providers.go wires MetadataPath onto the adapter in buildAdapters. tools/yaegi-check/main.go adds one cheap modelMeta config-override assertion to the existing attempt-accounting harness, proving resolveModelMeta's config-override layer and the context_window/pricing extension fields both run correctly under the real Yaegi interpreter. |
||
|
lukaszraczylo
|
bf23b41568 |
feat(admin): rates + discovery flag in API
Extend GET /admin/api/overview's existing providers payload rather than adding a sibling endpoint: the route already returns one providers array per request on the dashboard's 5s poll, is already admin-gated and stats-excluded, and the new fields are one more batched limiter read sharing that same request — a second endpoint would duplicate all three properties for no benefit. adminProviderView gains AttemptsDay/FailuresDay/AttemptsMinute/ FailuresMinute (the provider's own current-window counters) and ModelRates (a per-model breakdown, keyed by upstream model id, via the new adminModelRateView) — Feature A. buildAdminOverview builds every provider's and every (provider, model) pair's limitScope in two passes and reads them all through limiter.providerUsage in ONE storeGetMulti call, then slices the flat result back by index. adminProviderView also gains DiscoveryEnabled, mirroring a new providerSnapshot.discoveryEnabled field (registry.go) read straight off providerState — it is set once at construction and never mutated afterward, so it needs neither providerState's mutex nor a place in its existing snapshot() method's mutable-field return shape. Feature B: this lets the webui tell "discovery is off" apart from "discovery is on but has not refreshed yet", which previously both showed the same zero lastRefresh as an identical "refreshed never". |
||
|
lukaszraczylo
|
b39d4e92c9 |
fix(webui): query-safe expand state, prefixed search, accessible row toggle
Review fix round (5 Important + 3 folded) against the provider-model
accordion (
|
||
|
lukaszraczylo
|
f9b289f1ea |
feat(webui): provider model accordion
Clicking a provider row in the dashboard expands to show its model
list, with a search filter and copy-to-clipboard on each model.
Backend (registry.go/admin.go): providerSnapshot gains models []string
- the sorted explicit union discovered id set, computed in the SAME
providerState.mu critical section as lastRefresh/lastErr (not via a
separate knownIDs() call, which takes the same non-reentrant lock and
would deadlock) so the three values are never assembled from two
separate locked passes a concurrent finishRefresh could interleave
between. adminProviderView exposes it as models (never nil - a real,
if empty, JSON array). New tests: overview shape asserts sorted
models per provider; a new TestModelRegistry_Snapshot_ModelsStaleWhileError
proves a failed refresh after an earlier success keeps the previous
discovered set (existing stale-while-error semantics, no special
handling added); TestAdmin_SecretRedaction unaffected (model ids are
not secrets, still passes as-is).
Frontend: the accordion is NOT shadcn-vue's Accordion or Collapsible -
both wrap trigger+content in one div, which cannot legally sit
between two tr elements inside a table/tbody. Installed
Collapsible via the CLI to check (verified components.json's now-
defused font/iconLibrary omission holds under a real install: main.css
stayed byte-for-byte unchanged) and inspected its source: reka-ui's
CollapsibleContent does support an "as" prop that could force it onto
a tr, but its animation-measurement code (getBoundingClientRect +
inline transition styles) is untested in that shape and not worth
taking on for an instant show/hide with no transition - removed the
installed files, kept a plain reactive Set<string> + v-if second
TableRow, which reuses Accordion's own visual language (rotating
chevron, keyboard-accessible trigger row: role=button, tabindex,
Enter/Space handlers) without misusing a component built for block
content.
Model chips (new ModelChip.vue) show the full ROUTABLE id -
"{providerName}/{modelId}" unconditionally, so a discovered id that
already contains a slash (uni/deepseek-v4-flash-0731 on provider
"real") renders as real/uni/deepseek-v4-flash-0731 and stays
copy-paste-valid - and copy that id to the clipboard on click
(lib/clipboard.ts: navigator.clipboard.writeText with a
select-the-text fallback when the Clipboard API is unavailable, never
the deprecated execCommand('copy')), with a brief checkmark
copied-state affordance. No CSP change: Clipboard access is a browser
permission, not a CSP directive.
Search input (OverviewView.vue) filters the providers table to
providers with a matching model (case-insensitive substring) and the
aliases table to aliases whose own name or target matches, auto-
expanding matching providers while a query is active and filtering
each expanded provider's own chip list down to just its matches (so a
72-model provider does not stay a 72-chip wall for a 1-model search).
Manual expand/collapse state is tracked separately from the search's
own auto-expand set, so clearing the query restores exactly what the
user had open by hand - nothing to explicitly "restore", since the
auto-expand set was never written into the manual one.
Integration: TestAdminDashboard's overview assertion extended
(adminProviderModels helper) to check the openai provider's models
array contains dynamic.yml.tmpl's explicit "gpt-mock" model - stays
API-level (asserts the JSON payload), not the rendered HTML, so the
prefixed-display-form request is left at the frontend only per scope.
Regenerated via make admin-ui; admin_assets_gen.go included.
|
||
|
lukaszraczylo
|
ed2119cc99 |
fix(aliases): dedupe listing against emitted entries, not raw owners
listFor's alias dedupe keyed on the raw owners map, so a colliding discovered model whose provider is group-denied caused BOTH the real entry and the alias to be skipped - the id vanished from the listing entirely even though resolve still serves it via the alias. Track ids actually appended to out in a listed map instead, and skip the alias only when listed[alias] is true. Extends TestModelRegistry_ListFor_Alias_DedupedAgainstDiscoveredCollision with a provider-denied variant: group providers:["openai"], collision on the denied "acme" provider -> alias listed exactly once, owned_by openai. |
||
|
lukaszraczylo
|
f0730a70d7 |
fix(aliases): deterministic integration gate, warn-first fast path, deduped listing
Review fix, task 6 (1 Important + 3 folded): 1. TestModelAliases now polls GET /v1/models (waitForModelAlias) until "aliased/mock" appears, bounded 30s, before asserting - removes the stale-dynamic-config propagation window a plain healthy-200 check cannot rule out. Confirmed config-propagation, not alias-logic, per reviewer's config+error-text analysis. 2. registry.go resolve(): check the warned-set (aliasShadowWarned) before paying bareWinner's O(providerNames) scan on every aliased request - only the first request for a colliding alias pays that cost now. 3. listFor: skip an alias entry whose id already exists in owners (a real model already owns it) - previously produced duplicate ids when a discovered model collided with a configured alias. resolve's own alias-wins precedence is unchanged; only the listing is deduped. New test: TestModelRegistry_ListFor_Alias_DedupedAgainstDiscoveredCollision. 4. newAliasTestRegistry doc comment: "grp's given aliases" -> "the given aliases" (no grp parameter exists). |
||
|
lukaszraczylo
|
2090b31ae2 |
feat: operator-defined model aliases
Spec §5 (v0.2): Config.ModelAliases map[string]string, alias-first resolution precedence ahead of provider-prefix splitting, alias-OR-target authorization, lazy target resolution with a 404 naming both ids, /v1/models listing, admin overview alias table (JSON + dashboard), and construction-time validation (shape, prefix-shadow, explicit-model collision, chain, empty target). routes_unified.go/routes_media.go needed no resolution-call-site changes - both already pass the client's raw requested id through to registry.resolve, so aliases inherit the __alias echo and multipart rewrite machinery for free. cache.go needed no changes either - cacheKey already hashes requestedModel, so two aliases resolving to the same target already produce distinct cache entries. Adds registry_test.go validation table + precedence/authz/lazy-404/ listing coverage, a dedicated cache-key split test, chat/embeddings/ images/audio-transcriptions end-to-end tests, an extended admin overview test, and an integration test covering the openai-type vs anthropic-type echo asymmetry under real Traefik+Yaegi. |
||
|
lukaszraczylo
|
547b61f5a9 |
feat: read-only admin dashboard and JSON API
Adds GET /admin (single-page dashboard), GET /admin/api/overview, and GET /admin/api/usage per spec §4: providers/redis/cache/groups summary and per-user/group current-window usage+limits. Gated by Config.Admin (default off, not registered when disabled) and UserConfig.Admin. Gate order: disabled -> 404 fall-through; unauthenticated -> 401; authenticated non-admin -> 403; admin -> serve, counted against the admin's own request limits like any other route. Adds small, read-only, mutex-guarded hooks: modelRegistry.snapshot(), limiter.currentUsage()/redisStatus(), authStore.snapshot(). No secret field (API key, digest, redis password) is ever read into any admin response type; verified by a canary-literal redaction test. HTML/CSS/JS ship as a single Go raw-string const (Yaegi forbids go:embed); confirmed interpretable via make yaegi-check. |
||
|
lukaszraczylo
|
7939c3e74b |
test: integration suite with real Traefik, redis-shared limits, streaming assertions
Adds integration/ (separate module): docker compose stack with real Traefik v3.5 + localPlugins, mock openai/anthropic/gemini/tool providers, shared Redis, and a raw-TCP streaming timing probe. 6/7 tests pass; streaming incrementality fails on a confirmed external yaegi/Traefik bug (see task-15-report.md), not a plugin defect. Fixes 4 real Yaegi-interpretation bugs surfaced by running the plugin under real Traefik for the first time (none caught by go test or yaegi-check, which never exercises chat/embeddings/streaming/Redis): - .traefik.yml: add basePkg (Traefik derives the yaegi binding identifier from the module path, not the package's own declared name; without this override the plugin never loads at all). - registry.go: resolveAgainst's multi-value return panicked yaegi on an inline string concatenation; assign to a local first. - routes_unified.go: errors.As panicked yaegi on interpreted pointer error types; use plain type assertions (both types are always returned bare, never wrapped). - sse.go: readSSE's dispatch closure hit the same multi-value-assignment panic as registry.go; split into separate statements. - resp.go: respErr as a defined string type was indistinguishable from a plain string in a yaegi type assertion, misreading every successful Redis SELECT reply as failed and silently disabling rate limiting; changed to a struct. |
||
|
lukaszraczylo
|
0e73558f5c |
ci: yaegi catalog-analyzer gate harness
Add tools/yaegi-check, a separate Go module (llmgw-yaegi-check,
never a dependency of the plugin module) that replicates the
Traefik Plugin Catalog analyzer (piceus)'s yaegiMiddlewareCheck:
copy the repo into a temp GOPATH, interp.New + i.Use(stdlib.Symbols),
import the plugin package, eval CreateConfig(), decode the plugin's
real .traefik.yml testData into it via a hand-rolled YAML-subset
parser plus a JSON round-trip, reflect-Call New from outside the
interpreter, then exercise the returned handler with two real HTTP
requests (authenticated GET /v1/models against the testData key, and
the same route unauthenticated). Pins github.com/traefik/yaegi to
v0.16.1, confirmed as Traefik v3.5.0's own vendored version by
downloading its go.mod.
Wire cd tools/yaegi-check && GOWORK=off go run . /Users/nvm/Documents/projects/private/traefik-llmgateway
yaegi-check: OK to run the harness from repo root.
The harness caught a real Yaegi v0.16.1 limitation: calling a struct
FIELD of variadic func type (e.g. modelRegistry.log
func(string, ...any)) with two or more variadic arguments crashes
Yaegi's CFG builder ('index out of range'). The identical call shape
through a method or interface method is unaffected — bisected with a
standalone yaegi repro to confirm the exact trigger before touching
plugin code. Fixed by pre-formatting with fmt.Sprintf at
modelRegistry's three affected call sites (registry.go) so the field
is always called with exactly one variadic argument; documented the
constraint on both modelRegistry.log and limiter.logf so it is not
reintroduced. Verified the gate itself is load-bearing by temporarily
renaming CreateConfig (harness failed non-zero) and restoring it
(harness passed again).
Also fixes two golangci-lint findings in the new harness module
(govet shadow, gosec G301 directory permissions).
|
||
|
lukaszraczylo
|
e91f0e36ff |
fix(registry): recover refresh goroutine, per-provider warm-fill budget, exact-glob model authorization
1. refreshProvider (the plugin's only background goroutine) had no recover(): a panicking adapter's listModels would crash the whole Traefik process. Restructured around deferred recover + finishRefresh, so a panic is turned into a logged error, inFlight always releases, and the next interval retries. Documented as a scoped exception to limits.go's no-background-goroutine rule. 2. newGateway wrapped ctx in an outer 5s WithTimeout before warmFill, but warmFill already derives a fresh 5s budget per provider from whatever ctx it receives. The outer wrap made that a single budget shared across every discovery-enabled provider instead of a per-provider one - providers after the first could get a near-expired context and fail instantly. Dropped the outer wrapper; construction's worst case is now N * 5s, accepted. 3. group.allowsModel stripped at any first slash, so a glob like 'deepseek-*' could spuriously match a bare model id that merely contains a slash (e.g. an upstream's own 'uni/deepseek-v4-flash-0731' naming) even though the leading segment names no configured provider. allowsModel is now an exact glob with no internal splitting; resolveAgainst and listFor generate the provider-prefixed and bare-suffix candidates themselves, only when a prefix legitimately names a configured provider. Updated the Task-3 TestIdentify assertion to match. 4. Annotated the discarded json.Encode error in handleModels. Gate: go test ./... -count=1, -race, gofmt, vet, fieldalignment, golangci-lint (incl gosec) all clean. |
||
|
lukaszraczylo
|
fdb81640ec |
feat: model registry with discovery and aggregated /v1/models
Adds modelRegistry: per-provider explicit+discovered model sets, bare-id and provider-prefixed resolution with sorted-name collision precedence, group authorization pairing (allowsModel AND allowsProvider), and a group-filtered OpenAI-compatible /v1/models catalog. Discovery is lazy and throttled per provider: newGateway runs a synchronous 5s-bounded warm fill for discovery-enabled providers (non-fatal on error), and Gateway.ServeHTTP triggers a throttled, inFlight-guarded background refresh per provider per interval (default 1h, per-provider DiscoveryInterval parsed at construction). A failed refresh keeps the previous discovered set (stale-while-error). Wires GET /v1/models into ServeHTTP (auth required, 401 envelope otherwise) and gives Gateway its adapters/registry fields, both built in newGateway. newModelRegistry's signature gained an error return beyond the task brief's literal listing: an invalid DiscoveryInterval must be a constructor error per the controller ruling, which the brief's bare *modelRegistry signature could not express. |