mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-18 05:47:38 +00:00
feat(admin): add fleet statistics and admin read endpoints
Counters (always on when admin is enabled): - last seen per user and group, written asynchronously after admission, throttled to one write per scope per minute per replica - hourly provider and model attempts, failures and timeouts - failovers per provider, rejections per hour, and unpriced (402) refusals - cache hits, misses and cost saved - request counts per target and caller Opt-in counters under admin.stats (both default off): - userModel: usage per user and model - latency: latency histograms per provider and model All counters ride the existing Redis batches. A test measures three round trips per request with defaults and with every statistic on. A cache hit takes two. A model request is no longer counted for failed failover candidates. New admin endpoints: - usage/series and usage/totals - performance, with percentiles from the histograms - catalog, with the price billing actually uses and its source - consumers, with key source, grants and model access - config, with secrets redacted The overview gains features and a pricing summary. Usage gains lastSeen. usage/models gains offset and more detail fields. Reads are chunked at 1600 keys per round trip and capped at 64000 keys per request. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
This commit is contained in:
+8
-1
@@ -211,7 +211,14 @@ func (g *Gateway) handleTargetProxy(w http.ResponseWriter, r *http.Request, u *u
|
||||
writeLimitViolation(w, violation)
|
||||
return
|
||||
}
|
||||
g.limiter.countTargetRequest(targetScopeKind(kind), name)
|
||||
// P9 fix (admin dashboard redesign verify round): last-seen only for
|
||||
// an admitted request — recordLastSeen's own doc comment (limits.go).
|
||||
g.limiter.recordLastSeen(scopes, g.limiter.now())
|
||||
// Target x caller (always-on-with-admin, DECISIONS): u.name identifies
|
||||
// the caller whose own credential authorized this proxy — see
|
||||
// countTargetRequestBy's own doc comment (limits.go) for the id shape
|
||||
// and admin-stats gating.
|
||||
g.limiter.countTargetRequestBy(u.name, targetScopeKind(kind), name)
|
||||
|
||||
upstreamURL, err := buildUpstreamTargetURL(targetURL, rest, r.URL.RawQuery)
|
||||
if err != nil {
|
||||
|
||||
Reference in New Issue
Block a user