mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
lukaszraczylo
65b2f121c4
Counters (always on when admin is enabled): - last seen per user and group, written asynchronously after admission, throttled to one write per scope per minute per replica - hourly provider and model attempts, failures and timeouts - failovers per provider, rejections per hour, and unpriced (402) refusals - cache hits, misses and cost saved - request counts per target and caller Opt-in counters under admin.stats (both default off): - userModel: usage per user and model - latency: latency histograms per provider and model All counters ride the existing Redis batches. A test measures three round trips per request with defaults and with every statistic on. A cache hit takes two. A model request is no longer counted for failed failover candidates. New admin endpoints: - usage/series and usage/totals - performance, with percentiles from the histograms - catalog, with the price billing actually uses and its source - consumers, with key source, grants and model access - config, with secrets redacted The overview gains features and a pricing summary. Usage gains lastSeen. usage/models gains offset and more detail fields. Reads are chunked at 1600 keys per round trip and capped at 64000 keys per request. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
18 lines
763 B
Go
18 lines
763 B
Go
package traefikllmgateway
|
|
|
|
// needsExpire reports whether key does NOT currently have a CONFIRMED
|
|
// EXPIRE recorded for it — the single-key shape needsExpireBatch
|
|
// (redis_store.go) generalizes to a whole incrMulti/incrAndGetMulti
|
|
// batch. Every production call site now goes through needsExpireBatch;
|
|
// this stays as a test-only helper (admin-redesign WP-G cleanup) because
|
|
// redis_store_test.go's own coverage of expireSeen/commitExpire/
|
|
// forgetExpireLocked's bookkeeping asserts against it directly, one key
|
|
// at a time, which is easier to read than threading every assertion
|
|
// through a []counterIncr batch.
|
|
func (s *redisStore) needsExpire(key string) bool {
|
|
s.expireMu.Lock()
|
|
defer s.expireMu.Unlock()
|
|
_, ok := s.expireSeen[key]
|
|
return !ok
|
|
}
|