fix(limits): never fabricate zero counters, bound the rejection counter

Two correctness fixes from adversarial verification.

providerUsage returned every counter at its zero value on a
failed-closed store read, with no signal distinguishing that from a
real zero. metrics.go rendered the zero as a hard value. A single
Redis blip therefore read as a counter reset, and the recovery scrape
counted the whole prior total as a fresh increase, painting a phantom
multi-thousand-request spike from one transient error. Add storeDown
to providerCounters and set it on the failed-closed path, mirroring
scopeUsage's existing contract.

limiter.rejections grew without bound: nothing ever removed a scope
whose user was renamed or deleted from a hot-reloaded users file, so a
long-running process kept exporting a series for every name ever seen.
Bound the map with the same current/previous generation-rotation
authFailureTracker already uses, and evict a user's entry when
authStore.maybeReload performs a real reload. maybeReload now returns
whether it actually reloaded, so ServeHTTP only runs the eviction walk
on that rare event, never on every request.

Also corrects a stale comment: the rejection counter's store-down
branch fires on every request during a full outage, not rarely, as an
earlier comment claimed.
This commit is contained in:
2026-08-23 00:58:02 +01:00
parent d421f49ff1
commit 5700dd482d
4 changed files with 263 additions and 42 deletions
+8 -1
View File
@@ -830,7 +830,14 @@ func attachUsersFile(auth *authStore, path string, log gatewayLogger) error {
func (g *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request) {
sw := &statusTrackingWriter{ResponseWriter: w}
defer recoverPanic(sw, g)
g.auth.maybeReload()
// pruneRejectionsAfterReload only runs when maybeReload just performed
// a REAL reload (users_file.go's own doc comment explains why it must
// never run on every request) — a deleted file-sourced user must not
// keep exporting llmgateway_rate_limit_rejections_total forever
// (metrics.go).
if g.auth.maybeReload() {
g.pruneRejectionsAfterReload()
}
g.registry.maybeRefresh(r.Context())
if r.Method == http.MethodGet && r.URL.Path == "/v1/models" {