perf(limits): single-batch counter writes, bounded fallback retention

Review fix round (2 Important + 5 folded) on 08cfdb4/b3fb705.

1. counterStore gains incrMulti(entries []counterIncr) ([]int64, error).
   redisStore.incrMulti sends every entry's INCRBY+EXPIRE pair as ONE
   pipeline (ttlToSeconds extracted from incrBy, now shared);
   memoryStore.incrMulti loops incrBy for interface conformance.
   checkAndCount batches every scope's req:min/req:day/req:hour into one
   storeIncrMulti call (down from up to 9 round trips for a 3-scope
   request) and reads req:min/req:day back from the reply, no extra
   GETs. account batches every scope's nonzero tok-in/tok-out/cost
   entries into one call (down from up to 27). Violation math and
   fail-open/fail-closed/latched semantics are unchanged — only the
   round-trip count moves.

2. memoryStoreMaxTTL (48h) caps every TTL memoryStore actually applies,
   clamped inside incrBy so incrMulti's loop inherits it for free. The
   in-process fallback is continuity, not history: applying the real
   hour/day/month TTLs there was growing live-key count ~30x, multiplying
   sweepEvery's own documented "167x incrBy cliff" by the same factor.
   Sweep cadence is unchanged (still 30s-gated); the clamp shrinks what
   there is to sweep instead.

Folded: historyMaxSpan's margin comment corrected (~4 days, not a
month); checkAndCount's godoc names req:hour; the integration
current-hour assertion sums every point instead of reading only the
last one (an hour-rollover race the suite cannot control); a new
exact-boundary test proves tokin+tokout landing exactly on a
TokensPerDay limit still violates; README documents the history route
as unthrottled (admin-gate is its only trust boundary) and the
fallback's 48h retention ceiling.

Tests: countingIncrStore proves exactly one incrMulti call per
checkAndCount and per account, with correct per-scope/metric/window
values and unchanged violation math; a fake-RESP-server test pins
incrMulti's wire format (one pipeline, entry-ordered replies); TTL
clamp tests cover both the ceiling and the pass-through case at the
boundary.

Gates: go test ./... -count=1, -race, gofmt, vet, golangci-lint,
make yaegi-check all green. make integration: every test passes
except the pre-existing TestUsersFileHotReload flake, which is NOT
caused by this change — verified via an isolated worktree at the
prior commit (b3fb705) run under identical conditions (full suite,
same host), which fails identically. This host's Docker Desktop is
currently slow to propagate the bind-mounted users.json change into
the containers; auth.go/users_file.go are untouched by this diff.
This commit is contained in:
2026-08-21 01:02:28 +01:00
parent b3fb705b36
commit 5cc681496c
9 changed files with 634 additions and 113 deletions
+37 -21
View File
@@ -509,26 +509,38 @@ http:
accuracy.
- **Storage**: `redis` configured wires a hand-rolled, stdlib-only RESP2
client (`resp.go`) — no `go-redis`, which is not Yaegi-interpretable —
against Redis, Valkey, or Dragonfly. Semantics are deliberately
**at-least-once, never under-counting**: a lost reply after the server
already applied an `INCRBY` can cause a resend that over-counts by one
request's worth on that key. This is a fail-*safe* trade-off (more
restrictive than reality), never a fail-*open* one. Without `redis`
configured, or when it errors and `failOpen: true`, counters live in an
in-process map — correct for one Traefik replica, only approximate
across several, since each replica counts independently. Exact limits
across multiple Traefik replicas require the shared Redis. Every
counter, including the fallback in-process one, works as the usage-
history API's data source: a fallback-only deployment gets charts of
whatever the process itself accumulated, not an empty series.
- **Retention** (counter key TTL, distinct from a window's own length):
a minute key lives 2 minutes, an hour key 48 hours, a day key 35 days,
a month key 400 days — long enough for the usage-history API's largest
span (48 hourly / 35 daily / 13 monthly buckets) to always find a live
key, not just long enough to survive that bucket's own single rollover.
**Migration**: none — a deployed process's old, unsplit `tok` counter
keys simply expire on their existing TTL and are never read again;
nothing needs backfilling.
against Redis, Valkey, or Dragonfly. Every counter write (`checkAndCount`,
`account`) goes through ONE pipelined round trip for the whole request,
regardless of how many user/group/total counters it touches (perf
review, 2026-08-21) — a 3-scope request previously paid up to 9 (request
counting) or 27 (usage accounting) separate round trips. Semantics are
deliberately **at-least-once, never under-counting**: a lost reply after
the server already applied the pipeline can cause a resend that
over-counts every counter in it by its own delta. This is a
fail-*safe* trade-off (more restrictive than reality), never a
fail-*open* one.
- **Fallback** (no `redis` configured, or a Redis error with
`failOpen: true`): counters live in an in-process map — correct for one
Traefik replica, only approximate across several, since each replica
counts independently. Exact limits across multiple Traefik replicas
require the shared Redis. Every counter, including this fallback one,
works as the usage-history API's data source, but its retention is
capped at **48 hours regardless of window** (`memoryStoreMaxTTL`,
limits.go — ruling, 2026-08-21: the in-process fallback is continuity,
not history; applying the real day/month TTLs to it would grow its live
key count roughly 30x, multiplying the cost of its own periodic sweep by
the same factor). A fallback-only deployment's usage-history charts
therefore show at most the trailing 48 hours, never the full day/month
span Redis-backed deployments get — full retention needs the shared
Redis.
- **Retention** (Redis counter key TTL, distinct from a window's own
length): a minute key lives 2 minutes, an hour key 48 hours, a day key
35 days, a month key 400 days — long enough for the usage-history API's
largest span (48 hourly / 35 daily / 13 monthly buckets) to always find
a live key, not just long enough to survive that bucket's own single
rollover. **Migration**: none — a deployed process's old, unsplit `tok`
counter keys simply expire on their existing TTL and are never read
again; nothing needs backfilling.
## Retry
@@ -773,7 +785,11 @@ request to any of them falls through to the plugin's existing
`{"scope","metric","window","points":[{"bucket":"2026082114","value":123},...]}`.
Unlike the other two JSON routes, this one **never counts request
statistics** — it skips `checkAndCount` entirely, ahead of the same
change landing for every `/admin/api/*` route in a later change.
change landing for every `/admin/api/*` route in a later change. It is
therefore also **unthrottled**: no `requestsPerMinute`/`requestsPerDay`
limit ever applies to it, so its only trust boundary is the admin gate
itself (a valid admin key). An admin key holder can poll it as fast as
they like.
- **What's exposed**: provider names, types, base URLs (with any
userinfo/query string stripped before it's ever echoed), model counts,
discovery status, group/user names, membership, limits, and live usage