7 Commits

Include renames

Author	SHA1	Message	Date
lukaszraczylo	90eb58171e	perf(redis_store): send EXPIRE only once per key per process lifetime ... 31 of 74 commands per request in the interpreted-harness audit were EXPIRE, emitted unconditionally after every INCRBY. Deleting EXPIRE entirely (upper bound) measured 74->43 cmds/req, p50 867.9->697.9us (-19.6%), conc=16 throughput 1,477->2,336 req/s (+58%), allocs -26%. Route chosen: a bounded first-touch-per-key cache (expireOnce), not EVALSHA+Lua. windowKey (limits.go) already embeds each counter's own time bucket in its key string, and every TTL callers pass is deliberately far longer than its window's own natural length purely as a stale-key GC margin (limits.go's own doc comment on dayWindowTTL/monthWindowTTL/etc.), not a sliding-window mechanism — so once EXPIRE has been sent for an exact key, every further INCRBY against that SAME key happens well within its own bucket's window, comfortably inside the TTL's margin. Skipping the redundant resend cannot leave a key un-expiring: a bucket's key simply stops being written once its window rolls over (a new bucket string takes its place). expireOnce is bounded (expireOnceCacheMax=4096, FIFO eviction) so a long-lived process's ever-rolling buckets cannot grow it unboundedly; an evicted key just gets one redundant-but-harmless EXPIRE again, exactly as if never touched before. incrBy/incrMulti/incrAndGetMulti now build a variable-stride command list (each entry contributes its own INCRBY, plus EXPIRE only when expireOnce(key) is true) with an explicit incrReplyIdx/base tracking each entry's actual reply position — no longer a fixed i*2 stride, since a skipped EXPIRE shifts every later command left by one. EVALSHA+Lua was rejected as the shippable route here: it would rewrite every command-building and reply-index path, requires NOSCRIPT fallback handling, and every existing wire-level test (dozens of INCRBY+EXPIRE-scripted fakeRESPServer cases) would need rebuilding around SHA1 digests for no additional correctness benefit over the simpler first-touch cache.	2026-08-22 14:36:44 +01:00
lukaszraczylo	311dd20a8e	fix(limits): correct dangling comments and redisStore edge cases ... Independent review round approved the fusion itself (11 to 3 RTT/req, precedence verified branch by branch, latch/fail-open checked under Yaegi against a dead Redis). This commit addresses the follow-up items raised against that same change. Six doc comments in limits.go still named evaluateScope, budgetViolation, or tokenBudgetViolation as if they were live functions to consult. All three were removed by the fusion commit; the comments now point at checkAndCount and buildBudgetProbes, where the same logic lives today. One further comment claimed checkAndCount still slices a "storeIncrMulti result" — it now slices storeIncrAndGetMulti's own incrVals. TestCheckAndCount_FusedViolationPrecedence's doc comment claimed every subtest engineers two simultaneous violation conditions. The seventh engineers only one (the group violates; the user carries no limits at all) — it pins a different claim, that a later scope is still evaluated when an earlier one has nothing to check. The comment now says so. redisStore.incrAndGetMulti gained the same len()==0 guard incrMulti already has, and a table-driven test proves a RESP error or a non-integer value on a read key fails the whole call, matching respClient.getBatch's own equivalent table. Re-ran the interpreted Yaegi harness after these changes: RTT/req stays at 3.00, cmds/req stays at 74.00 — the fusion's throughput win is unaffected by this round's fixes.	2026-08-22 12:14:48 +01:00
lukaszraczylo	a418814b34	perf(limits): fuse admission reads into one round trip ... checkAndCount already batched its request-counter increments into one pipeline. evaluateScope then read every scope's token and cost budgets through up to four further serial round trips per scope. A request with a fully-limited user and group scope paid nine round trips before the upstream call started. This change fuses every budget read into the SAME pipeline as the increments, through a new incrAndGetMulti method on counterStore. The reads are safe to fuse because they touch disjoint keys (req:* versus tokin:*/tokout:*/cost:*), and tokens and cost are only ever written by account() after the response completes, never by checkAndCount itself. Violation precedence stays identical to the pre-fusion implementation: a user's own violation still wins over their group's, and within one scope the order stays requests-per-minute, requests-per-day, tokens-per-day, tokens-per-month, cost-per-day, cost-per-month. New table-driven tests pin this precedence directly, engineering two simultaneous violation conditions per case and asserting only the higher-precedence one is reported. evaluateScope, budgetViolation, and tokenBudgetViolation are removed; buildBudgetProbes plus a loop inline the same logic into checkAndCount, which is what lets their reads share the fused round trip. Measured on the interpreted Yaegi harness (yaegi v0.16.1, the runtime this plugin actually ships under): RTT/req for a fully-limited chat request drops from 10.99 to 3.00, with cmds/req unchanged at 74 (this is a round-trip fix, not a command-count or allocation change). Single-request throughput rises from 112 to 335 req/s; at concurrency 16, from 121 to 380 req/s. counterStore.incrAndGetMulti is a required interface method, not an optional capability behind a type assertion: Yaegi can silently return ok=false when a comma-ok assertion checks a compiled concrete value against a plugin-declared interface (Trap 4, documented in matchesSentinel's own doc comment). A plain interface method call carries no such risk and is the same mechanism incrMulti already uses in production.	2026-08-22 11:39:50 +01:00
lukaszraczylo	5cc681496c	perf(limits): single-batch counter writes, bounded fallback retention ... Review fix round (2 Important + 5 folded) on 08cfdb4/b3fb705. 1. counterStore gains incrMulti(entries []counterIncr) ([]int64, error). redisStore.incrMulti sends every entry's INCRBY+EXPIRE pair as ONE pipeline (ttlToSeconds extracted from incrBy, now shared); memoryStore.incrMulti loops incrBy for interface conformance. checkAndCount batches every scope's req:min/req:day/req:hour into one storeIncrMulti call (down from up to 9 round trips for a 3-scope request) and reads req:min/req:day back from the reply, no extra GETs. account batches every scope's nonzero tok-in/tok-out/cost entries into one call (down from up to 27). Violation math and fail-open/fail-closed/latched semantics are unchanged — only the round-trip count moves. 2. memoryStoreMaxTTL (48h) caps every TTL memoryStore actually applies, clamped inside incrBy so incrMulti's loop inherits it for free. The in-process fallback is continuity, not history: applying the real hour/day/month TTLs there was growing live-key count ~30x, multiplying sweepEvery's own documented "167x incrBy cliff" by the same factor. Sweep cadence is unchanged (still 30s-gated); the clamp shrinks what there is to sweep instead. Folded: historyMaxSpan's margin comment corrected (~4 days, not a month); checkAndCount's godoc names req:hour; the integration current-hour assertion sums every point instead of reading only the last one (an hour-rollover race the suite cannot control); a new exact-boundary test proves tokin+tokout landing exactly on a TokensPerDay limit still violates; README documents the history route as unthrottled (admin-gate is its only trust boundary) and the fallback's 48h retention ceiling. Tests: countingIncrStore proves exactly one incrMulti call per checkAndCount and per account, with correct per-scope/metric/window values and unchanged violation math; a fake-RESP-server test pins incrMulti's wire format (one pipeline, entry-ordered replies); TTL clamp tests cover both the ceiling and the pass-through case at the boundary. Gates: go test ./... -count=1, -race, gofmt, vet, golangci-lint, make yaegi-check all green. make integration: every test passes except the pre-existing TestUsersFileHotReload flake, which is NOT caused by this change — verified via an isolated worktree at the prior commit (b3fb705) run under identical conditions (full suite, same host), which fails identically. This host's Docker Desktop is currently slow to propagate the bind-mounted users.json change into the containers; auth.go/users_file.go are untouched by this diff.	2026-08-21 01:02:28 +01:00
lukaszraczylo	33bd8bc2cc	fix(admin): browser key entry, batched usage reads, hardened responses ... Review fixes (2 Important + 4 folded): 1. GET /admin now serves the HTML shell with no authentication - a browser navigating directly to the URL cannot attach a custom header. The page reads an admin key from sessionStorage and sends it as x-api-key on /admin/api/* fetches, falling back to an inline key-entry form on a missing key or any 401/403. Both JSON routes keep their original 401/403/200 gate. Spec ss4 amended in place. 2. GET /admin/api/usage no longer serializes 6 GETs per user/group scope on the shared Redis connection. respClient gains getBatch (one pipelined round trip); counterStore gains getMulti; limiter gains storeGetMulti mirroring storeGet's fail-open/fail-closed policy. currentUsage now issues one getMulti per scope. 3. Provider baseUrl is stripped of userinfo and query before it is ever echoed in the overview JSON (sanitizeBaseURL), stronger than net/url's own Redacted (which leaves a bare username in place). 4. limiter.redisStatus now also returns the last failure's timestamp, so the dashboard can render "last error (Ns ago)" instead of a bare message that reads as currently-broken indefinitely. 5. Both JSON admin responses now set X-Content-Type-Options: nosniff, Cache-Control: no-store, and the same CSP the HTML page uses. 6. Added a test proving a file-sourced (replaceFileUsers) admin user gets the same access an inline admin user does. go test ./... -count=1 -race, gofmt, vet, make yaegi-check all pass.	2026-08-20 19:46:35 +01:00
lukaszraczylo	7ee635cd27	fix(resp): bound store-call latency, cap wire lengths, cover fail-closed budget path ... Bound respClient's worst-case latency to one respCallTimeout per do/pipeline call instead of per read/write/handshake step. Compute a single absolute deadline at call entry, apply it once via conn.SetDeadline after connecting (covering the AUTH/SELECT handshake and the call's own command I/O), and skip the reconnect-retry when the failing attempt was a timeout (errors.As -> net.Error, Timeout()) — a peer that accepted the connection and went silent must not be asked the same non-question twice. Previously each read, write, and dial got its own fresh 2s budget and every caller queued behind the client's single mutex, so a hung (not merely refused) Redis-compatible server could stall one checkAndCount call for 16-32s+. Cap decodeBulk/decodeArray's wire-declared lengths (respMaxBulkLen 64MiB, respMaxArrayLen 1<<20) before allocating: an unguarded "$9223372036854775807\r\n" overflows n+2 into a negative slice length and panics make([]byte, n+2); a merely huge-but-valid length forces an outsized allocation either way. decodeBulk also now verifies its two trailing bytes are CRLF. Bound readLine to respMaxLineLen (64KiB), reading byte-by-byte so it can bail before ever buffering an unbounded line from a peer that never sends '\n'. Fix TestLimiter_FailClosed_BudgetReadReturnsStoreDownViolation: the erroringStore it used fails incrBy too, so checkAndCount's own increment loop already returned a storeDown violation before evaluateScope (and therefore budgetViolation) ever ran, leaving budgetViolation's storeDown branch uncovered. Replace it with a stub that only fails get, so the test's TokensPerDay limit actually drives evaluateScope into budgetViolation's fail-closed path. Document redis_store.go's incrBy as deliberately at-least-once: the reconnect-retry can re-send INCRBY after a lost reply (possible over-count), and fail-open can additionally count in the fallback — both fail-safe (more restrictive), never fail-open in the unsafe direction. Validate Redis.DB >= 0 in newConfiguredLimiter alongside Address, so a negative db is a constructor error rather than a SELECT -1 on the wire. New tests: a hung (accepts, never replies) fake RESP server proving a respClient call and a failOpen limiter call both stay bounded rather than stacking timeouts; decodeReply subtests for an oversized bulk length, a length just over the cap, a huge array length, a bad bulk trailer, and an oversized line; a negative-DB constructor-rejection test.	2026-08-20 02:16:07 +01:00
lukaszraczylo	5eab51bdef	feat: stdlib RESP client and redis-backed limit store with fail-open ... Add resp.go: a minimal stdlib-only RESP2 client (respClient) for a single Redis-compatible server. It lazily dials one mutex-guarded connection, applies 2s dial/read/write deadlines, sends AUTH (if a password is set) then SELECT on every new connection, and reconnects once on any I/O error before giving up. do and pipeline encode commands as "*N\r\n$len\r\n..." and decode +/-/:/$/* replies. Add redis_store.go: redisStore implements counterStore over respClient. incrBy pipelines INCRBY and EXPIRE in one round trip and returns INCRBY's value; get treats a missing key (RESP null bulk) as 0. Wire the limiter's fail-open/fail-closed policy in limits.go, which Task 5 stored but never read: a configured store's error now routes through storeIncrBy/storeGet. failOpen=true falls back to the in-process memoryStore transparently; failOpen=false makes checkAndCount return a storeDown violation and makes account drop the sample. Both paths log through an injectable, 30s-rate-limited logf. Wire newGateway (llmgateway.go) to build a redisStore-backed limiter when Config.Redis is set (password through resolveSecret, failOpen defaulting to true unless RedisConfig.FailOpen says otherwise) and a memory-only limiter otherwise, and to route pricing.go's unknown-model warnings through the gateway's own logf. Validate LimitsConfig at construction: a new LimitsConfig.validate rejects a negative int64 count limit or a negative/NaN/Inf cost limit, called from newAuthStore for every group and every user (inline and file-sourced) so a malformed limit is a constructor error instead of silently behaving as unlimited. Add RequestsPerDay, TokensPerMonth, and CostPerMonthUSD enforcement tests alongside the existing per-minute/day/cost-per-day cases, plus an erroringStore-backed test pair proving the fail-open (fallback counted) and fail-closed (storeDown violation) branches directly. Folded the validation work into this single commit rather than a second one: it interleaves with the fail-open changes across limits.go, limits_test.go, and llmgateway_test.go closely enough that splitting by hunk would be artificial.	2026-08-20 02:00:26 +01:00