mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
lukaszraczylo
5eab51bdef
Add resp.go: a minimal stdlib-only RESP2 client (respClient) for a single Redis-compatible server. It lazily dials one mutex-guarded connection, applies 2s dial/read/write deadlines, sends AUTH (if a password is set) then SELECT on every new connection, and reconnects once on any I/O error before giving up. do and pipeline encode commands as "*N\r\n$len\r\n..." and decode +/-/:/$/* replies. Add redis_store.go: redisStore implements counterStore over respClient. incrBy pipelines INCRBY and EXPIRE in one round trip and returns INCRBY's value; get treats a missing key (RESP null bulk) as 0. Wire the limiter's fail-open/fail-closed policy in limits.go, which Task 5 stored but never read: a configured store's error now routes through storeIncrBy/storeGet. failOpen=true falls back to the in-process memoryStore transparently; failOpen=false makes checkAndCount return a storeDown violation and makes account drop the sample. Both paths log through an injectable, 30s-rate-limited logf. Wire newGateway (llmgateway.go) to build a redisStore-backed limiter when Config.Redis is set (password through resolveSecret, failOpen defaulting to true unless RedisConfig.FailOpen says otherwise) and a memory-only limiter otherwise, and to route pricing.go's unknown-model warnings through the gateway's own logf. Validate LimitsConfig at construction: a new LimitsConfig.validate rejects a negative int64 count limit or a negative/NaN/Inf cost limit, called from newAuthStore for every group and every user (inline and file-sourced) so a malformed limit is a constructor error instead of silently behaving as unlimited. Add RequestsPerDay, TokensPerMonth, and CostPerMonthUSD enforcement tests alongside the existing per-minute/day/cost-per-day cases, plus an erroringStore-backed test pair proving the fail-open (fallback counted) and fail-closed (storeDown violation) branches directly. Folded the validation work into this single commit rather than a second one: it interleaves with the fail-open changes across limits.go, limits_test.go, and llmgateway_test.go closely enough that splitting by hunk would be artificial.