feat: stdlib RESP client and redis-backed limit store with fail-open

Add resp.go: a minimal stdlib-only RESP2 client (respClient) for a
single Redis-compatible server. It lazily dials one mutex-guarded
connection, applies 2s dial/read/write deadlines, sends AUTH (if a
password is set) then SELECT on every new connection, and reconnects
once on any I/O error before giving up. do and pipeline encode
commands as "*N\r\n$len\r\n..." and decode +/-/:/$/* replies.

Add redis_store.go: redisStore implements counterStore over
respClient. incrBy pipelines INCRBY and EXPIRE in one round trip and
returns INCRBY's value; get treats a missing key (RESP null bulk) as
0.

Wire the limiter's fail-open/fail-closed policy in limits.go, which
Task 5 stored but never read: a configured store's error now routes
through storeIncrBy/storeGet. failOpen=true falls back to the
in-process memoryStore transparently; failOpen=false makes
checkAndCount return a storeDown violation and makes account drop the
sample. Both paths log through an injectable, 30s-rate-limited logf.

Wire newGateway (llmgateway.go) to build a redisStore-backed limiter
when Config.Redis is set (password through resolveSecret, failOpen
defaulting to true unless RedisConfig.FailOpen says otherwise) and a
memory-only limiter otherwise, and to route pricing.go's unknown-model
warnings through the gateway's own logf.

Validate LimitsConfig at construction: a new LimitsConfig.validate
rejects a negative int64 count limit or a negative/NaN/Inf cost limit,
called from newAuthStore for every group and every user (inline and
file-sourced) so a malformed limit is a constructor error instead of
silently behaving as unlimited.

Add RequestsPerDay, TokensPerMonth, and CostPerMonthUSD enforcement
tests alongside the existing per-minute/day/cost-per-day cases, plus
an erroringStore-backed test pair proving the fail-open (fallback
counted) and fail-closed (storeDown violation) branches directly.

Folded the validation work into this single commit rather than a
second one: it interleaves with the fail-open changes across
limits.go, limits_test.go, and llmgateway_test.go closely enough that
splitting by hunk would be artificial.
This commit is contained in:
2026-08-20 02:00:26 +01:00
parent ed1b1d7b2f
commit 5eab51bdef
9 changed files with 1538 additions and 49 deletions
+6
View File
@@ -118,6 +118,9 @@ func newAuthStore(cfg *Config) (*authStore, error) {
nowFn: time.Now,
}
for name, gc := range cfg.Groups {
if err := gc.Limits.validate(); err != nil {
return nil, fmt.Errorf("llmgateway: group %q: %w", name, err)
}
a.groups[name] = &group{
limits: gc.Limits,
name: name,
@@ -155,6 +158,9 @@ func (a *authStore) buildEntry(uc *UserConfig) (*authEntry, error) {
if !ok {
return nil, fmt.Errorf("llmgateway: user %q references unknown group %q", uc.Name, uc.Group)
}
if err := uc.Limits.validate(); err != nil {
return nil, fmt.Errorf("llmgateway: user %q: %w", uc.Name, err)
}
key, err := resolveSecret(uc.APIKey)
if err != nil {
return nil, fmt.Errorf("llmgateway: user %q: %w", uc.Name, err)