fix(resp): bound store-call latency, cap wire lengths, cover fail-closed budget path

Bound respClient's worst-case latency to one respCallTimeout per
do/pipeline call instead of per read/write/handshake step. Compute a
single absolute deadline at call entry, apply it once via
conn.SetDeadline after connecting (covering the AUTH/SELECT handshake
and the call's own command I/O), and skip the reconnect-retry when the
failing attempt was a timeout (errors.As -> net.Error, Timeout()) — a
peer that accepted the connection and went silent must not be asked
the same non-question twice. Previously each read, write, and dial got
its own fresh 2s budget and every caller queued behind the client's
single mutex, so a hung (not merely refused) Redis-compatible server
could stall one checkAndCount call for 16-32s+.

Cap decodeBulk/decodeArray's wire-declared lengths (respMaxBulkLen
64MiB, respMaxArrayLen 1<<20) before allocating: an unguarded
"$9223372036854775807\r\n" overflows n+2 into a negative slice length
and panics make([]byte, n+2); a merely huge-but-valid length forces an
outsized allocation either way. decodeBulk also now verifies its two
trailing bytes are CRLF. Bound readLine to respMaxLineLen (64KiB),
reading byte-by-byte so it can bail before ever buffering an unbounded
line from a peer that never sends '\n'.

Fix TestLimiter_FailClosed_BudgetReadReturnsStoreDownViolation: the
erroringStore it used fails incrBy too, so checkAndCount's own
increment loop already returned a storeDown violation before
evaluateScope (and therefore budgetViolation) ever ran, leaving
budgetViolation's storeDown branch uncovered. Replace it with a stub
that only fails get, so the test's TokensPerDay limit actually drives
evaluateScope into budgetViolation's fail-closed path.

Document redis_store.go's incrBy as deliberately at-least-once: the
reconnect-retry can re-send INCRBY after a lost reply (possible
over-count), and fail-open can additionally count in the fallback —
both fail-safe (more restrictive), never fail-open in the unsafe
direction.

Validate Redis.DB >= 0 in newConfiguredLimiter alongside Address, so a
negative db is a constructor error rather than a SELECT -1 on the wire.

New tests: a hung (accepts, never replies) fake RESP server proving a
respClient call and a failOpen limiter call both stay bounded rather
than stacking timeouts; decodeReply subtests for an oversized bulk
length, a length just over the cap, a huge array length, a bad bulk
trailer, and an oversized line; a negative-DB constructor-rejection
test.
This commit is contained in:
2026-08-20 02:16:07 +01:00
parent 5eab51bdef
commit 7ee635cd27
7 changed files with 303 additions and 58 deletions
+23 -3
View File
@@ -433,18 +433,38 @@ func TestLimiter_FailClosed_StoreErrorReturnsStoreDownViolation(t *testing.T) {
}
}
// succeedIncrFailGetStore is a counterStore stub whose incrBy always
// succeeds and whose get always errors. It isolates budgetViolation's own
// storeDown branch (limits.go) from checkAndCount's earlier incrCounter
// one: an erroringStore that fails both methods makes checkAndCount's
// initial req:min/req:day increments fail closed and return before
// evaluateScope — and therefore budgetViolation — ever runs, so a test
// built on it cannot actually prove budgetViolation's own branch works.
type succeedIncrFailGetStore struct {
getErr error
}
func (s *succeedIncrFailGetStore) incrBy(string, int64, time.Duration) (int64, error) {
return 1, nil
}
func (s *succeedIncrFailGetStore) get(string) (int64, error) { return 0, s.getErr }
// TestLimiter_FailClosed_BudgetReadReturnsStoreDownViolation covers the
// budgetViolation fail-closed path specifically (as opposed to the
// request-counter incrCounter path covered above): a store error on a
// request-counter incrCounter path TestLimiter_FailClosed_
// StoreErrorReturnsStoreDownViolation covers above): a store error on a
// token/cost budget read also refuses the request when failOpen is false.
// The store's incrBy succeeds so checkAndCount's req:min/req:day
// increments pass and evaluateScope actually reaches budgetViolation; a
// TokensPerDay limit makes evaluateScope call it.
func TestLimiter_FailClosed_BudgetReadReturnsStoreDownViolation(t *testing.T) {
store := &erroringStore{err: errors.New("boom")}
store := &succeedIncrFailGetStore{getErr: errors.New("boom")}
l := newLimiter(store, false)
scopes := []limitScope{{kind: "user", id: "u", limits: &LimitsConfig{TokensPerDay: 100}}}
v := l.checkAndCount(scopes)
if v == nil {
t.Fatal("want a storeDown violation on a store error with failOpen=false")
t.Fatal("want a storeDown violation from the budget-read path with failOpen=false")
}
if !v.storeDown {
t.Error("v.storeDown = false, want true")