perf(limits): fuse admission reads into one round trip

checkAndCount already batched its request-counter increments into one
pipeline. evaluateScope then read every scope's token and cost budgets
through up to four further serial round trips per scope. A request with
a fully-limited user and group scope paid nine round trips before the
upstream call started.

This change fuses every budget read into the SAME pipeline as the
increments, through a new incrAndGetMulti method on counterStore. The
reads are safe to fuse because they touch disjoint keys (req:* versus
tokin:*/tokout:*/cost:*), and tokens and cost are only ever written by
account() after the response completes, never by checkAndCount itself.

Violation precedence stays identical to the pre-fusion implementation:
a user's own violation still wins over their group's, and within one
scope the order stays requests-per-minute, requests-per-day,
tokens-per-day, tokens-per-month, cost-per-day, cost-per-month. New
table-driven tests pin this precedence directly, engineering two
simultaneous violation conditions per case and asserting only the
higher-precedence one is reported.

evaluateScope, budgetViolation, and tokenBudgetViolation are removed;
buildBudgetProbes plus a loop inline the same logic into checkAndCount,
which is what lets their reads share the fused round trip.

Measured on the interpreted Yaegi harness (yaegi v0.16.1, the runtime
this plugin actually ships under): RTT/req for a fully-limited chat
request drops from 10.99 to 3.00, with cmds/req unchanged at 74 (this
is a round-trip fix, not a command-count or allocation change).
Single-request throughput rises from 112 to 335 req/s; at concurrency
16, from 121 to 380 req/s.

counterStore.incrAndGetMulti is a required interface method, not an
optional capability behind a type assertion: Yaegi can silently return
ok=false when a comma-ok assertion checks a compiled concrete value
against a plugin-declared interface (Trap 4, documented in
matchesSentinel's own doc comment). A plain interface method call
carries no such risk and is the same mechanism incrMulti already uses
in production.
This commit is contained in:
2026-08-22 11:39:50 +01:00
parent 2d10509e20
commit a418814b34
7 changed files with 595 additions and 162 deletions
+4
View File
@@ -1407,6 +1407,10 @@ func (s *countingMultiStore) getMulti(keys []string) ([]int64, error) {
func (s *countingMultiStore) incrMulti(entries []counterIncr) ([]int64, error) {
return make([]int64, len(entries)), nil
}
func (s *countingMultiStore) incrAndGetMulti(entries []counterIncr, reads []string) ([]int64, []int64, error) {
readVals, err := s.getMulti(reads)
return make([]int64, len(entries)), readVals, err
}
func TestLimiterCurrentUsage_BatchesOneGetMultiCallForAllScopes(t *testing.T) {
t.Parallel()