fix(retry): retries-after-first semantics, ctx-aware backoff, hardened classification

Critical: attempts now counts retries performed AFTER the first try
(default 1 -> two tries total), not the total try count. The prior
reading made retry:{enabled:true}'s own documented default perform zero
retries. (*retryPolicy).totalTries() is the single conversion point,
clamping any attempts<1 to 1 so do()'s loop bound can never compute <=0
and skip calling call() entirely (no (nil,nil) return path).

Backoff waits are now ctx-aware: sleepFn func(time.Duration) is replaced
by waitFn func(context.Context, time.Duration) bool, whose production
implementation (defaultWaitFn) selects between time.After(d) and
ctx.Done(). A context canceled mid-wait aborts the remaining retries
immediately instead of blocking out the full backoff.

Also: classify a request-build failure (invalid method/URL) as
non-transient via errRequestBuildFailed, since retrying an identically
malformed request only wastes attempts; drop the drain cap to a
dedicated 64KiB constant instead of the 32MiB response cap; fix a
t.Fatal call from a non-test goroutine in the mid-stream-death test;
drop the misleading omitempty on Config.Retry (a struct value is never
'empty' to encoding/json); document the Retry-After >2s-falls-back-to-
backoff behavior and cover it with a direct table test.

README: add the retry config table, a RetryConfig field reference, and
a Retry section documenting the semantics. examples/kubernetes.yaml
gains a retry block alongside its other optional sections.

Gate: go test -race, gofmt, vet, yaegi-check, golangci-lint (incl a
fieldalignment fix on two new test structs) all clean. 295 subtests
pass, 0 failed. integration/ untouched.
This commit is contained in:
2026-08-20 17:34:52 +01:00
parent 34968e27b2
commit 722e6f3d29
6 changed files with 587 additions and 150 deletions
+26 -12
View File
@@ -15,15 +15,21 @@ import (
// the middleware's YAML/testData. Every limit or list field follows the
// convention: zero/empty/omitted means unlimited/all.
type Config struct {
Providers map[string]*ProviderConfig `json:"providers,omitempty"`
Groups map[string]*GroupConfig `json:"groups,omitempty"`
Pricing map[string]*ModelPricing `json:"pricing,omitempty"`
MCPServers map[string]*TargetConfig `json:"mcpServers,omitempty"`
Agents map[string]*AgentConfig `json:"agents,omitempty"`
Users *UsersConfig `json:"users,omitempty"`
Redis *RedisConfig `json:"redis,omitempty"`
Retry RetryConfig `json:"retry,omitempty"`
PassthroughUnknown bool `json:"passthroughUnknown,omitempty"`
Providers map[string]*ProviderConfig `json:"providers,omitempty"`
Groups map[string]*GroupConfig `json:"groups,omitempty"`
Pricing map[string]*ModelPricing `json:"pricing,omitempty"`
MCPServers map[string]*TargetConfig `json:"mcpServers,omitempty"`
Agents map[string]*AgentConfig `json:"agents,omitempty"`
Users *UsersConfig `json:"users,omitempty"`
Redis *RedisConfig `json:"redis,omitempty"`
// Retry is a struct value, not a pointer, because its own Enabled
// field is the on/off signal (unlike Redis/Users, where the block's
// mere presence is the signal) — so its tag omits "omitempty":
// encoding/json never treats a struct value as "empty" regardless of
// its fields, so "omitempty" here would be a no-op that misleadingly
// implies otherwise.
Retry RetryConfig `json:"retry"`
PassthroughUnknown bool `json:"passthroughUnknown,omitempty"`
}
// ProviderConfig describes one upstream LLM provider.
@@ -85,9 +91,17 @@ type RedisConfig struct {
// Attempts and Backoff are validated (and defaulted, when left zero) by
// newRetryPolicy (retry.go) only when Enabled is true.
type RetryConfig struct {
Backoff string `json:"backoff,omitempty"`
Attempts int `json:"attempts,omitempty"`
Enabled bool `json:"enabled,omitempty"`
// Backoff is the base wait before the first retry (a Go duration
// string, e.g. "250ms"); it doubles on each further retry, capped at
// 2s per wait. Defaults to "250ms" when Enabled and left empty.
Backoff string `json:"backoff,omitempty"`
// Attempts is the number of retries performed AFTER the first try —
// not the total try count. 1 (the default, applied when Enabled and
// left at 0) allows one retry: two tries total. The maximum, 3,
// allows three retries: four tries total. A value outside 1..3 is a
// construction error.
Attempts int `json:"attempts,omitempty"`
Enabled bool `json:"enabled,omitempty"`
}
// ModelPricing overrides the built-in per-model price table.