mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-19 05:47:48 +00:00
fix(retry): retries-after-first semantics, ctx-aware backoff, hardened classification
Critical: attempts now counts retries performed AFTER the first try
(default 1 -> two tries total), not the total try count. The prior
reading made retry:{enabled:true}'s own documented default perform zero
retries. (*retryPolicy).totalTries() is the single conversion point,
clamping any attempts<1 to 1 so do()'s loop bound can never compute <=0
and skip calling call() entirely (no (nil,nil) return path).
Backoff waits are now ctx-aware: sleepFn func(time.Duration) is replaced
by waitFn func(context.Context, time.Duration) bool, whose production
implementation (defaultWaitFn) selects between time.After(d) and
ctx.Done(). A context canceled mid-wait aborts the remaining retries
immediately instead of blocking out the full backoff.
Also: classify a request-build failure (invalid method/URL) as
non-transient via errRequestBuildFailed, since retrying an identically
malformed request only wastes attempts; drop the drain cap to a
dedicated 64KiB constant instead of the 32MiB response cap; fix a
t.Fatal call from a non-test goroutine in the mid-stream-death test;
drop the misleading omitempty on Config.Retry (a struct value is never
'empty' to encoding/json); document the Retry-After >2s-falls-back-to-
backoff behavior and cover it with a direct table test.
README: add the retry config table, a RetryConfig field reference, and
a Retry section documenting the semantics. examples/kubernetes.yaml
gains a retry block alongside its other optional sections.
Gate: go test -race, gofmt, vet, yaegi-check, golangci-lint (incl a
fieldalignment fix on two new test structs) all clean. 295 subtests
pass, 0 failed. integration/ untouched.
This commit is contained in:
+26
-12
@@ -15,15 +15,21 @@ import (
|
||||
// the middleware's YAML/testData. Every limit or list field follows the
|
||||
// convention: zero/empty/omitted means unlimited/all.
|
||||
type Config struct {
|
||||
Providers map[string]*ProviderConfig `json:"providers,omitempty"`
|
||||
Groups map[string]*GroupConfig `json:"groups,omitempty"`
|
||||
Pricing map[string]*ModelPricing `json:"pricing,omitempty"`
|
||||
MCPServers map[string]*TargetConfig `json:"mcpServers,omitempty"`
|
||||
Agents map[string]*AgentConfig `json:"agents,omitempty"`
|
||||
Users *UsersConfig `json:"users,omitempty"`
|
||||
Redis *RedisConfig `json:"redis,omitempty"`
|
||||
Retry RetryConfig `json:"retry,omitempty"`
|
||||
PassthroughUnknown bool `json:"passthroughUnknown,omitempty"`
|
||||
Providers map[string]*ProviderConfig `json:"providers,omitempty"`
|
||||
Groups map[string]*GroupConfig `json:"groups,omitempty"`
|
||||
Pricing map[string]*ModelPricing `json:"pricing,omitempty"`
|
||||
MCPServers map[string]*TargetConfig `json:"mcpServers,omitempty"`
|
||||
Agents map[string]*AgentConfig `json:"agents,omitempty"`
|
||||
Users *UsersConfig `json:"users,omitempty"`
|
||||
Redis *RedisConfig `json:"redis,omitempty"`
|
||||
// Retry is a struct value, not a pointer, because its own Enabled
|
||||
// field is the on/off signal (unlike Redis/Users, where the block's
|
||||
// mere presence is the signal) — so its tag omits "omitempty":
|
||||
// encoding/json never treats a struct value as "empty" regardless of
|
||||
// its fields, so "omitempty" here would be a no-op that misleadingly
|
||||
// implies otherwise.
|
||||
Retry RetryConfig `json:"retry"`
|
||||
PassthroughUnknown bool `json:"passthroughUnknown,omitempty"`
|
||||
}
|
||||
|
||||
// ProviderConfig describes one upstream LLM provider.
|
||||
@@ -85,9 +91,17 @@ type RedisConfig struct {
|
||||
// Attempts and Backoff are validated (and defaulted, when left zero) by
|
||||
// newRetryPolicy (retry.go) only when Enabled is true.
|
||||
type RetryConfig struct {
|
||||
Backoff string `json:"backoff,omitempty"`
|
||||
Attempts int `json:"attempts,omitempty"`
|
||||
Enabled bool `json:"enabled,omitempty"`
|
||||
// Backoff is the base wait before the first retry (a Go duration
|
||||
// string, e.g. "250ms"); it doubles on each further retry, capped at
|
||||
// 2s per wait. Defaults to "250ms" when Enabled and left empty.
|
||||
Backoff string `json:"backoff,omitempty"`
|
||||
// Attempts is the number of retries performed AFTER the first try —
|
||||
// not the total try count. 1 (the default, applied when Enabled and
|
||||
// left at 0) allows one retry: two tries total. The maximum, 3,
|
||||
// allows three retries: four tries total. A value outside 1..3 is a
|
||||
// construction error.
|
||||
Attempts int `json:"attempts,omitempty"`
|
||||
Enabled bool `json:"enabled,omitempty"`
|
||||
}
|
||||
|
||||
// ModelPricing overrides the built-in per-model price table.
|
||||
|
||||
Reference in New Issue
Block a user