mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
docs(readme): note bounded fire-and-forget writes for provider counters
Re-read the provider/model success-rate paragraph against the FIXED
deadline behavior (v0.22 review round, round 2) per instruction — the
claim itself ("a real deadline counts as a failure") was already
accurate and needed no correction; the multi-wrap bug it was allegedly
describing never surfaced through the README, since the README
describes intended behavior, not the wrap-depth implementation detail
that broke it. Added: FOLDED-1's bound (64 in flight per process,
lossy-under-pressure by design) on the fire-and-forget write this
paragraph already described, and an explicit note that enforcement never
goes through this path.
This commit is contained in:
@@ -592,7 +592,12 @@ http:
|
||||
[Admin](#admin)) is the only reader; the write itself runs off the
|
||||
request's own goroutine (fire-and-forget — this is telemetry, and
|
||||
nothing in the request path waits on or gates against it), so it never
|
||||
adds latency to a response, streaming included.
|
||||
adds latency to a response, streaming included. That goroutine is
|
||||
bounded (64 in flight per process) and lossy under sustained pressure:
|
||||
a write that cannot start immediately is dropped, not queued — a
|
||||
deliberate trade-off against an unbounded goroutine pile-up if the
|
||||
configured store ever goes slow without going fully down. Enforcement
|
||||
(request/token/cost limits) never goes through this path at all.
|
||||
- **Native passthrough accounts provider-level only, never per-model.**
|
||||
`POST /{provider}/...` has no resolved model at request time — the
|
||||
upstream model, if any, only appears in the response body, read after
|
||||
|
||||
Reference in New Issue
Block a user