mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
fix(target-health): record a hung federated server as a failure
The previous round suppressed every context-caused federation error. A server that accepted the connection and then never answered therefore read "unknown" forever whenever federation carried the only traffic, which is the outage shape this feature exists to surface. Split the two causes instead. A server still waiting for a fan-out slot when the budget expires is never dialed, so the semaphore acquire now selects on the fan-out context and records nothing for it. A server this gateway did dial records a failure on a deadline, because it had its chance to answer. Only a caller hanging up stays unattributable. Document the split rule, correct a stale comment about a second record site, and state that "unknown" reads two ways rather than three. Claude-Session: https://claude.ai/code/session_01EebV5jyWvYocByNQ3oss1w
This commit is contained in:
@@ -1898,11 +1898,14 @@ plugin's config.
|
||||
only when the outbound call itself fails (`err != nil` from the
|
||||
backend call) — a JSON-RPC-level `error` in the backend's own
|
||||
response still counts as reachable, since the server answered. A
|
||||
context-caused failure (the caller canceling, or the shared fan-out
|
||||
deadline expiring) records nothing at all, the same client-cancel
|
||||
rule the per-server proxy above applies; a genuinely hung server is
|
||||
still caught by the active probe sweep or its own per-server proxy
|
||||
health.
|
||||
context-caused failure is split two ways: the caller hanging up
|
||||
(`context.Canceled`) records nothing, the same client-cancel rule the
|
||||
per-server proxy above applies, but a server this gateway actually
|
||||
dialed that never answers before the shared fan-out budget expires
|
||||
(`context.DeadlineExceeded`) records a real failure — it had a chance
|
||||
to answer and did not. A server still waiting for a free fan-out slot
|
||||
when that same budget expires — never dialed at all — records
|
||||
nothing, since this gateway never gave it the chance.
|
||||
- **Active probes are opt-in** (`targetHealth.enabled: true`) and lazy:
|
||||
no timer, no background goroutine running on its own. A probe sweep
|
||||
is triggered by `/metrics`, `GET /admin/api/targets`, `GET
|
||||
@@ -1929,8 +1932,7 @@ plugin's config.
|
||||
whether or not the target has ever succeeded). A target with one or
|
||||
two recent failures after a success still reads `"healthy"` until the
|
||||
threshold is actually crossed.
|
||||
- **`"unknown"` reads three ways**, all sharing one JSON `state`
|
||||
value:
|
||||
- **`"unknown"` reads two ways**, both sharing one JSON `state` value:
|
||||
1. never observed at all — no probe or passive traffic has reached
|
||||
this target yet. `GET /admin/api/targets` omits every health field
|
||||
but `state`/`consecutiveFailures` for this case.
|
||||
@@ -1941,9 +1943,10 @@ plugin's config.
|
||||
has never seen answer is not `"healthy"` either. Every other
|
||||
health field is still present here, since there is a real
|
||||
observation to show.
|
||||
3. either way, `llmgateway_target_healthy` emits no sample while
|
||||
`state` is `"unknown"` — a gap in the series is honest, a
|
||||
fabricated `1` is not.
|
||||
|
||||
Either way, `llmgateway_target_healthy` emits no sample while `state`
|
||||
is `"unknown"` — a gap in the series is honest, a fabricated `1` is
|
||||
not.
|
||||
|
||||
## Security notes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user