lukaszraczylo/traefik-llmgateway
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/traefik-llmgateway.git synced 2026-10-19 05:47:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(providers): add progress-based upstream request timeout

Browse Source 
Adapters previously issued upstream requests with no timeout at all.
A provider that accepted a connection and then stalled hung the
gateway request forever. newAdapterHTTPClient now sets
Transport.ResponseHeaderTimeout, and every response body read
(providers.go's upstreamBytes, routes_passthrough.go's proxyUpstream)
runs through a watchdogBody that resets on every successful read and
cancels the request once timeout passes with no progress.

The idle-progress design lets a long stream that keeps emitting tokens
run indefinitely. Only silence, never total duration, ends a request.

Config.RequestTimeout sets the global default (five minutes when
unset). ProviderConfig.RequestTimeout overrides it per provider. Both
are Go duration strings, validated at construction; a zero or negative
value is rejected rather than treated as unlimited, since that would
reintroduce the hang this feature fixes.

Native passthrough (routes_passthrough.go) and the MCP/A2A target
proxy (mcp_a2a.go) share the identical watchdog mechanism through
proxyUpstream's new timeout parameter.

A watchdog-fired timeout wraps errProviderTimeout and names the
provider. It never wraps context.Canceled, so it stays distinguishable
from a genuine client disconnect in every existing log classification.
This commit is contained in:
lukaszraczylo
2026-08-23 01:36:34 +01:00
parent a12b02188f
commit 503fc33933
12 changed files with 1130 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
retry_test.go
+1 -1
View File
@@ -616,7 +616,7 @@ func TestUpstreamJSON_RequestBuildFailure_NotRetried(t *testing.T) {
return true
}}
client := &http.Client{}
_, err := upstreamJSON(context.Background(), client, "IN VALID", "http://127.0.0.1:1/x", nil, nil, p)
_, err := upstreamJSON(context.Background(), client, "IN VALID", "http://127.0.0.1:1/x", nil, nil, p, defaultRequestTimeout, "p1")
if err == nil {
t.Fatal("upstreamJSON: want an error for an invalid method, got nil")
}