mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
feat(providers): add progress-based upstream request timeout
Adapters previously issued upstream requests with no timeout at all. A provider that accepted a connection and then stalled hung the gateway request forever. newAdapterHTTPClient now sets Transport.ResponseHeaderTimeout, and every response body read (providers.go's upstreamBytes, routes_passthrough.go's proxyUpstream) runs through a watchdogBody that resets on every successful read and cancels the request once timeout passes with no progress. The idle-progress design lets a long stream that keeps emitting tokens run indefinitely. Only silence, never total duration, ends a request. Config.RequestTimeout sets the global default (five minutes when unset). ProviderConfig.RequestTimeout overrides it per provider. Both are Go duration strings, validated at construction; a zero or negative value is rejected rather than treated as unlimited, since that would reintroduce the hang this feature fixes. Native passthrough (routes_passthrough.go) and the MCP/A2A target proxy (mcp_a2a.go) share the identical watchdog mechanism through proxyUpstream's new timeout parameter. A watchdog-fired timeout wraps errProviderTimeout and names the provider. It never wraps context.Canceled, so it stays distinguishable from a genuine client disconnect in every existing log classification.
This commit is contained in:
+2
-1
@@ -52,7 +52,8 @@ func (f *fakeAdapter) listModels(ctx context.Context) ([]string, error) {
|
||||
}
|
||||
return f.listModelsFn(ctx)
|
||||
}
|
||||
func (f *fakeAdapter) httpClient() *http.Client { return nil }
|
||||
func (f *fakeAdapter) httpClient() *http.Client { return nil }
|
||||
func (f *fakeAdapter) requestTimeout() time.Duration { return defaultRequestTimeout }
|
||||
|
||||
// recordingLog is a thread-safe log func for tests, capturing formatted
|
||||
// lines instead of writing to stderr.
|
||||
|
||||
Reference in New Issue
Block a user