mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-20 05:56:56 +00:00
fix(gateway): close billing, limit, routing and translation gaps
Billing: - Passthrough SSE streams now bill the usage the upstream reports. The gateway falls back to a request-size estimate only on a 2xx stream or a truncated JSON body. - A unified stream that fails after it starts is estimated, not billed at zero. - Billing uses the built-in LiteLLM price table and honours modelMeta free: true. - Pricing entries are validated. Cost arithmetic rounds the total and saturates on overflow. - Gemini thinking tokens count as completion tokens. Limits and storage: - A rejected request rolls back its counts on the other scopes, so one user cannot lock out a group. - Redis EXPIRE replies are checked. Keys recreated without a TTL are repaired at once. - A mutating pipeline is not retried after reply bytes arrive. - History, model totals and target usage report a store outage. Routing and translation: - Discovered ids with a provider-like prefix are listed only in owner/id form, so listing and routing agree. - Anthropic and Gemini discovery follows pagination. - The developer role maps to system. - stream_options.include_usage is honoured on translated streams. - tool_choice none keeps tools when history holds tool blocks. - refusal maps to content_filter. Data URI parameters are stripped. Streaming, retry and failover: - SSE accepts CR, LF and CRLF line ends. This closes a CR injection path. - Retry-After is never jittered. A cancelled wait returns the context error. - The watchdog timer stops at end of body. - Oversized bodies return 413. - A failed stream ends with an error event. - Failover restores pre-existing headers. Client cancels do not count as provider successes. Also: "admin" is a reserved provider name. metrics.path must start with "/". Pricing warnings go to the right gateway instance. The admin targets view sends access as null or [] so the two states differ. Upstream organisation headers are stripped. The users-file reload error is logged once per error. Dead code is removed, and test-only helpers move to test files. Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
This commit is contained in:
+12
-5
@@ -506,11 +506,18 @@ var configNamePattern = regexp.MustCompile(`^[a-zA-Z0-9._-]+$`)
|
||||
// own fixed top-level routes: "v1" is the unified API namespace
|
||||
// (/v1/chat/completions, /v1/models, /v1/embeddings, /v1/mcp/servers,
|
||||
// /v1/agents), "mcp" and "a2a" are the MCP/A2A target-proxy prefixes
|
||||
// (/mcp/{name}/..., /a2a/{name}/...). ServeHTTP checks those fixed routes
|
||||
// ahead of passthrough/target routing (llmgateway.go), so a provider, MCP
|
||||
// server, or agent configured under one of these names would never be
|
||||
// reachable — reject it here instead of silently shadowing it.
|
||||
var reservedConfigNames = map[string]bool{"v1": true, "mcp": true, "a2a": true}
|
||||
// (/mcp/{name}/..., /a2a/{name}/...), and "admin" for the dashboard's own
|
||||
// fixed /admin, /admin/assets/*, and /admin/api/* routes (review-auth.md
|
||||
// finding F9: a provider literally named "admin" would otherwise shadow
|
||||
// every one of those, an easy config typo away). ServeHTTP checks those
|
||||
// fixed routes ahead of passthrough/target routing (llmgateway.go), so a
|
||||
// provider, MCP server, or agent configured under one of these names
|
||||
// would never be reachable — reject it here instead of silently
|
||||
// shadowing it. Reserved unconditionally, the same as "v1"/"mcp"/"a2a"
|
||||
// already are, rather than only when admin.enabled: a provider named
|
||||
// "admin" configured before the dashboard is turned on would otherwise
|
||||
// silently lose its own routes the moment someone later enables it.
|
||||
var reservedConfigNames = map[string]bool{"v1": true, "mcp": true, "a2a": true, "admin": true}
|
||||
|
||||
// dotOnlyConfigNames are the two names configNamePattern's character class
|
||||
// permits (it allows ".") but that are rejected anyway: "." and ".." read
|
||||
|
||||
Reference in New Issue
Block a user