mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
fix(passthrough): close Content-Type bypass and matcher-parity bug
Independent review measured two real gaps in the round-1 model
enforcement and fixed both here, plus documentation and comment
regressions the round introduced.
MUST FIX: peekPassthroughModel's Content-Type gate was an ALLOW-list
("application/json" or skip") and fully client-controlled. The
reviewer sent "Content-Type: text/plain" with {"model":"EXPENSIVE"}
and got a 200 from a group scoped to a cheap model. This commit
replaces the whole gating approach:
1. A group with an empty Models list has nothing for allowsModel to
reject, so the request body is never read at all
(group.hasModelRestriction). This is the overwhelming majority of
traffic, including the operator's own "home" group, and removes
the memory-buffering concern entirely for it.
2. For a group WITH a restricted Models list, the Content-Type gate
is now a SKIP-list of genuinely binary types (multipart/, audio/,
image/, video/, application/octet-stream). Every other
Content-Type, including "text/plain" or none at all, is
inspected. The attacker no longer controls whether the check
runs.
3. The peek reads at most 64KiB (maxModelPeekBytes), restored via
io.MultiReader so the remainder streams straight to the upstream
instead of a second full buffer. A bounded JSON token walk
(scanTopLevelModel) finds a top-level "model" key even when the
rest of the body is truncated past the cap.
4. A restricted group whose peek window never finds "model" is
denied (403), fail closed. This also means a Gemini passthrough
request (its model lives in the URL, never the body) is always
denied for a model-restricted group; documented in
maxModelPeekBytes' own comment, the handler's doc comment, and
the README.
Matcher parity: the round-1 version glued providerName+"/" onto
whatever the body sent unconditionally, so an already-prefixed body
("openai/x") produced "openai/openai/x" and false-403'd a legitimate
client. allowsPassthroughModel now reuses modelRegistry.
splitConfiguredProvider — the same rule resolve/resolveAgainst apply
to a client-given id — to strip a leading segment only when it names
an actually configured provider, before generating candidates.
Also in this commit:
- Restored ~26 lines of doc comments fieldalignment -fix silently
dropped when it reordered the group struct (auth.go: cache,
cacheTTL) and ProviderConfig (llmgateway.go: MetadataPath, and the
Passthrough field's own new comment). Verified nothing else was
reordered by re-running fieldalignment: 0 issues.
- Documented the path.Match footgun a non-empty PassthroughPaths
inherits: "*" does not cross "/", so ["*"] denies almost all real
multi-segment passthrough traffic rather than allowing it. Empty is
the actual allow-all value. Documented on the config field, the
allowsPassthroughPath method, and the handler's own doc comment.
- Renamed the local "model" that shadowed the outer request-model
variable near extractPassthroughUsage's call site to "respModel" —
the new gating already scopes the outer one to a narrower block, so
this was no longer a live shadow, but the rename removes any future
ambiguity between the request's model and the response's.
- README: rewrote the two "Unified vs. passthrough" table cells that
described the OPPOSITE of this feature's actual behavior, added a
"Native passthrough model authorization" section, and added
providers.<name>.passthrough / groups.<name>.passthroughPaths rows
to the config tables.
New tests: the Content-Type-bypass regression (text/plain, absent,
form-urlencoded, all now 403), the matcher-parity case (already-
prefixed body allowed), fail-closed on a binary body for a restricted
group, skip-entirely for an unrestricted group, byte-identical body
forwarding for a body larger than the peek cap and for a
ContentLength<0 (chunked) request, and a scanTopLevelModel unit table
covering truncation before/after the model field, nested values, and
malformed input.
Gate re-run in full after these changes: gofmt, go vet, fieldalignment,
golangci-lint (0 issues), go test -race (ok), make yaegi-check (OK),
make integration (ok, real Traefik + Redis). Re-ran the interpreted
Yaegi harness: RTT/req unchanged at 3.00, cmds/req unchanged at 74.00.
This commit is contained in:
@@ -221,6 +221,7 @@ also a construction error, never a panic (`providers.go`, `mcp_a2a.go`,
|
||||
| `models` | `[]string` | `[]` | Explicit model ids this provider serves. Combined with any discovered ids. |
|
||||
| `discovery` | `bool` | `false` | Pull the provider's own model-listing endpoint at startup and on `discoveryInterval`. A discovery failure is logged and non-fatal; construction still succeeds on `models` alone. |
|
||||
| `metadataPath` | `string` | `""` (no metadata capture) | Only read by `openai`-type providers. A second endpoint, fetched alongside `discovery`, that reports per-model context length — for example `/api/v0/models` on an LM Studio server. Must start with `/` when set (checked at construction). See [Model metadata](#model-metadata). |
|
||||
| `passthrough` | `*bool` | `nil` (enabled) | `nil` or `true`: this provider's native passthrough route (`/{name}/...`) stays reachable, unchanged from before this field existed. `false`: the route is disabled — `ServeHTTP` treats it exactly like an unconfigured provider name, falling through to the ordinary unknown-route 404 without even reaching authentication. Does not affect the MCP/A2A target proxy, which has its own routing prefix. |
|
||||
|
||||
### `GroupConfig`
|
||||
|
||||
@@ -233,6 +234,7 @@ also a construction error, never a panic (`providers.go`, `mcp_a2a.go`,
|
||||
| `cacheTTL` | `string` | `""` (inherit) | Sets the TTL written when this group's own request populates a cache entry — see [Caching](#caching). It does not scope entries to a group: cache entries are shared across every group that can reach the model (see the Caching section's "User identity is not part of the key" note, which applies to group identity too), so this controls a write's TTL only, not which group can read the entry. Empty inherits the global TTL. A set value must be a valid, positive Go duration (`"30s"`, `"5m"`), and requires the top-level `cache` block to be configured — a construction error otherwise, the same "nothing to inherit from" reasoning as `cache: true` above. |
|
||||
| `mcpServers` | `[]string` | `[]` (all) | Glob-matched against a configured MCP server name. |
|
||||
| `agents` | `[]string` | `[]` (all) | Glob-matched against a configured agent name. |
|
||||
| `passthroughPaths` | `[]string` | `[]` (all) | Restricts which REST path this group's **native passthrough** requests may address — glob-matched (`path.Match`) against the path segment after the provider name (e.g. `v1/files` for a request to `/openai/v1/files`). Empty is allow-all, matching every other field on this table — **not** `["*"]`: `path.Match`'s `*` does not cross `/`, so a literal `["*"]` pattern denies every multi-segment `rest` (almost all real passthrough traffic). Leave this unset to actually allow everything. See [Unified vs. passthrough](#unified-vs-passthrough) for the related model-authorization rule. |
|
||||
|
||||
### `LimitsConfig`
|
||||
|
||||
@@ -403,8 +405,11 @@ kept — a bad edit to the file never breaks already-authenticated traffic.
|
||||
`group.allowsProvider` for the resolved provider — a model that exists
|
||||
but the caller's group cannot reach returns 403, distinct from a model no
|
||||
configured provider knows at all (404). Native passthrough
|
||||
(`/{provider}/...`) resolves no model at all and checks
|
||||
`group.allowsProvider` only — see the authorization row in [Unified vs.
|
||||
(`/{provider}/...`) always checks `group.allowsProvider`, and additionally
|
||||
checks `group.allowsModel` — against a `model` field peeked from the
|
||||
request body — whenever the group's `models` list is non-empty; see
|
||||
[Native passthrough model authorization](#native-passthrough-model-authorization)
|
||||
and the authorization row in [Unified vs.
|
||||
passthrough](#unified-vs-passthrough).
|
||||
- **Discovery**: `GET {baseUrl}/v1/models` for an `openai`-type provider,
|
||||
`GET {baseUrl}/v1/models` for `anthropic`, `GET {baseUrl}/v1beta/models`
|
||||
@@ -859,13 +864,33 @@ cached bodies for the same request.
|
||||
| | Unified (`/v1/...`) | Native passthrough (`/{provider}/...`) |
|
||||
|---|---|---|
|
||||
| Wire format | Always OpenAI-shaped in, OpenAI-shaped out. | The provider's own native format, untouched. |
|
||||
| Authorization | `group.allowsModel` AND `group.allowsProvider`, both checked against the resolved provider (`modelRegistry.resolve` — see [Model routing](#model-routing)). | `group.allowsProvider` only. Passthrough resolves no model at all, so there is nothing for `allowsModel` to gate. |
|
||||
| Authorization | `group.allowsModel` AND `group.allowsProvider`, both checked against the resolved provider (`modelRegistry.resolve` — see [Model routing](#model-routing)). | `group.allowsProvider`, always. `group.allowsModel` too, but ONLY when the group's `models` list is non-empty — see the model-enforcement note below the table. `passthroughPaths`, when set, also gates the REST path. |
|
||||
| Translation | `openai`-type: body forwarded verbatim (model id rewritten). `anthropic`/`gemini`: full bidirectional translation, including streaming, chunk by chunk. | None — a raw reverse proxy. |
|
||||
| Model alias in the response | Translated providers (`anthropic`, `gemini`) echo back the client's exact requested model string in the response's `model` field, even though the upstream call used the resolved provider model id. An `openai`-type response is a verbatim passthrough of the upstream body, so it carries whatever model id the upstream itself returned — this asymmetry is intentional, not a bug. | The upstream's own `model` field, verbatim — there is no alias to echo. |
|
||||
| Embeddings | `openai`-type: passthrough. `gemini`: mapped to `:embedContent`/`:batchEmbedContents`. `anthropic`: **501** — Anthropic's API has no embeddings endpoint. | Whatever the provider itself supports at that path; the gateway does not gate it. |
|
||||
| Embeddings | `openai`-type: passthrough. `gemini`: mapped to `:embedContent`/`:batchEmbedContents`. `anthropic`: **501** — Anthropic's API has no embeddings endpoint. | Whatever the provider itself supports at that path, subject to the same `models`/`passthroughPaths` authorization as every other native passthrough request. |
|
||||
| Unsupported parameters | A semantically meaningful field the target provider cannot express (e.g. `n > 1`, `logit_bias`, `logprobs` on Anthropic) is a **400**, never silently dropped. | Not applicable — the request reaches the provider exactly as sent. |
|
||||
| Non-2xx upstream response | Wrapped in the gateway's own error envelope, with the provider's own body embedded under `error.upstream`. | Forwarded to the client exactly as the upstream sent it — status, headers, and body. |
|
||||
|
||||
### Native passthrough model authorization
|
||||
|
||||
A group's `models` glob authorizes native passthrough traffic too, but
|
||||
only once it has something to authorize:
|
||||
|
||||
- **`models` empty (the default)** — nothing for `allowsModel` to reject,
|
||||
so the request body is never even read for this purpose. Every request
|
||||
reaches the provider exactly as it did before this feature existed.
|
||||
- **`models` non-empty** — the gateway reads up to 64KiB of a JSON request
|
||||
body looking for a top-level `model` field (a genuinely binary body —
|
||||
multipart, `audio/*`, `image/*`, `video/*`, `application/octet-stream`
|
||||
— is never read at all, since it cannot carry one). The field, when
|
||||
found, is checked against `models` in both its bare and
|
||||
`provider/model` forms. **A `model` field not found within that 64KiB
|
||||
window is DENIED (403)** — fail closed, not fail open. This includes a
|
||||
**Gemini** passthrough request: Gemini's model id lives in the URL
|
||||
path, never the JSON body, so a group with a restricted `models` list
|
||||
will see every Gemini passthrough request denied. Route Gemini traffic
|
||||
for a model-restricted group through the unified `/v1/*` API instead.
|
||||
|
||||
## Image and audio endpoints
|
||||
|
||||
`POST /v1/images/generations`, `POST /v1/audio/speech`, and
|
||||
|
||||
Reference in New Issue
Block a user