mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-13 04:58:50 +00:00
fix(passthrough): close Content-Type bypass and matcher-parity bug
Independent review measured two real gaps in the round-1 model
enforcement and fixed both here, plus documentation and comment
regressions the round introduced.
MUST FIX: peekPassthroughModel's Content-Type gate was an ALLOW-list
("application/json" or skip") and fully client-controlled. The
reviewer sent "Content-Type: text/plain" with {"model":"EXPENSIVE"}
and got a 200 from a group scoped to a cheap model. This commit
replaces the whole gating approach:
1. A group with an empty Models list has nothing for allowsModel to
reject, so the request body is never read at all
(group.hasModelRestriction). This is the overwhelming majority of
traffic, including the operator's own "home" group, and removes
the memory-buffering concern entirely for it.
2. For a group WITH a restricted Models list, the Content-Type gate
is now a SKIP-list of genuinely binary types (multipart/, audio/,
image/, video/, application/octet-stream). Every other
Content-Type, including "text/plain" or none at all, is
inspected. The attacker no longer controls whether the check
runs.
3. The peek reads at most 64KiB (maxModelPeekBytes), restored via
io.MultiReader so the remainder streams straight to the upstream
instead of a second full buffer. A bounded JSON token walk
(scanTopLevelModel) finds a top-level "model" key even when the
rest of the body is truncated past the cap.
4. A restricted group whose peek window never finds "model" is
denied (403), fail closed. This also means a Gemini passthrough
request (its model lives in the URL, never the body) is always
denied for a model-restricted group; documented in
maxModelPeekBytes' own comment, the handler's doc comment, and
the README.
Matcher parity: the round-1 version glued providerName+"/" onto
whatever the body sent unconditionally, so an already-prefixed body
("openai/x") produced "openai/openai/x" and false-403'd a legitimate
client. allowsPassthroughModel now reuses modelRegistry.
splitConfiguredProvider — the same rule resolve/resolveAgainst apply
to a client-given id — to strip a leading segment only when it names
an actually configured provider, before generating candidates.
Also in this commit:
- Restored ~26 lines of doc comments fieldalignment -fix silently
dropped when it reordered the group struct (auth.go: cache,
cacheTTL) and ProviderConfig (llmgateway.go: MetadataPath, and the
Passthrough field's own new comment). Verified nothing else was
reordered by re-running fieldalignment: 0 issues.
- Documented the path.Match footgun a non-empty PassthroughPaths
inherits: "*" does not cross "/", so ["*"] denies almost all real
multi-segment passthrough traffic rather than allowing it. Empty is
the actual allow-all value. Documented on the config field, the
allowsPassthroughPath method, and the handler's own doc comment.
- Renamed the local "model" that shadowed the outer request-model
variable near extractPassthroughUsage's call site to "respModel" —
the new gating already scopes the outer one to a narrower block, so
this was no longer a live shadow, but the rename removes any future
ambiguity between the request's model and the response's.
- README: rewrote the two "Unified vs. passthrough" table cells that
described the OPPOSITE of this feature's actual behavior, added a
"Native passthrough model authorization" section, and added
providers.<name>.passthrough / groups.<name>.passthroughPaths rows
to the config tables.
New tests: the Content-Type-bypass regression (text/plain, absent,
form-urlencoded, all now 403), the matcher-parity case (already-
prefixed body allowed), fail-closed on a binary body for a restricted
group, skip-entirely for an unrestricted group, byte-identical body
forwarding for a body larger than the peek cap and for a
ContentLength<0 (chunked) request, and a scanTopLevelModel unit table
covering truncation before/after the model field, nested values, and
malformed input.
Gate re-run in full after these changes: gofmt, go vet, fieldalignment,
golangci-lint (0 issues), go test -race (ok), make yaegi-check (OK),
make integration (ok, real Traefik + Redis). Re-ran the interpreted
Yaegi harness: RTT/req unchanged at 3.00, cmds/req unchanged at 74.00.
This commit is contained in:
@@ -221,6 +221,7 @@ also a construction error, never a panic (`providers.go`, `mcp_a2a.go`,
|
||||
| `models` | `[]string` | `[]` | Explicit model ids this provider serves. Combined with any discovered ids. |
|
||||
| `discovery` | `bool` | `false` | Pull the provider's own model-listing endpoint at startup and on `discoveryInterval`. A discovery failure is logged and non-fatal; construction still succeeds on `models` alone. |
|
||||
| `metadataPath` | `string` | `""` (no metadata capture) | Only read by `openai`-type providers. A second endpoint, fetched alongside `discovery`, that reports per-model context length — for example `/api/v0/models` on an LM Studio server. Must start with `/` when set (checked at construction). See [Model metadata](#model-metadata). |
|
||||
| `passthrough` | `*bool` | `nil` (enabled) | `nil` or `true`: this provider's native passthrough route (`/{name}/...`) stays reachable, unchanged from before this field existed. `false`: the route is disabled — `ServeHTTP` treats it exactly like an unconfigured provider name, falling through to the ordinary unknown-route 404 without even reaching authentication. Does not affect the MCP/A2A target proxy, which has its own routing prefix. |
|
||||
|
||||
### `GroupConfig`
|
||||
|
||||
@@ -233,6 +234,7 @@ also a construction error, never a panic (`providers.go`, `mcp_a2a.go`,
|
||||
| `cacheTTL` | `string` | `""` (inherit) | Sets the TTL written when this group's own request populates a cache entry — see [Caching](#caching). It does not scope entries to a group: cache entries are shared across every group that can reach the model (see the Caching section's "User identity is not part of the key" note, which applies to group identity too), so this controls a write's TTL only, not which group can read the entry. Empty inherits the global TTL. A set value must be a valid, positive Go duration (`"30s"`, `"5m"`), and requires the top-level `cache` block to be configured — a construction error otherwise, the same "nothing to inherit from" reasoning as `cache: true` above. |
|
||||
| `mcpServers` | `[]string` | `[]` (all) | Glob-matched against a configured MCP server name. |
|
||||
| `agents` | `[]string` | `[]` (all) | Glob-matched against a configured agent name. |
|
||||
| `passthroughPaths` | `[]string` | `[]` (all) | Restricts which REST path this group's **native passthrough** requests may address — glob-matched (`path.Match`) against the path segment after the provider name (e.g. `v1/files` for a request to `/openai/v1/files`). Empty is allow-all, matching every other field on this table — **not** `["*"]`: `path.Match`'s `*` does not cross `/`, so a literal `["*"]` pattern denies every multi-segment `rest` (almost all real passthrough traffic). Leave this unset to actually allow everything. See [Unified vs. passthrough](#unified-vs-passthrough) for the related model-authorization rule. |
|
||||
|
||||
### `LimitsConfig`
|
||||
|
||||
@@ -403,8 +405,11 @@ kept — a bad edit to the file never breaks already-authenticated traffic.
|
||||
`group.allowsProvider` for the resolved provider — a model that exists
|
||||
but the caller's group cannot reach returns 403, distinct from a model no
|
||||
configured provider knows at all (404). Native passthrough
|
||||
(`/{provider}/...`) resolves no model at all and checks
|
||||
`group.allowsProvider` only — see the authorization row in [Unified vs.
|
||||
(`/{provider}/...`) always checks `group.allowsProvider`, and additionally
|
||||
checks `group.allowsModel` — against a `model` field peeked from the
|
||||
request body — whenever the group's `models` list is non-empty; see
|
||||
[Native passthrough model authorization](#native-passthrough-model-authorization)
|
||||
and the authorization row in [Unified vs.
|
||||
passthrough](#unified-vs-passthrough).
|
||||
- **Discovery**: `GET {baseUrl}/v1/models` for an `openai`-type provider,
|
||||
`GET {baseUrl}/v1/models` for `anthropic`, `GET {baseUrl}/v1beta/models`
|
||||
@@ -859,13 +864,33 @@ cached bodies for the same request.
|
||||
| | Unified (`/v1/...`) | Native passthrough (`/{provider}/...`) |
|
||||
|---|---|---|
|
||||
| Wire format | Always OpenAI-shaped in, OpenAI-shaped out. | The provider's own native format, untouched. |
|
||||
| Authorization | `group.allowsModel` AND `group.allowsProvider`, both checked against the resolved provider (`modelRegistry.resolve` — see [Model routing](#model-routing)). | `group.allowsProvider` only. Passthrough resolves no model at all, so there is nothing for `allowsModel` to gate. |
|
||||
| Authorization | `group.allowsModel` AND `group.allowsProvider`, both checked against the resolved provider (`modelRegistry.resolve` — see [Model routing](#model-routing)). | `group.allowsProvider`, always. `group.allowsModel` too, but ONLY when the group's `models` list is non-empty — see the model-enforcement note below the table. `passthroughPaths`, when set, also gates the REST path. |
|
||||
| Translation | `openai`-type: body forwarded verbatim (model id rewritten). `anthropic`/`gemini`: full bidirectional translation, including streaming, chunk by chunk. | None — a raw reverse proxy. |
|
||||
| Model alias in the response | Translated providers (`anthropic`, `gemini`) echo back the client's exact requested model string in the response's `model` field, even though the upstream call used the resolved provider model id. An `openai`-type response is a verbatim passthrough of the upstream body, so it carries whatever model id the upstream itself returned — this asymmetry is intentional, not a bug. | The upstream's own `model` field, verbatim — there is no alias to echo. |
|
||||
| Embeddings | `openai`-type: passthrough. `gemini`: mapped to `:embedContent`/`:batchEmbedContents`. `anthropic`: **501** — Anthropic's API has no embeddings endpoint. | Whatever the provider itself supports at that path; the gateway does not gate it. |
|
||||
| Embeddings | `openai`-type: passthrough. `gemini`: mapped to `:embedContent`/`:batchEmbedContents`. `anthropic`: **501** — Anthropic's API has no embeddings endpoint. | Whatever the provider itself supports at that path, subject to the same `models`/`passthroughPaths` authorization as every other native passthrough request. |
|
||||
| Unsupported parameters | A semantically meaningful field the target provider cannot express (e.g. `n > 1`, `logit_bias`, `logprobs` on Anthropic) is a **400**, never silently dropped. | Not applicable — the request reaches the provider exactly as sent. |
|
||||
| Non-2xx upstream response | Wrapped in the gateway's own error envelope, with the provider's own body embedded under `error.upstream`. | Forwarded to the client exactly as the upstream sent it — status, headers, and body. |
|
||||
|
||||
### Native passthrough model authorization
|
||||
|
||||
A group's `models` glob authorizes native passthrough traffic too, but
|
||||
only once it has something to authorize:
|
||||
|
||||
- **`models` empty (the default)** — nothing for `allowsModel` to reject,
|
||||
so the request body is never even read for this purpose. Every request
|
||||
reaches the provider exactly as it did before this feature existed.
|
||||
- **`models` non-empty** — the gateway reads up to 64KiB of a JSON request
|
||||
body looking for a top-level `model` field (a genuinely binary body —
|
||||
multipart, `audio/*`, `image/*`, `video/*`, `application/octet-stream`
|
||||
— is never read at all, since it cannot carry one). The field, when
|
||||
found, is checked against `models` in both its bare and
|
||||
`provider/model` forms. **A `model` field not found within that 64KiB
|
||||
window is DENIED (403)** — fail closed, not fail open. This includes a
|
||||
**Gemini** passthrough request: Gemini's model id lives in the URL
|
||||
path, never the JSON body, so a group with a restricted `models` list
|
||||
will see every Gemini passthrough request denied. Route Gemini traffic
|
||||
for a model-restricted group through the unified `/v1/*` API instead.
|
||||
|
||||
## Image and audio endpoints
|
||||
|
||||
`POST /v1/images/generations`, `POST /v1/audio/speech`, and
|
||||
|
||||
@@ -25,15 +25,31 @@ type user struct {
|
||||
// group describes a group's access rules and default limits. Empty
|
||||
// providers, models, mcpServers, or agents means all are allowed.
|
||||
type group struct {
|
||||
limits *LimitsConfig
|
||||
cache *bool
|
||||
name string
|
||||
providers []string
|
||||
models []string
|
||||
mcpServers []string
|
||||
agents []string
|
||||
limits *LimitsConfig
|
||||
// cache is GroupConfig.Cache carried through unchanged: nil inherits
|
||||
// the global cache.enabled setting, non-nil overrides it for this
|
||||
// group's requests. Resolved by groupCacheEnabled (cache.go).
|
||||
cache *bool
|
||||
name string
|
||||
providers []string
|
||||
models []string
|
||||
mcpServers []string
|
||||
agents []string
|
||||
// passthroughPaths is GroupConfig.PassthroughPaths carried through
|
||||
// unchanged (security+performance audit, 2026-08-22) — see
|
||||
// allowsPassthroughPath below.
|
||||
passthroughPaths []string
|
||||
cacheTTL time.Duration
|
||||
// cacheTTL is GroupConfig.CacheTTL parsed and validated at construction
|
||||
// (newAuthStore below): 0 inherits the global responseCache's TTL
|
||||
// (effectiveTTL, cache.go); a positive value sets the TTL written
|
||||
// when THIS group's own request populates a cache entry. It is not a
|
||||
// per-group scope — cache entries are shared across every group that
|
||||
// can reach the model (groupCacheEnabled, cache.go), so a positive
|
||||
// cacheTTL only ever controls a write's TTL, never which group can
|
||||
// later read the entry. GroupConfig.CacheTTL == "" is the only input
|
||||
// that produces 0 here — every other value either becomes a positive
|
||||
// duration or fails newAuthStore as a constructor error.
|
||||
cacheTTL time.Duration
|
||||
}
|
||||
|
||||
// allowsProvider reports whether name matches one of the group's provider
|
||||
@@ -58,6 +74,20 @@ func (grp *group) allowsModel(id string) bool {
|
||||
return matchesGlob(grp.models, id)
|
||||
}
|
||||
|
||||
// hasModelRestriction reports whether grp's Models glob is non-empty —
|
||||
// whether there is anything for allowsModel to actually reject (security
|
||||
// review fix, 2026-08-22, round 2). handlePassthrough (routes_
|
||||
// passthrough.go) checks this BEFORE peeking a passthrough request's body
|
||||
// at all: a group with an empty Models list (matchesGlob's own
|
||||
// empty-means-all contract — the live cluster's "home" group and every
|
||||
// other group that has not opted into model restrictions) has nothing
|
||||
// allowsModel could ever deny, so there is no reason to read, buffer, or
|
||||
// even look at the request body for model enforcement — zero risk, and
|
||||
// exactly today's behavior for the overwhelming majority of traffic.
|
||||
func (grp *group) hasModelRestriction() bool {
|
||||
return len(grp.models) > 0
|
||||
}
|
||||
|
||||
// allowsMCP reports whether name matches one of the group's MCP-server glob
|
||||
// patterns. An empty pattern list allows every server.
|
||||
func (grp *group) allowsMCP(name string) bool {
|
||||
@@ -73,10 +103,13 @@ func (grp *group) allowsAgent(name string) bool {
|
||||
// allowsPassthroughPath reports whether rest — the path segment after the
|
||||
// provider name in a native passthrough request (passthroughRoute's own
|
||||
// "rest", routes_passthrough.go) — matches one of the group's
|
||||
// PassthroughPaths glob patterns (security+performance audit,
|
||||
// 2026-08-22). An empty pattern list allows every path, the same
|
||||
// fail-open default every other allowsX method on group already applies
|
||||
// (matchesGlob's own empty-means-all contract) — do not invert it.
|
||||
// PassthroughPaths glob patterns (security review, 2026-08-22). An empty
|
||||
// pattern list allows every path, the same fail-open default every other
|
||||
// allowsX method on group already applies (matchesGlob's own
|
||||
// empty-means-all contract) — do not invert it. See GroupConfig.
|
||||
// PassthroughPaths' own doc comment (llmgateway.go) for a path.Match
|
||||
// footgun a non-empty list inherits: "*" does not cross "/", so ["*"]
|
||||
// is NOT "allow everything".
|
||||
func (grp *group) allowsPassthroughPath(rest string) bool {
|
||||
return matchesGlob(grp.passthroughPaths, rest)
|
||||
}
|
||||
|
||||
+61
-19
@@ -65,14 +65,44 @@ type Config struct {
|
||||
|
||||
// ProviderConfig describes one upstream LLM provider.
|
||||
type ProviderConfig struct {
|
||||
Passthrough *bool `json:"passthrough,omitempty"`
|
||||
Type string `json:"type"`
|
||||
BaseURL string `json:"baseUrl,omitempty"`
|
||||
APIKey string `json:"apiKey"`
|
||||
DiscoveryInterval string `json:"discoveryInterval,omitempty"`
|
||||
MetadataPath string `json:"metadataPath,omitempty"`
|
||||
Models []string `json:"models,omitempty"`
|
||||
Discovery bool `json:"discovery,omitempty"`
|
||||
// Passthrough gates whether this provider's native passthrough route
|
||||
// ("/{providerName}/{rest...}", routes_passthrough.go's
|
||||
// handlePassthrough) is reachable at all. nil or true (the default)
|
||||
// preserves prior behavior exactly: every configured provider stays
|
||||
// reachable through native passthrough unless an operator explicitly
|
||||
// opts it out — FAIL-OPEN BY DESIGN, the same default direction as
|
||||
// every allow-list field in this package (GroupConfig's Providers/
|
||||
// Models/MCPServers/Agents/PassthroughPaths: empty/unset permits, an
|
||||
// explicit restriction narrows). Set false to disable a provider's
|
||||
// native passthrough entirely — e.g. an operator who wants clients
|
||||
// reaching a provider only through the unified /v1/* routes, never
|
||||
// its raw native API. A disabled provider's passthrough prefix is
|
||||
// treated exactly like an unconfigured one: ServeHTTP falls through
|
||||
// to the ordinary unknown-route 404, without even reaching auth
|
||||
// (security+performance audit, 2026-08-22). This does not affect the
|
||||
// MCP/A2A target proxy (mcp_a2a.go), which has its own routing
|
||||
// prefix and its own allow-list (GroupConfig.MCPServers/Agents) —
|
||||
// operators who route native tools like macstudio-rerank/parakeet-
|
||||
// mlx/piper-tts through THIS field's route must leave it unset or
|
||||
// true to keep that traffic flowing.
|
||||
Passthrough *bool `json:"passthrough,omitempty"`
|
||||
Type string `json:"type"`
|
||||
BaseURL string `json:"baseUrl,omitempty"`
|
||||
APIKey string `json:"apiKey"`
|
||||
DiscoveryInterval string `json:"discoveryInterval,omitempty"`
|
||||
// MetadataPath is an optional second discovery endpoint (feature
|
||||
// v0.23), fetched alongside the provider's normal listModels call
|
||||
// when set: a path such as "/api/v0/models" (LM Studio's own,
|
||||
// non-OpenAI-compatible models endpoint) that reports per-model
|
||||
// context length. Only openai-type adapters read this field
|
||||
// (provider_openai.go's openaiAdapter.fetchModelMetadata) — an
|
||||
// anthropic/gemini provider configuring it has no effect. Empty (the
|
||||
// default) captures no metadata, preserving prior behavior exactly.
|
||||
// A failed or absent fetch is always non-fatal to discovery itself:
|
||||
// no metadata is captured, nothing else changes.
|
||||
MetadataPath string `json:"metadataPath,omitempty"`
|
||||
Models []string `json:"models,omitempty"`
|
||||
Discovery bool `json:"discovery,omitempty"`
|
||||
}
|
||||
|
||||
// GroupConfig describes a group's access and limits. Empty/omitted
|
||||
@@ -103,17 +133,29 @@ type GroupConfig struct {
|
||||
Agents []string `json:"agents,omitempty"`
|
||||
// PassthroughPaths restricts which REST path this group's native
|
||||
// passthrough requests may address, glob-matched (matchesGlob,
|
||||
// auth.go) against passthroughRoute's own "rest" — the path segment
|
||||
// AFTER the provider name (e.g. "v1/files" for a request to
|
||||
// "/openai/v1/files"). Empty/omitted (the default) means every path
|
||||
// is allowed, preserving prior behavior exactly and matching every
|
||||
// other allow-list field's own empty-means-all semantics in this
|
||||
// struct — DO NOT invert that convention here. Set a non-empty list
|
||||
// to restrict a group to specific native endpoints (e.g.
|
||||
// ["v1/chat/completions"]), without narrowing any group that leaves
|
||||
// it unset (security+performance audit, 2026-08-22). The "home"
|
||||
// group's own providers/models/mcpServers/agents lists are already
|
||||
// empty (unrestricted); leaving this unset too keeps it that way.
|
||||
// auth.go, via Go's path.Match) against passthroughRoute's own
|
||||
// "rest" — the ESCAPED path segment (r.URL.EscapedPath()-derived,
|
||||
// never percent-decoded) AFTER the provider name (e.g. "v1/files" for
|
||||
// a request to "/openai/v1/files"). Empty/omitted (the default) means
|
||||
// every path is allowed, preserving prior behavior exactly and
|
||||
// matching every other allow-list field's own empty-means-all
|
||||
// semantics in this struct — DO NOT invert that convention here. Set
|
||||
// a non-empty list to restrict a group to specific native endpoints
|
||||
// (e.g. ["v1/chat/completions"]), without narrowing any group that
|
||||
// leaves it unset (security review, 2026-08-22). The "home" group's
|
||||
// own providers/models/mcpServers/agents lists are already empty
|
||||
// (unrestricted); leaving this unset too keeps it that way.
|
||||
//
|
||||
// FOOTGUN (path.Match's documented behavior, verified in review): "*"
|
||||
// matches within ONE path segment only — it does NOT cross "/". A
|
||||
// pattern of exactly ["*"] therefore does NOT mean "allow every
|
||||
// path": it denies every "rest" with more than one segment, which is
|
||||
// most real passthrough traffic (e.g. "v1/chat/completions", three
|
||||
// segments, never matches "*"). Leave PassthroughPaths unset/empty to
|
||||
// actually allow everything; if you need a genuine multi-segment
|
||||
// wildcard, list every segment explicitly (e.g. "v1/*") or repeat a
|
||||
// "*/*/*"-shaped pattern per depth — there is no "**" (path.Match has
|
||||
// no such construct).
|
||||
PassthroughPaths []string `json:"passthroughPaths,omitempty"`
|
||||
}
|
||||
|
||||
|
||||
+269
-59
@@ -25,6 +25,70 @@ const maxPassthroughBytes = 32 << 20
|
||||
// of any size — without ever waiting for the full body to buffer first.
|
||||
const maxAccountingTeeBytes = 4 << 20
|
||||
|
||||
// maxModelPeekBytes caps how much of a passthrough request body
|
||||
// peekPassthroughModel reads to find a top-level "model" field: 64KiB
|
||||
// (security review fix, 2026-08-22, round 2). This is deliberately far
|
||||
// smaller than maxPassthroughBytes: a legitimate LLM request always
|
||||
// carries "model" in its first bytes, so 64KiB is generous headroom
|
||||
// ahead of it while bounding memory to a small constant regardless of how
|
||||
// large the rest of the body (a long messages/documents array) is. The
|
||||
// body is never fully buffered to find this — see peekPassthroughModel's
|
||||
// own doc comment for the io.MultiReader restoration that lets the
|
||||
// remainder stream straight to the upstream, unread by this gateway.
|
||||
//
|
||||
// OPERATOR CONTRACT: a group with a non-empty Models list (GroupConfig,
|
||||
// llmgateway.go) enforces model authorization on every native passthrough
|
||||
// request whose body is inspected (see peekPassthroughModel's own gating)
|
||||
// — and a request whose "model" field does not appear within this 64KiB
|
||||
// window is DENIED (403), fail-closed, even if it would otherwise have
|
||||
// been allowed. This includes a Gemini passthrough request: Gemini's
|
||||
// model id lives in the URL path, never the JSON body, so a group with a
|
||||
// restricted Models list and Gemini passthrough traffic will see every
|
||||
// such request denied under this rule — route Gemini traffic for a
|
||||
// model-restricted group through the unified /v1/* API instead, where
|
||||
// modelRegistry.resolve enforces the same authorization against the
|
||||
// URL-independent "model" field those routes already require.
|
||||
const maxModelPeekBytes = 64 << 10
|
||||
|
||||
// passthroughBinaryContentTypePrefixes lists Content-Type prefixes
|
||||
// peekPassthroughModel treats as genuinely binary and never reads at all
|
||||
// (security review fix, 2026-08-22, round 2): multipart uploads
|
||||
// (parakeet-mlx's transcription passthrough shape) and raw audio/image/
|
||||
// video/octet-stream payloads carry no JSON "model" field to find, and
|
||||
// reading them would only cost memory for nothing.
|
||||
//
|
||||
// This is a SKIP-list, not an ALLOW-list — the deliberate inversion of
|
||||
// this gate's first version, which skipped the read unless Content-Type
|
||||
// contained "application/json". That allow-list shape let a client bypass
|
||||
// model enforcement outright merely by omitting Content-Type or sending
|
||||
// an unexpected value (e.g. "text/plain"): the reviewer measured a
|
||||
// {"model":"EXPENSIVE"} body reaching the upstream with a 200, because
|
||||
// the Content-Type check alone decided whether the check ran, and
|
||||
// Content-Type is entirely client-controlled. With a skip-list, every
|
||||
// Content-Type NOT matching one of these prefixes — including "text/plain"
|
||||
// and an ABSENT header — is inspected; the attacker no longer controls
|
||||
// whether the check runs, only (still) what non-binary Content-Type they
|
||||
// send.
|
||||
var passthroughBinaryContentTypePrefixes = []string{
|
||||
"multipart/",
|
||||
"audio/",
|
||||
"image/",
|
||||
"video/",
|
||||
"application/octet-stream",
|
||||
}
|
||||
|
||||
// isPassthroughBinaryContentType reports whether contentType matches one
|
||||
// of passthroughBinaryContentTypePrefixes, case-insensitively.
|
||||
func isPassthroughBinaryContentType(contentType string) bool {
|
||||
ct := strings.ToLower(contentType)
|
||||
for _, prefix := range passthroughBinaryContentTypePrefixes {
|
||||
if strings.HasPrefix(ct, prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// hopByHopHeaders lists the RFC 7230 §6.1 hop-by-hop headers, stripped
|
||||
// from both the outgoing upstream request and the response copied back
|
||||
// to the client — they describe this one TCP hop, not the end-to-end
|
||||
@@ -299,38 +363,166 @@ func extractPassthroughUsage(typeName, providerName string, body []byte) (usage,
|
||||
}
|
||||
}
|
||||
|
||||
// peekPassthroughModel best-effort reads a passthrough request's JSON body
|
||||
// far enough to extract its top-level "model" field, restoring r.Body
|
||||
// afterward — a fresh reader over the exact bytes read — so proxyUpstream
|
||||
// can still forward the request to the upstream unchanged (security+
|
||||
// performance audit, 2026-08-22). hasModel is false, with no error, for
|
||||
// every shape handlePassthrough's own doc comment documents as "no
|
||||
// inspectable model, fall back to provider-only auth": a non-JSON
|
||||
// Content-Type (Gemini's URL-embedded model; a multipart audio/image
|
||||
// upload — parakeet-mlx's transcription passthrough is exactly this
|
||||
// shape, and this check skips reading its body entirely, at no cost), a
|
||||
// body that fails to decode as JSON, or JSON with no non-empty top-level
|
||||
// "model" string. err is non-nil only for an actual body-read failure
|
||||
// (client disconnect, deadline) — the caller maps that to the same 400
|
||||
// runUnified's own body-read failure uses (routes_unified.go), rather
|
||||
// than silently treating an unreadable body as "provider-only, proceed".
|
||||
// peekPassthroughModel reads at most maxModelPeekBytes of a passthrough
|
||||
// request body to find its top-level "model" field, then restores r.Body
|
||||
// to stream the ORIGINAL, complete body to the upstream unchanged — the
|
||||
// bytes already read prepended (io.MultiReader) to whatever remains
|
||||
// unread on the real r.Body, never fully buffered in memory (security
|
||||
// review fix, 2026-08-22, round 2: the first version buffered the whole
|
||||
// body, up to 32MiB, into memory before forwarding it).
|
||||
//
|
||||
// Callers only invoke this once they have already decided enforcement is
|
||||
// required (handlePassthrough checks group.hasModelRestriction() first) —
|
||||
// this function itself has no fail-open/fail-closed opinion; hasModel
|
||||
// simply reports whether a non-empty top-level "model" STRING was found
|
||||
// within the peek window. It is skipped, at no cost — no read at all —
|
||||
// for a Content-Type isPassthroughBinaryContentType recognizes as
|
||||
// genuinely binary (multipart uploads, audio/image/video, octet-stream):
|
||||
// none of those carry a JSON "model" field to find. Every OTHER
|
||||
// Content-Type, including "text/plain" or an absent header, IS read: the
|
||||
// caller decides what a "not found" result means, not this gate — see
|
||||
// maxModelPeekBytes' own doc comment for why the caller's answer is
|
||||
// fail-closed.
|
||||
//
|
||||
// The scan (scanTopLevelModel) is a bounded JSON TOKEN walk, not a full
|
||||
// json.Unmarshal: the peek window is very often a PREFIX of a larger body
|
||||
// (a long messages/documents array trailing the model field), and
|
||||
// Unmarshal requires a complete, valid document — it would reject a
|
||||
// legitimately-truncated-but-found "model" as if it were malformed JSON.
|
||||
//
|
||||
// err is non-nil only for a genuine body-read failure (client disconnect,
|
||||
// deadline) distinct from merely reaching the end of a body shorter than
|
||||
// the peek window — the caller maps err to the same 400 runUnified's own
|
||||
// body-read failure uses (routes_unified.go).
|
||||
func peekPassthroughModel(r *http.Request) (model string, hasModel bool, err error) {
|
||||
if r.Body == nil || !strings.Contains(strings.ToLower(r.Header.Get("Content-Type")), "application/json") {
|
||||
if r.Body == nil || isPassthroughBinaryContentType(r.Header.Get("Content-Type")) {
|
||||
return "", false, nil
|
||||
}
|
||||
body, readErr := io.ReadAll(io.LimitReader(r.Body, maxPassthroughBytes))
|
||||
if readErr != nil {
|
||||
|
||||
head := make([]byte, maxModelPeekBytes)
|
||||
n, readErr := io.ReadFull(r.Body, head)
|
||||
switch readErr { //nolint:errorlint // io.ReadFull returns these sentinels bare, never wrapped — see its own doc comment
|
||||
case nil:
|
||||
// Read exactly maxModelPeekBytes: the body has at least that much
|
||||
// left unread on r.Body, restored below via io.MultiReader.
|
||||
case io.EOF, io.ErrUnexpectedEOF:
|
||||
// Body was shorter than the peek window (io.EOF: empty; io.
|
||||
// ErrUnexpectedEOF: 0 < n < len(head)) — head[:n] IS the whole
|
||||
// body; r.Body is now exhausted, so appending it below is a
|
||||
// harmless immediate EOF.
|
||||
default:
|
||||
return "", false, readErr
|
||||
}
|
||||
r.Body = io.NopCloser(bytes.NewReader(body))
|
||||
head = head[:n]
|
||||
r.Body = io.NopCloser(io.MultiReader(bytes.NewReader(head), r.Body))
|
||||
|
||||
var payload struct {
|
||||
Model string `json:"model"`
|
||||
model, hasModel = scanTopLevelModel(head)
|
||||
return model, hasModel, nil
|
||||
}
|
||||
|
||||
// scanTopLevelModel walks head — a bounded, possibly-truncated PREFIX of
|
||||
// a JSON request body (peekPassthroughModel's own maxModelPeekBytes cap)
|
||||
// — as a stream of JSON tokens (encoding/json's Decoder.Token, the
|
||||
// documented mechanism for parsing a document incrementally) and returns
|
||||
// the string value of its top-level "model" key, without ever requiring
|
||||
// head to be a complete, valid JSON document on its own: a legitimate
|
||||
// request's "model" field sits well within the cap even when a trailing
|
||||
// field (a long messages/documents array) does not, and this walk finds
|
||||
// it regardless of what truncation or garbage follows. found is false —
|
||||
// not an error — for anything else: head is not a JSON object at all,
|
||||
// "model" never appears among its top-level keys before head runs out,
|
||||
// or "model" is present but its value is not a string.
|
||||
func scanTopLevelModel(head []byte) (model string, found bool) {
|
||||
dec := json.NewDecoder(bytes.NewReader(head))
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
if json.Unmarshal(body, &payload) != nil || payload.Model == "" {
|
||||
return "", false, nil
|
||||
if d, ok := tok.(json.Delim); !ok || d != '{' {
|
||||
return "", false
|
||||
}
|
||||
return payload.Model, true, nil
|
||||
for dec.More() {
|
||||
keyTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
key, ok := keyTok.(string)
|
||||
if !ok {
|
||||
return "", false // malformed: expected an object key
|
||||
}
|
||||
valTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
if key == "model" {
|
||||
s, ok := valTok.(string)
|
||||
return s, ok && s != ""
|
||||
}
|
||||
if d, ok := valTok.(json.Delim); ok && (d == '{' || d == '[') {
|
||||
if err := skipJSONValue(dec); err != nil {
|
||||
return "", false
|
||||
}
|
||||
}
|
||||
// Otherwise valTok was a scalar (string/float64/bool/nil) other
|
||||
// than "model" — nothing further to do; loop to the next key.
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// skipJSONValue consumes the remainder of a nested JSON array/object
|
||||
// value whose opening delimiter dec has already emitted, so
|
||||
// scanTopLevelModel's own dec.More() call — which reports position in
|
||||
// whatever object/array dec is CURRENTLY inside — correctly reports the
|
||||
// outer top-level object's state again once this returns, rather than
|
||||
// still appearing "inside" the value just skipped.
|
||||
func skipJSONValue(dec *json.Decoder) error {
|
||||
depth := 1
|
||||
for depth > 0 {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d, ok := tok.(json.Delim); ok {
|
||||
switch d {
|
||||
case '{', '[':
|
||||
depth++
|
||||
case '}', ']':
|
||||
depth--
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// allowsPassthroughModel reports whether grp's Models glob authorizes
|
||||
// model for a native passthrough request already pinned to providerName
|
||||
// by the URL (security review, 2026-08-22, round 2 — fixes a matcher-
|
||||
// parity bug: gluing providerName+"/" onto model UNCONDITIONALLY, the
|
||||
// round-1 version's approach, produces "openai/openai/x" and false-403s a
|
||||
// legitimate client whose body already carries the prefixed form).
|
||||
//
|
||||
// This reuses modelRegistry.splitConfiguredProvider — the SAME rule
|
||||
// resolve/resolveAgainst apply to a client-given model id (registry.go):
|
||||
// a leading segment is stripped as a provider prefix ONLY when it names
|
||||
// an ACTUALLY CONFIGURED provider, never blindly assumed. Three
|
||||
// candidates are checked, covering this package's two established
|
||||
// conventions for turning one model reference into every glob form an
|
||||
// operator might reasonably write:
|
||||
// - model, exactly as the client's body sent it (resolveAgainst's own
|
||||
// "requestedID" candidate)
|
||||
// - bareModel, model's own suffix once any genuinely-configured
|
||||
// provider prefix is stripped — equal to model when it carried none
|
||||
// (resolveAgainst's own "upstreamModel" candidate)
|
||||
// - providerName+"/"+bareModel, the canonical id for the provider THIS
|
||||
// route is actually pinned to (listFor's own p+"/"+id synthesis) —
|
||||
// lets an operator write their glob against the addressed provider
|
||||
// even when the client sends a bare model id
|
||||
func (g *Gateway) allowsPassthroughModel(grp *group, providerName, model string) bool {
|
||||
bareModel := model
|
||||
if _, rest, ok := g.registry.splitConfiguredProvider(model); ok {
|
||||
bareModel = rest
|
||||
}
|
||||
return grp.allowsModel(model) || grp.allowsModel(bareModel) || grp.allowsModel(providerName+"/"+bareModel)
|
||||
}
|
||||
|
||||
// handlePassthrough implements the native provider passthrough route:
|
||||
@@ -343,59 +535,72 @@ func peekPassthroughModel(r *http.Request) (model string, hasModel bool, err err
|
||||
// providerHTTPError.
|
||||
//
|
||||
// Checks run in this order: group authorization — provider (403), model
|
||||
// when the body carries one (403), path allowlist (403) — before
|
||||
// capability checks (Upgrade→501, path validity→400) before rate limits
|
||||
// (429/503) — a caller who cannot use providerName/model/path at all
|
||||
// learns that first, rather than learning something about how they tried
|
||||
// to use it. The provider-level Passthrough toggle (ProviderConfig,
|
||||
// when the group restricts models at all (403), path allowlist (403) —
|
||||
// before capability checks (Upgrade→501, path validity→400) before rate
|
||||
// limits (429/503) — a caller who cannot use providerName/model/path at
|
||||
// all learns that first, rather than learning something about how they
|
||||
// tried to use it. The provider-level Passthrough toggle (ProviderConfig,
|
||||
// llmgateway.go) is checked earlier still, by ServeHTTP's own route gate
|
||||
// — a disabled provider never reaches this function at all, reported as
|
||||
// the ordinary unknown-route 404 instead.
|
||||
//
|
||||
// MODEL ENFORCEMENT (security+performance audit, 2026-08-22): when the
|
||||
// request body is JSON and carries a non-empty top-level "model" field
|
||||
// (peekPassthroughModel), it is checked against grp.allowsModel — both
|
||||
// the bare form and the "providerName/model" form, the identical
|
||||
// dual-candidate matcher the unified route's own resolveAgainst applies
|
||||
// (registry.go) — before proxying, closing the bypass where a tenant
|
||||
// scoped to one cheap model could otherwise reach the SAME provider's
|
||||
// entire native API (fine-tuning, files, batches, ...) on the operator's
|
||||
// key merely by asking natively instead of through /v1/chat/completions.
|
||||
// A body with no inspectable model (Gemini's URL-embedded model id, a
|
||||
// multipart upload, a non-JSON body) falls back to provider-only
|
||||
// authorization, unchanged from before this round — this is a strict
|
||||
// narrowing of what a passthrough request may address, never a new way
|
||||
// to allow one a plain grp.allowsProvider check would have refused.
|
||||
// MODEL ENFORCEMENT (security review, 2026-08-22, round 2 — closes a
|
||||
// round-1 bypass): enforcement runs ONLY for a group with a non-empty
|
||||
// Models list (grp.hasModelRestriction) — a group that has not opted into
|
||||
// model restriction has nothing allowsModel could reject, so the request
|
||||
// body is never even read, exactly as before this feature existed at all.
|
||||
// For a restricted group, peekPassthroughModel reads (bounded,
|
||||
// maxModelPeekBytes) for a top-level "model" field and this function
|
||||
// checks it via allowsPassthroughModel — the round-1 version instead
|
||||
// gated the READ ITSELF on Content-Type containing "application/json",
|
||||
// which a client fully controls: sending "text/plain", an unexpected
|
||||
// value, or no Content-Type at all skipped the check outright and let
|
||||
// {"model":"EXPENSIVE"} straight through. There is no such escape now: a
|
||||
// restricted group's every non-genuinely-binary request body is
|
||||
// inspected (isPassthroughBinaryContentType's skip-list, not an
|
||||
// allow-list), and one where "model" cannot be found within the peek
|
||||
// window is DENIED (403) — fail CLOSED, not open, the opposite direction
|
||||
// from a body with no inspectable model under round 1. This is
|
||||
// deliberate: it only affects a group that already opted into model
|
||||
// restriction, and a legitimate LLM request always carries "model" well
|
||||
// within maxModelPeekBytes. See maxModelPeekBytes' own doc comment for
|
||||
// the Gemini-passthrough interaction this creates (its model id lives in
|
||||
// the URL, never the body) and the operator-facing contract.
|
||||
//
|
||||
// PATH ALLOWLIST (same audit): GroupConfig.PassthroughPaths, when
|
||||
// PATH ALLOWLIST (same review): GroupConfig.PassthroughPaths, when
|
||||
// non-empty, additionally restricts which rest path this group's
|
||||
// passthrough requests may address (grp.allowsPassthroughPath). Empty
|
||||
// (the default, matchesGlob's own empty-means-all contract) allows every
|
||||
// path, exactly as before this field existed.
|
||||
// path, exactly as before this field existed — see its own doc comment
|
||||
// for a path.Match footgun this allow-list inherits (a literal "*"
|
||||
// pattern does NOT mean "allow everything").
|
||||
//
|
||||
// For a Gemini provider, rest is appended to base() exactly as the client
|
||||
// sent it: there is no model extraction or URL rewriting here, so a
|
||||
// Gemini passthrough client must address it with Gemini's own native URL
|
||||
// structure, including its "/v1beta/models/{model}:generateContent"
|
||||
// paths — injectAuth still sets the same x-goog-api-key header it sets
|
||||
// for every other Gemini request. Gemini's model id lives in that URL
|
||||
// path, not the JSON body, so peekPassthroughModel never finds one for a
|
||||
// Gemini request — a Gemini passthrough client is always authorized
|
||||
// provider-only, exactly as it was before model enforcement existed.
|
||||
// for every other Gemini request.
|
||||
func (g *Gateway) handlePassthrough(w http.ResponseWriter, r *http.Request, u *user, grp *group, providerName, rest string) {
|
||||
if !grp.allowsProvider(providerName) {
|
||||
writeOAIError(w, http.StatusForbidden, "invalid_request_error", "provider access denied")
|
||||
return
|
||||
}
|
||||
|
||||
model, hasModel, err := peekPassthroughModel(r)
|
||||
if err != nil {
|
||||
writeOAIError(w, http.StatusBadRequest, "invalid_request_error", "cannot read request body")
|
||||
return
|
||||
}
|
||||
if hasModel && !grp.allowsModel(model) && !grp.allowsModel(providerName+"/"+model) {
|
||||
writeOAIError(w, http.StatusForbidden, "invalid_request_error", "model access denied")
|
||||
return
|
||||
if grp.hasModelRestriction() {
|
||||
model, hasModel, err := peekPassthroughModel(r)
|
||||
if err != nil {
|
||||
writeOAIError(w, http.StatusBadRequest, "invalid_request_error", "cannot read request body")
|
||||
return
|
||||
}
|
||||
if !hasModel {
|
||||
writeOAIError(w, http.StatusForbidden, "invalid_request_error", "model could not be determined")
|
||||
return
|
||||
}
|
||||
if !g.allowsPassthroughModel(grp, providerName, model) {
|
||||
writeOAIError(w, http.StatusForbidden, "invalid_request_error", "model access denied")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
if !grp.allowsPassthroughPath(rest) {
|
||||
@@ -450,11 +655,16 @@ func (g *Gateway) handlePassthrough(w http.ResponseWriter, r *http.Request, u *u
|
||||
return
|
||||
}
|
||||
|
||||
respUsage, model, unmarshalErr := extractPassthroughUsage(adapter.typeName(), providerName, result.tee.buf.Bytes())
|
||||
// respModel: the RESPONSE body's own reported model id (extractPassthroughUsage
|
||||
// reads it, when present, from the upstream's reply) — never named
|
||||
// "model" here, so it can never be confused with (or shadow) the
|
||||
// REQUEST body's "model" field the enforcement block above already
|
||||
// consumed via peekPassthroughModel (review fix, 2026-08-22, round 2).
|
||||
respUsage, respModel, unmarshalErr := extractPassthroughUsage(adapter.typeName(), providerName, result.tee.buf.Bytes())
|
||||
if unmarshalErr != nil {
|
||||
g.logf("passthrough: response body did not decode as JSON for usage accounting (provider %q): %v", providerName, unmarshalErr)
|
||||
}
|
||||
cost := unifiedCostMicros(providerName+"/"+model, model, respUsage, g.cfg.Pricing)
|
||||
cost := unifiedCostMicros(providerName+"/"+respModel, respModel, respUsage, g.cfg.Pricing)
|
||||
g.limiter.account(scopes, respUsage, cost)
|
||||
}
|
||||
|
||||
|
||||
+316
-13
@@ -103,6 +103,94 @@ func TestExtractPassthroughUsage(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// --- unit tests: scanTopLevelModel (review fix, 2026-08-22, round 2) ---
|
||||
|
||||
func TestScanTopLevelModel(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
head string
|
||||
wantModel string
|
||||
wantFound bool
|
||||
}{
|
||||
{
|
||||
name: "model is the first key",
|
||||
head: `{"model":"gpt-4","temperature":0.7}`,
|
||||
wantModel: "gpt-4",
|
||||
wantFound: true,
|
||||
},
|
||||
{
|
||||
name: "model after a nested object",
|
||||
head: `{"metadata":{"a":1,"b":[1,2,3]},"model":"gpt-4"}`,
|
||||
wantModel: "gpt-4",
|
||||
wantFound: true,
|
||||
},
|
||||
{
|
||||
name: "model after a nested array of objects",
|
||||
head: `{"messages":[{"role":"user","content":"hi"},{"role":"assistant","content":"there"}],"model":"gpt-4"}`,
|
||||
wantModel: "gpt-4",
|
||||
wantFound: true,
|
||||
},
|
||||
{
|
||||
name: "model found even though the document is truncated AFTER it",
|
||||
head: `{"model":"gpt-4","messages":[{"role":"user","content":"this array is never closed`,
|
||||
wantModel: "gpt-4",
|
||||
wantFound: true,
|
||||
},
|
||||
{
|
||||
name: "truncated BEFORE model appears: not found",
|
||||
head: `{"messages":[{"role":"user","content":"a long message that eats the whole peek window and mo`,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "empty object: not found",
|
||||
head: `{}`,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "not a JSON object at the top level: not found",
|
||||
head: `["model","gpt-4"]`,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "model value is not a string: not found",
|
||||
head: `{"model":4}`,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "model value is an empty string: not found",
|
||||
head: `{"model":""}`,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "empty head: not found",
|
||||
head: ``,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "malformed JSON: not found",
|
||||
head: `not json at all`,
|
||||
wantFound: false,
|
||||
},
|
||||
{
|
||||
name: "deeply nested value ahead of model, correctly skipped",
|
||||
head: `{"a":{"b":{"c":[1,[2,3],{"d":4}]}},"model":"gpt-4"}`,
|
||||
wantModel: "gpt-4",
|
||||
wantFound: true,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
gotModel, gotFound := scanTopLevelModel([]byte(tt.head))
|
||||
if gotFound != tt.wantFound {
|
||||
t.Errorf("found = %v, want %v", gotFound, tt.wantFound)
|
||||
}
|
||||
if gotFound && gotModel != tt.wantModel {
|
||||
t.Errorf("model = %q, want %q", gotModel, tt.wantModel)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// --- unit tests: hasTraversalSegment ---
|
||||
|
||||
func TestHasTraversalSegment(t *testing.T) {
|
||||
@@ -1107,14 +1195,56 @@ func TestHandlePassthrough_ModelAllowed_ProviderPrefixedForm(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_NoInspectableModel_FallsBackToProviderOnly covers
|
||||
// every "no inspectable model" shape handlePassthrough's own doc comment
|
||||
// documents: a non-JSON Content-Type (a multipart upload — parakeet-mlx's
|
||||
// transcription passthrough shape). Even though the group's Models glob
|
||||
// would deny the literal string used here if it were checked, the request
|
||||
// must still succeed — falling back to provider-only authorization,
|
||||
// unchanged from before model enforcement existed.
|
||||
func TestHandlePassthrough_NoInspectableModel_FallsBackToProviderOnly(t *testing.T) {
|
||||
// TestHandlePassthrough_UnrestrictedGroup_SkipsModelPeekEntirely proves
|
||||
// ruling item 1 (review fix, 2026-08-22, round 2): a group with an EMPTY
|
||||
// Models list — the live cluster's "home" group and every group that has
|
||||
// not opted into model restriction — never even reads the request body
|
||||
// for model enforcement, regardless of Content-Type or body shape. This
|
||||
// is the correct "no inspectable model, and nothing to enforce anyway"
|
||||
// case; TestHandlePassthrough_RestrictedGroup_BinaryContentType_Returns403
|
||||
// below covers the opposite: the SAME binary body against a group that
|
||||
// DOES restrict models.
|
||||
func TestHandlePassthrough_UnrestrictedGroup_SkipsModelPeekEntirely(t *testing.T) {
|
||||
var upstreamCalled bool
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
upstreamCalled = true
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {}} // no Models restriction
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/openai/v1/audio/transcriptions", strings.NewReader("--boundary\r\nfake multipart body\r\n--boundary--"))
|
||||
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||
req.Header.Set("Content-Type", "multipart/form-data; boundary=boundary")
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200 (no model restriction, no peek at all), body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if !upstreamCalled {
|
||||
t.Error("upstream must be called: an unrestricted group's request body is never read for model enforcement")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_RestrictedGroup_BinaryContentType_Returns403 is
|
||||
// ruling item 4's fail-closed case for a genuinely binary body: a
|
||||
// RESTRICTED group's multipart/audio/image/video upload carries no
|
||||
// inspectable "model" field by nature, and peekPassthroughModel does not
|
||||
// even read it (isPassthroughBinaryContentType's skip-list) — but because
|
||||
// the group opted into model restriction, "not found" still denies,
|
||||
// rather than silently falling back to provider-only the way an
|
||||
// unrestricted group would.
|
||||
func TestHandlePassthrough_RestrictedGroup_BinaryContentType_Returns403(t *testing.T) {
|
||||
var upstreamCalled bool
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
upstreamCalled = true
|
||||
@@ -1138,18 +1268,105 @@ func TestHandlePassthrough_NoInspectableModel_FallsBackToProviderOnly(t *testing
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("status = %d, want 403 (restricted group, no model determinable), body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if upstreamCalled {
|
||||
t.Error("upstream must never be called: a restricted group with no determinable model must fail closed")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_ContentTypeBypass_Returns403 is the MUST-FIX
|
||||
// regression test: the round-1 gate skipped the body read (and so the
|
||||
// whole model check) for ANY Content-Type not containing
|
||||
// "application/json" — entirely client-controlled. The reviewer measured
|
||||
// {"model":"EXPENSIVE"} reaching the upstream with a 200 via
|
||||
// "Content-Type: text/plain". This table drives the identical body
|
||||
// through text/plain, an ABSENT Content-Type, and
|
||||
// application/x-www-form-urlencoded — none of them binary, so all three
|
||||
// must now be inspected and denied.
|
||||
func TestHandlePassthrough_ContentTypeBypass_Returns403(t *testing.T) {
|
||||
contentTypes := []string{"text/plain", "", "application/x-www-form-urlencoded"}
|
||||
for _, ct := range contentTypes {
|
||||
t.Run("Content-Type="+ct, func(t *testing.T) {
|
||||
var upstreamCalled bool
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
upstreamCalled = true
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {Models: []string{"gpt-cheap"}}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/openai/v1/fine-tuning/jobs", strings.NewReader(`{"model":"EXPENSIVE"}`))
|
||||
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||
if ct != "" {
|
||||
req.Header.Set("Content-Type", ct)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("status = %d, want 403 — the model check must not be skippable via Content-Type, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if upstreamCalled {
|
||||
t.Error("upstream must never be called: EXPENSIVE is not in the group's Models glob")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_MatcherParity_AlreadyPrefixedBody_Allowed is the
|
||||
// matcher-parity regression test (review fix, 2026-08-22, round 2): a
|
||||
// body whose "model" field already carries the "provider/model" form
|
||||
// (some client tooling always sends canonical ids) must be authorized
|
||||
// against a BARE group glob, not false-403'd by a naive
|
||||
// providerName+"/"+model concatenation producing "openai/openai/x".
|
||||
func TestHandlePassthrough_MatcherParity_AlreadyPrefixedBody_Allowed(t *testing.T) {
|
||||
var upstreamCalled bool
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
upstreamCalled = true
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {Models: []string{"x"}}} // bare glob
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/openai/v1/native-endpoint", strings.NewReader(`{"model":"openai/x"}`))
|
||||
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200 (falls back to provider-only auth), body=%s", rec.Code, rec.Body.String())
|
||||
t.Fatalf("status = %d, want 200 — the already-prefixed body must resolve against the bare glob, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if !upstreamCalled {
|
||||
t.Error("upstream must be called: a non-JSON body has no inspectable model, so model enforcement must not block it")
|
||||
t.Error("upstream must be called: splitConfiguredProvider must strip the client's own \"openai/\" prefix before matching, not double it")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_ModelBodyReadFailure_Returns400 covers
|
||||
// peekPassthroughModel's own body-read-failure path, distinct from a body
|
||||
// that merely fails to decode as JSON (which falls back to provider-only,
|
||||
// not an error).
|
||||
// with no determinable model (which fails closed with 403, not 400). The
|
||||
// group must have a Models restriction — otherwise the peek never runs
|
||||
// at all (ruling item 1) and this failure is never reached.
|
||||
func TestHandlePassthrough_ModelBodyReadFailure_Returns400(t *testing.T) {
|
||||
var upstreamCalled bool
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -1160,7 +1377,7 @@ func TestHandlePassthrough_ModelBodyReadFailure_Returns400(t *testing.T) {
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {Models: []string{"gpt-allowed"}}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
@@ -1183,6 +1400,92 @@ func TestHandlePassthrough_ModelBodyReadFailure_Returns400(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_BodyIntegrity_ByteIdentical_LargeBody proves
|
||||
// peekPassthroughModel's io.MultiReader restoration (review fix,
|
||||
// 2026-08-22, round 2) forwards the EXACT original body to the upstream,
|
||||
// even when the body is larger than maxModelPeekBytes (64KiB) — so the
|
||||
// remainder streams from the real, unread r.Body rather than a second
|
||||
// buffered copy. Exercises the "read exactly the cap, more remains" path
|
||||
// io.ReadFull's nil-error branch takes.
|
||||
func TestHandlePassthrough_BodyIntegrity_ByteIdentical_LargeBody(t *testing.T) {
|
||||
padding := strings.Repeat("x", maxModelPeekBytes*2) // forces the body well past the 64KiB peek cap
|
||||
body := `{"model":"gpt-allowed","padding":"` + padding + `"}`
|
||||
|
||||
var gotBody []byte
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {Models: []string{"gpt-allowed"}}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/openai/v1/native-endpoint", strings.NewReader(body))
|
||||
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if string(gotBody) != body {
|
||||
t.Errorf("upstream body length = %d, want %d bytes byte-identical to the original", len(gotBody), len(body))
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_BodyIntegrity_ChunkedContentLength proves the
|
||||
// same byte-identical forwarding when the request arrives with
|
||||
// ContentLength < 0 (chunked transfer / unknown length — httptest's own
|
||||
// stand-in for what a real chunked client connection produces), the
|
||||
// variant the reviewer specifically asked for alongside the large-body
|
||||
// case above.
|
||||
func TestHandlePassthrough_BodyIntegrity_ChunkedContentLength(t *testing.T) {
|
||||
body := `{"model":"gpt-allowed","messages":[{"role":"user","content":"hi"}]}`
|
||||
|
||||
var gotBody []byte
|
||||
var gotContentLength int64
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotContentLength = r.ContentLength
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"}}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {Models: []string{"gpt-allowed"}}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/openai/v1/native-endpoint", strings.NewReader(body))
|
||||
req.ContentLength = -1 // simulate chunked transfer / unknown length
|
||||
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if string(gotBody) != body {
|
||||
t.Errorf("upstream body = %q, want %q (byte-identical)", gotBody, body)
|
||||
}
|
||||
_ = gotContentLength // upstream's own negotiated length; not asserted, proxyUpstream already re-derives it from the reader
|
||||
}
|
||||
|
||||
// TestHandlePassthrough_ProviderPassthroughDisabled_Returns404 is the
|
||||
// per-provider toggle's off case: ProviderConfig.Passthrough=false makes
|
||||
// the native passthrough route behave exactly like an unconfigured
|
||||
|
||||
Reference in New Issue
Block a user