fix(routes): address review findings on the Anthropic Messages route

Preserve usage when response translation fails after a successful,
billed upstream call. The route previously discarded already-billed
tokens on that path. A client could induce unlimited free usage
against any budget via a truncated tool-call response.

Reject a streaming request with 400 instead of 501. Check it before
the rate limiter runs. A 501 made anthropic-sdk-python retry the
request, since it retries any status of 500 or higher. Each retry
also counted against the caller's rate limit before this fix.

Preserve block order when translating an Anthropic message to
OpenAI's shape. A tool_result block before other content must stay
before it. OpenAI otherwise rejects the translated conversation.

Echo the client's requested model alias in both the passthrough and
the translated response paths. The passthrough branch previously
returned the upstream's own model id instead.

Forward the anthropic-beta header to an anthropic-type upstream
through an explicit allowlist. Never forward Authorization,
x-api-key, Cookie, or hop-by-hop headers. Let the adapter's own
credential injection win regardless.

Bill Anthropic's prompt-cache tokens. anthropicUsagePayload now reads
cache_creation_input_tokens and cache_read_input_tokens too. It folds
all three into the billed prompt count. This applies to this route,
the existing outbound Anthropic path, and the native passthrough
route.

Extract the metered-request pipeline shared by the unified route and
this route into runMeteredCall. Thread an envelope writer through it
so each route answers in its own error shape. This removes the
duplication the original brief warned against. It also lets the
rate-limit and body-admission error paths answer in this route's own
shape.

Treat a non-boolean "stream" value as a streaming request. A string
or number value previously bypassed the streaming check silently.

Emit max_completion_tokens instead of max_tokens for OpenAI's
reasoning models. Those models reject max_tokens with a 400.

Preserve tool_result.is_error by prefixing string content with
"Error: ". OpenAI's tool message carries no error field of its own.
Log a warning when the thinking field is dropped instead of
discarding it silently.

Classify a response-translation failure distinctly in the log. An
operator must never see a genuine translation bug reported as an
upstream connection error.

Document the route in README.md. Cover both auth headers, the
non-streaming limitation, and which providers translate versus pass
through. Add the reviewer's yaegi-check probes to
tools/yaegi-check/main.go, adjusted for the fixes above.
This commit is contained in:
2026-08-22 22:48:50 +01:00
parent d47dd06a46
commit 25e73cb585
9 changed files with 1317 additions and 391 deletions
+90 -4
View File
@@ -28,6 +28,10 @@ combination.
and from Anthropic's and Google Gemini's own wire formats, so any
OpenAI-SDK-compatible client can talk to all three providers through one
API shape.
- An Anthropic-compatible surface (`POST /v1/messages`) for Anthropic SDK
clients — including Claude Code — to talk to any configured provider in
Anthropic's own Messages API shape; see [Anthropic Messages API
(`/v1/messages`)](#anthropic-messages-api-v1messages).
- Native passthrough per provider (`/{provider}/...`) for callers that want
a provider's own wire format untouched, with the gateway injecting that
provider's own API key and still enforcing auth and limits.
@@ -929,6 +933,80 @@ only once it has something to authorize:
will see every Gemini passthrough request denied. Route Gemini traffic
for a model-restricted group through the unified `/v1/*` API instead.
## Anthropic Messages API (`/v1/messages`)
`POST /v1/messages` serves Anthropic SDK clients — including Claude
Code — directly, in Anthropic's own Messages API request/response shape:
the inbound counterpart to the outbound-only Anthropic support the
unified routes already had. From model resolution through response
caching, usage accounting, and adapter-error handling, this route shares
one implementation with `/v1/chat/completions`
(`runMeteredCall`, `routes_unified.go`) — the same group authorization
(see [`GroupConfig`](#groupconfig)), [limits and
accounting](#limits-and-accounting), and [caching](#caching) semantics
apply. Only the wire shape and the client-facing error envelope are
specific to this route.
### Authentication
Anthropic SDKs authenticate with the `x-api-key` header, not
`Authorization: Bearer`. This route accepts **both** — the identical
credential lookup (`authStore.identify`, `auth.go`) every other route
uses already reads either header, `Authorization: Bearer` taking
priority if a request somehow carries both — so no separate credential
path exists for this route. A failed auth answers in Anthropic's own
error shape (`{"type":"error","error":{"type":"authentication_error",
"message":"..."}}`), never the OpenAI-shaped
`{"error":{"type":...,"code":...}}` every other route uses.
### Translation vs. passthrough
Behavior depends entirely on the resolved provider's configured `type`:
| Resolved provider `type` | Behavior |
|---|---|
| `anthropic` | **Passthrough.** The request reaches that provider's own `/v1/messages` endpoint untranslated — no message-body translation of any kind. The response reaches the client unchanged, field for field, with one exception: `"model"` is rewritten to the client's own requested alias, the same routing rewrite every route applies before any adapter call — not a translation of the response body. |
| `openai` or `gemini` | **Full bidirectional translation**, through the same `providerAdapter.chatCompletion` a `/v1/chat/completions` request against that provider already uses — its contract is OpenAI-shape in, OpenAI-shape out regardless of provider type. The Anthropic-shaped request is translated to OpenAI's shape before the call; that provider's OpenAI-shaped response is translated back to Anthropic's shape before it reaches the client. |
A few translation specifics worth knowing:
- **`anthropic-beta` is forwarded** to an `anthropic`-type upstream (an
explicit allowlist — `Authorization`, `x-api-key`, `Cookie`, and every
hop-by-hop header are never forwarded, and the gateway's own credential
injection always overrides whatever the allowlist copied), so Claude
Code and the SDKs can still opt into 1M context, token-efficient
tools, computer use, and other beta features through this route.
- **Prompt-cache tokens are billed.** Anthropic's
`cache_creation_input_tokens` and `cache_read_input_tokens` fold into
the same combined prompt-token count every other route's budgets
track, alongside `input_tokens` — not dropped, and not priced at a
separate cache-tier rate.
- **`tool_result.is_error`** has no native OpenAI tool-message field; a
string `content` gets an `"Error: "` prefix so an OpenAI-backed
agentic loop still sees the failure signal.
- **Reasoning models** (OpenAI's `o1`/`o3`/`o4`/`gpt-5` families) get
`max_completion_tokens` instead of `max_tokens` — the field those
models reject with a 400.
- **`thinking`** (Anthropic's extended-thinking config) has no OpenAI
equivalent and is dropped when translating to an `openai`/`gemini`
target; the gateway logs a warning naming the dropped field rather
than discarding it silently.
### Streaming is not yet supported
`"stream"` set to anything other than the JSON literal `false` (or left
unset) gets a clean **400** `invalid_request_error` — never silently
ignored, never answered non-streamed. This is deliberately a 400, not a
501: Anthropic's own SDKs retry any status ≥ 500, so a 501 here would
turn every streaming call — the default for Claude Code — into a
multi-request retry storm against the caller's own request-rate limit.
The rejection happens *before* the rate limiter runs, so a
streaming-rejected request never consumes budget either. A follow-up
release adds the Anthropic SSE event translator
(`message_start`/`content_block_delta`/`message_stop`) — a genuinely
different event model from OpenAI's own stream shape this plugin already
forwards on `/v1/chat/completions`.
## Image and audio endpoints
`POST /v1/images/generations`, `POST /v1/audio/speech`, and
@@ -1346,10 +1424,18 @@ the plugin's only non-standard-library runtime dependency.
- **A request rejected before it reaches a provider never consumes
request-rate quota.** On the unified routes, a malformed body, a missing
`model` field, or an unknown/denied model (404/403) is rejected before
`limiter.checkAndCount` runs. On native passthrough and the MCP/A2A
proxy, denied provider/target access (403), an unsupported `Upgrade`
request (501), and a path-traversal attempt (400) are all checked before
the limiter too. Once a request *does* reach a provider adapter,
`limiter.checkAndCount` runs. `/v1/messages` goes further for one
specific field: it decodes the body and rejects a streaming request
(400) *before* the rate limiter runs too, the opposite relative order
from the unified routes' own admission-before-decode default — a
deliberate, route-scoped exception (see [Anthropic Messages
API](#anthropic-messages-api-v1messages)), since Anthropic SDKs retry
any 5xx and would otherwise burn multiple requests of budget per
logical streaming call against a route that does not support streaming
yet. On native passthrough and the MCP/A2A proxy, denied provider/target
access (403), an unsupported `Upgrade` request (501), and a
path-traversal attempt (400) are all checked before the limiter too.
Once a request *does* reach a provider adapter,
though, a translate-time 400 (an unsupported parameter combination) or
Anthropic's embeddings 501 still count toward the caller's request-rate
limit — the limiter already ran by then — though neither ever bills a