mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-20 05:56:56 +00:00
fix(routes): preserve response bytes and close remaining test gaps
Replace the alias-echo rewrite on the Anthropic passthrough branch with a byte-preserving field replacement. The prior version decoded the response into a Go map and re-marshaled it. This silently corrupted a JSON integer beyond float64's exact range. It also rewrote 1.0 to 1, HTML-escaped angle brackets and ampersands, and re-sorted every key. A tool_use block's "input" is arbitrary model-generated JSON. A long numeric id inside tool arguments could be silently corrupted on a route documented as passthrough. rewriteModelField now walks the response with a JSON decoder and copies every field but "model" as raw bytes. Add a test with a 20-digit integer, a 1.0 float, and a literal "<" character inside tool_use input. It asserts the response is byte-identical to the source apart from the model field. Add a negative test for the anthropic-beta header allowlist. It asserts Authorization and Cookie never reach the upstream. It also asserts the upstream sees exactly one x-api-key value: the gateway's own key, never the client's. A single Get() call could miss a second, leaked value, since Set() and Add() interact in a way a naive check would not catch. The test checks the Values() length explicitly. Add a test case for the native passthrough route's prompt-cache billing. It fails if cache_creation_input_tokens or cache_read_input_tokens drop out of the billed prompt count. This closes a production billing change that previously carried no coverage at all. Decouple response delivery from usage extraction on the passthrough branch. A response body read failure, or a malformed usage field, no longer both discard the response and misreport the failure as an upstream connection error. A read failure still returns an error, now correctly classified as a translation failure. A malformed usage field no longer blocks the response at all: the client still gets it, and the existing zero-usage estimate takes over for accounting. Return the adapter's own usage value on a chatCompletion error in the translated path, instead of discarding it. This has no effect while streaming stays rejected up front. It prevents the same critical bug from resurfacing once streaming support lands. Normalize tool_result content to a plain string before applying the error prefix. The prior version only prefixed a Go string. Anthropic's array content form, which Claude Code sends, lost the is_error signal entirely. Absent content now becomes an empty string instead of the JSON null OpenAI rejects. Treat "stream": null the same as an absent key. The prior fix for a non-boolean stream value also rejected null. An SDK that serializes an unset optional field as null must still get an ordinary non-streaming response. Drop a yaegi-check assertion that could not fail. The fixture configures no cache, so the header it checked was never set regardless of what the code under test did. A mutation pass confirmed this: it deleted the line the assertion was meant to guard, and make yaegi-check still reported OK. Document that this route now intercepts POST /v1/messages unconditionally, even under passthroughUnknown, where it previously fell through to the next handler. Add the note to both the new README section and Known limitations. Correct the passthrough table's claim about response bytes to describe the actual guarantee.
This commit is contained in:
+44
-22
@@ -937,36 +937,58 @@ func openAIToolCallFromAnthropic(bm map[string]any) (map[string]any, error) {
|
||||
// openAIToolMessageFromAnthropic maps one Anthropic {"type":"tool_result",
|
||||
// "tool_use_id","content","is_error"} content block to an OpenAI
|
||||
// role:"tool" message ({"role":"tool","tool_call_id","content"}), the
|
||||
// mirror of anthropicToolResultBlock above. is_error (item 11 fix,
|
||||
// 2026-08-22 review) has no native OpenAI equivalent — a tool message
|
||||
// carries no error flag — so it is preserved via prefixErrorMarker
|
||||
// instead of being silently dropped: the previous version of this
|
||||
// function read only tool_use_id/content and discarded is_error
|
||||
// entirely, losing the failure signal for any OpenAI-backed agentic loop
|
||||
// that branches on whether a tool call succeeded.
|
||||
// mirror of anthropicToolResultBlock above. is_error (item 11/F4 fix,
|
||||
// 2026-08-22/23 review) has no native OpenAI equivalent — a tool message
|
||||
// carries no error flag — so it is preserved by prefixing "Error: " onto
|
||||
// content once openAIToolResultContentString (below) has normalized it
|
||||
// to a plain string, instead of being silently dropped: an earlier
|
||||
// version of this function read only tool_use_id/content and discarded
|
||||
// is_error entirely, and a second earlier version only prefixed a Go
|
||||
// string, still dropping the signal for Anthropic's array-content form.
|
||||
func openAIToolMessageFromAnthropic(bm map[string]any) map[string]any {
|
||||
toolUseID, _ := bm["tool_use_id"].(string)
|
||||
content := bm["content"]
|
||||
content := openAIToolResultContentString(bm["content"])
|
||||
if isErr, _ := bm["is_error"].(bool); isErr {
|
||||
content = prefixErrorMarker(content)
|
||||
content = "Error: " + content
|
||||
}
|
||||
return map[string]any{"role": "tool", "tool_call_id": toolUseID, "content": content}
|
||||
}
|
||||
|
||||
// prefixErrorMarker preserves Anthropic's tool_result.is_error signal
|
||||
// across the translation to OpenAI's tool message shape (item 11 fix,
|
||||
// 2026-08-22 review): a string content gets an "Error: " prefix so the
|
||||
// model still sees the failure signal in the one shape OpenAI's tool
|
||||
// message actually carries. Any other content shape (a block array, or
|
||||
// absent) is left as-is rather than guessing how to annotate it —
|
||||
// OpenAI-compatible upstreams generally infer failure from content
|
||||
// itself in that case, same as before this fix; only the common
|
||||
// string-content case previously lost the signal entirely.
|
||||
func prefixErrorMarker(content any) any {
|
||||
if s, ok := content.(string); ok {
|
||||
return "Error: " + s
|
||||
// openAIToolResultContentString normalizes an Anthropic tool_result
|
||||
// block's "content" into the single string OpenAI's tool message
|
||||
// "content" field requires (item 11/F4 fix, 2026-08-22/23 review). Three
|
||||
// source shapes all collapse to a string: a plain string passes through
|
||||
// unchanged; a content-block array (Anthropic permits this, and Claude
|
||||
// Code sends it for a tool_result carrying more than plain text) has its
|
||||
// text blocks concatenated; an absent/nil content becomes "". Handling
|
||||
// all three uniformly, here, is what lets openAIToolMessageFromAnthropic's
|
||||
// is_error prefix apply regardless of the source shape — the PREVIOUS
|
||||
// version only prefixed a Go string and left the array form unmarked
|
||||
// entirely, silently dropping the failure signal for real Claude Code
|
||||
// traffic (item 11 only covered half of it). It also fixes a second,
|
||||
// separate bug the array-only gap masked: an absent content previously
|
||||
// passed through as a literal Go nil, which this function's caller wrote
|
||||
// out as the JSON literal null — OpenAI rejects a null tool-message
|
||||
// content outright, where "" is accepted.
|
||||
func openAIToolResultContentString(content any) string {
|
||||
switch c := content.(type) {
|
||||
case string:
|
||||
return c
|
||||
case []any:
|
||||
var parts []string
|
||||
for _, block := range c {
|
||||
bm, ok := block.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if text, ok := bm["text"].(string); ok {
|
||||
parts = append(parts, text)
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, "")
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
return content
|
||||
}
|
||||
|
||||
// openAIMessagesFromAnthropic maps one Anthropic message ({"role",
|
||||
|
||||
Reference in New Issue
Block a user