mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-13 04:58:50 +00:00
fix(anthropic): merge parallel tool results, ordinal stream tool indices, array system content, strict stream decode
Review round 1 fixes (1 Critical + 5 Important) on the anthropic
adapter/translator.
- CRITICAL: consecutive role:"tool" messages each became their own
Anthropic user turn. Anthropic requires alternating user/assistant
turns and 400s on two consecutive same-role messages; parallel tool
calls (one assistant message with N tool_calls followed by N
role:"tool" messages) are the standard agentic transcript shape.
anthropicRequestFromOpenAI now accumulates consecutive tool messages
into one Anthropic user turn carrying all of their tool_result
blocks (anthropicToolResultBlock replaces anthropicToolResultMessage).
Added chat_parallel_tool_results fixture.
- Streaming tool_calls[].index passed Anthropic's raw content-block
index through, so a tool_use block after a text block emitted
index:1 instead of the required contiguous 0-based ordinal.
anthropicStreamState now tracks a per-stream tool-call ordinal
counter plus a content-block-index -> ordinal map, populated on
content_block_start and consulted on input_json_delta. Added the
stream_text_then_tool fixture (text block at index 0, tool_use at
index 1) proving the emitted index is 0.
- A system message with array-form content ([{"type":"text",...}])
was silently dropped (only a string content was handled). Added
systemTextFromContentParts: text parts join with "\n"; a non-text
part returns a *translateError instead of being dropped. Added
chat_system_array_content (success) and
chat_system_array_content_error fixtures.
- Malformed JSON on a recognized stream event (message_start,
content_block_start, content_block_delta, message_delta) was
swallowed as (nil, nil), which could silently corrupt a client's
reassembled tool-call arguments. All four now return an error
wrapping errUpstream; forwardStream already propagates that and
withholds the terminal [DONE]. The unrecognized-event-type and
error-event branches are unchanged. Added
TestAnthropicStreamState_MalformedEventReturnsError.
- providerAdapter.injectAuth's doc comment claimed a universal
empty-key no-op; updated to state the per-type divergence
(openai-type: keyless is a no-op; anthropic-type: the constructor
rejects an empty key, so injectAuth is never called with one).
- Added case-insensitive-Content-Type coverage for both adapters'
streaming-vs-JSON gate (previously untested): one subtest each in
provider_openai_test.go / provider_anthropic_test.go using a fake
upstream that sends "Text/Event-Stream; charset=utf-8", asserting
the streaming path (not the JSON fallback) is taken.
Gate: go test ./... -count=1 and -race both green; gofmt, go vet,
fieldalignment, golangci-lint (incl. gosec), govulncheck all clean.
This commit is contained in:
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What's the weather in Paris and Tokyo?"},
|
||||
{"role": "assistant", "content": null, "tool_calls": [
|
||||
{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": "{\"location\":\"Paris\"}"}},
|
||||
{"id": "call_2", "type": "function", "function": {"name": "get_weather", "arguments": "{\"location\":\"Tokyo\"}"}}
|
||||
]},
|
||||
{"role": "tool", "tool_call_id": "call_1", "content": "72F and sunny"},
|
||||
{"role": "tool", "tool_call_id": "call_2", "content": "50F and rainy"}
|
||||
],
|
||||
"max_tokens": 200
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What's the weather in Paris and Tokyo?"},
|
||||
{"role": "assistant", "content": [
|
||||
{"type": "tool_use", "id": "call_1", "name": "get_weather", "input": {"location": "Paris"}},
|
||||
{"type": "tool_use", "id": "call_2", "name": "get_weather", "input": {"location": "Tokyo"}}
|
||||
]},
|
||||
{"role": "user", "content": [
|
||||
{"type": "tool_result", "tool_use_id": "call_1", "content": "72F and sunny"},
|
||||
{"type": "tool_result", "tool_use_id": "call_2", "content": "50F and rainy"}
|
||||
]}
|
||||
],
|
||||
"max_tokens": 200
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "system", "content": [
|
||||
{"type": "text", "text": "You are a helpful assistant."},
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AAAA"}}
|
||||
]},
|
||||
{"role": "user", "content": "Hi"}
|
||||
],
|
||||
"max_tokens": 100
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"errContains": "system message content part type \"image_url\" is not supported for anthropic models", "notSupported": false}
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "system", "content": [
|
||||
{"type": "text", "text": "You are a helpful assistant."},
|
||||
{"type": "text", "text": "Always be concise."}
|
||||
]},
|
||||
{"role": "user", "content": "Hi"}
|
||||
],
|
||||
"max_tokens": 100
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"system": "You are a helpful assistant.\nAlways be concise.",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hi"}
|
||||
],
|
||||
"max_tokens": 100
|
||||
}
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_01MIX","type":"message","role":"assistant","content":[],"model":"claude-opus-5","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":8,"output_tokens":1}}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Let me check."}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_MIX1","name":"get_weather","input":{}}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"location\":\"Paris\"}"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":1}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":6}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"prompt": 8, "completion": 6}
|
||||
@@ -0,0 +1,7 @@
|
||||
[
|
||||
{"id":"chatcmpl-msg_01MIX","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01MIX","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"content":"Let me check."},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01MIX","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"toolu_MIX1","type":"function","function":{"name":"get_weather","arguments":""}}]},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01MIX","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"location\":\"Paris\"}"}}]},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01MIX","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}
|
||||
]
|
||||
Reference in New Issue
Block a user