mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
fix(anthropic): merge parallel tool results, ordinal stream tool indices, array system content, strict stream decode
Review round 1 fixes (1 Critical + 5 Important) on the anthropic
adapter/translator.
- CRITICAL: consecutive role:"tool" messages each became their own
Anthropic user turn. Anthropic requires alternating user/assistant
turns and 400s on two consecutive same-role messages; parallel tool
calls (one assistant message with N tool_calls followed by N
role:"tool" messages) are the standard agentic transcript shape.
anthropicRequestFromOpenAI now accumulates consecutive tool messages
into one Anthropic user turn carrying all of their tool_result
blocks (anthropicToolResultBlock replaces anthropicToolResultMessage).
Added chat_parallel_tool_results fixture.
- Streaming tool_calls[].index passed Anthropic's raw content-block
index through, so a tool_use block after a text block emitted
index:1 instead of the required contiguous 0-based ordinal.
anthropicStreamState now tracks a per-stream tool-call ordinal
counter plus a content-block-index -> ordinal map, populated on
content_block_start and consulted on input_json_delta. Added the
stream_text_then_tool fixture (text block at index 0, tool_use at
index 1) proving the emitted index is 0.
- A system message with array-form content ([{"type":"text",...}])
was silently dropped (only a string content was handled). Added
systemTextFromContentParts: text parts join with "\n"; a non-text
part returns a *translateError instead of being dropped. Added
chat_system_array_content (success) and
chat_system_array_content_error fixtures.
- Malformed JSON on a recognized stream event (message_start,
content_block_start, content_block_delta, message_delta) was
swallowed as (nil, nil), which could silently corrupt a client's
reassembled tool-call arguments. All four now return an error
wrapping errUpstream; forwardStream already propagates that and
withholds the terminal [DONE]. The unrecognized-event-type and
error-event branches are unchanged. Added
TestAnthropicStreamState_MalformedEventReturnsError.
- providerAdapter.injectAuth's doc comment claimed a universal
empty-key no-op; updated to state the per-type divergence
(openai-type: keyless is a no-op; anthropic-type: the constructor
rejects an empty key, so injectAuth is never called with one).
- Added case-insensitive-Content-Type coverage for both adapters'
streaming-vs-JSON gate (previously untested): one subtest each in
provider_openai_test.go / provider_anthropic_test.go using a fake
upstream that sends "Text/Event-Stream; charset=utf-8", asserting
the streaming path (not the JSON fallback) is taken.
Gate: go test ./... -count=1 and -race both green; gofmt, go vet,
fieldalignment, golangci-lint (incl. gosec), govulncheck all clean.
This commit is contained in:
@@ -270,6 +270,16 @@ func TestAnthropicStreamGolden_ToolUse(t *testing.T) {
|
||||
runStreamGolden(t, "stream_tooluse")
|
||||
}
|
||||
|
||||
// TestAnthropicStreamGolden_TextThenTool covers a text block followed by
|
||||
// a tool_use block (Anthropic content-block index 0 then 1) and proves
|
||||
// the emitted tool_calls[0].index is the 0-based tool-call ordinal (0),
|
||||
// not the raw Anthropic content-block index (1) — review fix: OpenAI's
|
||||
// contract requires tool_calls[].index to be contiguous from 0 over the
|
||||
// tool-call array, regardless of how many non-tool_use blocks preceded it.
|
||||
func TestAnthropicStreamGolden_TextThenTool(t *testing.T) {
|
||||
runStreamGolden(t, "stream_text_then_tool")
|
||||
}
|
||||
|
||||
// TestAnthropicStreamState_ErrorEvent proves an "error" SSE event
|
||||
// translates to one OpenAI-shaped {"error":{...}} chunk and a non-nil
|
||||
// terminal error, per the stream mapping table's error row — built
|
||||
@@ -307,6 +317,33 @@ func TestAnthropicStreamState_ErrorEvent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestAnthropicStreamState_MalformedEventReturnsError proves a malformed
|
||||
// (invalid-JSON) data payload on each recognized event type returns an
|
||||
// error wrapping errUpstream, rather than being swallowed as (nil, nil) —
|
||||
// review fix: a dropped input_json_delta would otherwise silently corrupt
|
||||
// the client's reassembled tool-call arguments. The "error" event type
|
||||
// itself already returned an error before this fix and is covered by
|
||||
// TestAnthropicStreamState_ErrorEvent, not repeated here.
|
||||
func TestAnthropicStreamState_MalformedEventReturnsError(t *testing.T) {
|
||||
const corrupt = `{"type":"message_start","message":{`
|
||||
|
||||
for _, event := range []string{"message_start", "content_block_start", "content_block_delta", "message_delta"} {
|
||||
t.Run(event, func(t *testing.T) {
|
||||
st := newAnthropicStreamState(goldenGatewayModel, goldenCreated)
|
||||
chunks, err := st.translate(sseEvent{event: event, data: []byte(corrupt)})
|
||||
if err == nil {
|
||||
t.Fatalf("translate(%s, corrupt data): want error, got nil (chunks=%v)", event, chunks)
|
||||
}
|
||||
if !errors.Is(err, errUpstream) {
|
||||
t.Errorf("translate(%s, corrupt data): err = %v, want errors.Is(err, errUpstream)", event, err)
|
||||
}
|
||||
if len(chunks) != 0 {
|
||||
t.Errorf("translate(%s, corrupt data): chunks = %v, want none", event, chunks)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestAnthropicListModels_404NotSpeciallyHandled proves listModels
|
||||
// returns a plain *providerHTTPError on a 404 (an older Anthropic API
|
||||
// version that predates the Models endpoint) rather than any special
|
||||
|
||||
Reference in New Issue
Block a user