mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
fix(anthropic): merge parallel tool results, ordinal stream tool indices, array system content, strict stream decode
Review round 1 fixes (1 Critical + 5 Important) on the anthropic
adapter/translator.
- CRITICAL: consecutive role:"tool" messages each became their own
Anthropic user turn. Anthropic requires alternating user/assistant
turns and 400s on two consecutive same-role messages; parallel tool
calls (one assistant message with N tool_calls followed by N
role:"tool" messages) are the standard agentic transcript shape.
anthropicRequestFromOpenAI now accumulates consecutive tool messages
into one Anthropic user turn carrying all of their tool_result
blocks (anthropicToolResultBlock replaces anthropicToolResultMessage).
Added chat_parallel_tool_results fixture.
- Streaming tool_calls[].index passed Anthropic's raw content-block
index through, so a tool_use block after a text block emitted
index:1 instead of the required contiguous 0-based ordinal.
anthropicStreamState now tracks a per-stream tool-call ordinal
counter plus a content-block-index -> ordinal map, populated on
content_block_start and consulted on input_json_delta. Added the
stream_text_then_tool fixture (text block at index 0, tool_use at
index 1) proving the emitted index is 0.
- A system message with array-form content ([{"type":"text",...}])
was silently dropped (only a string content was handled). Added
systemTextFromContentParts: text parts join with "\n"; a non-text
part returns a *translateError instead of being dropped. Added
chat_system_array_content (success) and
chat_system_array_content_error fixtures.
- Malformed JSON on a recognized stream event (message_start,
content_block_start, content_block_delta, message_delta) was
swallowed as (nil, nil), which could silently corrupt a client's
reassembled tool-call arguments. All four now return an error
wrapping errUpstream; forwardStream already propagates that and
withholds the terminal [DONE]. The unrecognized-event-type and
error-event branches are unchanged. Added
TestAnthropicStreamState_MalformedEventReturnsError.
- providerAdapter.injectAuth's doc comment claimed a universal
empty-key no-op; updated to state the per-type divergence
(openai-type: keyless is a no-op; anthropic-type: the constructor
rejects an empty key, so injectAuth is never called with one).
- Added case-insensitive-Content-Type coverage for both adapters'
streaming-vs-JSON gate (previously untested): one subtest each in
provider_openai_test.go / provider_anthropic_test.go using a fake
upstream that sends "Text/Event-Stream; charset=utf-8", asserting
the streaming path (not the JSON fallback) is taken.
Gate: go test ./... -count=1 and -race both green; gofmt, go vet,
fieldalignment, golangci-lint (incl. gosec), govulncheck all clean.
This commit is contained in:
@@ -364,6 +364,49 @@ func TestOpenAIAdapter_ChatCompletion_Streaming_UpstreamIgnoresStreamFallsBackTo
|
||||
}
|
||||
}
|
||||
|
||||
// TestOpenAIAdapter_ChatCompletion_Streaming_CaseInsensitiveContentType
|
||||
// proves the streaming-vs-JSON-fallback gate matches an upstream's
|
||||
// Content-Type case-insensitively — review fix: Content-Type values are
|
||||
// case-insensitive per RFC 7231, but the gate previously did a
|
||||
// case-sensitive substring match against "text/event-stream" and would
|
||||
// wrongly fall back to the JSON path for an upstream that sent, e.g.,
|
||||
// "Text/Event-Stream". Distinguishing evidence: the streaming path
|
||||
// extracts real usage from the SSE chunks and sets Content-Type to the
|
||||
// sseWriter's own lowercase "text/event-stream"; the JSON fallback path
|
||||
// would copy the upstream's mixed-case header verbatim and fail to parse
|
||||
// usage out of raw SSE text.
|
||||
func TestOpenAIAdapter_ChatCompletion_Streaming_CaseInsensitiveContentType(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "Text/Event-Stream; charset=utf-8")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
f, ok := w.(http.Flusher)
|
||||
if !ok {
|
||||
t.Fatalf("httptest ResponseWriter does not implement http.Flusher")
|
||||
}
|
||||
for _, frame := range streamFrames {
|
||||
fmt.Fprintf(w, "data: %s\n\n", frame)
|
||||
f.Flush()
|
||||
}
|
||||
fmt.Fprint(w, "data: [DONE]\n\n")
|
||||
f.Flush()
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
a := newOpenAIAdapter("p1", srv.URL, "sk-test")
|
||||
rec := newRecordingResponseWriter()
|
||||
|
||||
u, err := a.chatCompletion(context.Background(), rec, map[string]any{"model": "gpt-5", "stream": true})
|
||||
if err != nil {
|
||||
t.Fatalf("chatCompletion: %v", err)
|
||||
}
|
||||
if u.prompt != 7 || u.completion != 9 {
|
||||
t.Errorf("usage = %+v, want {prompt:7 completion:9} (proves the streaming path, not JSON fallback, was taken)", u)
|
||||
}
|
||||
if got := rec.header.Get("Content-Type"); got != "text/event-stream" {
|
||||
t.Errorf("Content-Type on client response = %q, want %q (sseWriter's own header; JSON fallback would copy the upstream's mixed-case value verbatim)", got, "text/event-stream")
|
||||
}
|
||||
}
|
||||
|
||||
// TestOpenAIAdapter_ChatCompletion_Upstream429 proves a non-2xx upstream
|
||||
// response returns a *providerHTTPError carrying the status and body,
|
||||
// with nothing written to the client — the caller decides envelope vs
|
||||
|
||||
Reference in New Issue
Block a user