mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-13 04:58:50 +00:00
feat: anthropic adapter with bidirectional translation incl streaming
Implements Task 9: the anthropic-type providerAdapter and its
request/response/stream translation to and from OpenAI's wire shape.
- translate_anthropic.go: anthropicRequestFromOpenAI (system messages
joined, content blocks incl. base64 data-URI images, tool_calls ->
tool_use, tool messages -> tool_result, max_tokens defaulting,
stop/stop_sequences, tool_choice incl. the none-drops-tools rule,
n>1/logit_bias/logprobs rejected), openAIResponseFromAnthropic
(content blocks joined/tool_calls re-marshaled, stop_reason mapping,
usage), and anthropicStreamState.translate mapping each SSE event to
zero or more OpenAI chat.completion.chunk payloads. translateError
carries a notSupported flag so the caller (Task 12) can tell a 400
from a 501.
- provider_anthropic.go: anthropicAdapter implementing providerAdapter.
newAnthropicAdapter errors on an empty API key per ruling (c), unlike
openai-type's keyless ruling. Headers: x-api-key, anthropic-version.
embeddings always returns a notSupported translateError without
calling upstream (Anthropic has no embeddings endpoint). listModels
GETs /v1/models and returns any non-2xx (incl. 404) as-is.
- providers.go: wires the anthropic buildAdapters case to
newAnthropicAdapter, propagating its constructor error.
- provider_openai_test.go: replaces the now-stale placeholder
TestBuildAdapters subtest with one asserting a keyed anthropic
provider builds successfully and a keyless one errors.
- translate_anthropic_test.go: golden-file harness walking
testdata/anthropic/{chat,resp}_*_in.json against sibling _want.json
(success) or _wanterr.json (*translateError) fixtures, plus a stream
harness driving each *.sse fixture through anthropicStreamState and
asserting the chunk list and usage().
- provider_anthropic_test.go: httptest-backed adapter wiring tests
(streaming + non-streaming), asserting OpenAI-shaped output on the
wire, per-event flushing, and the required auth headers.
Gate: go test ./... -count=1 and -race both green; gofmt, go vet,
fieldalignment, golangci-lint (incl. gosec) all clean.
This commit is contained in:
+10
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, how are you?"}
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"top_p": 0.9,
|
||||
"stop": "STOP",
|
||||
"user": "user-123"
|
||||
}
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, how are you?"}
|
||||
],
|
||||
"max_tokens": 4096,
|
||||
"temperature": 0.7,
|
||||
"top_p": 0.9,
|
||||
"stop_sequences": ["STOP"],
|
||||
"metadata": {"user_id": "user-123"}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": [
|
||||
{"type": "text", "text": "What is in this image?"},
|
||||
{"type": "image_url", "image_url": {"url": "https://example.com/cat.png"}}
|
||||
]}
|
||||
],
|
||||
"max_tokens": 300
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"errContains": "data: URI", "notSupported": false}
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": [
|
||||
{"type": "text", "text": "What is in this image?"},
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB"}}
|
||||
]}
|
||||
],
|
||||
"max_tokens": 300
|
||||
}
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "user", "content": [
|
||||
{"type": "text", "text": "What is in this image?"},
|
||||
{"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB"}}
|
||||
]}
|
||||
],
|
||||
"max_tokens": 300
|
||||
}
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [
|
||||
{"role": "system", "content": "You are a helpful assistant."},
|
||||
{"role": "system", "content": "Always be concise."},
|
||||
{"role": "user", "content": "What's the weather in Paris?"},
|
||||
{"role": "assistant", "content": null, "tool_calls": [
|
||||
{"id": "call_abc123", "type": "function", "function": {"name": "get_weather", "arguments": "{\"location\":\"Paris\"}"}}
|
||||
]},
|
||||
{"role": "tool", "tool_call_id": "call_abc123", "content": "72F and sunny"}
|
||||
],
|
||||
"max_tokens": 512,
|
||||
"stop": ["END", "STOP"],
|
||||
"tools": [
|
||||
{"type": "function", "function": {"name": "get_weather", "description": "Get current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
}
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"system": "You are a helpful assistant.\n\nAlways be concise.",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What's the weather in Paris?"},
|
||||
{"role": "assistant", "content": [
|
||||
{"type": "tool_use", "id": "call_abc123", "name": "get_weather", "input": {"location": "Paris"}}
|
||||
]},
|
||||
{"role": "user", "content": [
|
||||
{"type": "tool_result", "tool_use_id": "call_abc123", "content": "72F and sunny"}
|
||||
]}
|
||||
],
|
||||
"max_tokens": 512,
|
||||
"stop_sequences": ["END", "STOP"],
|
||||
"tools": [
|
||||
{"name": "get_weather", "description": "Get current weather for a location", "input_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}
|
||||
],
|
||||
"tool_choice": {"type": "auto"}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "Do something"}],
|
||||
"max_tokens": 100,
|
||||
"tools": [
|
||||
{"type": "function", "function": {"name": "noop", "description": "does nothing", "parameters": {"type": "object", "properties": {}}}}
|
||||
],
|
||||
"tool_choice": "none"
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "Do something"}],
|
||||
"max_tokens": 100
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "Do something"}],
|
||||
"max_tokens": 100,
|
||||
"tools": [
|
||||
{"type": "function", "function": {"name": "act", "description": "acts", "parameters": {"type": "object", "properties": {}}}}
|
||||
],
|
||||
"tool_choice": "required"
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "Do something"}],
|
||||
"max_tokens": 100,
|
||||
"tools": [
|
||||
{"name": "act", "description": "acts", "input_schema": {"type": "object", "properties": {}}}
|
||||
],
|
||||
"tool_choice": {"type": "any"}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "Do something"}],
|
||||
"max_tokens": 100,
|
||||
"tools": [
|
||||
{"type": "function", "function": {"name": "act", "description": "acts", "parameters": {"type": "object", "properties": {}}}}
|
||||
],
|
||||
"tool_choice": {"type": "function", "function": {"name": "act"}}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "Do something"}],
|
||||
"max_tokens": 100,
|
||||
"tools": [
|
||||
{"name": "act", "description": "acts", "input_schema": {"type": "object", "properties": {}}}
|
||||
],
|
||||
"tool_choice": {"type": "tool", "name": "act"}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
"logit_bias": {"50256": -100}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"errContains": "logit_bias is not supported for anthropic models", "notSupported": false}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
"logprobs": true
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"errContains": "logprobs is not supported for anthropic models", "notSupported": false}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"model": "claude-opus-5",
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
"n": 2
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"errContains": "n is not supported for anthropic models", "notSupported": false}
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"id": "msg_01TRUNC",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "text", "text": "This response was cut"}
|
||||
],
|
||||
"model": "claude-opus-5",
|
||||
"stop_reason": "max_tokens",
|
||||
"stop_sequence": null,
|
||||
"usage": {"input_tokens": 5, "output_tokens": 100}
|
||||
}
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"id": "chatcmpl-msg_01TRUNC",
|
||||
"object": "chat.completion",
|
||||
"created": 1734000000,
|
||||
"model": "gw-anthropic-model",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": "This response was cut"},
|
||||
"finish_reason": "length"
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 5, "completion_tokens": 100, "total_tokens": 105}
|
||||
}
|
||||
Vendored
+12
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"id": "msg_01XYZ",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "text", "text": "The capital of France is Paris."}
|
||||
],
|
||||
"model": "claude-opus-5",
|
||||
"stop_reason": "end_turn",
|
||||
"stop_sequence": null,
|
||||
"usage": {"input_tokens": 12, "output_tokens": 8}
|
||||
}
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"id": "chatcmpl-msg_01XYZ",
|
||||
"object": "chat.completion",
|
||||
"created": 1734000000,
|
||||
"model": "gw-anthropic-model",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": "The capital of France is Paris."},
|
||||
"finish_reason": "stop"
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 12, "completion_tokens": 8, "total_tokens": 20}
|
||||
}
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"id": "msg_01ABC",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "text", "text": "Let me check the weather."},
|
||||
{"type": "tool_use", "id": "toolu_01DEF", "name": "get_weather", "input": {"location": "Paris"}}
|
||||
],
|
||||
"model": "claude-opus-5",
|
||||
"stop_reason": "tool_use",
|
||||
"stop_sequence": null,
|
||||
"usage": {"input_tokens": 20, "output_tokens": 15}
|
||||
}
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"id": "chatcmpl-msg_01ABC",
|
||||
"object": "chat.completion",
|
||||
"created": 1734000000,
|
||||
"model": "gw-anthropic-model",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "Let me check the weather.",
|
||||
"tool_calls": [
|
||||
{"id": "toolu_01DEF", "type": "function", "function": {"name": "get_weather", "arguments": "{\"location\":\"Paris\"}"}}
|
||||
]
|
||||
},
|
||||
"finish_reason": "tool_calls"
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 20, "completion_tokens": 15, "total_tokens": 35}
|
||||
}
|
||||
Vendored
+24
@@ -0,0 +1,24 @@
|
||||
event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_01STREAM","type":"message","role":"assistant","content":[],"model":"claude-opus-5","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":10,"output_tokens":1}}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
|
||||
|
||||
event: ping
|
||||
data: {"type":"ping"}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" world"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":5}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
+1
@@ -0,0 +1 @@
|
||||
{"prompt": 10, "completion": 5}
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
[
|
||||
{"id":"chatcmpl-msg_01STREAM","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01STREAM","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01STREAM","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01STREAM","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
||||
]
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_01TOOL","type":"message","role":"assistant","content":[],"model":"claude-opus-5","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":15,"output_tokens":1}}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_01XYZ","name":"get_weather","input":{}}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"location\": \""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"Paris\"}"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":7}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"prompt": 15, "completion": 7}
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
[
|
||||
{"id":"chatcmpl-msg_01TOOL","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01TOOL","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"toolu_01XYZ","type":"function","function":{"name":"get_weather","arguments":""}}]},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01TOOL","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"location\": \""}}]},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01TOOL","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"Paris\"}"}}]},"finish_reason":null}]},
|
||||
{"id":"chatcmpl-msg_01TOOL","object":"chat.completion.chunk","created":1734000000,"model":"gw-anthropic-model","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}
|
||||
]
|
||||
Reference in New Issue
Block a user