Billing:
- Passthrough SSE streams now bill the usage the upstream reports.
The gateway falls back to a request-size estimate only on a 2xx
stream or a truncated JSON body.
- A unified stream that fails after it starts is estimated, not
billed at zero.
- Billing uses the built-in LiteLLM price table and honours
modelMeta free: true.
- Pricing entries are validated. Cost arithmetic rounds the total and
saturates on overflow.
- Gemini thinking tokens count as completion tokens.
Limits and storage:
- A rejected request rolls back its counts on the other scopes, so one
user cannot lock out a group.
- Redis EXPIRE replies are checked. Keys recreated without a TTL are
repaired at once.
- A mutating pipeline is not retried after reply bytes arrive.
- History, model totals and target usage report a store outage.
Routing and translation:
- Discovered ids with a provider-like prefix are listed only in
owner/id form, so listing and routing agree.
- Anthropic and Gemini discovery follows pagination.
- The developer role maps to system.
- stream_options.include_usage is honoured on translated streams.
- tool_choice none keeps tools when history holds tool blocks.
- refusal maps to content_filter. Data URI parameters are stripped.
Streaming, retry and failover:
- SSE accepts CR, LF and CRLF line ends. This closes a CR injection
path.
- Retry-After is never jittered. A cancelled wait returns the
context error.
- The watchdog timer stops at end of body.
- Oversized bodies return 413.
- A failed stream ends with an error event.
- Failover restores pre-existing headers. Client cancels do not count
as provider successes.
Also: "admin" is a reserved provider name. metrics.path must start
with "/". Pricing warnings go to the right gateway instance. The admin
targets view sends access as null or [] so the two states differ.
Upstream organisation headers are stripped. The users-file reload
error is logged once per error. Dead code is removed, and test-only
helpers move to test files.
Claude-Session: https://claude.ai/code/session_01V5jYJNxpDFdJkyyExpADrv
Review round 1 fixes (1 Critical + 5 Important) on the anthropic
adapter/translator.
- CRITICAL: consecutive role:"tool" messages each became their own
Anthropic user turn. Anthropic requires alternating user/assistant
turns and 400s on two consecutive same-role messages; parallel tool
calls (one assistant message with N tool_calls followed by N
role:"tool" messages) are the standard agentic transcript shape.
anthropicRequestFromOpenAI now accumulates consecutive tool messages
into one Anthropic user turn carrying all of their tool_result
blocks (anthropicToolResultBlock replaces anthropicToolResultMessage).
Added chat_parallel_tool_results fixture.
- Streaming tool_calls[].index passed Anthropic's raw content-block
index through, so a tool_use block after a text block emitted
index:1 instead of the required contiguous 0-based ordinal.
anthropicStreamState now tracks a per-stream tool-call ordinal
counter plus a content-block-index -> ordinal map, populated on
content_block_start and consulted on input_json_delta. Added the
stream_text_then_tool fixture (text block at index 0, tool_use at
index 1) proving the emitted index is 0.
- A system message with array-form content ([{"type":"text",...}])
was silently dropped (only a string content was handled). Added
systemTextFromContentParts: text parts join with "\n"; a non-text
part returns a *translateError instead of being dropped. Added
chat_system_array_content (success) and
chat_system_array_content_error fixtures.
- Malformed JSON on a recognized stream event (message_start,
content_block_start, content_block_delta, message_delta) was
swallowed as (nil, nil), which could silently corrupt a client's
reassembled tool-call arguments. All four now return an error
wrapping errUpstream; forwardStream already propagates that and
withholds the terminal [DONE]. The unrecognized-event-type and
error-event branches are unchanged. Added
TestAnthropicStreamState_MalformedEventReturnsError.
- providerAdapter.injectAuth's doc comment claimed a universal
empty-key no-op; updated to state the per-type divergence
(openai-type: keyless is a no-op; anthropic-type: the constructor
rejects an empty key, so injectAuth is never called with one).
- Added case-insensitive-Content-Type coverage for both adapters'
streaming-vs-JSON gate (previously untested): one subtest each in
provider_openai_test.go / provider_anthropic_test.go using a fake
upstream that sends "Text/Event-Stream; charset=utf-8", asserting
the streaming path (not the JSON fallback) is taken.
Gate: go test ./... -count=1 and -race both green; gofmt, go vet,
fieldalignment, golangci-lint (incl. gosec), govulncheck all clean.
Implements Task 9: the anthropic-type providerAdapter and its
request/response/stream translation to and from OpenAI's wire shape.
- translate_anthropic.go: anthropicRequestFromOpenAI (system messages
joined, content blocks incl. base64 data-URI images, tool_calls ->
tool_use, tool messages -> tool_result, max_tokens defaulting,
stop/stop_sequences, tool_choice incl. the none-drops-tools rule,
n>1/logit_bias/logprobs rejected), openAIResponseFromAnthropic
(content blocks joined/tool_calls re-marshaled, stop_reason mapping,
usage), and anthropicStreamState.translate mapping each SSE event to
zero or more OpenAI chat.completion.chunk payloads. translateError
carries a notSupported flag so the caller (Task 12) can tell a 400
from a 501.
- provider_anthropic.go: anthropicAdapter implementing providerAdapter.
newAnthropicAdapter errors on an empty API key per ruling (c), unlike
openai-type's keyless ruling. Headers: x-api-key, anthropic-version.
embeddings always returns a notSupported translateError without
calling upstream (Anthropic has no embeddings endpoint). listModels
GETs /v1/models and returns any non-2xx (incl. 404) as-is.
- providers.go: wires the anthropic buildAdapters case to
newAnthropicAdapter, propagating its constructor error.
- provider_openai_test.go: replaces the now-stale placeholder
TestBuildAdapters subtest with one asserting a keyed anthropic
provider builds successfully and a keyless one errors.
- translate_anthropic_test.go: golden-file harness walking
testdata/anthropic/{chat,resp}_*_in.json against sibling _want.json
(success) or _wanterr.json (*translateError) fixtures, plus a stream
harness driving each *.sse fixture through anthropicStreamState and
asserting the chunk list and usage().
- provider_anthropic_test.go: httptest-backed adapter wiring tests
(streaming + non-streaming), asserting OpenAI-shaped output on the
wire, per-event flushing, and the required auth headers.
Gate: go test ./... -count=1 and -race both green; gofmt, go vet,
fieldalignment, golangci-lint (incl. gosec) all clean.
lukaszraczylo