feat: unified chat and embeddings endpoints with limits and accounting

Implements Task 12: POST /v1/chat/completions and POST /v1/embeddings
through a shared pipeline (routes_unified.go: handleChat, handleEmbeddings,
runUnified) wired into ServeHTTP's dispatch in llmgateway.go.

Pipeline: decode (10MiB cap) -> resolve model -> enforce user/group
limits -> call adapter -> account usage, translating every failure mode
into its HTTP envelope per the task brief's carried rulings:

- ALIAS ECHO (ruling a): the client-requested model id is threaded to the
  adapter via the internal gatewayAliasKey convention and echoed back in
  anthropic/gemini's translated response model field; every adapter
  strips the key before building its upstream wire body so it never
  leaks to a real provider.
- Limit violations (ruling b): storeDown -> 503 server_error; otherwise
  429 rate_limit_error with Retry-After when the violation names a
  window.
- Stream-safe error handling (ruling c): statusTrackingWriter.wroteHeader
  gates whether an adapter error still gets an envelope; providerHTTPError
  wraps into {error:{message,type,code,upstream}}; translateError maps to
  400/501; context.Canceled logs quietly; anything else is a 502.
- Usage/accounting (ruling d): zero non-streaming usage gets a
  ceil(len(body)/4) prompt-token estimate; zero streaming usage logs once
  and accounts the request only. unifiedCostMicros extends costMicros'
  single-id lookup to canonical-then-bare without changing its signature.
- Scopes (ruling e): user scope only when u.limits is set, group scope
  only when grp.limits is set, user first.

routes_unified_test.go drives every case end-to-end through
Gateway.ServeHTTP against httptest fake upstreams (openai- and
anthropic-shaped): happy path with counters asserted via the limiter's
own store, incremental stream flushing, 401/403/404/429/503, embeddings,
alias echo through a provider-prefixed anthropic request, wrapped
upstream error envelopes, and a mid-stream drop that must not append a
second envelope to an already-committed response.

Gates: gofmt, go vet, fieldalignment, golangci-lint (incl. gosec) all
clean; go test ./... -race -count=1 passes.
This commit is contained in:
2026-08-20 10:45:12 +01:00
parent e91f0e36ff
commit dcfcfb7e48
6 changed files with 851 additions and 3 deletions
+9
View File
@@ -92,6 +92,14 @@ type chatStreamChunk struct {
// resulting usage-only chunk from a client that never asked for it while
// still capturing its usage for accounting.
func (a *openaiAdapter) chatCompletion(ctx context.Context, w http.ResponseWriter, req map[string]any) (usage, error) {
// gatewayAliasKey must never reach the real provider: an openai-type
// adapter forwards req verbatim as the upstream wire body, so a
// leftover alias entry would arrive as an unrecognized request field.
// This adapter's own response is a verbatim passthrough of whatever
// the upstream returns, so — unlike anthropic/gemini — there is no
// alias to echo back into it (documented in gatewayAliasKey's doc
// comment, routes_unified.go).
delete(req, gatewayAliasKey)
streaming, _ := req["stream"].(bool)
clientAskedUsage := false
if streaming {
@@ -135,6 +143,7 @@ func (a *openaiAdapter) chatCompletion(ctx context.Context, w http.ResponseWrite
// stream, so it is forwardJSON end to end, same as chatCompletion's
// non-streaming path.
func (a *openaiAdapter) embeddings(ctx context.Context, w http.ResponseWriter, req map[string]any) (usage, error) {
delete(req, gatewayAliasKey) // see chatCompletion's identical delete for why
resp, err := upstreamJSON(ctx, a.client, http.MethodPost, a.baseURL+"/v1/embeddings", a.requestHeaders(true), req)
if err != nil {
return usage{}, err