diff --git a/README.md b/README.md index 1b04bc1..65470ff 100644 --- a/README.md +++ b/README.md @@ -170,6 +170,7 @@ config accepts them as YAML, which decodes to the same JSON shape. | `retry` | `RetryConfig` | `{}` (disabled) | Same-provider retry for transient upstream failures — see [Retry](#retry). Omitted or `enabled: false` means no retry: every request makes exactly one upstream attempt, byte-identical to a gateway built before this field existed. | | `cache` | `CacheConfig` | `{}` (disabled) | Opt-in Redis-backed response cache for unified non-streaming chat/embeddings — see [Caching](#caching). Omitted or `enabled: false` means no caching, byte-identical to a gateway built before this field existed. | | `admin` | `*AdminConfig` | `nil` (disabled) | Read-only admin dashboard — see [Admin](#admin). `nil` or `enabled: false` means the `/admin*` routes are not registered at all. | +| `modelAliases` | `map[string]string` | `{}` | Operator-defined alias id → target model id — see [Model aliases](#model-aliases). Omitted or empty means no aliases, byte-identical to a gateway built before this field existed. | | `passthroughUnknown` | `bool` | `false` | `false`: a request matching none of the plugin's routes gets a 404 JSON envelope. `true`: it falls through to the router's own backing service. | Provider, `mcpServers`, and `agents` map **keys** (names) must match @@ -359,6 +360,85 @@ kept — a bad edit to the file never breaks already-authenticated traffic. **stale-while-error**, never an empty list just because one refresh attempt failed. +## Model aliases + +Operator-defined indirection: a client-facing alias id that resolves to a +real target model id. `modelAliases` is empty by default, and an empty map +changes nothing about model resolution — this is purely additive. + +```yaml +http: + middlewares: + llmgateway: + plugin: + llmgateway: + # ... providers, groups, users unchanged ... + modelAliases: + # Client-facing name -> real target. The target can itself be + # bare ("gpt-5-mini") or provider-prefixed + # ("anthropic/claude-sonnet-4-5"); either resolves through the + # normal rules in [Model routing](#model-routing) above. + aliased/coding: "anthropic/claude-sonnet-4-5" + aliased/fast: "gpt-5-mini" +``` + +- **Swapping the underlying model**: point an alias at a different target + and let Traefik's dynamic-config reload apply it — no gateway restart, + no client-side change. A client that has always addressed + `aliased/coding` keeps working unmodified after the operator moves that + alias to a newer model. +- **Precedence**: an EXACT alias match wins resolution before any other + rule, including `provider/model` prefix splitting — checked first in + `modelRegistry.resolve`. The target then resolves through the same + rules as any other id (bare, `provider/model`, or a discovered id). + Because of this, an alias whose id contains a `/` is rejected at + construction if the part before the `/` names an actually configured + provider (it would be shadowed by prefix resolution) — see the + validation rules below. +- **Authorization**: a group may use an alias when its `models` glob + matches the **alias name itself**, OR when it already grants access to + the **resolved target** — either is enough. `providers` still gates the + target's own provider as usual. +- **Validation at construction** (a bad alias config fails the same way a + bad provider config does — no gateway starts with a broken alias + table): + - the alias must be non-empty and contain no control characters; + - when the alias contains `/`, its prefix must not name a configured + provider (would be shadowed by prefix resolution); + - the alias must not equal an explicit `models` entry of any provider + (ambiguous — which one wins?); + - the target must not itself be another alias (no alias chains); + - the target must not be empty. + A target naming a model that is not yet known (awaiting a provider's + first discovery fetch, or simply never configured) is accepted at + construction — it resolves lazily. An unresolvable target at request + time returns 404 with a message naming both the alias and the missing + target. A target that is later discovered and collides with an + already-configured alias id is not a construction error either — the + alias always wins that resolution, and the gateway logs one warning + per such id. +- **`GET /v1/models` listing**: every alias the caller's group may use is + listed as a model object alongside real models — `id` is the alias, + `owned_by` is the target's resolved provider name. An alias whose + target does not resolve yet is omitted from the listing until it does. +- **Echo behavior**: exactly like the existing provider-prefix alias echo + (`__alias`, [Unified vs. passthrough](#unified-vs-passthrough)) — + `anthropic`- and `gemini`-type targets build their own response + envelope and echo the client's exact alias string back in its `model` + field. An `openai`-type target's response is a **verbatim passthrough** + of the upstream body, so its `model` field carries whatever the + upstream itself returned (typically the bare upstream model id) — + **not** the alias — the same documented v0.1 passthrough asymmetry, + unchanged by this feature. +- **Applies everywhere a model routes**: unified chat and embeddings, + `/v1/images/generations`, `/v1/audio/speech`, and + `/v1/audio/transcriptions` (including its multipart rewrite — the + upstream `model` field receives the resolved target's own upstream id, + never the alias, exactly like a provider-prefixed id already does). +- **Admin overview**: `GET /admin/api/overview` includes an `aliases` + array (`{alias, target}`, sorted by alias) — see [Admin](#admin). No + secrets involved. + ## Limits and accounting - Counters use **fixed windows**, keyed by UTC calendar boundaries (the @@ -592,10 +672,12 @@ request to any of them falls through to the plugin's existing the `x-api-key` header on this page's own `/admin/api/*` fetches. - **`GET /admin/api/overview`** and **`GET /admin/api/usage`** share one gate: unauthenticated → 401, authenticated non-admin → 403, admin → - serve. `overview` returns the provider list, groups, redis/cache - status, and the plugin version string. `usage` returns every user's and - every group's current-window counter values (req/min, req/day, tok/day, - tok/month, cost/day, cost/month) alongside their configured limits. + serve. `overview` returns the provider list, groups, the configured + model alias table (`alias`/`target` pairs, sorted by alias — see + [Model aliases](#model-aliases)), redis/cache status, and the plugin + version string. `usage` returns every user's and every group's + current-window counter values (req/min, req/day, tok/day, tok/month, + cost/day, cost/month) alongside their configured limits. - **What's exposed**: provider names, types, base URLs (with any userinfo/query string stripped before it's ever echoed), model counts, discovery status, group/user names, membership, limits, and live usage diff --git a/admin.go b/admin.go index 128aa52..c8824ff 100644 --- a/admin.go +++ b/admin.go @@ -194,6 +194,14 @@ type adminGroupView struct { MemberCount int `json:"memberCount"` } +// adminAliasView is one configured model alias's row in GET +// /admin/api/overview (spec §5, v0.2): the alias->target pair exactly as +// an operator wrote it. No secrets involved. +type adminAliasView struct { + Alias string `json:"alias"` + Target string `json:"target"` +} + // adminOverviewResponse is the full body of GET /admin/api/overview (spec // §4, v0.2). Every slice is sorted and built fresh per request — no // caching of the response itself — so the dashboard's 5s poll always @@ -202,6 +210,7 @@ type adminOverviewResponse struct { Version string `json:"version"` Providers []adminProviderView `json:"providers"` Groups []adminGroupView `json:"groups"` + Aliases []adminAliasView `json:"aliases"` Redis adminRedisView `json:"redis"` Cache adminCacheView `json:"cache"` } @@ -237,11 +246,18 @@ func (g *Gateway) buildAdminOverview() adminOverviewResponse { groups[i] = adminGroupView{Name: gs.name, Limits: gs.limits, MemberCount: gs.memberCount} } + aliasSnaps := g.registry.aliasSnapshot() + aliases := make([]adminAliasView, len(aliasSnaps)) + for i, a := range aliasSnaps { + aliases[i] = adminAliasView(a) // identical underlying field shape (alias, target string), differing only in json tags + } + return adminOverviewResponse{ Providers: providers, Redis: adminRedisView{Configured: configured, LastErr: redisLastErr, LastErrAt: redisLastErrAt}, Cache: adminCacheView{Enabled: cacheEnabled, TTL: ttl}, Groups: groups, + Aliases: aliases, Version: pluginVersion, } } @@ -443,6 +459,13 @@ const adminPageHTML = ` +
+

Model aliases

+ + +
AliasTarget
+
+

Infrastructure

@@ -542,6 +565,13 @@ const adminPageHTML = ` return tr; }); + setRows("aliases", data.aliases, function (a) { + var tr = document.createElement("tr"); + tr.appendChild(el("td", a.alias)); + tr.appendChild(el("td", a.target)); + return tr; + }); + var infra = document.getElementById("infra"); while (infra.firstChild) infra.removeChild(infra.firstChild); var redisText = "Redis: " + (data.redis.configured ? "configured" : "not configured"); diff --git a/admin_test.go b/admin_test.go index 482a73c..87f6474 100644 --- a/admin_test.go +++ b/admin_test.go @@ -195,6 +195,14 @@ func assertOAIErrorEnvelope(t *testing.T, rec *httptest.ResponseRecorder) { func TestAdminOverview_SortedShapeAndVersion(t *testing.T) { t.Parallel() cfg := newAdminTestConfig() + // Two aliases, deliberately out of sorted order, proving both the + // overview's aliases table (spec §5, v0.2) is populated AND sorted by + // alias — "zaliased/x" is a bare target ("z-model", zeta's own + // explicit model), "aaliased/y" is provider-prefixed. + cfg.ModelAliases = map[string]string{ + "zaliased/x": "z-model", + "aaliased/y": "alpha/a-model-1", + } h, _ := newAdminGatewayHandle(t, cfg) rec := httptest.NewRecorder() @@ -247,6 +255,19 @@ func TestAdminOverview_SortedShapeAndVersion(t *testing.T) { if got.Redis.Configured { t.Error("redis.configured must be false: cfg.Redis was never configured") } + + wantAliases := []adminAliasView{ + {Alias: "aaliased/y", Target: "alpha/a-model-1"}, + {Alias: "zaliased/x", Target: "z-model"}, + } + if len(got.Aliases) != len(wantAliases) { + t.Fatalf("aliases = %+v, want exactly %+v", got.Aliases, wantAliases) + } + for i, want := range wantAliases { + if got.Aliases[i] != want { + t.Errorf("aliases[%d] = %+v, want %+v (sorted by alias)", i, got.Aliases[i], want) + } + } } func TestAdminOverview_ProviderLastErrAfterFailedRefresh(t *testing.T) { diff --git a/cache_test.go b/cache_test.go index dc270dd..d1bc065 100644 --- a/cache_test.go +++ b/cache_test.go @@ -151,6 +151,34 @@ func TestCacheKey_ChatVsEmbeddings_DistinctKeys(t *testing.T) { } } +// TestCacheKey_TwoOperatorAliases_SameTarget_DistinctKeys is task 6's +// (spec §5, v0.2) dedicated cache-key regression, phrased in terms of +// operator-defined model aliases (modelAliases) rather than +// TestCacheKey_BareVsPrefixedAlias_DistinctKeys' bare-vs-prefixed pair +// above. Two DIFFERENT alias ids ("aliased/coding", "aliased/fast") +// configured to resolve to the identical provider+upstreamModel target +// must still hash to distinct cache entries: cacheKey already hashes +// requestedModel as real key material (this file's own doc comment, +// TestCacheKey_BareVsPrefixedAlias_DistinctKeys), and +// modelRegistry.resolveAliasTarget (registry.go) leaves that string +// completely untouched regardless of which alias resolved to which +// target — the two features compose with NO cache.go changes required. +func TestCacheKey_TwoOperatorAliases_SameTarget_DistinctKeys(t *testing.T) { + req := map[string]any{"messages": []any{"hi"}} + + codingAlias := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/coding", cacheEndpointChat, req) + fastAlias := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/fast", cacheEndpointChat, req) + if codingAlias == fastAlias { + t.Error("cacheKey identical for two different operator-defined aliases resolving to the identical provider+upstreamModel target") + } + + // Same alias twice must still hash identically — a genuine repeat + // request under the same alias is a genuine cache hit. + if got, want := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/coding", cacheEndpointChat, req), codingAlias; got != want { + t.Errorf("cacheKey differs for two requests using the identical alias: %q vs %q", got, want) + } +} + // --- validateCacheConfig: defaults and validation --- func TestValidateCacheConfig_DefaultsAppliedWhenZero(t *testing.T) { diff --git a/integration/integration_test.go b/integration/integration_test.go index c0a2ffc..9b08971 100644 --- a/integration/integration_test.go +++ b/integration/integration_test.go @@ -239,6 +239,55 @@ func TestUnifiedChatAllProviders(t *testing.T) { } } +// TestModelAliases covers v0.2 integration coverage: task 6 (spec §5). +// dynamic.yml.tmpl configures two operator-defined aliases — +// "aliased/mock" -> "gpt-mock" (openai-type) and "aliased/claude" -> +// "claude-mock" (anthropic-type) — proving alias resolution runs under +// real Traefik+Yaegi, not just go test, including the documented +// echo-behavior asymmetry between provider types (see +// registry_test.go's Gateway-level equivalents for the same assertion +// under go test). +func TestModelAliases(t *testing.T) { + // openai-type: the upstream response is forwarded verbatim + // (provider_openai.go), so its "model" field carries whatever the + // mock echoes back — the resolved upstream id "gpt-mock", not the + // alias. Correctness here is 200 + real content + the cache header + // unified routes always set, not the "model" field (documented v0.1 + // passthrough asymmetry, unchanged by aliasing). + openaiReq := map[string]any{ + "model": "aliased/mock", + "messages": []map[string]any{{"role": "user", "content": "hi"}}, + } + resp, body := doJSON(t, http.MethodPost, traefik1URL+"/v1/chat/completions", aliceKey, openaiReq) + if resp.StatusCode != http.StatusOK { + t.Fatalf("aliased openai chat: status = %d, body=%#v", resp.StatusCode, body) + } + if content := firstChoiceContent(t, body); content != "mock openai response" { + t.Errorf("aliased openai chat: content = %q, want %q", content, "mock openai response") + } + if got := resp.Header.Get("X-Llmgw-Cache"); got == "" { + t.Error("aliased openai chat: X-Llmgw-Cache header missing, want \"miss\" or \"hit\"") + } + + // anthropic-type: translate_anthropic.go builds its own response + // envelope and echoes the client's exact requested id (the alias) + // back into "model" — this DOES surface the alias. + anthropicReq := map[string]any{ + "model": "aliased/claude", + "messages": []map[string]any{{"role": "user", "content": "hi"}}, + } + resp2, body2 := doJSON(t, http.MethodPost, traefik1URL+"/v1/chat/completions", aliceKey, anthropicReq) + if resp2.StatusCode != http.StatusOK { + t.Fatalf("aliased anthropic chat: status = %d, body=%#v", resp2.StatusCode, body2) + } + if content := firstChoiceContent(t, body2); content != "mock anthropic response" { + t.Errorf("aliased anthropic chat: content = %q, want %q", content, "mock anthropic response") + } + if model, _ := body2["model"].(string); model != "aliased/claude" { + t.Errorf("aliased anthropic chat: response model = %q, want the client-requested alias %q echoed back", model, "aliased/claude") + } +} + // TestStreamingIncrementalityProbe covers integration test 3. // // Content correctness always runs, via assertStreamingContent: a host-side diff --git a/integration/traefik/dynamic.yml.tmpl b/integration/traefik/dynamic.yml.tmpl index 40f01d1..18e2285 100644 --- a/integration/traefik/dynamic.yml.tmpl +++ b/integration/traefik/dynamic.yml.tmpl @@ -98,9 +98,10 @@ http: mcpServers: tool: url: "http://mocktool:8080" - # v0.2 features (spec §1, §2, §4), all opt-in and off by default - # in v0.1: enabled here so the integration suite exercises every - # one under real Traefik+Yaegi, not just go test. + # v0.2 features (spec §1, §2, §4, §5), all opt-in and off by + # default in v0.1: enabled here so the integration suite + # exercises every one under real Traefik+Yaegi, not just go + # test. retry: enabled: true attempts: 1 # one retry after the first try — matches handleFlakyOpenAIChat's single forced 503 @@ -109,6 +110,15 @@ http: ttl: "2m" admin: enabled: true + # Task 6 (spec §5): one alias per mock provider type, so the + # integration suite proves the echo-behavior asymmetry under + # real Traefik+Yaegi — openai-type forwards the upstream body + # verbatim (its "model" field carries "gpt-mock", not the + # alias); anthropic-type builds its own envelope and echoes the + # client's exact alias string back. + modelAliases: + aliased/mock: "gpt-mock" + aliased/claude: "claude-mock" # Second, separate middleware instance carrying only the optional # real-upstream smoke test's "uni" provider (task 7, INTEGRATION_REAL=1 diff --git a/llmgateway.go b/llmgateway.go index 1a19f8a..bad82bd 100644 --- a/llmgateway.go +++ b/llmgateway.go @@ -27,6 +27,15 @@ type Config struct { // all — ServeHTTP falls through to its existing 404/passthroughUnknown // handling for those paths, preserving v0.1 behavior exactly. Admin *AdminConfig `json:"admin,omitempty"` + // ModelAliases maps an operator-defined alias id to a target model id + // (spec §5, v0.2) — e.g. {"aliased/coding": "anthropic/claude-sonnet-4-5"}. + // An exact alias match wins resolution before any other rule + // (modelRegistry.resolve, registry.go); the target then resolves + // through the normal rules. nil/omitted preserves v0.1 behavior + // exactly: resolve never consults an empty alias map, so no existing + // model id's resolution changes. Validated at construction + // (validateModelAliases, registry.go). + ModelAliases map[string]string `json:"modelAliases,omitempty"` // Retry is a struct value, not a pointer, because its own Enabled // field is the on/off signal (unlike Redis/Users, where the block's // mere presence is the signal) — so its tag omits "omitempty": diff --git a/registry.go b/registry.go index 21f232d..e5f62f4 100644 --- a/registry.go +++ b/registry.go @@ -5,6 +5,7 @@ import ( "errors" "fmt" "sort" + "strings" "sync" "time" ) @@ -160,7 +161,16 @@ type modelRegistry struct { // specific to a field holding a func value. Verified empirically // against yaegi v0.16.1 (see tools/yaegi-check) and does not apply to // g.logf/g.errorf, which are methods, not fields. - log func(string, ...any) + log func(string, ...any) + // aliases is the validated alias->target map from Config.ModelAliases + // (spec §5, v0.2), built once by newModelRegistry via + // validateModelAliases and never mutated afterward — resolve checks + // it first, ahead of splitConfiguredProvider/bareWinner (see resolve's + // own doc comment). nil (not merely empty) when the operator + // configured no aliases at all, so every alias-aware code path is a + // cheap nil-map lookup (always false/zero) that adds no branching + // cost to the v0.1 no-aliases case. + aliases map[string]string nowFn func() time.Time warned map[string]bool providerNames []string @@ -187,6 +197,12 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func } sort.Strings(m.providerNames) + // explicitModels is the union of every configured provider's EXPLICIT + // model ids only (never discovered ones — discovery has not run yet + // at this point in construction) — validateModelAliases' collision + // check needs exactly this set (spec §5: "alias must not equal an + // explicit model id of any provider"). + explicitModels := make(map[string]bool) for _, name := range m.providerNames { pc := cfg.Providers[name] if pc == nil { @@ -205,6 +221,7 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func explicit := make(map[string]bool, len(pc.Models)) for _, id := range pc.Models { explicit[id] = true + explicitModels[id] = true } m.states[name] = &providerState{ @@ -214,9 +231,87 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func interval: interval, } } + + aliases, err := validateModelAliases(cfg.ModelAliases, m.providerNames, explicitModels) + if err != nil { + return nil, err + } + m.aliases = aliases + return m, nil } +// isValidModelID reports whether id satisfies spec §5's alias model-id +// shape: non-empty, and free of ASCII control characters (0x00-0x1F and +// 0x7F) — the same class of raw value mcp_a2a.go's target-URL validation +// already rejects, applied here to a model/alias id string instead of a +// URL. +func isValidModelID(id string) bool { + if id == "" { + return false + } + for _, r := range id { + if r < 0x20 || r == 0x7f { + return false + } + } + return true +} + +// validateModelAliases validates cfg.ModelAliases per spec §5's +// validation table and returns the accepted alias->target map (nil when +// raw is empty — newGateway's default, zero-behavior-change case). +// providerNames is the registry's own sorted provider-name list (used for +// the prefix-shadow check); explicitModels is the union of every +// configured provider's EXPLICIT model ids only (used for the collision +// check) — a DISCOVERED model colliding with an alias is NOT a +// construction error (discovery is dynamic and runs after this), and is +// instead resolved in the alias's favor by resolve's own precedence; see +// resolve's warnAliasShadowsDiscoveredOnce call for the one-time runtime +// log that implies. +// +// raw is iterated in sorted alias-key order, not Go's randomized map +// order, so a config with more than one invalid alias always reports the +// same one first, deterministically, across repeated runs. +func validateModelAliases(raw map[string]string, providerNames []string, explicitModels map[string]bool) (map[string]string, error) { + if len(raw) == 0 { + return nil, nil + } + + providerSet := make(map[string]bool, len(providerNames)) + for _, p := range providerNames { + providerSet[p] = true + } + + names := make([]string, 0, len(raw)) + for alias := range raw { + names = append(names, alias) + } + sort.Strings(names) + + out := make(map[string]string, len(raw)) + for _, alias := range names { + target := raw[alias] + if !isValidModelID(alias) { + return nil, fmt.Errorf("llmgateway: modelAliases: alias %q is invalid: must be non-empty with no control characters", alias) + } + if prefix, _, hasSlash := strings.Cut(alias, "/"); hasSlash && providerSet[prefix] { + return nil, fmt.Errorf("llmgateway: modelAliases: alias %q is invalid: its prefix %q names a configured provider and would be shadowed by provider-prefix resolution", alias, prefix) + } + if explicitModels[alias] { + return nil, fmt.Errorf("llmgateway: modelAliases: alias %q collides with an explicitly configured model id of the same name", alias) + } + if target == "" { + return nil, fmt.Errorf("llmgateway: modelAliases: alias %q has an empty target", alias) + } + if _, targetIsAlias := raw[target]; targetIsAlias { + return nil, fmt.Errorf("llmgateway: modelAliases: alias %q targets %q, which is itself an alias (alias chains are not allowed)", alias, target) + } + out[alias] = target + } + return out, nil +} + // now returns the registry's current time, via nowFn — overridable in // tests to exercise refresh throttling deterministically. func (m *modelRegistry) now() time.Time { @@ -350,6 +445,17 @@ func (m *modelRegistry) bareWinner(id string) (string, bool) { // exactly-as-requested id the caller passed in is not returned separately — // the caller already holds it, in the id argument itself. // +// An EXACT alias match (spec §5, v0.2) wins before any other rule below — +// checked first, ahead of provider-prefix splitting, so an alias id that +// happens to look like "prefix/rest" is never mistaken for one (and +// validateModelAliases already rejects an alias whose prefix names an +// actually configured provider at construction, so this ordering never +// has to arbitrate a genuine conflict between the two forms). id also +// happening to collide with a since-DISCOVERED model of some provider +// (impossible for an explicit one; validateModelAliases already rejects +// that at construction) is resolved in the alias's favor by this same +// ordering — warnAliasShadowsDiscoveredOnce logs that once per id. +// // id in "provider/model" form (ruling (a): only when "provider" names a // configured provider) resolves directly against that provider. Any other // id is a bare id, resolved against the first configured provider (sorted @@ -359,14 +465,89 @@ func (m *modelRegistry) bareWinner(id string) (string, bool) { // exists but fails authorization returns errModelDenied, distinct from // errModelUnknown for a model no configured provider knows at all. func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string, string, error) { + if target, isAlias := m.aliases[id]; isAlias { + if _, discoveredCollision := m.bareWinner(id); discoveredCollision { + m.warnAliasShadowsDiscoveredOnce(id) + } + return m.resolveAliasTarget(id, target, grp) + } if providerName, rest, ok := m.splitConfiguredProvider(id); ok { - return m.resolveAgainst(providerName, rest, id, grp) + return m.resolveAgainst(providerName, rest, id, "", grp, errModelUnknown) } providerName, ok := m.bareWinner(id) if !ok { return nil, "", "", errModelUnknown } - return m.resolveAgainst(providerName, id, id, grp) + return m.resolveAgainst(providerName, id, id, "", grp, errModelUnknown) +} + +// aliasTargetError is resolveAgainst's notFoundErr when the provider it +// was called against, and the upstream model id on it, both came from +// resolving an alias's TARGET (resolveAliasTarget) rather than a direct +// client request — spec §5's "Targets referencing not-yet-discovered +// models are allowed at construction and resolve lazily; an unresolvable +// target at request time → 404 whose message names the alias AND the +// missing target". writeModelResolveError (routes_unified.go) type- +// asserts for *aliasTargetError explicitly, ahead of its +// errors.Is(errModelUnknown) switch — a plain type assertion, not +// errors.As, matching this package's established yaegi-safe convention +// for a pointer error type (providers.go's providerHTTPError doc +// comment: errors.As panics under yaegi checking whether an interpreted +// pointer type implements error). +type aliasTargetError struct { + alias, target string +} + +// Error names both the alias and its unresolved target, per spec §5. +func (e *aliasTargetError) Error() string { + return fmt.Sprintf("llmgateway: alias %q targets %q, which is not a known model", e.alias, e.target) +} + +// resolveAliasTarget finishes resolve for id already known to be an exact +// alias match: target is resolved through the SAME rules as any other id +// — provider-prefixed direct, or bare-id collision winner — via the same +// resolveAgainst helper, with two differences from a direct request: +// +// - Authorization (spec §5): "a group may use an alias when its model +// globs match the ALIAS name OR the resolved target (either +// grants)". alias is passed to resolveAgainst as its extraModelName, +// so a request denied by every target-side candidate can still +// succeed purely because the group's model glob names the alias +// itself. +// - A target that does not resolve to any known model at request time +// returns *aliasTargetError (naming both ids), not the bare +// errModelUnknown a direct request's unknown-model case returns. +func (m *modelRegistry) resolveAliasTarget(alias, target string, grp *group) (providerAdapter, string, string, error) { + notFound := &aliasTargetError{alias: alias, target: target} + if providerName, rest, ok := m.splitConfiguredProvider(target); ok { + return m.resolveAgainst(providerName, rest, target, alias, grp, notFound) + } + providerName, ok := m.bareWinner(target) + if !ok { + return nil, "", "", notFound + } + return m.resolveAgainst(providerName, target, target, alias, grp, notFound) +} + +// warnAliasShadowsDiscoveredOnce logs, at most once per colliding alias id +// for this registry's lifetime, that alias also names a model some +// provider's discovery fetch found after construction — resolve's +// documented precedence (its own doc comment above): the alias always +// wins, since only an EXPLICIT collision is rejected at construction +// (validateModelAliases); discovery is dynamic and runs after that check, +// so this collision can only be caught, and only warned about, here at +// request time. Keyed with an "alias-shadow:" prefix in the same m.warned +// map warnCollisionOnce uses, so the two unrelated warning kinds can never +// share a key and suppress each other. +func (m *modelRegistry) warnAliasShadowsDiscoveredOnce(alias string) { + m.warnedMu.Lock() + defer m.warnedMu.Unlock() + key := "alias-shadow:" + alias + if m.warned[key] { + return + } + m.warned[key] = true + m.log("%s", fmt.Sprintf("model registry: alias %q also names a discovered model; the alias takes precedence", alias)) } // resolveAgainst finishes resolve for a providerName already chosen (either @@ -386,11 +567,29 @@ func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string, // legitimately provider-prefixed in the first place (ruling: a pattern like // "deepseek-*" must not match a bare id that merely contains a slash, e.g. // "uni/deepseek-v4-flash-0731", when "uni" is not a configured provider). -func (m *modelRegistry) resolveAgainst(providerName, upstreamModel, requestedID string, grp *group) (providerAdapter, string, string, error) { +// +// extraModelName is a further authorization candidate checked via +// grp.allowsModel, alongside requestedID and upstreamModel: empty ("") for +// a direct (non-alias) resolution, or an alias's own id when resolveAliasTarget +// (spec §5, v0.2) is resolving that alias's target — "a group may use an +// alias when its model globs match the ALIAS name OR the resolved target +// (either grants)". +// +// notFoundErr is returned, unmodified, when upstreamModel is not known to +// providerName: resolve's own two call sites pass the bare errModelUnknown +// sentinel (unchanged from before this parameter existed — checked by "err +// != errModelUnknown" equality in registry_test.go, not errors.Is, so it +// must stay the exact sentinel value there); resolveAliasTarget passes a +// *aliasTargetError naming both the alias and its unresolved target (spec +// §5's 404 message requirement). +func (m *modelRegistry) resolveAgainst(providerName, upstreamModel, requestedID, extraModelName string, grp *group, notFoundErr error) (providerAdapter, string, string, error) { if !m.states[providerName].hasModel(upstreamModel) { - return nil, "", "", errModelUnknown + return nil, "", "", notFoundErr } allowed := grp.allowsModel(requestedID) || grp.allowsModel(upstreamModel) + if extraModelName != "" { + allowed = allowed || grp.allowsModel(extraModelName) + } if !allowed || !grp.allowsProvider(providerName) { return nil, "", "", errModelDenied } @@ -459,6 +658,33 @@ func (m *modelRegistry) listFor(grp *group) []map[string]any { } } + // Aliases (spec §5, v0.2): listed alongside real models only when + // both their target actually resolves right now (a target awaiting + // its provider's first discovery fetch is not listed — there is + // nothing yet to name as owned_by) and grp is authorized for it. + // resolveAliasTarget applies the exact same "alias name OR target" + // authorization rule request-time resolve uses (its own doc + // comment), so this listing can never promise access resolve would + // then deny. owned_by is recovered from the successful call's own + // canonical return value ("provider/upstreamModel") by splitting on + // the first "/" — exact, because a registry provider name can never + // itself contain one (configNamePattern, providers.go). An alias id + // that also happens to collide with a since-discovered model's own + // bare id (impossible for an EXPLICIT model — validateModelAliases + // already rejects that at construction) is not deduped against that + // model's own entry above: the same documented, alias-wins + // precedence edge resolve's warnAliasShadowsDiscoveredOnce logs + // about, accepted here too as rare enough not to warrant extra + // bookkeeping. + for alias, target := range m.aliases { + _, _, canonical, err := m.resolveAliasTarget(alias, target, grp) + if err != nil { + continue + } + providerName, _, _ := strings.Cut(canonical, "/") + out = append(out, modelObject(alias, providerName)) + } + sort.Slice(out, func(i, j int) bool { return out[i]["id"].(string) < out[j]["id"].(string) //nolint:forcetypeassert // modelObject always sets id to a string }) @@ -484,6 +710,33 @@ type providerSnapshot struct { modelCount int } +// aliasSnapshotEntry is one configured alias's read-only view for the +// admin dashboard (spec §5, v0.2): the pair exactly as an operator wrote +// it. No secrets involved. The target's actual resolved provider is +// deliberately not included — a target still awaiting its provider's +// first discovery fetch has none yet, and the configured pair is the +// meaningful, stable fact an operator wants to see either way. +type aliasSnapshotEntry struct { + Alias string + Target string +} + +// aliasSnapshot returns every configured alias's alias->target pair, +// sorted by alias — the admin dashboard's alias table (spec §5, v0.2). +func (m *modelRegistry) aliasSnapshot() []aliasSnapshotEntry { + names := make([]string, 0, len(m.aliases)) + for alias := range m.aliases { + names = append(names, alias) + } + sort.Strings(names) + + out := make([]aliasSnapshotEntry, len(names)) + for i, alias := range names { + out[i] = aliasSnapshotEntry{Alias: alias, Target: m.aliases[alias]} + } + return out +} + // snapshot returns every configured provider's read-only view, sorted by // name (m.providerNames is already sorted at construction) — the admin // dashboard's provider table (spec §4, v0.2). diff --git a/registry_test.go b/registry_test.go index a01f936..2f7117f 100644 --- a/registry_test.go +++ b/registry_test.go @@ -3,9 +3,11 @@ package traefikllmgateway import ( "context" "encoding/json" + "errors" "fmt" "net/http" "net/http/httptest" + "strings" "sync" "sync/atomic" "testing" @@ -893,3 +895,351 @@ func TestNewGateway_WarmFill_PopulatesDiscoveredModels_BeforeAnyRequest(t *testi t.Fatal("want disco-model known immediately after New returns (synchronous warm fill)") } } + +// --- model aliases (spec §5, v0.2) --- + +// newAliasTestRegistry builds a registry with one "openai" provider +// (explicit model "gpt-test") and grp's given aliases, failing the test on +// any construction error. log defaults to a no-op when nil. +func newAliasTestRegistry(t *testing.T, aliases map[string]string, log func(string, ...any)) *modelRegistry { + t.Helper() + if log == nil { + log = func(string, ...any) {} + } + adapters := map[string]providerAdapter{"openai": newFakeAdapter("openai")} + cfg := &Config{ + Providers: map[string]*ProviderConfig{"openai": {Models: []string{"gpt-test"}}}, + ModelAliases: aliases, + } + reg, err := newModelRegistry(adapters, cfg, log) + if err != nil { + t.Fatalf("newModelRegistry: %v", err) + } + return reg +} + +// TestNewModelRegistry_ModelAliases_ValidationTable covers every row of +// spec §5's validation table, plus a positive control. +func TestNewModelRegistry_ModelAliases_ValidationTable(t *testing.T) { + t.Parallel() + baseProviders := map[string]*ProviderConfig{"openai": {Models: []string{"gpt-test"}}} + adapters := map[string]providerAdapter{"openai": newFakeAdapter("openai")} + + tests := []struct { + aliases map[string]string + name string + wantErrText string // substring expected in the error; "" means construction must succeed + }{ + { + name: "empty alias id", + aliases: map[string]string{"": "gpt-test"}, + wantErrText: "invalid", + }, + { + name: "control character in alias id", + aliases: map[string]string{"bad\x7falias": "gpt-test"}, + wantErrText: "invalid", + }, + { + name: "alias prefix shadows a configured provider", + aliases: map[string]string{"openai/special": "gpt-test"}, + wantErrText: "shadowed", + }, + { + name: "alias collides with an explicit model id", + aliases: map[string]string{"gpt-test": "openai/gpt-test"}, + wantErrText: "collides", + }, + { + name: "alias chain: target is itself an alias", + aliases: map[string]string{"aliased/a": "aliased/b", "aliased/b": "openai/gpt-test"}, + wantErrText: "itself an alias", + }, + { + name: "empty target", + aliases: map[string]string{"aliased/x": ""}, + wantErrText: "empty target", + }, + { + name: "valid: bare target, no prefix collision", + aliases: map[string]string{"aliased/coding": "gpt-test"}, + }, + { + name: "valid: provider-prefixed target, unconfigured prefix on the alias itself", + aliases: map[string]string{"team/coding": "openai/gpt-test"}, + }, + { + name: "valid: target not yet known (lazy resolution, allowed at construction)", + aliases: map[string]string{"aliased/future": "openai/not-yet-discovered"}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + cfg := &Config{Providers: baseProviders, ModelAliases: tt.aliases} + _, err := newModelRegistry(adapters, cfg, func(string, ...any) {}) + if tt.wantErrText == "" { + if err != nil { + t.Fatalf("newModelRegistry: %v, want success", err) + } + return + } + if err == nil { + t.Fatal("newModelRegistry: want error, got nil") + } + if !strings.Contains(err.Error(), tt.wantErrText) { + t.Errorf("err = %v, want substring %q", err, tt.wantErrText) + } + }) + } +} + +// TestModelRegistry_Resolve_Alias_SlashLikeID_NotMistakenForProviderPrefix +// is the resolve-precedence regression from spec §5: an alias id +// containing "/" whose prefix names NO configured provider ("team" is not +// configured here — a prefix that IS configured is rejected at +// construction, so this is the only shape available to prove the alias +// map is checked, and wins, ahead of splitConfiguredProvider/bareWinner) +// must still resolve through the alias table, not fall through to +// errModelUnknown. +func TestModelRegistry_Resolve_Alias_SlashLikeID_NotMistakenForProviderPrefix(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"team/coding": "openai/gpt-test"}, nil) + + adapter, upstreamModel, canonical, err := reg.resolve("team/coding", allowAllGroup()) + if err != nil { + t.Fatalf("resolve: %v", err) + } + if adapter != reg.adapters["openai"] { + t.Errorf("adapter = %v, want openai adapter", adapter) + } + if upstreamModel != "gpt-test" { + t.Errorf("upstreamModel = %q, want %q", upstreamModel, "gpt-test") + } + if canonical != "openai/gpt-test" { + t.Errorf("canonical = %q, want %q", canonical, "openai/gpt-test") + } +} + +// TestModelRegistry_Resolve_Alias_BareTarget_ResolvesViaCollisionWinner +// proves resolveAliasTarget's bareWinner branch (a target with no +// provider prefix) applies the identical sorted-first collision rule a +// direct bare-id request already gets (ruling g). +func TestModelRegistry_Resolve_Alias_BareTarget_ResolvesViaCollisionWinner(t *testing.T) { + t.Parallel() + adapters := map[string]providerAdapter{ + "beta": newFakeAdapter("beta"), + "alpha": newFakeAdapter("alpha"), + } + cfg := &Config{ + Providers: map[string]*ProviderConfig{ + "beta": {Models: []string{"shared"}}, + "alpha": {Models: []string{"shared"}}, + }, + ModelAliases: map[string]string{"aliased/x": "shared"}, + } + reg, err := newModelRegistry(adapters, cfg, func(string, ...any) {}) + if err != nil { + t.Fatalf("newModelRegistry: %v", err) + } + + adapter, _, canonical, err := reg.resolve("aliased/x", allowAllGroup()) + if err != nil { + t.Fatalf("resolve: %v", err) + } + if adapter != adapters["alpha"] { + t.Errorf("adapter = %v, want alpha (sorted-first collision winner)", adapter) + } + if canonical != "alpha/shared" { + t.Errorf("canonical = %q, want %q", canonical, "alpha/shared") + } +} + +// --- resolve: alias authorization (spec §5's "alias name OR target, +// either grants") --- + +func TestModelRegistry_Resolve_Alias_AuthorizedViaAliasNameOnly(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil) + + // Matches the alias name; matches neither "openai/gpt-test" nor its + // bare suffix "gpt-test". + grp := &group{name: "narrow", models: []string{"aliased/coding"}} + if _, _, _, err := reg.resolve("aliased/coding", grp); err != nil { + t.Errorf("resolve: %v, want success (authorized via the alias name alone)", err) + } +} + +func TestModelRegistry_Resolve_Alias_AuthorizedViaTargetOnly(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil) + + // Matches the target's bare suffix; does not match the alias name. + grp := &group{name: "narrow", models: []string{"gpt-*"}} + if _, _, _, err := reg.resolve("aliased/coding", grp); err != nil { + t.Errorf("resolve: %v, want success (authorized via the resolved target alone)", err) + } +} + +func TestModelRegistry_Resolve_Alias_DeniedWhenNeitherAliasNorTargetMatch(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil) + + grp := &group{name: "narrow", models: []string{"claude-*"}} + if _, _, _, err := reg.resolve("aliased/coding", grp); err != errModelDenied { + t.Errorf("err = %v, want errModelDenied", err) + } +} + +func TestModelRegistry_Resolve_Alias_ProviderDenied_OverridesModelAuthz(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil) + + // The model glob would allow it via the alias name, but the group's + // providers list excludes "openai" entirely — allowsProvider still + // gates the resolved target's provider (spec §5's "allowsProvider + // still applies to the target's provider"). + grp := &group{name: "no-openai", providers: []string{"anthropic"}, models: []string{"aliased/coding"}} + if _, _, _, err := reg.resolve("aliased/coding", grp); err != errModelDenied { + t.Errorf("err = %v, want errModelDenied (provider gate must still apply)", err) + } +} + +// TestModelRegistry_Resolve_Alias_UnresolvedTarget_ReturnsAliasTargetError +// is spec §5's lazy-resolution 404: a target naming no currently-known +// model returns *aliasTargetError, whose message names both the alias and +// the missing target. +func TestModelRegistry_Resolve_Alias_UnresolvedTarget_ReturnsAliasTargetError(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil) + + _, _, _, err := reg.resolve("aliased/future", allowAllGroup()) + var aerr *aliasTargetError + if !errors.As(err, &aerr) { + t.Fatalf("err = %v (%T), want *aliasTargetError", err, err) + } + if !strings.Contains(aerr.Error(), "aliased/future") { + t.Errorf("error message = %q, want it to name the alias %q", aerr.Error(), "aliased/future") + } + if !strings.Contains(aerr.Error(), "openai/not-yet-discovered") { + t.Errorf("error message = %q, want it to name the target %q", aerr.Error(), "openai/not-yet-discovered") + } +} + +// TestModelRegistry_Resolve_Alias_TargetBecomesKnown_ResolvesAfterDiscovery +// proves the lazy side of "resolve lazily": a target that was unresolvable +// at construction succeeds once its provider's discovered set catches up +// (no re-validation needed — discovery mutates providerState directly). +func TestModelRegistry_Resolve_Alias_TargetBecomesKnown_ResolvesAfterDiscovery(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil) + + if _, _, _, err := reg.resolve("aliased/future", allowAllGroup()); err == nil { + t.Fatal("resolve before discovery: want an error, got nil") + } + + reg.states["openai"].finishRefresh(reg.now(), []string{"not-yet-discovered"}, nil) + + adapter, upstreamModel, canonical, err := reg.resolve("aliased/future", allowAllGroup()) + if err != nil { + t.Fatalf("resolve after discovery: %v, want success", err) + } + if adapter != reg.adapters["openai"] || upstreamModel != "not-yet-discovered" || canonical != "openai/not-yet-discovered" { + t.Errorf("resolve after discovery = (%v, %q, %q), want (openai adapter, %q, %q)", adapter, upstreamModel, canonical, "not-yet-discovered", "openai/not-yet-discovered") + } +} + +// TestModelRegistry_Resolve_Alias_PrecedesDiscoveredCollision_WarnsOnce +// covers the documented precedence edge (registry.go's resolve doc +// comment): an alias id that also happens to name a model discovered +// AFTER construction (impossible for an explicit model — that is a +// construction error) still resolves to the ALIAS's own target, and the +// gateway logs exactly one warning about the shadowed id, regardless of +// how many times resolve is called for it. +func TestModelRegistry_Resolve_Alias_PrecedesDiscoveredCollision_WarnsOnce(t *testing.T) { + t.Parallel() + adapters := map[string]providerAdapter{ + "openai": newFakeAdapter("openai"), + "acme": newFakeAdapter("acme"), + } + cfg := &Config{ + Providers: map[string]*ProviderConfig{ + "openai": {Models: []string{"gpt-test"}}, + "acme": {Discovery: true}, + }, + ModelAliases: map[string]string{"shared-id": "openai/gpt-test"}, + } + rl := &recordingLog{} + reg, err := newModelRegistry(adapters, cfg, rl.fn) + if err != nil { + t.Fatalf("newModelRegistry: %v", err) + } + // acme's discovery finds a model literally named "shared-id" — the + // same string as the configured alias, discovered only after + // construction, so validateModelAliases never saw it. + reg.states["acme"].finishRefresh(reg.now(), []string{"shared-id"}, nil) + + adapter, _, canonical, err := reg.resolve("shared-id", allowAllGroup()) + if err != nil { + t.Fatalf("resolve: %v", err) + } + if adapter != adapters["openai"] { + t.Errorf("adapter = %v, want openai (the alias, not acme's discovered model, must win)", adapter) + } + if canonical != "openai/gpt-test" { + t.Errorf("canonical = %q, want %q", canonical, "openai/gpt-test") + } + + if _, _, _, err := reg.resolve("shared-id", allowAllGroup()); err != nil { + t.Fatalf("second resolve: %v", err) + } + if got := rl.count(); got != 1 { + t.Errorf("collision-shadow log emitted %d times across 2 resolve calls, want exactly 1", got) + } +} + +// --- listFor: alias visibility (spec §5) --- + +func TestModelRegistry_ListFor_Alias_ListedWhenAuthorized_OwnedByTargetProvider(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil) + + got := reg.listFor(allowAllGroup()) + var found bool + for _, entry := range got { + if entry["id"] == "aliased/coding" { + found = true + if entry["owned_by"] != "openai" { + t.Errorf("aliased/coding owned_by = %v, want %q", entry["owned_by"], "openai") + } + } + } + if !found { + t.Errorf("listFor = %v, want an entry for the alias %q", got, "aliased/coding") + } +} + +func TestModelRegistry_ListFor_Alias_UnresolvedTarget_Omitted(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil) + + got := reg.listFor(allowAllGroup()) + for _, entry := range got { + if entry["id"] == "aliased/future" { + t.Errorf("listFor lists %q whose target does not resolve yet, want it omitted", "aliased/future") + } + } +} + +func TestModelRegistry_ListFor_Alias_NotAuthorized_Omitted(t *testing.T) { + t.Parallel() + reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil) + + grp := &group{name: "narrow", models: []string{"claude-*"}} + got := reg.listFor(grp) + for _, entry := range got { + if entry["id"] == "aliased/coding" { + t.Errorf("listFor lists %q for a group authorized for neither the alias nor its target", "aliased/coding") + } + } +} diff --git a/routes_media_test.go b/routes_media_test.go index 0890e42..4818c06 100644 --- a/routes_media_test.go +++ b/routes_media_test.go @@ -166,6 +166,38 @@ func TestHandleImagesGenerations_OpenAI_HappyPath_NativeForward(t *testing.T) { } } +// TestHandleImagesGenerations_ModelAlias_ResolvesToTargetUpstreamModel +// proves an operator-defined modelAliases entry (spec §5, v0.2) routes +// /v1/images/generations exactly like a bare or provider-prefixed id: the +// upstream call carries the target's own upstream model id, never the +// alias. +func TestHandleImagesGenerations_ModelAlias_ResolvesToTargetUpstreamModel(t *testing.T) { + var gotBody map[string]any + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _ = json.NewDecoder(r.Body).Decode(&gotBody) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{"created":1,"data":[{"b64_json":"xyz"}]}`)) + })) + defer srv.Close() + + cfg := newMediaTestConfig(srv.URL, "img-test") + cfg.ModelAliases = map[string]string{"aliased/image": "openai/img-test"} + gw := newMediaTestGateway(t, cfg) + + body := map[string]any{"model": "aliased/image", "prompt": "a cat", "n": 1} + req := newUnifiedRequest(t, http.MethodPost, imagesGenerationsPath, "sk-alice", body) + rec := httptest.NewRecorder() + gw.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String()) + } + if gotBody["model"] != "img-test" { + t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", gotBody["model"], "img-test") + } +} + // TestHandleImagesGenerations_StripsClientSuppliedAliasKey proves a // client independently including a literal "__alias" field in its own // JSON body (never one the gateway itself injects for a media route — @@ -726,6 +758,100 @@ func TestHandleAudioTranscriptions_PrefixedAlias_RewritesModelFieldOnly(t *testi } } +// TestHandleAudioTranscriptions_ModelAlias_RewritesModelFieldToTargetUpstreamID +// mirrors TestHandleAudioTranscriptions_PrefixedAlias_RewritesModelFieldOnly +// above, but through an operator-defined modelAliases entry (spec §5, +// v0.2) rather than a bare provider-prefixed id: the client's "model" +// form field ("aliased/whisper") differs from the resolved upstream model +// id ("whisper-test"), so rewriteMultipartModel (routes_media.go) must +// rebuild the multipart body — every part copied verbatim except "model", +// which becomes the target's own upstream id. The alias must never reach +// the real provider, the same invariant every other unified route +// already enforces. +func TestHandleAudioTranscriptions_ModelAlias_RewritesModelFieldToTargetUpstreamID(t *testing.T) { + const fileContent = "FAKEAUDIOBYTES-DISTINCTIVE-PAYLOAD" + var gotContentType string + var gotBody []byte + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotContentType = r.Header.Get("Content-Type") + gotBody, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{"text":"ok"}`)) + })) + defer srv.Close() + + cfg := newMediaTestConfig(srv.URL, "whisper-test") + cfg.ModelAliases = map[string]string{"aliased/whisper": "openai/whisper-test"} + gw := newMediaTestGateway(t, cfg) + + var buf bytes.Buffer + mw := multipart.NewWriter(&buf) + ffw, err := mw.CreateFormFile("file", "audio.wav") + if err != nil { + t.Fatalf("CreateFormFile: %v", err) + } + if _, err = ffw.Write([]byte(fileContent)); err != nil { + t.Fatalf("write file field: %v", err) + } + mfw, err := mw.CreateFormField("model") + if err != nil { + t.Fatalf("CreateFormField(model): %v", err) + } + if _, err = mfw.Write([]byte("aliased/whisper")); err != nil { + t.Fatalf("write model field: %v", err) + } + if err = mw.Close(); err != nil { + t.Fatalf("close multipart writer: %v", err) + } + + req := httptest.NewRequest(http.MethodPost, audioTranscriptionsPath, &buf) + req.Header.Set("Content-Type", mw.FormDataContentType()) + req.Header.Set("Authorization", "Bearer sk-alice") + rec := httptest.NewRecorder() + gw.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String()) + } + + _, params, err := mime.ParseMediaType(gotContentType) + if err != nil { + t.Fatalf("upstream Content-Type %q did not parse: %v", gotContentType, err) + } + mr := multipart.NewReader(bytes.NewReader(gotBody), params["boundary"]) + var gotModel string + var gotFile []byte + var sawModel, sawFile bool + for { + part, perr := mr.NextPart() + if perr == io.EOF { //nolint:errorlint // multipart.Reader.NextPart's own contract returns io.EOF bare + break + } + if perr != nil { + t.Fatalf("parse upstream multipart body: %v", perr) + } + switch part.FormName() { + case "model": + b, _ := io.ReadAll(part) + gotModel = string(b) + sawModel = true + case "file": + b, _ := io.ReadAll(part) + gotFile = b + sawFile = true + } + _ = part.Close() + } + + if !sawModel || gotModel != "whisper-test" { + t.Errorf("upstream model field = %q (present=%v), want the alias target's bare upstream id %q", gotModel, sawModel, "whisper-test") + } + if !sawFile || string(gotFile) != fileContent { + t.Errorf("upstream file part = %q (present=%v), want unchanged %q", gotFile, sawFile, fileContent) + } +} + // TestHandleAudioTranscriptions_OversizeBody_413 proves a multipart body // above maxRequestBytes is rejected with 413 (spec §3), the one media // route with an explicit oversize check — unlike images/speech's shared diff --git a/routes_unified.go b/routes_unified.go index 93634c6..27724dc 100644 --- a/routes_unified.go +++ b/routes_unified.go @@ -255,10 +255,19 @@ func unifiedCostMicros(canonical, bare string, u usage, overrides map[string]*Mo // writeModelResolveError maps a modelRegistry.resolve error to its HTTP // envelope: errModelUnknown is a 404 (no configured provider knows this // model), errModelDenied is a 403 (a real model the caller's group cannot -// use). Any other error is a defensive 500 — resolve's own contract -// promises only these two sentinels, so reaching this branch would be a -// programming error, not a client mistake. +// use). A *aliasTargetError (spec §5, v0.2) is also a 404, but with its +// own message naming both the alias and its unresolved target, checked +// first via a plain type assertion — not errors.As, matching this +// package's established yaegi-safe convention for a pointer error type +// (registry.go's aliasTargetError doc comment). Any other error is a +// defensive 500 — resolve's own contract promises only these sentinels/ +// types, so reaching this branch would be a programming error, not a +// client mistake. func writeModelResolveError(w http.ResponseWriter, err error) { + if aerr, ok := err.(*aliasTargetError); ok { + writeOAIError(w, http.StatusNotFound, "invalid_request_error", aerr.Error()) + return + } switch { case errors.Is(err, errModelUnknown): writeOAIError(w, http.StatusNotFound, "invalid_request_error", "unknown model") diff --git a/routes_unified_test.go b/routes_unified_test.go index 2e28ffc..a7aca85 100644 --- a/routes_unified_test.go +++ b/routes_unified_test.go @@ -533,6 +533,245 @@ func TestHandleChat_AnthropicProviderPrefixedModel_EchoesAliasInResponse(t *test } } +// --- model aliases end-to-end (spec §5, v0.2) --- + +// TestHandleChat_ModelAlias_AnthropicTarget_EchoesAliasInResponse mirrors +// TestHandleChat_AnthropicProviderPrefixedModel_EchoesAliasInResponse +// above, but through an operator-defined modelAliases entry rather than a +// bare provider-prefixed id: the upstream call uses the target's own +// upstream model id, and the translated response's "model" field echoes +// the client's exact alias string back (ruling a, ALIAS ECHO — aliases +// inherit the existing __alias machinery for free, per registry.go's +// resolve doc comment). +func TestHandleChat_ModelAlias_AnthropicTarget_EchoesAliasInResponse(t *testing.T) { + const anthResp = `{"id":"msg_01ABC","type":"message","role":"assistant","content":[{"type":"text","text":"hi there"}],"model":"claude-x","stop_reason":"end_turn","usage":{"input_tokens":10,"output_tokens":4}}` + + var gotBody []byte + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotBody, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(anthResp)) + })) + defer srv.Close() + + cfg := CreateConfig() + cfg.Providers = map[string]*ProviderConfig{ + "anthropic": {Type: "anthropic", BaseURL: srv.URL, APIKey: "sk-ant", Models: []string{"claude-x"}}, + } + cfg.ModelAliases = map[string]string{"aliased/coding": "anthropic/claude-x"} + cfg.Groups = map[string]*GroupConfig{"default": {}} + cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}} + next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}) + h, err := New(context.Background(), next, cfg, "llmgw") + if err != nil { + t.Fatalf("New: %v", err) + } + + const requestedID = "aliased/coding" + body := map[string]any{"model": requestedID, "messages": []any{map[string]any{"role": "user", "content": "hi"}}} + req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String()) + } + + var out map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil { + t.Fatalf("decode response: %v", err) + } + if out["model"] != requestedID { + t.Errorf("response model = %v, want the client-requested alias %q echoed back", out["model"], requestedID) + } + + sentUpstream := decodeJSONBody(t, gotBody) + if sentUpstream["model"] != "claude-x" { + t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", sentUpstream["model"], "claude-x") + } + if _, present := sentUpstream[gatewayAliasKey]; present { + t.Errorf("gatewayAliasKey %q leaked into the upstream request body: %v", gatewayAliasKey, sentUpstream) + } +} + +// TestHandleChat_ModelAlias_OpenAITarget_UpstreamEchoNotAlias documents the +// v0.1 passthrough asymmetry (routes_unified.go's gatewayAliasKey doc +// comment) as it applies to aliases: an openai-type target's response is +// forwarded verbatim, so its own "model" field carries whatever the mock +// upstream itself returned — the target's bare upstream id, NOT the +// alias. The gateway makes no attempt, and is not expected, to rewrite an +// openai-type body in flight. +func TestHandleChat_ModelAlias_OpenAITarget_UpstreamEchoNotAlias(t *testing.T) { + const respBody = `{"id":"c1","model":"gpt-test","choices":[{"message":{"role":"assistant","content":"hi"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}` + + var gotBody []byte + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotBody, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(respBody)) + })) + defer srv.Close() + + cfg := CreateConfig() + cfg.Providers = map[string]*ProviderConfig{ + "openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"gpt-test"}}, + } + cfg.ModelAliases = map[string]string{"aliased/fast": "openai/gpt-test"} + cfg.Groups = map[string]*GroupConfig{"default": {}} + cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}} + next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}) + h, err := New(context.Background(), next, cfg, "llmgw") + if err != nil { + t.Fatalf("New: %v", err) + } + + body := map[string]any{"model": "aliased/fast", "messages": []any{}} + req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String()) + } + var out map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil { + t.Fatalf("decode response: %v", err) + } + if out["model"] != "gpt-test" { + t.Errorf("response model = %v, want the openai-type passthrough's own upstream id %q, not the alias", out["model"], "gpt-test") + } + + sentUpstream := decodeJSONBody(t, gotBody) + if sentUpstream["model"] != "gpt-test" { + t.Errorf("upstream request model = %v, want bare upstream id %q", sentUpstream["model"], "gpt-test") + } +} + +// TestHandleEmbeddings_ModelAlias_ResolvesToTargetUpstreamModel proves +// aliasing applies to /v1/embeddings exactly like /v1/chat/completions — +// the upstream call uses the target's bare upstream model id. +func TestHandleEmbeddings_ModelAlias_ResolvesToTargetUpstreamModel(t *testing.T) { + const respBody = `{"object":"list","data":[{"embedding":[0.1]}],"usage":{"prompt_tokens":1,"total_tokens":1}}` + + var gotBody []byte + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotBody, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(respBody)) + })) + defer srv.Close() + + cfg := CreateConfig() + cfg.Providers = map[string]*ProviderConfig{ + "openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"text-embedding-3"}}, + } + cfg.ModelAliases = map[string]string{"aliased/embed": "openai/text-embedding-3"} + cfg.Groups = map[string]*GroupConfig{"default": {}} + cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}} + next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}) + h, err := New(context.Background(), next, cfg, "llmgw") + if err != nil { + t.Fatalf("New: %v", err) + } + + body := map[string]any{"model": "aliased/embed", "input": "hello"} + req := newUnifiedRequest(t, http.MethodPost, "/v1/embeddings", "sk-alice", body) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String()) + } + sentUpstream := decodeJSONBody(t, gotBody) + if sentUpstream["model"] != "text-embedding-3" { + t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", sentUpstream["model"], "text-embedding-3") + } +} + +// TestHandleChat_ModelAlias_AuthorizedViaAliasNameAlone_EndToEnd proves +// spec §5's authorization rule end-to-end: a group whose models glob +// matches only the ALIAS name (not the target, in either form) can still +// use it through the real ServeHTTP pipeline, not just modelRegistry.resolve +// in isolation. +func TestHandleChat_ModelAlias_AuthorizedViaAliasNameAlone_EndToEnd(t *testing.T) { + const respBody = `{"id":"c1","model":"gpt-test","choices":[{"message":{"role":"assistant","content":"hi"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(respBody)) + })) + defer srv.Close() + + cfg := CreateConfig() + cfg.Providers = map[string]*ProviderConfig{ + "openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"gpt-test"}}, + } + cfg.ModelAliases = map[string]string{"aliased/coding": "openai/gpt-test"} + // Matches only the alias name, not "openai/gpt-test" nor "gpt-test". + cfg.Groups = map[string]*GroupConfig{"narrow": {Models: []string{"aliased/coding"}}} + cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "narrow", APIKey: "sk-alice"}}} + next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}) + h, err := New(context.Background(), next, cfg, "llmgw") + if err != nil { + t.Fatalf("New: %v", err) + } + + body := map[string]any{"model": "aliased/coding", "messages": []any{}} + req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200 (authorized via the alias name alone), body=%s", rec.Code, rec.Body.String()) + } +} + +// TestHandleChat_ModelAlias_UnresolvedTarget_Returns404NamingAliasAndTarget +// covers spec §5's lazy-resolution 404 end-to-end: an alias whose target +// names no currently-known model returns 404 with a message naming both +// the alias and the missing target — not the generic "unknown model" text +// a plain unresolved bare/prefixed id gets. +func TestHandleChat_ModelAlias_UnresolvedTarget_Returns404NamingAliasAndTarget(t *testing.T) { + cfg := CreateConfig() + cfg.Providers = map[string]*ProviderConfig{ + "openai": {Type: "openai", APIKey: "k", Models: []string{"gpt-test"}}, + } + cfg.ModelAliases = map[string]string{"aliased/future": "openai/not-yet-discovered"} + cfg.Groups = map[string]*GroupConfig{"default": {}} + cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}} + next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}) + h, err := New(context.Background(), next, cfg, "llmgw") + if err != nil { + t.Fatalf("New: %v", err) + } + + body := map[string]any{"model": "aliased/future", "messages": []any{}} + req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + + if rec.Code != http.StatusNotFound { + t.Fatalf("status = %d, want 404, body=%s", rec.Code, rec.Body.String()) + } + var out struct { + Error struct { + Message string `json:"message"` + } `json:"error"` + } + if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil { + t.Fatalf("decode response: %v", err) + } + if !strings.Contains(out.Error.Message, "aliased/future") { + t.Errorf("error message = %q, want it to name the alias %q", out.Error.Message, "aliased/future") + } + if !strings.Contains(out.Error.Message, "openai/not-yet-discovered") { + t.Errorf("error message = %q, want it to name the target %q", out.Error.Message, "openai/not-yet-discovered") + } +} + // TestHandleChat_UpstreamNon2xx_WrapsProviderErrorEnvelope asserts a // non-2xx upstream response is passed through with its status code, but // wrapped in the gateway's own envelope with the raw upstream body