mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
feat: operator-defined model aliases
Spec §5 (v0.2): Config.ModelAliases map[string]string, alias-first resolution precedence ahead of provider-prefix splitting, alias-OR-target authorization, lazy target resolution with a 404 naming both ids, /v1/models listing, admin overview alias table (JSON + dashboard), and construction-time validation (shape, prefix-shadow, explicit-model collision, chain, empty target). routes_unified.go/routes_media.go needed no resolution-call-site changes - both already pass the client's raw requested id through to registry.resolve, so aliases inherit the __alias echo and multipart rewrite machinery for free. cache.go needed no changes either - cacheKey already hashes requestedModel, so two aliases resolving to the same target already produce distinct cache entries. Adds registry_test.go validation table + precedence/authz/lazy-404/ listing coverage, a dedicated cache-key split test, chat/embeddings/ images/audio-transcriptions end-to-end tests, an extended admin overview test, and an integration test covering the openai-type vs anthropic-type echo asymmetry under real Traefik+Yaegi.
This commit is contained in:
@@ -170,6 +170,7 @@ config accepts them as YAML, which decodes to the same JSON shape.
|
||||
| `retry` | `RetryConfig` | `{}` (disabled) | Same-provider retry for transient upstream failures — see [Retry](#retry). Omitted or `enabled: false` means no retry: every request makes exactly one upstream attempt, byte-identical to a gateway built before this field existed. |
|
||||
| `cache` | `CacheConfig` | `{}` (disabled) | Opt-in Redis-backed response cache for unified non-streaming chat/embeddings — see [Caching](#caching). Omitted or `enabled: false` means no caching, byte-identical to a gateway built before this field existed. |
|
||||
| `admin` | `*AdminConfig` | `nil` (disabled) | Read-only admin dashboard — see [Admin](#admin). `nil` or `enabled: false` means the `/admin*` routes are not registered at all. |
|
||||
| `modelAliases` | `map[string]string` | `{}` | Operator-defined alias id → target model id — see [Model aliases](#model-aliases). Omitted or empty means no aliases, byte-identical to a gateway built before this field existed. |
|
||||
| `passthroughUnknown` | `bool` | `false` | `false`: a request matching none of the plugin's routes gets a 404 JSON envelope. `true`: it falls through to the router's own backing service. |
|
||||
|
||||
Provider, `mcpServers`, and `agents` map **keys** (names) must match
|
||||
@@ -359,6 +360,85 @@ kept — a bad edit to the file never breaks already-authenticated traffic.
|
||||
**stale-while-error**, never an empty list just because one refresh
|
||||
attempt failed.
|
||||
|
||||
## Model aliases
|
||||
|
||||
Operator-defined indirection: a client-facing alias id that resolves to a
|
||||
real target model id. `modelAliases` is empty by default, and an empty map
|
||||
changes nothing about model resolution — this is purely additive.
|
||||
|
||||
```yaml
|
||||
http:
|
||||
middlewares:
|
||||
llmgateway:
|
||||
plugin:
|
||||
llmgateway:
|
||||
# ... providers, groups, users unchanged ...
|
||||
modelAliases:
|
||||
# Client-facing name -> real target. The target can itself be
|
||||
# bare ("gpt-5-mini") or provider-prefixed
|
||||
# ("anthropic/claude-sonnet-4-5"); either resolves through the
|
||||
# normal rules in [Model routing](#model-routing) above.
|
||||
aliased/coding: "anthropic/claude-sonnet-4-5"
|
||||
aliased/fast: "gpt-5-mini"
|
||||
```
|
||||
|
||||
- **Swapping the underlying model**: point an alias at a different target
|
||||
and let Traefik's dynamic-config reload apply it — no gateway restart,
|
||||
no client-side change. A client that has always addressed
|
||||
`aliased/coding` keeps working unmodified after the operator moves that
|
||||
alias to a newer model.
|
||||
- **Precedence**: an EXACT alias match wins resolution before any other
|
||||
rule, including `provider/model` prefix splitting — checked first in
|
||||
`modelRegistry.resolve`. The target then resolves through the same
|
||||
rules as any other id (bare, `provider/model`, or a discovered id).
|
||||
Because of this, an alias whose id contains a `/` is rejected at
|
||||
construction if the part before the `/` names an actually configured
|
||||
provider (it would be shadowed by prefix resolution) — see the
|
||||
validation rules below.
|
||||
- **Authorization**: a group may use an alias when its `models` glob
|
||||
matches the **alias name itself**, OR when it already grants access to
|
||||
the **resolved target** — either is enough. `providers` still gates the
|
||||
target's own provider as usual.
|
||||
- **Validation at construction** (a bad alias config fails the same way a
|
||||
bad provider config does — no gateway starts with a broken alias
|
||||
table):
|
||||
- the alias must be non-empty and contain no control characters;
|
||||
- when the alias contains `/`, its prefix must not name a configured
|
||||
provider (would be shadowed by prefix resolution);
|
||||
- the alias must not equal an explicit `models` entry of any provider
|
||||
(ambiguous — which one wins?);
|
||||
- the target must not itself be another alias (no alias chains);
|
||||
- the target must not be empty.
|
||||
A target naming a model that is not yet known (awaiting a provider's
|
||||
first discovery fetch, or simply never configured) is accepted at
|
||||
construction — it resolves lazily. An unresolvable target at request
|
||||
time returns 404 with a message naming both the alias and the missing
|
||||
target. A target that is later discovered and collides with an
|
||||
already-configured alias id is not a construction error either — the
|
||||
alias always wins that resolution, and the gateway logs one warning
|
||||
per such id.
|
||||
- **`GET /v1/models` listing**: every alias the caller's group may use is
|
||||
listed as a model object alongside real models — `id` is the alias,
|
||||
`owned_by` is the target's resolved provider name. An alias whose
|
||||
target does not resolve yet is omitted from the listing until it does.
|
||||
- **Echo behavior**: exactly like the existing provider-prefix alias echo
|
||||
(`__alias`, [Unified vs. passthrough](#unified-vs-passthrough)) —
|
||||
`anthropic`- and `gemini`-type targets build their own response
|
||||
envelope and echo the client's exact alias string back in its `model`
|
||||
field. An `openai`-type target's response is a **verbatim passthrough**
|
||||
of the upstream body, so its `model` field carries whatever the
|
||||
upstream itself returned (typically the bare upstream model id) —
|
||||
**not** the alias — the same documented v0.1 passthrough asymmetry,
|
||||
unchanged by this feature.
|
||||
- **Applies everywhere a model routes**: unified chat and embeddings,
|
||||
`/v1/images/generations`, `/v1/audio/speech`, and
|
||||
`/v1/audio/transcriptions` (including its multipart rewrite — the
|
||||
upstream `model` field receives the resolved target's own upstream id,
|
||||
never the alias, exactly like a provider-prefixed id already does).
|
||||
- **Admin overview**: `GET /admin/api/overview` includes an `aliases`
|
||||
array (`{alias, target}`, sorted by alias) — see [Admin](#admin). No
|
||||
secrets involved.
|
||||
|
||||
## Limits and accounting
|
||||
|
||||
- Counters use **fixed windows**, keyed by UTC calendar boundaries (the
|
||||
@@ -592,10 +672,12 @@ request to any of them falls through to the plugin's existing
|
||||
the `x-api-key` header on this page's own `/admin/api/*` fetches.
|
||||
- **`GET /admin/api/overview`** and **`GET /admin/api/usage`** share one
|
||||
gate: unauthenticated → 401, authenticated non-admin → 403, admin →
|
||||
serve. `overview` returns the provider list, groups, redis/cache
|
||||
status, and the plugin version string. `usage` returns every user's and
|
||||
every group's current-window counter values (req/min, req/day, tok/day,
|
||||
tok/month, cost/day, cost/month) alongside their configured limits.
|
||||
serve. `overview` returns the provider list, groups, the configured
|
||||
model alias table (`alias`/`target` pairs, sorted by alias — see
|
||||
[Model aliases](#model-aliases)), redis/cache status, and the plugin
|
||||
version string. `usage` returns every user's and every group's
|
||||
current-window counter values (req/min, req/day, tok/day, tok/month,
|
||||
cost/day, cost/month) alongside their configured limits.
|
||||
- **What's exposed**: provider names, types, base URLs (with any
|
||||
userinfo/query string stripped before it's ever echoed), model counts,
|
||||
discovery status, group/user names, membership, limits, and live usage
|
||||
|
||||
@@ -194,6 +194,14 @@ type adminGroupView struct {
|
||||
MemberCount int `json:"memberCount"`
|
||||
}
|
||||
|
||||
// adminAliasView is one configured model alias's row in GET
|
||||
// /admin/api/overview (spec §5, v0.2): the alias->target pair exactly as
|
||||
// an operator wrote it. No secrets involved.
|
||||
type adminAliasView struct {
|
||||
Alias string `json:"alias"`
|
||||
Target string `json:"target"`
|
||||
}
|
||||
|
||||
// adminOverviewResponse is the full body of GET /admin/api/overview (spec
|
||||
// §4, v0.2). Every slice is sorted and built fresh per request — no
|
||||
// caching of the response itself — so the dashboard's 5s poll always
|
||||
@@ -202,6 +210,7 @@ type adminOverviewResponse struct {
|
||||
Version string `json:"version"`
|
||||
Providers []adminProviderView `json:"providers"`
|
||||
Groups []adminGroupView `json:"groups"`
|
||||
Aliases []adminAliasView `json:"aliases"`
|
||||
Redis adminRedisView `json:"redis"`
|
||||
Cache adminCacheView `json:"cache"`
|
||||
}
|
||||
@@ -237,11 +246,18 @@ func (g *Gateway) buildAdminOverview() adminOverviewResponse {
|
||||
groups[i] = adminGroupView{Name: gs.name, Limits: gs.limits, MemberCount: gs.memberCount}
|
||||
}
|
||||
|
||||
aliasSnaps := g.registry.aliasSnapshot()
|
||||
aliases := make([]adminAliasView, len(aliasSnaps))
|
||||
for i, a := range aliasSnaps {
|
||||
aliases[i] = adminAliasView(a) // identical underlying field shape (alias, target string), differing only in json tags
|
||||
}
|
||||
|
||||
return adminOverviewResponse{
|
||||
Providers: providers,
|
||||
Redis: adminRedisView{Configured: configured, LastErr: redisLastErr, LastErrAt: redisLastErrAt},
|
||||
Cache: adminCacheView{Enabled: cacheEnabled, TTL: ttl},
|
||||
Groups: groups,
|
||||
Aliases: aliases,
|
||||
Version: pluginVersion,
|
||||
}
|
||||
}
|
||||
@@ -443,6 +459,13 @@ const adminPageHTML = `<!doctype html>
|
||||
</tr></thead><tbody></tbody></table>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Model aliases</h2>
|
||||
<table id="aliases"><thead><tr>
|
||||
<th>Alias</th><th>Target</th>
|
||||
</tr></thead><tbody></tbody></table>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Infrastructure</h2>
|
||||
<div id="infra"></div>
|
||||
@@ -542,6 +565,13 @@ const adminPageHTML = `<!doctype html>
|
||||
return tr;
|
||||
});
|
||||
|
||||
setRows("aliases", data.aliases, function (a) {
|
||||
var tr = document.createElement("tr");
|
||||
tr.appendChild(el("td", a.alias));
|
||||
tr.appendChild(el("td", a.target));
|
||||
return tr;
|
||||
});
|
||||
|
||||
var infra = document.getElementById("infra");
|
||||
while (infra.firstChild) infra.removeChild(infra.firstChild);
|
||||
var redisText = "Redis: " + (data.redis.configured ? "configured" : "not configured");
|
||||
|
||||
@@ -195,6 +195,14 @@ func assertOAIErrorEnvelope(t *testing.T, rec *httptest.ResponseRecorder) {
|
||||
func TestAdminOverview_SortedShapeAndVersion(t *testing.T) {
|
||||
t.Parallel()
|
||||
cfg := newAdminTestConfig()
|
||||
// Two aliases, deliberately out of sorted order, proving both the
|
||||
// overview's aliases table (spec §5, v0.2) is populated AND sorted by
|
||||
// alias — "zaliased/x" is a bare target ("z-model", zeta's own
|
||||
// explicit model), "aaliased/y" is provider-prefixed.
|
||||
cfg.ModelAliases = map[string]string{
|
||||
"zaliased/x": "z-model",
|
||||
"aaliased/y": "alpha/a-model-1",
|
||||
}
|
||||
h, _ := newAdminGatewayHandle(t, cfg)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
@@ -247,6 +255,19 @@ func TestAdminOverview_SortedShapeAndVersion(t *testing.T) {
|
||||
if got.Redis.Configured {
|
||||
t.Error("redis.configured must be false: cfg.Redis was never configured")
|
||||
}
|
||||
|
||||
wantAliases := []adminAliasView{
|
||||
{Alias: "aaliased/y", Target: "alpha/a-model-1"},
|
||||
{Alias: "zaliased/x", Target: "z-model"},
|
||||
}
|
||||
if len(got.Aliases) != len(wantAliases) {
|
||||
t.Fatalf("aliases = %+v, want exactly %+v", got.Aliases, wantAliases)
|
||||
}
|
||||
for i, want := range wantAliases {
|
||||
if got.Aliases[i] != want {
|
||||
t.Errorf("aliases[%d] = %+v, want %+v (sorted by alias)", i, got.Aliases[i], want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdminOverview_ProviderLastErrAfterFailedRefresh(t *testing.T) {
|
||||
|
||||
@@ -151,6 +151,34 @@ func TestCacheKey_ChatVsEmbeddings_DistinctKeys(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestCacheKey_TwoOperatorAliases_SameTarget_DistinctKeys is task 6's
|
||||
// (spec §5, v0.2) dedicated cache-key regression, phrased in terms of
|
||||
// operator-defined model aliases (modelAliases) rather than
|
||||
// TestCacheKey_BareVsPrefixedAlias_DistinctKeys' bare-vs-prefixed pair
|
||||
// above. Two DIFFERENT alias ids ("aliased/coding", "aliased/fast")
|
||||
// configured to resolve to the identical provider+upstreamModel target
|
||||
// must still hash to distinct cache entries: cacheKey already hashes
|
||||
// requestedModel as real key material (this file's own doc comment,
|
||||
// TestCacheKey_BareVsPrefixedAlias_DistinctKeys), and
|
||||
// modelRegistry.resolveAliasTarget (registry.go) leaves that string
|
||||
// completely untouched regardless of which alias resolved to which
|
||||
// target — the two features compose with NO cache.go changes required.
|
||||
func TestCacheKey_TwoOperatorAliases_SameTarget_DistinctKeys(t *testing.T) {
|
||||
req := map[string]any{"messages": []any{"hi"}}
|
||||
|
||||
codingAlias := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/coding", cacheEndpointChat, req)
|
||||
fastAlias := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/fast", cacheEndpointChat, req)
|
||||
if codingAlias == fastAlias {
|
||||
t.Error("cacheKey identical for two different operator-defined aliases resolving to the identical provider+upstreamModel target")
|
||||
}
|
||||
|
||||
// Same alias twice must still hash identically — a genuine repeat
|
||||
// request under the same alias is a genuine cache hit.
|
||||
if got, want := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/coding", cacheEndpointChat, req), codingAlias; got != want {
|
||||
t.Errorf("cacheKey differs for two requests using the identical alias: %q vs %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// --- validateCacheConfig: defaults and validation ---
|
||||
|
||||
func TestValidateCacheConfig_DefaultsAppliedWhenZero(t *testing.T) {
|
||||
|
||||
@@ -239,6 +239,55 @@ func TestUnifiedChatAllProviders(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelAliases covers v0.2 integration coverage: task 6 (spec §5).
|
||||
// dynamic.yml.tmpl configures two operator-defined aliases —
|
||||
// "aliased/mock" -> "gpt-mock" (openai-type) and "aliased/claude" ->
|
||||
// "claude-mock" (anthropic-type) — proving alias resolution runs under
|
||||
// real Traefik+Yaegi, not just go test, including the documented
|
||||
// echo-behavior asymmetry between provider types (see
|
||||
// registry_test.go's Gateway-level equivalents for the same assertion
|
||||
// under go test).
|
||||
func TestModelAliases(t *testing.T) {
|
||||
// openai-type: the upstream response is forwarded verbatim
|
||||
// (provider_openai.go), so its "model" field carries whatever the
|
||||
// mock echoes back — the resolved upstream id "gpt-mock", not the
|
||||
// alias. Correctness here is 200 + real content + the cache header
|
||||
// unified routes always set, not the "model" field (documented v0.1
|
||||
// passthrough asymmetry, unchanged by aliasing).
|
||||
openaiReq := map[string]any{
|
||||
"model": "aliased/mock",
|
||||
"messages": []map[string]any{{"role": "user", "content": "hi"}},
|
||||
}
|
||||
resp, body := doJSON(t, http.MethodPost, traefik1URL+"/v1/chat/completions", aliceKey, openaiReq)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("aliased openai chat: status = %d, body=%#v", resp.StatusCode, body)
|
||||
}
|
||||
if content := firstChoiceContent(t, body); content != "mock openai response" {
|
||||
t.Errorf("aliased openai chat: content = %q, want %q", content, "mock openai response")
|
||||
}
|
||||
if got := resp.Header.Get("X-Llmgw-Cache"); got == "" {
|
||||
t.Error("aliased openai chat: X-Llmgw-Cache header missing, want \"miss\" or \"hit\"")
|
||||
}
|
||||
|
||||
// anthropic-type: translate_anthropic.go builds its own response
|
||||
// envelope and echoes the client's exact requested id (the alias)
|
||||
// back into "model" — this DOES surface the alias.
|
||||
anthropicReq := map[string]any{
|
||||
"model": "aliased/claude",
|
||||
"messages": []map[string]any{{"role": "user", "content": "hi"}},
|
||||
}
|
||||
resp2, body2 := doJSON(t, http.MethodPost, traefik1URL+"/v1/chat/completions", aliceKey, anthropicReq)
|
||||
if resp2.StatusCode != http.StatusOK {
|
||||
t.Fatalf("aliased anthropic chat: status = %d, body=%#v", resp2.StatusCode, body2)
|
||||
}
|
||||
if content := firstChoiceContent(t, body2); content != "mock anthropic response" {
|
||||
t.Errorf("aliased anthropic chat: content = %q, want %q", content, "mock anthropic response")
|
||||
}
|
||||
if model, _ := body2["model"].(string); model != "aliased/claude" {
|
||||
t.Errorf("aliased anthropic chat: response model = %q, want the client-requested alias %q echoed back", model, "aliased/claude")
|
||||
}
|
||||
}
|
||||
|
||||
// TestStreamingIncrementalityProbe covers integration test 3.
|
||||
//
|
||||
// Content correctness always runs, via assertStreamingContent: a host-side
|
||||
|
||||
@@ -98,9 +98,10 @@ http:
|
||||
mcpServers:
|
||||
tool:
|
||||
url: "http://mocktool:8080"
|
||||
# v0.2 features (spec §1, §2, §4), all opt-in and off by default
|
||||
# in v0.1: enabled here so the integration suite exercises every
|
||||
# one under real Traefik+Yaegi, not just go test.
|
||||
# v0.2 features (spec §1, §2, §4, §5), all opt-in and off by
|
||||
# default in v0.1: enabled here so the integration suite
|
||||
# exercises every one under real Traefik+Yaegi, not just go
|
||||
# test.
|
||||
retry:
|
||||
enabled: true
|
||||
attempts: 1 # one retry after the first try — matches handleFlakyOpenAIChat's single forced 503
|
||||
@@ -109,6 +110,15 @@ http:
|
||||
ttl: "2m"
|
||||
admin:
|
||||
enabled: true
|
||||
# Task 6 (spec §5): one alias per mock provider type, so the
|
||||
# integration suite proves the echo-behavior asymmetry under
|
||||
# real Traefik+Yaegi — openai-type forwards the upstream body
|
||||
# verbatim (its "model" field carries "gpt-mock", not the
|
||||
# alias); anthropic-type builds its own envelope and echoes the
|
||||
# client's exact alias string back.
|
||||
modelAliases:
|
||||
aliased/mock: "gpt-mock"
|
||||
aliased/claude: "claude-mock"
|
||||
|
||||
# Second, separate middleware instance carrying only the optional
|
||||
# real-upstream smoke test's "uni" provider (task 7, INTEGRATION_REAL=1
|
||||
|
||||
@@ -27,6 +27,15 @@ type Config struct {
|
||||
// all — ServeHTTP falls through to its existing 404/passthroughUnknown
|
||||
// handling for those paths, preserving v0.1 behavior exactly.
|
||||
Admin *AdminConfig `json:"admin,omitempty"`
|
||||
// ModelAliases maps an operator-defined alias id to a target model id
|
||||
// (spec §5, v0.2) — e.g. {"aliased/coding": "anthropic/claude-sonnet-4-5"}.
|
||||
// An exact alias match wins resolution before any other rule
|
||||
// (modelRegistry.resolve, registry.go); the target then resolves
|
||||
// through the normal rules. nil/omitted preserves v0.1 behavior
|
||||
// exactly: resolve never consults an empty alias map, so no existing
|
||||
// model id's resolution changes. Validated at construction
|
||||
// (validateModelAliases, registry.go).
|
||||
ModelAliases map[string]string `json:"modelAliases,omitempty"`
|
||||
// Retry is a struct value, not a pointer, because its own Enabled
|
||||
// field is the on/off signal (unlike Redis/Users, where the block's
|
||||
// mere presence is the signal) — so its tag omits "omitempty":
|
||||
|
||||
+258
-5
@@ -5,6 +5,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
@@ -160,7 +161,16 @@ type modelRegistry struct {
|
||||
// specific to a field holding a func value. Verified empirically
|
||||
// against yaegi v0.16.1 (see tools/yaegi-check) and does not apply to
|
||||
// g.logf/g.errorf, which are methods, not fields.
|
||||
log func(string, ...any)
|
||||
log func(string, ...any)
|
||||
// aliases is the validated alias->target map from Config.ModelAliases
|
||||
// (spec §5, v0.2), built once by newModelRegistry via
|
||||
// validateModelAliases and never mutated afterward — resolve checks
|
||||
// it first, ahead of splitConfiguredProvider/bareWinner (see resolve's
|
||||
// own doc comment). nil (not merely empty) when the operator
|
||||
// configured no aliases at all, so every alias-aware code path is a
|
||||
// cheap nil-map lookup (always false/zero) that adds no branching
|
||||
// cost to the v0.1 no-aliases case.
|
||||
aliases map[string]string
|
||||
nowFn func() time.Time
|
||||
warned map[string]bool
|
||||
providerNames []string
|
||||
@@ -187,6 +197,12 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func
|
||||
}
|
||||
sort.Strings(m.providerNames)
|
||||
|
||||
// explicitModels is the union of every configured provider's EXPLICIT
|
||||
// model ids only (never discovered ones — discovery has not run yet
|
||||
// at this point in construction) — validateModelAliases' collision
|
||||
// check needs exactly this set (spec §5: "alias must not equal an
|
||||
// explicit model id of any provider").
|
||||
explicitModels := make(map[string]bool)
|
||||
for _, name := range m.providerNames {
|
||||
pc := cfg.Providers[name]
|
||||
if pc == nil {
|
||||
@@ -205,6 +221,7 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func
|
||||
explicit := make(map[string]bool, len(pc.Models))
|
||||
for _, id := range pc.Models {
|
||||
explicit[id] = true
|
||||
explicitModels[id] = true
|
||||
}
|
||||
|
||||
m.states[name] = &providerState{
|
||||
@@ -214,9 +231,87 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func
|
||||
interval: interval,
|
||||
}
|
||||
}
|
||||
|
||||
aliases, err := validateModelAliases(cfg.ModelAliases, m.providerNames, explicitModels)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.aliases = aliases
|
||||
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// isValidModelID reports whether id satisfies spec §5's alias model-id
|
||||
// shape: non-empty, and free of ASCII control characters (0x00-0x1F and
|
||||
// 0x7F) — the same class of raw value mcp_a2a.go's target-URL validation
|
||||
// already rejects, applied here to a model/alias id string instead of a
|
||||
// URL.
|
||||
func isValidModelID(id string) bool {
|
||||
if id == "" {
|
||||
return false
|
||||
}
|
||||
for _, r := range id {
|
||||
if r < 0x20 || r == 0x7f {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// validateModelAliases validates cfg.ModelAliases per spec §5's
|
||||
// validation table and returns the accepted alias->target map (nil when
|
||||
// raw is empty — newGateway's default, zero-behavior-change case).
|
||||
// providerNames is the registry's own sorted provider-name list (used for
|
||||
// the prefix-shadow check); explicitModels is the union of every
|
||||
// configured provider's EXPLICIT model ids only (used for the collision
|
||||
// check) — a DISCOVERED model colliding with an alias is NOT a
|
||||
// construction error (discovery is dynamic and runs after this), and is
|
||||
// instead resolved in the alias's favor by resolve's own precedence; see
|
||||
// resolve's warnAliasShadowsDiscoveredOnce call for the one-time runtime
|
||||
// log that implies.
|
||||
//
|
||||
// raw is iterated in sorted alias-key order, not Go's randomized map
|
||||
// order, so a config with more than one invalid alias always reports the
|
||||
// same one first, deterministically, across repeated runs.
|
||||
func validateModelAliases(raw map[string]string, providerNames []string, explicitModels map[string]bool) (map[string]string, error) {
|
||||
if len(raw) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
providerSet := make(map[string]bool, len(providerNames))
|
||||
for _, p := range providerNames {
|
||||
providerSet[p] = true
|
||||
}
|
||||
|
||||
names := make([]string, 0, len(raw))
|
||||
for alias := range raw {
|
||||
names = append(names, alias)
|
||||
}
|
||||
sort.Strings(names)
|
||||
|
||||
out := make(map[string]string, len(raw))
|
||||
for _, alias := range names {
|
||||
target := raw[alias]
|
||||
if !isValidModelID(alias) {
|
||||
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q is invalid: must be non-empty with no control characters", alias)
|
||||
}
|
||||
if prefix, _, hasSlash := strings.Cut(alias, "/"); hasSlash && providerSet[prefix] {
|
||||
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q is invalid: its prefix %q names a configured provider and would be shadowed by provider-prefix resolution", alias, prefix)
|
||||
}
|
||||
if explicitModels[alias] {
|
||||
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q collides with an explicitly configured model id of the same name", alias)
|
||||
}
|
||||
if target == "" {
|
||||
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q has an empty target", alias)
|
||||
}
|
||||
if _, targetIsAlias := raw[target]; targetIsAlias {
|
||||
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q targets %q, which is itself an alias (alias chains are not allowed)", alias, target)
|
||||
}
|
||||
out[alias] = target
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// now returns the registry's current time, via nowFn — overridable in
|
||||
// tests to exercise refresh throttling deterministically.
|
||||
func (m *modelRegistry) now() time.Time {
|
||||
@@ -350,6 +445,17 @@ func (m *modelRegistry) bareWinner(id string) (string, bool) {
|
||||
// exactly-as-requested id the caller passed in is not returned separately —
|
||||
// the caller already holds it, in the id argument itself.
|
||||
//
|
||||
// An EXACT alias match (spec §5, v0.2) wins before any other rule below —
|
||||
// checked first, ahead of provider-prefix splitting, so an alias id that
|
||||
// happens to look like "prefix/rest" is never mistaken for one (and
|
||||
// validateModelAliases already rejects an alias whose prefix names an
|
||||
// actually configured provider at construction, so this ordering never
|
||||
// has to arbitrate a genuine conflict between the two forms). id also
|
||||
// happening to collide with a since-DISCOVERED model of some provider
|
||||
// (impossible for an explicit one; validateModelAliases already rejects
|
||||
// that at construction) is resolved in the alias's favor by this same
|
||||
// ordering — warnAliasShadowsDiscoveredOnce logs that once per id.
|
||||
//
|
||||
// id in "provider/model" form (ruling (a): only when "provider" names a
|
||||
// configured provider) resolves directly against that provider. Any other
|
||||
// id is a bare id, resolved against the first configured provider (sorted
|
||||
@@ -359,14 +465,89 @@ func (m *modelRegistry) bareWinner(id string) (string, bool) {
|
||||
// exists but fails authorization returns errModelDenied, distinct from
|
||||
// errModelUnknown for a model no configured provider knows at all.
|
||||
func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string, string, error) {
|
||||
if target, isAlias := m.aliases[id]; isAlias {
|
||||
if _, discoveredCollision := m.bareWinner(id); discoveredCollision {
|
||||
m.warnAliasShadowsDiscoveredOnce(id)
|
||||
}
|
||||
return m.resolveAliasTarget(id, target, grp)
|
||||
}
|
||||
if providerName, rest, ok := m.splitConfiguredProvider(id); ok {
|
||||
return m.resolveAgainst(providerName, rest, id, grp)
|
||||
return m.resolveAgainst(providerName, rest, id, "", grp, errModelUnknown)
|
||||
}
|
||||
providerName, ok := m.bareWinner(id)
|
||||
if !ok {
|
||||
return nil, "", "", errModelUnknown
|
||||
}
|
||||
return m.resolveAgainst(providerName, id, id, grp)
|
||||
return m.resolveAgainst(providerName, id, id, "", grp, errModelUnknown)
|
||||
}
|
||||
|
||||
// aliasTargetError is resolveAgainst's notFoundErr when the provider it
|
||||
// was called against, and the upstream model id on it, both came from
|
||||
// resolving an alias's TARGET (resolveAliasTarget) rather than a direct
|
||||
// client request — spec §5's "Targets referencing not-yet-discovered
|
||||
// models are allowed at construction and resolve lazily; an unresolvable
|
||||
// target at request time → 404 whose message names the alias AND the
|
||||
// missing target". writeModelResolveError (routes_unified.go) type-
|
||||
// asserts for *aliasTargetError explicitly, ahead of its
|
||||
// errors.Is(errModelUnknown) switch — a plain type assertion, not
|
||||
// errors.As, matching this package's established yaegi-safe convention
|
||||
// for a pointer error type (providers.go's providerHTTPError doc
|
||||
// comment: errors.As panics under yaegi checking whether an interpreted
|
||||
// pointer type implements error).
|
||||
type aliasTargetError struct {
|
||||
alias, target string
|
||||
}
|
||||
|
||||
// Error names both the alias and its unresolved target, per spec §5.
|
||||
func (e *aliasTargetError) Error() string {
|
||||
return fmt.Sprintf("llmgateway: alias %q targets %q, which is not a known model", e.alias, e.target)
|
||||
}
|
||||
|
||||
// resolveAliasTarget finishes resolve for id already known to be an exact
|
||||
// alias match: target is resolved through the SAME rules as any other id
|
||||
// — provider-prefixed direct, or bare-id collision winner — via the same
|
||||
// resolveAgainst helper, with two differences from a direct request:
|
||||
//
|
||||
// - Authorization (spec §5): "a group may use an alias when its model
|
||||
// globs match the ALIAS name OR the resolved target (either
|
||||
// grants)". alias is passed to resolveAgainst as its extraModelName,
|
||||
// so a request denied by every target-side candidate can still
|
||||
// succeed purely because the group's model glob names the alias
|
||||
// itself.
|
||||
// - A target that does not resolve to any known model at request time
|
||||
// returns *aliasTargetError (naming both ids), not the bare
|
||||
// errModelUnknown a direct request's unknown-model case returns.
|
||||
func (m *modelRegistry) resolveAliasTarget(alias, target string, grp *group) (providerAdapter, string, string, error) {
|
||||
notFound := &aliasTargetError{alias: alias, target: target}
|
||||
if providerName, rest, ok := m.splitConfiguredProvider(target); ok {
|
||||
return m.resolveAgainst(providerName, rest, target, alias, grp, notFound)
|
||||
}
|
||||
providerName, ok := m.bareWinner(target)
|
||||
if !ok {
|
||||
return nil, "", "", notFound
|
||||
}
|
||||
return m.resolveAgainst(providerName, target, target, alias, grp, notFound)
|
||||
}
|
||||
|
||||
// warnAliasShadowsDiscoveredOnce logs, at most once per colliding alias id
|
||||
// for this registry's lifetime, that alias also names a model some
|
||||
// provider's discovery fetch found after construction — resolve's
|
||||
// documented precedence (its own doc comment above): the alias always
|
||||
// wins, since only an EXPLICIT collision is rejected at construction
|
||||
// (validateModelAliases); discovery is dynamic and runs after that check,
|
||||
// so this collision can only be caught, and only warned about, here at
|
||||
// request time. Keyed with an "alias-shadow:" prefix in the same m.warned
|
||||
// map warnCollisionOnce uses, so the two unrelated warning kinds can never
|
||||
// share a key and suppress each other.
|
||||
func (m *modelRegistry) warnAliasShadowsDiscoveredOnce(alias string) {
|
||||
m.warnedMu.Lock()
|
||||
defer m.warnedMu.Unlock()
|
||||
key := "alias-shadow:" + alias
|
||||
if m.warned[key] {
|
||||
return
|
||||
}
|
||||
m.warned[key] = true
|
||||
m.log("%s", fmt.Sprintf("model registry: alias %q also names a discovered model; the alias takes precedence", alias))
|
||||
}
|
||||
|
||||
// resolveAgainst finishes resolve for a providerName already chosen (either
|
||||
@@ -386,11 +567,29 @@ func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string,
|
||||
// legitimately provider-prefixed in the first place (ruling: a pattern like
|
||||
// "deepseek-*" must not match a bare id that merely contains a slash, e.g.
|
||||
// "uni/deepseek-v4-flash-0731", when "uni" is not a configured provider).
|
||||
func (m *modelRegistry) resolveAgainst(providerName, upstreamModel, requestedID string, grp *group) (providerAdapter, string, string, error) {
|
||||
//
|
||||
// extraModelName is a further authorization candidate checked via
|
||||
// grp.allowsModel, alongside requestedID and upstreamModel: empty ("") for
|
||||
// a direct (non-alias) resolution, or an alias's own id when resolveAliasTarget
|
||||
// (spec §5, v0.2) is resolving that alias's target — "a group may use an
|
||||
// alias when its model globs match the ALIAS name OR the resolved target
|
||||
// (either grants)".
|
||||
//
|
||||
// notFoundErr is returned, unmodified, when upstreamModel is not known to
|
||||
// providerName: resolve's own two call sites pass the bare errModelUnknown
|
||||
// sentinel (unchanged from before this parameter existed — checked by "err
|
||||
// != errModelUnknown" equality in registry_test.go, not errors.Is, so it
|
||||
// must stay the exact sentinel value there); resolveAliasTarget passes a
|
||||
// *aliasTargetError naming both the alias and its unresolved target (spec
|
||||
// §5's 404 message requirement).
|
||||
func (m *modelRegistry) resolveAgainst(providerName, upstreamModel, requestedID, extraModelName string, grp *group, notFoundErr error) (providerAdapter, string, string, error) {
|
||||
if !m.states[providerName].hasModel(upstreamModel) {
|
||||
return nil, "", "", errModelUnknown
|
||||
return nil, "", "", notFoundErr
|
||||
}
|
||||
allowed := grp.allowsModel(requestedID) || grp.allowsModel(upstreamModel)
|
||||
if extraModelName != "" {
|
||||
allowed = allowed || grp.allowsModel(extraModelName)
|
||||
}
|
||||
if !allowed || !grp.allowsProvider(providerName) {
|
||||
return nil, "", "", errModelDenied
|
||||
}
|
||||
@@ -459,6 +658,33 @@ func (m *modelRegistry) listFor(grp *group) []map[string]any {
|
||||
}
|
||||
}
|
||||
|
||||
// Aliases (spec §5, v0.2): listed alongside real models only when
|
||||
// both their target actually resolves right now (a target awaiting
|
||||
// its provider's first discovery fetch is not listed — there is
|
||||
// nothing yet to name as owned_by) and grp is authorized for it.
|
||||
// resolveAliasTarget applies the exact same "alias name OR target"
|
||||
// authorization rule request-time resolve uses (its own doc
|
||||
// comment), so this listing can never promise access resolve would
|
||||
// then deny. owned_by is recovered from the successful call's own
|
||||
// canonical return value ("provider/upstreamModel") by splitting on
|
||||
// the first "/" — exact, because a registry provider name can never
|
||||
// itself contain one (configNamePattern, providers.go). An alias id
|
||||
// that also happens to collide with a since-discovered model's own
|
||||
// bare id (impossible for an EXPLICIT model — validateModelAliases
|
||||
// already rejects that at construction) is not deduped against that
|
||||
// model's own entry above: the same documented, alias-wins
|
||||
// precedence edge resolve's warnAliasShadowsDiscoveredOnce logs
|
||||
// about, accepted here too as rare enough not to warrant extra
|
||||
// bookkeeping.
|
||||
for alias, target := range m.aliases {
|
||||
_, _, canonical, err := m.resolveAliasTarget(alias, target, grp)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
providerName, _, _ := strings.Cut(canonical, "/")
|
||||
out = append(out, modelObject(alias, providerName))
|
||||
}
|
||||
|
||||
sort.Slice(out, func(i, j int) bool {
|
||||
return out[i]["id"].(string) < out[j]["id"].(string) //nolint:forcetypeassert // modelObject always sets id to a string
|
||||
})
|
||||
@@ -484,6 +710,33 @@ type providerSnapshot struct {
|
||||
modelCount int
|
||||
}
|
||||
|
||||
// aliasSnapshotEntry is one configured alias's read-only view for the
|
||||
// admin dashboard (spec §5, v0.2): the pair exactly as an operator wrote
|
||||
// it. No secrets involved. The target's actual resolved provider is
|
||||
// deliberately not included — a target still awaiting its provider's
|
||||
// first discovery fetch has none yet, and the configured pair is the
|
||||
// meaningful, stable fact an operator wants to see either way.
|
||||
type aliasSnapshotEntry struct {
|
||||
Alias string
|
||||
Target string
|
||||
}
|
||||
|
||||
// aliasSnapshot returns every configured alias's alias->target pair,
|
||||
// sorted by alias — the admin dashboard's alias table (spec §5, v0.2).
|
||||
func (m *modelRegistry) aliasSnapshot() []aliasSnapshotEntry {
|
||||
names := make([]string, 0, len(m.aliases))
|
||||
for alias := range m.aliases {
|
||||
names = append(names, alias)
|
||||
}
|
||||
sort.Strings(names)
|
||||
|
||||
out := make([]aliasSnapshotEntry, len(names))
|
||||
for i, alias := range names {
|
||||
out[i] = aliasSnapshotEntry{Alias: alias, Target: m.aliases[alias]}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// snapshot returns every configured provider's read-only view, sorted by
|
||||
// name (m.providerNames is already sorted at construction) — the admin
|
||||
// dashboard's provider table (spec §4, v0.2).
|
||||
|
||||
@@ -3,9 +3,11 @@ package traefikllmgateway
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
@@ -893,3 +895,351 @@ func TestNewGateway_WarmFill_PopulatesDiscoveredModels_BeforeAnyRequest(t *testi
|
||||
t.Fatal("want disco-model known immediately after New returns (synchronous warm fill)")
|
||||
}
|
||||
}
|
||||
|
||||
// --- model aliases (spec §5, v0.2) ---
|
||||
|
||||
// newAliasTestRegistry builds a registry with one "openai" provider
|
||||
// (explicit model "gpt-test") and grp's given aliases, failing the test on
|
||||
// any construction error. log defaults to a no-op when nil.
|
||||
func newAliasTestRegistry(t *testing.T, aliases map[string]string, log func(string, ...any)) *modelRegistry {
|
||||
t.Helper()
|
||||
if log == nil {
|
||||
log = func(string, ...any) {}
|
||||
}
|
||||
adapters := map[string]providerAdapter{"openai": newFakeAdapter("openai")}
|
||||
cfg := &Config{
|
||||
Providers: map[string]*ProviderConfig{"openai": {Models: []string{"gpt-test"}}},
|
||||
ModelAliases: aliases,
|
||||
}
|
||||
reg, err := newModelRegistry(adapters, cfg, log)
|
||||
if err != nil {
|
||||
t.Fatalf("newModelRegistry: %v", err)
|
||||
}
|
||||
return reg
|
||||
}
|
||||
|
||||
// TestNewModelRegistry_ModelAliases_ValidationTable covers every row of
|
||||
// spec §5's validation table, plus a positive control.
|
||||
func TestNewModelRegistry_ModelAliases_ValidationTable(t *testing.T) {
|
||||
t.Parallel()
|
||||
baseProviders := map[string]*ProviderConfig{"openai": {Models: []string{"gpt-test"}}}
|
||||
adapters := map[string]providerAdapter{"openai": newFakeAdapter("openai")}
|
||||
|
||||
tests := []struct {
|
||||
aliases map[string]string
|
||||
name string
|
||||
wantErrText string // substring expected in the error; "" means construction must succeed
|
||||
}{
|
||||
{
|
||||
name: "empty alias id",
|
||||
aliases: map[string]string{"": "gpt-test"},
|
||||
wantErrText: "invalid",
|
||||
},
|
||||
{
|
||||
name: "control character in alias id",
|
||||
aliases: map[string]string{"bad\x7falias": "gpt-test"},
|
||||
wantErrText: "invalid",
|
||||
},
|
||||
{
|
||||
name: "alias prefix shadows a configured provider",
|
||||
aliases: map[string]string{"openai/special": "gpt-test"},
|
||||
wantErrText: "shadowed",
|
||||
},
|
||||
{
|
||||
name: "alias collides with an explicit model id",
|
||||
aliases: map[string]string{"gpt-test": "openai/gpt-test"},
|
||||
wantErrText: "collides",
|
||||
},
|
||||
{
|
||||
name: "alias chain: target is itself an alias",
|
||||
aliases: map[string]string{"aliased/a": "aliased/b", "aliased/b": "openai/gpt-test"},
|
||||
wantErrText: "itself an alias",
|
||||
},
|
||||
{
|
||||
name: "empty target",
|
||||
aliases: map[string]string{"aliased/x": ""},
|
||||
wantErrText: "empty target",
|
||||
},
|
||||
{
|
||||
name: "valid: bare target, no prefix collision",
|
||||
aliases: map[string]string{"aliased/coding": "gpt-test"},
|
||||
},
|
||||
{
|
||||
name: "valid: provider-prefixed target, unconfigured prefix on the alias itself",
|
||||
aliases: map[string]string{"team/coding": "openai/gpt-test"},
|
||||
},
|
||||
{
|
||||
name: "valid: target not yet known (lazy resolution, allowed at construction)",
|
||||
aliases: map[string]string{"aliased/future": "openai/not-yet-discovered"},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
cfg := &Config{Providers: baseProviders, ModelAliases: tt.aliases}
|
||||
_, err := newModelRegistry(adapters, cfg, func(string, ...any) {})
|
||||
if tt.wantErrText == "" {
|
||||
if err != nil {
|
||||
t.Fatalf("newModelRegistry: %v, want success", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("newModelRegistry: want error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), tt.wantErrText) {
|
||||
t.Errorf("err = %v, want substring %q", err, tt.wantErrText)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelRegistry_Resolve_Alias_SlashLikeID_NotMistakenForProviderPrefix
|
||||
// is the resolve-precedence regression from spec §5: an alias id
|
||||
// containing "/" whose prefix names NO configured provider ("team" is not
|
||||
// configured here — a prefix that IS configured is rejected at
|
||||
// construction, so this is the only shape available to prove the alias
|
||||
// map is checked, and wins, ahead of splitConfiguredProvider/bareWinner)
|
||||
// must still resolve through the alias table, not fall through to
|
||||
// errModelUnknown.
|
||||
func TestModelRegistry_Resolve_Alias_SlashLikeID_NotMistakenForProviderPrefix(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"team/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
adapter, upstreamModel, canonical, err := reg.resolve("team/coding", allowAllGroup())
|
||||
if err != nil {
|
||||
t.Fatalf("resolve: %v", err)
|
||||
}
|
||||
if adapter != reg.adapters["openai"] {
|
||||
t.Errorf("adapter = %v, want openai adapter", adapter)
|
||||
}
|
||||
if upstreamModel != "gpt-test" {
|
||||
t.Errorf("upstreamModel = %q, want %q", upstreamModel, "gpt-test")
|
||||
}
|
||||
if canonical != "openai/gpt-test" {
|
||||
t.Errorf("canonical = %q, want %q", canonical, "openai/gpt-test")
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelRegistry_Resolve_Alias_BareTarget_ResolvesViaCollisionWinner
|
||||
// proves resolveAliasTarget's bareWinner branch (a target with no
|
||||
// provider prefix) applies the identical sorted-first collision rule a
|
||||
// direct bare-id request already gets (ruling g).
|
||||
func TestModelRegistry_Resolve_Alias_BareTarget_ResolvesViaCollisionWinner(t *testing.T) {
|
||||
t.Parallel()
|
||||
adapters := map[string]providerAdapter{
|
||||
"beta": newFakeAdapter("beta"),
|
||||
"alpha": newFakeAdapter("alpha"),
|
||||
}
|
||||
cfg := &Config{
|
||||
Providers: map[string]*ProviderConfig{
|
||||
"beta": {Models: []string{"shared"}},
|
||||
"alpha": {Models: []string{"shared"}},
|
||||
},
|
||||
ModelAliases: map[string]string{"aliased/x": "shared"},
|
||||
}
|
||||
reg, err := newModelRegistry(adapters, cfg, func(string, ...any) {})
|
||||
if err != nil {
|
||||
t.Fatalf("newModelRegistry: %v", err)
|
||||
}
|
||||
|
||||
adapter, _, canonical, err := reg.resolve("aliased/x", allowAllGroup())
|
||||
if err != nil {
|
||||
t.Fatalf("resolve: %v", err)
|
||||
}
|
||||
if adapter != adapters["alpha"] {
|
||||
t.Errorf("adapter = %v, want alpha (sorted-first collision winner)", adapter)
|
||||
}
|
||||
if canonical != "alpha/shared" {
|
||||
t.Errorf("canonical = %q, want %q", canonical, "alpha/shared")
|
||||
}
|
||||
}
|
||||
|
||||
// --- resolve: alias authorization (spec §5's "alias name OR target,
|
||||
// either grants") ---
|
||||
|
||||
func TestModelRegistry_Resolve_Alias_AuthorizedViaAliasNameOnly(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
// Matches the alias name; matches neither "openai/gpt-test" nor its
|
||||
// bare suffix "gpt-test".
|
||||
grp := &group{name: "narrow", models: []string{"aliased/coding"}}
|
||||
if _, _, _, err := reg.resolve("aliased/coding", grp); err != nil {
|
||||
t.Errorf("resolve: %v, want success (authorized via the alias name alone)", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestModelRegistry_Resolve_Alias_AuthorizedViaTargetOnly(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
// Matches the target's bare suffix; does not match the alias name.
|
||||
grp := &group{name: "narrow", models: []string{"gpt-*"}}
|
||||
if _, _, _, err := reg.resolve("aliased/coding", grp); err != nil {
|
||||
t.Errorf("resolve: %v, want success (authorized via the resolved target alone)", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestModelRegistry_Resolve_Alias_DeniedWhenNeitherAliasNorTargetMatch(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
grp := &group{name: "narrow", models: []string{"claude-*"}}
|
||||
if _, _, _, err := reg.resolve("aliased/coding", grp); err != errModelDenied {
|
||||
t.Errorf("err = %v, want errModelDenied", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestModelRegistry_Resolve_Alias_ProviderDenied_OverridesModelAuthz(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
// The model glob would allow it via the alias name, but the group's
|
||||
// providers list excludes "openai" entirely — allowsProvider still
|
||||
// gates the resolved target's provider (spec §5's "allowsProvider
|
||||
// still applies to the target's provider").
|
||||
grp := &group{name: "no-openai", providers: []string{"anthropic"}, models: []string{"aliased/coding"}}
|
||||
if _, _, _, err := reg.resolve("aliased/coding", grp); err != errModelDenied {
|
||||
t.Errorf("err = %v, want errModelDenied (provider gate must still apply)", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelRegistry_Resolve_Alias_UnresolvedTarget_ReturnsAliasTargetError
|
||||
// is spec §5's lazy-resolution 404: a target naming no currently-known
|
||||
// model returns *aliasTargetError, whose message names both the alias and
|
||||
// the missing target.
|
||||
func TestModelRegistry_Resolve_Alias_UnresolvedTarget_ReturnsAliasTargetError(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil)
|
||||
|
||||
_, _, _, err := reg.resolve("aliased/future", allowAllGroup())
|
||||
var aerr *aliasTargetError
|
||||
if !errors.As(err, &aerr) {
|
||||
t.Fatalf("err = %v (%T), want *aliasTargetError", err, err)
|
||||
}
|
||||
if !strings.Contains(aerr.Error(), "aliased/future") {
|
||||
t.Errorf("error message = %q, want it to name the alias %q", aerr.Error(), "aliased/future")
|
||||
}
|
||||
if !strings.Contains(aerr.Error(), "openai/not-yet-discovered") {
|
||||
t.Errorf("error message = %q, want it to name the target %q", aerr.Error(), "openai/not-yet-discovered")
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelRegistry_Resolve_Alias_TargetBecomesKnown_ResolvesAfterDiscovery
|
||||
// proves the lazy side of "resolve lazily": a target that was unresolvable
|
||||
// at construction succeeds once its provider's discovered set catches up
|
||||
// (no re-validation needed — discovery mutates providerState directly).
|
||||
func TestModelRegistry_Resolve_Alias_TargetBecomesKnown_ResolvesAfterDiscovery(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil)
|
||||
|
||||
if _, _, _, err := reg.resolve("aliased/future", allowAllGroup()); err == nil {
|
||||
t.Fatal("resolve before discovery: want an error, got nil")
|
||||
}
|
||||
|
||||
reg.states["openai"].finishRefresh(reg.now(), []string{"not-yet-discovered"}, nil)
|
||||
|
||||
adapter, upstreamModel, canonical, err := reg.resolve("aliased/future", allowAllGroup())
|
||||
if err != nil {
|
||||
t.Fatalf("resolve after discovery: %v, want success", err)
|
||||
}
|
||||
if adapter != reg.adapters["openai"] || upstreamModel != "not-yet-discovered" || canonical != "openai/not-yet-discovered" {
|
||||
t.Errorf("resolve after discovery = (%v, %q, %q), want (openai adapter, %q, %q)", adapter, upstreamModel, canonical, "not-yet-discovered", "openai/not-yet-discovered")
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelRegistry_Resolve_Alias_PrecedesDiscoveredCollision_WarnsOnce
|
||||
// covers the documented precedence edge (registry.go's resolve doc
|
||||
// comment): an alias id that also happens to name a model discovered
|
||||
// AFTER construction (impossible for an explicit model — that is a
|
||||
// construction error) still resolves to the ALIAS's own target, and the
|
||||
// gateway logs exactly one warning about the shadowed id, regardless of
|
||||
// how many times resolve is called for it.
|
||||
func TestModelRegistry_Resolve_Alias_PrecedesDiscoveredCollision_WarnsOnce(t *testing.T) {
|
||||
t.Parallel()
|
||||
adapters := map[string]providerAdapter{
|
||||
"openai": newFakeAdapter("openai"),
|
||||
"acme": newFakeAdapter("acme"),
|
||||
}
|
||||
cfg := &Config{
|
||||
Providers: map[string]*ProviderConfig{
|
||||
"openai": {Models: []string{"gpt-test"}},
|
||||
"acme": {Discovery: true},
|
||||
},
|
||||
ModelAliases: map[string]string{"shared-id": "openai/gpt-test"},
|
||||
}
|
||||
rl := &recordingLog{}
|
||||
reg, err := newModelRegistry(adapters, cfg, rl.fn)
|
||||
if err != nil {
|
||||
t.Fatalf("newModelRegistry: %v", err)
|
||||
}
|
||||
// acme's discovery finds a model literally named "shared-id" — the
|
||||
// same string as the configured alias, discovered only after
|
||||
// construction, so validateModelAliases never saw it.
|
||||
reg.states["acme"].finishRefresh(reg.now(), []string{"shared-id"}, nil)
|
||||
|
||||
adapter, _, canonical, err := reg.resolve("shared-id", allowAllGroup())
|
||||
if err != nil {
|
||||
t.Fatalf("resolve: %v", err)
|
||||
}
|
||||
if adapter != adapters["openai"] {
|
||||
t.Errorf("adapter = %v, want openai (the alias, not acme's discovered model, must win)", adapter)
|
||||
}
|
||||
if canonical != "openai/gpt-test" {
|
||||
t.Errorf("canonical = %q, want %q", canonical, "openai/gpt-test")
|
||||
}
|
||||
|
||||
if _, _, _, err := reg.resolve("shared-id", allowAllGroup()); err != nil {
|
||||
t.Fatalf("second resolve: %v", err)
|
||||
}
|
||||
if got := rl.count(); got != 1 {
|
||||
t.Errorf("collision-shadow log emitted %d times across 2 resolve calls, want exactly 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
// --- listFor: alias visibility (spec §5) ---
|
||||
|
||||
func TestModelRegistry_ListFor_Alias_ListedWhenAuthorized_OwnedByTargetProvider(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
got := reg.listFor(allowAllGroup())
|
||||
var found bool
|
||||
for _, entry := range got {
|
||||
if entry["id"] == "aliased/coding" {
|
||||
found = true
|
||||
if entry["owned_by"] != "openai" {
|
||||
t.Errorf("aliased/coding owned_by = %v, want %q", entry["owned_by"], "openai")
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("listFor = %v, want an entry for the alias %q", got, "aliased/coding")
|
||||
}
|
||||
}
|
||||
|
||||
func TestModelRegistry_ListFor_Alias_UnresolvedTarget_Omitted(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil)
|
||||
|
||||
got := reg.listFor(allowAllGroup())
|
||||
for _, entry := range got {
|
||||
if entry["id"] == "aliased/future" {
|
||||
t.Errorf("listFor lists %q whose target does not resolve yet, want it omitted", "aliased/future")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestModelRegistry_ListFor_Alias_NotAuthorized_Omitted(t *testing.T) {
|
||||
t.Parallel()
|
||||
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||
|
||||
grp := &group{name: "narrow", models: []string{"claude-*"}}
|
||||
got := reg.listFor(grp)
|
||||
for _, entry := range got {
|
||||
if entry["id"] == "aliased/coding" {
|
||||
t.Errorf("listFor lists %q for a group authorized for neither the alias nor its target", "aliased/coding")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -166,6 +166,38 @@ func TestHandleImagesGenerations_OpenAI_HappyPath_NativeForward(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImagesGenerations_ModelAlias_ResolvesToTargetUpstreamModel
|
||||
// proves an operator-defined modelAliases entry (spec §5, v0.2) routes
|
||||
// /v1/images/generations exactly like a bare or provider-prefixed id: the
|
||||
// upstream call carries the target's own upstream model id, never the
|
||||
// alias.
|
||||
func TestHandleImagesGenerations_ModelAlias_ResolvesToTargetUpstreamModel(t *testing.T) {
|
||||
var gotBody map[string]any
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_ = json.NewDecoder(r.Body).Decode(&gotBody)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(`{"created":1,"data":[{"b64_json":"xyz"}]}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := newMediaTestConfig(srv.URL, "img-test")
|
||||
cfg.ModelAliases = map[string]string{"aliased/image": "openai/img-test"}
|
||||
gw := newMediaTestGateway(t, cfg)
|
||||
|
||||
body := map[string]any{"model": "aliased/image", "prompt": "a cat", "n": 1}
|
||||
req := newUnifiedRequest(t, http.MethodPost, imagesGenerationsPath, "sk-alice", body)
|
||||
rec := httptest.NewRecorder()
|
||||
gw.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if gotBody["model"] != "img-test" {
|
||||
t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", gotBody["model"], "img-test")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImagesGenerations_StripsClientSuppliedAliasKey proves a
|
||||
// client independently including a literal "__alias" field in its own
|
||||
// JSON body (never one the gateway itself injects for a media route —
|
||||
@@ -726,6 +758,100 @@ func TestHandleAudioTranscriptions_PrefixedAlias_RewritesModelFieldOnly(t *testi
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleAudioTranscriptions_ModelAlias_RewritesModelFieldToTargetUpstreamID
|
||||
// mirrors TestHandleAudioTranscriptions_PrefixedAlias_RewritesModelFieldOnly
|
||||
// above, but through an operator-defined modelAliases entry (spec §5,
|
||||
// v0.2) rather than a bare provider-prefixed id: the client's "model"
|
||||
// form field ("aliased/whisper") differs from the resolved upstream model
|
||||
// id ("whisper-test"), so rewriteMultipartModel (routes_media.go) must
|
||||
// rebuild the multipart body — every part copied verbatim except "model",
|
||||
// which becomes the target's own upstream id. The alias must never reach
|
||||
// the real provider, the same invariant every other unified route
|
||||
// already enforces.
|
||||
func TestHandleAudioTranscriptions_ModelAlias_RewritesModelFieldToTargetUpstreamID(t *testing.T) {
|
||||
const fileContent = "FAKEAUDIOBYTES-DISTINCTIVE-PAYLOAD"
|
||||
var gotContentType string
|
||||
var gotBody []byte
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotContentType = r.Header.Get("Content-Type")
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(`{"text":"ok"}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := newMediaTestConfig(srv.URL, "whisper-test")
|
||||
cfg.ModelAliases = map[string]string{"aliased/whisper": "openai/whisper-test"}
|
||||
gw := newMediaTestGateway(t, cfg)
|
||||
|
||||
var buf bytes.Buffer
|
||||
mw := multipart.NewWriter(&buf)
|
||||
ffw, err := mw.CreateFormFile("file", "audio.wav")
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFormFile: %v", err)
|
||||
}
|
||||
if _, err = ffw.Write([]byte(fileContent)); err != nil {
|
||||
t.Fatalf("write file field: %v", err)
|
||||
}
|
||||
mfw, err := mw.CreateFormField("model")
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFormField(model): %v", err)
|
||||
}
|
||||
if _, err = mfw.Write([]byte("aliased/whisper")); err != nil {
|
||||
t.Fatalf("write model field: %v", err)
|
||||
}
|
||||
if err = mw.Close(); err != nil {
|
||||
t.Fatalf("close multipart writer: %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, audioTranscriptionsPath, &buf)
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||
rec := httptest.NewRecorder()
|
||||
gw.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
_, params, err := mime.ParseMediaType(gotContentType)
|
||||
if err != nil {
|
||||
t.Fatalf("upstream Content-Type %q did not parse: %v", gotContentType, err)
|
||||
}
|
||||
mr := multipart.NewReader(bytes.NewReader(gotBody), params["boundary"])
|
||||
var gotModel string
|
||||
var gotFile []byte
|
||||
var sawModel, sawFile bool
|
||||
for {
|
||||
part, perr := mr.NextPart()
|
||||
if perr == io.EOF { //nolint:errorlint // multipart.Reader.NextPart's own contract returns io.EOF bare
|
||||
break
|
||||
}
|
||||
if perr != nil {
|
||||
t.Fatalf("parse upstream multipart body: %v", perr)
|
||||
}
|
||||
switch part.FormName() {
|
||||
case "model":
|
||||
b, _ := io.ReadAll(part)
|
||||
gotModel = string(b)
|
||||
sawModel = true
|
||||
case "file":
|
||||
b, _ := io.ReadAll(part)
|
||||
gotFile = b
|
||||
sawFile = true
|
||||
}
|
||||
_ = part.Close()
|
||||
}
|
||||
|
||||
if !sawModel || gotModel != "whisper-test" {
|
||||
t.Errorf("upstream model field = %q (present=%v), want the alias target's bare upstream id %q", gotModel, sawModel, "whisper-test")
|
||||
}
|
||||
if !sawFile || string(gotFile) != fileContent {
|
||||
t.Errorf("upstream file part = %q (present=%v), want unchanged %q", gotFile, sawFile, fileContent)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleAudioTranscriptions_OversizeBody_413 proves a multipart body
|
||||
// above maxRequestBytes is rejected with 413 (spec §3), the one media
|
||||
// route with an explicit oversize check — unlike images/speech's shared
|
||||
|
||||
+12
-3
@@ -255,10 +255,19 @@ func unifiedCostMicros(canonical, bare string, u usage, overrides map[string]*Mo
|
||||
// writeModelResolveError maps a modelRegistry.resolve error to its HTTP
|
||||
// envelope: errModelUnknown is a 404 (no configured provider knows this
|
||||
// model), errModelDenied is a 403 (a real model the caller's group cannot
|
||||
// use). Any other error is a defensive 500 — resolve's own contract
|
||||
// promises only these two sentinels, so reaching this branch would be a
|
||||
// programming error, not a client mistake.
|
||||
// use). A *aliasTargetError (spec §5, v0.2) is also a 404, but with its
|
||||
// own message naming both the alias and its unresolved target, checked
|
||||
// first via a plain type assertion — not errors.As, matching this
|
||||
// package's established yaegi-safe convention for a pointer error type
|
||||
// (registry.go's aliasTargetError doc comment). Any other error is a
|
||||
// defensive 500 — resolve's own contract promises only these sentinels/
|
||||
// types, so reaching this branch would be a programming error, not a
|
||||
// client mistake.
|
||||
func writeModelResolveError(w http.ResponseWriter, err error) {
|
||||
if aerr, ok := err.(*aliasTargetError); ok {
|
||||
writeOAIError(w, http.StatusNotFound, "invalid_request_error", aerr.Error())
|
||||
return
|
||||
}
|
||||
switch {
|
||||
case errors.Is(err, errModelUnknown):
|
||||
writeOAIError(w, http.StatusNotFound, "invalid_request_error", "unknown model")
|
||||
|
||||
@@ -533,6 +533,245 @@ func TestHandleChat_AnthropicProviderPrefixedModel_EchoesAliasInResponse(t *test
|
||||
}
|
||||
}
|
||||
|
||||
// --- model aliases end-to-end (spec §5, v0.2) ---
|
||||
|
||||
// TestHandleChat_ModelAlias_AnthropicTarget_EchoesAliasInResponse mirrors
|
||||
// TestHandleChat_AnthropicProviderPrefixedModel_EchoesAliasInResponse
|
||||
// above, but through an operator-defined modelAliases entry rather than a
|
||||
// bare provider-prefixed id: the upstream call uses the target's own
|
||||
// upstream model id, and the translated response's "model" field echoes
|
||||
// the client's exact alias string back (ruling a, ALIAS ECHO — aliases
|
||||
// inherit the existing __alias machinery for free, per registry.go's
|
||||
// resolve doc comment).
|
||||
func TestHandleChat_ModelAlias_AnthropicTarget_EchoesAliasInResponse(t *testing.T) {
|
||||
const anthResp = `{"id":"msg_01ABC","type":"message","role":"assistant","content":[{"type":"text","text":"hi there"}],"model":"claude-x","stop_reason":"end_turn","usage":{"input_tokens":10,"output_tokens":4}}`
|
||||
|
||||
var gotBody []byte
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(anthResp))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{
|
||||
"anthropic": {Type: "anthropic", BaseURL: srv.URL, APIKey: "sk-ant", Models: []string{"claude-x"}},
|
||||
}
|
||||
cfg.ModelAliases = map[string]string{"aliased/coding": "anthropic/claude-x"}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
const requestedID = "aliased/coding"
|
||||
body := map[string]any{"model": requestedID, "messages": []any{map[string]any{"role": "user", "content": "hi"}}}
|
||||
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
var out map[string]any
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
|
||||
t.Fatalf("decode response: %v", err)
|
||||
}
|
||||
if out["model"] != requestedID {
|
||||
t.Errorf("response model = %v, want the client-requested alias %q echoed back", out["model"], requestedID)
|
||||
}
|
||||
|
||||
sentUpstream := decodeJSONBody(t, gotBody)
|
||||
if sentUpstream["model"] != "claude-x" {
|
||||
t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", sentUpstream["model"], "claude-x")
|
||||
}
|
||||
if _, present := sentUpstream[gatewayAliasKey]; present {
|
||||
t.Errorf("gatewayAliasKey %q leaked into the upstream request body: %v", gatewayAliasKey, sentUpstream)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleChat_ModelAlias_OpenAITarget_UpstreamEchoNotAlias documents the
|
||||
// v0.1 passthrough asymmetry (routes_unified.go's gatewayAliasKey doc
|
||||
// comment) as it applies to aliases: an openai-type target's response is
|
||||
// forwarded verbatim, so its own "model" field carries whatever the mock
|
||||
// upstream itself returned — the target's bare upstream id, NOT the
|
||||
// alias. The gateway makes no attempt, and is not expected, to rewrite an
|
||||
// openai-type body in flight.
|
||||
func TestHandleChat_ModelAlias_OpenAITarget_UpstreamEchoNotAlias(t *testing.T) {
|
||||
const respBody = `{"id":"c1","model":"gpt-test","choices":[{"message":{"role":"assistant","content":"hi"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`
|
||||
|
||||
var gotBody []byte
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(respBody))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{
|
||||
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"gpt-test"}},
|
||||
}
|
||||
cfg.ModelAliases = map[string]string{"aliased/fast": "openai/gpt-test"}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
body := map[string]any{"model": "aliased/fast", "messages": []any{}}
|
||||
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
var out map[string]any
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
|
||||
t.Fatalf("decode response: %v", err)
|
||||
}
|
||||
if out["model"] != "gpt-test" {
|
||||
t.Errorf("response model = %v, want the openai-type passthrough's own upstream id %q, not the alias", out["model"], "gpt-test")
|
||||
}
|
||||
|
||||
sentUpstream := decodeJSONBody(t, gotBody)
|
||||
if sentUpstream["model"] != "gpt-test" {
|
||||
t.Errorf("upstream request model = %v, want bare upstream id %q", sentUpstream["model"], "gpt-test")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleEmbeddings_ModelAlias_ResolvesToTargetUpstreamModel proves
|
||||
// aliasing applies to /v1/embeddings exactly like /v1/chat/completions —
|
||||
// the upstream call uses the target's bare upstream model id.
|
||||
func TestHandleEmbeddings_ModelAlias_ResolvesToTargetUpstreamModel(t *testing.T) {
|
||||
const respBody = `{"object":"list","data":[{"embedding":[0.1]}],"usage":{"prompt_tokens":1,"total_tokens":1}}`
|
||||
|
||||
var gotBody []byte
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(respBody))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{
|
||||
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"text-embedding-3"}},
|
||||
}
|
||||
cfg.ModelAliases = map[string]string{"aliased/embed": "openai/text-embedding-3"}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
body := map[string]any{"model": "aliased/embed", "input": "hello"}
|
||||
req := newUnifiedRequest(t, http.MethodPost, "/v1/embeddings", "sk-alice", body)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
sentUpstream := decodeJSONBody(t, gotBody)
|
||||
if sentUpstream["model"] != "text-embedding-3" {
|
||||
t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", sentUpstream["model"], "text-embedding-3")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleChat_ModelAlias_AuthorizedViaAliasNameAlone_EndToEnd proves
|
||||
// spec §5's authorization rule end-to-end: a group whose models glob
|
||||
// matches only the ALIAS name (not the target, in either form) can still
|
||||
// use it through the real ServeHTTP pipeline, not just modelRegistry.resolve
|
||||
// in isolation.
|
||||
func TestHandleChat_ModelAlias_AuthorizedViaAliasNameAlone_EndToEnd(t *testing.T) {
|
||||
const respBody = `{"id":"c1","model":"gpt-test","choices":[{"message":{"role":"assistant","content":"hi"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(respBody))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{
|
||||
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"gpt-test"}},
|
||||
}
|
||||
cfg.ModelAliases = map[string]string{"aliased/coding": "openai/gpt-test"}
|
||||
// Matches only the alias name, not "openai/gpt-test" nor "gpt-test".
|
||||
cfg.Groups = map[string]*GroupConfig{"narrow": {Models: []string{"aliased/coding"}}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "narrow", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
body := map[string]any{"model": "aliased/coding", "messages": []any{}}
|
||||
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200 (authorized via the alias name alone), body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleChat_ModelAlias_UnresolvedTarget_Returns404NamingAliasAndTarget
|
||||
// covers spec §5's lazy-resolution 404 end-to-end: an alias whose target
|
||||
// names no currently-known model returns 404 with a message naming both
|
||||
// the alias and the missing target — not the generic "unknown model" text
|
||||
// a plain unresolved bare/prefixed id gets.
|
||||
func TestHandleChat_ModelAlias_UnresolvedTarget_Returns404NamingAliasAndTarget(t *testing.T) {
|
||||
cfg := CreateConfig()
|
||||
cfg.Providers = map[string]*ProviderConfig{
|
||||
"openai": {Type: "openai", APIKey: "k", Models: []string{"gpt-test"}},
|
||||
}
|
||||
cfg.ModelAliases = map[string]string{"aliased/future": "openai/not-yet-discovered"}
|
||||
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
|
||||
body := map[string]any{"model": "aliased/future", "messages": []any{}}
|
||||
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Fatalf("status = %d, want 404, body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
var out struct {
|
||||
Error struct {
|
||||
Message string `json:"message"`
|
||||
} `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
|
||||
t.Fatalf("decode response: %v", err)
|
||||
}
|
||||
if !strings.Contains(out.Error.Message, "aliased/future") {
|
||||
t.Errorf("error message = %q, want it to name the alias %q", out.Error.Message, "aliased/future")
|
||||
}
|
||||
if !strings.Contains(out.Error.Message, "openai/not-yet-discovered") {
|
||||
t.Errorf("error message = %q, want it to name the target %q", out.Error.Message, "openai/not-yet-discovered")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleChat_UpstreamNon2xx_WrapsProviderErrorEnvelope asserts a
|
||||
// non-2xx upstream response is passed through with its status code, but
|
||||
// wrapped in the gateway's own envelope with the raw upstream body
|
||||
|
||||
Reference in New Issue
Block a user