mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-20 05:56:56 +00:00
feat: operator-defined model aliases
Spec §5 (v0.2): Config.ModelAliases map[string]string, alias-first resolution precedence ahead of provider-prefix splitting, alias-OR-target authorization, lazy target resolution with a 404 naming both ids, /v1/models listing, admin overview alias table (JSON + dashboard), and construction-time validation (shape, prefix-shadow, explicit-model collision, chain, empty target). routes_unified.go/routes_media.go needed no resolution-call-site changes - both already pass the client's raw requested id through to registry.resolve, so aliases inherit the __alias echo and multipart rewrite machinery for free. cache.go needed no changes either - cacheKey already hashes requestedModel, so two aliases resolving to the same target already produce distinct cache entries. Adds registry_test.go validation table + precedence/authz/lazy-404/ listing coverage, a dedicated cache-key split test, chat/embeddings/ images/audio-transcriptions end-to-end tests, an extended admin overview test, and an integration test covering the openai-type vs anthropic-type echo asymmetry under real Traefik+Yaegi.
This commit is contained in:
@@ -170,6 +170,7 @@ config accepts them as YAML, which decodes to the same JSON shape.
|
|||||||
| `retry` | `RetryConfig` | `{}` (disabled) | Same-provider retry for transient upstream failures — see [Retry](#retry). Omitted or `enabled: false` means no retry: every request makes exactly one upstream attempt, byte-identical to a gateway built before this field existed. |
|
| `retry` | `RetryConfig` | `{}` (disabled) | Same-provider retry for transient upstream failures — see [Retry](#retry). Omitted or `enabled: false` means no retry: every request makes exactly one upstream attempt, byte-identical to a gateway built before this field existed. |
|
||||||
| `cache` | `CacheConfig` | `{}` (disabled) | Opt-in Redis-backed response cache for unified non-streaming chat/embeddings — see [Caching](#caching). Omitted or `enabled: false` means no caching, byte-identical to a gateway built before this field existed. |
|
| `cache` | `CacheConfig` | `{}` (disabled) | Opt-in Redis-backed response cache for unified non-streaming chat/embeddings — see [Caching](#caching). Omitted or `enabled: false` means no caching, byte-identical to a gateway built before this field existed. |
|
||||||
| `admin` | `*AdminConfig` | `nil` (disabled) | Read-only admin dashboard — see [Admin](#admin). `nil` or `enabled: false` means the `/admin*` routes are not registered at all. |
|
| `admin` | `*AdminConfig` | `nil` (disabled) | Read-only admin dashboard — see [Admin](#admin). `nil` or `enabled: false` means the `/admin*` routes are not registered at all. |
|
||||||
|
| `modelAliases` | `map[string]string` | `{}` | Operator-defined alias id → target model id — see [Model aliases](#model-aliases). Omitted or empty means no aliases, byte-identical to a gateway built before this field existed. |
|
||||||
| `passthroughUnknown` | `bool` | `false` | `false`: a request matching none of the plugin's routes gets a 404 JSON envelope. `true`: it falls through to the router's own backing service. |
|
| `passthroughUnknown` | `bool` | `false` | `false`: a request matching none of the plugin's routes gets a 404 JSON envelope. `true`: it falls through to the router's own backing service. |
|
||||||
|
|
||||||
Provider, `mcpServers`, and `agents` map **keys** (names) must match
|
Provider, `mcpServers`, and `agents` map **keys** (names) must match
|
||||||
@@ -359,6 +360,85 @@ kept — a bad edit to the file never breaks already-authenticated traffic.
|
|||||||
**stale-while-error**, never an empty list just because one refresh
|
**stale-while-error**, never an empty list just because one refresh
|
||||||
attempt failed.
|
attempt failed.
|
||||||
|
|
||||||
|
## Model aliases
|
||||||
|
|
||||||
|
Operator-defined indirection: a client-facing alias id that resolves to a
|
||||||
|
real target model id. `modelAliases` is empty by default, and an empty map
|
||||||
|
changes nothing about model resolution — this is purely additive.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
http:
|
||||||
|
middlewares:
|
||||||
|
llmgateway:
|
||||||
|
plugin:
|
||||||
|
llmgateway:
|
||||||
|
# ... providers, groups, users unchanged ...
|
||||||
|
modelAliases:
|
||||||
|
# Client-facing name -> real target. The target can itself be
|
||||||
|
# bare ("gpt-5-mini") or provider-prefixed
|
||||||
|
# ("anthropic/claude-sonnet-4-5"); either resolves through the
|
||||||
|
# normal rules in [Model routing](#model-routing) above.
|
||||||
|
aliased/coding: "anthropic/claude-sonnet-4-5"
|
||||||
|
aliased/fast: "gpt-5-mini"
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Swapping the underlying model**: point an alias at a different target
|
||||||
|
and let Traefik's dynamic-config reload apply it — no gateway restart,
|
||||||
|
no client-side change. A client that has always addressed
|
||||||
|
`aliased/coding` keeps working unmodified after the operator moves that
|
||||||
|
alias to a newer model.
|
||||||
|
- **Precedence**: an EXACT alias match wins resolution before any other
|
||||||
|
rule, including `provider/model` prefix splitting — checked first in
|
||||||
|
`modelRegistry.resolve`. The target then resolves through the same
|
||||||
|
rules as any other id (bare, `provider/model`, or a discovered id).
|
||||||
|
Because of this, an alias whose id contains a `/` is rejected at
|
||||||
|
construction if the part before the `/` names an actually configured
|
||||||
|
provider (it would be shadowed by prefix resolution) — see the
|
||||||
|
validation rules below.
|
||||||
|
- **Authorization**: a group may use an alias when its `models` glob
|
||||||
|
matches the **alias name itself**, OR when it already grants access to
|
||||||
|
the **resolved target** — either is enough. `providers` still gates the
|
||||||
|
target's own provider as usual.
|
||||||
|
- **Validation at construction** (a bad alias config fails the same way a
|
||||||
|
bad provider config does — no gateway starts with a broken alias
|
||||||
|
table):
|
||||||
|
- the alias must be non-empty and contain no control characters;
|
||||||
|
- when the alias contains `/`, its prefix must not name a configured
|
||||||
|
provider (would be shadowed by prefix resolution);
|
||||||
|
- the alias must not equal an explicit `models` entry of any provider
|
||||||
|
(ambiguous — which one wins?);
|
||||||
|
- the target must not itself be another alias (no alias chains);
|
||||||
|
- the target must not be empty.
|
||||||
|
A target naming a model that is not yet known (awaiting a provider's
|
||||||
|
first discovery fetch, or simply never configured) is accepted at
|
||||||
|
construction — it resolves lazily. An unresolvable target at request
|
||||||
|
time returns 404 with a message naming both the alias and the missing
|
||||||
|
target. A target that is later discovered and collides with an
|
||||||
|
already-configured alias id is not a construction error either — the
|
||||||
|
alias always wins that resolution, and the gateway logs one warning
|
||||||
|
per such id.
|
||||||
|
- **`GET /v1/models` listing**: every alias the caller's group may use is
|
||||||
|
listed as a model object alongside real models — `id` is the alias,
|
||||||
|
`owned_by` is the target's resolved provider name. An alias whose
|
||||||
|
target does not resolve yet is omitted from the listing until it does.
|
||||||
|
- **Echo behavior**: exactly like the existing provider-prefix alias echo
|
||||||
|
(`__alias`, [Unified vs. passthrough](#unified-vs-passthrough)) —
|
||||||
|
`anthropic`- and `gemini`-type targets build their own response
|
||||||
|
envelope and echo the client's exact alias string back in its `model`
|
||||||
|
field. An `openai`-type target's response is a **verbatim passthrough**
|
||||||
|
of the upstream body, so its `model` field carries whatever the
|
||||||
|
upstream itself returned (typically the bare upstream model id) —
|
||||||
|
**not** the alias — the same documented v0.1 passthrough asymmetry,
|
||||||
|
unchanged by this feature.
|
||||||
|
- **Applies everywhere a model routes**: unified chat and embeddings,
|
||||||
|
`/v1/images/generations`, `/v1/audio/speech`, and
|
||||||
|
`/v1/audio/transcriptions` (including its multipart rewrite — the
|
||||||
|
upstream `model` field receives the resolved target's own upstream id,
|
||||||
|
never the alias, exactly like a provider-prefixed id already does).
|
||||||
|
- **Admin overview**: `GET /admin/api/overview` includes an `aliases`
|
||||||
|
array (`{alias, target}`, sorted by alias) — see [Admin](#admin). No
|
||||||
|
secrets involved.
|
||||||
|
|
||||||
## Limits and accounting
|
## Limits and accounting
|
||||||
|
|
||||||
- Counters use **fixed windows**, keyed by UTC calendar boundaries (the
|
- Counters use **fixed windows**, keyed by UTC calendar boundaries (the
|
||||||
@@ -592,10 +672,12 @@ request to any of them falls through to the plugin's existing
|
|||||||
the `x-api-key` header on this page's own `/admin/api/*` fetches.
|
the `x-api-key` header on this page's own `/admin/api/*` fetches.
|
||||||
- **`GET /admin/api/overview`** and **`GET /admin/api/usage`** share one
|
- **`GET /admin/api/overview`** and **`GET /admin/api/usage`** share one
|
||||||
gate: unauthenticated → 401, authenticated non-admin → 403, admin →
|
gate: unauthenticated → 401, authenticated non-admin → 403, admin →
|
||||||
serve. `overview` returns the provider list, groups, redis/cache
|
serve. `overview` returns the provider list, groups, the configured
|
||||||
status, and the plugin version string. `usage` returns every user's and
|
model alias table (`alias`/`target` pairs, sorted by alias — see
|
||||||
every group's current-window counter values (req/min, req/day, tok/day,
|
[Model aliases](#model-aliases)), redis/cache status, and the plugin
|
||||||
tok/month, cost/day, cost/month) alongside their configured limits.
|
version string. `usage` returns every user's and every group's
|
||||||
|
current-window counter values (req/min, req/day, tok/day, tok/month,
|
||||||
|
cost/day, cost/month) alongside their configured limits.
|
||||||
- **What's exposed**: provider names, types, base URLs (with any
|
- **What's exposed**: provider names, types, base URLs (with any
|
||||||
userinfo/query string stripped before it's ever echoed), model counts,
|
userinfo/query string stripped before it's ever echoed), model counts,
|
||||||
discovery status, group/user names, membership, limits, and live usage
|
discovery status, group/user names, membership, limits, and live usage
|
||||||
|
|||||||
@@ -194,6 +194,14 @@ type adminGroupView struct {
|
|||||||
MemberCount int `json:"memberCount"`
|
MemberCount int `json:"memberCount"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// adminAliasView is one configured model alias's row in GET
|
||||||
|
// /admin/api/overview (spec §5, v0.2): the alias->target pair exactly as
|
||||||
|
// an operator wrote it. No secrets involved.
|
||||||
|
type adminAliasView struct {
|
||||||
|
Alias string `json:"alias"`
|
||||||
|
Target string `json:"target"`
|
||||||
|
}
|
||||||
|
|
||||||
// adminOverviewResponse is the full body of GET /admin/api/overview (spec
|
// adminOverviewResponse is the full body of GET /admin/api/overview (spec
|
||||||
// §4, v0.2). Every slice is sorted and built fresh per request — no
|
// §4, v0.2). Every slice is sorted and built fresh per request — no
|
||||||
// caching of the response itself — so the dashboard's 5s poll always
|
// caching of the response itself — so the dashboard's 5s poll always
|
||||||
@@ -202,6 +210,7 @@ type adminOverviewResponse struct {
|
|||||||
Version string `json:"version"`
|
Version string `json:"version"`
|
||||||
Providers []adminProviderView `json:"providers"`
|
Providers []adminProviderView `json:"providers"`
|
||||||
Groups []adminGroupView `json:"groups"`
|
Groups []adminGroupView `json:"groups"`
|
||||||
|
Aliases []adminAliasView `json:"aliases"`
|
||||||
Redis adminRedisView `json:"redis"`
|
Redis adminRedisView `json:"redis"`
|
||||||
Cache adminCacheView `json:"cache"`
|
Cache adminCacheView `json:"cache"`
|
||||||
}
|
}
|
||||||
@@ -237,11 +246,18 @@ func (g *Gateway) buildAdminOverview() adminOverviewResponse {
|
|||||||
groups[i] = adminGroupView{Name: gs.name, Limits: gs.limits, MemberCount: gs.memberCount}
|
groups[i] = adminGroupView{Name: gs.name, Limits: gs.limits, MemberCount: gs.memberCount}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
aliasSnaps := g.registry.aliasSnapshot()
|
||||||
|
aliases := make([]adminAliasView, len(aliasSnaps))
|
||||||
|
for i, a := range aliasSnaps {
|
||||||
|
aliases[i] = adminAliasView(a) // identical underlying field shape (alias, target string), differing only in json tags
|
||||||
|
}
|
||||||
|
|
||||||
return adminOverviewResponse{
|
return adminOverviewResponse{
|
||||||
Providers: providers,
|
Providers: providers,
|
||||||
Redis: adminRedisView{Configured: configured, LastErr: redisLastErr, LastErrAt: redisLastErrAt},
|
Redis: adminRedisView{Configured: configured, LastErr: redisLastErr, LastErrAt: redisLastErrAt},
|
||||||
Cache: adminCacheView{Enabled: cacheEnabled, TTL: ttl},
|
Cache: adminCacheView{Enabled: cacheEnabled, TTL: ttl},
|
||||||
Groups: groups,
|
Groups: groups,
|
||||||
|
Aliases: aliases,
|
||||||
Version: pluginVersion,
|
Version: pluginVersion,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -443,6 +459,13 @@ const adminPageHTML = `<!doctype html>
|
|||||||
</tr></thead><tbody></tbody></table>
|
</tr></thead><tbody></tbody></table>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<h2>Model aliases</h2>
|
||||||
|
<table id="aliases"><thead><tr>
|
||||||
|
<th>Alias</th><th>Target</th>
|
||||||
|
</tr></thead><tbody></tbody></table>
|
||||||
|
</section>
|
||||||
|
|
||||||
<section>
|
<section>
|
||||||
<h2>Infrastructure</h2>
|
<h2>Infrastructure</h2>
|
||||||
<div id="infra"></div>
|
<div id="infra"></div>
|
||||||
@@ -542,6 +565,13 @@ const adminPageHTML = `<!doctype html>
|
|||||||
return tr;
|
return tr;
|
||||||
});
|
});
|
||||||
|
|
||||||
|
setRows("aliases", data.aliases, function (a) {
|
||||||
|
var tr = document.createElement("tr");
|
||||||
|
tr.appendChild(el("td", a.alias));
|
||||||
|
tr.appendChild(el("td", a.target));
|
||||||
|
return tr;
|
||||||
|
});
|
||||||
|
|
||||||
var infra = document.getElementById("infra");
|
var infra = document.getElementById("infra");
|
||||||
while (infra.firstChild) infra.removeChild(infra.firstChild);
|
while (infra.firstChild) infra.removeChild(infra.firstChild);
|
||||||
var redisText = "Redis: " + (data.redis.configured ? "configured" : "not configured");
|
var redisText = "Redis: " + (data.redis.configured ? "configured" : "not configured");
|
||||||
|
|||||||
@@ -195,6 +195,14 @@ func assertOAIErrorEnvelope(t *testing.T, rec *httptest.ResponseRecorder) {
|
|||||||
func TestAdminOverview_SortedShapeAndVersion(t *testing.T) {
|
func TestAdminOverview_SortedShapeAndVersion(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
cfg := newAdminTestConfig()
|
cfg := newAdminTestConfig()
|
||||||
|
// Two aliases, deliberately out of sorted order, proving both the
|
||||||
|
// overview's aliases table (spec §5, v0.2) is populated AND sorted by
|
||||||
|
// alias — "zaliased/x" is a bare target ("z-model", zeta's own
|
||||||
|
// explicit model), "aaliased/y" is provider-prefixed.
|
||||||
|
cfg.ModelAliases = map[string]string{
|
||||||
|
"zaliased/x": "z-model",
|
||||||
|
"aaliased/y": "alpha/a-model-1",
|
||||||
|
}
|
||||||
h, _ := newAdminGatewayHandle(t, cfg)
|
h, _ := newAdminGatewayHandle(t, cfg)
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
@@ -247,6 +255,19 @@ func TestAdminOverview_SortedShapeAndVersion(t *testing.T) {
|
|||||||
if got.Redis.Configured {
|
if got.Redis.Configured {
|
||||||
t.Error("redis.configured must be false: cfg.Redis was never configured")
|
t.Error("redis.configured must be false: cfg.Redis was never configured")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
wantAliases := []adminAliasView{
|
||||||
|
{Alias: "aaliased/y", Target: "alpha/a-model-1"},
|
||||||
|
{Alias: "zaliased/x", Target: "z-model"},
|
||||||
|
}
|
||||||
|
if len(got.Aliases) != len(wantAliases) {
|
||||||
|
t.Fatalf("aliases = %+v, want exactly %+v", got.Aliases, wantAliases)
|
||||||
|
}
|
||||||
|
for i, want := range wantAliases {
|
||||||
|
if got.Aliases[i] != want {
|
||||||
|
t.Errorf("aliases[%d] = %+v, want %+v (sorted by alias)", i, got.Aliases[i], want)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAdminOverview_ProviderLastErrAfterFailedRefresh(t *testing.T) {
|
func TestAdminOverview_ProviderLastErrAfterFailedRefresh(t *testing.T) {
|
||||||
|
|||||||
@@ -151,6 +151,34 @@ func TestCacheKey_ChatVsEmbeddings_DistinctKeys(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestCacheKey_TwoOperatorAliases_SameTarget_DistinctKeys is task 6's
|
||||||
|
// (spec §5, v0.2) dedicated cache-key regression, phrased in terms of
|
||||||
|
// operator-defined model aliases (modelAliases) rather than
|
||||||
|
// TestCacheKey_BareVsPrefixedAlias_DistinctKeys' bare-vs-prefixed pair
|
||||||
|
// above. Two DIFFERENT alias ids ("aliased/coding", "aliased/fast")
|
||||||
|
// configured to resolve to the identical provider+upstreamModel target
|
||||||
|
// must still hash to distinct cache entries: cacheKey already hashes
|
||||||
|
// requestedModel as real key material (this file's own doc comment,
|
||||||
|
// TestCacheKey_BareVsPrefixedAlias_DistinctKeys), and
|
||||||
|
// modelRegistry.resolveAliasTarget (registry.go) leaves that string
|
||||||
|
// completely untouched regardless of which alias resolved to which
|
||||||
|
// target — the two features compose with NO cache.go changes required.
|
||||||
|
func TestCacheKey_TwoOperatorAliases_SameTarget_DistinctKeys(t *testing.T) {
|
||||||
|
req := map[string]any{"messages": []any{"hi"}}
|
||||||
|
|
||||||
|
codingAlias := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/coding", cacheEndpointChat, req)
|
||||||
|
fastAlias := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/fast", cacheEndpointChat, req)
|
||||||
|
if codingAlias == fastAlias {
|
||||||
|
t.Error("cacheKey identical for two different operator-defined aliases resolving to the identical provider+upstreamModel target")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Same alias twice must still hash identically — a genuine repeat
|
||||||
|
// request under the same alias is a genuine cache hit.
|
||||||
|
if got, want := cacheKey("anthropic", "claude-sonnet-4-5", "aliased/coding", cacheEndpointChat, req), codingAlias; got != want {
|
||||||
|
t.Errorf("cacheKey differs for two requests using the identical alias: %q vs %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// --- validateCacheConfig: defaults and validation ---
|
// --- validateCacheConfig: defaults and validation ---
|
||||||
|
|
||||||
func TestValidateCacheConfig_DefaultsAppliedWhenZero(t *testing.T) {
|
func TestValidateCacheConfig_DefaultsAppliedWhenZero(t *testing.T) {
|
||||||
|
|||||||
@@ -239,6 +239,55 @@ func TestUnifiedChatAllProviders(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestModelAliases covers v0.2 integration coverage: task 6 (spec §5).
|
||||||
|
// dynamic.yml.tmpl configures two operator-defined aliases —
|
||||||
|
// "aliased/mock" -> "gpt-mock" (openai-type) and "aliased/claude" ->
|
||||||
|
// "claude-mock" (anthropic-type) — proving alias resolution runs under
|
||||||
|
// real Traefik+Yaegi, not just go test, including the documented
|
||||||
|
// echo-behavior asymmetry between provider types (see
|
||||||
|
// registry_test.go's Gateway-level equivalents for the same assertion
|
||||||
|
// under go test).
|
||||||
|
func TestModelAliases(t *testing.T) {
|
||||||
|
// openai-type: the upstream response is forwarded verbatim
|
||||||
|
// (provider_openai.go), so its "model" field carries whatever the
|
||||||
|
// mock echoes back — the resolved upstream id "gpt-mock", not the
|
||||||
|
// alias. Correctness here is 200 + real content + the cache header
|
||||||
|
// unified routes always set, not the "model" field (documented v0.1
|
||||||
|
// passthrough asymmetry, unchanged by aliasing).
|
||||||
|
openaiReq := map[string]any{
|
||||||
|
"model": "aliased/mock",
|
||||||
|
"messages": []map[string]any{{"role": "user", "content": "hi"}},
|
||||||
|
}
|
||||||
|
resp, body := doJSON(t, http.MethodPost, traefik1URL+"/v1/chat/completions", aliceKey, openaiReq)
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
t.Fatalf("aliased openai chat: status = %d, body=%#v", resp.StatusCode, body)
|
||||||
|
}
|
||||||
|
if content := firstChoiceContent(t, body); content != "mock openai response" {
|
||||||
|
t.Errorf("aliased openai chat: content = %q, want %q", content, "mock openai response")
|
||||||
|
}
|
||||||
|
if got := resp.Header.Get("X-Llmgw-Cache"); got == "" {
|
||||||
|
t.Error("aliased openai chat: X-Llmgw-Cache header missing, want \"miss\" or \"hit\"")
|
||||||
|
}
|
||||||
|
|
||||||
|
// anthropic-type: translate_anthropic.go builds its own response
|
||||||
|
// envelope and echoes the client's exact requested id (the alias)
|
||||||
|
// back into "model" — this DOES surface the alias.
|
||||||
|
anthropicReq := map[string]any{
|
||||||
|
"model": "aliased/claude",
|
||||||
|
"messages": []map[string]any{{"role": "user", "content": "hi"}},
|
||||||
|
}
|
||||||
|
resp2, body2 := doJSON(t, http.MethodPost, traefik1URL+"/v1/chat/completions", aliceKey, anthropicReq)
|
||||||
|
if resp2.StatusCode != http.StatusOK {
|
||||||
|
t.Fatalf("aliased anthropic chat: status = %d, body=%#v", resp2.StatusCode, body2)
|
||||||
|
}
|
||||||
|
if content := firstChoiceContent(t, body2); content != "mock anthropic response" {
|
||||||
|
t.Errorf("aliased anthropic chat: content = %q, want %q", content, "mock anthropic response")
|
||||||
|
}
|
||||||
|
if model, _ := body2["model"].(string); model != "aliased/claude" {
|
||||||
|
t.Errorf("aliased anthropic chat: response model = %q, want the client-requested alias %q echoed back", model, "aliased/claude")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestStreamingIncrementalityProbe covers integration test 3.
|
// TestStreamingIncrementalityProbe covers integration test 3.
|
||||||
//
|
//
|
||||||
// Content correctness always runs, via assertStreamingContent: a host-side
|
// Content correctness always runs, via assertStreamingContent: a host-side
|
||||||
|
|||||||
@@ -98,9 +98,10 @@ http:
|
|||||||
mcpServers:
|
mcpServers:
|
||||||
tool:
|
tool:
|
||||||
url: "http://mocktool:8080"
|
url: "http://mocktool:8080"
|
||||||
# v0.2 features (spec §1, §2, §4), all opt-in and off by default
|
# v0.2 features (spec §1, §2, §4, §5), all opt-in and off by
|
||||||
# in v0.1: enabled here so the integration suite exercises every
|
# default in v0.1: enabled here so the integration suite
|
||||||
# one under real Traefik+Yaegi, not just go test.
|
# exercises every one under real Traefik+Yaegi, not just go
|
||||||
|
# test.
|
||||||
retry:
|
retry:
|
||||||
enabled: true
|
enabled: true
|
||||||
attempts: 1 # one retry after the first try — matches handleFlakyOpenAIChat's single forced 503
|
attempts: 1 # one retry after the first try — matches handleFlakyOpenAIChat's single forced 503
|
||||||
@@ -109,6 +110,15 @@ http:
|
|||||||
ttl: "2m"
|
ttl: "2m"
|
||||||
admin:
|
admin:
|
||||||
enabled: true
|
enabled: true
|
||||||
|
# Task 6 (spec §5): one alias per mock provider type, so the
|
||||||
|
# integration suite proves the echo-behavior asymmetry under
|
||||||
|
# real Traefik+Yaegi — openai-type forwards the upstream body
|
||||||
|
# verbatim (its "model" field carries "gpt-mock", not the
|
||||||
|
# alias); anthropic-type builds its own envelope and echoes the
|
||||||
|
# client's exact alias string back.
|
||||||
|
modelAliases:
|
||||||
|
aliased/mock: "gpt-mock"
|
||||||
|
aliased/claude: "claude-mock"
|
||||||
|
|
||||||
# Second, separate middleware instance carrying only the optional
|
# Second, separate middleware instance carrying only the optional
|
||||||
# real-upstream smoke test's "uni" provider (task 7, INTEGRATION_REAL=1
|
# real-upstream smoke test's "uni" provider (task 7, INTEGRATION_REAL=1
|
||||||
|
|||||||
@@ -27,6 +27,15 @@ type Config struct {
|
|||||||
// all — ServeHTTP falls through to its existing 404/passthroughUnknown
|
// all — ServeHTTP falls through to its existing 404/passthroughUnknown
|
||||||
// handling for those paths, preserving v0.1 behavior exactly.
|
// handling for those paths, preserving v0.1 behavior exactly.
|
||||||
Admin *AdminConfig `json:"admin,omitempty"`
|
Admin *AdminConfig `json:"admin,omitempty"`
|
||||||
|
// ModelAliases maps an operator-defined alias id to a target model id
|
||||||
|
// (spec §5, v0.2) — e.g. {"aliased/coding": "anthropic/claude-sonnet-4-5"}.
|
||||||
|
// An exact alias match wins resolution before any other rule
|
||||||
|
// (modelRegistry.resolve, registry.go); the target then resolves
|
||||||
|
// through the normal rules. nil/omitted preserves v0.1 behavior
|
||||||
|
// exactly: resolve never consults an empty alias map, so no existing
|
||||||
|
// model id's resolution changes. Validated at construction
|
||||||
|
// (validateModelAliases, registry.go).
|
||||||
|
ModelAliases map[string]string `json:"modelAliases,omitempty"`
|
||||||
// Retry is a struct value, not a pointer, because its own Enabled
|
// Retry is a struct value, not a pointer, because its own Enabled
|
||||||
// field is the on/off signal (unlike Redis/Users, where the block's
|
// field is the on/off signal (unlike Redis/Users, where the block's
|
||||||
// mere presence is the signal) — so its tag omits "omitempty":
|
// mere presence is the signal) — so its tag omits "omitempty":
|
||||||
|
|||||||
+258
-5
@@ -5,6 +5,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"sort"
|
"sort"
|
||||||
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
@@ -160,7 +161,16 @@ type modelRegistry struct {
|
|||||||
// specific to a field holding a func value. Verified empirically
|
// specific to a field holding a func value. Verified empirically
|
||||||
// against yaegi v0.16.1 (see tools/yaegi-check) and does not apply to
|
// against yaegi v0.16.1 (see tools/yaegi-check) and does not apply to
|
||||||
// g.logf/g.errorf, which are methods, not fields.
|
// g.logf/g.errorf, which are methods, not fields.
|
||||||
log func(string, ...any)
|
log func(string, ...any)
|
||||||
|
// aliases is the validated alias->target map from Config.ModelAliases
|
||||||
|
// (spec §5, v0.2), built once by newModelRegistry via
|
||||||
|
// validateModelAliases and never mutated afterward — resolve checks
|
||||||
|
// it first, ahead of splitConfiguredProvider/bareWinner (see resolve's
|
||||||
|
// own doc comment). nil (not merely empty) when the operator
|
||||||
|
// configured no aliases at all, so every alias-aware code path is a
|
||||||
|
// cheap nil-map lookup (always false/zero) that adds no branching
|
||||||
|
// cost to the v0.1 no-aliases case.
|
||||||
|
aliases map[string]string
|
||||||
nowFn func() time.Time
|
nowFn func() time.Time
|
||||||
warned map[string]bool
|
warned map[string]bool
|
||||||
providerNames []string
|
providerNames []string
|
||||||
@@ -187,6 +197,12 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func
|
|||||||
}
|
}
|
||||||
sort.Strings(m.providerNames)
|
sort.Strings(m.providerNames)
|
||||||
|
|
||||||
|
// explicitModels is the union of every configured provider's EXPLICIT
|
||||||
|
// model ids only (never discovered ones — discovery has not run yet
|
||||||
|
// at this point in construction) — validateModelAliases' collision
|
||||||
|
// check needs exactly this set (spec §5: "alias must not equal an
|
||||||
|
// explicit model id of any provider").
|
||||||
|
explicitModels := make(map[string]bool)
|
||||||
for _, name := range m.providerNames {
|
for _, name := range m.providerNames {
|
||||||
pc := cfg.Providers[name]
|
pc := cfg.Providers[name]
|
||||||
if pc == nil {
|
if pc == nil {
|
||||||
@@ -205,6 +221,7 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func
|
|||||||
explicit := make(map[string]bool, len(pc.Models))
|
explicit := make(map[string]bool, len(pc.Models))
|
||||||
for _, id := range pc.Models {
|
for _, id := range pc.Models {
|
||||||
explicit[id] = true
|
explicit[id] = true
|
||||||
|
explicitModels[id] = true
|
||||||
}
|
}
|
||||||
|
|
||||||
m.states[name] = &providerState{
|
m.states[name] = &providerState{
|
||||||
@@ -214,9 +231,87 @@ func newModelRegistry(adapters map[string]providerAdapter, cfg *Config, log func
|
|||||||
interval: interval,
|
interval: interval,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
aliases, err := validateModelAliases(cfg.ModelAliases, m.providerNames, explicitModels)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
m.aliases = aliases
|
||||||
|
|
||||||
return m, nil
|
return m, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// isValidModelID reports whether id satisfies spec §5's alias model-id
|
||||||
|
// shape: non-empty, and free of ASCII control characters (0x00-0x1F and
|
||||||
|
// 0x7F) — the same class of raw value mcp_a2a.go's target-URL validation
|
||||||
|
// already rejects, applied here to a model/alias id string instead of a
|
||||||
|
// URL.
|
||||||
|
func isValidModelID(id string) bool {
|
||||||
|
if id == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, r := range id {
|
||||||
|
if r < 0x20 || r == 0x7f {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// validateModelAliases validates cfg.ModelAliases per spec §5's
|
||||||
|
// validation table and returns the accepted alias->target map (nil when
|
||||||
|
// raw is empty — newGateway's default, zero-behavior-change case).
|
||||||
|
// providerNames is the registry's own sorted provider-name list (used for
|
||||||
|
// the prefix-shadow check); explicitModels is the union of every
|
||||||
|
// configured provider's EXPLICIT model ids only (used for the collision
|
||||||
|
// check) — a DISCOVERED model colliding with an alias is NOT a
|
||||||
|
// construction error (discovery is dynamic and runs after this), and is
|
||||||
|
// instead resolved in the alias's favor by resolve's own precedence; see
|
||||||
|
// resolve's warnAliasShadowsDiscoveredOnce call for the one-time runtime
|
||||||
|
// log that implies.
|
||||||
|
//
|
||||||
|
// raw is iterated in sorted alias-key order, not Go's randomized map
|
||||||
|
// order, so a config with more than one invalid alias always reports the
|
||||||
|
// same one first, deterministically, across repeated runs.
|
||||||
|
func validateModelAliases(raw map[string]string, providerNames []string, explicitModels map[string]bool) (map[string]string, error) {
|
||||||
|
if len(raw) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
providerSet := make(map[string]bool, len(providerNames))
|
||||||
|
for _, p := range providerNames {
|
||||||
|
providerSet[p] = true
|
||||||
|
}
|
||||||
|
|
||||||
|
names := make([]string, 0, len(raw))
|
||||||
|
for alias := range raw {
|
||||||
|
names = append(names, alias)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
|
||||||
|
out := make(map[string]string, len(raw))
|
||||||
|
for _, alias := range names {
|
||||||
|
target := raw[alias]
|
||||||
|
if !isValidModelID(alias) {
|
||||||
|
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q is invalid: must be non-empty with no control characters", alias)
|
||||||
|
}
|
||||||
|
if prefix, _, hasSlash := strings.Cut(alias, "/"); hasSlash && providerSet[prefix] {
|
||||||
|
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q is invalid: its prefix %q names a configured provider and would be shadowed by provider-prefix resolution", alias, prefix)
|
||||||
|
}
|
||||||
|
if explicitModels[alias] {
|
||||||
|
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q collides with an explicitly configured model id of the same name", alias)
|
||||||
|
}
|
||||||
|
if target == "" {
|
||||||
|
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q has an empty target", alias)
|
||||||
|
}
|
||||||
|
if _, targetIsAlias := raw[target]; targetIsAlias {
|
||||||
|
return nil, fmt.Errorf("llmgateway: modelAliases: alias %q targets %q, which is itself an alias (alias chains are not allowed)", alias, target)
|
||||||
|
}
|
||||||
|
out[alias] = target
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
// now returns the registry's current time, via nowFn — overridable in
|
// now returns the registry's current time, via nowFn — overridable in
|
||||||
// tests to exercise refresh throttling deterministically.
|
// tests to exercise refresh throttling deterministically.
|
||||||
func (m *modelRegistry) now() time.Time {
|
func (m *modelRegistry) now() time.Time {
|
||||||
@@ -350,6 +445,17 @@ func (m *modelRegistry) bareWinner(id string) (string, bool) {
|
|||||||
// exactly-as-requested id the caller passed in is not returned separately —
|
// exactly-as-requested id the caller passed in is not returned separately —
|
||||||
// the caller already holds it, in the id argument itself.
|
// the caller already holds it, in the id argument itself.
|
||||||
//
|
//
|
||||||
|
// An EXACT alias match (spec §5, v0.2) wins before any other rule below —
|
||||||
|
// checked first, ahead of provider-prefix splitting, so an alias id that
|
||||||
|
// happens to look like "prefix/rest" is never mistaken for one (and
|
||||||
|
// validateModelAliases already rejects an alias whose prefix names an
|
||||||
|
// actually configured provider at construction, so this ordering never
|
||||||
|
// has to arbitrate a genuine conflict between the two forms). id also
|
||||||
|
// happening to collide with a since-DISCOVERED model of some provider
|
||||||
|
// (impossible for an explicit one; validateModelAliases already rejects
|
||||||
|
// that at construction) is resolved in the alias's favor by this same
|
||||||
|
// ordering — warnAliasShadowsDiscoveredOnce logs that once per id.
|
||||||
|
//
|
||||||
// id in "provider/model" form (ruling (a): only when "provider" names a
|
// id in "provider/model" form (ruling (a): only when "provider" names a
|
||||||
// configured provider) resolves directly against that provider. Any other
|
// configured provider) resolves directly against that provider. Any other
|
||||||
// id is a bare id, resolved against the first configured provider (sorted
|
// id is a bare id, resolved against the first configured provider (sorted
|
||||||
@@ -359,14 +465,89 @@ func (m *modelRegistry) bareWinner(id string) (string, bool) {
|
|||||||
// exists but fails authorization returns errModelDenied, distinct from
|
// exists but fails authorization returns errModelDenied, distinct from
|
||||||
// errModelUnknown for a model no configured provider knows at all.
|
// errModelUnknown for a model no configured provider knows at all.
|
||||||
func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string, string, error) {
|
func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string, string, error) {
|
||||||
|
if target, isAlias := m.aliases[id]; isAlias {
|
||||||
|
if _, discoveredCollision := m.bareWinner(id); discoveredCollision {
|
||||||
|
m.warnAliasShadowsDiscoveredOnce(id)
|
||||||
|
}
|
||||||
|
return m.resolveAliasTarget(id, target, grp)
|
||||||
|
}
|
||||||
if providerName, rest, ok := m.splitConfiguredProvider(id); ok {
|
if providerName, rest, ok := m.splitConfiguredProvider(id); ok {
|
||||||
return m.resolveAgainst(providerName, rest, id, grp)
|
return m.resolveAgainst(providerName, rest, id, "", grp, errModelUnknown)
|
||||||
}
|
}
|
||||||
providerName, ok := m.bareWinner(id)
|
providerName, ok := m.bareWinner(id)
|
||||||
if !ok {
|
if !ok {
|
||||||
return nil, "", "", errModelUnknown
|
return nil, "", "", errModelUnknown
|
||||||
}
|
}
|
||||||
return m.resolveAgainst(providerName, id, id, grp)
|
return m.resolveAgainst(providerName, id, id, "", grp, errModelUnknown)
|
||||||
|
}
|
||||||
|
|
||||||
|
// aliasTargetError is resolveAgainst's notFoundErr when the provider it
|
||||||
|
// was called against, and the upstream model id on it, both came from
|
||||||
|
// resolving an alias's TARGET (resolveAliasTarget) rather than a direct
|
||||||
|
// client request — spec §5's "Targets referencing not-yet-discovered
|
||||||
|
// models are allowed at construction and resolve lazily; an unresolvable
|
||||||
|
// target at request time → 404 whose message names the alias AND the
|
||||||
|
// missing target". writeModelResolveError (routes_unified.go) type-
|
||||||
|
// asserts for *aliasTargetError explicitly, ahead of its
|
||||||
|
// errors.Is(errModelUnknown) switch — a plain type assertion, not
|
||||||
|
// errors.As, matching this package's established yaegi-safe convention
|
||||||
|
// for a pointer error type (providers.go's providerHTTPError doc
|
||||||
|
// comment: errors.As panics under yaegi checking whether an interpreted
|
||||||
|
// pointer type implements error).
|
||||||
|
type aliasTargetError struct {
|
||||||
|
alias, target string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Error names both the alias and its unresolved target, per spec §5.
|
||||||
|
func (e *aliasTargetError) Error() string {
|
||||||
|
return fmt.Sprintf("llmgateway: alias %q targets %q, which is not a known model", e.alias, e.target)
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveAliasTarget finishes resolve for id already known to be an exact
|
||||||
|
// alias match: target is resolved through the SAME rules as any other id
|
||||||
|
// — provider-prefixed direct, or bare-id collision winner — via the same
|
||||||
|
// resolveAgainst helper, with two differences from a direct request:
|
||||||
|
//
|
||||||
|
// - Authorization (spec §5): "a group may use an alias when its model
|
||||||
|
// globs match the ALIAS name OR the resolved target (either
|
||||||
|
// grants)". alias is passed to resolveAgainst as its extraModelName,
|
||||||
|
// so a request denied by every target-side candidate can still
|
||||||
|
// succeed purely because the group's model glob names the alias
|
||||||
|
// itself.
|
||||||
|
// - A target that does not resolve to any known model at request time
|
||||||
|
// returns *aliasTargetError (naming both ids), not the bare
|
||||||
|
// errModelUnknown a direct request's unknown-model case returns.
|
||||||
|
func (m *modelRegistry) resolveAliasTarget(alias, target string, grp *group) (providerAdapter, string, string, error) {
|
||||||
|
notFound := &aliasTargetError{alias: alias, target: target}
|
||||||
|
if providerName, rest, ok := m.splitConfiguredProvider(target); ok {
|
||||||
|
return m.resolveAgainst(providerName, rest, target, alias, grp, notFound)
|
||||||
|
}
|
||||||
|
providerName, ok := m.bareWinner(target)
|
||||||
|
if !ok {
|
||||||
|
return nil, "", "", notFound
|
||||||
|
}
|
||||||
|
return m.resolveAgainst(providerName, target, target, alias, grp, notFound)
|
||||||
|
}
|
||||||
|
|
||||||
|
// warnAliasShadowsDiscoveredOnce logs, at most once per colliding alias id
|
||||||
|
// for this registry's lifetime, that alias also names a model some
|
||||||
|
// provider's discovery fetch found after construction — resolve's
|
||||||
|
// documented precedence (its own doc comment above): the alias always
|
||||||
|
// wins, since only an EXPLICIT collision is rejected at construction
|
||||||
|
// (validateModelAliases); discovery is dynamic and runs after that check,
|
||||||
|
// so this collision can only be caught, and only warned about, here at
|
||||||
|
// request time. Keyed with an "alias-shadow:" prefix in the same m.warned
|
||||||
|
// map warnCollisionOnce uses, so the two unrelated warning kinds can never
|
||||||
|
// share a key and suppress each other.
|
||||||
|
func (m *modelRegistry) warnAliasShadowsDiscoveredOnce(alias string) {
|
||||||
|
m.warnedMu.Lock()
|
||||||
|
defer m.warnedMu.Unlock()
|
||||||
|
key := "alias-shadow:" + alias
|
||||||
|
if m.warned[key] {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
m.warned[key] = true
|
||||||
|
m.log("%s", fmt.Sprintf("model registry: alias %q also names a discovered model; the alias takes precedence", alias))
|
||||||
}
|
}
|
||||||
|
|
||||||
// resolveAgainst finishes resolve for a providerName already chosen (either
|
// resolveAgainst finishes resolve for a providerName already chosen (either
|
||||||
@@ -386,11 +567,29 @@ func (m *modelRegistry) resolve(id string, grp *group) (providerAdapter, string,
|
|||||||
// legitimately provider-prefixed in the first place (ruling: a pattern like
|
// legitimately provider-prefixed in the first place (ruling: a pattern like
|
||||||
// "deepseek-*" must not match a bare id that merely contains a slash, e.g.
|
// "deepseek-*" must not match a bare id that merely contains a slash, e.g.
|
||||||
// "uni/deepseek-v4-flash-0731", when "uni" is not a configured provider).
|
// "uni/deepseek-v4-flash-0731", when "uni" is not a configured provider).
|
||||||
func (m *modelRegistry) resolveAgainst(providerName, upstreamModel, requestedID string, grp *group) (providerAdapter, string, string, error) {
|
//
|
||||||
|
// extraModelName is a further authorization candidate checked via
|
||||||
|
// grp.allowsModel, alongside requestedID and upstreamModel: empty ("") for
|
||||||
|
// a direct (non-alias) resolution, or an alias's own id when resolveAliasTarget
|
||||||
|
// (spec §5, v0.2) is resolving that alias's target — "a group may use an
|
||||||
|
// alias when its model globs match the ALIAS name OR the resolved target
|
||||||
|
// (either grants)".
|
||||||
|
//
|
||||||
|
// notFoundErr is returned, unmodified, when upstreamModel is not known to
|
||||||
|
// providerName: resolve's own two call sites pass the bare errModelUnknown
|
||||||
|
// sentinel (unchanged from before this parameter existed — checked by "err
|
||||||
|
// != errModelUnknown" equality in registry_test.go, not errors.Is, so it
|
||||||
|
// must stay the exact sentinel value there); resolveAliasTarget passes a
|
||||||
|
// *aliasTargetError naming both the alias and its unresolved target (spec
|
||||||
|
// §5's 404 message requirement).
|
||||||
|
func (m *modelRegistry) resolveAgainst(providerName, upstreamModel, requestedID, extraModelName string, grp *group, notFoundErr error) (providerAdapter, string, string, error) {
|
||||||
if !m.states[providerName].hasModel(upstreamModel) {
|
if !m.states[providerName].hasModel(upstreamModel) {
|
||||||
return nil, "", "", errModelUnknown
|
return nil, "", "", notFoundErr
|
||||||
}
|
}
|
||||||
allowed := grp.allowsModel(requestedID) || grp.allowsModel(upstreamModel)
|
allowed := grp.allowsModel(requestedID) || grp.allowsModel(upstreamModel)
|
||||||
|
if extraModelName != "" {
|
||||||
|
allowed = allowed || grp.allowsModel(extraModelName)
|
||||||
|
}
|
||||||
if !allowed || !grp.allowsProvider(providerName) {
|
if !allowed || !grp.allowsProvider(providerName) {
|
||||||
return nil, "", "", errModelDenied
|
return nil, "", "", errModelDenied
|
||||||
}
|
}
|
||||||
@@ -459,6 +658,33 @@ func (m *modelRegistry) listFor(grp *group) []map[string]any {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Aliases (spec §5, v0.2): listed alongside real models only when
|
||||||
|
// both their target actually resolves right now (a target awaiting
|
||||||
|
// its provider's first discovery fetch is not listed — there is
|
||||||
|
// nothing yet to name as owned_by) and grp is authorized for it.
|
||||||
|
// resolveAliasTarget applies the exact same "alias name OR target"
|
||||||
|
// authorization rule request-time resolve uses (its own doc
|
||||||
|
// comment), so this listing can never promise access resolve would
|
||||||
|
// then deny. owned_by is recovered from the successful call's own
|
||||||
|
// canonical return value ("provider/upstreamModel") by splitting on
|
||||||
|
// the first "/" — exact, because a registry provider name can never
|
||||||
|
// itself contain one (configNamePattern, providers.go). An alias id
|
||||||
|
// that also happens to collide with a since-discovered model's own
|
||||||
|
// bare id (impossible for an EXPLICIT model — validateModelAliases
|
||||||
|
// already rejects that at construction) is not deduped against that
|
||||||
|
// model's own entry above: the same documented, alias-wins
|
||||||
|
// precedence edge resolve's warnAliasShadowsDiscoveredOnce logs
|
||||||
|
// about, accepted here too as rare enough not to warrant extra
|
||||||
|
// bookkeeping.
|
||||||
|
for alias, target := range m.aliases {
|
||||||
|
_, _, canonical, err := m.resolveAliasTarget(alias, target, grp)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
providerName, _, _ := strings.Cut(canonical, "/")
|
||||||
|
out = append(out, modelObject(alias, providerName))
|
||||||
|
}
|
||||||
|
|
||||||
sort.Slice(out, func(i, j int) bool {
|
sort.Slice(out, func(i, j int) bool {
|
||||||
return out[i]["id"].(string) < out[j]["id"].(string) //nolint:forcetypeassert // modelObject always sets id to a string
|
return out[i]["id"].(string) < out[j]["id"].(string) //nolint:forcetypeassert // modelObject always sets id to a string
|
||||||
})
|
})
|
||||||
@@ -484,6 +710,33 @@ type providerSnapshot struct {
|
|||||||
modelCount int
|
modelCount int
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// aliasSnapshotEntry is one configured alias's read-only view for the
|
||||||
|
// admin dashboard (spec §5, v0.2): the pair exactly as an operator wrote
|
||||||
|
// it. No secrets involved. The target's actual resolved provider is
|
||||||
|
// deliberately not included — a target still awaiting its provider's
|
||||||
|
// first discovery fetch has none yet, and the configured pair is the
|
||||||
|
// meaningful, stable fact an operator wants to see either way.
|
||||||
|
type aliasSnapshotEntry struct {
|
||||||
|
Alias string
|
||||||
|
Target string
|
||||||
|
}
|
||||||
|
|
||||||
|
// aliasSnapshot returns every configured alias's alias->target pair,
|
||||||
|
// sorted by alias — the admin dashboard's alias table (spec §5, v0.2).
|
||||||
|
func (m *modelRegistry) aliasSnapshot() []aliasSnapshotEntry {
|
||||||
|
names := make([]string, 0, len(m.aliases))
|
||||||
|
for alias := range m.aliases {
|
||||||
|
names = append(names, alias)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
|
||||||
|
out := make([]aliasSnapshotEntry, len(names))
|
||||||
|
for i, alias := range names {
|
||||||
|
out[i] = aliasSnapshotEntry{Alias: alias, Target: m.aliases[alias]}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
// snapshot returns every configured provider's read-only view, sorted by
|
// snapshot returns every configured provider's read-only view, sorted by
|
||||||
// name (m.providerNames is already sorted at construction) — the admin
|
// name (m.providerNames is already sorted at construction) — the admin
|
||||||
// dashboard's provider table (spec §4, v0.2).
|
// dashboard's provider table (spec §4, v0.2).
|
||||||
|
|||||||
@@ -3,9 +3,11 @@ package traefikllmgateway
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -893,3 +895,351 @@ func TestNewGateway_WarmFill_PopulatesDiscoveredModels_BeforeAnyRequest(t *testi
|
|||||||
t.Fatal("want disco-model known immediately after New returns (synchronous warm fill)")
|
t.Fatal("want disco-model known immediately after New returns (synchronous warm fill)")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- model aliases (spec §5, v0.2) ---
|
||||||
|
|
||||||
|
// newAliasTestRegistry builds a registry with one "openai" provider
|
||||||
|
// (explicit model "gpt-test") and grp's given aliases, failing the test on
|
||||||
|
// any construction error. log defaults to a no-op when nil.
|
||||||
|
func newAliasTestRegistry(t *testing.T, aliases map[string]string, log func(string, ...any)) *modelRegistry {
|
||||||
|
t.Helper()
|
||||||
|
if log == nil {
|
||||||
|
log = func(string, ...any) {}
|
||||||
|
}
|
||||||
|
adapters := map[string]providerAdapter{"openai": newFakeAdapter("openai")}
|
||||||
|
cfg := &Config{
|
||||||
|
Providers: map[string]*ProviderConfig{"openai": {Models: []string{"gpt-test"}}},
|
||||||
|
ModelAliases: aliases,
|
||||||
|
}
|
||||||
|
reg, err := newModelRegistry(adapters, cfg, log)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newModelRegistry: %v", err)
|
||||||
|
}
|
||||||
|
return reg
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNewModelRegistry_ModelAliases_ValidationTable covers every row of
|
||||||
|
// spec §5's validation table, plus a positive control.
|
||||||
|
func TestNewModelRegistry_ModelAliases_ValidationTable(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
baseProviders := map[string]*ProviderConfig{"openai": {Models: []string{"gpt-test"}}}
|
||||||
|
adapters := map[string]providerAdapter{"openai": newFakeAdapter("openai")}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
aliases map[string]string
|
||||||
|
name string
|
||||||
|
wantErrText string // substring expected in the error; "" means construction must succeed
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "empty alias id",
|
||||||
|
aliases: map[string]string{"": "gpt-test"},
|
||||||
|
wantErrText: "invalid",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "control character in alias id",
|
||||||
|
aliases: map[string]string{"bad\x7falias": "gpt-test"},
|
||||||
|
wantErrText: "invalid",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "alias prefix shadows a configured provider",
|
||||||
|
aliases: map[string]string{"openai/special": "gpt-test"},
|
||||||
|
wantErrText: "shadowed",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "alias collides with an explicit model id",
|
||||||
|
aliases: map[string]string{"gpt-test": "openai/gpt-test"},
|
||||||
|
wantErrText: "collides",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "alias chain: target is itself an alias",
|
||||||
|
aliases: map[string]string{"aliased/a": "aliased/b", "aliased/b": "openai/gpt-test"},
|
||||||
|
wantErrText: "itself an alias",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "empty target",
|
||||||
|
aliases: map[string]string{"aliased/x": ""},
|
||||||
|
wantErrText: "empty target",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "valid: bare target, no prefix collision",
|
||||||
|
aliases: map[string]string{"aliased/coding": "gpt-test"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "valid: provider-prefixed target, unconfigured prefix on the alias itself",
|
||||||
|
aliases: map[string]string{"team/coding": "openai/gpt-test"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "valid: target not yet known (lazy resolution, allowed at construction)",
|
||||||
|
aliases: map[string]string{"aliased/future": "openai/not-yet-discovered"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
cfg := &Config{Providers: baseProviders, ModelAliases: tt.aliases}
|
||||||
|
_, err := newModelRegistry(adapters, cfg, func(string, ...any) {})
|
||||||
|
if tt.wantErrText == "" {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newModelRegistry: %v, want success", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("newModelRegistry: want error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tt.wantErrText) {
|
||||||
|
t.Errorf("err = %v, want substring %q", err, tt.wantErrText)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestModelRegistry_Resolve_Alias_SlashLikeID_NotMistakenForProviderPrefix
|
||||||
|
// is the resolve-precedence regression from spec §5: an alias id
|
||||||
|
// containing "/" whose prefix names NO configured provider ("team" is not
|
||||||
|
// configured here — a prefix that IS configured is rejected at
|
||||||
|
// construction, so this is the only shape available to prove the alias
|
||||||
|
// map is checked, and wins, ahead of splitConfiguredProvider/bareWinner)
|
||||||
|
// must still resolve through the alias table, not fall through to
|
||||||
|
// errModelUnknown.
|
||||||
|
func TestModelRegistry_Resolve_Alias_SlashLikeID_NotMistakenForProviderPrefix(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"team/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
adapter, upstreamModel, canonical, err := reg.resolve("team/coding", allowAllGroup())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolve: %v", err)
|
||||||
|
}
|
||||||
|
if adapter != reg.adapters["openai"] {
|
||||||
|
t.Errorf("adapter = %v, want openai adapter", adapter)
|
||||||
|
}
|
||||||
|
if upstreamModel != "gpt-test" {
|
||||||
|
t.Errorf("upstreamModel = %q, want %q", upstreamModel, "gpt-test")
|
||||||
|
}
|
||||||
|
if canonical != "openai/gpt-test" {
|
||||||
|
t.Errorf("canonical = %q, want %q", canonical, "openai/gpt-test")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestModelRegistry_Resolve_Alias_BareTarget_ResolvesViaCollisionWinner
|
||||||
|
// proves resolveAliasTarget's bareWinner branch (a target with no
|
||||||
|
// provider prefix) applies the identical sorted-first collision rule a
|
||||||
|
// direct bare-id request already gets (ruling g).
|
||||||
|
func TestModelRegistry_Resolve_Alias_BareTarget_ResolvesViaCollisionWinner(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
adapters := map[string]providerAdapter{
|
||||||
|
"beta": newFakeAdapter("beta"),
|
||||||
|
"alpha": newFakeAdapter("alpha"),
|
||||||
|
}
|
||||||
|
cfg := &Config{
|
||||||
|
Providers: map[string]*ProviderConfig{
|
||||||
|
"beta": {Models: []string{"shared"}},
|
||||||
|
"alpha": {Models: []string{"shared"}},
|
||||||
|
},
|
||||||
|
ModelAliases: map[string]string{"aliased/x": "shared"},
|
||||||
|
}
|
||||||
|
reg, err := newModelRegistry(adapters, cfg, func(string, ...any) {})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newModelRegistry: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
adapter, _, canonical, err := reg.resolve("aliased/x", allowAllGroup())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolve: %v", err)
|
||||||
|
}
|
||||||
|
if adapter != adapters["alpha"] {
|
||||||
|
t.Errorf("adapter = %v, want alpha (sorted-first collision winner)", adapter)
|
||||||
|
}
|
||||||
|
if canonical != "alpha/shared" {
|
||||||
|
t.Errorf("canonical = %q, want %q", canonical, "alpha/shared")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- resolve: alias authorization (spec §5's "alias name OR target,
|
||||||
|
// either grants") ---
|
||||||
|
|
||||||
|
func TestModelRegistry_Resolve_Alias_AuthorizedViaAliasNameOnly(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
// Matches the alias name; matches neither "openai/gpt-test" nor its
|
||||||
|
// bare suffix "gpt-test".
|
||||||
|
grp := &group{name: "narrow", models: []string{"aliased/coding"}}
|
||||||
|
if _, _, _, err := reg.resolve("aliased/coding", grp); err != nil {
|
||||||
|
t.Errorf("resolve: %v, want success (authorized via the alias name alone)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModelRegistry_Resolve_Alias_AuthorizedViaTargetOnly(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
// Matches the target's bare suffix; does not match the alias name.
|
||||||
|
grp := &group{name: "narrow", models: []string{"gpt-*"}}
|
||||||
|
if _, _, _, err := reg.resolve("aliased/coding", grp); err != nil {
|
||||||
|
t.Errorf("resolve: %v, want success (authorized via the resolved target alone)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModelRegistry_Resolve_Alias_DeniedWhenNeitherAliasNorTargetMatch(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
grp := &group{name: "narrow", models: []string{"claude-*"}}
|
||||||
|
if _, _, _, err := reg.resolve("aliased/coding", grp); err != errModelDenied {
|
||||||
|
t.Errorf("err = %v, want errModelDenied", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModelRegistry_Resolve_Alias_ProviderDenied_OverridesModelAuthz(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
// The model glob would allow it via the alias name, but the group's
|
||||||
|
// providers list excludes "openai" entirely — allowsProvider still
|
||||||
|
// gates the resolved target's provider (spec §5's "allowsProvider
|
||||||
|
// still applies to the target's provider").
|
||||||
|
grp := &group{name: "no-openai", providers: []string{"anthropic"}, models: []string{"aliased/coding"}}
|
||||||
|
if _, _, _, err := reg.resolve("aliased/coding", grp); err != errModelDenied {
|
||||||
|
t.Errorf("err = %v, want errModelDenied (provider gate must still apply)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestModelRegistry_Resolve_Alias_UnresolvedTarget_ReturnsAliasTargetError
|
||||||
|
// is spec §5's lazy-resolution 404: a target naming no currently-known
|
||||||
|
// model returns *aliasTargetError, whose message names both the alias and
|
||||||
|
// the missing target.
|
||||||
|
func TestModelRegistry_Resolve_Alias_UnresolvedTarget_ReturnsAliasTargetError(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil)
|
||||||
|
|
||||||
|
_, _, _, err := reg.resolve("aliased/future", allowAllGroup())
|
||||||
|
var aerr *aliasTargetError
|
||||||
|
if !errors.As(err, &aerr) {
|
||||||
|
t.Fatalf("err = %v (%T), want *aliasTargetError", err, err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(aerr.Error(), "aliased/future") {
|
||||||
|
t.Errorf("error message = %q, want it to name the alias %q", aerr.Error(), "aliased/future")
|
||||||
|
}
|
||||||
|
if !strings.Contains(aerr.Error(), "openai/not-yet-discovered") {
|
||||||
|
t.Errorf("error message = %q, want it to name the target %q", aerr.Error(), "openai/not-yet-discovered")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestModelRegistry_Resolve_Alias_TargetBecomesKnown_ResolvesAfterDiscovery
|
||||||
|
// proves the lazy side of "resolve lazily": a target that was unresolvable
|
||||||
|
// at construction succeeds once its provider's discovered set catches up
|
||||||
|
// (no re-validation needed — discovery mutates providerState directly).
|
||||||
|
func TestModelRegistry_Resolve_Alias_TargetBecomesKnown_ResolvesAfterDiscovery(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil)
|
||||||
|
|
||||||
|
if _, _, _, err := reg.resolve("aliased/future", allowAllGroup()); err == nil {
|
||||||
|
t.Fatal("resolve before discovery: want an error, got nil")
|
||||||
|
}
|
||||||
|
|
||||||
|
reg.states["openai"].finishRefresh(reg.now(), []string{"not-yet-discovered"}, nil)
|
||||||
|
|
||||||
|
adapter, upstreamModel, canonical, err := reg.resolve("aliased/future", allowAllGroup())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolve after discovery: %v, want success", err)
|
||||||
|
}
|
||||||
|
if adapter != reg.adapters["openai"] || upstreamModel != "not-yet-discovered" || canonical != "openai/not-yet-discovered" {
|
||||||
|
t.Errorf("resolve after discovery = (%v, %q, %q), want (openai adapter, %q, %q)", adapter, upstreamModel, canonical, "not-yet-discovered", "openai/not-yet-discovered")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestModelRegistry_Resolve_Alias_PrecedesDiscoveredCollision_WarnsOnce
|
||||||
|
// covers the documented precedence edge (registry.go's resolve doc
|
||||||
|
// comment): an alias id that also happens to name a model discovered
|
||||||
|
// AFTER construction (impossible for an explicit model — that is a
|
||||||
|
// construction error) still resolves to the ALIAS's own target, and the
|
||||||
|
// gateway logs exactly one warning about the shadowed id, regardless of
|
||||||
|
// how many times resolve is called for it.
|
||||||
|
func TestModelRegistry_Resolve_Alias_PrecedesDiscoveredCollision_WarnsOnce(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
adapters := map[string]providerAdapter{
|
||||||
|
"openai": newFakeAdapter("openai"),
|
||||||
|
"acme": newFakeAdapter("acme"),
|
||||||
|
}
|
||||||
|
cfg := &Config{
|
||||||
|
Providers: map[string]*ProviderConfig{
|
||||||
|
"openai": {Models: []string{"gpt-test"}},
|
||||||
|
"acme": {Discovery: true},
|
||||||
|
},
|
||||||
|
ModelAliases: map[string]string{"shared-id": "openai/gpt-test"},
|
||||||
|
}
|
||||||
|
rl := &recordingLog{}
|
||||||
|
reg, err := newModelRegistry(adapters, cfg, rl.fn)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newModelRegistry: %v", err)
|
||||||
|
}
|
||||||
|
// acme's discovery finds a model literally named "shared-id" — the
|
||||||
|
// same string as the configured alias, discovered only after
|
||||||
|
// construction, so validateModelAliases never saw it.
|
||||||
|
reg.states["acme"].finishRefresh(reg.now(), []string{"shared-id"}, nil)
|
||||||
|
|
||||||
|
adapter, _, canonical, err := reg.resolve("shared-id", allowAllGroup())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolve: %v", err)
|
||||||
|
}
|
||||||
|
if adapter != adapters["openai"] {
|
||||||
|
t.Errorf("adapter = %v, want openai (the alias, not acme's discovered model, must win)", adapter)
|
||||||
|
}
|
||||||
|
if canonical != "openai/gpt-test" {
|
||||||
|
t.Errorf("canonical = %q, want %q", canonical, "openai/gpt-test")
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, _, _, err := reg.resolve("shared-id", allowAllGroup()); err != nil {
|
||||||
|
t.Fatalf("second resolve: %v", err)
|
||||||
|
}
|
||||||
|
if got := rl.count(); got != 1 {
|
||||||
|
t.Errorf("collision-shadow log emitted %d times across 2 resolve calls, want exactly 1", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- listFor: alias visibility (spec §5) ---
|
||||||
|
|
||||||
|
func TestModelRegistry_ListFor_Alias_ListedWhenAuthorized_OwnedByTargetProvider(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
got := reg.listFor(allowAllGroup())
|
||||||
|
var found bool
|
||||||
|
for _, entry := range got {
|
||||||
|
if entry["id"] == "aliased/coding" {
|
||||||
|
found = true
|
||||||
|
if entry["owned_by"] != "openai" {
|
||||||
|
t.Errorf("aliased/coding owned_by = %v, want %q", entry["owned_by"], "openai")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
t.Errorf("listFor = %v, want an entry for the alias %q", got, "aliased/coding")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModelRegistry_ListFor_Alias_UnresolvedTarget_Omitted(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/future": "openai/not-yet-discovered"}, nil)
|
||||||
|
|
||||||
|
got := reg.listFor(allowAllGroup())
|
||||||
|
for _, entry := range got {
|
||||||
|
if entry["id"] == "aliased/future" {
|
||||||
|
t.Errorf("listFor lists %q whose target does not resolve yet, want it omitted", "aliased/future")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModelRegistry_ListFor_Alias_NotAuthorized_Omitted(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
reg := newAliasTestRegistry(t, map[string]string{"aliased/coding": "openai/gpt-test"}, nil)
|
||||||
|
|
||||||
|
grp := &group{name: "narrow", models: []string{"claude-*"}}
|
||||||
|
got := reg.listFor(grp)
|
||||||
|
for _, entry := range got {
|
||||||
|
if entry["id"] == "aliased/coding" {
|
||||||
|
t.Errorf("listFor lists %q for a group authorized for neither the alias nor its target", "aliased/coding")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -166,6 +166,38 @@ func TestHandleImagesGenerations_OpenAI_HappyPath_NativeForward(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestHandleImagesGenerations_ModelAlias_ResolvesToTargetUpstreamModel
|
||||||
|
// proves an operator-defined modelAliases entry (spec §5, v0.2) routes
|
||||||
|
// /v1/images/generations exactly like a bare or provider-prefixed id: the
|
||||||
|
// upstream call carries the target's own upstream model id, never the
|
||||||
|
// alias.
|
||||||
|
func TestHandleImagesGenerations_ModelAlias_ResolvesToTargetUpstreamModel(t *testing.T) {
|
||||||
|
var gotBody map[string]any
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_ = json.NewDecoder(r.Body).Decode(&gotBody)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write([]byte(`{"created":1,"data":[{"b64_json":"xyz"}]}`))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
cfg := newMediaTestConfig(srv.URL, "img-test")
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/image": "openai/img-test"}
|
||||||
|
gw := newMediaTestGateway(t, cfg)
|
||||||
|
|
||||||
|
body := map[string]any{"model": "aliased/image", "prompt": "a cat", "n": 1}
|
||||||
|
req := newUnifiedRequest(t, http.MethodPost, imagesGenerationsPath, "sk-alice", body)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
gw.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
if gotBody["model"] != "img-test" {
|
||||||
|
t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", gotBody["model"], "img-test")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHandleImagesGenerations_StripsClientSuppliedAliasKey proves a
|
// TestHandleImagesGenerations_StripsClientSuppliedAliasKey proves a
|
||||||
// client independently including a literal "__alias" field in its own
|
// client independently including a literal "__alias" field in its own
|
||||||
// JSON body (never one the gateway itself injects for a media route —
|
// JSON body (never one the gateway itself injects for a media route —
|
||||||
@@ -726,6 +758,100 @@ func TestHandleAudioTranscriptions_PrefixedAlias_RewritesModelFieldOnly(t *testi
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestHandleAudioTranscriptions_ModelAlias_RewritesModelFieldToTargetUpstreamID
|
||||||
|
// mirrors TestHandleAudioTranscriptions_PrefixedAlias_RewritesModelFieldOnly
|
||||||
|
// above, but through an operator-defined modelAliases entry (spec §5,
|
||||||
|
// v0.2) rather than a bare provider-prefixed id: the client's "model"
|
||||||
|
// form field ("aliased/whisper") differs from the resolved upstream model
|
||||||
|
// id ("whisper-test"), so rewriteMultipartModel (routes_media.go) must
|
||||||
|
// rebuild the multipart body — every part copied verbatim except "model",
|
||||||
|
// which becomes the target's own upstream id. The alias must never reach
|
||||||
|
// the real provider, the same invariant every other unified route
|
||||||
|
// already enforces.
|
||||||
|
func TestHandleAudioTranscriptions_ModelAlias_RewritesModelFieldToTargetUpstreamID(t *testing.T) {
|
||||||
|
const fileContent = "FAKEAUDIOBYTES-DISTINCTIVE-PAYLOAD"
|
||||||
|
var gotContentType string
|
||||||
|
var gotBody []byte
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gotContentType = r.Header.Get("Content-Type")
|
||||||
|
gotBody, _ = io.ReadAll(r.Body)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write([]byte(`{"text":"ok"}`))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
cfg := newMediaTestConfig(srv.URL, "whisper-test")
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/whisper": "openai/whisper-test"}
|
||||||
|
gw := newMediaTestGateway(t, cfg)
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
mw := multipart.NewWriter(&buf)
|
||||||
|
ffw, err := mw.CreateFormFile("file", "audio.wav")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateFormFile: %v", err)
|
||||||
|
}
|
||||||
|
if _, err = ffw.Write([]byte(fileContent)); err != nil {
|
||||||
|
t.Fatalf("write file field: %v", err)
|
||||||
|
}
|
||||||
|
mfw, err := mw.CreateFormField("model")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateFormField(model): %v", err)
|
||||||
|
}
|
||||||
|
if _, err = mfw.Write([]byte("aliased/whisper")); err != nil {
|
||||||
|
t.Fatalf("write model field: %v", err)
|
||||||
|
}
|
||||||
|
if err = mw.Close(); err != nil {
|
||||||
|
t.Fatalf("close multipart writer: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodPost, audioTranscriptionsPath, &buf)
|
||||||
|
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||||
|
req.Header.Set("Authorization", "Bearer sk-alice")
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
gw.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
_, params, err := mime.ParseMediaType(gotContentType)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("upstream Content-Type %q did not parse: %v", gotContentType, err)
|
||||||
|
}
|
||||||
|
mr := multipart.NewReader(bytes.NewReader(gotBody), params["boundary"])
|
||||||
|
var gotModel string
|
||||||
|
var gotFile []byte
|
||||||
|
var sawModel, sawFile bool
|
||||||
|
for {
|
||||||
|
part, perr := mr.NextPart()
|
||||||
|
if perr == io.EOF { //nolint:errorlint // multipart.Reader.NextPart's own contract returns io.EOF bare
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if perr != nil {
|
||||||
|
t.Fatalf("parse upstream multipart body: %v", perr)
|
||||||
|
}
|
||||||
|
switch part.FormName() {
|
||||||
|
case "model":
|
||||||
|
b, _ := io.ReadAll(part)
|
||||||
|
gotModel = string(b)
|
||||||
|
sawModel = true
|
||||||
|
case "file":
|
||||||
|
b, _ := io.ReadAll(part)
|
||||||
|
gotFile = b
|
||||||
|
sawFile = true
|
||||||
|
}
|
||||||
|
_ = part.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
if !sawModel || gotModel != "whisper-test" {
|
||||||
|
t.Errorf("upstream model field = %q (present=%v), want the alias target's bare upstream id %q", gotModel, sawModel, "whisper-test")
|
||||||
|
}
|
||||||
|
if !sawFile || string(gotFile) != fileContent {
|
||||||
|
t.Errorf("upstream file part = %q (present=%v), want unchanged %q", gotFile, sawFile, fileContent)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHandleAudioTranscriptions_OversizeBody_413 proves a multipart body
|
// TestHandleAudioTranscriptions_OversizeBody_413 proves a multipart body
|
||||||
// above maxRequestBytes is rejected with 413 (spec §3), the one media
|
// above maxRequestBytes is rejected with 413 (spec §3), the one media
|
||||||
// route with an explicit oversize check — unlike images/speech's shared
|
// route with an explicit oversize check — unlike images/speech's shared
|
||||||
|
|||||||
+12
-3
@@ -255,10 +255,19 @@ func unifiedCostMicros(canonical, bare string, u usage, overrides map[string]*Mo
|
|||||||
// writeModelResolveError maps a modelRegistry.resolve error to its HTTP
|
// writeModelResolveError maps a modelRegistry.resolve error to its HTTP
|
||||||
// envelope: errModelUnknown is a 404 (no configured provider knows this
|
// envelope: errModelUnknown is a 404 (no configured provider knows this
|
||||||
// model), errModelDenied is a 403 (a real model the caller's group cannot
|
// model), errModelDenied is a 403 (a real model the caller's group cannot
|
||||||
// use). Any other error is a defensive 500 — resolve's own contract
|
// use). A *aliasTargetError (spec §5, v0.2) is also a 404, but with its
|
||||||
// promises only these two sentinels, so reaching this branch would be a
|
// own message naming both the alias and its unresolved target, checked
|
||||||
// programming error, not a client mistake.
|
// first via a plain type assertion — not errors.As, matching this
|
||||||
|
// package's established yaegi-safe convention for a pointer error type
|
||||||
|
// (registry.go's aliasTargetError doc comment). Any other error is a
|
||||||
|
// defensive 500 — resolve's own contract promises only these sentinels/
|
||||||
|
// types, so reaching this branch would be a programming error, not a
|
||||||
|
// client mistake.
|
||||||
func writeModelResolveError(w http.ResponseWriter, err error) {
|
func writeModelResolveError(w http.ResponseWriter, err error) {
|
||||||
|
if aerr, ok := err.(*aliasTargetError); ok {
|
||||||
|
writeOAIError(w, http.StatusNotFound, "invalid_request_error", aerr.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
switch {
|
switch {
|
||||||
case errors.Is(err, errModelUnknown):
|
case errors.Is(err, errModelUnknown):
|
||||||
writeOAIError(w, http.StatusNotFound, "invalid_request_error", "unknown model")
|
writeOAIError(w, http.StatusNotFound, "invalid_request_error", "unknown model")
|
||||||
|
|||||||
@@ -533,6 +533,245 @@ func TestHandleChat_AnthropicProviderPrefixedModel_EchoesAliasInResponse(t *test
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- model aliases end-to-end (spec §5, v0.2) ---
|
||||||
|
|
||||||
|
// TestHandleChat_ModelAlias_AnthropicTarget_EchoesAliasInResponse mirrors
|
||||||
|
// TestHandleChat_AnthropicProviderPrefixedModel_EchoesAliasInResponse
|
||||||
|
// above, but through an operator-defined modelAliases entry rather than a
|
||||||
|
// bare provider-prefixed id: the upstream call uses the target's own
|
||||||
|
// upstream model id, and the translated response's "model" field echoes
|
||||||
|
// the client's exact alias string back (ruling a, ALIAS ECHO — aliases
|
||||||
|
// inherit the existing __alias machinery for free, per registry.go's
|
||||||
|
// resolve doc comment).
|
||||||
|
func TestHandleChat_ModelAlias_AnthropicTarget_EchoesAliasInResponse(t *testing.T) {
|
||||||
|
const anthResp = `{"id":"msg_01ABC","type":"message","role":"assistant","content":[{"type":"text","text":"hi there"}],"model":"claude-x","stop_reason":"end_turn","usage":{"input_tokens":10,"output_tokens":4}}`
|
||||||
|
|
||||||
|
var gotBody []byte
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gotBody, _ = io.ReadAll(r.Body)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write([]byte(anthResp))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
cfg := CreateConfig()
|
||||||
|
cfg.Providers = map[string]*ProviderConfig{
|
||||||
|
"anthropic": {Type: "anthropic", BaseURL: srv.URL, APIKey: "sk-ant", Models: []string{"claude-x"}},
|
||||||
|
}
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/coding": "anthropic/claude-x"}
|
||||||
|
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||||
|
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||||
|
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||||
|
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
const requestedID = "aliased/coding"
|
||||||
|
body := map[string]any{"model": requestedID, "messages": []any{map[string]any{"role": "user", "content": "hi"}}}
|
||||||
|
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
var out map[string]any
|
||||||
|
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
|
||||||
|
t.Fatalf("decode response: %v", err)
|
||||||
|
}
|
||||||
|
if out["model"] != requestedID {
|
||||||
|
t.Errorf("response model = %v, want the client-requested alias %q echoed back", out["model"], requestedID)
|
||||||
|
}
|
||||||
|
|
||||||
|
sentUpstream := decodeJSONBody(t, gotBody)
|
||||||
|
if sentUpstream["model"] != "claude-x" {
|
||||||
|
t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", sentUpstream["model"], "claude-x")
|
||||||
|
}
|
||||||
|
if _, present := sentUpstream[gatewayAliasKey]; present {
|
||||||
|
t.Errorf("gatewayAliasKey %q leaked into the upstream request body: %v", gatewayAliasKey, sentUpstream)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleChat_ModelAlias_OpenAITarget_UpstreamEchoNotAlias documents the
|
||||||
|
// v0.1 passthrough asymmetry (routes_unified.go's gatewayAliasKey doc
|
||||||
|
// comment) as it applies to aliases: an openai-type target's response is
|
||||||
|
// forwarded verbatim, so its own "model" field carries whatever the mock
|
||||||
|
// upstream itself returned — the target's bare upstream id, NOT the
|
||||||
|
// alias. The gateway makes no attempt, and is not expected, to rewrite an
|
||||||
|
// openai-type body in flight.
|
||||||
|
func TestHandleChat_ModelAlias_OpenAITarget_UpstreamEchoNotAlias(t *testing.T) {
|
||||||
|
const respBody = `{"id":"c1","model":"gpt-test","choices":[{"message":{"role":"assistant","content":"hi"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`
|
||||||
|
|
||||||
|
var gotBody []byte
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gotBody, _ = io.ReadAll(r.Body)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write([]byte(respBody))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
cfg := CreateConfig()
|
||||||
|
cfg.Providers = map[string]*ProviderConfig{
|
||||||
|
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"gpt-test"}},
|
||||||
|
}
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/fast": "openai/gpt-test"}
|
||||||
|
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||||
|
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||||
|
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||||
|
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
body := map[string]any{"model": "aliased/fast", "messages": []any{}}
|
||||||
|
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
var out map[string]any
|
||||||
|
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
|
||||||
|
t.Fatalf("decode response: %v", err)
|
||||||
|
}
|
||||||
|
if out["model"] != "gpt-test" {
|
||||||
|
t.Errorf("response model = %v, want the openai-type passthrough's own upstream id %q, not the alias", out["model"], "gpt-test")
|
||||||
|
}
|
||||||
|
|
||||||
|
sentUpstream := decodeJSONBody(t, gotBody)
|
||||||
|
if sentUpstream["model"] != "gpt-test" {
|
||||||
|
t.Errorf("upstream request model = %v, want bare upstream id %q", sentUpstream["model"], "gpt-test")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleEmbeddings_ModelAlias_ResolvesToTargetUpstreamModel proves
|
||||||
|
// aliasing applies to /v1/embeddings exactly like /v1/chat/completions —
|
||||||
|
// the upstream call uses the target's bare upstream model id.
|
||||||
|
func TestHandleEmbeddings_ModelAlias_ResolvesToTargetUpstreamModel(t *testing.T) {
|
||||||
|
const respBody = `{"object":"list","data":[{"embedding":[0.1]}],"usage":{"prompt_tokens":1,"total_tokens":1}}`
|
||||||
|
|
||||||
|
var gotBody []byte
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gotBody, _ = io.ReadAll(r.Body)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write([]byte(respBody))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
cfg := CreateConfig()
|
||||||
|
cfg.Providers = map[string]*ProviderConfig{
|
||||||
|
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"text-embedding-3"}},
|
||||||
|
}
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/embed": "openai/text-embedding-3"}
|
||||||
|
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||||
|
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||||
|
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||||
|
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
body := map[string]any{"model": "aliased/embed", "input": "hello"}
|
||||||
|
req := newUnifiedRequest(t, http.MethodPost, "/v1/embeddings", "sk-alice", body)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want 200, body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
sentUpstream := decodeJSONBody(t, gotBody)
|
||||||
|
if sentUpstream["model"] != "text-embedding-3" {
|
||||||
|
t.Errorf("upstream request model = %v, want the alias target's bare upstream id %q", sentUpstream["model"], "text-embedding-3")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleChat_ModelAlias_AuthorizedViaAliasNameAlone_EndToEnd proves
|
||||||
|
// spec §5's authorization rule end-to-end: a group whose models glob
|
||||||
|
// matches only the ALIAS name (not the target, in either form) can still
|
||||||
|
// use it through the real ServeHTTP pipeline, not just modelRegistry.resolve
|
||||||
|
// in isolation.
|
||||||
|
func TestHandleChat_ModelAlias_AuthorizedViaAliasNameAlone_EndToEnd(t *testing.T) {
|
||||||
|
const respBody = `{"id":"c1","model":"gpt-test","choices":[{"message":{"role":"assistant","content":"hi"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write([]byte(respBody))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
cfg := CreateConfig()
|
||||||
|
cfg.Providers = map[string]*ProviderConfig{
|
||||||
|
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "k", Models: []string{"gpt-test"}},
|
||||||
|
}
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/coding": "openai/gpt-test"}
|
||||||
|
// Matches only the alias name, not "openai/gpt-test" nor "gpt-test".
|
||||||
|
cfg.Groups = map[string]*GroupConfig{"narrow": {Models: []string{"aliased/coding"}}}
|
||||||
|
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "narrow", APIKey: "sk-alice"}}}
|
||||||
|
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||||
|
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
body := map[string]any{"model": "aliased/coding", "messages": []any{}}
|
||||||
|
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want 200 (authorized via the alias name alone), body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleChat_ModelAlias_UnresolvedTarget_Returns404NamingAliasAndTarget
|
||||||
|
// covers spec §5's lazy-resolution 404 end-to-end: an alias whose target
|
||||||
|
// names no currently-known model returns 404 with a message naming both
|
||||||
|
// the alias and the missing target — not the generic "unknown model" text
|
||||||
|
// a plain unresolved bare/prefixed id gets.
|
||||||
|
func TestHandleChat_ModelAlias_UnresolvedTarget_Returns404NamingAliasAndTarget(t *testing.T) {
|
||||||
|
cfg := CreateConfig()
|
||||||
|
cfg.Providers = map[string]*ProviderConfig{
|
||||||
|
"openai": {Type: "openai", APIKey: "k", Models: []string{"gpt-test"}},
|
||||||
|
}
|
||||||
|
cfg.ModelAliases = map[string]string{"aliased/future": "openai/not-yet-discovered"}
|
||||||
|
cfg.Groups = map[string]*GroupConfig{"default": {}}
|
||||||
|
cfg.Users = &UsersConfig{Inline: []*UserConfig{{Name: "alice", Group: "default", APIKey: "sk-alice"}}}
|
||||||
|
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
|
||||||
|
h, err := New(context.Background(), next, cfg, "llmgw")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
body := map[string]any{"model": "aliased/future", "messages": []any{}}
|
||||||
|
req := newUnifiedRequest(t, http.MethodPost, "/v1/chat/completions", "sk-alice", body)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusNotFound {
|
||||||
|
t.Fatalf("status = %d, want 404, body=%s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
var out struct {
|
||||||
|
Error struct {
|
||||||
|
Message string `json:"message"`
|
||||||
|
} `json:"error"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
|
||||||
|
t.Fatalf("decode response: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(out.Error.Message, "aliased/future") {
|
||||||
|
t.Errorf("error message = %q, want it to name the alias %q", out.Error.Message, "aliased/future")
|
||||||
|
}
|
||||||
|
if !strings.Contains(out.Error.Message, "openai/not-yet-discovered") {
|
||||||
|
t.Errorf("error message = %q, want it to name the target %q", out.Error.Message, "openai/not-yet-discovered")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHandleChat_UpstreamNon2xx_WrapsProviderErrorEnvelope asserts a
|
// TestHandleChat_UpstreamNon2xx_WrapsProviderErrorEnvelope asserts a
|
||||||
// non-2xx upstream response is passed through with its status code, but
|
// non-2xx upstream response is passed through with its status code, but
|
||||||
// wrapped in the gateway's own envelope with the raw upstream body
|
// wrapped in the gateway's own envelope with the raw upstream body
|
||||||
|
|||||||
Reference in New Issue
Block a user