mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-17 05:37:43 +00:00
feat(models): emit the context window under three client dialects
OpenAI's /v1/models schema carries no context field, so every client that wants one reads a vendor dialect, and a client whose dialect is absent does not ask — it substitutes its own default. GET /v1/models now emits the same resolved integer under "context_window", "context_length" (the OpenRouter spelling) and "max_model_len" (the vLLM spelling). All three are omitted together when the window is unknown, exactly as the single field was. Measured case: omp 18.1.15's openai-models-list discovery resolves `max_model_len ?? context_length ?? <bundled catalog> ?? 128000` and never reads context_window, so a 1048576-context local model reached the client as 128000, which also clamped its maxTokens. The metadata fetcher already parses all of these dialects on the inbound side (provider_openai.go), so this makes the outbound direction speak what the inbound one already understands. Claude-Session: https://claude.ai/code/session_0168ukk3uwp8PgiuRAzeLPXp
This commit is contained in:
@@ -375,3 +375,36 @@ func TestModelRegistry_RefreshProvider_MetadataPanicDoesNotFailDiscovery(t *test
|
||||
t.Error("want at least one log line recording the metadata-capture panic")
|
||||
}
|
||||
}
|
||||
|
||||
// TestModelObject_ContextDialectMirrors proves every model object carries its
|
||||
// context window under all three dialect keys OpenAI-compatible clients
|
||||
// actually read — "context_window", "context_length" and "max_model_len" —
|
||||
// always the identical integer, and that all three vanish together when the
|
||||
// window is unknown.
|
||||
//
|
||||
// Client-side reason this exists (2026-09-10): omp's openai-models-list
|
||||
// discovery reads `max_model_len ?? context_length ?? <bundled catalog> ??
|
||||
// 128000` and never looks at context_window, so a 1M-context local model was
|
||||
// advertised to it as the 128000 default. Other clients read the OpenRouter
|
||||
// spelling (context_length). One resolved value, three spellings, no
|
||||
// per-client configuration.
|
||||
//
|
||||
// Mutation that must make this test fail: drop either mirror key from
|
||||
// modelObject, or let one carry a different value from context_window.
|
||||
func TestModelObject_ContextDialectMirrors(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
known := modelObject("m", "prov", resolvedModelMeta{ContextTokens: 1048576, ContextKnown: true})
|
||||
for _, key := range []string{"context_window", "context_length", "max_model_len"} {
|
||||
if known[key] != 1048576 {
|
||||
t.Errorf("known[%q] = %v, want 1048576", key, known[key])
|
||||
}
|
||||
}
|
||||
|
||||
unknown := modelObject("m", "prov", resolvedModelMeta{})
|
||||
for _, key := range []string{"context_window", "context_length", "max_model_len"} {
|
||||
if v, present := unknown[key]; present {
|
||||
t.Errorf("unknown[%q] present (%v), want omitted entirely", key, v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user