lukaszraczylo/traefik-llmgateway
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/traefik-llmgateway.git synced 2026-10-18 05:47:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(models): emit the context window under three client dialects

Browse Source 
OpenAI's /v1/models schema carries no context field, so every client that
wants one reads a vendor dialect, and a client whose dialect is absent does
not ask — it substitutes its own default. GET /v1/models now emits the same
resolved integer under "context_window", "context_length" (the OpenRouter
spelling) and "max_model_len" (the vLLM spelling). All three are omitted
together when the window is unknown, exactly as the single field was.

Measured case: omp 18.1.15's openai-models-list discovery resolves
`max_model_len ?? context_length ?? <bundled catalog> ?? 128000` and never
reads context_window, so a 1048576-context local model reached the client as
128000, which also clamped its maxTokens.

The metadata fetcher already parses all of these dialects on the inbound
side (provider_openai.go), so this makes the outbound direction speak what
the inbound one already understands.

Claude-Session: https://claude.ai/code/session_0168ukk3uwp8PgiuRAzeLPXp
This commit is contained in:
lukaszraczylo
2026-09-10 02:16:49 +01:00
parent ffa27ce831
commit 8b49bbf2a7
3 changed files with 63 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
registry.go
+19 -7
View File
@@ -1563,17 +1563,29 @@ func (m *modelRegistry) aliasEntriesFor(grp *group) map[string]map[string]any {
}
// modelObject builds one OpenAI-compatible model list entry, extended
// with two OpenAI-compat-safe extension fields (feature v0.23):
// "context_window" (int) and "pricing" ({"input_per_mtok_usd",
// "output_per_mtok_usd"} floats, USD per million tokens) — both omitted
// entirely when meta reports them unknown, never emitted as a
// misleading zero. "pricing" IS emitted with zeros for an explicitly
// free model (meta.CostKnown true, both cost fields 0) — that is a
// known, meaningful zero, not an absent one.
// with OpenAI-compat-safe extension fields (feature v0.23): the context
// window and "pricing" ({"input_per_mtok_usd", "output_per_mtok_usd"}
// floats, USD per million tokens) — both omitted entirely when meta
// reports them unknown, never emitted as a misleading zero. "pricing" IS
// emitted with zeros for an explicitly free model (meta.CostKnown true,
// both cost fields 0) — that is a known, meaningful zero, not an absent
// one.
//
// The context window is emitted under THREE keys carrying the identical
// integer: "context_window", "context_length" (the OpenRouter spelling)
// and "max_model_len" (the vLLM spelling). OpenAI's own /v1/models
// schema has no context field at all, so every client that wants one
// reads some vendor dialect, and a client whose dialect is missing
// silently substitutes its own default instead of asking. Emitting all
// three costs two map entries and removes that whole failure class —
// the same reason the metadata FETCHER already parses all of these
// dialects on the inbound side (provider_openai.go's modelMetadata).
func modelObject(id, ownedBy string, meta resolvedModelMeta) map[string]any {
obj := map[string]any{"id": id, "object": "model", "owned_by": ownedBy}
if meta.ContextKnown {
obj["context_window"] = meta.ContextTokens
obj["context_length"] = meta.ContextTokens
obj["max_model_len"] = meta.ContextTokens
}
if meta.CostKnown {
obj["pricing"] = map[string]any{