mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-16 05:37:28 +00:00
feat(models): emit the context window under three client dialects
OpenAI's /v1/models schema carries no context field, so every client that wants one reads a vendor dialect, and a client whose dialect is absent does not ask — it substitutes its own default. GET /v1/models now emits the same resolved integer under "context_window", "context_length" (the OpenRouter spelling) and "max_model_len" (the vLLM spelling). All three are omitted together when the window is unknown, exactly as the single field was. Measured case: omp 18.1.15's openai-models-list discovery resolves `max_model_len ?? context_length ?? <bundled catalog> ?? 128000` and never reads context_window, so a 1048576-context local model reached the client as 128000, which also clamped its maxTokens. The metadata fetcher already parses all of these dialects on the inbound side (provider_openai.go), so this makes the outbound direction speak what the inbound one already understands. Claude-Session: https://claude.ai/code/session_0168ukk3uwp8PgiuRAzeLPXp
This commit is contained in:
@@ -666,8 +666,8 @@ identical file with nothing to review.
|
||||
|
||||
### `GET /v1/models` extension fields
|
||||
|
||||
Two OpenAI-compat-safe extra fields on each model object, present only
|
||||
when known:
|
||||
OpenAI-compat-safe extra fields on each model object, present only when
|
||||
known:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -675,11 +675,19 @@ when known:
|
||||
"object": "model",
|
||||
"owned_by": "openai",
|
||||
"context_window": 400000,
|
||||
"context_length": 400000,
|
||||
"max_model_len": 400000,
|
||||
"pricing": {"input_per_mtok_usd": 1.25, "output_per_mtok_usd": 10}
|
||||
}
|
||||
```
|
||||
|
||||
- `context_window` — an integer token count, omitted when unknown.
|
||||
- `context_window`, `context_length`, `max_model_len` — the same integer
|
||||
token count under all three dialect spellings, omitted together when
|
||||
unknown. OpenAI's own schema has no context field, so each client reads
|
||||
whichever vendor dialect it was built against: `context_length` is the
|
||||
OpenRouter spelling, `max_model_len` the vLLM one. A client whose
|
||||
dialect is missing does not ask — it silently substitutes its own
|
||||
default (128k is a common one), so all three are emitted.
|
||||
- `pricing` — `{input_per_mtok_usd, output_per_mtok_usd}`, both floats
|
||||
in USD per 1,000,000 tokens. Omitted when unknown; present with `0`
|
||||
values for an explicitly free model — a known zero, not an absent
|
||||
|
||||
Reference in New Issue
Block a user