OpenAI's /v1/models schema carries no context field, so every client that
wants one reads a vendor dialect, and a client whose dialect is absent does
not ask — it substitutes its own default. GET /v1/models now emits the same
resolved integer under "context_window", "context_length" (the OpenRouter
spelling) and "max_model_len" (the vLLM spelling). All three are omitted
together when the window is unknown, exactly as the single field was.
Measured case: omp 18.1.15's openai-models-list discovery resolves
`max_model_len ?? context_length ?? <bundled catalog> ?? 128000` and never
reads context_window, so a 1048576-context local model reached the client as
128000, which also clamped its maxTokens.
The metadata fetcher already parses all of these dialects on the inbound
side (provider_openai.go), so this makes the outbound direction speak what
the inbound one already understands.
Claude-Session: https://claude.ai/code/session_0168ukk3uwp8PgiuRAzeLPXp
Add a three-state circuit breaker to providerState: closed, open, and
half-open. It is driven only by discovery-refresh outcomes, never by
request-path traffic.
A provider whose refresh fails breakerThreshold times in a row opens
the breaker and backs off before its next half-open probe, instead of
retrying every discovery interval forever. A failed probe reopens the
breaker with a doubled backoff, capped at breakerOpenMax. A successful
probe closes the breaker again, so a recovered provider never stays
dead.
The failure classification reuses limits.go's isTransient and
isDeadlineExceeded, the same rule recordProviderAttempt already
applies to request-path attempts. It does not add a second
classification.
Add Config.Breaker (BreakerConfig), validated and defaulted by
validateBreakerConfig. Every default preserves current behavior for a
provider that never fails: a closed breaker never changes the existing
interval-based refresh gating.
Export modelRegistry.providerUsable(name) as the queryable health
predicate. A sibling branch (feat/failover) will use it to skip
unusable providers before routing. This commit does not touch the
request path or routes_unified.go.
warmFill/refreshProvider used to reuse listModels' own context for the
subsequent captureModelMetadata call — a listModels attempt that took
real, non-trivial time (or timed out outright) guaranteed the metadata
fetch an already-spent, near-expired context, producing a spurious
deadline error and log line every warm fill or refresh for an otherwise
healthy provider. Both call sites now give captureModelMetadata its own
fresh context.WithTimeout budget, computed after listModels returns.
(review SHOULD-4)
captureModelMetadata also gains its own deferred recover, isolating a
panic inside fetchModelMetadata (or a future modelMetadataFetcher
implementation) from refreshProvider's OUTER recover — which would
otherwise overwrite the (already-nil, already-correct) err variable from
a successful listModels call and cause finishRefresh to discard a
freshly fetched, genuinely good model list over an unrelated bug in the
metadata path. "Metadata must never fail discovery" is now enforced on
the panic path, not just the plain-error-return one already handled.
(review SHOULD-5)
New regression tests: a fresh-timeout-budget check (a slow simulated
listModels must not shrink the metadata fetch's own deadline) and a
panic-in-fetchModelMetadata check (discovery must still succeed and
record both models).
Config gets two new surfaces (llmgateway.go): Config.ModelMeta
(map[string]*ModelMetaConfig, keyed by exact "provider/model" or a
bare model id) for operator overrides, and ProviderConfig.MetadataPath
for openai-type providers whose discovery should also capture per-model
context (LM Studio's native /api/v0/models shape).
modelmeta.go adds the layering itself: resolveModelMeta applies config
override > discovered context > builtin table > absent for context, and
config override/free > builtin > absent for cost (discovery never
reports cost). resolveAliasModelMeta implements alias inheritance: an
alias inherits its target's fully resolved metadata unless the alias
itself has its own modelMeta entry, applied per field. validateModelMeta
rejects negative values and a free:true/non-zero-cost contradiction at
construction.
registry.go wires this into the request path: providerState gains a
discoveredContext map (stale-while-error, mirroring discovered's own
convention) with contextFor/setDiscoveredContext; a new optional
modelMetadataFetcher interface, type-asserted (interpreted-to-interpreted,
documented as yaegi-safe against the Trap-4 restriction) rather than
added to providerAdapter, lets warmFill/refreshProvider capture metadata
alongside listModels without touching anthropic/gemini adapters at all.
modelObject/listFor now attach resolved context_window/pricing to every
GET /v1/models entry, including aliases (resolveMetaFor/
resolveMetaForAlias/resolveMetaForAliasName).
provider_openai.go implements modelMetadataFetcher: fetchModelMetadata
reads ProviderConfig.MetadataPath, preferring loaded_context_length over
max_context_length when the LM Studio response reports both. providers.go
wires MetadataPath onto the adapter in buildAdapters.
tools/yaegi-check/main.go adds one cheap modelMeta config-override
assertion to the existing attempt-accounting harness, proving
resolveModelMeta's config-override layer and the context_window/pricing
extension fields both run correctly under the real Yaegi interpreter.
lukaszraczylo