feat(registry): model metadata capture and resolution

Config gets two new surfaces (llmgateway.go): Config.ModelMeta
(map[string]*ModelMetaConfig, keyed by exact "provider/model" or a
bare model id) for operator overrides, and ProviderConfig.MetadataPath
for openai-type providers whose discovery should also capture per-model
context (LM Studio's native /api/v0/models shape).

modelmeta.go adds the layering itself: resolveModelMeta applies config
override > discovered context > builtin table > absent for context, and
config override/free > builtin > absent for cost (discovery never
reports cost). resolveAliasModelMeta implements alias inheritance: an
alias inherits its target's fully resolved metadata unless the alias
itself has its own modelMeta entry, applied per field. validateModelMeta
rejects negative values and a free:true/non-zero-cost contradiction at
construction.

registry.go wires this into the request path: providerState gains a
discoveredContext map (stale-while-error, mirroring discovered's own
convention) with contextFor/setDiscoveredContext; a new optional
modelMetadataFetcher interface, type-asserted (interpreted-to-interpreted,
documented as yaegi-safe against the Trap-4 restriction) rather than
added to providerAdapter, lets warmFill/refreshProvider capture metadata
alongside listModels without touching anthropic/gemini adapters at all.
modelObject/listFor now attach resolved context_window/pricing to every
GET /v1/models entry, including aliases (resolveMetaFor/
resolveMetaForAlias/resolveMetaForAliasName).

provider_openai.go implements modelMetadataFetcher: fetchModelMetadata
reads ProviderConfig.MetadataPath, preferring loaded_context_length over
max_context_length when the LM Studio response reports both. providers.go
wires MetadataPath onto the adapter in buildAdapters.

tools/yaegi-check/main.go adds one cheap modelMeta config-override
assertion to the existing attempt-accounting harness, proving
resolveModelMeta's config-override layer and the context_window/pricing
extension fields both run correctly under the real Yaegi interpreter.
This commit is contained in:
2026-08-22 01:35:21 +01:00
parent e8398e6d49
commit 0c61296c73
10 changed files with 1331 additions and 15 deletions
+50 -6
View File
@@ -36,6 +36,20 @@ type Config struct {
// model id's resolution changes. Validated at construction
// (validateModelAliases, registry.go).
ModelAliases map[string]string `json:"modelAliases,omitempty"`
// ModelMeta declares per-model metadata overrides (feature v0.23):
// context window size and per-token cost, keyed by an exact
// "provider/model" id or a bare model id (applying wherever that bare
// id resolves — modelmeta.go's lookupModelMetaConfig checks the exact
// form first). This ALWAYS wins over discovery-captured context or
// the built-in LiteLLM-synced table (resolveModelMeta, modelmeta.go)
// — distinct from Pricing above, which drives request cost
// ACCOUNTING (costMicros); ModelMeta drives metadata EXPOSURE
// (GET /v1/models' context_window/pricing extension fields, the
// admin dashboard) and never affects billing. nil/omitted preserves
// prior behavior exactly: every model's metadata falls through to
// discovery/builtin/absent, same as before this feature existed.
// Validated at construction (validateModelMeta, modelmeta.go).
ModelMeta map[string]*ModelMetaConfig `json:"modelMeta,omitempty"`
// Retry is a struct value, not a pointer, because its own Enabled
// field is the on/off signal (unlike Redis/Users, where the block's
// mere presence is the signal) — so its tag omits "omitempty":
@@ -51,12 +65,23 @@ type Config struct {
// ProviderConfig describes one upstream LLM provider.
type ProviderConfig struct {
Type string `json:"type"`
BaseURL string `json:"baseUrl,omitempty"`
APIKey string `json:"apiKey"`
DiscoveryInterval string `json:"discoveryInterval,omitempty"`
Models []string `json:"models,omitempty"`
Discovery bool `json:"discovery,omitempty"`
Type string `json:"type"`
BaseURL string `json:"baseUrl,omitempty"`
APIKey string `json:"apiKey"`
DiscoveryInterval string `json:"discoveryInterval,omitempty"`
// MetadataPath is an optional second discovery endpoint (feature
// v0.23), fetched alongside the provider's normal listModels call
// when set: a path such as "/api/v0/models" (LM Studio's own,
// non-OpenAI-compatible models endpoint) that reports per-model
// context length. Only openai-type adapters read this field
// (provider_openai.go's openaiAdapter.fetchModelMetadata) — an
// anthropic/gemini provider configuring it has no effect. Empty (the
// default) captures no metadata, preserving prior behavior exactly.
// A failed or absent fetch is always non-fatal to discovery itself:
// no metadata is captured, nothing else changes.
MetadataPath string `json:"metadataPath,omitempty"`
Models []string `json:"models,omitempty"`
Discovery bool `json:"discovery,omitempty"`
}
// GroupConfig describes a group's access and limits. Empty/omitted
@@ -183,6 +208,25 @@ type ModelPricing struct {
OutputPerM float64 `json:"outputPerM"`
}
// ModelMetaConfig overrides one model's context window and per-token
// cost (feature v0.23, Config.ModelMeta) — layered ahead of discovery-
// captured context and the built-in LiteLLM-synced table
// (resolveModelMeta, modelmeta.go). ContextTokens left at 0 falls
// through to the next layer rather than overriding with an explicit
// unknown; the same holds for InputCostPerMTokMicroUSD/
// OutputCostPerMTokMicroUSD when Free is false. Free, when true,
// explicitly zeroes both cost fields regardless of whatever they are
// set to — a KNOWN zero cost, distinct from the "not overridden here"
// fallback a zero cost field means when Free is false. Setting Free
// true together with a non-zero cost field is rejected at construction
// (validateModelMeta) as a contradictory config.
type ModelMetaConfig struct {
ContextTokens int `json:"contextTokens,omitempty"`
InputCostPerMTokMicroUSD int64 `json:"inputCostPerMTokMicroUsd,omitempty"`
OutputCostPerMTokMicroUSD int64 `json:"outputCostPerMTokMicroUsd,omitempty"`
Free bool `json:"free,omitempty"`
}
// TargetConfig is a proxied upstream target (MCP server).
type TargetConfig struct {
URL string `json:"url"`