mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-12 04:54:45 +00:00
lukaszraczylo
e8398e6d49
Add tools/pricing-sync: a standalone, compiled Go tool that fetches
LiteLLM's model_prices_and_context_window.json at build time and
regenerates pricing_data_gen.go, a plain map[string]builtinModelMeta
literal keyed by bare model id (context window, input/output cost in
micro-USD per million tokens).
Pruning rule (documented on the tool's own package doc comment): keep
only entries whose litellm_provider matches this gateway's own provider
types (openai, anthropic, deepseek, xai, minimax, cerebras, dashscope
for qwen/alibaba, and moonshot for Kimi models) and whose mode is chat
or embedding, dropping everything else — a 3111-entry table is a real
startup cost under Yaegi interpretation, so the generated table stays at
236 entries. Bare-id collisions across LiteLLM keys resolve
deterministically: a bare key beats a provider-prefixed one, then
provider priority, then lexicographic order.
Naming bridges: a small, data-driven table (namingBridges) duplicates an
entry under a second bare id when a provider's own live API disagrees
with LiteLLM's naming for the identical model — seeded with MiniMax's
lightning/highspeed dual naming, verified against a live deployment.
Coverage-audit follow-up (verified against a live fetch of the same
source, 2026-08): moonshot's own "litellm_provider" tag carries 23
real Kimi entries (kimi-k2.5, kimi-k3, ...), added to the allowlist as a
first-class provider rather than a per-id bridge, since the table is
already keyed by bare id regardless of provider. Two claims from a
coverage-audit request could NOT be verified against this same fetched
file and were deliberately NOT implemented: no key anywhere in the
3111-entry source ends in ":free" (an openrouter free-tier naming
convention the request assumed exists in this file), and no
"qwen/"-prefixed key exists at all — see the report for the full
grounding note. Also confirmed already-correct by design, no change
needed: audio/responses-mode OpenAI entries ("codex-mini-latest" and
friends) are excluded by the existing mode allowlist, not by any
provider special-case; image/video media models are excluded the same
way.
builtinModelMeta itself lives in its own file (builtin_model_meta.go),
separate from modelmeta.go's resolution logic, so this commit's table
and generator compile standalone.
make pricing-sync regenerates the table; add its gitignore entry for the
tool's own compiled binary.
26 lines
1.2 KiB
Go
26 lines
1.2 KiB
Go
package traefikllmgateway
|
|
|
|
// builtinModelMeta is one built-in, LiteLLM-synced catalog entry (feature
|
|
// v0.23): a model's context window and per-token cost, in the same
|
|
// micro-USD-per-million-tokens unit ModelMetaConfig uses (modelmeta.go's
|
|
// microUSDPerUSD). Zero in any field means "this table has no data for
|
|
// it" — resolveModelMeta (modelmeta.go) only ever trusts a field greater
|
|
// than zero, never a bare zero value, so there is no ambiguity with a
|
|
// genuinely free or zero-context entry (this table never has one:
|
|
// pricing_data_gen.go's generator only ever emits a field it found a
|
|
// positive upstream value for). Regenerated wholesale by `make
|
|
// pricing-sync` (tools/pricing-sync) — see pricing_data_gen.go's own
|
|
// header for the source snapshot and pruning rule, and
|
|
// tools/pricing-sync/main.go's own doc comment for the full pruning/
|
|
// naming-bridge design.
|
|
//
|
|
// Kept in its own file, separate from modelmeta.go's resolution logic:
|
|
// this type is the generated table's own data shape, needed to compile
|
|
// pricing_data_gen.go on its own, independent of the resolveModelMeta/
|
|
// registry wiring layered on top of it.
|
|
type builtinModelMeta struct {
|
|
ContextTokens int
|
|
InputCostPerMTokMicroUSD int64
|
|
OutputCostPerMTokMicroUSD int64
|
|
}
|