mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-13 04:58:50 +00:00
feat(pricing): litellm sync generator + table
Add tools/pricing-sync: a standalone, compiled Go tool that fetches
LiteLLM's model_prices_and_context_window.json at build time and
regenerates pricing_data_gen.go, a plain map[string]builtinModelMeta
literal keyed by bare model id (context window, input/output cost in
micro-USD per million tokens).
Pruning rule (documented on the tool's own package doc comment): keep
only entries whose litellm_provider matches this gateway's own provider
types (openai, anthropic, deepseek, xai, minimax, cerebras, dashscope
for qwen/alibaba, and moonshot for Kimi models) and whose mode is chat
or embedding, dropping everything else — a 3111-entry table is a real
startup cost under Yaegi interpretation, so the generated table stays at
236 entries. Bare-id collisions across LiteLLM keys resolve
deterministically: a bare key beats a provider-prefixed one, then
provider priority, then lexicographic order.
Naming bridges: a small, data-driven table (namingBridges) duplicates an
entry under a second bare id when a provider's own live API disagrees
with LiteLLM's naming for the identical model — seeded with MiniMax's
lightning/highspeed dual naming, verified against a live deployment.
Coverage-audit follow-up (verified against a live fetch of the same
source, 2026-08): moonshot's own "litellm_provider" tag carries 23
real Kimi entries (kimi-k2.5, kimi-k3, ...), added to the allowlist as a
first-class provider rather than a per-id bridge, since the table is
already keyed by bare id regardless of provider. Two claims from a
coverage-audit request could NOT be verified against this same fetched
file and were deliberately NOT implemented: no key anywhere in the
3111-entry source ends in ":free" (an openrouter free-tier naming
convention the request assumed exists in this file), and no
"qwen/"-prefixed key exists at all — see the report for the full
grounding note. Also confirmed already-correct by design, no change
needed: audio/responses-mode OpenAI entries ("codex-mini-latest" and
friends) are excluded by the existing mode allowlist, not by any
provider special-case; image/video media models are excluded the same
way.
builtinModelMeta itself lives in its own file (builtin_model_meta.go),
separate from modelmeta.go's resolution logic, so this commit's table
and generator compile standalone.
make pricing-sync regenerates the table; add its gitignore entry for the
tool's own compiled binary.
This commit is contained in:
@@ -5,6 +5,7 @@
|
||||
docs/superpowers/
|
||||
*.wasm
|
||||
tools/yaegi-check/llmgw-yaegi-check
|
||||
tools/pricing-sync/llmgw-pricing-sync
|
||||
dist/
|
||||
|
||||
# webui/'s build output (dist/, matched above) and npm dependency tree —
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
# two of them concurrently would corrupt each other's containers.
|
||||
.NOTPARALLEL:
|
||||
|
||||
.PHONY: test lint yaegi-check admin-ui integration integration-keep integration-up integration-wait integration-down
|
||||
.PHONY: test lint yaegi-check admin-ui pricing-sync integration integration-keep integration-up integration-wait integration-down
|
||||
|
||||
test:
|
||||
go test ./... -count=1
|
||||
@@ -32,6 +32,18 @@ admin-ui:
|
||||
yaegi-check:
|
||||
cd tools/yaegi-check && GOWORK=off go run . $(CURDIR)
|
||||
|
||||
# pricing-sync regenerates the repo-root pricing_data_gen.go: the
|
||||
# built-in, LiteLLM-synced per-model context/cost table feature v0.23's
|
||||
# metadata resolution falls back to (modelmeta.go's resolveModelMeta).
|
||||
# Fetches LiteLLM's model_prices_and_context_window.json at BUILD TIME
|
||||
# only — see tools/pricing-sync/main.go's own doc comment for the
|
||||
# pruning rule and naming-bridge mechanism. Not run by CI (network
|
||||
# fetch, and the generated table is committed); an operator re-runs this
|
||||
# by hand when LiteLLM's own pricing drifts.
|
||||
pricing-sync:
|
||||
cd tools/pricing-sync && GOWORK=off go run . $(CURDIR)
|
||||
gofmt -w pricing_data_gen.go
|
||||
|
||||
INTEGRATION_COMPOSE := integration/docker-compose.yml
|
||||
INTEGRATION_HEALTH_URL := http://localhost:19081/v1/models
|
||||
INTEGRATION_ALICE_KEY := sk-int-alice
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
package traefikllmgateway
|
||||
|
||||
// builtinModelMeta is one built-in, LiteLLM-synced catalog entry (feature
|
||||
// v0.23): a model's context window and per-token cost, in the same
|
||||
// micro-USD-per-million-tokens unit ModelMetaConfig uses (modelmeta.go's
|
||||
// microUSDPerUSD). Zero in any field means "this table has no data for
|
||||
// it" — resolveModelMeta (modelmeta.go) only ever trusts a field greater
|
||||
// than zero, never a bare zero value, so there is no ambiguity with a
|
||||
// genuinely free or zero-context entry (this table never has one:
|
||||
// pricing_data_gen.go's generator only ever emits a field it found a
|
||||
// positive upstream value for). Regenerated wholesale by `make
|
||||
// pricing-sync` (tools/pricing-sync) — see pricing_data_gen.go's own
|
||||
// header for the source snapshot and pruning rule, and
|
||||
// tools/pricing-sync/main.go's own doc comment for the full pruning/
|
||||
// naming-bridge design.
|
||||
//
|
||||
// Kept in its own file, separate from modelmeta.go's resolution logic:
|
||||
// this type is the generated table's own data shape, needed to compile
|
||||
// pricing_data_gen.go on its own, independent of the resolveModelMeta/
|
||||
// registry wiring layered on top of it.
|
||||
type builtinModelMeta struct {
|
||||
ContextTokens int
|
||||
InputCostPerMTokMicroUSD int64
|
||||
OutputCostPerMTokMicroUSD int64
|
||||
}
|
||||
@@ -0,0 +1,249 @@
|
||||
package traefikllmgateway
|
||||
|
||||
// Code generated by tools/pricing-sync from LiteLLM's
|
||||
// model_prices_and_context_window.json — DO NOT EDIT.
|
||||
//
|
||||
// Source: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json
|
||||
// Snapshot: 2026-08-22, 236 entries kept (pruned from 3111 upstream entries —
|
||||
// see tools/pricing-sync/main.go's own doc comment for the exact pruning
|
||||
// rule and bare-id collision resolution).
|
||||
//
|
||||
// Regenerate with: make pricing-sync
|
||||
var builtinModelMetaTable = map[string]builtinModelMeta{
|
||||
"MiniMax-M2": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"MiniMax-M2.1": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"MiniMax-M2.1-highspeed": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"MiniMax-M2.1-lightning": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"MiniMax-M2.5": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"MiniMax-M2.5-highspeed": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"MiniMax-M2.5-lightning": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"MiniMax-M3": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"chat-latest": {ContextTokens: 400000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"chatgpt-4o-latest": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-3-7-sonnet-20250219": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-3-haiku-20240307": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 250000, OutputCostPerMTokMicroUSD: 1250000},
|
||||
"claude-3-opus-20240229": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"claude-4-opus-20250514": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"claude-4-sonnet-20250514": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-fable-5": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 50000000},
|
||||
"claude-haiku-4-5": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"claude-haiku-4-5-20251001": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"claude-mythos-5": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 50000000},
|
||||
"claude-mythos-preview": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 50000000},
|
||||
"claude-opus-4-1": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"claude-opus-4-1-20250805": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"claude-opus-4-20250514": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"claude-opus-4-5": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-4-5-20251101": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-4-6": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-4-6-20260205": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-4-7": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-4-7-20260416": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-4-8": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-opus-5": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"claude-sonnet-4-20250514": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-sonnet-4-5": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-sonnet-4-5-20250929": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-sonnet-4-6": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"claude-sonnet-5": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"daybreak-blue-latest": {ContextTokens: 1050000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"daybreak-red-latest": {ContextTokens: 400000, InputCostPerMTokMicroUSD: 12500000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"deepseek-chat": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 280000, OutputCostPerMTokMicroUSD: 420000},
|
||||
"deepseek-coder": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 140000, OutputCostPerMTokMicroUSD: 280000},
|
||||
"deepseek-r1": {ContextTokens: 65536, InputCostPerMTokMicroUSD: 550000, OutputCostPerMTokMicroUSD: 2190000},
|
||||
"deepseek-reasoner": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 280000, OutputCostPerMTokMicroUSD: 420000},
|
||||
"deepseek-v3": {ContextTokens: 65536, InputCostPerMTokMicroUSD: 270000, OutputCostPerMTokMicroUSD: 1100000},
|
||||
"deepseek-v3.2": {ContextTokens: 163840, InputCostPerMTokMicroUSD: 280000, OutputCostPerMTokMicroUSD: 400000},
|
||||
"deepseek-v4-flash": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 440000, OutputCostPerMTokMicroUSD: 1320000},
|
||||
"deepseek-v4-flash-0731": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 400000},
|
||||
"deepseek-v4-pro": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1320000, OutputCostPerMTokMicroUSD: 3960000},
|
||||
"ft:gpt-3.5-turbo": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"ft:gpt-3.5-turbo-0125": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"ft:gpt-3.5-turbo-0613": {ContextTokens: 4096, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"ft:gpt-3.5-turbo-1106": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"ft:gpt-4-0613": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 30000000, OutputCostPerMTokMicroUSD: 60000000},
|
||||
"ft:gpt-4.1-2025-04-14": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 12000000},
|
||||
"ft:gpt-4.1-mini-2025-04-14": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 800000, OutputCostPerMTokMicroUSD: 3200000},
|
||||
"ft:gpt-4.1-nano-2025-04-14": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 800000},
|
||||
"ft:gpt-4o-2024-08-06": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 3750000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"ft:gpt-4o-2024-11-20": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 3750000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"ft:gpt-4o-mini-2024-07-18": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"ft:o4-mini-2025-04-16": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 4000000, OutputCostPerMTokMicroUSD: 16000000},
|
||||
"glm-5.1": {ContextTokens: 202745, InputCostPerMTokMicroUSD: 1400000, OutputCostPerMTokMicroUSD: 4400000},
|
||||
"glm-5.2": {ContextTokens: 1048576, InputCostPerMTokMicroUSD: 1400000, OutputCostPerMTokMicroUSD: 4400000},
|
||||
"gpt-3.5-turbo": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 500000, OutputCostPerMTokMicroUSD: 1500000},
|
||||
"gpt-3.5-turbo-0125": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 500000, OutputCostPerMTokMicroUSD: 1500000},
|
||||
"gpt-3.5-turbo-1106": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"gpt-3.5-turbo-16k": {ContextTokens: 16385, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"gpt-4": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 30000000, OutputCostPerMTokMicroUSD: 60000000},
|
||||
"gpt-4-0125-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-4-0314": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 30000000, OutputCostPerMTokMicroUSD: 60000000},
|
||||
"gpt-4-0613": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 30000000, OutputCostPerMTokMicroUSD: 60000000},
|
||||
"gpt-4-1106-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-4-turbo": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-4-turbo-2024-04-09": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-4-turbo-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 10000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-4.1": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 8000000},
|
||||
"gpt-4.1-2025-04-14": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 8000000},
|
||||
"gpt-4.1-mini": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1600000},
|
||||
"gpt-4.1-mini-2025-04-14": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1600000},
|
||||
"gpt-4.1-nano": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 100000, OutputCostPerMTokMicroUSD: 400000},
|
||||
"gpt-4.1-nano-2025-04-14": {ContextTokens: 1047576, InputCostPerMTokMicroUSD: 100000, OutputCostPerMTokMicroUSD: 400000},
|
||||
"gpt-4o": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-2024-05-13": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"gpt-4o-2024-08-06": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-2024-11-20": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-audio-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-audio-preview-2024-12-17": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-audio-preview-2025-06-03": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-mini": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"gpt-4o-mini-2024-07-18": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"gpt-4o-mini-audio-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"gpt-4o-mini-audio-preview-2024-12-17": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"gpt-4o-mini-search-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"gpt-4o-mini-search-preview-2025-03-11": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"gpt-4o-search-preview": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-4o-search-preview-2025-03-11": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5-2025-08-07": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5-chat": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5-chat-latest": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5-mini": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 250000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"gpt-5-mini-2025-08-07": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 250000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"gpt-5-nano": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 50000, OutputCostPerMTokMicroUSD: 400000},
|
||||
"gpt-5-nano-2025-08-07": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 50000, OutputCostPerMTokMicroUSD: 400000},
|
||||
"gpt-5-search-api": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5-search-api-2025-10-14": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5.1": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5.1-2025-11-13": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5.1-chat-latest": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-5.2": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1750000, OutputCostPerMTokMicroUSD: 14000000},
|
||||
"gpt-5.2-2025-12-11": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 1750000, OutputCostPerMTokMicroUSD: 14000000},
|
||||
"gpt-5.2-chat-latest": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 1750000, OutputCostPerMTokMicroUSD: 14000000},
|
||||
"gpt-5.3-chat-latest": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 1750000, OutputCostPerMTokMicroUSD: 14000000},
|
||||
"gpt-5.4": {ContextTokens: 1050000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"gpt-5.4-2026-03-05": {ContextTokens: 1050000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"gpt-5.4-mini": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 750000, OutputCostPerMTokMicroUSD: 4500000},
|
||||
"gpt-5.4-mini-2026-03-17": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 750000, OutputCostPerMTokMicroUSD: 4500000},
|
||||
"gpt-5.4-nano": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 1250000},
|
||||
"gpt-5.4-nano-2026-03-17": {ContextTokens: 272000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 1250000},
|
||||
"gpt-5.5": {ContextTokens: 1050000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-5.5-2026-04-23": {ContextTokens: 1050000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-5.6": {ContextTokens: 922000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-5.6-cyber": {ContextTokens: 400000, InputCostPerMTokMicroUSD: 12500000, OutputCostPerMTokMicroUSD: 75000000},
|
||||
"gpt-5.6-luna": {ContextTokens: 922000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"gpt-5.6-sol": {ContextTokens: 922000, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 30000000},
|
||||
"gpt-5.6-terra": {ContextTokens: 922000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 12000000},
|
||||
"gpt-audio": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-audio-1.5": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-audio-2025-08-28": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"gpt-audio-mini": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"gpt-audio-mini-2025-10-06": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"gpt-audio-mini-2025-12-15": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"gpt-oss-120b": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 350000, OutputCostPerMTokMicroUSD: 750000},
|
||||
"grok-2": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"grok-2-1212": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"grok-2-latest": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"grok-2-vision": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"grok-2-vision-1212": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"grok-2-vision-latest": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 10000000},
|
||||
"grok-3": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-3-beta": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-3-fast-beta": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"grok-3-fast-latest": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 25000000},
|
||||
"grok-3-latest": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-3-mini": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-3-mini-beta": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-3-mini-fast": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"grok-3-mini-fast-beta": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"grok-3-mini-fast-latest": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"grok-3-mini-latest": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-4-0709": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-4-1-fast": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-1-fast-non-reasoning": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-1-fast-non-reasoning-latest": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-1-fast-reasoning": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-1-fast-reasoning-latest": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-fast-non-reasoning": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-fast-reasoning": {ContextTokens: 2000000, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 500000},
|
||||
"grok-4-latest": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-4.20-0309-non-reasoning": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.20-0309-reasoning": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.20-beta-0309-non-reasoning": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.20-beta-0309-reasoning": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.20-multi-agent-0309": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.20-multi-agent-beta-0309": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.3": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.3-latest": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 1250000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"grok-4.5": {ContextTokens: 500000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"grok-4.5-latest": {ContextTokens: 500000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"grok-4.6": {ContextTokens: 500000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"grok-beta": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"grok-build-0.1": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"grok-code-fast": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"grok-code-fast-1": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"grok-code-fast-1-0825": {ContextTokens: 256000, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"grok-vision-beta": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 5000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"kimi-k2-0711-preview": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"kimi-k2-0905-preview": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"kimi-k2-thinking": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"kimi-k2-thinking-turbo": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 1150000, OutputCostPerMTokMicroUSD: 8000000},
|
||||
"kimi-k2-turbo-preview": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 1150000, OutputCostPerMTokMicroUSD: 8000000},
|
||||
"kimi-k2.5": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 3000000},
|
||||
"kimi-k2.6": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 950000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"kimi-k2.7-code": {ContextTokens: 229376, InputCostPerMTokMicroUSD: 950000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"kimi-k3": {ContextTokens: 1048576, InputCostPerMTokMicroUSD: 3000000, OutputCostPerMTokMicroUSD: 15000000},
|
||||
"kimi-latest": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"kimi-latest-128k": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"kimi-latest-32k": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 3000000},
|
||||
"kimi-latest-8k": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"kimi-thinking-preview": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 2500000},
|
||||
"llama-3.3-70b": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 850000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"llama3.1-70b": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 600000, OutputCostPerMTokMicroUSD: 600000},
|
||||
"llama3.1-8b": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 100000, OutputCostPerMTokMicroUSD: 100000},
|
||||
"moonshot-v1-128k": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"moonshot-v1-128k-0430": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"moonshot-v1-128k-vision-preview": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"moonshot-v1-32k": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 3000000},
|
||||
"moonshot-v1-32k-0430": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 3000000},
|
||||
"moonshot-v1-32k-vision-preview": {ContextTokens: 32768, InputCostPerMTokMicroUSD: 1000000, OutputCostPerMTokMicroUSD: 3000000},
|
||||
"moonshot-v1-8k": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"moonshot-v1-8k-0430": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"moonshot-v1-8k-vision-preview": {ContextTokens: 8192, InputCostPerMTokMicroUSD: 200000, OutputCostPerMTokMicroUSD: 2000000},
|
||||
"moonshot-v1-auto": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 5000000},
|
||||
"o1": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 60000000},
|
||||
"o1-2024-12-17": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 15000000, OutputCostPerMTokMicroUSD: 60000000},
|
||||
"o3": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 8000000},
|
||||
"o3-2025-04-16": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 8000000},
|
||||
"o3-mini": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 1100000, OutputCostPerMTokMicroUSD: 4400000},
|
||||
"o3-mini-2025-01-31": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 1100000, OutputCostPerMTokMicroUSD: 4400000},
|
||||
"o4-mini": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 1100000, OutputCostPerMTokMicroUSD: 4400000},
|
||||
"o4-mini-2025-04-16": {ContextTokens: 200000, InputCostPerMTokMicroUSD: 1100000, OutputCostPerMTokMicroUSD: 4400000},
|
||||
"qwen-3-32b": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 800000},
|
||||
"qwen-coder": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 300000, OutputCostPerMTokMicroUSD: 1500000},
|
||||
"qwen-max": {ContextTokens: 30720, InputCostPerMTokMicroUSD: 1600000, OutputCostPerMTokMicroUSD: 6400000},
|
||||
"qwen-plus": {ContextTokens: 129024, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"qwen-plus-2025-01-25": {ContextTokens: 129024, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"qwen-plus-2025-04-28": {ContextTokens: 129024, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"qwen-plus-2025-07-14": {ContextTokens: 129024, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"qwen-turbo": {ContextTokens: 129024, InputCostPerMTokMicroUSD: 50000, OutputCostPerMTokMicroUSD: 200000},
|
||||
"qwen-turbo-2024-11-01": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 50000, OutputCostPerMTokMicroUSD: 200000},
|
||||
"qwen-turbo-2025-04-28": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 50000, OutputCostPerMTokMicroUSD: 200000},
|
||||
"qwen-turbo-latest": {ContextTokens: 1000000, InputCostPerMTokMicroUSD: 50000, OutputCostPerMTokMicroUSD: 200000},
|
||||
"qwen3-next-80b-a3b-instruct": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"qwen3-next-80b-a3b-thinking": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 150000, OutputCostPerMTokMicroUSD: 1200000},
|
||||
"qwen3-vl-235b-a22b-instruct": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 1600000},
|
||||
"qwen3-vl-235b-a22b-thinking": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 400000, OutputCostPerMTokMicroUSD: 4000000},
|
||||
"qwen3-vl-32b-instruct": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 160000, OutputCostPerMTokMicroUSD: 640000},
|
||||
"qwen3-vl-32b-thinking": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 160000, OutputCostPerMTokMicroUSD: 2870000},
|
||||
"qwen3.7-max": {ContextTokens: 991808, InputCostPerMTokMicroUSD: 2500000, OutputCostPerMTokMicroUSD: 7500000},
|
||||
"qwen3.8-max": {ContextTokens: 991808, InputCostPerMTokMicroUSD: 2000000, OutputCostPerMTokMicroUSD: 6000000},
|
||||
"qwq-plus": {ContextTokens: 98304, InputCostPerMTokMicroUSD: 800000, OutputCostPerMTokMicroUSD: 2400000},
|
||||
"text-embedding-3-large": {ContextTokens: 8191, InputCostPerMTokMicroUSD: 130000, OutputCostPerMTokMicroUSD: 0},
|
||||
"text-embedding-3-small": {ContextTokens: 8191, InputCostPerMTokMicroUSD: 20000, OutputCostPerMTokMicroUSD: 0},
|
||||
"text-embedding-ada-002": {ContextTokens: 8191, InputCostPerMTokMicroUSD: 100000, OutputCostPerMTokMicroUSD: 0},
|
||||
"text-embedding-ada-002-v2": {ContextTokens: 8191, InputCostPerMTokMicroUSD: 100000, OutputCostPerMTokMicroUSD: 0},
|
||||
"zai-glm-4.6": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2250000, OutputCostPerMTokMicroUSD: 2750000},
|
||||
"zai-glm-4.7": {ContextTokens: 128000, InputCostPerMTokMicroUSD: 2250000, OutputCostPerMTokMicroUSD: 2750000},
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
module llmgw-pricing-sync
|
||||
|
||||
go 1.22
|
||||
@@ -0,0 +1,497 @@
|
||||
// Command pricing-sync regenerates the plugin's repo-root
|
||||
// pricing_data_gen.go: a built-in, LiteLLM-synced table of per-model
|
||||
// context window and per-token cost (feature v0.23's builtin layer,
|
||||
// modelmeta.go's builtinModelMeta/builtinModelMetaTable), which
|
||||
// resolveModelMeta falls back to when neither an operator's modelMeta
|
||||
// config override nor a provider's own discovery capture knows a value.
|
||||
//
|
||||
// It fetches LiteLLM's community-maintained
|
||||
// model_prices_and_context_window.json (litellmSourceURL below — 3111
|
||||
// entries as of this feature's own snapshot, 2026-08) at BUILD TIME
|
||||
// only, never at plugin runtime: the plugin module stays free of any
|
||||
// network dependency, and the generated table is a plain, checked-in Go
|
||||
// map literal an operator's `go build`/Yaegi interpretation never
|
||||
// fetches anything to produce.
|
||||
//
|
||||
// # Pruning rule
|
||||
//
|
||||
// A 3111-entry table in Yaegi-interpreted source is a real startup cost
|
||||
// (parsing and evaluating a large map literal under an AST interpreter
|
||||
// is far slower than under a compiler) — this tool prunes aggressively,
|
||||
// on three independent axes, ALL of which an entry must pass to survive:
|
||||
//
|
||||
// 1. Provider allowlist (allowedProviders below): only LiteLLM's own
|
||||
// "litellm_provider" tag values this gateway's provider TYPES can
|
||||
// meaningfully match against an operator's OWN model ids —
|
||||
// "openai", "anthropic", "deepseek", "xai" (Grok), "minimax",
|
||||
// "cerebras", "dashscope" (Alibaba Cloud's DashScope API, which is
|
||||
// where LiteLLM files its Qwen models — verified against a live
|
||||
// fetch of the source JSON while building this generator, 2026-08:
|
||||
// there is no "qwen" or "alibaba" litellm_provider tag, and no
|
||||
// "qwen/"-prefixed key exists in the source either), and "moonshot"
|
||||
// (Kimi models — coverage-audit addendum, 2026-08). Every other
|
||||
// provider LiteLLM tracks (bedrock, azure, vertex_ai, openrouter,
|
||||
// novita, github_copilot, ...) is dropped outright — this gateway
|
||||
// has no adapter type that would ever route to most of them under a
|
||||
// bare model id anyway. Notably NOT allowlisted despite being
|
||||
// considered during the coverage audit: "openrouter" (no entry in
|
||||
// the fetched source carries a ":free"-suffixed key — a claimed
|
||||
// free-tier naming convention that does not appear in this file as
|
||||
// fetched) and "novita" (where LiteLLM does track a few otherwise-
|
||||
// uncovered ids, e.g. xiaomi's mimo-v2.5 family, but adding either
|
||||
// whole catalog — openrouter alone re-adds ~100 entries — cuts
|
||||
// against this rule's own "keep the interpreted table small" goal;
|
||||
// left as an operator decision, not made unilaterally here).
|
||||
// 2. Mode allowlist (allowedModes below): "chat" and "embedding" only —
|
||||
// LiteLLM also prices image generation, audio transcription/speech,
|
||||
// moderation, and rerank endpoints under the same file, none of
|
||||
// which this feature's context-window/cost metadata is meant to
|
||||
// describe.
|
||||
// 3. A well-formed entry: both input_cost_per_token and
|
||||
// output_cost_per_token must decode as JSON numbers (many LiteLLM
|
||||
// entries price only per-image or per-request and leave these
|
||||
// null); an entry whose JSON shape does not even decode into
|
||||
// litellmEntry at all (LiteLLM's own "sample_spec" schema-example
|
||||
// key mixes description STRINGS into fields this tool expects as
|
||||
// numbers) is skipped rather than aborting the whole sync — this is
|
||||
// a defensive fallback, not just a hardcoded "sample_spec" skip, so
|
||||
// a future upstream schema-example entry with some other key name
|
||||
// degrades the same way.
|
||||
//
|
||||
// # Bare-id collision resolution
|
||||
//
|
||||
// The plugin's builtin tables (both the pre-existing builtinPricing,
|
||||
// pricing.go, and this feature's builtinModelMetaTable) are keyed by
|
||||
// BARE model id only, never "provider/model" — an operator's own
|
||||
// provider config key never matches LiteLLM's canonical provider name,
|
||||
// so a provider-qualified builtin entry would never be reachable via
|
||||
// resolveModelMeta's actual lookup (modelmeta.go). LiteLLM itself often
|
||||
// carries BOTH a bare key ("deepseek-chat") and one or more
|
||||
// provider-prefixed keys for the identical or a re-hosted model
|
||||
// ("deepseek/deepseek-chat", sometimes "dashscope/deepseek-v4-flash"
|
||||
// with DIFFERENT pricing served through a different host) — chooseWinner
|
||||
// resolves this deterministically: prefer a key that carries no
|
||||
// "provider/" prefix at all, then the earliest provider in
|
||||
// providerPriority (declared in the same order as allowedProviders'
|
||||
// documentation above), then the lexicographically smallest full key as
|
||||
// a final, always-deterministic tiebreak.
|
||||
//
|
||||
// # Naming bridges
|
||||
//
|
||||
// LiteLLM's own model naming occasionally disagrees with what a
|
||||
// provider's own live API reports for the IDENTICAL model — not a
|
||||
// pricing disagreement (chooseWinner's problem above), but a pure label
|
||||
// mismatch. namingBridges (below) is a small, explicitly data-driven
|
||||
// table for exactly this: each entry matches a bare id's suffix under
|
||||
// one provider and additionally emits an identical-valued duplicate
|
||||
// under a second suffix, so a future one-off naming split is a one-line
|
||||
// table addition, not new code. Never invents a value: a bridge only
|
||||
// ever duplicates an entry LiteLLM already priced, and never overwrites
|
||||
// an id LiteLLM already prices directly under its own bridged name.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"io"
|
||||
"math"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// litellmSourceURL is LiteLLM's community-maintained model price/context
|
||||
// table, fetched fresh on every `make pricing-sync` run — never at
|
||||
// plugin runtime (this tool's own doc comment above).
|
||||
const litellmSourceURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
|
||||
|
||||
// fetchTimeout bounds the network fetch: generous for a ~2MB JSON file
|
||||
// over a normal connection, while still failing a hung request instead
|
||||
// of blocking a `make pricing-sync` run indefinitely.
|
||||
const fetchTimeout = 60 * time.Second
|
||||
|
||||
// microUSDPerUSDPerToken converts a LiteLLM per-SINGLE-token USD price
|
||||
// to micro-USD PER MILLION tokens in one multiplication:
|
||||
// cost_per_token (USD/token) * 1_000_000 (tokens/Mtok) * 1_000_000
|
||||
// (micro-USD/USD) = cost_per_token * 1e12.
|
||||
const microUSDPerUSDPerToken = 1e12
|
||||
|
||||
// sampleSpecKey is LiteLLM's own schema-example entry — explicitly named
|
||||
// here (in addition to the generic decode-failure fallback the package
|
||||
// doc comment describes) so the pruning rule reads unambiguously for
|
||||
// this specific, known case.
|
||||
const sampleSpecKey = "sample_spec"
|
||||
|
||||
// allowedProviders is the pruning rule's provider allowlist — see the
|
||||
// package doc comment for the full rationale and the qwen/alibaba ->
|
||||
// dashscope mapping.
|
||||
var allowedProviders = map[string]bool{
|
||||
"openai": true,
|
||||
"anthropic": true,
|
||||
"deepseek": true,
|
||||
"xai": true, // Grok
|
||||
"minimax": true,
|
||||
"cerebras": true,
|
||||
"dashscope": true, // Qwen / Alibaba Cloud
|
||||
// moonshot: Kimi models (coverage-audit addendum, 2026-08) —
|
||||
// verified live against a fetch of the source JSON: 23 entries,
|
||||
// litellm_provider "moonshot", all mode "chat", bare ids like
|
||||
// "kimi-k2.5"/"kimi-k3". Added as a first-class allowlisted provider
|
||||
// (not a per-id bridge): builtinModelMetaTable is already keyed by
|
||||
// bare id only (bareModelID strips ANY matching allowlisted
|
||||
// provider's own "provider/" prefix), so an operator's own
|
||||
// Kimi-serving deployment — whether addressed directly or through an
|
||||
// aggregator such as Alibaba's DashScope — resolves against these
|
||||
// bare ids the same way any other allowlisted provider's entries do.
|
||||
"moonshot": true,
|
||||
}
|
||||
|
||||
// providerPriority orders allowedProviders for chooseWinner's collision
|
||||
// tiebreak — same set, as a stable slice instead of a randomly-ordered
|
||||
// map.
|
||||
var providerPriority = []string{"openai", "anthropic", "deepseek", "xai", "minimax", "cerebras", "dashscope", "moonshot"}
|
||||
|
||||
// allowedModes is the pruning rule's mode allowlist — see the package
|
||||
// doc comment.
|
||||
var allowedModes = map[string]bool{"chat": true, "embedding": true}
|
||||
|
||||
// namingBridge is one entry in namingBridges — see the package doc
|
||||
// comment's "Naming bridges" section.
|
||||
type namingBridge struct {
|
||||
provider string
|
||||
fromSuffix string
|
||||
toSuffix string
|
||||
}
|
||||
|
||||
// namingBridges documents known cases where a LiteLLM key's own naming
|
||||
// differs from what an operator's live discovery reports for the exact
|
||||
// same model. Seeded with MiniMax's dual naming (generator addendum,
|
||||
// 2026-08): LiteLLM's own model list calls the fast tier "-lightning"
|
||||
// (its international/global naming); MiniMax's own live API
|
||||
// (api.minimax.io, verified against a running deployment discovering
|
||||
// bare ids MiniMax-M2, -M2.1, -M2.1-highspeed, -M2.5, -M2.5-highspeed,
|
||||
// -M2.7, -M2.7-highspeed, -M3) reports the identical models as
|
||||
// "-highspeed" — same weights, same price, a different marketing label
|
||||
// per region. -M2.7/-M2.7-highspeed have no LiteLLM entry at all as of
|
||||
// this generator's own snapshot (verified live) and are deliberately
|
||||
// left uncovered here — applyNamingBridges only ever duplicates an
|
||||
// entry LiteLLM already priced, never invents one; -M2.7 is a candidate
|
||||
// for an operator's own modelMeta config override instead.
|
||||
var namingBridges = []namingBridge{
|
||||
{provider: "minimax", fromSuffix: "-lightning", toSuffix: "-highspeed"},
|
||||
}
|
||||
|
||||
// applyNamingBridges adds one extra table entry per namingBridges match:
|
||||
// for each winning candidate whose provider and bare-id suffix match a
|
||||
// bridge, it duplicates that entry (identical ContextTokens/cost — the
|
||||
// SAME model under a different label, not a different price point)
|
||||
// under the bridged id. A bridged id that collides with an id already
|
||||
// present in table (because LiteLLM prices it directly, under its own
|
||||
// entry) is left untouched — an entry LiteLLM already prices directly
|
||||
// always wins over a bridged duplicate.
|
||||
func applyNamingBridges(table map[string]builtinEntry, winners map[string]candidate) {
|
||||
for bare, w := range winners {
|
||||
for _, br := range namingBridges {
|
||||
if w.provider != br.provider || !strings.HasSuffix(bare, br.fromSuffix) {
|
||||
continue
|
||||
}
|
||||
bridged := strings.TrimSuffix(bare, br.fromSuffix) + br.toSuffix
|
||||
if _, exists := table[bridged]; exists {
|
||||
continue
|
||||
}
|
||||
table[bridged] = w.entry
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// litellmEntry is the subset of one LiteLLM model entry this tool reads.
|
||||
// Cost and max-token fields are *float64, not *int: most LiteLLM entries
|
||||
// report integer token counts, but a handful (observed live among xai's
|
||||
// own entries, e.g. "xai/grok-4-fast-reasoning") report
|
||||
// max_input_tokens/max_tokens as a JSON float (2000000.0) — decoding
|
||||
// those into *int would fail outright and drop the entry.
|
||||
type litellmEntry struct {
|
||||
InputCostPerToken *float64 `json:"input_cost_per_token"`
|
||||
OutputCostPerToken *float64 `json:"output_cost_per_token"`
|
||||
MaxInputTokens *float64 `json:"max_input_tokens"`
|
||||
MaxTokens *float64 `json:"max_tokens"`
|
||||
LiteLLMProvider string `json:"litellm_provider"`
|
||||
Mode string `json:"mode"`
|
||||
}
|
||||
|
||||
// builtinEntry mirrors the plugin's own builtinModelMeta (modelmeta.go)
|
||||
// field for field — kept as a separate, tool-local type rather than
|
||||
// importing the plugin package, so this generator never depends on the
|
||||
// (interpreted-only-constrained) plugin module at all.
|
||||
type builtinEntry struct {
|
||||
ContextTokens int
|
||||
InputCostPerMTokMicroUSD int64
|
||||
OutputCostPerMTokMicroUSD int64
|
||||
}
|
||||
|
||||
// candidate is one surviving LiteLLM key considered for a given bare
|
||||
// model id — chooseWinner's input, when more than one key collapses to
|
||||
// the same bare id.
|
||||
type candidate struct {
|
||||
fullKey string
|
||||
provider string
|
||||
entry builtinEntry
|
||||
bare bool // true when fullKey itself carried no "provider/" prefix
|
||||
}
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "pricing-sync: FAIL:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
repoRoot := "."
|
||||
if len(os.Args) > 1 {
|
||||
repoRoot = os.Args[1]
|
||||
}
|
||||
outPath := filepath.Join(repoRoot, "pricing_data_gen.go")
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), fetchTimeout)
|
||||
defer cancel()
|
||||
|
||||
data, err := fetchLiteLLMPrices(ctx, litellmSourceURL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("fetch %s: %w", litellmSourceURL, err)
|
||||
}
|
||||
|
||||
table, totalUpstream, err := parseAndPrune(data)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse and prune: %w", err)
|
||||
}
|
||||
|
||||
src, err := renderGoSource(table, sourceMeta{
|
||||
snapshot: time.Now().UTC().Format("2006-01-02"),
|
||||
totalUpstream: totalUpstream,
|
||||
kept: len(table),
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("render generated source: %w", err)
|
||||
}
|
||||
|
||||
if err := os.WriteFile(outPath, []byte(src), 0o644); err != nil { //nolint:gosec // generated Go source, not a secret
|
||||
return fmt.Errorf("write %s: %w", outPath, err)
|
||||
}
|
||||
|
||||
fmt.Printf("pricing-sync: wrote %s: %d entries kept (pruned from %d upstream entries)\n", outPath, len(table), totalUpstream)
|
||||
return nil
|
||||
}
|
||||
|
||||
// fetchLiteLLMPrices issues the network fetch, returning the raw
|
||||
// response body.
|
||||
func fetchLiteLLMPrices(ctx context.Context, url string) ([]byte, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build request: %w", err)
|
||||
}
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("do request: %w", err)
|
||||
}
|
||||
defer resp.Body.Close() //nolint:errcheck // read-side close
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return nil, fmt.Errorf("status %d", resp.StatusCode)
|
||||
}
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read body: %w", err)
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
|
||||
// parseAndPrune decodes data (LiteLLM's top-level {key: entry} JSON
|
||||
// object) and applies the pruning rule (package doc comment): the
|
||||
// provider and mode allowlists, well-formed cost fields, and
|
||||
// chooseWinner's deterministic bare-id collision resolution. It returns
|
||||
// the pruned table and the total number of top-level keys data carried
|
||||
// (including ones this function skips), for the summary line run prints.
|
||||
func parseAndPrune(data []byte) (map[string]builtinEntry, int, error) {
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal(data, &raw); err != nil {
|
||||
return nil, 0, fmt.Errorf("decode top-level object: %w", err)
|
||||
}
|
||||
|
||||
byBare := make(map[string][]candidate)
|
||||
for key, rawEntry := range raw {
|
||||
if key == sampleSpecKey {
|
||||
continue
|
||||
}
|
||||
var e litellmEntry
|
||||
if err := json.Unmarshal(rawEntry, &e); err != nil {
|
||||
// Not this tool's problem to solve generically — an entry
|
||||
// whose shape does not match litellmEntry at all (the
|
||||
// package doc comment's "sample_spec-shaped" fallback) is
|
||||
// simply not a model this table can describe.
|
||||
continue
|
||||
}
|
||||
if !allowedProviders[e.LiteLLMProvider] || !allowedModes[e.Mode] {
|
||||
continue
|
||||
}
|
||||
if e.InputCostPerToken == nil || e.OutputCostPerToken == nil {
|
||||
continue
|
||||
}
|
||||
|
||||
bare, isBareKey := bareModelID(key, e.LiteLLMProvider)
|
||||
byBare[bare] = append(byBare[bare], candidate{
|
||||
fullKey: key,
|
||||
provider: e.LiteLLMProvider,
|
||||
bare: isBareKey,
|
||||
entry: builtinEntry{
|
||||
ContextTokens: contextTokens(e),
|
||||
InputCostPerMTokMicroUSD: microUSDPerMTok(*e.InputCostPerToken),
|
||||
OutputCostPerMTokMicroUSD: microUSDPerMTok(*e.OutputCostPerToken),
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
winners := make(map[string]candidate, len(byBare))
|
||||
for bare, candidates := range byBare {
|
||||
winners[bare] = chooseWinner(candidates)
|
||||
}
|
||||
|
||||
table := make(map[string]builtinEntry, len(winners))
|
||||
for bare, w := range winners {
|
||||
table[bare] = w.entry
|
||||
}
|
||||
applyNamingBridges(table, winners)
|
||||
|
||||
return table, len(raw), nil
|
||||
}
|
||||
|
||||
// bareModelID strips a literal "{provider}/" prefix from key when
|
||||
// present, returning the bare id and whether key itself already WAS
|
||||
// bare (carried no such prefix) — chooseWinner's first, strongest
|
||||
// collision tiebreak.
|
||||
func bareModelID(key, provider string) (bare string, isBareKey bool) {
|
||||
prefix := provider + "/"
|
||||
if strings.HasPrefix(key, prefix) {
|
||||
return strings.TrimPrefix(key, prefix), false
|
||||
}
|
||||
return key, true
|
||||
}
|
||||
|
||||
// contextTokens picks e's context window: max_input_tokens when
|
||||
// reported, else max_tokens (LiteLLM's own documented fallback — see
|
||||
// this repo's litellm_prices.json "sample_spec" field descriptions),
|
||||
// else 0 (unknown — resolveModelMeta, modelmeta.go, never trusts a
|
||||
// zero).
|
||||
func contextTokens(e litellmEntry) int {
|
||||
if e.MaxInputTokens != nil && *e.MaxInputTokens > 0 {
|
||||
return int(math.Round(*e.MaxInputTokens))
|
||||
}
|
||||
if e.MaxTokens != nil && *e.MaxTokens > 0 {
|
||||
return int(math.Round(*e.MaxTokens))
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// microUSDPerMTok converts a LiteLLM per-token USD price to micro-USD
|
||||
// per million tokens, rounded to the nearest micro-USD — the same
|
||||
// round-to-nearest discipline pricing.go's costMicros already documents
|
||||
// for the plugin's own cost-accounting math, to avoid float64
|
||||
// representation drift undershooting by one unit.
|
||||
func microUSDPerMTok(costPerToken float64) int64 {
|
||||
return int64(math.Round(costPerToken * microUSDPerUSDPerToken))
|
||||
}
|
||||
|
||||
// chooseWinner picks the deterministic survivor among candidates that
|
||||
// all collapsed to the same bare model id — see the package doc
|
||||
// comment's "Bare-id collision resolution" section for the full
|
||||
// rationale. candidates is never empty (only ever built by appending at
|
||||
// least one entry, parseAndPrune above).
|
||||
func chooseWinner(candidates []candidate) candidate {
|
||||
best := candidates[0]
|
||||
for _, c := range candidates[1:] {
|
||||
if candidateLess(c, best) {
|
||||
best = c
|
||||
}
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
// candidateLess reports whether a should win over b: a bare key (no
|
||||
// "provider/" prefix) beats a prefixed one; failing that, an earlier
|
||||
// providerPriority index wins; failing that, the lexicographically
|
||||
// smaller full key wins — always deterministic, since fullKey is unique
|
||||
// per candidate by construction.
|
||||
func candidateLess(a, b candidate) bool {
|
||||
if a.bare != b.bare {
|
||||
return a.bare
|
||||
}
|
||||
ai, bi := providerPriorityIndex(a.provider), providerPriorityIndex(b.provider)
|
||||
if ai != bi {
|
||||
return ai < bi
|
||||
}
|
||||
return a.fullKey < b.fullKey
|
||||
}
|
||||
|
||||
// providerPriorityIndex returns provider's index in providerPriority, or
|
||||
// len(providerPriority) if somehow absent (defensive only — every
|
||||
// candidate's provider already passed the allowedProviders check, which
|
||||
// is drawn from the exact same set).
|
||||
func providerPriorityIndex(provider string) int {
|
||||
for i, p := range providerPriority {
|
||||
if p == provider {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return len(providerPriority)
|
||||
}
|
||||
|
||||
// sourceMeta is renderGoSource's header-comment material.
|
||||
type sourceMeta struct {
|
||||
snapshot string
|
||||
totalUpstream int
|
||||
kept int
|
||||
}
|
||||
|
||||
// renderGoSource formats table as the generated pricing_data_gen.go
|
||||
// source: a documented header, then a single, gofmt-canonical map
|
||||
// literal — plain, no generics (the plugin package is Yaegi-interpreted;
|
||||
// see this tool's own package doc comment and modelmeta.go's
|
||||
// builtinModelMeta doc comment) — entries sorted by key for a stable,
|
||||
// reviewable diff on every regeneration.
|
||||
func renderGoSource(table map[string]builtinEntry, meta sourceMeta) (string, error) {
|
||||
keys := make([]string, 0, len(table))
|
||||
for k := range table {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
|
||||
var b strings.Builder
|
||||
b.WriteString("package traefikllmgateway\n\n")
|
||||
fmt.Fprintf(&b, "// Code generated by tools/pricing-sync from LiteLLM's\n")
|
||||
fmt.Fprintf(&b, "// model_prices_and_context_window.json — DO NOT EDIT.\n")
|
||||
fmt.Fprintf(&b, "//\n")
|
||||
fmt.Fprintf(&b, "// Source: %s\n", litellmSourceURL)
|
||||
fmt.Fprintf(&b, "// Snapshot: %s, %d entries kept (pruned from %d upstream entries —\n", meta.snapshot, meta.kept, meta.totalUpstream)
|
||||
fmt.Fprintf(&b, "// see tools/pricing-sync/main.go's own doc comment for the exact pruning\n")
|
||||
fmt.Fprintf(&b, "// rule and bare-id collision resolution).\n")
|
||||
fmt.Fprintf(&b, "//\n")
|
||||
fmt.Fprintf(&b, "// Regenerate with: make pricing-sync\n")
|
||||
b.WriteString("var builtinModelMetaTable = map[string]builtinModelMeta{\n")
|
||||
for _, k := range keys {
|
||||
e := table[k]
|
||||
fmt.Fprintf(&b, "\t%q: {ContextTokens: %d, InputCostPerMTokMicroUSD: %d, OutputCostPerMTokMicroUSD: %d},\n",
|
||||
k, e.ContextTokens, e.InputCostPerMTokMicroUSD, e.OutputCostPerMTokMicroUSD)
|
||||
}
|
||||
b.WriteString("}\n")
|
||||
|
||||
formatted, err := format.Source([]byte(b.String()))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("gofmt generated source: %w", err)
|
||||
}
|
||||
return string(formatted), nil
|
||||
}
|
||||
@@ -0,0 +1,244 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// fixtureJSON is a small, hand-built stand-in for LiteLLM's real
|
||||
// model_prices_and_context_window.json, covering every branch of the
|
||||
// pruning rule documented on this package (main.go's own doc comment):
|
||||
// the sample_spec schema-example entry, a disallowed provider, a
|
||||
// disallowed mode, null cost fields, a bare-vs-provider-prefixed
|
||||
// collision with identical values, a three-way collision with
|
||||
// DIFFERING values across providers (deepseek-v4-flash, mirroring the
|
||||
// real dashscope/deepseek disagreement this tool was built against), an
|
||||
// xai entry with float-valued (not int) max_input_tokens/max_tokens, an
|
||||
// embedding-mode entry with a genuine zero output cost, and an entry
|
||||
// with no max_input_tokens/max_tokens at all.
|
||||
const fixtureJSON = `{
|
||||
"sample_spec": {
|
||||
"litellm_provider": "one of https://docs.litellm.ai/docs/providers",
|
||||
"input_cost_per_token": 0.0,
|
||||
"output_cost_per_token": 0.0,
|
||||
"max_input_tokens": "max input tokens, if the provider specifies it."
|
||||
},
|
||||
"gpt-5": {
|
||||
"litellm_provider": "openai", "mode": "chat",
|
||||
"input_cost_per_token": 1.25e-06, "output_cost_per_token": 1.0e-05,
|
||||
"max_input_tokens": 400000, "max_tokens": 128000
|
||||
},
|
||||
"bedrock/some-model": {
|
||||
"litellm_provider": "bedrock", "mode": "chat",
|
||||
"input_cost_per_token": 1e-06, "output_cost_per_token": 1e-06,
|
||||
"max_input_tokens": 100000
|
||||
},
|
||||
"openai/dalle": {
|
||||
"litellm_provider": "openai", "mode": "image_generation",
|
||||
"output_cost_per_image": 0.04
|
||||
},
|
||||
"openai/null-cost": {
|
||||
"litellm_provider": "openai", "mode": "chat",
|
||||
"input_cost_per_token": null, "output_cost_per_token": null,
|
||||
"max_input_tokens": 1000
|
||||
},
|
||||
"deepseek-chat": {
|
||||
"litellm_provider": "deepseek", "mode": "chat",
|
||||
"input_cost_per_token": 2.8e-07, "output_cost_per_token": 4.2e-07,
|
||||
"max_input_tokens": 131072, "max_tokens": 8192
|
||||
},
|
||||
"deepseek/deepseek-chat": {
|
||||
"litellm_provider": "deepseek", "mode": "chat",
|
||||
"input_cost_per_token": 2.8e-07, "output_cost_per_token": 4.2e-07,
|
||||
"max_input_tokens": 131072, "max_tokens": 8192
|
||||
},
|
||||
"dashscope/deepseek-v4-flash": {
|
||||
"litellm_provider": "dashscope", "mode": "chat",
|
||||
"input_cost_per_token": 2e-07, "output_cost_per_token": 4e-07,
|
||||
"max_input_tokens": 1000000
|
||||
},
|
||||
"deepseek-v4-flash": {
|
||||
"litellm_provider": "deepseek", "mode": "chat",
|
||||
"input_cost_per_token": 4.4e-07, "output_cost_per_token": 1.32e-06,
|
||||
"max_input_tokens": 131072
|
||||
},
|
||||
"deepseek/deepseek-v4-flash": {
|
||||
"litellm_provider": "deepseek", "mode": "chat",
|
||||
"input_cost_per_token": 4.4e-07, "output_cost_per_token": 1.32e-06,
|
||||
"max_input_tokens": 131072
|
||||
},
|
||||
"xai/grok-4-fast-reasoning": {
|
||||
"litellm_provider": "xai", "mode": "chat",
|
||||
"input_cost_per_token": 2e-07, "output_cost_per_token": 5e-07,
|
||||
"max_input_tokens": 2000000.0, "max_tokens": 2000000.0
|
||||
},
|
||||
"openai/whisper-notmode": {
|
||||
"litellm_provider": "openai", "mode": "audio_transcription",
|
||||
"input_cost_per_token": 1e-06, "output_cost_per_token": 1e-06,
|
||||
"max_input_tokens": 16000
|
||||
},
|
||||
"text-embedding-3-large": {
|
||||
"litellm_provider": "openai", "mode": "embedding",
|
||||
"input_cost_per_token": 1.3e-07, "output_cost_per_token": 0.0,
|
||||
"max_input_tokens": 8191, "max_tokens": 8191
|
||||
},
|
||||
"openai/no-max-tokens-at-all": {
|
||||
"litellm_provider": "openai", "mode": "chat",
|
||||
"input_cost_per_token": 1e-06, "output_cost_per_token": 2e-06
|
||||
},
|
||||
"minimax/MiniMax-M2.1-lightning": {
|
||||
"litellm_provider": "minimax", "mode": "chat",
|
||||
"input_cost_per_token": 3e-07, "output_cost_per_token": 2.4e-06,
|
||||
"max_input_tokens": 1000000
|
||||
},
|
||||
"claude-haiku-4-5": {
|
||||
"litellm_provider": "anthropic", "mode": "chat",
|
||||
"input_cost_per_token": 1e-06, "output_cost_per_token": 5e-06,
|
||||
"max_input_tokens": 200000, "max_tokens": 64000
|
||||
},
|
||||
"moonshot/kimi-k2.5": {
|
||||
"litellm_provider": "moonshot", "mode": "chat",
|
||||
"input_cost_per_token": 6e-07, "output_cost_per_token": 2.5e-06,
|
||||
"max_input_tokens": 262144
|
||||
}
|
||||
}`
|
||||
|
||||
// fixtureUpstreamKeyCount is fixtureJSON's total top-level key count
|
||||
// (including every pruned-out entry and sample_spec) — parseAndPrune's
|
||||
// second return value.
|
||||
const fixtureUpstreamKeyCount = 17
|
||||
|
||||
// TestParseAndPrune_Golden is the generator's golden test (task
|
||||
// requirement): a small fixture JSON in, an exact expected pruned table
|
||||
// out, covering every pruning-rule branch fixtureJSON's own doc comment
|
||||
// lists.
|
||||
func TestParseAndPrune_Golden(t *testing.T) {
|
||||
got, totalUpstream, err := parseAndPrune([]byte(fixtureJSON))
|
||||
if err != nil {
|
||||
t.Fatalf("parseAndPrune: %v", err)
|
||||
}
|
||||
if totalUpstream != fixtureUpstreamKeyCount {
|
||||
t.Errorf("totalUpstream = %d, want %d", totalUpstream, fixtureUpstreamKeyCount)
|
||||
}
|
||||
|
||||
want := map[string]builtinEntry{
|
||||
"gpt-5": {ContextTokens: 400000, InputCostPerMTokMicroUSD: 1_250_000, OutputCostPerMTokMicroUSD: 10_000_000},
|
||||
// deepseek-chat: bare key wins over the identical-valued
|
||||
// "deepseek/deepseek-chat" prefixed key (rule: bare beats
|
||||
// prefixed regardless of value equality).
|
||||
"deepseek-chat": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 280_000, OutputCostPerMTokMicroUSD: 420_000},
|
||||
// deepseek-v4-flash: three colliding keys, DIFFERING values
|
||||
// between "dashscope/..." and the deepseek-provider ones — the
|
||||
// bare "deepseek-v4-flash" key wins (rule 1: bare beats
|
||||
// prefixed), not the higher- or lower-priced alternative.
|
||||
"deepseek-v4-flash": {ContextTokens: 131072, InputCostPerMTokMicroUSD: 440_000, OutputCostPerMTokMicroUSD: 1_320_000},
|
||||
"grok-4-fast-reasoning": {ContextTokens: 2_000_000, InputCostPerMTokMicroUSD: 200_000, OutputCostPerMTokMicroUSD: 500_000},
|
||||
// text-embedding-3-large: a genuine, known zero output cost
|
||||
// (embeddings bill input tokens only) must survive, not be
|
||||
// treated as "missing".
|
||||
"text-embedding-3-large": {ContextTokens: 8191, InputCostPerMTokMicroUSD: 130_000, OutputCostPerMTokMicroUSD: 0},
|
||||
// no-max-tokens-at-all: neither max_input_tokens nor max_tokens
|
||||
// present at all -> ContextTokens 0 (unknown), cost still kept.
|
||||
"no-max-tokens-at-all": {ContextTokens: 0, InputCostPerMTokMicroUSD: 1_000_000, OutputCostPerMTokMicroUSD: 2_000_000},
|
||||
// MiniMax-M2.1-lightning: LiteLLM's own entry, kept as-is.
|
||||
"MiniMax-M2.1-lightning": {ContextTokens: 1_000_000, InputCostPerMTokMicroUSD: 300_000, OutputCostPerMTokMicroUSD: 2_400_000},
|
||||
// MiniMax-M2.1-highspeed: the naming-bridge duplicate (generator
|
||||
// addendum) — identical values, no LiteLLM entry of its own in
|
||||
// this fixture.
|
||||
"MiniMax-M2.1-highspeed": {ContextTokens: 1_000_000, InputCostPerMTokMicroUSD: 300_000, OutputCostPerMTokMicroUSD: 2_400_000},
|
||||
// Coverage-audit sanity (addendum, 2026-08): openai-family and
|
||||
// anthropic bare ids resolve as LiteLLM's own canonical bare
|
||||
// keys, with no bridging or prefix-stripping needed — "gpt-5"
|
||||
// above already covers the openai family; this covers anthropic.
|
||||
"claude-haiku-4-5": {ContextTokens: 200_000, InputCostPerMTokMicroUSD: 1_000_000, OutputCostPerMTokMicroUSD: 5_000_000},
|
||||
// moonshot: Kimi models (coverage-audit addendum) — a
|
||||
// provider-prefixed key, bareModelID strips "moonshot/" the same
|
||||
// way it strips any other allowlisted provider's own prefix.
|
||||
"kimi-k2.5": {ContextTokens: 262144, InputCostPerMTokMicroUSD: 600_000, OutputCostPerMTokMicroUSD: 2_500_000},
|
||||
}
|
||||
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("parseAndPrune(fixtureJSON) =\n%#v\nwant\n%#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRenderGoSource_ValidSortedGo asserts renderGoSource produces
|
||||
// syntactically valid, gofmt-canonical Go source (parseable by go/
|
||||
// parser) whose entries appear in sorted key order, and that its header
|
||||
// documents the pruning provenance (source URL, snapshot, counts).
|
||||
func TestRenderGoSource_ValidSortedGo(t *testing.T) {
|
||||
table := map[string]builtinEntry{
|
||||
"zzz-model": {ContextTokens: 1, InputCostPerMTokMicroUSD: 1, OutputCostPerMTokMicroUSD: 1},
|
||||
"aaa-model": {ContextTokens: 2, InputCostPerMTokMicroUSD: 2, OutputCostPerMTokMicroUSD: 2},
|
||||
}
|
||||
src, err := renderGoSource(table, sourceMeta{snapshot: "2026-08-22", totalUpstream: 3111, kept: 2})
|
||||
if err != nil {
|
||||
t.Fatalf("renderGoSource: %v", err)
|
||||
}
|
||||
|
||||
if _, err := parser.ParseFile(token.NewFileSet(), "pricing_data_gen.go", src, parser.AllErrors); err != nil {
|
||||
t.Fatalf("generated source does not parse as valid Go: %v\n%s", err, src)
|
||||
}
|
||||
if !strings.Contains(src, "package traefikllmgateway") {
|
||||
t.Error("generated source missing package declaration")
|
||||
}
|
||||
if !strings.Contains(src, litellmSourceURL) {
|
||||
t.Error("generated source missing source URL provenance comment")
|
||||
}
|
||||
if !strings.Contains(src, "2026-08-22") || !strings.Contains(src, "3111") {
|
||||
t.Error("generated source missing snapshot date / upstream count provenance comment")
|
||||
}
|
||||
if got, want := strings.Index(src, `"aaa-model"`), strings.Index(src, `"zzz-model"`); got == -1 || want == -1 || got > want {
|
||||
t.Errorf(`entries not sorted: index("aaa-model")=%d, index("zzz-model")=%d, want aaa before zzz`, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestChooseWinner_Deterministic asserts chooseWinner's tiebreak order
|
||||
// (bare > provider priority > lexicographic full key) directly, beyond
|
||||
// what the golden test above exercises indirectly.
|
||||
func TestChooseWinner_Deterministic(t *testing.T) {
|
||||
cands := []candidate{
|
||||
{fullKey: "dashscope/x", provider: "dashscope", bare: false, entry: builtinEntry{ContextTokens: 1}},
|
||||
{fullKey: "deepseek/x", provider: "deepseek", bare: false, entry: builtinEntry{ContextTokens: 2}},
|
||||
{fullKey: "x", provider: "deepseek", bare: true, entry: builtinEntry{ContextTokens: 3}},
|
||||
}
|
||||
if got := chooseWinner(cands); got.entry.ContextTokens != 3 {
|
||||
t.Errorf("chooseWinner picked ContextTokens=%d, want 3 (the bare candidate)", got.entry.ContextTokens)
|
||||
}
|
||||
|
||||
// With no bare candidate, provider priority decides: "deepseek"
|
||||
// precedes "dashscope" in providerPriority.
|
||||
noBare := []candidate{
|
||||
{fullKey: "dashscope/x", provider: "dashscope", bare: false, entry: builtinEntry{ContextTokens: 1}},
|
||||
{fullKey: "deepseek/x", provider: "deepseek", bare: false, entry: builtinEntry{ContextTokens: 2}},
|
||||
}
|
||||
if got := chooseWinner(noBare); got.entry.ContextTokens != 2 {
|
||||
t.Errorf("chooseWinner picked ContextTokens=%d, want 2 (deepseek outranks dashscope)", got.entry.ContextTokens)
|
||||
}
|
||||
}
|
||||
|
||||
// TestApplyNamingBridges_DoesNotOverwriteExistingEntry asserts a bridged
|
||||
// id that LiteLLM already prices directly (its own independent winning
|
||||
// candidate) is left untouched by the bridge — an entry LiteLLM prices
|
||||
// directly always wins over a bridged duplicate (generator addendum).
|
||||
func TestApplyNamingBridges_DoesNotOverwriteExistingEntry(t *testing.T) {
|
||||
winners := map[string]candidate{
|
||||
"MiniMax-M2.1-lightning": {provider: "minimax", entry: builtinEntry{ContextTokens: 1_000_000, InputCostPerMTokMicroUSD: 300_000, OutputCostPerMTokMicroUSD: 2_400_000}},
|
||||
// A directly-priced, independent "-highspeed" entry with a
|
||||
// DIFFERENT value than the lightning entry would bridge to.
|
||||
"MiniMax-M2.1-highspeed": {provider: "minimax", entry: builtinEntry{ContextTokens: 500_000, InputCostPerMTokMicroUSD: 999, OutputCostPerMTokMicroUSD: 999}},
|
||||
}
|
||||
table := make(map[string]builtinEntry, len(winners))
|
||||
for bare, w := range winners {
|
||||
table[bare] = w.entry
|
||||
}
|
||||
applyNamingBridges(table, winners)
|
||||
|
||||
want := builtinEntry{ContextTokens: 500_000, InputCostPerMTokMicroUSD: 999, OutputCostPerMTokMicroUSD: 999}
|
||||
if got := table["MiniMax-M2.1-highspeed"]; got != want {
|
||||
t.Errorf(`table["MiniMax-M2.1-highspeed"] = %+v, want %+v (its own direct LiteLLM entry, not the bridged duplicate)`, got, want)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user