fix(accounting): bound upstream-controlled usage and counter keys

Passthrough accounting read the model id, the usage figures and the
response size from the upstream response without limits.

- Mint a model counter scope only for ids the provider serves. An
  upstream that echoed new ids created unbounded counter keys.
- Drop a reported token or cost figure above a fixed ceiling. One huge
  figure locked a user out of every budget for the whole window.
- Bill an echoed unpriced model at the price of the model that passed
  the cost-budget gate.
- Cap a body-size prompt estimate at one million tokens.
- Bound the total bytes that concurrent passthrough accounting tees
  hold. A tee that cannot reserve room falls back to the estimate.
- Add cache.accountHits. When true, a cache hit charges the cached usage
  to the user. The default stays false.
This commit is contained in:
2026-10-15 22:19:57 +01:00
parent e5fe48a9a6
commit 8c4bd16a22
7 changed files with 306 additions and 8 deletions
+1 -1
View File
@@ -3355,7 +3355,7 @@ func TestHandlePassthrough_ModelLatencyAndUModelSuppressed_ProviderHistogramsOnl
cfg := CreateConfig()
cfg.Providers = map[string]*ProviderConfig{
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up"},
"openai": {Type: "openai", BaseURL: srv.URL, APIKey: "sk-up", Models: []string{"gpt-native-x"}},
}
cfg.Groups = map[string]*GroupConfig{"default": {}}
cfg.Users = &UsersConfig{Inline: []*UserConfig{