fix(accounting): bound upstream-controlled usage and counter keys

Passthrough accounting read the model id, the usage figures and the
response size from the upstream response without limits.

- Mint a model counter scope only for ids the provider serves. An
  upstream that echoed new ids created unbounded counter keys.
- Drop a reported token or cost figure above a fixed ceiling. One huge
  figure locked a user out of every budget for the whole window.
- Bill an echoed unpriced model at the price of the model that passed
  the cost-budget gate.
- Cap a body-size prompt estimate at one million tokens.
- Bound the total bytes that concurrent passthrough accounting tees
  hold. A tee that cannot reserve room falls back to the estimate.
- Add cache.accountHits. When true, a cache hit charges the cached usage
  to the user. The default stays false.
This commit is contained in:
2026-10-15 22:19:57 +01:00
parent e5fe48a9a6
commit 8c4bd16a22
7 changed files with 306 additions and 8 deletions
+7
View File
@@ -1555,6 +1555,13 @@ func (m *modelRegistry) bareWinner(id string, grp *group, extraIDs ...string) (s
return visible[0], true
}
// providerHasModel reports whether provider's explicit or discovered model
// set contains id.
func (m *modelRegistry) providerHasModel(provider, id string) bool {
st, ok := m.states[provider]
return ok && st.hasModel(id)
}
// resolve maps a client-requested model id to the adapter that serves it.
// It returns the adapter, the upstream model id to send that adapter (the
// id form the provider itself knows, with any gateway-side provider prefix