lukaszraczylo/traefik-llmgateway
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/traefik-llmgateway.git synced 2026-10-19 05:47:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(admin): add per-model usage statistics to the charts section

Browse Source 
The Charts view could break usage down by user, group or total, but not
by model. Per-model data existed only as day-window attempt and failure
counts, which measure availability rather than delivered usage.

A served model now accumulates the same four metrics a user or group
does. The scope kind is "model" and its id is the canonical
"provider/model", so a request that fails over counts against the
provider that answered it, and one bare id served by two providers ranks
as two rows.

The request counter is written after the response, in the batch that
already writes tokens and cost. That is the first point at which the
serving model is known, so no extra round trip is needed and admission
never double counts.

GET /admin/api/usage/models ranks models for one metric and window. It
returns only models with non-zero usage, by operator requirement: the
catalog runs to hundreds of models and a padded ranking is unreadable.
The read stays one storeGetMulti over the catalog.

The Charts view gains a Models tab with its own metric picker, and the
scope picker gains the models that have traffic.
This commit is contained in:
lukaszraczylo
2026-08-27 23:30:22 +01:00
parent bcd7c32c3d
commit 4bb49b4fe9
15 changed files with 1236 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
routes_passthrough.go
+6 -2
View File
@@ -895,8 +895,12 @@ func (g *Gateway) handlePassthrough(w http.ResponseWriter, r *http.Request, u *u
if unmarshalErr != nil {
g.logf("passthrough: response body did not decode as JSON for usage accounting (provider %q): %v", providerName, unmarshalErr)
}
cost := unifiedCostMicros(providerName+"/"+respModel, respModel, respUsage, g.cfg.Pricing)
g.limiter.account(scopes, respUsage, cost)
canonical := providerName + "/" + respModel
cost := unifiedCostMicros(canonical, respModel, respUsage, g.cfg.Pricing)
// withModelScope drops the model scope when respModel is empty (the
// upstream reported no model id), so a passthrough reply the gateway
// cannot attribute still accounts to user/group/total as before.
g.limiter.account(withModelScope(scopes, canonical), respUsage, cost)
}
// proxyResult is what proxyUpstream reports back to its caller once it has