feat(metrics): expose token-accounting provenance as a metric dimension

Token accounting has three provenances today: reported (real provider
usage), estimated (a non-streaming response with no usage, prompt
substituted from request body size), and unbilled (a streaming response
with no usage, request counted but zero tokens billed). Nothing on any
read surface distinguished them before this change.

Add a per-process provenanceStore (metrics.go), recorded in
runUnified right after the existing estimated/unbilled classification
(routes_unified.go), for a candidate whose response actually completed.
A failed-over candidate is never recorded, so ordinary failover noise
never pollutes the unbilled signal.

Expose two new counter families on GET /metrics:
llmgateway_usage_provenance_requests_total and
llmgateway_usage_provenance_tokens_total, both labeled by provider and
provenance. Together they answer the three operator questions this
change targets: what fraction of billed tokens came from real reported
usage, which providers produce estimated or unbilled accounting, and
how often the unbilled case happens at all. Both families are
in-process only, so sum() is the correct cross-replica aggregation,
matching this file's other per-process families.

Billing itself is unchanged: this only records which provenance an
already-billed number took, never a new number.

Tests cover reported/estimated/unbilled classification, that billing
matches the pre-existing contract exactly, and that estimated and
unbilled never collapse into one series. Each behavioral assertion was
verified against a real mutation of the source before being kept.
This commit is contained in:
2026-08-23 19:44:50 +01:00
parent b02203a650
commit 7c27233ca3
5 changed files with 573 additions and 5 deletions
+28
View File
@@ -555,6 +555,34 @@ func (g *Gateway) runMeteredCall(sw *statusTrackingWriter, r *http.Request, scop
g.logf("%s: usage for model %q logged as estimated (%d prompt tokens derived from request body size, not the provider's reported usage)", logPrefix, cand.canonical, result.prompt)
}
// Usage provenance (feat: expose token-accounting provenance),
// recorded ONLY for a candidate whose own response actually
// completed (callErr == nil): a candidate that failed over never
// bills more than a zero-total no-op (this function's own doc
// comment above), and counting THAT as "unbilled" would drown the
// one signal this feature exists to surface — a provider silently
// serving completions free against every budget — in ordinary
// failover noise. "unbilled" is re-derived from streaming &&
// result.total() == 0 rather than a separate flag carried out of
// the block above: result.prompt/result.estimated are only ever
// mutated by that block's non-streaming branch, so this condition
// is true here if and only if the streaming branch above ran.
// Gated on metricsEnabled exactly like withLatencyRecorder's own
// wiring above, so a deployment with metrics off pays nothing
// extra here. Observation only: nothing recorded here feeds back
// into result or the account call above, both of which already
// ran — see g.recordUsageProvenance's own doc comment (metrics.go).
if callErr == nil && metricsEnabled(g.cfg) {
provenance := provenanceReported
switch {
case result.estimated:
provenance = provenanceEstimated
case streaming && result.total() == 0:
provenance = provenanceUnbilled
}
g.recordUsageProvenance(providerName, provenance, result.total())
}
// A miss stores the response after everything above has already
// run — accounting must never be skipped or delayed waiting on a
// cache write. Only a genuine upstream 200 is stored (spec §2's