feat(metrics): expose token-accounting provenance as a metric dimension

Token accounting has three provenances today: reported (real provider
usage), estimated (a non-streaming response with no usage, prompt
substituted from request body size), and unbilled (a streaming response
with no usage, request counted but zero tokens billed). Nothing on any
read surface distinguished them before this change.

Add a per-process provenanceStore (metrics.go), recorded in
runUnified right after the existing estimated/unbilled classification
(routes_unified.go), for a candidate whose response actually completed.
A failed-over candidate is never recorded, so ordinary failover noise
never pollutes the unbilled signal.

Expose two new counter families on GET /metrics:
llmgateway_usage_provenance_requests_total and
llmgateway_usage_provenance_tokens_total, both labeled by provider and
provenance. Together they answer the three operator questions this
change targets: what fraction of billed tokens came from real reported
usage, which providers produce estimated or unbilled accounting, and
how often the unbilled case happens at all. Both families are
in-process only, so sum() is the correct cross-replica aggregation,
matching this file's other per-process families.

Billing itself is unchanged: this only records which provenance an
already-billed number took, never a new number.

Tests cover reported/estimated/unbilled classification, that billing
matches the pre-existing contract exactly, and that estimated and
unbilled never collapse into one series. Each behavioral assertion was
verified against a real mutation of the source before being kept.
This commit is contained in:
2026-08-23 19:44:50 +01:00
parent b02203a650
commit 7c27233ca3
5 changed files with 573 additions and 5 deletions
+15 -1
View File
@@ -597,7 +597,17 @@ type Gateway struct {
// identical "safe even off a bare &Gateway{} literal" convention
// above.
latency *latencyStore
name string
// provenance is the in-process usage-accounting-provenance accumulator
// (feat: expose token-accounting provenance, metrics.go's
// provenanceStore) — recordUsageProvenance's only write target, and
// writeProvenanceMetrics/buildAdminProvenanceViews' (admin.go) own
// read source. Always constructed, here, by newGateway, even when
// Config.Metrics is nil or disabled, mirroring g.latency's own
// identical "always-present, usually-empty, nil-receiver-safe"
// convention immediately above — see that field's own doc comment for
// the full reasoning, which applies here unchanged.
provenance *provenanceStore
name string
// failoverLogGate rate-limits runMeteredCall's generic "failing over"
// log line (routes_unified.go) to once per storeErrorLogEvery
// (adversarial-review fix, F10) — reuses auth.go's own logGate type,
@@ -744,6 +754,10 @@ func newGateway(ctx context.Context, next http.Handler, config *Config, name str
// comment above for why an always-present, usually-empty store is the
// right default rather than a conditional nil.
g.latency = newLatencyStore()
// feat: expose token-accounting provenance — always constructed, here,
// regardless of whether Config.Metrics is nil/disabled, mirroring
// g.latency's own identical "always-present" ruling immediately above.
g.provenance = newProvenanceStore()
// A malformed Config.Metrics.AllowedCIDRs entry is a constructor
// error (validate-at-construction), not a silently-ignored allowlist