mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-19 05:47:48 +00:00
feat(metrics): expose token-accounting provenance as a metric dimension
Token accounting has three provenances today: reported (real provider usage), estimated (a non-streaming response with no usage, prompt substituted from request body size), and unbilled (a streaming response with no usage, request counted but zero tokens billed). Nothing on any read surface distinguished them before this change. Add a per-process provenanceStore (metrics.go), recorded in runUnified right after the existing estimated/unbilled classification (routes_unified.go), for a candidate whose response actually completed. A failed-over candidate is never recorded, so ordinary failover noise never pollutes the unbilled signal. Expose two new counter families on GET /metrics: llmgateway_usage_provenance_requests_total and llmgateway_usage_provenance_tokens_total, both labeled by provider and provenance. Together they answer the three operator questions this change targets: what fraction of billed tokens came from real reported usage, which providers produce estimated or unbilled accounting, and how often the unbilled case happens at all. Both families are in-process only, so sum() is the correct cross-replica aggregation, matching this file's other per-process families. Billing itself is unchanged: this only records which provenance an already-billed number took, never a new number. Tests cover reported/estimated/unbilled classification, that billing matches the pre-existing contract exactly, and that estimated and unbilled never collapse into one series. Each behavioral assertion was verified against a real mutation of the source before being kept.
This commit is contained in:
+15
-1
@@ -597,7 +597,17 @@ type Gateway struct {
|
||||
// identical "safe even off a bare &Gateway{} literal" convention
|
||||
// above.
|
||||
latency *latencyStore
|
||||
name string
|
||||
// provenance is the in-process usage-accounting-provenance accumulator
|
||||
// (feat: expose token-accounting provenance, metrics.go's
|
||||
// provenanceStore) — recordUsageProvenance's only write target, and
|
||||
// writeProvenanceMetrics/buildAdminProvenanceViews' (admin.go) own
|
||||
// read source. Always constructed, here, by newGateway, even when
|
||||
// Config.Metrics is nil or disabled, mirroring g.latency's own
|
||||
// identical "always-present, usually-empty, nil-receiver-safe"
|
||||
// convention immediately above — see that field's own doc comment for
|
||||
// the full reasoning, which applies here unchanged.
|
||||
provenance *provenanceStore
|
||||
name string
|
||||
// failoverLogGate rate-limits runMeteredCall's generic "failing over"
|
||||
// log line (routes_unified.go) to once per storeErrorLogEvery
|
||||
// (adversarial-review fix, F10) — reuses auth.go's own logGate type,
|
||||
@@ -744,6 +754,10 @@ func newGateway(ctx context.Context, next http.Handler, config *Config, name str
|
||||
// comment above for why an always-present, usually-empty store is the
|
||||
// right default rather than a conditional nil.
|
||||
g.latency = newLatencyStore()
|
||||
// feat: expose token-accounting provenance — always constructed, here,
|
||||
// regardless of whether Config.Metrics is nil/disabled, mirroring
|
||||
// g.latency's own identical "always-present" ruling immediately above.
|
||||
g.provenance = newProvenanceStore()
|
||||
|
||||
// A malformed Config.Metrics.AllowedCIDRs entry is a constructor
|
||||
// error (validate-at-construction), not a silently-ignored allowlist
|
||||
|
||||
Reference in New Issue
Block a user