Skip to content

Epic: Comprehensive Metrics System #443

@psschwei

Description

@psschwei

Description:
Add metrics collection for key operational indicators: token usage, latency, costs, errors, and sampling/validation outcomes. This provides quantitative observability complementing the existing tracing system.

Current State:

  • No metrics collection system exists
  • Token usage is extracted during backend operations but only used for tracing attributes
  • No latency histograms, error counters, or cost tracking
  • No Prometheus or OTLP metrics export

Acceptance Criteria:

  • Token usage counters by model/backend
  • Latency histograms including TTFB for streaming
  • Cost tracking with pricing registry
  • Error counters by semantic type
  • Prometheus and OTLP export options
  • Operational counters for sampling/validation/tools

Metadata

Metadata

Assignees

Labels

area/telemetryOTel spans, metrics, tracing, semconv

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions