TokenLeak

Find the bugs behind your LLM bill spikes.

TokenLeak is a local-first CLI that analyzes LLM usage logs and shows where your token budget leaked — expensive models, costly endpoints, runaway sessions, repeated agent loops, long prompts, and sudden cost spikes.

No dashboard. No SaaS. No API key required. Logs in, leak report out.

Install

pip install tokenleak

Usage

# Analyze a usage CSV
tokenleak analyze usage.csv

# Compare against a previous period (spot spikes)
tokenleak analyze usage.csv --compare previous.csv

# Auto-detect and normalize any supported format, then analyze
tokenleak analyze-real raw_export.csv --compare raw_previous.csv --out reports/

# Save the report to a file or directory
tokenleak analyze usage.csv --out report.txt

# Use your own price table
tokenleak analyze usage.csv --pricing my_pricing.json

Supported CSV formats

analyze-real auto-detects and normalizes these:

OpenAI-style export — timestamp,model,input_tokens,output_tokens,num_requests[,cost_usd] Generic app logs — timestamp,model,route,user,session,prompt_tokens,completion_tokens[,cost] TokenLeak native — timestamp,model,endpoint,session_id,tokens_in,tokens_out[,cost]

If a row has no cost column, TokenLeak computes it from the pricing table.

Example output

$ tokenleak analyze-real examples/openai_usage.csv

TOTAL COST
----------------------------------------------------------------------
  Current:      $0.59
  Requests:     7
  Tokens (in):  82,000
  Tokens (out): 9,200

TOP EXPENSIVE MODELS
----------------------------------------------------------------------
  1. gpt-5.5                                         $0.58 ( 98.9%)
  2. gpt-5-mini                                      $0.01 (  1.1%)

⚠️  LONG PROMPT WARNING
----------------------------------------------------------------------
  Average input tokens: 11,714
  Consider chunking or summarizing inputs

Pricing

Costs come from a bundled, dated pricing.json with 2026 rates in USD per 1M tokens, sourced from official provider docs (Anthropic, OpenAI, Google). Model names are prefix-matched longest-key-first, so a specific version (gpt-5.5) is never shadowed by a shorter prefix (gpt-5).

• A model that isn't in the table is priced at a documented fallback rate and flagged with an ⚠️ ESTIMATED (UNKNOWN) MODELS warning — never a silent guess.
• Override or extend the rates with your own file: --pricing my_pricing.json (its models merge over the bundled ones).

Architecture

• pricing.py — load/merge the price table; get_cost() → (cost, is_estimated).
• analyzer.py — pure logic: load, validate, normalize, analyze, compare.
• report.py — formats the plain-text report.
• cli.py — Click commands (analyze, analyze-real).

Development

pip install -e ".[dev]"
pytest
ruff check .

License

MIT — see LICENSE.