# US-045 — Dual-rate billing: separate input and output token prices

Status: in progress
Priority: High (billing correctness before friends test; providers all price this way)
Stage: Designed

## Context

Today the ledger has one `price_per_1k_tokens` per model, and the two proxy
paths don't even agree on what they count:

- **Non-streaming** bills `usage.total_tokens` (prompt + completion) at the
  blended rate (`_billable_non_stream_tokens`).
- **Streaming** bills `min(observed output deltas, reported total)` — output
  only in practice (`_billable_stream_tokens`).
- The HF pricing refresher (issue 23) averages a provider's input/output
  rates 50/50 (`blended_price_per_1k_tokens`), which misprices asymmetric
  models — e.g. Qwen3.6-35B-A3B on deepinfra is $0.15/1M in, $0.95/1M out.

Decision (user, 2026-07-06): charge **both** input and output tokens, at
**two separate rates**, same as other providers.

## Design

1. **`BillingLedger`** stores `{model: (input_per_1k, output_per_1k)}`.
   - `set_prices(model, input_per_1k, output_per_1k)` (new);
     `set_price(model, p)` keeps working and sets both.
   - `prices_for(model) -> (input, output)` (new); `price_for(model)` returns
     the blended average for back-compat (estimators/logs).
   - `charge_request(...)` gains keyword `input_tokens`/`output_tokens`; when
     provided, `cost = in_rate·in/1k + out_rate·out/1k` and the event records
     the split. Without them, legacy behavior (blended × total) — old events
     and gossip replicas replay unchanged (`cost` stays the applied field).
2. **Token counting** (`server.py`):
   - Non-stream: prefer `usage.prompt_tokens`/`completion_tokens`; fall back
     to content estimates (`_estimate_prompt_tokens`, observed completion),
     capped by `max_tokens` bounds as today.
   - Stream (direct + relay): output = observed deltas as today; input =
     `usage.prompt_tokens` when a usage chunk appears, else the prompt
     estimate from the request body. `_stream_line_tokens` returns the parsed
     usage triple instead of just the total.
3. **Presets**: `input_price_per_1k_tokens` / `output_price_per_1k_tokens`
   (dual keys win; `price_per_1k_tokens` alone still means "both rates").
   Qwen3.6-35B-A3B: input 0.00012, output 0.00076 (80% of deepinfra).
4. **HF refresher**: applies 80% of each side separately via `set_prices`
   (all alias keys); change log keeps recording the blended pair for history
   continuity.
5. **Spend cap** (`--max-charge-per-request`): estimate =
   `in_rate·prompt_estimate + out_rate·completion_limit`.

## Acceptance criteria

- Streamed and non-streamed requests for the same exchange bill the same
  split (input charged in both)
- A model with asymmetric provider rates bills input and output differently;
  `usage_for` / billing events expose the split
- Old persisted billing events replay byte-identically (balances unchanged)
- HF refresh sets both rates from the marketplace row, not the average
- Spend cap uses the dual rates
- `python -m pytest` passes from repo root