# ADR-0013: Rolling RPM statistics, smart assignment scoring, and throughput routing

## Status: Accepted

## Context

The tracker made routing and assignment decisions blind to actual network traffic.
Three related improvements were needed and designed together:

1. **Model usage statistics** — how many requests per model, so the tracker knows demand
2. **Smart assignment** — assign new nodes to where demand × unmet coverage is highest
3. **Throughput routing** — when multiple nodes can complete a route, pick the faster ones

## Decisions

### 1. Rolling RPM counters

`_RollingCounter` is a circular-bucket structure where each slot covers a fixed time epoch.
Recording a value for the current epoch increments that slot; an expired slot is silently reset
on the next write. Three windows per model:

| Window | Buckets | Bucket size | Total span |
|--------|---------|-------------|------------|
| per_minute | 60 | 60 s | 1 hour |
| per_hour | 24 | 3600 s | 1 day |
| per_day | 30 | 86400 s | ~1 month |

`rpm()` sums all non-stale buckets and divides by total window minutes.

Alternative: exponential moving average (simpler, single float). Rejected because EMA
cannot be persisted and restored without loss, and cannot be accurately merged from peer
slices (each tracker runs its own requests, so merging EMA values doesn't give the true
combined rate).

### 2. Per-tracker stat slices + additive gossip

Each tracker keeps only its own request slice. Gossip exchanges these slices and each tracker
stores a `{tracker_url → {model → rpms}}` map. `get_combined_stats()` sums all slices.

This is additive: if tracker A sees 10 RPM for model X and tracker B sees 5 RPM, combined
is 15 RPM. Slices are keyed by tracker URL so a stale peer update simply overwrites its
own key without corrupting other peers' data.

Alternative: one global aggregator. Rejected — single point of failure, contradicts the
distributed model.

### 3. Assignment scoring formula

```
score(model) = (demand_rpm + 1.0) × (coverage_deficit + 0.01)
```

- `demand_rpm` = `get_combined_stats()[model]["rpm_last_hour"]`
- `coverage_deficit` = fraction of model layers with zero-node coverage ∈ [0.0, 1.0]
- `+1.0` floor: zero-traffic models still compete by coverage
- `+0.01` floor: fully-covered models can still attract nodes if they have high demand

The product ensures both dimensions matter: high demand but full coverage scores lower
than high demand with partial coverage. Pure coverage deficits without traffic score
lower than even modest traffic combined with any gap.

`price_per_token: 0.0` is returned in the assignment response, reserved for future billing.

### 4. Throughput tiebreak in route selection

```
effective_throughput(node) = benchmark_tokens_per_sec / (queue_depth + 1)
```

`_select_route` uses this as a tiebreak only: when two candidates reach the same
maximum `shard_end`, the one with higher effective throughput is preferred.
Coverage maximization remains the primary objective.

`benchmark_tokens_per_sec` comes from the hardware profile at registration.
`queue_depth` comes from the most recent heartbeat. A busy node (high queue)
is deprioritized without being excluded.

### 5. SQLite persistence

Stats are saved to SQLite (configurable via `--stats-db PATH`) every 60 seconds and
on clean shutdown. Schema: `model_rpm_buckets(model, window, bucket_idx, bucket_epoch, count)`.
The circular-bucket structure maps directly — each slot is one row.

## Consequences

- Tracker startup is slightly slower when loading a large stats DB (sub-second for typical sizes)
- Peer gossip adds one round-trip per gossip interval per peer
- `price_per_token` is a stable wire field; future billing sets it to a real value
- `effective_throughput` depends on `benchmark_tokens_per_sec` being set correctly at registration; nodes that don't set it get the default `1.0` and are treated as slowest