2b439e8a5f
Issue files (.scratch/issues/20-29): retrospective specs for all work done in the current sprint — hardening, route-timeout, start-layer protocol, heartbeat stats, availability map, rolling RPM, smart assignment, throughput routing, routing tests, relay outbound client. ADRs (docs/adr/0011-0014): 0011 — Auto-shard from memory budget and tracker network assignment 0012 — X-Meshnet-Start-Layer overlapping shard execution protocol 0013 — Rolling RPM statistics, smart assignment scoring, throughput routing 0014 — Relay outbound client for NAT/internet pipeline hops prd.json: US-020 through US-029 added, all marked done. ralph_progress.py now shows 29/29 complete (100%). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
90 lines
3.8 KiB
Markdown
90 lines
3.8 KiB
Markdown
# ADR-0013: Rolling RPM statistics, smart assignment scoring, and throughput routing
|
||
|
||
## Status: Accepted
|
||
|
||
## Context
|
||
|
||
The tracker made routing and assignment decisions blind to actual network traffic.
|
||
Three related improvements were needed and designed together:
|
||
|
||
1. **Model usage statistics** — how many requests per model, so the tracker knows demand
|
||
2. **Smart assignment** — assign new nodes to where demand × unmet coverage is highest
|
||
3. **Throughput routing** — when multiple nodes can complete a route, pick the faster ones
|
||
|
||
## Decisions
|
||
|
||
### 1. Rolling RPM counters
|
||
|
||
`_RollingCounter` is a circular-bucket structure where each slot covers a fixed time epoch.
|
||
Recording a value for the current epoch increments that slot; an expired slot is silently reset
|
||
on the next write. Three windows per model:
|
||
|
||
| Window | Buckets | Bucket size | Total span |
|
||
|--------|---------|-------------|------------|
|
||
| per_minute | 60 | 60 s | 1 hour |
|
||
| per_hour | 24 | 3600 s | 1 day |
|
||
| per_day | 30 | 86400 s | ~1 month |
|
||
|
||
`rpm()` sums all non-stale buckets and divides by total window minutes.
|
||
|
||
Alternative: exponential moving average (simpler, single float). Rejected because EMA
|
||
cannot be persisted and restored without loss, and cannot be accurately merged from peer
|
||
slices (each tracker runs its own requests, so merging EMA values doesn't give the true
|
||
combined rate).
|
||
|
||
### 2. Per-tracker stat slices + additive gossip
|
||
|
||
Each tracker keeps only its own request slice. Gossip exchanges these slices and each tracker
|
||
stores a `{tracker_url → {model → rpms}}` map. `get_combined_stats()` sums all slices.
|
||
|
||
This is additive: if tracker A sees 10 RPM for model X and tracker B sees 5 RPM, combined
|
||
is 15 RPM. Slices are keyed by tracker URL so a stale peer update simply overwrites its
|
||
own key without corrupting other peers' data.
|
||
|
||
Alternative: one global aggregator. Rejected — single point of failure, contradicts the
|
||
distributed model.
|
||
|
||
### 3. Assignment scoring formula
|
||
|
||
```
|
||
score(model) = (demand_rpm + 1.0) × (coverage_deficit + 0.01)
|
||
```
|
||
|
||
- `demand_rpm` = `get_combined_stats()[model]["rpm_last_hour"]`
|
||
- `coverage_deficit` = fraction of model layers with zero-node coverage ∈ [0.0, 1.0]
|
||
- `+1.0` floor: zero-traffic models still compete by coverage
|
||
- `+0.01` floor: fully-covered models can still attract nodes if they have high demand
|
||
|
||
The product ensures both dimensions matter: high demand but full coverage scores lower
|
||
than high demand with partial coverage. Pure coverage deficits without traffic score
|
||
lower than even modest traffic combined with any gap.
|
||
|
||
`price_per_token: 0.0` is returned in the assignment response, reserved for future billing.
|
||
|
||
### 4. Throughput tiebreak in route selection
|
||
|
||
```
|
||
effective_throughput(node) = benchmark_tokens_per_sec / (queue_depth + 1)
|
||
```
|
||
|
||
`_select_route` uses this as a tiebreak only: when two candidates reach the same
|
||
maximum `shard_end`, the one with higher effective throughput is preferred.
|
||
Coverage maximization remains the primary objective.
|
||
|
||
`benchmark_tokens_per_sec` comes from the hardware profile at registration.
|
||
`queue_depth` comes from the most recent heartbeat. A busy node (high queue)
|
||
is deprioritized without being excluded.
|
||
|
||
### 5. SQLite persistence
|
||
|
||
Stats are saved to SQLite (configurable via `--stats-db PATH`) every 60 seconds and
|
||
on clean shutdown. Schema: `model_rpm_buckets(model, window, bucket_idx, bucket_epoch, count)`.
|
||
The circular-bucket structure maps directly — each slot is one row.
|
||
|
||
## Consequences
|
||
|
||
- Tracker startup is slightly slower when loading a large stats DB (sub-second for typical sizes)
|
||
- Peer gossip adds one round-trip per gossip interval per peer
|
||
- `price_per_token` is a stable wire field; future billing sets it to a real value
|
||
- `effective_throughput` depends on `benchmark_tokens_per_sec` being set correctly at registration; nodes that don't set it get the default `1.0` and are treated as slowest
|