docs: add US-020–029 issue files, ADR 0011–0014, update prd.json to 29/29
Issue files (.scratch/issues/20-29): retrospective specs for all work done in the current sprint — hardening, route-timeout, start-layer protocol, heartbeat stats, availability map, rolling RPM, smart assignment, throughput routing, routing tests, relay outbound client. ADRs (docs/adr/0011-0014): 0011 — Auto-shard from memory budget and tracker network assignment 0012 — X-Meshnet-Start-Layer overlapping shard execution protocol 0013 — Rolling RPM statistics, smart assignment scoring, throughput routing 0014 — Relay outbound client for NAT/internet pipeline hops prd.json: US-020 through US-029 added, all marked done. ralph_progress.py now shows 29/29 complete (100%). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
89
docs/adr/0013-rolling-stats-smart-routing.md
Normal file
89
docs/adr/0013-rolling-stats-smart-routing.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# ADR-0013: Rolling RPM statistics, smart assignment scoring, and throughput routing
|
||||
|
||||
## Status: Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The tracker made routing and assignment decisions blind to actual network traffic.
|
||||
Three related improvements were needed and designed together:
|
||||
|
||||
1. **Model usage statistics** — how many requests per model, so the tracker knows demand
|
||||
2. **Smart assignment** — assign new nodes to where demand × unmet coverage is highest
|
||||
3. **Throughput routing** — when multiple nodes can complete a route, pick the faster ones
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Rolling RPM counters
|
||||
|
||||
`_RollingCounter` is a circular-bucket structure where each slot covers a fixed time epoch.
|
||||
Recording a value for the current epoch increments that slot; an expired slot is silently reset
|
||||
on the next write. Three windows per model:
|
||||
|
||||
| Window | Buckets | Bucket size | Total span |
|
||||
|--------|---------|-------------|------------|
|
||||
| per_minute | 60 | 60 s | 1 hour |
|
||||
| per_hour | 24 | 3600 s | 1 day |
|
||||
| per_day | 30 | 86400 s | ~1 month |
|
||||
|
||||
`rpm()` sums all non-stale buckets and divides by total window minutes.
|
||||
|
||||
Alternative: exponential moving average (simpler, single float). Rejected because EMA
|
||||
cannot be persisted and restored without loss, and cannot be accurately merged from peer
|
||||
slices (each tracker runs its own requests, so merging EMA values doesn't give the true
|
||||
combined rate).
|
||||
|
||||
### 2. Per-tracker stat slices + additive gossip
|
||||
|
||||
Each tracker keeps only its own request slice. Gossip exchanges these slices and each tracker
|
||||
stores a `{tracker_url → {model → rpms}}` map. `get_combined_stats()` sums all slices.
|
||||
|
||||
This is additive: if tracker A sees 10 RPM for model X and tracker B sees 5 RPM, combined
|
||||
is 15 RPM. Slices are keyed by tracker URL so a stale peer update simply overwrites its
|
||||
own key without corrupting other peers' data.
|
||||
|
||||
Alternative: one global aggregator. Rejected — single point of failure, contradicts the
|
||||
distributed model.
|
||||
|
||||
### 3. Assignment scoring formula
|
||||
|
||||
```
|
||||
score(model) = (demand_rpm + 1.0) × (coverage_deficit + 0.01)
|
||||
```
|
||||
|
||||
- `demand_rpm` = `get_combined_stats()[model]["rpm_last_hour"]`
|
||||
- `coverage_deficit` = fraction of model layers with zero-node coverage ∈ [0.0, 1.0]
|
||||
- `+1.0` floor: zero-traffic models still compete by coverage
|
||||
- `+0.01` floor: fully-covered models can still attract nodes if they have high demand
|
||||
|
||||
The product ensures both dimensions matter: high demand but full coverage scores lower
|
||||
than high demand with partial coverage. Pure coverage deficits without traffic score
|
||||
lower than even modest traffic combined with any gap.
|
||||
|
||||
`price_per_token: 0.0` is returned in the assignment response, reserved for future billing.
|
||||
|
||||
### 4. Throughput tiebreak in route selection
|
||||
|
||||
```
|
||||
effective_throughput(node) = benchmark_tokens_per_sec / (queue_depth + 1)
|
||||
```
|
||||
|
||||
`_select_route` uses this as a tiebreak only: when two candidates reach the same
|
||||
maximum `shard_end`, the one with higher effective throughput is preferred.
|
||||
Coverage maximization remains the primary objective.
|
||||
|
||||
`benchmark_tokens_per_sec` comes from the hardware profile at registration.
|
||||
`queue_depth` comes from the most recent heartbeat. A busy node (high queue)
|
||||
is deprioritized without being excluded.
|
||||
|
||||
### 5. SQLite persistence
|
||||
|
||||
Stats are saved to SQLite (configurable via `--stats-db PATH`) every 60 seconds and
|
||||
on clean shutdown. Schema: `model_rpm_buckets(model, window, bucket_idx, bucket_epoch, count)`.
|
||||
The circular-bucket structure maps directly — each slot is one row.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Tracker startup is slightly slower when loading a large stats DB (sub-second for typical sizes)
|
||||
- Peer gossip adds one round-trip per gossip interval per peer
|
||||
- `price_per_token` is a stable wire field; future billing sets it to a real value
|
||||
- `effective_throughput` depends on `benchmark_tokens_per_sec` being set correctly at registration; nodes that don't set it get the default `1.0` and are treated as slowest
|
||||
Reference in New Issue
Block a user