docs: add US-020–029 issue files, ADR 0011–0014, update prd.json to 29/29

Issue files (.scratch/issues/20-29): retrospective specs for all work
done in the current sprint — hardening, route-timeout, start-layer
protocol, heartbeat stats, availability map, rolling RPM, smart
assignment, throughput routing, routing tests, relay outbound client.

ADRs (docs/adr/0011-0014):
  0011 — Auto-shard from memory budget and tracker network assignment
  0012 — X-Meshnet-Start-Layer overlapping shard execution protocol
  0013 — Rolling RPM statistics, smart assignment scoring, throughput routing
  0014 — Relay outbound client for NAT/internet pipeline hops

prd.json: US-020 through US-029 added, all marked done. ralph_progress.py
now shows 29/29 complete (100%).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs/adr/0011-auto-shard-and-network-assignment.md

@@ -0,0 +1,73 @@
+# ADR-0011: Auto-shard from memory budget and tracker-managed network assignment
+
+## Status: Accepted
+
+## Context
+
+Early node startup required explicit `--shard-start` and `--shard-end` flags. This is
+fine for expert operators but a barrier to new participants who don't know how many layers
+their GPU can hold. Two improvements were needed:
+
+1. **Auto-detect shard range**: fetch `num_hidden_layers` from the model's `config.json`
+   and compute how many layers fit in available VRAM.
+2. **Network-aware assignment**: instead of each node picking its own shard, the tracker
+   knows the current coverage map and can tell the node which gap to fill.
+
+## Decisions
+
+### 1. Layer count from HuggingFace config
+
+`AutoConfig.from_pretrained(model_id)` downloads only `config.json` (~1 KB, no weights).
+`cfg.num_hidden_layers` gives the total layer count. The node uses this to set
+`shard_end = num_layers - 1` when no explicit range is given.
+
+A curated `MODEL_CATALOG` in `model_catalog.py` provides layer counts for common models
+without any network call — HuggingFace is only hit for uncatalogued repos.
+
+### 2. VRAM-aware shard sizing
+
+`hardware.detect_hardware()` returns `vram_mb`. The node sends this to
+`/v1/network/assign?device=cuda&vram_mb=<n>&hf_repo=<repo>`. The tracker responds with
+a `{shard_start, shard_end}` gap that fits within the reported VRAM budget using the
+`bytes_per_layer` table from the model preset.
+
+When the tracker has no registered nodes for the model yet, `gap_found: false` is
+returned and the node defaults to the full model.
+
+### 3. --memory override
+
+`--memory MB` allows overriding the detected VRAM. Useful for CPU nodes (which report 0
+VRAM) that want to serve a specific slice using system RAM.
+
+### 4. Tracker network assignment endpoint
+
+`GET /v1/network/assign` replaces the old `GET /v1/nodes/assign`. It accepts
+`device`, `vram_mb`, and optionally `hf_repo`. It returns:
+
+```json
+{
+  "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+  "shard_start": 12,
+  "shard_end": 23,
+  "num_layers": 24,
+  "gap_found": true,
+  "price_per_token": 0.0
+}
+```
+
+`price_per_token` is reserved at 0.0 for future billing integration.
+
+## Alternatives rejected
+
+**Fixed shard table per model**: would require updating the code for every new model.
+HuggingFace config fetch is more general.
+
+**Node computes its own gap**: requires the node to know the full coverage map. The
+tracker already has this; having the tracker compute the assignment is cleaner.
+
+## Consequences
+
+- Nodes can join the network with a single command: `meshnet-node start --tracker <url>`
+- The tracker is now the authoritative source for shard assignment
+- VRAM budgets are advisory — nodes can still pin a range with explicit flags
+- `price_per_token: 0.0` is a stable protocol field; future billing sets it to a real value

docs/adr/0012-start-layer-overlapping-shards.md

@@ -0,0 +1,67 @@
+# ADR-0012: X-Meshnet-Start-Layer protocol for overlapping shard execution
+
+## Status: Accepted
+
+## Context
+
+The greedy route-selection algorithm picks a minimal set of nodes whose shard ranges
+collectively cover all model layers. This is exact when shard ranges are disjoint
+(node A: 0–11, node B: 12–23). But two nodes with overlapping ranges can also cover
+the full model (node A: 0–15, node B: 10–23).
+
+Without coordination, node B would re-run layers 10–15 on top of an activation tensor
+that already has those layers applied — producing silently wrong output.
+
+The question is: who resolves the overlap, and how?
+
+## Options considered
+
+**A. Tracker injects start_layer per hop (chosen)**  
+The tracker knows the full route when it builds `X-Meshnet-Route`. It computes
+`covered_up_to` as it walks the route and sets `start_layer = covered_up_to + 1`
+for each subsequent hop. The head node forwards this per-hop in
+`X-Meshnet-Start-Layer`. No peer-to-peer negotiation needed.
+
+**B. Each node negotiates with the next**  
+Node A would tell node B "I ran layers 0–15, you start from 16". This requires
+node A to know node B's shard range, which means an extra tracker lookup or
+exposing shard metadata in the activation wire protocol.
+
+**C. Strict non-overlapping enforcement**  
+Reject any route that contains overlapping nodes. Simpler but limits redundancy:
+two nodes with the same shard can't form a route even if their combined coverage
+is complete.
+
+## Decision
+
+Option A. The tracker is already the central coordinator; it already knows every
+node's shard range. Injecting `start_layer` at route-build time costs nothing and
+keeps the node implementation simple.
+
+## Wire protocol
+
+`X-Meshnet-Route` (JSON array, injected by tracker into the first-hop request):
+
+```json
+[
+  {"endpoint": "http://node-b:7002", "start_layer": 12, "relay_addr": null},
+  {"endpoint": "http://node-c:7003", "start_layer": 20}
+]
+```
+
+`X-Meshnet-Start-Layer` (integer header, forwarded by head node to each downstream hop):
+
+```
+X-Meshnet-Start-Layer: 12
+```
+
+The receiving node passes `start_layer` to `backend.forward_bytes(start_layer=12)`.
+The model shard skips transformer blocks below index 12.
+
+## Consequences
+
+- Overlapping shard registrations are valid and useful for redundancy
+- Route selection does not need to enforce disjoint ranges
+- The tracker carries the full route context; nodes are stateless w.r.t. routing
+- `start_layer` must be preserved through the relay path (included in hop dict)
+- Backward compatibility: if `start_layer` is absent, the node runs from its registered `shard_start`

docs/adr/0013-rolling-stats-smart-routing.md

@@ -0,0 +1,89 @@
+# ADR-0013: Rolling RPM statistics, smart assignment scoring, and throughput routing
+
+## Status: Accepted
+
+## Context
+
+The tracker made routing and assignment decisions blind to actual network traffic.
+Three related improvements were needed and designed together:
+
+1. **Model usage statistics** — how many requests per model, so the tracker knows demand
+2. **Smart assignment** — assign new nodes to where demand × unmet coverage is highest
+3. **Throughput routing** — when multiple nodes can complete a route, pick the faster ones
+
+## Decisions
+
+### 1. Rolling RPM counters
+
+`_RollingCounter` is a circular-bucket structure where each slot covers a fixed time epoch.
+Recording a value for the current epoch increments that slot; an expired slot is silently reset
+on the next write. Three windows per model:
+
+| Window | Buckets | Bucket size | Total span |
+|--------|---------|-------------|------------|
+| per_minute | 60 | 60 s | 1 hour |
+| per_hour | 24 | 3600 s | 1 day |
+| per_day | 30 | 86400 s | ~1 month |
+
+`rpm()` sums all non-stale buckets and divides by total window minutes.
+
+Alternative: exponential moving average (simpler, single float). Rejected because EMA
+cannot be persisted and restored without loss, and cannot be accurately merged from peer
+slices (each tracker runs its own requests, so merging EMA values doesn't give the true
+combined rate).
+
+### 2. Per-tracker stat slices + additive gossip
+
+Each tracker keeps only its own request slice. Gossip exchanges these slices and each tracker
+stores a `{tracker_url → {model → rpms}}` map. `get_combined_stats()` sums all slices.
+
+This is additive: if tracker A sees 10 RPM for model X and tracker B sees 5 RPM, combined
+is 15 RPM. Slices are keyed by tracker URL so a stale peer update simply overwrites its
+own key without corrupting other peers' data.
+
+Alternative: one global aggregator. Rejected — single point of failure, contradicts the
+distributed model.
+
+### 3. Assignment scoring formula
+
+```
+score(model) = (demand_rpm + 1.0) × (coverage_deficit + 0.01)
+```
+
+- `demand_rpm` = `get_combined_stats()[model]["rpm_last_hour"]`
+- `coverage_deficit` = fraction of model layers with zero-node coverage ∈ [0.0, 1.0]
+- `+1.0` floor: zero-traffic models still compete by coverage
+- `+0.01` floor: fully-covered models can still attract nodes if they have high demand
+
+The product ensures both dimensions matter: high demand but full coverage scores lower
+than high demand with partial coverage. Pure coverage deficits without traffic score
+lower than even modest traffic combined with any gap.
+
+`price_per_token: 0.0` is returned in the assignment response, reserved for future billing.
+
+### 4. Throughput tiebreak in route selection
+
+```
+effective_throughput(node) = benchmark_tokens_per_sec / (queue_depth + 1)
+```
+
+`_select_route` uses this as a tiebreak only: when two candidates reach the same
+maximum `shard_end`, the one with higher effective throughput is preferred.
+Coverage maximization remains the primary objective.
+
+`benchmark_tokens_per_sec` comes from the hardware profile at registration.
+`queue_depth` comes from the most recent heartbeat. A busy node (high queue)
+is deprioritized without being excluded.
+
+### 5. SQLite persistence
+
+Stats are saved to SQLite (configurable via `--stats-db PATH`) every 60 seconds and
+on clean shutdown. Schema: `model_rpm_buckets(model, window, bucket_idx, bucket_epoch, count)`.
+The circular-bucket structure maps directly — each slot is one row.
+
+## Consequences
+
+- Tracker startup is slightly slower when loading a large stats DB (sub-second for typical sizes)
+- Peer gossip adds one round-trip per gossip interval per peer
+- `price_per_token` is a stable wire field; future billing sets it to a real value
+- `effective_throughput` depends on `benchmark_tokens_per_sec` being set correctly at registration; nodes that don't set it get the default `1.0` and are treated as slowest

docs/adr/0014-relay-outbound-client.md

@@ -0,0 +1,120 @@
+# ADR-0014: Relay outbound client for NAT/internet pipeline hops
+
+## Status: Accepted
+
+## Context
+
+ADR-0010 describes the relay server: a public WebSocket hub where nodes behind NAT
+connect outbound and register as reachable peers. That ADR focused on the *inbound*
+side: how the tracker reaches a behind-NAT node for the initial chat request.
+
+The *pipeline hop* problem is different: when node A has the head shard and node B
+(behind NAT) has the tail shard, node A must forward binary activations to node B
+for *every generated token*. Direct HTTP from A to B is blocked. The relay must
+carry this per-hop activation traffic.
+
+### Why this is harder than tracker → node
+
+The tracker-to-node relay (ADR-0010) proxies a single JSON request. The activation
+hop carries raw bfloat16 tensors — binary data that must survive round-tripping
+through the relay's JSON message envelope without precision loss.
+
+Also, the relay `/rpc/{peer_id}` endpoint (one WebSocket connection per request)
+must be opened and closed for every token in the autoregressive loop. Latency
+of connection setup matters.
+
+## Options considered
+
+**A. Relay hop (WebSocket per hop, chosen)**  
+Node A opens a WebSocket to `wss://relay/rpc/{peer_id_B}`, sends the activation,
+receives the response, closes. The relay's `_handle_rpc` forwards it to B's persistent
+connection via the existing `relay-http-request` envelope mechanism.
+
+Pros: reuses the existing relay server unchanged. Each hop is independent; failures don't
+affect other requests.
+
+Cons: WebSocket connection setup adds ~50–150 ms per hop on a fast relay. For
+autoregressive inference (N tokens × M hops), this adds up.
+
+**B. Persistent per-session tunnel**  
+Node A opens a persistent WebSocket to the relay for the duration of an inference session
+and multiplexes all token hops over it.
+
+Pros: amortises connection setup across tokens.
+
+Cons: requires session-level state on the relay; complicates relay shutdown/failover;
+the current relay is stateless by design. Deferred for a future optimization.
+
+**C. Tracker-proxied activations**  
+Route all activation traffic through the tracker's HTTP proxy.
+
+Cons: the tracker is the control plane, not the data plane. High-volume binary tensor
+traffic through the tracker would saturate it. Rejected.
+
+## Decision
+
+Option A — per-hop WebSocket relay. Simple, reuses existing infrastructure, correct.
+Option B is noted as a future optimization when activation-path latency becomes the
+bottleneck.
+
+## Protocol
+
+```
+Node A opens WS → wss://relay/rpc/{peer_id_B}
+Node A sends:
+  {
+    "request_id": "<hex>",
+    "method": "POST",
+    "path": "/forward",
+    "headers": { "X-Meshnet-Shape": "...", "X-Meshnet-Start-Layer": "12", ... },
+    "body_base64": "<base64(bfloat16 tensor)>"
+  }
+
+Relay forwards to Node B as relay-http-request envelope.
+Node B's RelayHttpBridge decodes body_base64, calls POST /forward locally.
+Response:
+  {
+    "request_id": "<hex>",
+    "status": 200,
+    "headers": { "x-meshnet-shape": "...", "content-type": "application/octet-stream" },
+    "body_base64": "<base64(output tensor)>"   ← for binary responses
+    # OR
+    "body": "<json string>"                    ← for text (last-hop decode)
+  }
+Relay sends response JSON back to Node A.
+Node A decodes body_base64, continues pipeline.
+```
+
+### Binary data through JSON: base64
+
+Raw bfloat16 bytes cannot safely transit JSON (no UTF-8 guarantee, lossy decode).
+`body_base64` carries the tensor as base64; the bridge decodes it before calling
+the local HTTP endpoint, and re-encodes the response. No precision loss.
+
+Text responses (final hop, `application/json`) use `body` (plain string) for efficiency.
+
+### Fallback
+
+If `_relay_hop` raises (relay unreachable, peer disconnected), `_run_downstream_pipeline`
+logs a warning and retries via direct HTTP. If both fail, the hop returns a pipeline error
+string and the token is skipped.
+
+### Tracker injection
+
+The tracker's `_handle_proxy_chat` includes `relay_addr` in each downstream hop dict
+when the node has one registered:
+
+```json
+{"endpoint": "http://172.29.x.x:7002", "start_layer": 12, "relay_addr": "wss://relay/rpc/abc123"}
+```
+
+The head node reads `relay_addr` from the injected `X-Meshnet-Route` header and calls
+`_relay_hop` instead of direct HTTP.
+
+## Consequences
+
+- Nodes behind NAT (WSL2, 5G, home routers) can now participate in distributed pipeline inference without opening firewall ports
+- `relay_addr` is a stable registration field; nodes without a relay omit it and receive direct HTTP hops
+- Per-hop WebSocket setup adds latency proportional to relay RTT; acceptable for prototype, optimize later with persistent tunnels
+- Base64 encoding increases payload size by ~33%; acceptable for prototype
+- The relay server remains stateless and horizontally scalable; only the persistent per-peer `/ws` connections are stateful