docs: add US-020–029 issue files, ADR 0011–0014, update prd.json to 29/29

Issue files (.scratch/issues/20-29): retrospective specs for all work
done in the current sprint — hardening, route-timeout, start-layer
protocol, heartbeat stats, availability map, rolling RPM, smart
assignment, throughput routing, routing tests, relay outbound client.

ADRs (docs/adr/0011-0014):
  0011 — Auto-shard from memory budget and tracker network assignment
  0012 — X-Meshnet-Start-Layer overlapping shard execution protocol
  0013 — Rolling RPM statistics, smart assignment scoring, throughput routing
  0014 — Relay outbound client for NAT/internet pipeline hops

prd.json: US-020 through US-029 added, all marked done. ralph_progress.py
now shows 29/29 complete (100%).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dobromir Popov
2026-06-30 22:15:41 +03:00
parent d0307fcc84
commit 2b439e8a5f
15 changed files with 870 additions and 40 deletions

View File

@@ -0,0 +1,67 @@
# ADR-0012: X-Meshnet-Start-Layer protocol for overlapping shard execution
## Status: Accepted
## Context
The greedy route-selection algorithm picks a minimal set of nodes whose shard ranges
collectively cover all model layers. This is exact when shard ranges are disjoint
(node A: 011, node B: 1223). But two nodes with overlapping ranges can also cover
the full model (node A: 015, node B: 1023).
Without coordination, node B would re-run layers 1015 on top of an activation tensor
that already has those layers applied — producing silently wrong output.
The question is: who resolves the overlap, and how?
## Options considered
**A. Tracker injects start_layer per hop (chosen)**
The tracker knows the full route when it builds `X-Meshnet-Route`. It computes
`covered_up_to` as it walks the route and sets `start_layer = covered_up_to + 1`
for each subsequent hop. The head node forwards this per-hop in
`X-Meshnet-Start-Layer`. No peer-to-peer negotiation needed.
**B. Each node negotiates with the next**
Node A would tell node B "I ran layers 015, you start from 16". This requires
node A to know node B's shard range, which means an extra tracker lookup or
exposing shard metadata in the activation wire protocol.
**C. Strict non-overlapping enforcement**
Reject any route that contains overlapping nodes. Simpler but limits redundancy:
two nodes with the same shard can't form a route even if their combined coverage
is complete.
## Decision
Option A. The tracker is already the central coordinator; it already knows every
node's shard range. Injecting `start_layer` at route-build time costs nothing and
keeps the node implementation simple.
## Wire protocol
`X-Meshnet-Route` (JSON array, injected by tracker into the first-hop request):
```json
[
{"endpoint": "http://node-b:7002", "start_layer": 12, "relay_addr": null},
{"endpoint": "http://node-c:7003", "start_layer": 20}
]
```
`X-Meshnet-Start-Layer` (integer header, forwarded by head node to each downstream hop):
```
X-Meshnet-Start-Layer: 12
```
The receiving node passes `start_layer` to `backend.forward_bytes(start_layer=12)`.
The model shard skips transformer blocks below index 12.
## Consequences
- Overlapping shard registrations are valid and useful for redundancy
- Route selection does not need to enforce disjoint ranges
- The tracker carries the full route context; nodes are stateless w.r.t. routing
- `start_layer` must be preserved through the relay path (included in hop dict)
- Backward compatibility: if `start_layer` is absent, the node runs from its registered `shard_start`