2b439e8a5f
Issue files (.scratch/issues/20-29): retrospective specs for all work done in the current sprint — hardening, route-timeout, start-layer protocol, heartbeat stats, availability map, rolling RPM, smart assignment, throughput routing, routing tests, relay outbound client. ADRs (docs/adr/0011-0014): 0011 — Auto-shard from memory budget and tracker network assignment 0012 — X-Meshnet-Start-Layer overlapping shard execution protocol 0013 — Rolling RPM statistics, smart assignment scoring, throughput routing 0014 — Relay outbound client for NAT/internet pipeline hops prd.json: US-020 through US-029 added, all marked done. ralph_progress.py now shows 29/29 complete (100%). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
68 lines
2.6 KiB
Markdown
68 lines
2.6 KiB
Markdown
# ADR-0012: X-Meshnet-Start-Layer protocol for overlapping shard execution
|
||
|
||
## Status: Accepted
|
||
|
||
## Context
|
||
|
||
The greedy route-selection algorithm picks a minimal set of nodes whose shard ranges
|
||
collectively cover all model layers. This is exact when shard ranges are disjoint
|
||
(node A: 0–11, node B: 12–23). But two nodes with overlapping ranges can also cover
|
||
the full model (node A: 0–15, node B: 10–23).
|
||
|
||
Without coordination, node B would re-run layers 10–15 on top of an activation tensor
|
||
that already has those layers applied — producing silently wrong output.
|
||
|
||
The question is: who resolves the overlap, and how?
|
||
|
||
## Options considered
|
||
|
||
**A. Tracker injects start_layer per hop (chosen)**
|
||
The tracker knows the full route when it builds `X-Meshnet-Route`. It computes
|
||
`covered_up_to` as it walks the route and sets `start_layer = covered_up_to + 1`
|
||
for each subsequent hop. The head node forwards this per-hop in
|
||
`X-Meshnet-Start-Layer`. No peer-to-peer negotiation needed.
|
||
|
||
**B. Each node negotiates with the next**
|
||
Node A would tell node B "I ran layers 0–15, you start from 16". This requires
|
||
node A to know node B's shard range, which means an extra tracker lookup or
|
||
exposing shard metadata in the activation wire protocol.
|
||
|
||
**C. Strict non-overlapping enforcement**
|
||
Reject any route that contains overlapping nodes. Simpler but limits redundancy:
|
||
two nodes with the same shard can't form a route even if their combined coverage
|
||
is complete.
|
||
|
||
## Decision
|
||
|
||
Option A. The tracker is already the central coordinator; it already knows every
|
||
node's shard range. Injecting `start_layer` at route-build time costs nothing and
|
||
keeps the node implementation simple.
|
||
|
||
## Wire protocol
|
||
|
||
`X-Meshnet-Route` (JSON array, injected by tracker into the first-hop request):
|
||
|
||
```json
|
||
[
|
||
{"endpoint": "http://node-b:7002", "start_layer": 12, "relay_addr": null},
|
||
{"endpoint": "http://node-c:7003", "start_layer": 20}
|
||
]
|
||
```
|
||
|
||
`X-Meshnet-Start-Layer` (integer header, forwarded by head node to each downstream hop):
|
||
|
||
```
|
||
X-Meshnet-Start-Layer: 12
|
||
```
|
||
|
||
The receiving node passes `start_layer` to `backend.forward_bytes(start_layer=12)`.
|
||
The model shard skips transformer blocks below index 12.
|
||
|
||
## Consequences
|
||
|
||
- Overlapping shard registrations are valid and useful for redundancy
|
||
- Route selection does not need to enforce disjoint ranges
|
||
- The tracker carries the full route context; nodes are stateless w.r.t. routing
|
||
- `start_layer` must be preserved through the relay path (included in hop dict)
|
||
- Backward compatibility: if `start_layer` is absent, the node runs from its registered `shard_start`
|