Files

Dobromir Popov 2b439e8a5f docs: add US-020–029 issue files, ADR 0011–0014, update prd.json to 29/29

Issue files (.scratch/issues/20-29): retrospective specs for all work
done in the current sprint — hardening, route-timeout, start-layer
protocol, heartbeat stats, availability map, rolling RPM, smart
assignment, throughput routing, routing tests, relay outbound client.

ADRs (docs/adr/0011-0014):
  0011 — Auto-shard from memory budget and tracker network assignment
  0012 — X-Meshnet-Start-Layer overlapping shard execution protocol
  0013 — Rolling RPM statistics, smart assignment scoring, throughput routing
  0014 — Relay outbound client for NAT/internet pipeline hops

prd.json: US-020 through US-029 added, all marked done. ralph_progress.py
now shows 29/29 complete (100%).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-06-30 22:15:41 +03:00

2.6 KiB

Raw Blame History

ADR-0012: X-Meshnet-Start-Layer protocol for overlapping shard execution

Status: Accepted

Context

The greedy route-selection algorithm picks a minimal set of nodes whose shard ranges collectively cover all model layers. This is exact when shard ranges are disjoint (node A: 0–11, node B: 12–23). But two nodes with overlapping ranges can also cover the full model (node A: 0–15, node B: 10–23).

Without coordination, node B would re-run layers 10–15 on top of an activation tensor that already has those layers applied — producing silently wrong output.

The question is: who resolves the overlap, and how?

Options considered

A. Tracker injects start_layer per hop (chosen)
The tracker knows the full route when it builds X-Meshnet-Route. It computes covered_up_to as it walks the route and sets start_layer = covered_up_to + 1 for each subsequent hop. The head node forwards this per-hop in X-Meshnet-Start-Layer. No peer-to-peer negotiation needed.

B. Each node negotiates with the next
Node A would tell node B "I ran layers 0–15, you start from 16". This requires node A to know node B's shard range, which means an extra tracker lookup or exposing shard metadata in the activation wire protocol.

C. Strict non-overlapping enforcement
Reject any route that contains overlapping nodes. Simpler but limits redundancy: two nodes with the same shard can't form a route even if their combined coverage is complete.

Decision

Option A. The tracker is already the central coordinator; it already knows every node's shard range. Injecting start_layer at route-build time costs nothing and keeps the node implementation simple.

Wire protocol

X-Meshnet-Route (JSON array, injected by tracker into the first-hop request):

[
  {"endpoint": "http://node-b:7002", "start_layer": 12, "relay_addr": null},
  {"endpoint": "http://node-c:7003", "start_layer": 20}
]

X-Meshnet-Start-Layer (integer header, forwarded by head node to each downstream hop):

X-Meshnet-Start-Layer: 12

The receiving node passes start_layer to backend.forward_bytes(start_layer=12). The model shard skips transformer blocks below index 12.

Consequences

Overlapping shard registrations are valid and useful for redundancy
Route selection does not need to enforce disjoint ranges
The tracker carries the full route context; nodes are stateless w.r.t. routing
start_layer must be preserved through the relay path (included in hop dict)
Backward compatibility: if start_layer is absent, the node runs from its registered shard_start

2.6 KiB Raw Blame History Unescape Escape