# ADR-0012: X-Meshnet-Start-Layer protocol for overlapping shard execution

## Status: Accepted

## Context

The greedy route-selection algorithm picks a minimal set of nodes whose shard ranges
collectively cover all model layers. This is exact when shard ranges are disjoint
(node A: 0–11, node B: 12–23). But two nodes with overlapping ranges can also cover
the full model (node A: 0–15, node B: 10–23).

Without coordination, node B would re-run layers 10–15 on top of an activation tensor
that already has those layers applied — producing silently wrong output.

The question is: who resolves the overlap, and how?

## Options considered

**A. Tracker injects start_layer per hop (chosen)**  
The tracker knows the full route when it builds `X-Meshnet-Route`. It computes
`covered_up_to` as it walks the route and sets `start_layer = covered_up_to + 1`
for each subsequent hop. The head node forwards this per-hop in
`X-Meshnet-Start-Layer`. No peer-to-peer negotiation needed.

**B. Each node negotiates with the next**  
Node A would tell node B "I ran layers 0–15, you start from 16". This requires
node A to know node B's shard range, which means an extra tracker lookup or
exposing shard metadata in the activation wire protocol.

**C. Strict non-overlapping enforcement**  
Reject any route that contains overlapping nodes. Simpler but limits redundancy:
two nodes with the same shard can't form a route even if their combined coverage
is complete.

## Decision

Option A. The tracker is already the central coordinator; it already knows every
node's shard range. Injecting `start_layer` at route-build time costs nothing and
keeps the node implementation simple.

## Wire protocol

`X-Meshnet-Route` (JSON array, injected by tracker into the first-hop request):

```json
[
  {"endpoint": "http://node-b:7002", "start_layer": 12, "relay_addr": null},
  {"endpoint": "http://node-c:7003", "start_layer": 20}
]
```

`X-Meshnet-Start-Layer` (integer header, forwarded by head node to each downstream hop):

```
X-Meshnet-Start-Layer: 12
```

The receiving node passes `start_layer` to `backend.forward_bytes(start_layer=12)`.
The model shard skips transformer blocks below index 12.

## Consequences

- Overlapping shard registrations are valid and useful for redundancy
- Route selection does not need to enforce disjoint ranges
- The tracker carries the full route context; nodes are stateless w.r.t. routing
- `start_layer` must be preserved through the relay path (included in hop dict)
- Backward compatibility: if `start_layer` is absent, the node runs from its registered `shard_start`