feat(us-022): X-Meshnet-Start-Layer pipeline protocol for overlapping shards
When _select_route picks two nodes with overlapping registrations (e.g.
A:0-22 and B:20-24), the tracker now injects start_layer per hop so B
executes only layers 23-24, not 20-24.
- model_backend: forward_bytes + _run_layers accept start_layer offset;
clamped to shard_start to prevent out-of-bounds indexing
- torch_server: _handle_binary_forward reads X-Meshnet-Start-Layer header;
_run_downstream_pipeline sends it per hop; route is now list[tuple[str,int]]
- server: proxy injects {endpoint, start_layer} objects in X-Meshnet-Route;
/v1/route response includes start_layer per node in the nodes list
- test: fake backends accept start_layer=None kwarg
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -36,7 +36,7 @@ class _FakeBackend:
|
||||
position_ids_header=None,
|
||||
)
|
||||
|
||||
def forward_bytes(self, body, shape, attention_mask_header, position_ids_header):
|
||||
def forward_bytes(self, body, shape, attention_mask_header, position_ids_header, start_layer=None):
|
||||
assert shape == [1, 6, 8]
|
||||
return TensorPayload(
|
||||
body=body,
|
||||
@@ -50,7 +50,7 @@ class _FakeTailBackend(_FakeBackend):
|
||||
is_head = False
|
||||
is_tail = True
|
||||
|
||||
def forward_bytes(self, body, shape, attention_mask_header, position_ids_header):
|
||||
def forward_bytes(self, body, shape, attention_mask_header, position_ids_header, start_layer=None):
|
||||
assert len(body) == 1 * 6 * 8 * 2
|
||||
return " Paris"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user