Commit Graph

7 Commits

Author SHA1 Message Date
Dobromir Popov
08e9c22ccf Merge origin/master: streaming progress, dashboard call wall, and heartbeat scaffolding.
Resolve conflicts in dashboard.html (Call wall + live TPS/queue from remote) and server.py (proxy progress logging, request id forwarding, current_requests on node entries).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 17:44:18 +03:00
Dobromir Popov
e81d989f39 dash QOL 2026-07-07 17:37:38 +03:00
Dobromir Popov
3eb7c6b93e fixing streaming 2026-07-07 16:06:05 +02:00
Dobromir Popov
6c46f96aaf relaying/ RPC 2026-06-30 18:30:54 +03:00
Dobromir Popov
96af82892f feat(us-022): X-Meshnet-Start-Layer pipeline protocol for overlapping shards
When _select_route picks two nodes with overlapping registrations (e.g.
A:0-22 and B:20-24), the tracker now injects start_layer per hop so B
executes only layers 23-24, not 20-24.

- model_backend: forward_bytes + _run_layers accept start_layer offset;
  clamped to shard_start to prevent out-of-bounds indexing
- torch_server: _handle_binary_forward reads X-Meshnet-Start-Layer header;
  _run_downstream_pipeline sends it per hop; route is now list[tuple[str,int]]
- server: proxy injects {endpoint, start_layer} objects in X-Meshnet-Route;
  /v1/route response includes start_layer per node in the nodes list
- test: fake backends accept start_layer=None kwarg

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 13:14:45 +03:00
Dobromir Popov
1bdfce657d inference working 2026-06-29 23:54:35 +03:00
Dobromir Popov
2690d9b9ba feat: add real PyTorch model backend 2026-06-29 15:54:40 +03:00