feat: tracker-as-first-layer-node inference entry point (US-014)
- Tracker: add GET /v1/tracker-nodes/<model> returning nodes registered with tracker_mode=true whose shard_start matches the model's first layer - Node: StubNodeServer and TorchNodeServer accept tracker_mode/tracker_url; when tracker_mode=True (or auto-detected via shard_start==0 for Torch), /v1/chat/completions is served alongside /forward - TorchNodeServer: full pipeline implementation — encode_prompt → route selection via tracker → binary forward through remaining hops → decode - Gateway: _handle_chat_completions checks _get_tracker_nodes() first and proxies round-robin to tracker-nodes; falls back to existing direct pipeline when none found (preserves all US-005 backward compat) - CLI: --tracker-mode and --tracker-url flags added to meshnet-node start - Test: two stub tracker-nodes + two mid-shard nodes for gpt2; 10 requests; round-robin 5/5 split verified; all OpenAI-format responses validated - All 78 tests pass Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -346,13 +346,14 @@
|
||||
"Commit only this story's changes"
|
||||
],
|
||||
"priority": 14,
|
||||
"passes": false,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-inference-network/issues/14-tracker-as-node.md",
|
||||
"dependsOn": [
|
||||
"US-012",
|
||||
"US-013"
|
||||
],
|
||||
"status": "open"
|
||||
"status": "done",
|
||||
"completionNotes": "Implemented: (1) Tracker GET /v1/tracker-nodes/<model> endpoint returning nodes registered with tracker_mode=true at shard_start==0. (2) StubNodeServer and TorchNodeServer now accept tracker_mode/tracker_url params; when tracker_mode=True, /v1/chat/completions is served alongside /forward. (3) TorchNodeServer auto-detects tracker mode when shard_start==0. (4) Gateway _handle_chat_completions checks for tracker-nodes first via _get_tracker_nodes(), proxies round-robin if found, falls back to existing direct pipeline if none (backward compat). (5) CLI --tracker-mode and --tracker-url flags added. (6) Integration test: 2 tracker-nodes + 2 mid-shard nodes for gpt2; 10 requests; round-robin verified (5/5 split); all responses valid OpenAI format. All 78 tests pass."
|
||||
},
|
||||
{
|
||||
"id": "US-015",
|
||||
|
||||
Reference in New Issue
Block a user