25 lines
1.7 KiB
Markdown
25 lines
1.7 KiB
Markdown
Status: done
|
|
|
|
# 03 — Tracker: node registration + route selection
|
|
|
|
## What to build
|
|
|
|
Replace the hardcoded two-node route from issue 02 with a real tracker. Nodes register themselves with the tracker on startup (endpoint, shard range, hardware profile, node score). The gateway queries the tracker for an inference route for a given model preset instead of using a static list. The tracker returns an ordered list of node endpoints whose shards collectively cover all layers.
|
|
|
|
The tracker runs as a lightweight HTTP service. Node score is initially a simple placeholder (e.g. fixed value or random) — real throughput/latency scoring comes later. The tracker must correctly reject route requests when no registered nodes cover a required shard range and return an appropriate error.
|
|
|
|
This issue establishes the tracker's registration and routing API contract, which issues 04 (node startup) and 05 (OpenAI gateway) both depend on.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] A node can register with the tracker via HTTP, providing its endpoint, shard range, and a hardware profile
|
|
- [ ] The gateway queries the tracker with a model preset name and receives an ordered list of node endpoints forming a complete inference route
|
|
- [ ] The end-to-end integration test from issue 02 still passes with the tracker now in the loop (no hardcoded routes)
|
|
- [ ] The tracker returns a clear error response when no route is available for a requested model preset
|
|
- [ ] Nodes that fail to heartbeat within a configurable window are removed from the registry
|
|
- [ ] The tracker's registration and route-selection HTTP API is defined (paths, request/response shapes) in the tracker package
|
|
|
|
## Blocked by
|
|
|
|
- `02-two-node-shard-pipeline.md`
|