24 lines
1.5 KiB
Markdown
24 lines
1.5 KiB
Markdown
Status: done
|
|
|
|
# 02 — Two-node shard pipeline
|
|
|
|
## What to build
|
|
|
|
Extend the single-node smoke test so that inference is routed through exactly two nodes, each serving half of a stub model's layers. Activation tensors must travel from the gateway to node A, then from node A to node B, and the final output returns to the gateway. This proves the distributed shard pipeline works end-to-end before real models or a real tracker are introduced.
|
|
|
|
The two nodes run in the same process as the test (no real network required). The gateway hardcodes the two-node route for now — dynamic route selection comes in issue 03. The activation tensor protocol (shape, dtype, serialisation format) must be established here, as all later issues depend on it.
|
|
|
|
Use the domain vocabulary from `CONTEXT.md`: the ordered list of nodes is an **inference route**; each node's layer range is a **shard**; the tensors passed between nodes are activations (not "data" or "chunks").
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] The integration test spins up two stub nodes, each configured with a non-overlapping shard range
|
|
- [ ] A `POST /v1/chat/completions` request results in activation tensors flowing node-A → node-B (verifiable via test assertions or logs)
|
|
- [ ] The gateway assembles the final response correctly from node-B's output
|
|
- [ ] The test passes with no external services running
|
|
- [ ] The activation tensor serialisation format is documented in a short inline comment (shape, dtype, wire format) — this becomes the contract all future nodes implement
|
|
|
|
## Blocked by
|
|
|
|
- `01-monorepo-scaffold.md`
|