d1e75ddded
Move issues (01–29) and PRD from .scratch/distributed-inference-network/ into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path and rewrite docs/agents/issue-tracker.md to reflect the new layout. The distributed_inference_network.egg-info/docs/ mirror is a build artifact already covered by *.egg-info/ in .gitignore — not committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1.6 KiB
Status: ready-for-agent
09 — P2P shard swarm
What to build
Once a node has downloaded a shard, it seeds that shard to other nodes that are assigned the same shard. New nodes with the same shard assignment should download from the swarm (peer nodes) rather than HuggingFace when peers are available, reducing HuggingFace load and speeding up propagation of popular shards.
The shard swarm is keyed by (model_preset, shard_index). Nodes announce their shard availability to the tracker on registration. When a new node requests a shard download, the tracker returns a list of peer endpoints that hold the shard alongside the HuggingFace fallback URL. The node tries peers first; falls back to HuggingFace if no peers respond within a timeout.
Peers serve shard data over a simple chunked HTTP endpoint — no custom P2P protocol needed in v1. The node client displays download source (peer or HuggingFace) during shard download.
Acceptance criteria
- A node that has downloaded a shard is listed as a peer for that shard in the tracker
- A second node assigned the same shard downloads it from the first node (peer) rather than HuggingFace (verified by asserting no HuggingFace HTTP calls are made in the test)
- If no peers are available, the node falls back to HuggingFace without error
- The node client logs whether the shard was downloaded from a peer or HuggingFace
- Shard chunks are verified against a checksum before being marked complete
- An integration test: node A downloads shard from HuggingFace stub; node B with same assignment downloads from node A
Blocked by
04-node-client-startup.md