dual billing; tracker to node model sharing
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# US-044 — Tracker as model-file source; nodes download only their shard
|
||||
|
||||
Status: planned
|
||||
Status: in progress
|
||||
Priority: High (blocks multi-machine big-model serving; pairs with US-042)
|
||||
Stage: Designed (grill remaining decisions before build)
|
||||
|
||||
@@ -56,10 +56,30 @@ What exists already (build on it, don't duplicate):
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Two-machine test: machine A (tracker + node, holds full snapshot) serves
|
||||
- [x] Tracker can be started with `--models-dir PATH` / `MESHNET_MODELS_DIR`
|
||||
and advertises a local model-file source in assignment responses when it has
|
||||
a matching HF snapshot.
|
||||
- [x] Tracker serves a tar stream containing only the safetensors files selected
|
||||
for the assigned layer range plus config/tokenizer/index metadata.
|
||||
- [x] Node downloader keeps exact-shard peers first, then races tracker model
|
||||
sources against a HuggingFace `snapshot_download(..., allow_patterns=...)`
|
||||
subset download, using the first successful source.
|
||||
- [x] Real PyTorch model startup can use tracker `full_url` sources to fetch
|
||||
the full local snapshot over LAN before `from_pretrained`, so local-network
|
||||
testing no longer has to pull from HuggingFace first.
|
||||
- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
|
||||
layers 0–k; machine B joins with no model and receives **only** the files
|
||||
for its assigned range from A — nothing fetched from HF
|
||||
- Machine B's resident memory scales with its shard size, not model size
|
||||
- Checksums verified end-to-end; corrupted transfer falls back cleanly
|
||||
- Single-node/full-model flows unchanged
|
||||
- `python -m pytest` passes from repo root
|
||||
- [ ] Machine B's resident memory scales with its shard size, not model size
|
||||
- [ ] Checksums verified end-to-end; corrupted transfer falls back cleanly
|
||||
- [x] Single-node/full-model flows unchanged
|
||||
- [ ] `python -m pytest` passes from repo root
|
||||
|
||||
## Implementation notes
|
||||
|
||||
- 2026-07-06: Added the tracker/node download path. For immediate Qwen3.6-35B
|
||||
LAN testing, real PyTorch nodes fetch the full snapshot from the tracker via
|
||||
`full_url` and race HuggingFace as fallback. Remaining hard half is true
|
||||
partial model materialization: the backend can prefer a downloaded local
|
||||
model directory, but Transformers still needs a `meta`-device load path that
|
||||
materializes only assigned layers.
|
||||
|
||||
Reference in New Issue
Block a user