dual billing; tracker to node model sharing

This commit is contained in:
Dobromir Popov
2026-07-06 17:31:11 +03:00
parent ccb69c41e3
commit 2e696be80f
14 changed files with 1092 additions and 41 deletions

View File

@@ -1,6 +1,6 @@
# US-044 — Tracker as model-file source; nodes download only their shard
Status: planned
Status: in progress
Priority: High (blocks multi-machine big-model serving; pairs with US-042)
Stage: Designed (grill remaining decisions before build)
@@ -56,10 +56,30 @@ What exists already (build on it, don't duplicate):
## Acceptance criteria
- Two-machine test: machine A (tracker + node, holds full snapshot) serves
- [x] Tracker can be started with `--models-dir PATH` / `MESHNET_MODELS_DIR`
and advertises a local model-file source in assignment responses when it has
a matching HF snapshot.
- [x] Tracker serves a tar stream containing only the safetensors files selected
for the assigned layer range plus config/tokenizer/index metadata.
- [x] Node downloader keeps exact-shard peers first, then races tracker model
sources against a HuggingFace `snapshot_download(..., allow_patterns=...)`
subset download, using the first successful source.
- [x] Real PyTorch model startup can use tracker `full_url` sources to fetch
the full local snapshot over LAN before `from_pretrained`, so local-network
testing no longer has to pull from HuggingFace first.
- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
layers 0k; machine B joins with no model and receives **only** the files
for its assigned range from A — nothing fetched from HF
- Machine B's resident memory scales with its shard size, not model size
- Checksums verified end-to-end; corrupted transfer falls back cleanly
- Single-node/full-model flows unchanged
- `python -m pytest` passes from repo root
- [ ] Machine B's resident memory scales with its shard size, not model size
- [ ] Checksums verified end-to-end; corrupted transfer falls back cleanly
- [x] Single-node/full-model flows unchanged
- [ ] `python -m pytest` passes from repo root
## Implementation notes
- 2026-07-06: Added the tracker/node download path. For immediate Qwen3.6-35B
LAN testing, real PyTorch nodes fetch the full snapshot from the tracker via
`full_url` and race HuggingFace as fallback. Remaining hard half is true
partial model materialization: the backend can prefer a downloaded local
model directory, but Transformers still needs a `meta`-device load path that
materializes only assigned layers.