tracker download fix

This commit is contained in:
Dobromir Popov
2026-07-06 18:36:42 +03:00
parent 2e696be80f
commit d151dd5484
6 changed files with 159 additions and 43 deletions

View File

@@ -38,6 +38,17 @@ What exists already (build on it, don't duplicate):
subset (new) → HF `snapshot_download` with `allow_patterns` for the same
subset (new — stop downloading the whole repo even from HF) → full snapshot
(last resort).
- The `allow_patterns` subset must not depend on the tracker having a local
snapshot: when the tracker has no `--models-dir` match for a repo (or
hasn't cached it yet — the common case for a fresh public tracker),
`model_sources` comes back empty and `download_shard` falls straight to
`_download_huggingface_subset(..., allow_patterns=None)`, i.e. the full
repo. Reported 2026-07-06: a CPU node assigned layers 02 of
`unsloth/Qwen3.6-35B-A3B` (42 safetensor shards) sat downloading the
entire model unauthenticated because of this. Fix: fetch
`model.safetensors.index.json` + `config.json` directly from HF (a few
KB) and compute the same layer-scoped file subset client-side, so the
HF-fallback path is filtered even with an empty `model_sources`.
4. **Partial LOAD (the hard half).** Downloading a subset is wasted unless the
node stops instantiating the full model: build the model skeleton on the
`meta` device, materialize only assigned layers (+embeddings/norm/head as
@@ -64,6 +75,11 @@ What exists already (build on it, don't duplicate):
- [x] Node downloader keeps exact-shard peers first, then races tracker model
sources against a HuggingFace `snapshot_download(..., allow_patterns=...)`
subset download, using the first successful source.
- [ ] When no tracker model source is available at all, the HuggingFace
fallback still computes `allow_patterns` from the repo's own
`model.safetensors.index.json` (fetched directly, not via the tracker) —
it never silently downloads the full model just because the tracker has
nothing cached.
- [x] Real PyTorch model startup can use tracker `full_url` sources to fetch
the full local snapshot over LAN before `from_pretrained`, so local-network
testing no longer has to pull from HuggingFace first.