US-047 — Model-source race: visibility, sane timeouts, quiet tracker aborts

Status: in progress Priority: High (follow-up to US-044/US-046; blocks usable LAN downloads)

Context

Reported 2026-07-06 (Windows CPU node, 79.2 GB RAM, --tracker http://192.168.0.179:8080 --model Qwen3.6-35B-A3B):

Startup prints (auto-join unavailable: HTTP Error 503) even though the user explicitly named a model. The auto-join query (/v1/network/assign) never sends the requested model, so a fresh tracker + a caller too small for the recommended preset 503s (expected per US-046) — but the whole auto-join step is pointless when the user already picked a model: the /v1/nodes/assign?model=… call right after it succeeds (assigned layers 0–2 with tracker model_sources).
The tracker-vs-HuggingFace race then starts, but only HuggingFace shows progress (hf tqdm bars). The tracker tar download prints nothing and swallows every failure (except Exception: return None), so the node appears to download only from slow HF; the user killed it. Tracker-side log showed the tar stream reset mid-archive.add — with no way to tell whether the client timed out or the user aborted.
_download_model_source inherits peer_timeout (2.0 s) as its urlopen socket timeout. Any 2 s read stall during a multi-GB tar stream silently kills the tracker source and leaves HF as the only contender.
Every client abort spams the tracker console with a full BrokenPipeError/ConnectionResetError traceback from socketserver.

startup.py: skip the network auto-join query entirely when a model was explicitly requested (model set and not "stub-model"); path 3b (/v1/nodes/assign?model=…) is the authoritative one there.
downloader.py: model-source downloads get their own timeout constant (30 s socket timeout) instead of the 2 s peer-probe timeout. Peer shard downloads keep 2 s — they run sequentially before the race, and a dead peer must not hang startup for 30 s; the race is concurrent so a slow source costs nothing.
downloader.py: progress + failure visibility for the race — _download_model_source prints received bytes every 512 MB and prints the exception when a source fails, so "downloads only from HF" can never happen silently again.
Tracker _handle_model_files_download: catch BrokenPipeError/ConnectionResetError around the tar stream and log a single line instead of a traceback.

Node started with an explicit --model never queries /v1/network/assign and never prints auto-join unavailable.
During the race, tracker-source progress lines appear alongside HF tqdm output; a failing tracker source prints its exception.
A ≥2 s read stall no longer aborts a tracker model-source download (30 s socket timeout).
Client disconnect during /v1/model-files/download logs one line on the tracker, no traceback.
python -m pytest passes from repo root.