Default 30s replaces the hardcoded 5s. Wired through TorchNodeServer →
_TorchHTTPServer → _get_remaining_route. Available on both the legacy
`start` subcommand and the default wizard path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Wrap wfile.write in _TrackerHandler._send_json with except BrokenPipeError
- Replace uuid4 node IDs with deterministic SHA-256 hash of endpoint+model+shards
so nodes keep the same ID on re-registration after tracker restart
- /v1/models now lists HF-repo models (not just preset models)
- /v1/coverage/{model} now resolves HF repos, not just preset names
- /v1/route response includes node_id alongside endpoint
- startup.py exposes tracker_node_id on node object and prints it in dashboard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
socket.getfqdn() returns *.localdomain names that other machines on
the same LAN (especially cross-OS) cannot resolve via DNS. When the
node is bound to 0.0.0.0 and --advertise-host is not given, probe the
outbound IP by connecting a UDP socket toward the tracker — this picks
the correct interface IP without sending any data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous logic caught 404 on heartbeat and re-registered, but the re-
registration failed silently if the tracker wasn't fully ready yet.
New approach: after any connection-refused streak, the heartbeat loop
switches to re-registration mode and keeps retrying until the tracker
accepts the registration (instead of sending heartbeats into a fresh
registry that doesn't know this node). Falls back to the 404 path for
the case where a node is purged without a full tracker restart.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the tracker restarts, nodes' registrations are lost. The heartbeat loop
was catching the 404 and printing a warning but never re-registering, leaving
the node permanently invisible until manually restarted.
Fix: on HTTP 404 heartbeat response, the loop re-posts the original
registration payload to /v1/nodes/register and updates the node_id and
heartbeat URL for subsequent beats. This also handles tracker expiry races.
The register_payload is now passed into _start_heartbeat so the thread has
everything needed for re-registration without reaching back into run_startup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/route now resolves HF model nodes (by hf_repo or short name)
in addition to preset models, using the same greedy interval-cover logic.
This allows distributed inference routing across two nodes each holding
half the model.
Endpoint dedup: re-registering the same endpoint atomically replaces the
old entry so stale registrations don't accumulate across node restarts.
Purge logging: tracker now prints when a node expires due to missed
heartbeats so operators can see dead nodes being removed.
Timing fix: heartbeat timeout raised from 30s to 90s (3 missed beats);
node heartbeat interval lowered from 30s to 20s to maintain margin.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker /v1/network/assign now accepts an optional `hf_repo` query param
to restrict assignment to a specific model, and returns `gap_found: bool`
so callers know whether they received a real gap vs a redundancy slot.
Node startup with --model-id (no explicit shard args) now queries the
tracker first for an uncovered gap for that model before defaulting to
full coverage (0..n-1). This means a second node with --model-id will
serve only the missing layers, not the whole model again.
Auto-join fallback (no --model-id) now prints why it fell through
instead of silently switching to stub-model.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker now prints a line when a node registers and on every heartbeat
received. Node prints its assigned node_id after successful registration
and starts a daemon heartbeat thread (30s interval) that logs each send.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracker:
- _NodeEntry gains hf_repo + num_layers fields (parsed from register body)
- GET /v1/network/assign — finds the biggest uncovered shard gap across
registered HF-model nodes; returns {hf_repo, shard_start, shard_end, num_layers}
- Returns 503 when no HF-model nodes are registered yet
Node startup:
- When model_id is set: registers with tracker including hf_repo + num_layers
so other nodes can auto-join this model
- When model_id is empty/None: queries /v1/network/assign, gets assigned the
missing layers, loads TorchNodeServer with the assigned shard automatically
- Fixes empty-string model_id leaking from DEFAULTS (treats "" same as None)
Usage: `meshnet-node start --tracker http://localhost:8080 --quantization bfloat16`
Node discovers what to serve and joins the network without any model flags.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
