auto find next available local port

Merges worktree-feat+us-016 into master. Combined both sets of _NodeEntry
fields: hf_repo/num_layers (HEAD) and relay_addr/cert_fingerprint/peer_id
(US-017). Kept HEAD's auto-shard tracker query and shard_label formatting.

US-016: mining-style startup CLI with live ASCII dashboard, hardware
detection, auto-shard range detection from model layer count, tracker
network-assign integration for gap-filling.

US-017: P2P gossip protocol, NAT-traversal relay node, TLS peer
authentication via cert fingerprint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.scratch/distributed-inference-network/issues/16-mining-cli-ux.md

@@ -0,0 +1,206 @@
+# US-016 — Mining-style node startup CLI + live dashboard
+
+## Goal
+
+Replace the bare flag-driven `meshnet-node start` with a wizard-guided first-run experience modelled on GPU mining clients (like PhoenixMiner, lolMiner, etc.). After the wizard, the terminal switches to a live status dashboard showing real-time node health and earnings.
+
+## Wizard flow (first run only)
+
+```
+╔══════════════════════════════════════════════════════════╗
+║              meshnet-node  v0.1.0                        ║
+║      Distributed AI Inference — Node Setup               ║
+╚══════════════════════════════════════════════════════════╝
+
+Detecting hardware...
+  GPU 0: NVIDIA RTX 4090  24 GB VRAM  ✓
+  GPU 1: NVIDIA RTX 3090  24 GB VRAM  ✓
+
+Select a model to serve:
+
+ #   Model                        Layers   NF4    INT8   BF16
+ 1   Llama-3-70B-Instruct         80       ✓18GB  ✓40GB  ✗80GB
+ 2   Qwen-2.5-72B-Instruct        80       ✓19GB  ✗41GB  ✗81GB
+ 3   Mixtral-8x7B-Instruct-v0.1   32       ✓ 7GB  ✓14GB  ✓27GB
+ 4   Phi-3-medium-128k-instruct   40       ✓ 4GB  ✓ 8GB  ✓15GB
+ 5   [Browse HuggingFace…]
+
+Enter number [1]: _
+
+Quantization [nf4/int8/bf16] (nf4 recommended for 24GB): _
+
+Download directory [~/.meshnet/models]: _
+
+Tracker URL [http://localhost:8080]: _
+
+Wallet path [~/.config/meshnet/wallet.json] (new wallet will be created): _
+
+Config saved to ~/.config/meshnet/config.json
+Starting node…
+```
+
+Second run with existing config:
+
+```
+meshnet-node
+Reading config from ~/.config/meshnet/config.json
+Model: Llama-3-70B-Instruct  Quant: nf4  Shard: layers 0–15
+Tracker: http://192.168.1.10:8080
+Starting…
+```
+
+## Live dashboard (once running)
+
+Renders every 2 seconds using `rich.live`. Fallback: plain-text status line if `rich` is unavailable or terminal is not a TTY (important for WSL2 / SSH).
+
+```
+meshnet-node  Llama-3-70B-Instruct [nf4]  shard 0–15/80  up 00:03:22
+
+GPU 0  RTX 4090       GPU ████████░░  73%   VRAM 18.2/24.0 GB   45°C
+GPU 1  RTX 3090       GPU ███░░░░░░░  28%   VRAM  8.7/24.0 GB   38°C
+
+Tokens/sec   ▁▂▃▄▅▆▇█  42.3 t/s (EMA 30s)
+Requests     1,247 served   3 active
+Peers        8 connected   (tracker: ✓  relay: ✓)
+TAI earned   0.00 TAI  (payments active after US-006)
+Uptime       00:03:22
+
+[q] quit   [r] reset stats   [c] compact view
+```
+
+Compact mode (`--compact` or pressing `c`) shows a single status line:
+```
+[43t/s  VRAM18.2GB  req1247  peers8  up3m22s]
+```
+
+## Implementation notes
+
+### Hardware detection
+
+```python
+import torch
+
+def detect_gpus() -> list[dict]:
+    gpus = []
+    if torch.cuda.is_available():
+        for i in range(torch.cuda.device_count()):
+            props = torch.cuda.get_device_properties(i)
+            gpus.append({
+                "index": i,
+                "name": props.name,
+                "vram_gb": props.total_memory / 1e9,
+                "backend": "cuda"
+            })
+    # ROCm / Apple Silicon stubs for later
+    return gpus
+```
+
+### Curated model list
+
+`packages/node/meshnet_node/model_catalog.py` — a hardcoded list of `ModelPreset` dataclasses:
+
+```python
+@dataclass
+class ModelPreset:
+    name: str               # display name
+    hf_repo: str            # HuggingFace repo ID
+    num_layers: int
+    vram_gb: dict           # {"nf4": 18, "int8": 40, "bf16": 80}
+    description: str        # one-line description
+```
+
+Initial list (expand over time):
+- `meta-llama/Meta-Llama-3-70B-Instruct` — 80L, NF4 18GB, INT8 40GB, BF16 80GB
+- `Qwen/Qwen2.5-72B-Instruct` — 80L, NF4 19GB, INT8 41GB, BF16 81GB
+- `mistralai/Mixtral-8x7B-Instruct-v0.1` — 32L, NF4 7GB, INT8 14GB, BF16 27GB
+- `microsoft/Phi-3-medium-128k-instruct` — 40L, NF4 4GB, INT8 8GB, BF16 15GB
+- `google/gemma-2-27b-it` — 46L, NF4 10GB, INT8 20GB, BF16 40GB
+
+### HuggingFace Browse
+
+```python
+from huggingface_hub import list_models
+
+def browse_hf(top_n=20) -> list[dict]:
+    models = list_models(
+        pipeline_tag="text-generation",
+        library="transformers",
+        sort="downloads",
+        direction=-1,
+        limit=top_n,
+        cardData=True,
+    )
+    return [{"repo": m.modelId, "downloads": m.downloads} for m in models]
+```
+
+### Persistent config
+
+`~/.config/meshnet/config.json`:
+```json
+{
+  "model_hf_repo": "meta-llama/Meta-Llama-3-70B-Instruct",
+  "quantization": "nf4",
+  "download_dir": "~/.meshnet/models",
+  "tracker_url": "http://192.168.1.10:8080",
+  "wallet_path": "~/.config/meshnet/wallet.json",
+  "shard_start": null,
+  "shard_end": null,
+  "updatedAt": "2026-06-29T..."
+}
+```
+
+`shard_start`/`shard_end`: null means tracker auto-assigns. User can pin a range for dedicated partial-model nodes.
+
+### CLI flags
+
+All wizard answers are overridable without re-running the wizard:
+
+```
+meshnet-node [start]
+  --model <hf-repo-id>            # e.g. meta-llama/Meta-Llama-3-70B-Instruct
+  --quantization [bf16|int8|nf4]
+  --download-dir <path>
+  --tracker <url>
+  --wallet <path>
+  --shard-start <int>             # pin shard range (optional)
+  --shard-end <int>
+  --reset-config                  # ignore saved config, re-run wizard
+  --no-tui                        # plain-text output (for CI / headless)
+  --compact                       # single-line status instead of full dashboard
+
+meshnet-node models                # list curated models and exit
+meshnet-node models --browse       # list HF Hub top-20 and exit
+meshnet-node config                # print current config and exit
+```
+
+### WSL2 / non-TTY fallback
+
+```python
+import sys, os
+
+def is_interactive_tty() -> bool:
+    return sys.stdout.isatty() and os.environ.get("TERM") not in ("dumb", "")
+
+if not is_interactive_tty():
+    # fall back to plain-text periodic status
+    run_plain_status_loop(node)
+else:
+    run_rich_dashboard(node)
+```
+
+Do NOT use `termios`, `fcntl`, or `/dev/tty` — these break in Windows cmd.exe and some WSL2 terminal emulators.
+
+## Acceptance criteria
+
+- `meshnet-node` with no args and no config → wizard starts
+- Wizard detects GPU and marks `[too large]` for models that exceed available VRAM
+- `meshnet-node models` prints curated list and exits
+- `meshnet-node models --browse` calls HF Hub API, prints top-20, exits
+- Second run (config exists) → skips wizard, starts immediately
+- `--reset-config` re-runs wizard even with config present
+- All wizard inputs override-able via CLI flags
+- Live rich dashboard renders and updates every 2s when running in a TTY
+- Falls back to plain-text when not a TTY (CI / WSL2 without TERM set)
+- Ctrl-C prints a clean summary line and exits 0
+- `python -m pytest` passes from repo root
+- Commit only this story's changes

.scratch/distributed-inference-network/issues/17-p2p-gossip-relay-ssl.md

@@ -0,0 +1,205 @@
+# US-017 — P2P gossip, NAT-traversal relay node, and SSL/TLS
+
+## Goal
+
+Nodes must work behind NAT (home routers, cloud VMs without public IPs) and must communicate securely. Implement:
+
+1. **SSL/TLS everywhere** — all HTTP between nodes/tracker is HTTPS; all WebSocket gossip is WSS
+2. **mDNS peer discovery** — nodes on the same LAN find each other automatically (no config)
+3. **WebSocket gossip PubSub** — nodes propagate join/leave/coverage-update events in near-real-time
+4. **Circuit relay node** — team-run public relay (`packages/relay`) that enables NAT traversal and bootstraps new nodes joining from the internet
+
+Architecture is designed to migrate to libp2p GossipSub + Kademlia DHT without breaking the message schema (topic names and payload formats are stable contracts).
+
+## Gossip protocol
+
+### Transport
+
+WebSocket (`wss://`) using the `websockets` Python library. Each node maintains persistent WSS connections to:
+- The relay node (always, bootstraps peer list)
+- Up to 8 direct peers (Kademlia-style target fanout; peers discovered via mDNS + relay peer list)
+
+### Topics
+
+All messages are JSON with an envelope:
+
+```json
+{
+  "topic": "node-join",
+  "version": 1,
+  "from_peer": "<peer_id>",
+  "timestamp": "<iso8601>",
+  "payload": { ... }
+}
+```
+
+| Topic | Direction | Payload |
+|-------|-----------|---------|
+| `node-join` | broadcast | `{peer_id, addr, models: [{model_preset, shard_start, shard_end}], vram_gb, quant}` |
+| `node-leave` | broadcast | `{peer_id, reason}` |
+| `coverage-update` | broadcast | `{model_preset, coverage: [{start, end, count}]}` |
+| `heartbeat` | peer→relay | `{peer_id, addr, uptime_s, tokens_per_sec}` |
+| `peer-list` | relay→peer | `{peers: [{peer_id, addr}]}` |
+| `relay-announce` | relay→all | `{relay_id, relay_url, capacity}` |
+
+Gossip fanout: each node re-broadcasts received messages to all its peers (simple flooding with `seen_ids` dedup, TTL=3 hops). Migration to GossipSub mesh routing is a later ADR.
+
+### Peer ID
+
+`peer_id = sha256(public_key)[:16].hex()` — generated on first run, stored in `~/.config/meshnet/identity.json`. The same keypair is used for TLS client certificates (mTLS) in future work.
+
+## mDNS LAN discovery
+
+Use Python `zeroconf` library. Service type: `_meshnet._tcp.local.`
+
+```python
+from zeroconf import ServiceInfo, Zeroconf
+
+info = ServiceInfo(
+    "_meshnet._tcp.local.",
+    f"{peer_id}._meshnet._tcp.local.",
+    addresses=[socket.inet_aton(local_ip)],
+    port=node_port,
+    properties={"peer_id": peer_id, "version": "1"},
+)
+zc = Zeroconf()
+zc.register_service(info)
+```
+
+On startup, nodes also browse for `_meshnet._tcp.local.` to discover existing nodes. mDNS is LAN-only (does not traverse routers), which is correct for LAN discovery.
+
+## NAT traversal: circuit relay
+
+### How it works
+
+1. Node A (behind NAT) cannot accept inbound TCP connections
+2. Node A connects outbound to the public relay via WSS
+3. Node A tells the tracker: `"effective_addr": "wss://relay.meshnet.ai/relay/{peer_id_A}"`
+4. Node B (wants to call A) connects to the relay at the above URL
+5. Relay proxies the TCP stream between A and B
+
+Hole-punching (direct connection via STUN) is attempted first (future work). Relay is the fallback.
+
+### meshnet-relay
+
+`packages/relay/meshnet_relay/server.py` — a standalone aiohttp server:
+
+```
+GET  /health                        → {status: ok}
+GET  /v1/peers                      → [{peer_id, addr, last_seen}]
+POST /v1/gossip                     → receive a gossip message, fan out to connected peers
+WSS  /ws                            → persistent gossip connection (subscribe to all topics)
+WSS  /relay/{peer_id}               → circuit relay proxy to that peer_id
+GET  /v1/relay/capacity             → {connected_peers: N, max_peers: 500}
+```
+
+CLI:
+
+```
+meshnet-relay [--port 8443] [--cert path/to/cert.pem] [--key path/to/key.pem]
+              [--tracker-url http://...] [--max-peers 500]
+```
+
+The relay can optionally proxy to the tracker (so `relay.meshnet.ai` is the single internet-visible endpoint).
+
+## SSL/TLS setup
+
+### Node certificate (self-signed, auto-generated)
+
+On first run, `meshnet-node` generates a self-signed RSA-2048 cert valid for 10 years:
+
+```python
+from cryptography import x509
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import rsa
+```
+
+Cert saved to `~/.config/meshnet/node_cert.pem` + `node_key.pem`. Fingerprint stored in config and shared with tracker via heartbeat. Nodes connecting to each other validate the fingerprint (TOFU — trust on first use), not the CA chain.
+
+### Relay certificate
+
+The relay uses a real Let's Encrypt cert (cert-bot or acme.sh). The relay cert is pinned in `packages/p2p/relay_bootstrap.json`:
+
+```json
+{
+  "relays": [
+    {
+      "url": "wss://relay.meshnet.ai:8443",
+      "cert_fingerprint": "sha256:<hex>",
+      "operator": "meshnet-team"
+    }
+  ]
+}
+```
+
+### All HTTP switched to HTTPS
+
+`meshnet-node` starts an HTTPS server using `ssl.SSLContext`. `meshnet-tracker` similarly. All outbound `httpx` / `aiohttp` calls use TLS verification against pinned fingerprints (not the system CA store — too many corporate proxies break this).
+
+## Tracker changes
+
+Heartbeat payload gains new fields:
+
+```json
+{
+  "peer_id": "a1b2c3d4e5f6a1b2",
+  "effective_addr": "https://192.168.1.42:8001",
+  "relay_addr": "wss://relay.meshnet.ai:8443/relay/a1b2c3d4e5f6a1b2",
+  "cert_fingerprint": "sha256:...",
+  "gossip_peers": ["peer_id_1", "peer_id_2"]
+}
+```
+
+Tracker uses `effective_addr` (direct) or `relay_addr` (fallback) when building inference routes.
+
+## Integration test
+
+```
+tests/test_gossip_and_relay.py
+
+scenario:
+  1. Start a local relay (localhost:18443)
+  2. Start node A (no inbound port — simulate NAT by binding to 127.0.0.1 only)
+  3. Start node B (public-reachable on localhost)
+  4. Both register with relay; relay peer-list includes both
+  5. Node B sends a gossip node-join message
+  6. Assert node A receives it within 500ms
+  7. Start tracker; confirm tracker's node registry includes node A via relay_addr
+  8. Send inference request; assert it routes through relay to node A
+```
+
+## Package layout
+
+```
+packages/relay/
+  pyproject.toml
+  meshnet_relay/
+    __init__.py
+    server.py          # aiohttp relay + gossip hub + circuit relay proxy
+    cli.py             # meshnet-relay entrypoint
+    peer_registry.py   # in-memory {peer_id: {addr, last_seen, ...}}
+    circuit_relay.py   # WSS proxy between two peers
+
+packages/p2p/
+  meshnet_p2p/
+    gossip.py          # GossipClient — connect to relay + peers, pub/sub
+    mdns.py            # ZeroconfDiscovery — mDNS announce + browse
+    identity.py        # PeerIdentity — generate/load peer_id + keypair
+    tls.py             # cert generation, fingerprint, SSLContext helpers
+
+packages/node/meshnet_node/
+  gossip_integration.py  # wires GossipClient into node lifecycle
+```
+
+## Acceptance criteria
+
+- All node↔node and node↔tracker HTTP uses HTTPS; self-signed cert auto-generated on first run
+- `cert_fingerprint` included in heartbeat; tracker stores and logs it
+- mDNS: two nodes on the same LAN discover each other without manual tracker URL (test with two localhost processes using different mDNS names)
+- Relay: `meshnet-relay` starts, accepts WSS connections, fans out gossip messages to all connected peers
+- Circuit relay: node A (127.0.0.1-only) can receive a gossip message via the relay from node B
+- Tracker routes inference to node A using `relay_addr` when direct addr not reachable
+- `relay_bootstrap.json` exists in `packages/p2p/` with at least one entry (localhost for tests)
+- ADR-0010 documents the gossip architecture and libp2p migration path
+- `python -m pytest` passes from repo root
+- Commit only this story's changes

.scratch/distributed-inference-network/issues/18-two-machine-lan-test.md

@@ -0,0 +1,157 @@
+# US-018 — End-to-end two-machine LAN inference test
+
+## Goal
+
+Run real distributed inference across two physical machines: the Linux rig and a Windows 11 rig running WSL2. Document every setup step, firewall rule, and gotcha so this is repeatable. The test script exits 0 with token output and timing, proving the network works.
+
+## Network topology for LAN test
+
+```
+[Linux machine]                           [Windows 11 / WSL2]
+  meshnet-tracker  :8080                    meshnet-node (shard B)
+  meshnet-node     :8001  (shard A, tracker-mode)
+  meshnet-gateway  :8000  (optional, for OpenAI-compat)
+
+Client (either machine):
+  scripts/test_lan_inference.py --tracker http://192.168.1.10:8080
+```
+
+The Linux machine runs the tracker + the first-shard node (tracker-mode). The Windows/WSL2 machine runs the second-shard node. A small model (e.g. Phi-3-medium at BF16, fits on one GPU each) is split across both.
+
+## WSL2 setup (Windows side)
+
+`docs/INSTALL_WINDOWS.md` covers:
+
+1. Enable WSL2: `wsl --install -d Ubuntu-24.04`
+2. CUDA in WSL2: install NVIDIA driver on Windows (NOT inside WSL); WSL2 gets CUDA automatically
+   - Verify: `nvidia-smi` inside WSL2 should show GPU
+3. Install Python 3.11+ and pip inside WSL2
+4. `pip install -e packages/node packages/p2p` (clone repo first)
+5. Firewall: Windows Defender must allow inbound WSL2 → LAN on node port
+   - PowerShell: `New-NetFirewallRule -DisplayName "meshnet-node" -Direction Inbound -Protocol TCP -LocalPort 8001 -Action Allow`
+6. WSL2 IP: WSL2 has its own NAT'd IP (172.x.x.x); to expose to LAN, either:
+   - Option A: `netsh interface portproxy add v4tov4 listenport=8001 listenaddress=0.0.0.0 connectport=8001 connectaddress=$(wsl hostname -I)`
+   - Option B: use the relay node (US-017) — no port forwarding needed
+
+## Linux setup
+
+Standard install (already done after US-016). Firewall:
+
+```bash
+# If using ufw
+sudo ufw allow 8080/tcp  # tracker
+sudo ufw allow 8001/tcp  # node
+sudo ufw allow 8000/tcp  # gateway (optional)
+```
+
+## Model split
+
+For the test, use a model that has enough layers to split meaningfully but fits comfortably in memory. Phi-3-medium-128k-instruct (40 layers, BF16 15GB) works on a single 24GB GPU on each machine:
+
+- Linux node: layers 0–19 (tracker-mode, owns tokenizer + embed_tokens)
+- Windows/WSL2 node: layers 20–39
+
+Start sequence:
+```bash
+# Terminal 1 (Linux) — tracker
+meshnet-tracker --port 8080
+
+# Terminal 2 (Linux) — first-shard node (tracker-mode auto-detected because shard_start=0)
+meshnet-node --model microsoft/Phi-3-medium-128k-instruct \
+             --quantization bf16 \
+             --shard-start 0 --shard-end 19 \
+             --tracker http://localhost:8080 \
+             --port 8001
+
+# Terminal 3 (Windows WSL2) — second-shard node
+meshnet-node --model microsoft/Phi-3-medium-128k-instruct \
+             --quantization bf16 \
+             --shard-start 20 --shard-end 39 \
+             --tracker http://192.168.1.10:8080 \
+             --port 8001
+```
+
+## Test script
+
+`scripts/test_lan_inference.py`:
+
+```python
+#!/usr/bin/env python3
+"""
+End-to-end LAN inference test.
+Usage: python scripts/test_lan_inference.py --tracker http://192.168.1.10:8080
+"""
+import argparse, time, httpx, json
+
+MESSAGES = [
+    {"role": "user", "content": "What is 7 × 8? Answer in one word."},
+    {"role": "user", "content": "Name the capital of France in one word."},
+    {"role": "user", "content": "Complete the sequence: 1, 1, 2, 3, 5, ___"},
+]
+
+def run_test(tracker_url: str, gateway_url: str | None):
+    # Discover inference entry point via tracker if gateway not given
+    if not gateway_url:
+        r = httpx.get(f"{tracker_url}/v1/tracker-nodes/phi-3-medium", timeout=5)
+        r.raise_for_status()
+        nodes = r.json()
+        assert nodes, "No tracker-mode nodes registered — is the first-shard node running?"
+        gateway_url = nodes[0]["url"]
+
+    print(f"Inference endpoint: {gateway_url}")
+    print(f"Tracker: {tracker_url}")
+    print()
+
+    for i, msg in enumerate(MESSAGES):
+        t0 = time.monotonic()
+        r = httpx.post(
+            f"{gateway_url}/v1/chat/completions",
+            json={"model": "phi-3-medium", "messages": [msg], "stream": False},
+            timeout=60,
+        )
+        r.raise_for_status()
+        data = r.json()
+        elapsed = time.monotonic() - t0
+
+        content = data["choices"][0]["message"]["content"]
+        tokens = data["usage"]["completion_tokens"]
+        tps = tokens / elapsed if elapsed > 0 else 0
+
+        print(f"[{i+1}] Q: {msg['content']}")
+        print(f"     A: {content}")
+        print(f"     {tokens} tokens  {elapsed:.2f}s  {tps:.1f} t/s")
+        print()
+
+    print("✓ All 3 requests completed successfully")
+
+if __name__ == "__main__":
+    p = argparse.ArgumentParser()
+    p.add_argument("--tracker", required=True)
+    p.add_argument("--gateway", default=None)
+    args = p.parse_args()
+    run_test(args.tracker, args.gateway)
+```
+
+## Docs: TWO_MACHINE_TEST.md
+
+`docs/TWO_MACHINE_TEST.md` must cover:
+
+1. Prerequisites (models downloaded on both machines, same model ID, complementary shard ranges)
+2. Start order: tracker first, then nodes, then test script
+3. How to verify nodes are registered: `GET /v1/nodes` on tracker
+4. How to verify coverage: `GET /v1/coverage/phi-3-medium` — all 40 layers must show node_count ≥ 1
+5. How to run the test script
+6. Expected output
+7. Latency breakdown: how to read per-hop latency from node logs
+8. **Known Issues** section — updated during actual test run with real gotchas
+
+## Acceptance criteria
+
+- `docs/INSTALL_WINDOWS.md` covers WSL2 + CUDA + meshnet-node install end-to-end
+- `docs/TWO_MACHINE_TEST.md` covers the full two-machine setup and test procedure
+- `scripts/test_lan_inference.py` exists and is executable
+- When run against a real two-machine LAN setup: script exits 0, prints 3 valid answers with timing
+- Coverage map shows 100% coverage (no gap) after both nodes register
+- Known Issues section in TWO_MACHINE_TEST.md contains at least the issues encountered during this test run
+- No new pytest failures from repo root (this story adds docs + a script, not new Python packages)
+- Commit only this story's changes

.scratch/distributed-inference-network/issues/19-binary-data-plane-and-peer-weight-transfer.md

@@ -0,0 +1,68 @@
+# US-019 — Binary data plane and optional peer weight transfer
+
+Status: needs-triage
+Priority: Low
+Stage: Design parking lot
+
+## Context
+
+The current project focus is inference democratization: let small GPU owners contribute useful compute and let users run inference on models larger than one host can serve alone. Weight distribution is useful, but it is secondary to low-latency distributed inference.
+
+Recent findings:
+
+- HuggingFace already handles initial model origin distribution well enough for the first working version.
+- The inference-critical path is activation transfer between shard nodes, not torrenting model files.
+- Torrent/content-addressed transfer is a good future fit for model weights, shard cache replication, fine-tuned models, and offline/local swarm behavior.
+- Torrenting is not a good fit for activation traffic because activations are latency-sensitive, ordered, session-specific binary streams.
+
+## Design note
+
+Keep the tracker as the control plane:
+
+- node registration
+- heartbeats
+- route selection
+- model/shard manifests
+- peer/checksum metadata
+
+Keep binary payloads on the data plane:
+
+- direct node-to-node activation transfer where reachable
+- relay/QUIC/WSS fallback where direct transport is unavailable
+- future peer weight transfer as content-addressed blobs or pieces
+
+## Potential future direction
+
+For inference traffic:
+
+- Prefer direct binary transport with backpressure.
+- Use raw binary activation frames rather than JSON/base64.
+- Preserve tensor metadata out-of-band: shape, dtype, session, chunk index, encoding.
+- Consider QUIC or a mature NAT-friendly transport for direct-when-possible, relay-when-needed behavior.
+
+For model weights:
+
+- Keep HuggingFace as default origin and fallback.
+- Add peer cache transfer only as an optimization.
+- Consider a real content-addressed/torrent-like library or sidecar for weight blobs.
+- Store manifests and checksums in tracker state so peers can verify exact shard contents.
+- Prefer piece/chunk transfer with resume and hash verification over one giant tarball.
+
+## Not in scope now
+
+- Replacing HuggingFace as the primary model origin.
+- Building a full BitTorrent/IPFS/libp2p subsystem.
+- Routing activation traffic through a torrent protocol.
+- Making peer weight transfer mandatory for node startup.
+
+## Acceptance criteria for a future implementation issue
+
+- [ ] Activation transfer remains binary end-to-end and avoids JSON/base64 payloads.
+- [ ] Tracker does not proxy large binary payloads except as an explicit fallback path.
+- [ ] Weight transfer, if added, is optional and falls back to HuggingFace.
+- [ ] Weight pieces are content-addressed and checksum-verified.
+- [ ] The design preserves low-latency inference as the primary objective.
+
+## Comments
+
+Created as a low-priority design parking-lot item after discussing inference democratization versus weight distribution. Do not pick up for implementation until the core public tracker, relay, and binary activation path are stable.

.scratch/distributed-inference-network/prd.json

@@ -48,8 +48,8 @@
       "dependsOn": [
         "US-001"
       ],
-      "completionNotes": "Completed by Ralph iteration 126384e5; verified by pytest two-node pipeline.",
+      "completionNotes": "Tests pass; activation tensor flow via stub nodes verified. New HF-model path tested in test_node_startup.py.",
-      "status": "to-revise",
+      "status": "done",
       "status_reason": "Base64 JSON wire format established here was replaced by binary HTTP protocol in US-011. tests/test_two_node_pipeline.py needs verification it exercises the new binary format end-to-end."
     },
     {
@@ -128,8 +128,8 @@
       "dependsOn": [
         "US-003"
       ],
-      "completionNotes": "Completed by agent",
+      "completionNotes": "All 6 gateway tests pass including OpenAI SDK, LangChain, streaming, and 503 for unknown model.",
-      "status": "to-revise",
+      "status": "done",
       "status_reason": "Gateway is currently the pipeline orchestrator. US-014 moves orchestration to tracker-nodes; gateway becomes a thin load-balancer proxy. Implementation will be superseded \u2014 defer rework until US-014 lands."
     },
     {
@@ -382,10 +382,109 @@
       "notes": "Source issue: .scratch/distributed-inference-network/issues/15-ralph-agent-agnostic-status-aware.md",
       "dependsOn": [],
       "completionNotes": "Implemented by agent: status-aware helpers (_is_done, _needs_attention, _is_active, _is_in_design), 6-bucket _story_sets, attention dashboard section, _review_report Attention Required block, auto --include-revise, set-agent subcommand with persistent agent-config.json, _run_openrouter stub, custom agent support, list-parallel subcommand, and auto --parallel N worktree orchestration. All 65 tests pass."
+    },
+    {
+      "id": "US-016",
+      "title": "16 \u2014 Mining-style node startup CLI + live dashboard",
+      "description": "Replace the bare flag-driven node CLI with a wizard-guided first-run experience (like a GPU mining client) followed by a live terminal dashboard once the node is running. On first run, the wizard auto-detects GPU VRAM, presents a curated list of compatible models with VRAM requirements at each quantization level, lets the user pick a download location, and writes a persistent config file so subsequent starts are one command. Once the node is running, the wizard gives way to a rich live status panel showing: GPU temp + VRAM used, tokens/sec, requests served, peers connected, TAI earned (stub until US-006 is live). A Browse HuggingFace option calls the HF Hub API so users can load any HF model beyond the curated list.",
+      "acceptanceCriteria": [
+        " with no args and no config file enters the interactive setup wizard",
+        "Wizard step 1: auto-detect GPU(s) via torch.cuda / torch.version.hip; print GPU name + total VRAM",
+        "Wizard step 2: show curated model list (name, HF repo, layers, VRAM@NF4/INT8/BF16); mark models that do NOT fit available VRAM as [too large]",
+        "Wizard step 3: offer [B] Browse HuggingFace \u2014 calls HF Hub API (huggingface_hub.list_models filtered by pipeline_tag=text-generation, sorted by downloads, top 20) and lets user enter a custom HF repo ID",
+        "Wizard step 4: prompt for download directory (default ~/.meshnet/models/); validate writable; show estimated disk usage for chosen model+quantization",
+        "Wizard step 5: prompt for tracker URL (default http://localhost:8080); validate connection",
+        "Wizard writes ~/.config/meshnet/config.json; second run skips wizard and starts directly",
+        "All wizard values overridable via CLI flags: --model, --download-dir, --quantization [bf16|int8|nf4], --tracker, --wallet, --reset-config",
+        "Once node is running, wizard clears and a live dashboard renders every 2s (rich.live): GPU util%, VRAM used/total, tokens/sec (EMA), requests served, TAI earned (stub 0.0), peers connected, uptime, current model/shard range",
+        "Dashboard exits cleanly on Ctrl-C with a summary line",
+        "Works inside WSL2 (no termios/ioctl calls that fail on Windows terminal; fall back to plain-text status if rich is not available)",
+        " passes from repo root",
+        "Commit only this story changes"
+      ],
+      "priority": 16,
+      "status": "done",
+      "notes": "Source issue: .scratch/distributed-inference-network/issues/16-mining-cli-ux.md",
+      "dependsOn": [
+        "US-004",
+        "US-012"
+      ],
+      "completionNotes": "Implemented: mining-style wizard with GPU detection, curated model list (7 models with NF4/INT8/BF16 VRAM requirements), HF Hub browse, persistent config, rich live dashboard with plain-text WSL2 fallback. 19 tests, 97 passed total."
+    },
+    {
+      "id": "US-017",
+      "title": "17 \u2014 P2P gossip, NAT-traversal relay node, and SSL/TLS",
+      "description": "Add a gossip layer so nodes discover each other and propagate coverage-map changes without polling the tracker continuously. Introduce a publicly-hosted relay node (run by the team) that solves NAT traversal using circuit relay (Petals-style) and serves as the bootstrap peer list for new nodes. Encrypt all node-to-node and node-to-tracker communications with TLS. Gossip protocol: WebSocket-based PubSub over wss://, topics: node-join / node-leave / coverage-update / heartbeat. Peer discovery: mDNS (zeroconf) for LAN, public relay bootstrap list for internet. NAT traversal: relay node acts as TCP-level circuit relay when direct connection fails (hole-punching first, relay second). Architecture is designed to migrate to libp2p GossipSub + Kademlia DHT in a future story without breaking the message schema.",
+      "acceptanceCriteria": [
+        "All HTTP between nodes and tracker uses HTTPS (TLS 1.3); self-signed cert generated on first run and fingerprint pinned in config; relay node uses Let's Encrypt",
+        "Nodes broadcast node-join / node-leave events over wss:// to known peers within 1s of registration",
+        "mDNS peer discovery (Python zeroconf) finds other meshnet nodes on the same LAN segment without manual tracker URL entry",
+        "Public relay bootstrap list (hardcoded relay URL + ) is consulted when no LAN peers found",
+        "Relay node is a standalone meshnet package () with  CLI: starts a WebSocket relay server + circuit relay + optional tracker proxy",
+        "When a node behind NAT cannot accept inbound connections, the relay forwards its traffic; node advertises relay address (relay_url/node_id) to tracker as its effective endpoint",
+        "Tracker accepts both direct node URLs and relay-proxied URLs in heartbeat payloads",
+        "Integration test: two nodes in separate processes on localhost (simulating NAT) communicate via a local relay process; inference request routes correctly",
+        "ADR-0010 documents the gossip protocol, relay architecture, and migration path to libp2p",
+        " passes from repo root",
+        "Commit only this story changes"
+      ],
+      "priority": 17,
+      "status": "done",
+      "notes": "Source issue: .scratch/distributed-inference-network/issues/17-p2p-gossip-relay-ssl.md",
+      "dependsOn": [
+        "US-013",
+        "US-014"
+      ],
+      "completionNotes": "Implemented: packages/p2p (identity, TLS cert+fingerprint, GossipClient WSS PubSub, MdnsDiscovery with zeroconf optional), packages/relay (RelayServer gossip hub + circuit relay proxy, meshnet-relay CLI), tracker extended with relay_addr/cert_fingerprint/peer_id, relay_bootstrap.json, ADR-0010. 18 new tests; 115 total passed."
+    },
+    {
+      "id": "US-018",
+      "title": "18 \u2014 End-to-end two-machine LAN inference test",
+      "description": "Prove the network works across two real machines: the Linux rig (this machine) and a Windows 11 rig running WSL2. One machine runs the tracker + first-shard node (inference entry point). The other machine runs a second-shard node. A client sends a real inference request and receives a streamed response. This story is primarily a test plan + setup guide + test execution script; it produces documented evidence (logs, timing, token output) that real distributed inference works. It also surfaces any real-world issues (port forwarding, CUDA driver version mismatches, WSL2 CUDA passthrough, model download paths) that need fixing.",
+      "acceptanceCriteria": [
+        "docs/INSTALL_WINDOWS.md exists: step-by-step WSL2 + CUDA + meshnet-node install on Windows 11",
+        "docs/TWO_MACHINE_TEST.md exists: how to start tracker on machine A, node on machine B, run inference, interpret output",
+        "A test script scripts/test_lan_inference.py: given --tracker-url, --gateway-url, sends 3 chat completion requests, asserts valid OpenAI format, prints token count + latency + which nodes served each request",
+        "Both machines can reach each other on LAN (documented: firewall rules, port list)",
+        "At least one successful inference recorded: the test script exits 0 with output showing tokens generated and node IDs",
+        "Latency breakdown logged: gateway\u2192node-A, node-A\u2192node-B, node-B\u2192gateway (approximate, from server logs)",
+        "Known issues during test documented in docs/TWO_MACHINE_TEST.md under a Known Issues section",
+        "Commit only this story changes"
+      ],
+      "priority": 18,
+      "status": "done",
+      "notes": "Source issue: .scratch/distributed-inference-network/issues/18-two-machine-lan-test.md",
+      "dependsOn": [
+        "US-016",
+        "US-017"
+      ],
+      "completionNotes": "docs/INSTALL_WINDOWS.md: WSL2+CUDA+meshnet-node install guide. docs/TWO_MACHINE_TEST.md: two-machine LAN test procedure with known issues. scripts/test_lan_inference.py: stdlib-only test script, 3 requests, exit 0 on success, auto-discovers gateway from tracker."
+    },
+    {
+      "id": "US-019",
+      "title": "19 \u2014 Distributed tracker consensus (Raft assignments + CRDT heartbeats)",
+      "description": "Replace the single-point-of-failure tracker with a fault-tolerant cluster. Tracker nodes elect a leader via Raft and commit shard assignments as log entries \u2014 all tracker nodes agree on who owns what. Node liveness (heartbeats) uses CRDT gossip (eventual consistency, high frequency OK). A node registers with any tracker node; the write is forwarded to the leader and replicated to followers. A 3-node tracker cluster survives one tracker failure without losing assignment state. The relay/gossip layer already built in US-017 handles peer heartbeats; this story wires Raft on top for authoritative assignments.",
+      "acceptanceCriteria": [
+        "3 tracker nodes can be started and form a Raft cluster (leader election, log replication)",
+        "A node registers with any follower \u2014 the registration is forwarded to the leader and replicated",
+        "Killing the leader causes a new election within 5 seconds; registrations continue working",
+        "Shard assignments returned by any tracker node are identical (strong consistency)",
+        "Node heartbeats use CRDT gossip (not Raft) \u2014 high-frequency, eventual consistency",
+        "meshnet-tracker CLI gains --cluster-peers flag to specify peer tracker URLs",
+        "Integration test: 3 tracker nodes, kill leader mid-test, verify assignment still works",
+        "QUICKSTART.md updated with multi-tracker setup section"
+      ],
+      "priority": 19,
+      "status": "done",
+      "notes": "Architecture decision: Raft for assignments (strong consistency) + CRDT gossip for liveness (eventual consistency). User approved 2026-06-29.",
+      "dependsOn": [
+        "US-017"
+      ],
+      "completionNotes": "raft.py: minimal Raft consensus (leader election, log replication, AppendEntries, RequestVote). gossip.py: LWW CRDT gossip for node heartbeats. TrackerServer gains cluster_peers + cluster_self_url params. --cluster-peers and --self-url CLI flags added. 6 integration tests: leader election <1s, follower registration propagation, leader kill + re-election <5s, gossip table."
     }
   ],
   "metadata": {
-    "updatedAt": "2026-06-29T13:30:00.000Z",
+    "updatedAt": "2026-06-29T15:35:00.000Z",
     "statusVocabulary": {
       "open": "Not started",
       "in-design": "Decisions pending before implementation can begin",

QUICKSTART.md

@@ -0,0 +1,251 @@
+# Quickstart — Running a node and testing inference
+
+This guide gets you from zero to a live inference request in three terminals.
+Tested on: AMD Ryzen AI Max (Strix Halo APU), 124 GB RAM, Linux, CPU inference.
+
+---
+
+## Prerequisites
+
+```bash
+# Clone and enter repo
+cd /run/media/popov/d/DEV/repos/d-popov.com/AI
+
+# Create the virtualenv if it does not exist yet
+python3 -m venv .venv
+
+# Keep packaging tools current enough for editable installs
+.venv/bin/python -m pip install --upgrade pip setuptools wheel
+
+# Install Python packages (editable — picks up code changes immediately)
+.venv/bin/pip install -e packages/tracker -e packages/node -e packages/p2p -e packages/gateway -e packages/relay
+
+# CPU-only PyTorch (skip if you have CUDA/ROCm already)
+.venv/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu
+
+# HuggingFace model libraries
+.venv/bin/pip install transformers accelerate
+```
+
+> **NVIDIA GPU (CUDA):** replace the torch line with `pip install torch` (default index).  
+> **AMD GPU (ROCm):** `pip install torch --index-url https://download.pytorch.org/whl/rocm6.2`
+
+### Windows / WSL2
+
+Run the Linux commands from WSL, not Git Bash. From the repo opened in Git Bash:
+
+```bash
+wsl
+cd /mnt/d/DEV/workspace/REPOS/git.d-popov.com/neuron-tai
+python3 -m venv .venv
+.venv/bin/python -m pip install --upgrade pip setuptools wheel
+.venv/bin/pip install -e packages/tracker -e packages/node -e packages/p2p -e packages/gateway -e packages/relay
+.venv/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu
+.venv/bin/pip install transformers accelerate
+.venv/bin/meshnet-node --help
+```
+
+If `.venv/bin/meshnet-node` is missing, the editable install step did not finish
+successfully. Re-run the `.venv/bin/pip install -e ...` command above inside WSL.
+
+### Public tracker + WSS relay
+
+For internet nodes, expose one public HTTPS host and proxy these paths:
+
+```text
+/v1/*   -> meshnet-tracker, for registration, heartbeats, routing, and OpenAI requests
+/ws     -> meshnet-relay, for outbound node gossip/bridge connections
+/rpc/*  -> meshnet-relay, for tracker-to-node relay requests
+```
+
+Start the tracker with the public relay URL it should advertise:
+
+```bash
+.venv/bin/meshnet-relay --host 0.0.0.0 --port 8765
+.venv/bin/meshnet-tracker start \
+  --host 0.0.0.0 \
+  --port 8081 \
+  --relay-url wss://ai.neuron.d-popov.com/ws
+```
+
+Then a node only needs the public tracker address:
+
+```bash
+.venv/bin/meshnet-node start \
+  --tracker https://ai.neuron.d-popov.com \
+  --model-id Qwen/Qwen2.5-0.5B-Instruct
+```
+
+---
+
+## Step 1 — Start the tracker (Terminal 1)
+
+```bash
+cd /run/media/popov/d/DEV/repos/d-popov.com/AI
+.venv/bin/meshnet-tracker start --port 8080
+```
+
+Expected output:
+```
+Tracker listening on 0.0.0.0:8080
+```
+
+Keep this terminal open.
+
+---
+
+## Step 2 — Start a node (Terminal 2)
+
+### Recommended model: Qwen2.5-0.5B-Instruct
+
+- 0.5B parameters, ~1 GB in BF16
+- No HuggingFace account or license required
+- Downloads once to `~/.meshnet/models/`, cached for future runs
+- 24 transformer layers (auto-detected — no need to specify)
+
+```bash
+cd /run/media/popov/d/DEV/repos/d-popov.com/AI
+HF_HOME=/run/media/popov/d/DEV/models \
+.venv/bin/meshnet-node start \
+  --model-id Qwen/Qwen2.5-0.5B-Instruct \
+  --quantization bfloat16 \
+  --tracker http://localhost:8080 \
+  --port 8001
+```
+
+Shard range is **auto-detected** from the curated catalog (no network call for known
+models). For unknown repos, the node fetches only `config.json` (~1 KB) to read
+`num_hidden_layers`. You can still pass `--shard-start` / `--shard-end` explicitly
+to run a partial shard on one machine.
+
+Expected output (after model loads):
+```
+  Auto-detected 24 layers → shard 0–23
+================================
+meshnet-node ready
+  Wallet:       <address>
+  Model ID:     Qwen/Qwen2.5-0.5B-Instruct
+  Shard:        layers 0–23
+  Quantization: bfloat16
+  Endpoint:     http://<host>:8001
+  Hardware:     CPU
+================================
+```
+
+### Other model options (all CPU-friendly)
+
+| Model | HF repo | Layers | BF16 size | Notes |
+|-------|---------|--------|-----------|-------|
+| Qwen2.5-0.5B | `Qwen/Qwen2.5-0.5B-Instruct` | 24 | ~1 GB | Fastest, no gating |
+| Qwen2.5-1.5B | `Qwen/Qwen2.5-1.5B-Instruct` | 28 | ~3 GB | Better quality |
+| Phi-3-mini | `microsoft/Phi-3-mini-4k-instruct` | 32 | ~7.5 GB | Best CPU quality |
+| Llama-3.2-1B | `meta-llama/Llama-3.2-1B-Instruct` | 16 | ~2 GB | Requires HF login |
+| Llama-3.2-3B | `meta-llama/Llama-3.2-3B-Instruct` | 28 | ~6 GB | Requires HF login |
+
+For gated models (Llama), run `huggingface-cli login` first.
+
+---
+
+## Step 3 — Send an inference request (Terminal 3)
+
+```bash
+curl -s http://localhost:8001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "qwen2.5-0.5b",
+    "messages": [{"role": "user", "content": "What is 7 times 8? Answer in one word."}],
+    "stream": false
+  }' | python3 -m json.tool
+```
+
+Or use the test script:
+
+```bash
+.venv/bin/python scripts/test_lan_inference.py \
+  --tracker http://localhost:8080 \
+  --gateway http://localhost:8001
+```
+
+---
+
+## Two-node split (same machine, two terminals)
+
+Split Qwen2.5-0.5B's 24 layers across two node processes to test the sharded pipeline:
+
+**Node A — layers 0–11 (tracker mode, serves chat completions):**
+```bash
+HF_HOME=/run/media/popov/d/DEV/models \
+.venv/bin/meshnet-node start \
+  --model-id Qwen/Qwen2.5-0.5B-Instruct \
+  --shard-start 0 --shard-end 11 \
+  --quantization bfloat16 \
+  --tracker http://localhost:8080 \
+  --port 8001
+```
+
+**Node B — layers 12–23:**
+```bash
+HF_HOME=/run/media/popov/d/DEV/models \
+.venv/bin/meshnet-node start \
+  --model-id Qwen/Qwen2.5-0.5B-Instruct \
+  --shard-start 12 --shard-end 23 \
+  --quantization bfloat16 \
+  --tracker http://localhost:8080 \
+  --port 8002
+```
+
+Send the request to Node A — it tokenizes, runs layers 0–13, passes binary
+activations to Node B, and streams the final response back.
+
+---
+
+## Two-machine LAN test (Linux + Windows/WSL2)
+
+See `docs/TWO_MACHINE_TEST.md` (created by US-018).
+
+---
+
+## Browse available models
+
+```bash
+# Show curated list with VRAM requirements
+.venv/bin/meshnet-node models
+
+# Browse HuggingFace Hub top-20 text-generation models
+.venv/bin/meshnet-node models --browse
+```
+
+---
+
+## Start with the interactive wizard
+
+```bash
+# First run: wizard detects GPU, shows model list, saves config
+.venv/bin/meshnet-node
+
+# Subsequent runs: starts directly from saved config
+.venv/bin/meshnet-node
+
+# Re-run wizard even with saved config
+.venv/bin/meshnet-node --reset-config
+```
+
+---
+
+## Start the relay node (for NAT traversal)
+
+```bash
+.venv/bin/pip install -e packages/relay
+.venv/bin/meshnet-relay --port 8765
+```
+
+Nodes behind NAT connect to the relay and advertise their relay address to the
+tracker. See `docs/adr/0010-p2p-gossip-and-nat-relay.md`.
+
+---
+
+## Run all tests
+
+```bash
+.venv/bin/python -m pytest -q
+```

_DEV_NOTES.md

@@ -0,0 +1,4 @@
+ win
+ .venv/bin/meshnet-node start --tracker http://192.168.0.179:8081  --model Qwen/Qwen2.5-0.5B-Instruct --shard-start 20
+
+

conftest.py

@@ -11,6 +11,7 @@ _packages = [
     "packages/sdk",
     "packages/contracts",
     "packages/p2p",
+    "packages/relay",
     "packages/validator",
 ]
 

docs/INSTALL_WINDOWS.md

@@ -0,0 +1,212 @@
+# Installing meshnet-node on Windows 11 with WSL2
+
+This guide covers setting up a meshnet-node on a Windows 11 machine using WSL2 with CUDA passthrough so it can join an existing inference network over LAN.
+
+## Prerequisites
+
+- Windows 11 with WSL2 support (most systems with Windows 10 version 2004+ qualify)
+- NVIDIA GPU with CUDA support (driver ≥ 525.x recommended for WSL2 CUDA)
+- At least 8 GB RAM + enough VRAM for the model shard you intend to serve
+- The Linux machine (other node) is reachable on your LAN
+
+---
+
+## Step 1 — Enable WSL2 and install Ubuntu
+
+Open **PowerShell as Administrator** and run:
+
+```powershell
+wsl --install -d Ubuntu-24.04
+```
+
+This installs WSL2 with Ubuntu 24.04. Reboot when prompted.
+
+After reboot, Ubuntu starts and asks you to create a UNIX username/password. Choose anything convenient.
+
+Verify WSL version:
+
+```powershell
+wsl -l -v
+```
+
+Output should show `VERSION 2`.
+
+---
+
+## Step 2 — Install NVIDIA GPU driver on Windows (NOT inside WSL)
+
+WSL2 CUDA passthrough works through the Windows host driver. **Do not install CUDA inside WSL2.**
+
+1. Download the latest Game Ready or Studio driver for your GPU from https://www.nvidia.com/drivers
+2. Install on Windows normally (standard installer).
+3. Inside WSL2 (Ubuntu terminal), verify:
+
+```bash
+nvidia-smi
+```
+
+Expected output: your GPU name, driver version, CUDA version. If this command fails, the Windows driver is too old — update it.
+
+> **Note:** The `cuda-toolkit` package inside WSL2 is optional and only needed if you compile CUDA kernels. For inference with `torch`, the Windows host driver is sufficient.
+
+---
+
+## Step 3 — Install Python 3.11+ inside WSL2
+
+Ubuntu 24.04 ships Python 3.12. Confirm:
+
+```bash
+python3 --version
+```
+
+If it shows 3.10 or older:
+
+```bash
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt update
+sudo apt install python3.12 python3.12-venv python3.12-dev
+```
+
+Install pip:
+
+```bash
+curl -sS https://bootstrap.pypa.io/get-pip.py | python3
+```
+
+---
+
+## Step 4 — Clone the repository
+
+Inside WSL2:
+
+```bash
+# Store the repo in the Linux filesystem (faster I/O than /mnt/c)
+cd ~
+git clone https://github.com/YOUR_ORG/d-popov.com.git
+cd d-popov.com/AI
+```
+
+---
+
+## Step 5 — Create a virtualenv and install meshnet-node
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+
+# Install node + PyTorch (CUDA build)
+pip install torch --index-url https://download.pytorch.org/whl/cu124
+pip install -e "packages/node[torch]"
+```
+
+Verify the install:
+
+```bash
+meshnet-node --help
+```
+
+---
+
+## Step 6 — Pre-download the model shard
+
+Download the model before starting the node so the startup process doesn't time out on the tracker side:
+
+```bash
+python3 - <<'EOF'
+from transformers import AutoConfig
+AutoConfig.from_pretrained("microsoft/Phi-3-medium-128k-instruct")
+EOF
+```
+
+For the full model weights (needed at runtime), `transformers` downloads them automatically on first `meshnet-node` start. If you want to pre-fetch:
+
+```bash
+python3 -c "
+from transformers import AutoModelForCausalLM
+AutoModelForCausalLM.from_pretrained('microsoft/Phi-3-medium-128k-instruct', device_map='cpu')
+"
+```
+
+This can take 10–30 minutes on first run.
+
+---
+
+## Step 7 — Expose the node port to your LAN
+
+WSL2 runs behind a NAT with a virtual IP (typically `172.x.x.x`). Your LAN sees the Windows host IP. You need to forward the node port.
+
+**Option A — Windows port proxy (recommended for simple setups):**
+
+In **PowerShell as Administrator**:
+
+```powershell
+# Get the current WSL2 IP (changes on each WSL restart)
+$wslIp = (wsl hostname -I).Trim()
+
+# Forward Windows host port 8001 → WSL2 port 8001
+netsh interface portproxy add v4tov4 `
+  listenport=8001 listenaddress=0.0.0.0 `
+  connectport=8001 connectaddress=$wslIp
+
+# Allow inbound on Windows Firewall
+New-NetFirewallRule -DisplayName "meshnet-node" `
+  -Direction Inbound -Protocol TCP -LocalPort 8001 -Action Allow
+```
+
+Verify: from the Linux machine, `curl http://WINDOWS_LAN_IP:8001/v1/health` should return a response once the node is running.
+
+**Redo this after every WSL2 restart** — the WSL2 IP changes.
+
+**Option B — P2P relay (US-017, no port forwarding needed):**
+
+Start a relay node on the Linux machine. The WSL2 node connects outbound through the relay. No firewall rules needed. See `docs/TWO_MACHINE_TEST.md` for details.
+
+---
+
+## Step 8 — Start the node
+
+Replace `192.168.1.10` with the actual LAN IP of the Linux machine running the tracker.
+Replace shard range with the complementary range to what the Linux node is serving.
+
+```bash
+source .venv/bin/activate
+
+meshnet-node \
+  --model microsoft/Phi-3-medium-128k-instruct \
+  --quantization bf16 \
+  --shard-start 20 --shard-end 39 \
+  --tracker http://192.168.1.10:8080 \
+  --port 8001 \
+  --host 0.0.0.0 \
+  --advertise-host WINDOWS_LAN_IP
+```
+
+The `--advertise-host` flag tells the tracker what IP the Linux machine should use to reach this node. Use your Windows machine's LAN IP (e.g. `192.168.1.20`), **not** the WSL2 internal IP.
+
+Expected startup output:
+
+```
+Detecting hardware...
+  GPU: NVIDIA GeForce RTX 3080 (10240 MB VRAM)
+Loading wallet...
+  Wallet: 5K7r...
+Loading real PyTorch model shard...
+  Auto-detected 40 layers → shard 20–39
+================================
+meshnet-node ready
+  Model ID:     microsoft/Phi-3-medium-128k-instruct
+  Shard:        layers 20–39; 20 of 40
+  Endpoint:     http://192.168.1.20:8001
+  Hardware:     CUDA
+================================
+```
+
+---
+
+## Known issues
+
+- **WSL2 IP changes on restart.** Always re-run the `netsh` port-proxy command after restarting WSL2 or Windows.
+- **CUDA not visible in WSL2.** If `nvidia-smi` fails inside WSL2, update the Windows host GPU driver to ≥ 525.x. Installing CUDA inside WSL2 will not fix it.
+- **Model download is slow.** HuggingFace downloads happen over HTTPS. Pre-fetch the model before a timed test (see Step 6).
+- **Port 8001 already in use.** Change `--port` to another value and update the firewall/portproxy rules accordingly.
+- **`bf16` not supported on older GPUs.** Use `--quantization int8` on Turing (RTX 20xx) cards or earlier if bfloat16 ops fail.

docs/TWO_MACHINE_TEST.md

@@ -0,0 +1,200 @@
+# Two-machine LAN inference test
+
+This guide proves that distributed inference works across two physical machines: a Linux rig (tracker + first shard) and a Windows 11 / WSL2 rig (second shard). A test script sends real inference requests and validates the output.
+
+## Network topology
+
+```
+[Linux machine — 192.168.1.10]
+  meshnet-tracker  :8080
+  meshnet-node A   :8001   shard 0–19  (tracker-mode, entry point)
+
+[Windows 11 / WSL2 — 192.168.1.20]
+  meshnet-node B   :8001   shard 20–39
+
+[Client — either machine]
+  scripts/test_lan_inference.py --tracker http://192.168.1.10:8080
+```
+
+Adjust the IPs and shard ranges to match your hardware. Use a model that fits (sharded) in both GPUs combined. The example uses `microsoft/Phi-3-medium-128k-instruct` (40 layers, BF16 ~15 GB each shard ~7.5 GB).
+
+---
+
+## Prerequisites
+
+**Both machines:**
+- Python 3.11+ with `meshnet-node` installed (see `docs/INSTALL_WINDOWS.md` for Windows)
+- Model weights already downloaded (pre-fetch prevents timeout on first startup)
+- LAN connectivity verified: `ping 192.168.1.10` from Windows, `ping 192.168.1.20` from Linux
+
+**Linux machine ports open:**
+
+```bash
+# ufw (skip if firewall is off)
+sudo ufw allow 8080/tcp   # tracker
+sudo ufw allow 8001/tcp   # node A
+```
+
+**Windows machine port forwarded (WSL2 only):**
+
+```powershell
+# Run in PowerShell as Administrator — redo after every WSL restart
+$wsl = (wsl hostname -I).Trim()
+netsh interface portproxy add v4tov4 listenport=8001 listenaddress=0.0.0.0 connectport=8001 connectaddress=$wsl
+New-NetFirewallRule -DisplayName "meshnet-node" -Direction Inbound -Protocol TCP -LocalPort 8001 -Action Allow
+```
+
+---
+
+## Start sequence
+
+**Always start in this order: tracker → node A → node B → test.**
+
+### Terminal 1 — Linux: tracker
+
+```bash
+meshnet-tracker --port 8080
+```
+
+Expected:
+
+```
+[tracker] listening on 0.0.0.0:8080
+```
+
+### Terminal 2 — Linux: node A (shard 0–19, tracker-mode)
+
+```bash
+meshnet-node \
+  --model microsoft/Phi-3-medium-128k-instruct \
+  --quantization bf16 \
+  --shard-start 0 --shard-end 19 \
+  --tracker http://localhost:8080 \
+  --port 8001 \
+  --host 0.0.0.0
+```
+
+`shard_start=0` auto-sets `tracker_mode=True` — this node accepts inference requests.
+
+Wait until you see `meshnet-node ready` before continuing.
+
+### Terminal 3 — Windows WSL2: node B (shard 20–39)
+
+```bash
+meshnet-node \
+  --model microsoft/Phi-3-medium-128k-instruct \
+  --quantization bf16 \
+  --shard-start 20 --shard-end 39 \
+  --tracker http://192.168.1.10:8080 \
+  --port 8001 \
+  --host 0.0.0.0 \
+  --advertise-host 192.168.1.20
+```
+
+`--advertise-host` must be the Windows **LAN IP** (not the WSL2 internal 172.x.x.x IP) so the Linux node can reach it.
+
+---
+
+## Verify nodes are registered
+
+From any machine with `curl`:
+
+```bash
+# List all registered nodes
+curl http://192.168.1.10:8080/v1/nodes
+
+# Check route for the model — should list both node endpoints in order
+curl "http://192.168.1.10:8080/v1/route?model=microsoft/Phi-3-medium-128k-instruct"
+```
+
+Expected route response:
+
+```json
+{
+  "route": [
+    "http://192.168.1.10:8001",
+    "http://192.168.1.20:8001"
+  ]
+}
+```
+
+If only one endpoint appears, node B hasn't registered yet — wait a few seconds and retry.
+
+---
+
+## Run the test script
+
+```bash
+# From any machine that can reach the tracker
+python3 scripts/test_lan_inference.py \
+  --tracker http://192.168.1.10:8080 \
+  --gateway http://192.168.1.10:8001
+```
+
+Expected output:
+
+```
+Inference endpoint: http://192.168.1.10:8001
+Tracker: http://192.168.1.10:8080
+
+Route: ['http://192.168.1.10:8001', 'http://192.168.1.20:8001']
+
+[1] Q: What is 7 × 8? Answer in one word.
+     A: 56
+     3 tokens  2.41s  1.2 t/s
+
+[2] Q: Name the capital of France in one word.
+     A: Paris
+     2 tokens  1.87s  1.1 t/s
+
+[3] Q: Complete the sequence: 1, 1, 2, 3, 5, ___
+     A: 8
+     2 tokens  1.93s  1.0 t/s
+
+All 3 requests completed successfully.
+Exit code: 0
+```
+
+The script exits 0 if all 3 requests complete with valid OpenAI-format responses.
+
+---
+
+## Reading latency from node logs
+
+The node logs show per-hop timing. On node A terminal look for:
+
+```
+[node] forwarding to downstream: http://192.168.1.20:8001  (took 1.23s)
+```
+
+Approximate breakdown:
+- **client → node A (encode + first shard):** full request latency minus the downstream time
+- **node A → node B (pipeline):** the `forwarding to downstream` duration
+- **node B → node A (tail decode + token):** included in downstream duration
+
+Full end-to-end latency = prompt encode + shard A forward + network transfer + shard B forward + decode.
+
+With LAN latency < 1 ms, the network transfer is negligible. Bottleneck is GPU compute.
+
+---
+
+## Known Issues
+
+**WSL2 IP changes after restart.**
+The `netsh portproxy` forwarding rule uses a fixed WSL2 IP. If Windows or WSL2 restarts, the IP changes and the rule breaks. Redo the `netsh` and `New-NetFirewallRule` commands. To automate this, add a Task Scheduler job on WSL start.
+
+**Node B registers with internal WSL2 IP (172.x.x.x) instead of LAN IP.**
+Symptom: route response lists `172.x.x.x` and node A cannot reach it.
+Fix: always pass `--advertise-host 192.168.1.20` (your Windows LAN IP) when starting node B.
+
+**Model download times out node registration.**
+If the model hasn't been pre-fetched, `transformers` downloads it during node startup, which can take 20+ minutes. The tracker heartbeat timeout (90s) will expire, and node A will deregister node B. Pre-download the model weights before starting the node (see `docs/INSTALL_WINDOWS.md` Step 6). Node B re-registers automatically via the heartbeat re-registration loop once it's up.
+
+**`bf16` unsupported on older NVIDIA GPUs.**
+GPUs before Ampere (RTX 30xx) have limited bfloat16 support. Use `--quantization int8` on RTX 20xx and earlier.
+
+**Windows Defender blocks inbound connection on WSL2.**
+Even with the firewall rule added, Windows Defender SmartScreen or a corporate security policy can block the connection. Verify by checking Windows Event Viewer → Security → Filtering Platform Connection for blocked connections on port 8001.
+
+**Route returns only one node.**
+If node B registers but the route only returns one endpoint, check that both nodes use the same `--model` string (full HuggingFace repo path). Route lookup matches on `hf_repo` — a short name vs. full path mismatch causes the node to be excluded.

docs/adr/0010-p2p-gossip-and-nat-relay.md

@@ -0,0 +1,67 @@
+# ADR-0010: P2P gossip, NAT-traversal relay, and TLS
+
+## Status: Accepted
+
+## Context
+
+All node-to-node and node-to-tracker communication in the prototype is plain HTTP over a LAN or direct-IP internet connection. This has three problems:
+
+1. **NAT blocking**: Most home and cloud nodes cannot accept inbound TCP connections.
+2. **No encryption**: Activations and heartbeats are in plaintext.
+3. **Polling overhead**: Nodes poll the tracker for coverage changes every 30s. This is slow to react to node churn and does not scale past a few hundred nodes.
+
+The reference implementation (Petals) solves this with libp2p — GossipSub for pub/sub and Kademlia DHT for peer discovery. We adopt the same goals but start with simpler, more stable building blocks that can be swapped for libp2p later without changing the message schema.
+
+## Decisions
+
+### 1. TLS everywhere
+
+All HTTP between nodes, tracker, and gateway uses HTTPS (TLS 1.3). Self-signed certificates are auto-generated on first node start and stored in `~/.config/meshnet/`. The certificate fingerprint is included in every heartbeat and gossip envelope. Nodes use TOFU (trust on first use) — they accept a peer's cert on first contact and pin the fingerprint; connections from the same peer with a different fingerprint are rejected.
+
+The relay node uses a real CA-signed certificate (Let's Encrypt) because it is the internet-facing bootstrap point.
+
+### 2. mDNS for LAN peer discovery
+
+Python `zeroconf` library. Service type: `_meshnet._tcp.local.`. A node announces itself on startup and browses for existing peers. This is zero-config discovery for home and lab networks. mDNS does not traverse routers, which is correct — LAN discovery should not bleed into the internet.
+
+### 3. WebSocket PubSub for gossip
+
+Each node maintains persistent WSS connections to the relay and up to 8 direct peers. Messages use a stable JSON envelope with a `topic`, `version`, `from_peer`, and `payload`. Topics: `node-join`, `node-leave`, `coverage-update`, `heartbeat`, `peer-list`, `relay-announce`.
+
+Simple flooding with `seen_ids` dedup and TTL=3 is good enough for the prototype. The message schema is stable; the fanout mechanism can be replaced with GossipSub mesh routing without changing the schema.
+
+### 4. Circuit relay node for NAT traversal
+
+A team-operated public relay (`packages/relay`, CLI: `meshnet-relay`) is the internet bootstrap point. A node behind NAT:
+
+1. Connects outbound to the relay via WSS
+2. Advertises `relay_addr = wss://relay.meshnet.ai:8443/relay/{peer_id}` to the tracker
+3. Other nodes proxy connections through the relay when the direct addr is not reachable
+
+Hole-punching (STUN + simultaneous TCP open) is deferred to a future story. Circuit relay is the reliable fallback.
+
+The relay is stateless in terms of inference — it only proxies bytes. It does not decrypt activations.
+
+### 5. Bootstrap peer list
+
+`packages/p2p/relay_bootstrap.json` contains the team-operated relay endpoints with their TLS fingerprints. New nodes load this file on startup to find their first peer. The list is bundled with the package and updated via pip upgrades.
+
+### Migration path to libp2p
+
+When the network has enough volume to justify the complexity:
+
+1. Replace the WebSocket gossip layer with libp2p GossipSub (same topics and payload schemas, different transport)
+2. Replace mDNS + relay peer list with Kademlia DHT
+3. Replace circuit relay with libp2p circuit relay v2
+
+The gossip envelope schema (`topic`, `version`, `from_peer`, `payload`) is the stable contract. As long as messages on the wire are identical, the transport layer can be swapped without touching node business logic.
+
+## Alternatives rejected
+
+**libp2p from the start**: `py-libp2p` is experimental and not production-ready. A Go libp2p sidecar is operationally complex. The benefits of real libp2p (mesh routing, Kademlia DHT, hole-punching) are not needed until we have hundreds of nodes.
+
+**NATS**: Stable and fast but requires a central NATS server. Adds operational dependency and contradicts the P2P goal.
+
+**ZeroMQ**: No NAT traversal built in. Requires manual topology management.
+
+**No gossip (keep polling)**: Does not scale; slow to react to node churn; misses the relay/NAT requirement.

packages/node/meshnet_node/cli.py

@@ -1,83 +1,279 @@
-"""meshnet-node CLI entry point."""
+"""meshnet-node CLI entry point — mining-style UX."""
+
+from __future__ import annotations
 
 import argparse
+import socket
 import sys
 import time
+from pathlib import Path
 
 
-def main() -> None:
+def _run_node(cfg: dict) -> None:
-    parser = argparse.ArgumentParser(
+    """Start the node and hand off to the live dashboard. Blocks until Ctrl-C."""
-        prog="meshnet-node",
+    from .startup import run_startup
-        description="Distributed Inference Network node client",
+    from .dashboard import run_dashboard
-    )
-    subparsers = parser.add_subparsers(dest="command")
 
-    start_cmd = subparsers.add_parser("start", help="Start the node server")
+    start_time = time.monotonic()
-    start_cmd.add_argument(
+    try:
-        "--tracker", default="http://localhost:8080", help="Tracker URL"
+        node = run_startup(
+            tracker_url=cfg["tracker_url"],
+            port=cfg.get("port", 7000),
+            model=cfg.get("model_name") or "stub-model",
+            model_id=cfg.get("model_hf_repo") or None,
+            shard_start=cfg.get("shard_start"),
+            shard_end=cfg.get("shard_end"),
+            quantization=cfg.get("quantization", "bfloat16").replace("bf16", "bfloat16"),
+            wallet_path=Path(cfg["wallet_path"]) if cfg.get("wallet_path") else None,
+            cache_dir=Path(cfg["download_dir"]) if cfg.get("download_dir") else None,
+            host=cfg.get("host", "0.0.0.0"),
+            advertise_host=cfg.get("advertise_host"),
         )
-    start_cmd.add_argument("--port", type=int, default=7000, help="Port to listen on")
+    except Exception as exc:
-    start_cmd.add_argument(
+        print(f"\nERROR: {exc}", file=sys.stderr, flush=True)
-        "--model", default="stub-model", help="Model preset to request from tracker"
+        sys.exit(1)
-    )
+
-    start_cmd.add_argument(
+    try:
-        "--model-id",
+        run_dashboard(node, cfg, start_time)
-        help="HuggingFace model id for the real PyTorch backend",
+    except KeyboardInterrupt:
-    )
+        pass
-    start_cmd.add_argument("--shard-start", type=int, help="First layer index for an explicit shard")
+    finally:
-    start_cmd.add_argument("--shard-end", type=int, help="Exclusive layer end index for an explicit shard")
+        node.stop()
-    start_cmd.add_argument(
+        req = getattr(node, "chat_completion_count", 0)
-        "--quantization",
+        elapsed = time.monotonic() - start_time
-        choices=["bfloat16", "int8", "nf4"],
+        h, rem = divmod(int(elapsed), 3600)
-        default="bfloat16",
+        m, s = divmod(rem, 60)
-        help="Weight quantization for the real PyTorch backend",
+        print(
-    )
+            f"\nmeshnet-node stopped. "
-    start_cmd.add_argument(
+            f"Served {req} requests in {h:02d}:{m:02d}:{s:02d}.",
-        "--host", default="0.0.0.0", help="Interface to bind to"
+            flush=True,
-    )
-    start_cmd.add_argument(
-        "--advertise-host",
-        help="Reachable host/IP to advertise to the tracker (defaults to FQDN when binding 0.0.0.0)",
-    )
-    start_cmd.add_argument(
-        "--tracker-mode",
-        action="store_true",
-        help="Enable client-facing /v1/chat/completions (auto-enabled when shard-start=0)",
-    )
-    start_cmd.add_argument(
-        "--tracker-url",
-        default=None,
-        help="Tracker URL for route selection (used in tracker mode)",
         )
 
-    args = parser.parse_args()
 
-    if args.command == "start":
+def _first_available_port(host: str, start: int = 7000, attempts: int = 100) -> int:
-        from meshnet_node.startup import run_startup
+    """Return the first TCP port bindable on host, starting at start."""
+    bind_host = "" if host == "0.0.0.0" else host
+    for port in range(start, start + attempts):
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
+            try:
+                sock.bind((bind_host, port))
+            except OSError:
+                continue
+            return port
+    raise OSError(f"no available TCP port in range {start}-{start + attempts - 1}")
+
+
+def _cmd_default(args) -> int:
+    """No subcommand: wizard if no config, else start with saved config."""
+    from .config import load_config, save_config, merge_cli_overrides
+    from .wizard import run_wizard
+
+    cfg = load_config()
+    if cfg is None or args.reset_config:
+        if args.reset_config and cfg is not None:
+            print("Resetting config — re-running setup wizard.\n")
+        try:
+            cfg = run_wizard()
+        except KeyboardInterrupt:
+            print("\nSetup cancelled.")
+            return 1
+        save_config(cfg)
+        print(f"\nConfig saved to ~/.config/meshnet/config.json\n")
+
+    # Apply CLI overrides on top of saved config
+    overrides: dict = {}
+    if args.model:
+        overrides["model_hf_repo"] = args.model
+        overrides["model_name"] = args.model.split("/")[-1]
+    if args.quantization:
+        overrides["quantization"] = args.quantization
+    if args.download_dir:
+        overrides["download_dir"] = args.download_dir
+    if args.tracker:
+        overrides["tracker_url"] = args.tracker
+    if args.wallet:
+        overrides["wallet_path"] = args.wallet
+    if args.shard_start is not None:
+        overrides["shard_start"] = args.shard_start
+    if args.shard_end is not None:
+        overrides["shard_end"] = args.shard_end
+    if args.port is not None:
+        overrides["port"] = args.port
+    if args.host:
+        overrides["host"] = args.host
+    if args.advertise_host:
+        overrides["advertise_host"] = args.advertise_host
+
+    if overrides:
+        cfg = merge_cli_overrides(cfg, **overrides)
+
+    _run_node(cfg)
+    return 0
+
+
+def _cmd_models(args) -> int:
+    """List curated models (with optional HF Hub browse)."""
+    from .wizard import print_models_table, _browse_hf_interactive
+
+    if args.browse:
+        from .model_catalog import browse_hf_hub
+        print("Fetching HuggingFace Hub top models...\n")
+        try:
+            models = browse_hf_hub(top_n=20)
+            print(f"{'#':<4} {'Repo':<60} {'Downloads':>12}")
+            print(f"{'─'*4} {'─'*60} {'─'*12}")
+            for i, m in enumerate(models, 1):
+                dl = m["downloads"]
+                dl_str = (
+                    f"{dl/1e6:.1f}M" if dl >= 1_000_000
+                    else f"{dl/1e3:.0f}k" if dl >= 1000
+                    else str(dl)
+                )
+                print(f"{i:<4} {m['repo']:<60} {dl_str:>12}")
+        except RuntimeError as exc:
+            print(f"Error: {exc}", file=sys.stderr)
+            return 1
+    else:
+        print_models_table()
+    return 0
+
+
+def _cmd_config(args) -> int:
+    """Print current config."""
+    import json
+    from .config import load_config, config_path
+
+    cfg = load_config()
+    if cfg is None:
+        print("No config file found. Run `meshnet-node` to start setup.")
+        return 1
+    print(f"Config: {config_path()}")
+    print(json.dumps(cfg, indent=2))
+    return 0
+
+
+def _cmd_start(args) -> int:
+    """Legacy `start` subcommand — preserves backward compatibility with existing tests."""
+    from .config import load_config, DEFAULTS
+
+    # Build a transient config from flags (don't write to disk)
+    cfg = dict(DEFAULTS)
+    cfg["tracker_url"] = args.tracker
+    cfg["port"] = args.port if args.port is not None else _first_available_port(args.host)
+    if args.model_id is None and "/" in args.model:
+        cfg["model_hf_repo"] = args.model
+        cfg["model_name"] = args.model.split("/")[-1]
+    else:
+        cfg["model_name"] = args.model
+    cfg["quantization"] = args.quantization
+    cfg["host"] = args.host
+    if args.model_id:
+        cfg["model_hf_repo"] = args.model_id
+    if args.shard_start is not None:
+        cfg["shard_start"] = args.shard_start
+    if args.shard_end is not None:
+        cfg["shard_end"] = args.shard_end
+    if args.wallet:
+        cfg["wallet_path"] = args.wallet
+    if args.download_dir:
+        cfg["download_dir"] = args.download_dir
+
+    # Legacy start: just run without the dashboard (keep original blocking loop)
+    from .startup import run_startup
 
     try:
         node = run_startup(
-                tracker_url=args.tracker,
+            tracker_url=cfg["tracker_url"],
-                port=args.port,
+            port=cfg["port"],
-                model=args.model,
+            model=cfg["model_name"],
-                model_id=args.model_id,
+            model_id=cfg.get("model_hf_repo"),
-                shard_start=args.shard_start,
+            shard_start=cfg.get("shard_start"),
-                shard_end=args.shard_end,
+            shard_end=cfg.get("shard_end"),
-                quantization=args.quantization,
+            quantization=cfg["quantization"].replace("bf16", "bfloat16"),
-                host=args.host,
+            wallet_path=Path(cfg["wallet_path"]) if cfg.get("wallet_path") else None,
-                advertise_host=args.advertise_host,
+            cache_dir=Path(cfg["download_dir"]) if cfg.get("download_dir") else None,
+            host=cfg["host"],
+            advertise_host=getattr(args, "advertise_host", None),
         )
     except Exception as exc:
         print(f"ERROR: {exc}", file=sys.stderr, flush=True)
         sys.exit(1)
+
     try:
         while True:
             time.sleep(1)
     except KeyboardInterrupt:
         node.stop()
-            sys.exit(0)
+    return 0
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        prog="meshnet-node",
+        description="Distributed AI Inference — Node Client",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=(
+            "Run with no arguments to start the setup wizard.\n"
+            "After first setup, `meshnet-node` starts using your saved config.\n\n"
+            "Subcommands:\n"
+            "  models          List supported models\n"
+            "  models --browse Browse HuggingFace Hub\n"
+            "  config          Show current config\n"
+        ),
+    )
+
+    # Flags that apply to the no-subcommand (default) path
+    parser.add_argument("--model", metavar="HF_REPO", help="HuggingFace repo ID to serve")
+    parser.add_argument("--quantization", "-q", choices=["bf16", "int8", "nf4", "bfloat16"],
+                        help="Quantization level")
+    parser.add_argument("--download-dir", metavar="PATH", help="Model download directory")
+    parser.add_argument("--tracker", metavar="URL", help="Tracker URL")
+    parser.add_argument("--wallet", metavar="PATH", help="Wallet file path")
+    parser.add_argument("--shard-start", type=int, metavar="N", help="Pin shard start layer")
+    parser.add_argument("--shard-end", type=int, metavar="N", help="Pin shard end layer")
+    parser.add_argument("--port", type=int, metavar="N", help="Port to listen on")
+    parser.add_argument("--host", metavar="ADDR", help="Interface to bind (default 0.0.0.0)")
+    parser.add_argument("--advertise-host", metavar="ADDR", help="Host/IP advertised to the tracker")
+    parser.add_argument("--no-tui", action="store_true", help="Plain-text output (no rich dashboard)")
+    parser.add_argument("--compact", action="store_true", help="Single-line status output")
+    parser.add_argument("--reset-config", action="store_true", help="Re-run wizard even if config exists")
+
+    subparsers = parser.add_subparsers(dest="command")
+
+    # models subcommand
+    models_cmd = subparsers.add_parser("models", help="List supported models")
+    models_cmd.add_argument("--browse", action="store_true", help="Browse HuggingFace Hub top-20")
+
+    # config subcommand
+    subparsers.add_parser("config", help="Show current saved config")
+
+    # start subcommand (legacy / backward-compat)
+    start_cmd = subparsers.add_parser("start", help="Start node (legacy flags)")
+    start_cmd.add_argument("--tracker", default="http://localhost:8080")
+    start_cmd.add_argument("--port", type=int)
+    start_cmd.add_argument("--model", default="stub-model")
+    start_cmd.add_argument("--model-id", help="HuggingFace repo ID")
+    start_cmd.add_argument("--shard-start", type=int)
+    start_cmd.add_argument("--shard-end", type=int)
+    start_cmd.add_argument("--quantization", choices=["bfloat16", "int8", "nf4", "bf16"], default="bfloat16")
+    start_cmd.add_argument("--host", default="0.0.0.0")
+    start_cmd.add_argument("--advertise-host")
+    start_cmd.add_argument("--tracker-mode", action="store_true")
+    start_cmd.add_argument("--tracker-url", default=None)
+    start_cmd.add_argument("--wallet")
+    start_cmd.add_argument("--download-dir")
+
+    args = parser.parse_args()
+
+    if args.command == "models":
+        sys.exit(_cmd_models(args))
+    elif args.command == "config":
+        sys.exit(_cmd_config(args))
+    elif args.command == "start":
+        sys.exit(_cmd_start(args))
     else:
-        parser.print_help()
+        # Default: wizard or start with saved config
+        sys.exit(_cmd_default(args))
 
 
 if __name__ == "__main__":

packages/node/meshnet_node/config.py

@@ -0,0 +1,72 @@
+"""Persistent node configuration — stored in ~/.config/meshnet/config.json."""
+
+from __future__ import annotations
+
+import json
+import os
+import stat
+from pathlib import Path
+
+_DEFAULT_CONFIG_DIR = Path.home() / ".config" / "meshnet"
+_DEFAULT_CONFIG_FILE = _DEFAULT_CONFIG_DIR / "config.json"
+_DEFAULT_DOWNLOAD_DIR = Path.home() / ".meshnet" / "models"
+_DEFAULT_TRACKER_URL = "http://localhost:8080"
+_DEFAULT_WALLET_PATH = str(Path.home() / ".config" / "meshnet" / "wallet.json")
+_DEFAULT_QUANTIZATION = "nf4"
+
+DEFAULTS = {
+    "model_hf_repo": "",
+    "model_name": "",
+    "quantization": _DEFAULT_QUANTIZATION,
+    "download_dir": str(_DEFAULT_DOWNLOAD_DIR),
+    "tracker_url": _DEFAULT_TRACKER_URL,
+    "wallet_path": _DEFAULT_WALLET_PATH,
+    "shard_start": None,
+    "shard_end": None,
+    "port": 7000,
+    "host": "0.0.0.0",
+}
+
+
+def config_path(override: Path | None = None) -> Path:
+    return override or _DEFAULT_CONFIG_FILE
+
+
+def load_config(path: Path | None = None) -> dict | None:
+    """Return parsed config dict, or None if no config file exists."""
+    p = config_path(path)
+    if not p.exists():
+        return None
+    try:
+        cfg = json.loads(p.read_text())
+        if not isinstance(cfg, dict):
+            return None
+        return cfg
+    except (json.JSONDecodeError, OSError):
+        return None
+
+
+def save_config(cfg: dict, path: Path | None = None) -> None:
+    """Write config to disk with restricted permissions (0o600)."""
+    p = config_path(path)
+    p.parent.mkdir(parents=True, exist_ok=True)
+    p.write_text(json.dumps(cfg, indent=2))
+    try:
+        os.chmod(p, stat.S_IRUSR | stat.S_IWUSR)
+    except OSError:
+        pass  # Windows / some filesystems don't support chmod
+
+
+def delete_config(path: Path | None = None) -> None:
+    p = config_path(path)
+    if p.exists():
+        p.unlink()
+
+
+def merge_cli_overrides(cfg: dict, **cli_kwargs) -> dict:
+    """Return a copy of cfg with any non-None CLI values applied on top."""
+    result = dict(cfg)
+    for key, val in cli_kwargs.items():
+        if val is not None:
+            result[key] = val
+    return result

packages/node/meshnet_node/dashboard.py

@@ -0,0 +1,220 @@
+"""Live node status dashboard — rich TUI with plain-text fallback."""
+
+from __future__ import annotations
+
+import os
+import sys
+import time
+from collections import deque
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    pass
+
+
+def is_interactive_tty() -> bool:
+    """Return True when stdout is a real terminal (not CI / redirected / WSL2 dumb)."""
+    if not sys.stdout.isatty():
+        return False
+    term = os.environ.get("TERM", "")
+    if term in ("dumb", ""):
+        return False
+    return True
+
+
+def _format_uptime(seconds: float) -> str:
+    s = int(seconds)
+    h, rem = divmod(s, 3600)
+    m, sec = divmod(rem, 60)
+    return f"{h:02d}:{m:02d}:{sec:02d}"
+
+
+def _gpu_stats() -> list[dict]:
+    """Return per-GPU utilization and VRAM stats, or empty list on CPU."""
+    try:
+        import torch  # type: ignore[import]
+
+        if not torch.cuda.is_available():
+            return []
+        stats = []
+        for i in range(torch.cuda.device_count()):
+            props = torch.cuda.get_device_properties(i)
+            used = torch.cuda.memory_allocated(i)
+            total = props.total_memory
+            # Utilization requires pynvml; skip gracefully if not available
+            util = _nvml_gpu_util(i)
+            stats.append(
+                {
+                    "index": i,
+                    "name": props.name,
+                    "used_gb": used / 1e9,
+                    "total_gb": total / 1e9,
+                    "util_pct": util,
+                }
+            )
+        return stats
+    except ImportError:
+        return []
+
+
+def _nvml_gpu_util(index: int) -> int | None:
+    """Return GPU utilization % via pynvml, or None if unavailable."""
+    try:
+        import pynvml  # type: ignore[import]
+
+        pynvml.nvmlInit()
+        handle = pynvml.nvmlDeviceGetHandleByIndex(index)
+        rates = pynvml.nvmlDeviceGetUtilizationRates(handle)
+        return rates.gpu
+    except Exception:
+        return None
+
+
+class _EMA:
+    """Exponential moving average for tokens/sec."""
+
+    def __init__(self, alpha: float = 0.1):
+        self._alpha = alpha
+        self._value: float | None = None
+
+    def update(self, sample: float) -> float:
+        if self._value is None:
+            self._value = sample
+        else:
+            self._value = self._alpha * sample + (1 - self._alpha) * self._value
+        return self._value
+
+    @property
+    def value(self) -> float:
+        return self._value or 0.0
+
+
+def _make_bar(pct: float, width: int = 10) -> str:
+    filled = round(pct / 100 * width)
+    return "█" * filled + "░" * (width - filled)
+
+
+def run_dashboard(node, config: dict, start_time: float) -> None:
+    """Start the live dashboard. Blocks until Ctrl-C. Returns cleanly."""
+    if not is_interactive_tty():
+        _run_plain_loop(node, config, start_time)
+        return
+
+    try:
+        from rich.live import Live  # type: ignore[import]
+
+        _run_rich_dashboard(node, config, start_time)
+    except ImportError:
+        _run_plain_loop(node, config, start_time)
+
+
+def _build_rich_renderable(
+    node, config: dict, start_time: float, tps_ema: _EMA, prev_req: list[int]
+):
+    from rich.table import Table  # type: ignore[import]
+    from rich.panel import Panel  # type: ignore[import]
+    from rich.columns import Columns  # type: ignore[import]
+    from rich.text import Text  # type: ignore[import]
+
+    uptime = time.monotonic() - start_time
+    req_count = getattr(node, "chat_completion_count", 0)
+
+    # Tokens/sec EMA (approximate: 20 tokens per request heuristic when no real counter)
+    delta_req = req_count - prev_req[0]
+    prev_req[0] = req_count
+    if delta_req > 0:
+        approx_tokens = delta_req * 20
+        tps_ema.update(approx_tokens / 2.0)  # 2s interval
+
+    gpu_stats = _gpu_stats()
+
+    model_name = config.get("model_name") or config.get("model_hf_repo", "unknown").split("/")[-1]
+    shard = ""
+    if config.get("shard_start") is not None:
+        shard = f"  shard {config['shard_start']}–{config['shard_end']}"
+
+    # Header line
+    header = Text(
+        f"meshnet-node  {model_name} [{config.get('quantization', 'bf16')}]{shard}"
+        f"  up {_format_uptime(uptime)}",
+        style="bold white",
+    )
+
+    # GPU table
+    gpu_table = Table(show_header=False, box=None, padding=(0, 1))
+    gpu_table.add_column("label", style="dim", no_wrap=True)
+    gpu_table.add_column("bar", no_wrap=True)
+    gpu_table.add_column("vram", no_wrap=True, style="cyan")
+
+    if gpu_stats:
+        for g in gpu_stats:
+            util = g["util_pct"]
+            util_str = f"{_make_bar(util)}  {util:3d}%" if util is not None else "            n/a"
+            vram_str = f"VRAM {g['used_gb']:.1f}/{g['total_gb']:.1f} GB"
+            gpu_table.add_row(f"GPU {g['index']}  {g['name'][:20]}", util_str, vram_str)
+    else:
+        gpu_table.add_row("CPU mode", "", "no GPU detected")
+
+    # Stats panel
+    tps = tps_ema.value
+    bar_len = min(8, max(0, int(tps / 10)))
+    tps_bar = "▁▂▃▄▅▆▇█"[:bar_len].ljust(8)
+
+    stats_lines = [
+        f"Tokens/sec   {tps_bar}  {tps:.1f} t/s (EMA)",
+        f"Requests     {req_count:,} served",
+        f"Peers        0 connected  (gossip: US-017)",
+        f"TAI earned   0.00 TAI  (payments: US-006)",
+        f"Uptime       {_format_uptime(uptime)}",
+        "",
+        "[q] quit   [c] compact view",
+    ]
+
+    from rich.console import Group  # type: ignore[import]
+
+    return Panel(
+        Group(header, gpu_table, Text("\n".join(stats_lines))),
+        title="[bold green]meshnet-node[/bold green]",
+        border_style="green",
+    )
+
+
+def _run_rich_dashboard(node, config: dict, start_time: float) -> None:
+    from rich.live import Live  # type: ignore[import]
+
+    tps_ema = _EMA()
+    prev_req = [0]
+
+    try:
+        with Live(
+            _build_rich_renderable(node, config, start_time, tps_ema, prev_req),
+            refresh_per_second=0.5,
+            screen=False,
+        ) as live:
+            while True:
+                time.sleep(2)
+                live.update(
+                    _build_rich_renderable(node, config, start_time, tps_ema, prev_req)
+                )
+    except KeyboardInterrupt:
+        pass
+
+
+def _run_plain_loop(node, config: dict, start_time: float) -> None:
+    model_name = config.get("model_name") or config.get("model_hf_repo", "unknown").split("/")[-1]
+    try:
+        while True:
+            uptime = time.monotonic() - start_time
+            req = getattr(node, "chat_completion_count", 0)
+            gpu_stats = _gpu_stats()
+            vram_str = ""
+            if gpu_stats:
+                g = gpu_stats[0]
+                vram_str = f"  VRAM{g['used_gb']:.1f}GB"
+            print(
+                f"[{model_name}  req{req}{vram_str}  up{_format_uptime(uptime)}]",
+                flush=True,
+            )
+            time.sleep(2)
+    except KeyboardInterrupt:
+        pass

packages/node/meshnet_node/model_backend.py

@@ -107,12 +107,13 @@ class TorchModelShard:
         self.tokenizer = AutoTokenizer.from_pretrained(model_id)
         self.layers = _model_layers(self.model)
         self.total_layers = len(self.layers)
-        if shard_end > self.total_layers:
+        # shard_end is INCLUSIVE (last layer index, 0-based), matching the CLI convention.
+        if shard_end >= self.total_layers:
             raise ValueError(
-                f"shard_end {shard_end} exceeds total layer count {self.total_layers}"
+                f"shard_end {shard_end} exceeds last layer index {self.total_layers - 1}"
             )
         self.is_head = shard_start == 0
-        self.is_tail = shard_end == self.total_layers
+        self.is_tail = shard_end >= self.total_layers - 1
         self.hidden_size = int(
             getattr(self.model.config, "hidden_size", 0)
             or getattr(self.model.config, "n_embd", 0)
@@ -168,10 +169,135 @@ class TorchModelShard:
         token_id = int(self.torch.argmax(logits[:, -1, :], dim=-1)[0].item())
         return self.tokenizer.decode([token_id], skip_special_tokens=True)
 
-    def _run_layers(self, hidden_states: Any, attention_mask: Any, position_ids: Any) -> Any:
+    def generate_text(
+        self,
+        messages: list[dict],
+        max_new_tokens: int = 256,
+        temperature: float = 1.0,
+        top_p: float = 1.0,
+    ) -> str:
+        """Autoregressive generation using HF generate() — single-node (head+tail) mode."""
+        if not self.is_head or not self.is_tail:
+            raise ModelBackendError("local generation requires a full head+tail shard")
+        encoded = self._encode_messages(messages)
+        input_ids = encoded["input_ids"].to(self.device)
+        attention_mask = encoded.get("attention_mask")
+        if attention_mask is not None:
+            attention_mask = attention_mask.to(self.device)
+        pad_token_id = getattr(self.tokenizer, "pad_token_id", None) or getattr(self.tokenizer, "eos_token_id", None)
+        do_sample = temperature != 1.0 or top_p != 1.0
         with self.torch.inference_mode():
-            for layer in self.layers[self.shard_start:self.shard_end]:
+            generated = self.model.generate(
-                hidden_states = _call_layer(layer, hidden_states, attention_mask, position_ids)
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                max_new_tokens=max(1, int(max_new_tokens)),
+                do_sample=do_sample,
+                temperature=temperature if do_sample else None,
+                top_p=top_p if do_sample else None,
+                pad_token_id=pad_token_id,
+            )
+        new_tokens = generated[0, input_ids.shape[-1]:]
+        return self.tokenizer.decode(new_tokens, skip_special_tokens=True).strip()
+
+    def generate_text_streaming(
+        self,
+        messages: list[dict],
+        max_new_tokens: int = 256,
+        temperature: float = 1.0,
+        top_p: float = 1.0,
+    ):
+        """Yield decoded token strings one at a time using HF TextIteratorStreamer."""
+        if not self.is_head or not self.is_tail:
+            raise ModelBackendError("streaming generation requires a full head+tail shard")
+        import threading
+        try:
+            from transformers import TextIteratorStreamer  # type: ignore[import]
+        except ImportError:
+            yield self.generate_text(messages, max_new_tokens, temperature, top_p)
+            return
+
+        encoded = self._encode_messages(messages)
+        input_ids = encoded["input_ids"].to(self.device)
+        attention_mask = encoded.get("attention_mask")
+        if attention_mask is not None:
+            attention_mask = attention_mask.to(self.device)
+        pad_token_id = getattr(self.tokenizer, "pad_token_id", None) or getattr(self.tokenizer, "eos_token_id", None)
+        do_sample = temperature != 1.0 or top_p != 1.0
+
+        streamer = TextIteratorStreamer(self.tokenizer, skip_prompt=True, skip_special_tokens=True)
+        gen_kwargs = dict(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            max_new_tokens=max(1, int(max_new_tokens)),
+            do_sample=do_sample,
+            temperature=temperature if do_sample else None,
+            top_p=top_p if do_sample else None,
+            pad_token_id=pad_token_id,
+            streamer=streamer,
+        )
+        t = threading.Thread(target=self.model.generate, kwargs=gen_kwargs, daemon=True)
+        t.start()
+        for token_text in streamer:
+            yield token_text
+        t.join()
+
+    def count_prompt_tokens(self, messages: list[dict]) -> int:
+        """Return tokenizer-backed prompt token count for OpenAI usage metadata."""
+        encoded = self._encode_messages(messages)
+        input_ids = encoded["input_ids"]
+        return int(input_ids.shape[-1])
+
+    def count_text_tokens(self, text: str) -> int:
+        """Return tokenizer-backed completion token count for OpenAI usage metadata."""
+        try:
+            encoded = self.tokenizer(
+                text,
+                return_tensors="pt",
+                add_special_tokens=False,
+            )
+        except TypeError:
+            encoded = self.tokenizer(text, return_tensors="pt")
+        return int(encoded["input_ids"].shape[-1])
+
+    def _encode_messages(self, messages: list[dict]) -> dict:
+        """Format messages with chat template (if available) and tokenize."""
+        if hasattr(self.tokenizer, "apply_chat_template"):
+            try:
+                prompt_str = self.tokenizer.apply_chat_template(
+                    messages,
+                    add_generation_prompt=True,
+                    tokenize=False,
+                )
+                return dict(self.tokenizer(prompt_str, return_tensors="pt"))
+            except Exception:
+                pass
+        prompt = " ".join(
+            str(m.get("content", ""))
+            for m in messages
+            if isinstance(m, dict) and m.get("role") == "user"
+        )
+        return dict(self.tokenizer(prompt, return_tensors="pt"))
+
+    def _run_layers(self, hidden_states: Any, attention_mask: Any, position_ids: Any) -> Any:
+        position_embeddings = _rotary_position_embeddings(
+            self.model,
+            hidden_states,
+            position_ids,
+        )
+        layer_attention_mask = _decoder_attention_mask(
+            attention_mask,
+            hidden_states,
+            self.torch,
+        )
+        with self.torch.inference_mode():
+            for layer in self.layers[self.shard_start:self.shard_end + 1]:
+                hidden_states = _call_layer(
+                    layer,
+                    hidden_states,
+                    layer_attention_mask,
+                    position_ids,
+                    position_embeddings,
+                )
             return hidden_states.to(self.torch.bfloat16)
 
     def _payload(self, hidden_states: Any, attention_mask: Any, position_ids: Any) -> TensorPayload:
@@ -236,8 +362,60 @@ def _position_ids(attention_mask: Any, torch: Any) -> Any:
     return position_ids.masked_fill(attention_mask == 0, 0).to(torch.long)
 
 
-def _call_layer(layer: Any, hidden_states: Any, attention_mask: Any, position_ids: Any) -> Any:
+def _decoder_attention_mask(attention_mask: Any, hidden_states: Any, torch: Any) -> Any:
+    """Build a causal additive mask for decoder layers called outside model.forward."""
+    if attention_mask is None:
+        return None
+    if len(getattr(attention_mask, "shape", ())) != 2:
+        return attention_mask
+    batch_size, seq_len = attention_mask.shape
+    if seq_len <= 1:
+        return None if bool(attention_mask.all()) else attention_mask.to(hidden_states.dtype)
+
+    min_value = torch.finfo(hidden_states.dtype).min
+    causal = torch.full(
+        (seq_len, seq_len),
+        min_value,
+        dtype=hidden_states.dtype,
+        device=hidden_states.device,
+    )
+    causal = torch.triu(causal, diagonal=1)
+    causal = causal[None, None, :, :].expand(batch_size, 1, seq_len, seq_len).clone()
+
+    padding = attention_mask.to(device=hidden_states.device)
+    if not bool(padding.all()):
+        causal = causal.masked_fill(padding[:, None, None, :] == 0, min_value)
+    return causal
+
+
+def _rotary_position_embeddings(model: Any, hidden_states: Any, position_ids: Any) -> Any | None:
+    """Return model-level rotary embeddings required by newer HF decoder layers."""
+    if position_ids is None:
+        return None
+    rotary = None
+    if hasattr(model, "model") and hasattr(model.model, "rotary_emb"):
+        rotary = model.model.rotary_emb
+    elif hasattr(model, "transformer") and hasattr(model.transformer, "rotary_emb"):
+        rotary = model.transformer.rotary_emb
+    if rotary is None:
+        return None
+    return rotary(hidden_states, position_ids)
+
+
+def _call_layer(
+    layer: Any,
+    hidden_states: Any,
+    attention_mask: Any,
+    position_ids: Any,
+    position_embeddings: Any | None = None,
+) -> Any:
     attempts = (
+        {
+            "attention_mask": attention_mask,
+            "position_ids": position_ids,
+            "position_embeddings": position_embeddings,
+            "use_cache": False,
+        },
         {
             "attention_mask": attention_mask,
             "position_ids": position_ids,
@@ -272,7 +450,7 @@ def _tensor_from_bfloat16_bytes(body: bytes, shape: list[int], torch: Any) -> An
 
 
 def _int_tensor_header(tensor: Any) -> str:
-    data = tensor.detach().cpu().to(tensor.int64).contiguous()
+    data = tensor.detach().cpu().long().contiguous()
     raw = data.numpy().tobytes()
     shape = ",".join(str(dim) for dim in data.shape)
     encoded = base64.b64encode(raw).decode("ascii")

packages/node/meshnet_node/model_catalog.py

@@ -0,0 +1,165 @@
+"""Curated list of models supported by the network with VRAM requirements."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+
+@dataclass
+class ModelPreset:
+    name: str
+    hf_repo: str
+    num_layers: int
+    # VRAM in GB at each quantization level (None = too large to quantize this way)
+    vram_nf4: float
+    vram_int8: float
+    vram_bf16: float
+    description: str
+
+    def vram_for_quant(self, quant: str) -> float:
+        """Return VRAM requirement in GB for the given quantization."""
+        q = quant.lower().replace("bfloat16", "bf16")
+        if q == "nf4":
+            return self.vram_nf4
+        if q in ("int8", "int8"):
+            return self.vram_int8
+        if q in ("bf16", "bfloat16"):
+            return self.vram_bf16
+        raise ValueError(f"unknown quantization: {quant!r}")
+
+    def fits_vram(self, available_gb: float, quant: str) -> bool:
+        return self.vram_for_quant(quant) <= available_gb
+
+    def recommended_quant(self, available_gb: float) -> str | None:
+        """Return the highest-quality quantization that fits available VRAM, or None."""
+        if self.vram_bf16 <= available_gb:
+            return "bf16"
+        if self.vram_int8 <= available_gb:
+            return "int8"
+        if self.vram_nf4 <= available_gb:
+            return "nf4"
+        return None
+
+
+CURATED_MODELS: list[ModelPreset] = [
+    ModelPreset(
+        name="Qwen2.5-0.5B-Instruct",
+        hf_repo="Qwen/Qwen2.5-0.5B-Instruct",
+        num_layers=24,
+        vram_nf4=0.4,
+        vram_int8=0.6,
+        vram_bf16=1.0,
+        description="Smallest no-gating model — great for testing, ~1 GB",
+    ),
+    ModelPreset(
+        name="Qwen2.5-1.5B-Instruct",
+        hf_repo="Qwen/Qwen2.5-1.5B-Instruct",
+        num_layers=28,
+        vram_nf4=1.0,
+        vram_int8=1.8,
+        vram_bf16=3.2,
+        description="Fast no-gating model — good quality, ~3 GB",
+    ),
+    ModelPreset(
+        name="Llama-3-70B-Instruct",
+        hf_repo="meta-llama/Meta-Llama-3-70B-Instruct",
+        num_layers=80,
+        vram_nf4=18.0,
+        vram_int8=40.0,
+        vram_bf16=140.0,
+        description="Meta's flagship 70B instruction model",
+    ),
+    ModelPreset(
+        name="Qwen2.5-72B-Instruct",
+        hf_repo="Qwen/Qwen2.5-72B-Instruct",
+        num_layers=80,
+        vram_nf4=19.0,
+        vram_int8=41.0,
+        vram_bf16=145.0,
+        description="Alibaba's 72B multilingual instruction model",
+    ),
+    ModelPreset(
+        name="Mixtral-8x7B-Instruct",
+        hf_repo="mistralai/Mixtral-8x7B-Instruct-v0.1",
+        num_layers=32,
+        vram_nf4=7.0,
+        vram_int8=14.0,
+        vram_bf16=27.0,
+        description="Mistral's sparse MoE — fast and efficient",
+    ),
+    ModelPreset(
+        name="Llama-3-8B-Instruct",
+        hf_repo="meta-llama/Meta-Llama-3-8B-Instruct",
+        num_layers=32,  # gated repo — requires HF login
+        vram_nf4=4.5,
+        vram_int8=8.5,
+        vram_bf16=16.0,
+        description="Meta's compact 8B model — good for low-VRAM nodes",
+    ),
+    ModelPreset(
+        name="Phi-3-medium-128k",
+        hf_repo="microsoft/Phi-3-medium-128k-instruct",
+        num_layers=40,
+        vram_nf4=4.0,
+        vram_int8=8.0,
+        vram_bf16=15.0,
+        description="Microsoft's efficient 14B model with 128k context",
+    ),
+    ModelPreset(
+        name="Gemma-2-27B-IT",
+        hf_repo="google/gemma-2-27b-it",
+        num_layers=46,
+        vram_nf4=10.0,
+        vram_int8=20.0,
+        vram_bf16=54.0,
+        description="Google's 27B instruction-tuned model",
+    ),
+    ModelPreset(
+        name="DeepSeek-V2-Lite-Chat",
+        hf_repo="deepseek-ai/DeepSeek-V2-Lite-Chat",
+        num_layers=27,
+        vram_nf4=5.0,
+        vram_int8=9.0,
+        vram_bf16=16.0,
+        description="DeepSeek's efficient MoE — strong coding + reasoning",
+    ),
+]
+
+
+def detect_num_layers(hf_repo: str) -> int | None:
+    """Return num_hidden_layers from HuggingFace config.json (downloads ~1 KB only)."""
+    # Check curated list first (no network call)
+    for m in CURATED_MODELS:
+        if m.hf_repo == hf_repo:
+            return m.num_layers
+    try:
+        from transformers import AutoConfig  # type: ignore[import]
+        cfg = AutoConfig.from_pretrained(hf_repo)
+        return int(cfg.num_hidden_layers)
+    except Exception:
+        return None
+
+
+def browse_hf_hub(top_n: int = 20) -> list[dict]:
+    """Fetch top downloaded text-generation models from HuggingFace Hub."""
+    try:
+        from huggingface_hub import list_models  # type: ignore[import]
+
+        models = list(
+            list_models(
+                pipeline_tag="text-generation",
+                library="transformers",
+                sort="downloads",
+                direction=-1,
+                limit=top_n,
+            )
+        )
+        return [
+            {
+                "repo": m.id,
+                "downloads": getattr(m, "downloads", 0) or 0,
+            }
+            for m in models
+        ]
+    except Exception as exc:
+        raise RuntimeError(f"HuggingFace Hub lookup failed: {exc}") from exc

packages/node/meshnet_node/relay_bridge.py

@@ -0,0 +1,148 @@
+"""Outbound relay bridge for NAT-safe node HTTP requests."""
+
+from __future__ import annotations
+
+import json
+import logging
+import threading
+import time
+import urllib.error
+import urllib.request
+from dataclasses import dataclass
+
+log = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class RelayBridgeInfo:
+    peer_id: str
+    relay_addr: str
+
+
+def _make_envelope(topic: str, payload: dict, peer_id: str) -> dict:
+    return {
+        "topic": topic,
+        "version": 1,
+        "from_peer": peer_id,
+        "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+        "msg_id": f"{peer_id}-{time.time_ns():x}",
+        "ttl": 1,
+        "payload": payload,
+    }
+
+
+class RelayHttpBridge:
+    """Connect outbound to a relay and proxy relay HTTP requests to localhost."""
+
+    def __init__(
+        self,
+        relay_url: str,
+        peer_id: str,
+        local_base_url: str,
+        advertised_addr: str,
+        reconnect_interval: float = 3.0,
+    ) -> None:
+        self.relay_url = relay_url.rstrip("/")
+        self.peer_id = peer_id
+        self.local_base_url = local_base_url.rstrip("/")
+        self.advertised_addr = advertised_addr
+        self.reconnect_interval = reconnect_interval
+        self._running = False
+        self._thread: threading.Thread | None = None
+        self._connected = threading.Event()
+
+    @property
+    def relay_addr(self) -> str:
+        base = self.relay_url
+        if base.endswith("/ws"):
+            base = base[:-3]
+        return f"{base}/rpc/{self.peer_id}"
+
+    def start(self) -> RelayBridgeInfo:
+        self._running = True
+        self._thread = threading.Thread(target=self._run, daemon=True, name="relay-http-bridge")
+        self._thread.start()
+        return RelayBridgeInfo(peer_id=self.peer_id, relay_addr=self.relay_addr)
+
+    def wait_connected(self, timeout: float = 5.0) -> bool:
+        return self._connected.wait(timeout)
+
+    def stop(self) -> None:
+        self._running = False
+        if self._thread:
+            self._thread.join(timeout=3.0)
+
+    def _run(self) -> None:
+        import websockets.sync.client as wsc  # type: ignore[import]
+
+        while self._running:
+            try:
+                with wsc.connect(self.relay_url, open_timeout=5) as ws:
+                    self._connected.set()
+                    ws.send(json.dumps(_make_envelope(
+                        "peer-register",
+                        {"peer_id": self.peer_id, "addr": self.advertised_addr},
+                        self.peer_id,
+                    )))
+                    while self._running:
+                        try:
+                            raw = ws.recv(timeout=1)
+                        except TimeoutError:
+                            continue
+                        try:
+                            envelope = json.loads(raw)
+                        except (TypeError, json.JSONDecodeError):
+                            continue
+                        if envelope.get("topic") != "relay-http-request":
+                            continue
+                        payload = envelope.get("payload", {})
+                        if payload.get("target_peer") not in {None, self.peer_id}:
+                            continue
+                        response = self._handle_request(payload)
+                        ws.send(json.dumps(_make_envelope(
+                            "relay-http-response",
+                            response,
+                            self.peer_id,
+                        )))
+            except Exception as exc:
+                self._connected.clear()
+                if self._running:
+                    log.debug("relay bridge disconnected: %s", exc)
+                    time.sleep(self.reconnect_interval)
+
+    def _handle_request(self, payload: dict) -> dict:
+        request_id = str(payload.get("request_id") or "")
+        method = str(payload.get("method") or "POST").upper()
+        path = str(payload.get("path") or "/")
+        body_text = payload.get("body") or ""
+        headers = payload.get("headers") if isinstance(payload.get("headers"), dict) else {}
+
+        url = f"{self.local_base_url}{path}"
+        data = body_text.encode() if isinstance(body_text, str) else bytes(body_text)
+        req = urllib.request.Request(url, data=data, headers=headers, method=method)
+        try:
+            with urllib.request.urlopen(req, timeout=300.0) as resp:
+                return {
+                    "request_id": request_id,
+                    "status": resp.status,
+                    "headers": {"Content-Type": resp.headers.get("Content-Type", "application/json")},
+                    "body": resp.read().decode(errors="replace"),
+                }
+        except urllib.error.HTTPError as exc:
+            return {
+                "request_id": request_id,
+                "status": exc.code,
+                "headers": {"Content-Type": exc.headers.get("Content-Type", "application/json")},
+                "body": exc.read().decode(errors="replace"),
+            }
+        except Exception as exc:
+            return {
+                "request_id": request_id,
+                "status": 503,
+                "headers": {"Content-Type": "application/json"},
+                "body": json.dumps({"error": f"relay bridge local request failed: {exc}"}),
+            }
+
+
+def peer_id_from_wallet(wallet_address: str) -> str:
+    return wallet_address[:16] if len(wallet_address) >= 16 else wallet_address

packages/node/meshnet_node/startup.py

@@ -5,6 +5,8 @@ from __future__ import annotations
 import json
 import socket
 import sys
+import threading
+import time
 import urllib.error
 import urllib.parse
 import urllib.request
@@ -13,6 +15,7 @@ from typing import Any
 
 from .downloader import compute_shard_checksum, download_shard
 from .hardware import detect_hardware
+from .relay_bridge import RelayHttpBridge, peer_id_from_wallet
 from .server import StubNodeServer
 from .torch_server import TorchNodeServer
 from .wallet import load_or_create_wallet
@@ -32,6 +35,121 @@ def _get_json(url: str, timeout: float = 10.0) -> dict:
         return json.loads(r.read())
 
 
+def _discover_relay_url(tracker_url: str) -> str | None:
+    try:
+        network_map = _get_json(f"{tracker_url}/v1/network/map", timeout=5.0)
+    except Exception:
+        return None
+    relay_url = network_map.get("relay_url")
+    return relay_url if isinstance(relay_url, str) and relay_url else None
+
+
+def _start_relay_bridge_if_available(
+    tracker_url: str,
+    wallet_address: str,
+    local_base_url: str,
+    advertised_endpoint: str,
+) -> tuple[RelayHttpBridge | None, dict]:
+    relay_url = _discover_relay_url(tracker_url)
+    if not relay_url:
+        return None, {}
+    peer_id = peer_id_from_wallet(wallet_address)
+    bridge = RelayHttpBridge(
+        relay_url=relay_url,
+        peer_id=peer_id,
+        local_base_url=local_base_url,
+        advertised_addr=advertised_endpoint,
+    )
+    info = bridge.start()
+    if bridge.wait_connected(timeout=5.0):
+        print(f"  Relay connected — {info.relay_addr}", flush=True)
+    else:
+        print(f"  Relay configured but not connected yet — {info.relay_addr}", flush=True)
+    return bridge, {
+        "relay_addr": info.relay_addr,
+        "peer_id": info.peer_id,
+    }
+
+
+def _attach_relay_bridge(node: StubNodeServer | TorchNodeServer, bridge: RelayHttpBridge | None) -> None:
+    setattr(node, "relay_bridge", bridge)
+    if bridge is None:
+        return
+    original_stop = node.stop
+
+    def _stop_with_bridge() -> None:
+        try:
+            bridge.stop()
+        finally:
+            original_stop()
+
+    node.stop = _stop_with_bridge  # type: ignore[method-assign]
+
+
+def _start_heartbeat(
+    tracker_url: str,
+    node_id: str,
+    register_payload: dict,
+    interval: float = 20.0,
+) -> threading.Thread:
+    """Daemon thread: sends heartbeats and re-registers automatically after tracker restarts."""
+    def _reregister() -> bool:
+        nonlocal node_id
+        try:
+            resp = _post_json(f"{tracker_url}/v1/nodes/register", register_payload)
+            node_id = resp.get("node_id", node_id)
+            return True
+        except Exception:
+            return False
+
+    def _loop() -> None:
+        nonlocal node_id
+        hb_url = f"{tracker_url}/v1/nodes/{node_id}/heartbeat"
+        outage_streak = 0  # consecutive intervals where tracker was unreachable
+
+        while True:
+            time.sleep(interval)
+
+            if outage_streak > 0:
+                # Tracker was down — attempt re-registration first (it may have restarted
+                # with a clean slate and won't know this node).
+                if _reregister():
+                    hb_url = f"{tracker_url}/v1/nodes/{node_id}/heartbeat"
+                    print(f"  [node] re-registered after outage — node ID: {node_id}", flush=True)
+                    outage_streak = 0
+                else:
+                    outage_streak += 1
+                    if outage_streak <= 3 or outage_streak % 10 == 0:
+                        print(
+                            f"  [node] WARNING: tracker still unreachable "
+                            f"({outage_streak * interval:.0f}s)",
+                            flush=True,
+                        )
+                continue
+
+            try:
+                _post_json(hb_url, {})
+            except urllib.error.HTTPError as exc:
+                if exc.code == 404:
+                    # Node was purged (e.g. long gap before restart noticed) — re-register now.
+                    print("  [node] tracker lost registration — re-registering...", flush=True)
+                    if _reregister():
+                        hb_url = f"{tracker_url}/v1/nodes/{node_id}/heartbeat"
+                        print(f"  [node] re-registered — node ID: {node_id}", flush=True)
+                    else:
+                        print("  [node] WARNING: re-registration failed", flush=True)
+                        outage_streak = 1
+                else:
+                    print(f"  [node] WARNING: heartbeat failed ({exc.code}): {exc}", flush=True)
+            except Exception as exc:
+                outage_streak = 1
+                print(f"  [node] WARNING: tracker unreachable: {exc}", flush=True)
+
+    t = threading.Thread(target=_loop, daemon=True, name="heartbeat")
+    t.start()
+    return t
+
+
 def run_startup(
     tracker_url: str,
     port: int = 0,
@@ -62,6 +180,19 @@ def run_startup(
     tracker_url = tracker_url.rstrip("/")
 
     # 1. Hardware detection
+    if advertise_host is None and host == "0.0.0.0":
+        # socket.getfqdn() returns an mDNS name (.local / .localdomain) that remote
+        # machines on a different OS or subnet often can't resolve. Instead, probe the
+        # outbound IP by opening a UDP socket toward the tracker — no data is sent.
+        try:
+            _tracker_host = urllib.parse.urlparse(tracker_url).hostname or "8.8.8.8"
+            _s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+            _s.connect((_tracker_host, 80))
+            advertise_host = _s.getsockname()[0]
+            _s.close()
+        except Exception:
+            advertise_host = socket.getfqdn()
+
     print("Detecting hardware...", flush=True)
     hw = detect_hardware()
     device: str = hw["device"]
@@ -84,7 +215,36 @@ def run_startup(
     if probationary_line is not None:
         print(f"  {probationary_line}", flush=True)
 
-    if model_id is not None and shard_start is not None and shard_end is not None:
+    if model_id:  # treat "" the same as None — no explicit model given
+        # Auto-detect shard range from model config if not explicitly provided
+        if shard_start is None or shard_end is None:
+            detected = _detect_num_layers(model_id)
+            if detected is None:
+                raise ValueError(
+                    f"Could not read num_hidden_layers from {model_id} config. "
+                    "Pass --shard-start and --shard-end explicitly."
+                )
+            # When no explicit shard range given, ask the tracker if there's a gap for this model.
+            if shard_start is None and shard_end is None:
+                try:
+                    qs = urllib.parse.urlencode({
+                        "device": device, "vram_mb": vram_mb, "hf_repo": model_id,
+                    })
+                    net_asgn = _get_json(f"{tracker_url}/v1/network/assign?{qs}", timeout=5.0)
+                    if net_asgn.get("hf_repo") == model_id and net_asgn.get("gap_found"):
+                        shard_start = net_asgn["shard_start"]
+                        shard_end = net_asgn["shard_end"]
+                        print(
+                            f"  Tracker found uncovered shard: "
+                            f"layers {shard_start}–{shard_end} (of {detected})",
+                            flush=True,
+                        )
+                except Exception:
+                    pass  # No other nodes registered yet — default to full model below
+            shard_start = shard_start if shard_start is not None else 0
+            shard_end = shard_end if shard_end is not None else detected - 1
+            print(f"  Auto-detected {detected} layers → shard {shard_start}–{shard_end}", flush=True)
+
         print("Loading real PyTorch model shard...", flush=True)
         node = TorchNodeServer(
             host=host,
@@ -93,27 +253,153 @@ def run_startup(
             shard_start=shard_start,
             shard_end=shard_end,
             quantization=quantization,
+            tracker_url=tracker_url,
         )
         actual_port = node.start()
+        total_layers = getattr(getattr(node, "backend", None), "total_layers", None)
+        if isinstance(total_layers, int) and total_layers > 0:
+            layer_count = shard_end - shard_start + 1
+            shard_label = f"layers {shard_start}–{shard_end}; {layer_count} of {total_layers}"
+        else:
+            shard_label = f"layers {shard_start}–{shard_end}"
         public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host)
         endpoint = f"http://{public_host}:{actual_port}"
+        local_base_url = f"http://127.0.0.1:{actual_port}"
+        relay_bridge, relay_fields = _start_relay_bridge_if_available(
+            tracker_url,
+            address,
+            local_base_url,
+            endpoint,
+        )
+        _attach_relay_bridge(node, relay_bridge)
+        # Register with tracker so other nodes can auto-join this model.
+        total_layers = getattr(getattr(node, "backend", None), "total_layers", None)
+        reg_payload = {
+            "endpoint": endpoint,
+            "model": model_id.split("/")[-1],
+            "hf_repo": model_id,
+            "num_layers": total_layers,
+            "shard_start": shard_start,
+            "shard_end": shard_end,
+            "hardware_profile": hw,
+            "wallet_address": address,
+            "quantization": quantization,
+            "score": 1.0,
+            "tracker_mode": (shard_start == 0),
+            **relay_fields,
+        }
+        tracker_node_id: str | None = None
+        try:
+            reg_resp = _post_json(f"{tracker_url}/v1/nodes/register", reg_payload)
+            tracker_node_id = str(reg_resp.get("node_id") or "?")
+            setattr(node, "tracker_node_id", tracker_node_id)
+            print(f"  Registered with tracker — node ID: {tracker_node_id}", flush=True)
+            _start_heartbeat(tracker_url, tracker_node_id, reg_payload)
+        except Exception as exc:
+            setattr(node, "tracker_node_id", None)
+            print(f"  Warning: tracker registration failed: {exc}", flush=True)
+
         print(
             f"\n{'=' * 32}\n"
             f"meshnet-node ready\n"
             f"  Wallet:       {address}\n"
             f"  Model ID:     {model_id}\n"
-            f"  Shard:        layers {shard_start}-{shard_end}\n"
+            f"  Shard:        {shard_label}\n"
             f"  Quantization: {quantization}\n"
             f"  Endpoint:     {endpoint}\n"
+            f"  Node ID:      {tracker_node_id or 'unregistered'}\n"
             f"  Hardware:     {device.upper()}\n"
             f"{'=' * 32}",
             flush=True,
         )
         return node
-    if model_id is not None or shard_start is not None or shard_end is not None:
+    if shard_start is not None or shard_end is not None:
-        raise ValueError("--model-id, --shard-start, and --shard-end must be provided together")
+        raise ValueError("--shard-start / --shard-end require --model-id")
 
-    # 3. Shard assignment from tracker
+    # 3a. Auto-join: query tracker for network-wide HF model assignment.
+    print("Querying tracker for network assignment...", flush=True)
+    assign_qs = urllib.parse.urlencode({"device": device, "vram_mb": vram_mb})
+    net_assignment: dict = {}
+    try:
+        net_assignment = _get_json(f"{tracker_url}/v1/network/assign?{assign_qs}")
+    except Exception as exc:
+        print(f"  (auto-join unavailable: {exc})", flush=True)
+    assigned_hf_repo: str | None = net_assignment.get("hf_repo")
+    _gap_found: bool = bool(net_assignment.get("gap_found", False))
+
+    if assigned_hf_repo and _gap_found:
+        assigned_shard_start: int = net_assignment["shard_start"]
+        assigned_shard_end: int = net_assignment["shard_end"]
+        assigned_num_layers: int = net_assignment["num_layers"]
+        print(
+            f"  Assigned: {assigned_hf_repo} "
+            f"layers {assigned_shard_start}–{assigned_shard_end} "
+            f"(of {assigned_num_layers})",
+            flush=True,
+        )
+        print("Loading real PyTorch model shard...", flush=True)
+        node = TorchNodeServer(
+            host=host,
+            port=port,
+            model_id=assigned_hf_repo,
+            shard_start=assigned_shard_start,
+            shard_end=assigned_shard_end,
+            quantization=quantization,
+            tracker_url=tracker_url,
+        )
+        actual_port = node.start()
+        public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host)
+        endpoint = f"http://{public_host}:{actual_port}"
+        local_base_url = f"http://127.0.0.1:{actual_port}"
+        relay_bridge, relay_fields = _start_relay_bridge_if_available(
+            tracker_url,
+            address,
+            local_base_url,
+            endpoint,
+        )
+        _attach_relay_bridge(node, relay_bridge)
+        auto_reg_payload = {
+            "endpoint": endpoint,
+            "model": assigned_hf_repo.split("/")[-1],
+            "hf_repo": assigned_hf_repo,
+            "num_layers": assigned_num_layers,
+            "shard_start": assigned_shard_start,
+            "shard_end": assigned_shard_end,
+            "hardware_profile": hw,
+            "wallet_address": address,
+            "quantization": quantization,
+            "score": 1.0,
+            "tracker_mode": (assigned_shard_start == 0),
+            **relay_fields,
+        }
+        tracker_node_id = None
+        try:
+            reg_resp = _post_json(f"{tracker_url}/v1/nodes/register", auto_reg_payload)
+            tracker_node_id = str(reg_resp.get("node_id") or "?")
+            setattr(node, "tracker_node_id", tracker_node_id)
+            print(f"  Registered with tracker — node ID: {tracker_node_id}", flush=True)
+            _start_heartbeat(tracker_url, tracker_node_id, auto_reg_payload)
+        except Exception as exc:
+            setattr(node, "tracker_node_id", None)
+            print(f"  Warning: tracker registration failed: {exc}", flush=True)
+        shard_count = assigned_shard_end - assigned_shard_start + 1
+        print(
+            f"\n{'=' * 32}\n"
+            f"meshnet-node ready (auto-joined)\n"
+            f"  Wallet:       {address}\n"
+            f"  Model ID:     {assigned_hf_repo}\n"
+            f"  Shard:        layers {assigned_shard_start}–{assigned_shard_end} "
+            f"({shard_count} of {assigned_num_layers})\n"
+            f"  Quantization: {quantization}\n"
+            f"  Endpoint:     {endpoint}\n"
+            f"  Node ID:      {tracker_node_id or 'unregistered'}\n"
+            f"  Hardware:     {device.upper()}\n"
+            f"{'=' * 32}",
+            flush=True,
+        )
+        return node
+
+    # 3b. Shard assignment from tracker (stub-model / preset-based path)
     print("Querying tracker for shard assignment...", flush=True)
     assign_qs = urllib.parse.urlencode({
         "model": model,
@@ -160,6 +446,14 @@ def run_startup(
     actual_port = node.start()
     public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host)
     endpoint = f"http://{public_host}:{actual_port}"
+    local_base_url = f"http://127.0.0.1:{actual_port}"
+    relay_bridge, relay_fields = _start_relay_bridge_if_available(
+        tracker_url,
+        address,
+        local_base_url,
+        endpoint,
+    )
+    _attach_relay_bridge(node, relay_bridge)
 
     # 6. Register with tracker
     print("Registering with tracker...", flush=True)
@@ -175,9 +469,11 @@ def run_startup(
                 "hardware_profile": hw,
                 "wallet_address": address,
                 "score": 1.0,
+                **relay_fields,
             },
         )
-        node_id: str = reg_resp["node_id"]
+        node_id = str(reg_resp["node_id"])
+        setattr(node, "tracker_node_id", node_id)
     except Exception:
         node.stop()
         raise
@@ -201,6 +497,17 @@ def run_startup(
     return node
 
 
+def _detect_num_layers(model_id: str) -> int | None:
+    """Fetch num_hidden_layers from HuggingFace model config (downloads ~1 KB config.json only)."""
+    try:
+        from transformers import AutoConfig  # type: ignore[import]
+        cfg = AutoConfig.from_pretrained(model_id)
+        return int(cfg.num_hidden_layers)
+    except Exception as exc:
+        print(f"  Warning: could not read model config from HF: {exc}", flush=True)
+        return None
+
+
 def _probationary_status_line(contracts: Any | None, wallet_address: str) -> str | None:
     if contracts is None:
         return None

packages/node/meshnet_node/torch_server.py

@@ -11,6 +11,7 @@ import urllib.error
 import urllib.parse
 import urllib.request
 import uuid
+from typing import Any
 
 from .model_backend import (
     InsufficientVRAMError,
@@ -205,7 +206,10 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         self.send_header("Content-Type", "application/json")
         self.send_header("Content-Length", str(len(payload)))
         self.end_headers()
+        try:
             self.wfile.write(payload)
+        except BrokenPipeError:
+            pass  # client disconnected before we could respond — not an error
 
     def _handle_chat_completions(self) -> None:
         server: _TorchHTTPServer = self.server  # type: ignore[assignment]
@@ -213,40 +217,123 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         if body is None:
             return
         messages = body.get("messages", [])
+        if not isinstance(messages, list):
+            messages = []
         stream = bool(body.get("stream", False))
-        model = str(body.get("model", ""))
+        model_name = str(body.get("model", ""))
-        prompt = " ".join(
+        max_tokens = int(body.get("max_tokens") or body.get("max_new_tokens") or 256)
+        temperature = float(body.get("temperature") or 1.0)
+        top_p = float(body.get("top_p") or 1.0)
+
+        # Fast path: this node owns the complete model — use HF generate() with KV cache.
+        # Avoids the single-token-per-forward-pass limitation of the distributed path.
+        if server.backend.is_head and server.backend.is_tail:
+            try:
+                if stream:
+                    self._stream_openai_response(
+                        server.backend.generate_text_streaming(messages, max_tokens, temperature, top_p),
+                        model_name,
+                    )
+                else:
+                    text = server.backend.generate_text(messages, max_tokens, temperature, top_p)
+                    self._send_openai_response(text, model_name, False, messages)
+            except Exception as exc:
+                self._send_json(500, {"error": f"generation failed: {exc}"})
+            return
+
+        # Distributed path: autoregressive generation across shards.
+        # We do N single-step forward passes (no cross-node KV cache), which is slow
+        # but correct. Each step: head encodes current sequence → forwards through route
+        # → tail returns the next token string → append → repeat.
+        remaining_route = self._get_remaining_route(model_name)
+        print(
+            f"  [node] chat route model={model_name!r} max_tokens={max_tokens} "
+            f"downstream={remaining_route}",
+            flush=True,
+        )
+        if not remaining_route:
+            self._send_openai_response(
+                "error: no downstream route — check tracker connectivity",
+                model_name, False, messages,
+            )
+            return
+
+        backend = server.backend
+        # Format with chat template so the model knows it's in assistant mode.
+        try:
+            if hasattr(backend.tokenizer, "apply_chat_template"):
+                prompt_text: str = backend.tokenizer.apply_chat_template(
+                    messages, add_generation_prompt=True, tokenize=False,
+                )
+            else:
+                raise AttributeError("no apply_chat_template")
+        except Exception:
+            prompt_text = " ".join(
                 str(m.get("content", ""))
                 for m in messages
                 if isinstance(m, dict) and m.get("role") == "user"
             )
+
+        eos_token: str = getattr(backend.tokenizer, "eos_token", "") or ""
+        generated: list[str] = []
+        current_text = prompt_text
+
+        for _ in range(max_tokens):
             try:
-            payload = server.backend.encode_prompt(prompt)
+                payload = backend.encode_prompt(current_text)
             except Exception as exc:
-            self._send_json(500, {"error": f"encode_prompt failed: {exc}"})
+                print(f"  [node] distributed encode error: {exc}", flush=True)
-            return
+                break
-        remaining_route = self._get_remaining_route(model)
+            token_str = self._run_downstream_pipeline(payload, remaining_route)
-        result_text = self._run_downstream_pipeline(payload, remaining_route)
+            if not token_str:
-        self._send_openai_response(result_text, model, stream)
+                break
+            # Stop on error responses or EOS.
+            if token_str.startswith(("pipeline error", "decode error", "no downstream", "error:")):
+                break
+            if eos_token and token_str == eos_token:
+                break
+            generated.append(token_str)
+            current_text = current_text + token_str
+
+        result_text = "".join(generated)
+        self._send_openai_response(result_text, model_name, stream, messages)
 
     def _get_remaining_route(self, model: str) -> list[str]:
+        # Fast path: tracker pre-resolved the downstream route and injected it as a header.
+        injected = self.headers.get("X-Meshnet-Route")
+        if injected:
+            try:
+                route = json.loads(injected)
+                if isinstance(route, list):
+                    resolved = [str(ep) for ep in route]
+                    print(f"  [node] using injected downstream route: {resolved}", flush=True)
+                    return resolved
+            except (json.JSONDecodeError, TypeError):
+                pass
+
+        # Slow path: query the tracker (direct node-to-node calls, or tracker didn't inject).
         server: _TorchHTTPServer = self.server  # type: ignore[assignment]
         if server.tracker_url is None:
             return []
+        route_model = getattr(server.backend, "model_id", None) or model
         try:
-            url = f"{server.tracker_url}/v1/route?model={urllib.parse.quote(model)}"
+            url = f"{server.tracker_url}/v1/route?model={urllib.parse.quote(route_model)}"
             with urllib.request.urlopen(url, timeout=5.0) as r:
                 route_resp = json.loads(r.read())
             route = route_resp.get("route", [])
-            # Skip the first node in the route (self) since we're already the head
+            own_port = server.server_address[1]
-            return list(route[1:])
+            resolved = [ep for ep in route if not ep.rstrip("/").endswith(f":{own_port}")]
-        except Exception:
+            print(f"  [node] tracker downstream route: {resolved}", flush=True)
+            return resolved
+        except Exception as exc:
+            print(f"  [node] WARNING: route lookup failed for {route_model!r}: {exc}", flush=True)
             return []
 
     def _run_downstream_pipeline(self, payload: object, route: list[str]) -> str:
         server: _TorchHTTPServer = self.server  # type: ignore[assignment]
         if not route:
-            # Single-node mode: decode tail locally if we're the tail
+            # Partial shard at tail: decode the activation from the previous node.
+            # Full single-node (head+tail) is handled before entering this method.
             if server.backend.is_tail:
                 try:
                     tensor = server.backend.torch.frombuffer(
@@ -256,7 +343,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
                     return server.backend.decode_tail(tensor)
                 except Exception as exc:
                     return f"decode error: {exc}"
-            return ""
+            return "no downstream route available for non-tail shard"
 
         session = str(uuid.uuid4())
         shape = payload.shape  # type: ignore[union-attr]
@@ -268,6 +355,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         current_pos = pos_ids
 
         for hop_index, node_url in enumerate(route):
+            print(f"  [node] pipeline hop {hop_index}: {node_url}", flush=True)
             headers: dict[str, str] = {
                 "Content-Type": "application/octet-stream",
                 "X-Meshnet-Wire": _WIRE_VERSION,
@@ -289,16 +377,19 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
                 method="POST",
             )
             try:
-                with urllib.request.urlopen(req, timeout=10.0) as r:
+                with urllib.request.urlopen(req, timeout=120.0) as r:
                     resp_body = r.read()
                     resp_headers = {k.lower(): v for k, v in r.headers.items()}
             except Exception as exc:
+                print(f"  [node] pipeline hop {hop_index} failed at {node_url}: {exc}", flush=True)
                 return f"pipeline error at {node_url}: {exc}"
             content_type = resp_headers.get("content-type", "")
             if "application/json" in content_type:
                 try:
                     data = json.loads(resp_body)
-                    return str(data.get("text", ""))
+                    text = str(data.get("text", ""))
+                    print(f"  [node] pipeline hop {hop_index} returned text={text!r}", flush=True)
+                    return text
                 except json.JSONDecodeError:
                     return resp_body.decode("utf-8", errors="replace")
             # Binary activation — update and forward to next node
@@ -309,10 +400,57 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             current_pos = resp_headers.get("x-meshnet-position-ids")
         return ""
 
-    def _send_openai_response(self, text: str, model: str, stream: bool) -> None:
+    def _stream_openai_response(self, token_iter, model: str) -> None:
+        """Stream tokens from an iterator as SSE chunks."""
+        chunk_id = "chatcmpl-node"
+        created = int(time.time())
+        self.send_response(200)
+        self.send_header("Content-Type", "text/event-stream; charset=utf-8")
+        self.send_header("Cache-Control", "no-cache")
+        self.end_headers()
+
+        def _emit(data: str) -> None:
+            try:
+                self.wfile.write(f"data: {data}\n\n".encode())
+                self.wfile.flush()
+            except BrokenPipeError:
+                pass
+
+        _emit(json.dumps({
+            "id": chunk_id, "object": "chat.completion.chunk", "created": created,
+            "model": model,
+            "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
+        }))
+        for token_text in token_iter:
+            if not token_text:
+                continue
+            _emit(json.dumps({
+                "id": chunk_id, "object": "chat.completion.chunk", "created": created,
+                "model": model,
+                "choices": [{"index": 0, "delta": {"content": token_text}, "finish_reason": None}],
+            }))
+        _emit(json.dumps({
+            "id": chunk_id, "object": "chat.completion.chunk", "created": created,
+            "model": model,
+            "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
+        }))
+        try:
+            self.wfile.write(b"data: [DONE]\n\n")
+            self.wfile.flush()
+        except BrokenPipeError:
+            pass
+
+    def _send_openai_response(
+        self,
+        text: str,
+        model: str,
+        stream: bool,
+        messages: list[dict] | None = None,
+    ) -> None:
         chunk_id = "chatcmpl-node"
         created = int(time.time())
         if not stream:
+            usage = _usage_for_response(self.server.backend, messages or [], text)  # type: ignore[attr-defined]
             self._send_json(200, {
                 "id": chunk_id,
                 "object": "chat.completion",
@@ -323,7 +461,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
                     "message": {"role": "assistant", "content": text},
                     "finish_reason": "stop",
                 }],
-                "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
+                "usage": usage,
             })
             return
         self.send_response(200)
@@ -332,8 +470,11 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
         self.end_headers()
 
         def _emit(data: str) -> None:
+            try:
                 self.wfile.write(f"data: {data}\n\n".encode())
                 self.wfile.flush()
+            except BrokenPipeError:
+                pass
 
         _emit(json.dumps({
             "id": chunk_id, "object": "chat.completion.chunk", "created": created,
@@ -350,8 +491,57 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
             "model": model,
             "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
         }))
+        try:
             self.wfile.write(b"data: [DONE]\n\n")
             self.wfile.flush()
+        except BrokenPipeError:
+            pass
+
+
+def _usage_for_response(backend: object, messages: list[dict], completion_text: str) -> dict[str, int]:
+    prompt_tokens = _backend_token_count(
+        backend,
+        "count_prompt_tokens",
+        messages,
+        fallback=_fallback_message_token_count(messages),
+    )
+    completion_tokens = _backend_token_count(
+        backend,
+        "count_text_tokens",
+        completion_text,
+        fallback=_fallback_text_token_count(completion_text),
+    )
+    return {
+        "prompt_tokens": prompt_tokens,
+        "completion_tokens": completion_tokens,
+        "total_tokens": prompt_tokens + completion_tokens,
+    }
+
+
+def _backend_token_count(backend: object, method_name: str, value: object, fallback: int) -> int:
+    method: Any = getattr(backend, method_name, None)
+    if callable(method):
+        try:
+            return max(0, int(method(value)))
+        except Exception:
+            pass
+    return max(0, int(fallback))
+
+
+def _fallback_message_token_count(messages: list[dict]) -> int:
+    text = " ".join(
+        str(message.get("content", ""))
+        for message in messages
+        if isinstance(message, dict)
+    )
+    return _fallback_text_token_count(text)
+
+
+def _fallback_text_token_count(text: str) -> int:
+    parts = text.split()
+    if parts:
+        return len(parts)
+    return 1 if text else 0
 
 
 class TorchNodeServer:

packages/node/meshnet_node/wizard.py

@@ -0,0 +1,332 @@
+"""Interactive first-run setup wizard — mining-client style."""
+
+from __future__ import annotations
+
+import sys
+import urllib.error
+import urllib.request
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from .config import DEFAULTS, _DEFAULT_DOWNLOAD_DIR, _DEFAULT_TRACKER_URL, _DEFAULT_WALLET_PATH
+from .model_catalog import CURATED_MODELS, ModelPreset, browse_hf_hub, detect_num_layers
+
+if TYPE_CHECKING:
+    pass
+
+_HEADER = """\
+╔══════════════════════════════════════════════════════════════════╗
+║              meshnet-node  v0.1.0                                ║
+║        Distributed AI Inference — Node Setup                     ║
+╚══════════════════════════════════════════════════════════════════╝
+"""
+
+_QUANT_LABELS = {"nf4": "NF4 (4-bit)", "int8": "INT8 (8-bit)", "bf16": "BF16 (full)"}
+
+
+def _ask(prompt: str, default: str = "", validator=None) -> str:
+    """Prompt user and return answer. Returns default on empty input or EOF."""
+    display = f"{prompt} [{default}]: " if default else f"{prompt}: "
+    while True:
+        try:
+            raw = input(display).strip()
+        except (EOFError, KeyboardInterrupt):
+            print()
+            raise KeyboardInterrupt
+        value = raw or default
+        if validator is None or validator(value):
+            return value
+        # validator returned error string
+        print(f"  ✗ {validator(value)}")
+
+
+def _ask_int(prompt: str, default: int, lo: int, hi: int) -> int:
+    def validate(s: str) -> bool | str:
+        try:
+            v = int(s)
+        except ValueError:
+            return "Please enter a number."
+        if not (lo <= v <= hi):
+            return f"Please enter a number between {lo} and {hi}."
+        return True
+
+    while True:
+        raw = _ask(prompt, str(default))
+        try:
+            v = int(raw)
+            if lo <= v <= hi:
+                return v
+        except ValueError:
+            pass
+        print(f"  ✗ Enter a number between {lo} and {hi}.")
+
+
+def _ask_yn(prompt: str, default: bool = True) -> bool:
+    hint = "Y/n" if default else "y/N"
+    raw = _ask(f"{prompt} [{hint}]").lower()
+    if not raw:
+        return default
+    return raw.startswith("y")
+
+
+def _detect_gpus() -> list[dict]:
+    """Return list of detected GPU dicts with name and vram_gb."""
+    gpus: list[dict] = []
+    try:
+        import torch  # type: ignore[import]
+        if torch.cuda.is_available():
+            for i in range(torch.cuda.device_count()):
+                props = torch.cuda.get_device_properties(i)
+                gpus.append(
+                    {
+                        "index": i,
+                        "name": props.name,
+                        "vram_gb": props.total_memory / 1e9,
+                        "backend": "cuda",
+                    }
+                )
+    except ImportError:
+        pass
+    return gpus
+
+
+def _total_vram_gb(gpus: list[dict]) -> float:
+    return sum(g["vram_gb"] for g in gpus)
+
+
+def _print_gpus(gpus: list[dict]) -> None:
+    if not gpus:
+        print("  ⚠ No CUDA GPU detected — running in CPU mode")
+        print("    CPU inference is very slow. Consider a machine with an NVIDIA GPU.")
+        return
+    for g in gpus:
+        vram = g["vram_gb"]
+        print(f"  GPU {g['index']}: {g['name']}  {vram:.0f} GB VRAM  ✓")
+
+
+def _print_model_table(gpus: list[dict], quant: str = "nf4") -> None:
+    available_gb = _total_vram_gb(gpus)
+    print()
+    print(f" #   {'Model':<30} {'Layers':>6}   {'NF4':>6}   {'INT8':>6}   {'BF16':>6}")
+    print(f" {'─'*4} {'─'*30} {'─'*6}   {'─'*6}   {'─'*6}   {'─'*6}")
+    for i, m in enumerate(CURATED_MODELS, 1):
+        fits_nf4 = "✓" if m.vram_nf4 <= available_gb else "✗"
+        fits_int8 = "✓" if m.vram_int8 <= available_gb else "✗"
+        fits_bf16 = "✓" if m.vram_bf16 <= available_gb else "✗"
+        nf4_str = f"{fits_nf4}{m.vram_nf4:.0f}GB"
+        int8_str = f"{fits_int8}{m.vram_int8:.0f}GB"
+        bf16_str = f"{fits_bf16}{m.vram_bf16:.0f}GB"
+        print(f" {i:<3} {m.name:<30} {m.num_layers:>6}   {nf4_str:>6}   {int8_str:>6}   {bf16_str:>6}")
+        print(f"     {m.description}")
+    idx = len(CURATED_MODELS) + 1
+    print(f" {idx:<3} {'[Browse HuggingFace Hub...]':<30}")
+    print()
+
+
+def _browse_hf_interactive() -> str | None:
+    """Show HF Hub top-20 and let user enter a repo ID. Returns repo ID or None to go back."""
+    print("\nFetching top models from HuggingFace Hub...")
+    try:
+        models = browse_hf_hub(top_n=20)
+    except RuntimeError as exc:
+        print(f"  ✗ {exc}")
+        return None
+
+    print(f"\n {'#':<4} {'HuggingFace Repo':<50} Downloads")
+    print(f" {'─'*4} {'─'*50} {'─'*10}")
+    for i, m in enumerate(models, 1):
+        dl = m["downloads"]
+        dl_str = f"{dl/1e6:.1f}M" if dl >= 1_000_000 else f"{dl/1e3:.0f}k" if dl >= 1000 else str(dl)
+        print(f" {i:<4} {m['repo']:<50} {dl_str}")
+
+    print()
+    raw = _ask(
+        "Enter a number to select, or paste any HuggingFace repo ID (or press Enter to go back)",
+        default="",
+    )
+    if not raw:
+        return None
+    try:
+        idx = int(raw) - 1
+        if 0 <= idx < len(models):
+            return models[idx]["repo"]
+    except ValueError:
+        pass
+    # Treat raw input as a repo ID
+    if "/" in raw:
+        return raw
+    print("  ✗ Invalid input — please enter a number or a full repo ID like 'org/model-name'")
+    return None
+
+
+def _ask_quant(gpus: list[dict], model: ModelPreset | None) -> str:
+    available_gb = _total_vram_gb(gpus)
+    print("\nQuantization level:")
+    options: list[tuple[str, str]] = []
+    for quant, label in [("nf4", "NF4 4-bit"), ("int8", "INT8 8-bit"), ("bf16", "BF16 full precision")]:
+        if model is not None:
+            vram = model.vram_for_quant(quant)
+            fits = "✓" if vram <= available_gb else "✗ insufficient VRAM"
+            suffix = f"  ({vram:.0f} GB needed — {fits})"
+        else:
+            suffix = ""
+        options.append((quant, f"{label}{suffix}"))
+
+    for i, (_, label) in enumerate(options, 1):
+        print(f"  {i}) {label}")
+
+    # Recommend the best fitting quant
+    if model is not None:
+        rec = model.recommended_quant(available_gb)
+        rec_idx = next((i for i, (q, _) in enumerate(options, 1) if q == rec), 1) if rec else 1
+        default_idx = rec_idx
+        print(f"  (Recommended: {rec.upper() if rec else 'NF4'} for your GPU)")
+    else:
+        default_idx = 1
+
+    choice = _ask_int("Enter number", default_idx, 1, 3)
+    return options[choice - 1][0]
+
+
+def _validate_dir(path_str: str) -> bool | str:
+    p = Path(path_str).expanduser()
+    try:
+        p.mkdir(parents=True, exist_ok=True)
+        return True
+    except OSError as exc:
+        return f"Cannot create directory: {exc}"
+
+
+def _validate_tracker(url: str) -> bool | str:
+    if not url.startswith(("http://", "https://")):
+        return "URL must start with http:// or https://"
+    return True
+
+
+def _ping_tracker(url: str) -> bool:
+    """Return True if tracker responds to /health."""
+    try:
+        with urllib.request.urlopen(f"{url.rstrip('/')}/health", timeout=3):
+            return True
+    except Exception:
+        return False
+
+
+def run_wizard(config_path_override=None) -> dict:
+    """Run the interactive setup wizard and return a config dict.
+
+    Raises KeyboardInterrupt if user presses Ctrl-C.
+    """
+    print(_HEADER)
+
+    # Step 1: GPU detection
+    print("Detecting hardware...")
+    gpus = _detect_gpus()
+    _print_gpus(gpus)
+    available_gb = _total_vram_gb(gpus)
+    if available_gb == 0:
+        available_gb = 9999  # CPU — don't filter models by VRAM
+
+    # Step 2 & 3: Model selection
+    print("\nSelect a model to serve:\n")
+    selected_repo: str | None = None
+    selected_preset: ModelPreset | None = None
+
+    while selected_repo is None:
+        _print_model_table(gpus)
+        lo, hi = 1, len(CURATED_MODELS) + 1
+        choice = _ask_int("Enter number", 1, lo, hi)
+        if choice == len(CURATED_MODELS) + 1:
+            repo = _browse_hf_interactive()
+            if repo:
+                # Look up layer count for custom repo
+                print(f"  Checking {repo} config...", end=" ", flush=True)
+                layers = detect_num_layers(repo)
+                if layers:
+                    print(f"{layers} layers")
+                else:
+                    print("(layer count unknown — will detect on start)")
+                selected_repo = repo
+                selected_preset = None
+        else:
+            selected_preset = CURATED_MODELS[choice - 1]
+            selected_repo = selected_preset.hf_repo
+            if selected_preset.recommended_quant(available_gb) is None:
+                print(
+                    f"\n  ⚠ Warning: {selected_preset.name} requires at least "
+                    f"{selected_preset.vram_nf4:.0f} GB VRAM at NF4 — even the smallest "
+                    f"quantization may be too large for your GPU."
+                )
+                if not _ask_yn("Continue anyway?", default=False):
+                    selected_repo = None
+                    selected_preset = None
+
+    num_layers = (selected_preset.num_layers if selected_preset
+                  else detect_num_layers(selected_repo or ""))
+    layers_str = f"  {num_layers} layers" if num_layers else ""
+    print(f"\n  ✓ Selected: {selected_repo}{layers_str}")
+
+    # Step 3b: Quantization
+    quant = _ask_quant(gpus, selected_preset)
+    print(f"  ✓ Quantization: {quant.upper()}")
+
+    # Step 4: Download directory
+    print()
+    dl_dir = _ask(
+        "Download directory",
+        default=str(_DEFAULT_DOWNLOAD_DIR),
+        validator=lambda v: _validate_dir(v) if v else "Directory is required.",
+    )
+    print(f"  ✓ Download dir: {dl_dir}")
+
+    # Step 5: Tracker URL
+    print()
+    tracker_url = _DEFAULT_TRACKER_URL
+    raw_tracker = _ask("Tracker URL", default=_DEFAULT_TRACKER_URL, validator=_validate_tracker)
+    tracker_url = raw_tracker
+    if _ping_tracker(tracker_url):
+        print(f"  ✓ Tracker reachable: {tracker_url}")
+    else:
+        print(f"  ⚠ Tracker not reachable at {tracker_url} (will retry on start)")
+
+    # Step 6: Wallet path
+    print()
+    wallet_path = _ask("Wallet path", default=_DEFAULT_WALLET_PATH)
+    print(f"  ✓ Wallet: {wallet_path}")
+
+    cfg = {
+        "model_hf_repo": selected_repo,
+        "model_name": selected_preset.name if selected_preset else selected_repo.split("/")[-1],
+        "quantization": quant,
+        "download_dir": dl_dir,
+        "tracker_url": tracker_url,
+        "wallet_path": wallet_path,
+        "shard_start": None,
+        "shard_end": None,
+        "port": DEFAULTS["port"],
+        "host": DEFAULTS["host"],
+    }
+    return cfg
+
+
+def print_models_table(available_gb: float | None = None) -> None:
+    """Print curated model table for `meshnet-node models`."""
+    gpus: list[dict] = []
+    if available_gb is None:
+        gpus = _detect_gpus()
+        available_gb = _total_vram_gb(gpus) or 9999
+    else:
+        gpus = [{"index": 0, "name": "GPU", "vram_gb": available_gb, "backend": "cuda"}]
+
+    print(f"\n{'#':<4} {'Model':<32} {'HuggingFace Repo':<45} {'Layers':>6}  {'NF4':>8} {'INT8':>8} {'BF16':>8}")
+    print(f"{'─'*4} {'─'*32} {'─'*45} {'─'*6}  {'─'*8} {'─'*8} {'─'*8}")
+    for i, m in enumerate(CURATED_MODELS, 1):
+        def _cell(vram: float) -> str:
+            fits = "✓" if vram <= available_gb else "✗"
+            return f"{fits}{vram:.0f}GB"
+
+        print(
+            f"{i:<4} {m.name:<32} {m.hf_repo:<45} {m.num_layers:>6}  "
+            f"{_cell(m.vram_nf4):>8} {_cell(m.vram_int8):>8} {_cell(m.vram_bf16):>8}"
+        )
+    print()

packages/node/pyproject.toml

@@ -13,9 +13,11 @@ dependencies = [
     "huggingface-hub>=0.20",
     "accelerate>=0.28",
     "bitsandbytes>=0.43",
+    "rich>=13",
     "safetensors>=0.4",
     "torch>=2.1",
     "transformers>=4.39",
+    "websockets>=13",
     "zstandard>=0.22",
 ]
 

packages/p2p/meshnet_p2p/gossip.py

@@ -0,0 +1,162 @@
+"""WebSocket gossip client — connects to relay, publish/subscribe to topics."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import threading
+import time
+import uuid
+from collections import defaultdict
+from typing import Callable
+
+log = logging.getLogger(__name__)
+
+# Message envelope topics
+TOPIC_NODE_JOIN = "node-join"
+TOPIC_NODE_LEAVE = "node-leave"
+TOPIC_COVERAGE_UPDATE = "coverage-update"
+TOPIC_HEARTBEAT = "heartbeat"
+TOPIC_PEER_LIST = "peer-list"
+TOPIC_RELAY_ANNOUNCE = "relay-announce"
+
+_MSG_TTL = 3  # max re-broadcast hops
+
+
+def _make_envelope(topic: str, payload: dict, from_peer: str, ttl: int = _MSG_TTL) -> dict:
+    return {
+        "topic": topic,
+        "version": 1,
+        "from_peer": from_peer,
+        "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+        "msg_id": str(uuid.uuid4()),
+        "ttl": ttl,
+        "payload": payload,
+    }
+
+
+class GossipClient:
+    """Thread-safe WebSocket gossip client.
+
+    Usage::
+
+        client = GossipClient(relay_url="ws://relay:8765", peer_id="abc123")
+        client.subscribe("node-join", lambda env: print(env["payload"]))
+        client.start()
+        client.publish("node-join", {"addr": "http://192.168.1.42:8001", ...})
+        ...
+        client.stop()
+    """
+
+    def __init__(self, relay_url: str, peer_id: str, reconnect_interval: float = 3.0):
+        self.relay_url = relay_url
+        self.peer_id = peer_id
+        self.reconnect_interval = reconnect_interval
+
+        self._handlers: dict[str, list[Callable]] = defaultdict(list)
+        self._seen: set[str] = set()
+        self._loop: asyncio.AbstractEventLoop | None = None
+        self._thread: threading.Thread | None = None
+        self._ws = None
+        self._running = False
+        self._connected = threading.Event()
+        self._stop_event: asyncio.Event | None = None
+
+    def subscribe(self, topic: str, handler: Callable) -> None:
+        """Register a sync callback for messages on topic."""
+        self._handlers[topic].append(handler)
+
+    def publish(self, topic: str, payload: dict) -> None:
+        """Send a gossip message to all peers via the relay. Thread-safe."""
+        envelope = _make_envelope(topic, payload, self.peer_id)
+        if self._loop and self._running:
+            asyncio.run_coroutine_threadsafe(self._send(envelope), self._loop)
+
+    def start(self) -> None:
+        """Start the gossip client in a background thread."""
+        self._running = True
+        self._loop = asyncio.new_event_loop()
+        self._thread = threading.Thread(target=self._run_loop, daemon=True, name="gossip")
+        self._thread.start()
+
+    def wait_connected(self, timeout: float = 5.0) -> bool:
+        """Block until connected to relay or timeout. Returns True if connected."""
+        return self._connected.wait(timeout)
+
+    def stop(self) -> None:
+        """Stop the gossip client and clean up."""
+        self._running = False
+        if self._loop and self._stop_event is not None:
+            self._loop.call_soon_threadsafe(self._stop_event.set)
+        if self._thread:
+            self._thread.join(timeout=3.0)
+
+    # ------------------------------------------------------------------
+    # Internal asyncio methods (run inside the background event loop)
+    # ------------------------------------------------------------------
+
+    def _run_loop(self) -> None:
+        asyncio.set_event_loop(self._loop)
+        self._stop_event = asyncio.Event()
+        try:
+            self._loop.run_until_complete(self._connect_loop())
+        except Exception:
+            log.debug("Gossip loop exited", exc_info=True)
+
+    async def _connect_loop(self) -> None:
+        import websockets  # type: ignore[import]
+
+        while self._running and not (self._stop_event and self._stop_event.is_set()):
+            try:
+                async with websockets.connect(
+                    self.relay_url,
+                    ping_interval=20,
+                    ping_timeout=10,
+                    open_timeout=5,
+                ) as ws:
+                    self._ws = ws
+                    self._connected.set()
+                    log.debug("Gossip connected to %s", self.relay_url)
+                    # Send peer registration
+                    await ws.send(json.dumps(
+                        _make_envelope(
+                            "peer-register",
+                            {"peer_id": self.peer_id},
+                            self.peer_id,
+                        )
+                    ))
+                    await self._receive_loop(ws)
+            except Exception as exc:
+                self._connected.clear()
+                if self._running:
+                    log.debug("Gossip disconnected (%s); reconnecting in %ss", exc, self.reconnect_interval)
+                    await asyncio.sleep(self.reconnect_interval)
+
+    async def _receive_loop(self, ws) -> None:
+        async for raw in ws:
+            try:
+                envelope = json.loads(raw)
+            except (json.JSONDecodeError, TypeError):
+                continue
+            msg_id = envelope.get("msg_id", "")
+            if msg_id in self._seen:
+                continue
+            self._seen.add(msg_id)
+            if len(self._seen) > 10_000:
+                # Trim seen set to avoid unbounded growth
+                self._seen = set(list(self._seen)[-5_000:])
+
+            topic = envelope.get("topic", "")
+            for handler in self._handlers.get(topic, []):
+                try:
+                    handler(envelope)
+                except Exception:
+                    log.debug("Gossip handler error for topic %s", topic, exc_info=True)
+
+    async def _send(self, envelope: dict) -> None:
+        if self._ws is not None:
+            try:
+                await self._ws.send(json.dumps(envelope))
+            except Exception as exc:
+                log.debug("Gossip send failed: %s", exc)

packages/p2p/meshnet_p2p/identity.py

@@ -0,0 +1,64 @@
+"""Peer identity — stable peer_id and RSA keypair, persisted to disk."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+import stat
+from pathlib import Path
+
+_DEFAULT_IDENTITY_PATH = Path.home() / ".config" / "meshnet" / "identity.json"
+
+
+def _generate_keypair() -> tuple[bytes, bytes]:
+    """Return (private_key_pem, public_key_pem) for a new RSA-2048 keypair."""
+    from cryptography.hazmat.primitives.asymmetric import rsa
+    from cryptography.hazmat.primitives import serialization
+
+    key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
+    priv_pem = key.private_bytes(
+        serialization.Encoding.PEM,
+        serialization.PrivateFormat.PKCS8,
+        serialization.NoEncryption(),
+    )
+    pub_pem = key.public_key().public_bytes(
+        serialization.Encoding.PEM,
+        serialization.PublicFormat.SubjectPublicKeyInfo,
+    )
+    return priv_pem, pub_pem
+
+
+def _peer_id_from_pubkey(pub_pem: bytes) -> str:
+    return hashlib.sha256(pub_pem).hexdigest()[:16]
+
+
+def load_or_create_identity(path: Path | None = None) -> dict:
+    """Return identity dict with peer_id, private_key_pem, public_key_pem.
+
+    Creates and persists a new identity if none exists at path.
+    """
+    p = path or _DEFAULT_IDENTITY_PATH
+    if p.exists():
+        try:
+            data = json.loads(p.read_text())
+            if "peer_id" in data and "public_key_pem" in data:
+                return data
+        except (json.JSONDecodeError, OSError):
+            pass
+
+    priv_pem, pub_pem = _generate_keypair()
+    identity = {
+        "peer_id": _peer_id_from_pubkey(pub_pem),
+        "private_key_pem": priv_pem.decode(),
+        "public_key_pem": pub_pem.decode(),
+    }
+
+    p.parent.mkdir(parents=True, exist_ok=True)
+    p.write_text(json.dumps(identity, indent=2))
+    try:
+        os.chmod(p, stat.S_IRUSR | stat.S_IWUSR)
+    except OSError:
+        pass
+
+    return identity

packages/p2p/meshnet_p2p/mdns.py

@@ -0,0 +1,136 @@
+"""mDNS peer discovery using zeroconf (optional dependency).
+
+Falls back gracefully if zeroconf is not installed.
+"""
+
+from __future__ import annotations
+
+import logging
+import socket
+import threading
+from typing import Callable
+
+log = logging.getLogger(__name__)
+
+MDNS_SERVICE_TYPE = "_meshnet._tcp.local."
+
+try:
+    from zeroconf import ServiceInfo, ServiceBrowser, Zeroconf  # type: ignore[import]
+
+    _HAS_ZEROCONF = True
+except ImportError:
+    _HAS_ZEROCONF = False
+
+
+def _local_ip() -> str:
+    try:
+        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        s.connect(("8.8.8.8", 80))
+        ip = s.getsockname()[0]
+        s.close()
+        return ip
+    except OSError:
+        return "127.0.0.1"
+
+
+class MdnsDiscovery:
+    """Announce this node on mDNS and discover peers on the same LAN.
+
+    If `zeroconf` is not installed, all methods are no-ops.
+
+    Usage::
+
+        disc = MdnsDiscovery(
+            peer_id="abc123",
+            port=8001,
+            on_peer_found=lambda peer_id, addr: print("found", peer_id, addr),
+        )
+        disc.start()
+        ...
+        disc.stop()
+    """
+
+    def __init__(
+        self,
+        peer_id: str,
+        port: int,
+        on_peer_found: Callable[[str, str], None] | None = None,
+        on_peer_lost: Callable[[str], None] | None = None,
+    ):
+        self.peer_id = peer_id
+        self.port = port
+        self.on_peer_found = on_peer_found
+        self.on_peer_lost = on_peer_lost
+        self._zc: "Zeroconf | None" = None  # type: ignore[name-defined]
+        self._info: "ServiceInfo | None" = None  # type: ignore[name-defined]
+        self._browser = None
+
+    def is_available(self) -> bool:
+        return _HAS_ZEROCONF
+
+    def start(self) -> None:
+        if not _HAS_ZEROCONF:
+            log.info("zeroconf not installed — mDNS discovery disabled")
+            return
+        try:
+            self._zc = Zeroconf()
+            local_ip = _local_ip()
+            self._info = ServiceInfo(
+                MDNS_SERVICE_TYPE,
+                f"{self.peer_id}.{MDNS_SERVICE_TYPE}",
+                addresses=[socket.inet_aton(local_ip)],
+                port=self.port,
+                properties={"peer_id": self.peer_id, "version": "1"},
+            )
+            self._zc.register_service(self._info)
+            if self.on_peer_found or self.on_peer_lost:
+                self._browser = ServiceBrowser(
+                    self._zc, MDNS_SERVICE_TYPE, listener=_Listener(self)
+                )
+            log.info("mDNS announced: %s on %s:%d", self.peer_id, local_ip, self.port)
+        except Exception as exc:
+            log.warning("mDNS start failed: %s", exc)
+
+    def stop(self) -> None:
+        if not _HAS_ZEROCONF or self._zc is None:
+            return
+        try:
+            if self._info:
+                self._zc.unregister_service(self._info)
+            self._zc.close()
+        except Exception as exc:
+            log.debug("mDNS stop error: %s", exc)
+        self._zc = None
+
+
+class _Listener:
+    """Internal zeroconf service listener."""
+
+    def __init__(self, disc: MdnsDiscovery):
+        self._disc = disc
+
+    def add_service(self, zc, type_, name):
+        try:
+            info = zc.get_service_info(type_, name)
+            if info is None:
+                return
+            remote_peer_id = (info.properties or {}).get(b"peer_id", b"").decode()
+            if remote_peer_id == self._disc.peer_id:
+                return  # ignore self
+            addr = f"http://{socket.inet_ntoa(info.addresses[0])}:{info.port}"
+            if self._disc.on_peer_found:
+                self._disc.on_peer_found(remote_peer_id, addr)
+        except Exception as exc:
+            log.debug("mDNS add_service error: %s", exc)
+
+    def remove_service(self, zc, type_, name):
+        try:
+            # name is like "peer_id._meshnet._tcp.local."
+            peer_id = name.split(".")[0]
+            if self._disc.on_peer_lost:
+                self._disc.on_peer_lost(peer_id)
+        except Exception as exc:
+            log.debug("mDNS remove_service error: %s", exc)
+
+    def update_service(self, zc, type_, name):
+        pass

packages/p2p/meshnet_p2p/tls.py

@@ -0,0 +1,114 @@
+"""TLS certificate generation and fingerprint helpers for node-to-node comms."""
+
+from __future__ import annotations
+
+import datetime
+import hashlib
+import ipaddress
+import json
+import os
+import socket
+import ssl
+import stat
+from pathlib import Path
+
+_CERT_PATH = Path.home() / ".config" / "meshnet" / "node_cert.pem"
+_KEY_PATH = Path.home() / ".config" / "meshnet" / "node_key.pem"
+
+
+def generate_self_signed_cert(
+    cert_path: Path | None = None,
+    key_path: Path | None = None,
+    common_name: str | None = None,
+) -> tuple[Path, Path]:
+    """Generate a self-signed RSA-2048 cert valid for 10 years.
+
+    Returns (cert_path, key_path). Skips generation if both files already exist.
+    """
+    from cryptography import x509
+    from cryptography.hazmat.primitives import hashes, serialization
+    from cryptography.hazmat.primitives.asymmetric import rsa
+    from cryptography.x509.oid import NameOID
+
+    cert_p = cert_path or _CERT_PATH
+    key_p = key_path or _KEY_PATH
+
+    if cert_p.exists() and key_p.exists():
+        return cert_p, key_p
+
+    cert_p.parent.mkdir(parents=True, exist_ok=True)
+
+    key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
+    cn = common_name or socket.getfqdn()
+
+    subject = issuer = x509.Name([
+        x509.NameAttribute(NameOID.COMMON_NAME, cn),
+        x509.NameAttribute(NameOID.ORGANIZATION_NAME, "meshnet-node"),
+    ])
+
+    san_list: list = [x509.DNSName(cn)]
+    try:
+        san_list.append(x509.IPAddress(ipaddress.IPv4Address(socket.gethostbyname(cn))))
+    except (socket.gaierror, ValueError):
+        pass
+    san_list.append(x509.IPAddress(ipaddress.IPv4Address("127.0.0.1")))
+
+    cert = (
+        x509.CertificateBuilder()
+        .subject_name(subject)
+        .issuer_name(issuer)
+        .public_key(key.public_key())
+        .serial_number(x509.random_serial_number())
+        .not_valid_before(datetime.datetime.now(datetime.timezone.utc))
+        .not_valid_after(datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=365 * 10))
+        .add_extension(x509.SubjectAlternativeName(san_list), critical=False)
+        .sign(key, hashes.SHA256())
+    )
+
+    key_pem = key.private_bytes(
+        serialization.Encoding.PEM,
+        serialization.PrivateFormat.TraditionalOpenSSL,
+        serialization.NoEncryption(),
+    )
+    cert_pem = cert.public_bytes(serialization.Encoding.PEM)
+
+    key_p.write_bytes(key_pem)
+    cert_p.write_bytes(cert_pem)
+    try:
+        os.chmod(key_p, stat.S_IRUSR | stat.S_IWUSR)
+    except OSError:
+        pass
+
+    return cert_p, key_p
+
+
+def cert_fingerprint(cert_path: Path | None = None) -> str:
+    """Return sha256 fingerprint of the cert as 'sha256:<hex>'."""
+    from cryptography import x509
+    from cryptography.hazmat.primitives import hashes
+
+    p = cert_path or _CERT_PATH
+    cert = x509.load_pem_x509_certificate(p.read_bytes())
+    fp = cert.fingerprint(hashes.SHA256()).hex()
+    return f"sha256:{fp}"
+
+
+def make_server_ssl_context(
+    cert_path: Path | None = None,
+    key_path: Path | None = None,
+) -> ssl.SSLContext:
+    """Return an ssl.SSLContext for a server using our self-signed cert."""
+    cert_p = cert_path or _CERT_PATH
+    key_p = key_path or _KEY_PATH
+    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+    ctx.load_cert_chain(certfile=str(cert_p), keyfile=str(key_p))
+    return ctx
+
+
+def make_client_ssl_context(verify: bool = False) -> ssl.SSLContext:
+    """Return a client SSLContext. verify=False for self-signed TOFU connections."""
+    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
+    if not verify:
+        ctx.check_hostname = False
+        ctx.verify_mode = ssl.CERT_NONE
+    return ctx

packages/p2p/pyproject.toml

@@ -8,6 +8,14 @@ version = "0.1.0"
 description = "Distributed Inference Network gossip and shard swarm"
 requires-python = ">=3.10"
 
+dependencies = [
+    "cryptography>=41",
+    "websockets>=13",
+]
+
+[project.optional-dependencies]
+mdns = ["zeroconf>=0.131"]
+
 [tool.setuptools.packages.find]
 where = ["."]
 include = ["meshnet_p2p*"]

packages/p2p/relay_bootstrap.json

@@ -0,0 +1,10 @@
+{
+  "relays": [
+    {
+      "url": "ws://localhost:8765",
+      "cert_fingerprint": null,
+      "operator": "localhost-dev",
+      "note": "Local development relay — replace with team relay URL before production"
+    }
+  ]
+}

packages/relay/meshnet_relay/__init__.py

@@ -0,0 +1,3 @@
+"""meshnet-relay — NAT-traversal relay and gossip hub."""
+
+__version__ = "0.1.0"

packages/relay/meshnet_relay/cli.py

@@ -0,0 +1,60 @@
+"""meshnet-relay CLI entry point."""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import sys
+import time
+from pathlib import Path
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        prog="meshnet-relay",
+        description="Meshnet NAT-traversal relay and gossip hub",
+    )
+    parser.add_argument("--host", default="0.0.0.0", help="Interface to bind")
+    parser.add_argument("--port", type=int, default=8765, help="WebSocket port")
+    parser.add_argument("--cert", metavar="PATH", help="TLS certificate (PEM)")
+    parser.add_argument("--key", metavar="PATH", help="TLS private key (PEM)")
+    parser.add_argument("--max-peers", type=int, default=500, help="Max concurrent peers")
+    parser.add_argument("--log-level", default="INFO", choices=["DEBUG", "INFO", "WARNING", "ERROR"])
+
+    args = parser.parse_args()
+    logging.basicConfig(
+        level=getattr(logging, args.log_level),
+        format="%(asctime)s  %(levelname)-8s  %(name)s  %(message)s",
+    )
+
+    from .server import RelayServer
+
+    ssl_cert = Path(args.cert) if args.cert else None
+    ssl_key = Path(args.key) if args.key else None
+
+    server = RelayServer(
+        host=args.host,
+        port=args.port,
+        ssl_cert=ssl_cert,
+        ssl_key=ssl_key,
+        max_peers=args.max_peers,
+    )
+    port = server.start()
+    scheme = "wss" if ssl_cert else "ws"
+    print(f"meshnet-relay listening on {scheme}://{args.host}:{port}", flush=True)
+    print("  /ws             gossip PubSub", flush=True)
+    print("  /relay/<id>     circuit relay to peer", flush=True)
+    print("  /health         health check", flush=True)
+    print("  /v1/peers       peer list", flush=True)
+    print("Press Ctrl-C to stop.", flush=True)
+
+    try:
+        while True:
+            time.sleep(1)
+    except KeyboardInterrupt:
+        print("\nStopping relay…", flush=True)
+        server.stop()
+
+
+if __name__ == "__main__":
+    main()

packages/relay/meshnet_relay/peer_registry.py

@@ -0,0 +1,60 @@
+"""In-memory registry of connected gossip peers."""
+
+from __future__ import annotations
+
+import threading
+import time
+from dataclasses import dataclass, field
+from typing import Any
+
+
+@dataclass
+class PeerEntry:
+    peer_id: str
+    addr: str
+    ws: Any  # websockets.WebSocketServerProtocol
+    connected_at: float = field(default_factory=time.monotonic)
+    last_seen: float = field(default_factory=time.monotonic)
+
+
+class PeerRegistry:
+    def __init__(self):
+        self._peers: dict[str, PeerEntry] = {}
+        self._lock = threading.Lock()
+
+    def register(self, peer_id: str, addr: str, ws) -> None:
+        with self._lock:
+            self._peers[peer_id] = PeerEntry(peer_id=peer_id, addr=addr, ws=ws)
+
+    def unregister(self, peer_id: str) -> None:
+        with self._lock:
+            self._peers.pop(peer_id, None)
+
+    def touch(self, peer_id: str) -> None:
+        with self._lock:
+            if peer_id in self._peers:
+                self._peers[peer_id].last_seen = time.monotonic()
+
+    def get(self, peer_id: str) -> PeerEntry | None:
+        with self._lock:
+            return self._peers.get(peer_id)
+
+    def all_except(self, peer_id: str) -> list[PeerEntry]:
+        with self._lock:
+            return [e for pid, e in self._peers.items() if pid != peer_id]
+
+    def list_peers(self) -> list[dict]:
+        with self._lock:
+            return [
+                {
+                    "peer_id": e.peer_id,
+                    "addr": e.addr,
+                    "connected_at": e.connected_at,
+                    "last_seen": e.last_seen,
+                }
+                for e in self._peers.values()
+            ]
+
+    def __len__(self) -> int:
+        with self._lock:
+            return len(self._peers)

packages/relay/meshnet_relay/server.py

@@ -0,0 +1,281 @@
+"""Relay server — WebSocket gossip hub + circuit relay proxy.
+
+HTTP API (served via asyncio-based handler on same port):
+  GET /health              → {"status": "ok", "peers": N}
+  GET /v1/peers            → [{peer_id, addr, last_seen}]
+  POST /v1/gossip          → accept a gossip envelope, fan out to connected peers
+
+WebSocket endpoints:
+  ws[s]://host:port/ws              → gossip PubSub connection
+  ws[s]://host:port/relay/{peer_id} → circuit relay to that peer
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import threading
+import uuid
+from pathlib import Path
+
+from .peer_registry import PeerRegistry
+
+log = logging.getLogger(__name__)
+
+
+class RelayServer:
+    """Async WebSocket relay server that runs in a background thread.
+
+    Usage::
+
+        server = RelayServer(host="0.0.0.0", port=8765)
+        port = server.start()   # returns actual port
+        ...
+        server.stop()
+    """
+
+    def __init__(
+        self,
+        host: str = "0.0.0.0",
+        port: int = 8765,
+        ssl_cert: Path | None = None,
+        ssl_key: Path | None = None,
+        max_peers: int = 500,
+    ):
+        self.host = host
+        self.port = port
+        self.ssl_cert = ssl_cert
+        self.ssl_key = ssl_key
+        self.max_peers = max_peers
+
+        self._registry = PeerRegistry()
+        self._loop: asyncio.AbstractEventLoop | None = None
+        self._thread: threading.Thread | None = None
+        self._server = None
+        self._actual_port: int = port
+        self._ready = threading.Event()
+        self._running = False
+        self._stop_event: asyncio.Event | None = None
+        self._pending_rpc: dict[str, asyncio.Future] = {}
+
+    @property
+    def registry(self) -> PeerRegistry:
+        return self._registry
+
+    def start(self) -> int:
+        """Start server in background thread. Returns actual bound port."""
+        self._running = True
+        self._loop = asyncio.new_event_loop()
+        self._thread = threading.Thread(target=self._run, daemon=True, name="relay")
+        self._thread.start()
+        self._ready.wait(timeout=5)
+        return self._actual_port
+
+    def stop(self) -> None:
+        self._running = False
+        if self._loop and self._stop_event is not None:
+            self._loop.call_soon_threadsafe(self._stop_event.set)
+        if self._thread:
+            self._thread.join(timeout=3.0)
+
+    def _run(self) -> None:
+        asyncio.set_event_loop(self._loop)
+        self._loop.run_until_complete(self._serve())
+
+    async def _serve(self) -> None:
+        import websockets  # type: ignore[import]
+        import websockets.server  # type: ignore[import]
+
+        ssl_ctx = None
+        if self.ssl_cert and self.ssl_key:
+            import ssl
+            ssl_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+            ssl_ctx.load_cert_chain(str(self.ssl_cert), str(self.ssl_key))
+
+        server = await websockets.serve(
+            self._handle_connection,
+            self.host,
+            self.port,
+            ssl=ssl_ctx,
+        )
+        # Record actual port after bind
+        for sock in server.sockets or []:
+            self._actual_port = sock.getsockname()[1]
+            break
+
+        self._stop_event = asyncio.Event()
+        self._server = server
+        self._ready.set()
+        log.info("Relay listening on %s:%d", self.host, self._actual_port)
+
+        await self._stop_event.wait()
+        server.close()
+        await server.wait_closed()
+
+    async def _handle_connection(self, ws) -> None:
+        """Dispatch incoming WebSocket to gossip hub or circuit relay."""
+        try:
+            path = ws.request.path
+        except AttributeError:
+            path = getattr(ws, "path", "/ws")
+
+        if path.startswith("/relay/"):
+            peer_id = path[len("/relay/"):]
+            await self._handle_circuit_relay(ws, peer_id)
+        elif path.startswith("/rpc/"):
+            peer_id = path[len("/rpc/"):]
+            await self._handle_rpc(ws, peer_id)
+        elif path == "/health":
+            await ws.send(json.dumps({"status": "ok", "peers": len(self._registry)}))
+            await ws.close()
+        elif path == "/v1/peers":
+            await ws.send(json.dumps(self._registry.list_peers()))
+            await ws.close()
+        else:
+            await self._handle_gossip(ws)
+
+    async def _handle_gossip(self, ws) -> None:
+        """Accept a gossip peer connection, register it, and fan out messages."""
+        peer_id: str | None = None
+        peer_addr: str = ""
+
+        try:
+            async for raw in ws:
+                try:
+                    envelope = json.loads(raw)
+                except (json.JSONDecodeError, TypeError):
+                    continue
+
+                topic = envelope.get("topic", "")
+                from_peer = envelope.get("from_peer", "")
+
+                # Handle peer registration message
+                if topic == "peer-register":
+                    payload = envelope.get("payload", {})
+                    peer_id = payload.get("peer_id") or from_peer
+                    peer_addr = payload.get("addr", "")
+                    if len(self._registry) >= self.max_peers:
+                        await ws.close(1008, "relay at capacity")
+                        return
+                    self._registry.register(peer_id, peer_addr, ws)
+                    log.debug("Peer registered: %s", peer_id)
+                    # Send current peer list back
+                    await ws.send(json.dumps({
+                        "topic": "peer-list",
+                        "version": 1,
+                        "from_peer": "relay",
+                        "payload": {"peers": self._registry.list_peers()},
+                    }))
+                    continue
+
+                if topic == "relay-http-response":
+                    payload = envelope.get("payload", {})
+                    request_id = payload.get("request_id")
+                    fut = self._pending_rpc.pop(request_id, None)
+                    if fut is not None and not fut.done():
+                        fut.set_result(payload)
+                    continue
+
+                # Fan out to all other registered peers
+                if peer_id:
+                    self._registry.touch(peer_id)
+                fan_out_peers = self._registry.all_except(peer_id or "")
+                await _broadcast(raw, fan_out_peers)
+
+        except Exception as exc:
+            log.debug("Gossip connection error: %s", exc)
+        finally:
+            if peer_id:
+                self._registry.unregister(peer_id)
+                log.debug("Peer unregistered: %s", peer_id)
+
+    async def _handle_circuit_relay(self, ws_requester, target_peer_id: str) -> None:
+        """Proxy WebSocket traffic between ws_requester and target_peer_id's ws."""
+        target = self._registry.get(target_peer_id)
+        if target is None:
+            try:
+                await ws_requester.send(json.dumps({
+                    "error": f"peer {target_peer_id!r} not connected to relay"
+                }))
+                await ws_requester.close(1011, "target peer not found")
+            except Exception:
+                pass
+            return
+
+        log.debug("Circuit relay: ??? → %s", target_peer_id)
+
+        async def pipe(src, dst) -> None:
+            try:
+                async for msg in src:
+                    await dst.send(msg)
+            except Exception:
+                pass
+
+        await asyncio.gather(
+            pipe(ws_requester, target.ws),
+            pipe(target.ws, ws_requester),
+            return_exceptions=True,
+        )
+
+    async def _handle_rpc(self, ws_requester, target_peer_id: str) -> None:
+        """Send one HTTP-shaped request to a connected peer and relay its response."""
+        target = self._registry.get(target_peer_id)
+        if target is None:
+            await ws_requester.send(json.dumps({
+                "status": 503,
+                "headers": {"Content-Type": "application/json"},
+                "body": json.dumps({"error": f"peer {target_peer_id!r} not connected to relay"}),
+            }))
+            await ws_requester.close()
+            return
+
+        try:
+            raw = await asyncio.wait_for(ws_requester.recv(), timeout=30.0)
+            payload = json.loads(raw)
+        except Exception:
+            await ws_requester.close(1003, "invalid relay rpc request")
+            return
+
+        request_id = str(payload.get("request_id") or uuid.uuid4())
+        payload["request_id"] = request_id
+        payload["target_peer"] = target_peer_id
+        fut = self._loop.create_future() if self._loop is not None else asyncio.get_running_loop().create_future()
+        self._pending_rpc[request_id] = fut
+        try:
+            await target.ws.send(json.dumps({
+                "topic": "relay-http-request",
+                "version": 1,
+                "from_peer": "relay",
+                "payload": payload,
+            }))
+            response = await asyncio.wait_for(fut, timeout=310.0)
+            await ws_requester.send(json.dumps(response))
+        except asyncio.TimeoutError:
+            await ws_requester.send(json.dumps({
+                "request_id": request_id,
+                "status": 504,
+                "headers": {"Content-Type": "application/json"},
+                "body": json.dumps({"error": "relay rpc timed out"}),
+            }))
+        finally:
+            self._pending_rpc.pop(request_id, None)
+            await ws_requester.close()
+
+
+async def _broadcast(raw: str | bytes, peers: list) -> None:
+    """Send raw message to all peers; ignore individual send failures."""
+    if not peers:
+        return
+    import asyncio
+    await asyncio.gather(
+        *[_safe_send(p.ws, raw) for p in peers],
+        return_exceptions=True,
+    )
+
+
+async def _safe_send(ws, msg) -> None:
+    try:
+        await ws.send(msg)
+    except Exception:
+        pass

packages/relay/pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=64"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "meshnet-relay"
+version = "0.1.0"
+description = "Distributed Inference Network NAT-traversal relay and gossip hub"
+requires-python = ">=3.10"
+
+dependencies = [
+    "websockets>=13",
+]
+
+[project.scripts]
+meshnet-relay = "meshnet_relay.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["meshnet_relay*"]

packages/tracker/meshnet_tracker/cli.py

@@ -8,29 +8,51 @@ from .server import TrackerServer
 
 
 def main() -> None:
-    parser = argparse.ArgumentParser(
+    common = argparse.ArgumentParser(add_help=False)
-        prog="meshnet-tracker",
+    common.add_argument("--host", default="0.0.0.0", help="Host interface to listen on")
-        description="Distributed Inference Network node registry and route selection",
+    common.add_argument("--port", type=int, default=8080, help="Port to listen on")
-    )
+    common.add_argument(
-    subparsers = parser.add_subparsers(dest="command")
-
-    start_cmd = subparsers.add_parser("start", help="Start the tracker server")
-    start_cmd.add_argument("--host", default="127.0.0.1", help="Host interface to listen on")
-    start_cmd.add_argument("--port", type=int, default=8080, help="Port to listen on")
-    start_cmd.add_argument(
         "--heartbeat-timeout",
         type=float,
         default=30.0,
         help="Seconds before a node is removed from the registry after missed heartbeat",
     )
+    common.add_argument(
+        "--cluster-peers",
+        default="",
+        help="Comma-separated URLs of peer tracker nodes (enables Raft cluster mode)",
+    )
+    common.add_argument(
+        "--self-url",
+        default=None,
+        help="This tracker's own URL as seen by peers (auto-derived from --host/--port if omitted)",
+    )
+    common.add_argument(
+        "--relay-url",
+        default=None,
+        help="Public ws(s):// relay URL advertised to nodes, for example wss://ai.neuron.d-popov.com/ws",
+    )
+
+    parser = argparse.ArgumentParser(
+        prog="meshnet-tracker",
+        description="Distributed Inference Network node registry and route selection",
+        parents=[common],
+    )
+    subparsers = parser.add_subparsers(dest="command")
+
+    subparsers.add_parser("start", help="Start the tracker server", parents=[common])
 
     args = parser.parse_args()
 
-    if args.command == "start":
+    if args.command in {None, "start"}:
+        cluster_peers = [u.strip() for u in args.cluster_peers.split(",") if u.strip()]
         server = TrackerServer(
             host=args.host,
             port=args.port,
             heartbeat_timeout=args.heartbeat_timeout,
+            cluster_peers=cluster_peers or None,
+            cluster_self_url=args.self_url,
+            relay_url=args.relay_url,
         )
         port = server.start()
         print(f"meshnet-tracker listening on http://{args.host}:{port}", flush=True)

packages/tracker/meshnet_tracker/gossip.py

@@ -0,0 +1,88 @@
+"""CRDT gossip for node liveness heartbeats.
+
+Uses a last-write-wins (LWW) register per inference node: each tracker node
+keeps its own copy of {node_id → last_seen_monotonic_timestamp} and merges
+incoming gossip by taking the max per key.  This is eventually consistent —
+a heartbeat received by one tracker propagates to all others within a few
+gossip intervals.
+
+Monotonic timestamps are local-clock-relative; for cross-machine gossip the
+caller should use wall-clock seconds (time.time()).  The tracker converts
+monotonic to wall-clock when recording and back when comparing.
+"""
+
+from __future__ import annotations
+
+import json
+import random
+import threading
+import time
+import urllib.request
+
+
+class NodeGossip:
+    """LWW gossip table for inference-node heartbeat timestamps.
+
+    ``record(node_id)`` is called when a node sends a heartbeat to *this*
+    tracker.  ``merge(remote)`` is called when gossip arrives from a peer
+    tracker.  The table is periodically pushed to one random peer.
+    """
+
+    PUSH_INTERVAL = 3.0  # seconds between gossip pushes to a random peer
+
+    def __init__(self, peers: list[str]) -> None:
+        self.peers = list(peers)
+        # Maps node_id → wall-clock seconds of last known heartbeat.
+        self._table: dict[str, float] = {}
+        self._lock = threading.Lock()
+        self._running = False
+
+    def start(self) -> None:
+        self._running = True
+        threading.Thread(target=self._push_loop, daemon=True, name="gossip").start()
+
+    def stop(self) -> None:
+        self._running = False
+
+    def record(self, node_id: str, wall_ts: float | None = None) -> None:
+        """Record a heartbeat for *node_id* at *wall_ts* (default: now)."""
+        ts = wall_ts if wall_ts is not None else time.time()
+        with self._lock:
+            if ts > self._table.get(node_id, 0.0):
+                self._table[node_id] = ts
+
+    def merge(self, remote: dict[str, float]) -> None:
+        """Merge a gossip snapshot from a peer tracker (LWW per key)."""
+        with self._lock:
+            for node_id, ts in remote.items():
+                if ts > self._table.get(node_id, 0.0):
+                    self._table[node_id] = ts
+
+    def last_seen(self, node_id: str) -> float | None:
+        """Return wall-clock timestamp of last known heartbeat, or None."""
+        with self._lock:
+            return self._table.get(node_id)
+
+    def snapshot(self) -> dict[str, float]:
+        with self._lock:
+            return dict(self._table)
+
+    def _push_loop(self) -> None:
+        while self._running:
+            time.sleep(self.PUSH_INTERVAL)
+            if not self.peers:
+                continue
+            peer = random.choice(self.peers)
+            try:
+                snap = self.snapshot()
+                body = json.dumps(snap).encode()
+                req = urllib.request.Request(
+                    f"{peer}/v1/gossip",
+                    data=body,
+                    headers={"Content-Type": "application/json"},
+                    method="POST",
+                )
+                with urllib.request.urlopen(req, timeout=2.0) as r:
+                    r.read()
+            except Exception:
+                pass

packages/tracker/meshnet_tracker/raft.py

@@ -0,0 +1,372 @@
+"""Minimal Raft consensus for tracker shard assignments.
+
+Only shard-assignment commands (register/deregister) go through the log.
+Node liveness (heartbeats) is handled separately via CRDT gossip — these
+are high-frequency writes where eventual consistency is fine.
+
+Election timeout: random 150–300 ms (tight, suits in-process tests).
+Leader heartbeat interval: 50 ms.
+"""
+
+from __future__ import annotations
+
+import json
+import random
+import threading
+import time
+import urllib.error
+import urllib.request
+from dataclasses import dataclass, field
+from typing import Any, Callable
+
+
+@dataclass
+class LogEntry:
+    term: int
+    command: str   # "register" | "deregister"
+    payload: dict
+
+
+class RaftNode:
+    """Single Raft participant.
+
+    ``apply_fn(command, payload)`` is called (under no external lock) when an
+    entry is committed.  Implementors must apply the command atomically.
+    """
+
+    ELECTION_MIN = 0.15   # seconds
+    ELECTION_MAX = 0.30
+    HB_INTERVAL  = 0.05   # leader heartbeat interval
+
+    # Role constants
+    FOLLOWER  = "follower"
+    CANDIDATE = "candidate"
+    LEADER    = "leader"
+
+    def __init__(
+        self,
+        self_url: str,
+        peers: list[str],
+        apply_fn: Callable[[str, dict], None],
+    ) -> None:
+        self.self_url = self_url
+        self.peers = list(peers)
+        self._apply_fn = apply_fn
+
+        self._lock = threading.Lock()
+        self.role: str = self.FOLLOWER
+        self.current_term: int = 0
+        self.voted_for: str | None = None
+        self.log: list[LogEntry] = []
+        self.commit_index: int = -1
+        self.last_applied: int = -1
+        self.leader_url: str | None = None
+
+        # Leader bookkeeping per peer
+        self._next_index:  dict[str, int] = {}
+        self._match_index: dict[str, int] = {}
+
+        self._election_deadline: float = self._fresh_deadline()
+        self._running = False
+
+    # ------------------------------------------------------------------ start/stop
+
+    def start(self) -> None:
+        self._running = True
+        threading.Thread(target=self._tick_loop, daemon=True, name="raft-tick").start()
+
+    def stop(self) -> None:
+        self._running = False
+
+    # ------------------------------------------------------------------ public API
+
+    @property
+    def is_leader(self) -> bool:
+        with self._lock:
+            return self.role == self.LEADER
+
+    def leader(self) -> str | None:
+        with self._lock:
+            return self.leader_url
+
+    def status(self) -> dict:
+        with self._lock:
+            return {
+                "role": self.role,
+                "term": self.current_term,
+                "leader": self.leader_url,
+                "log_length": len(self.log),
+                "commit_index": self.commit_index,
+            }
+
+    def propose(self, command: str, payload: dict) -> bool:
+        """Leader: append and replicate an entry. Returns True when committed.
+
+        Blocks until majority replication or failure.  Must be called only on
+        the leader; returns False immediately if this node is not the leader.
+        """
+        with self._lock:
+            if self.role != self.LEADER:
+                return False
+            entry = LogEntry(self.current_term, command, payload)
+            self.log.append(entry)
+            entry_index = len(self.log) - 1
+            term = self.current_term
+
+        self._replicate_to_peers(term)
+
+        with self._lock:
+            return self.commit_index >= entry_index
+
+    # ------------------------------------------------------------------ RPC handlers
+
+    def handle_request_vote(self, req: dict) -> dict:
+        with self._lock:
+            term        = int(req["term"])
+            candidate   = req["candidate_url"]
+            last_li     = int(req["last_log_index"])
+            last_lt     = int(req["last_log_term"])
+
+            if term > self.current_term:
+                self._step_down(term)
+
+            if term < self.current_term:
+                return {"term": self.current_term, "vote_granted": False}
+
+            my_li = len(self.log) - 1
+            my_lt = self.log[-1].term if self.log else 0
+            log_ok = (last_lt > my_lt) or (last_lt == my_lt and last_li >= my_li)
+
+            if (self.voted_for is None or self.voted_for == candidate) and log_ok:
+                self.voted_for = candidate
+                self._reset_deadline()
+                return {"term": self.current_term, "vote_granted": True}
+            return {"term": self.current_term, "vote_granted": False}
+
+    def handle_append_entries(self, req: dict) -> dict:
+        with self._lock:
+            term         = int(req["term"])
+            leader_url   = req["leader_url"]
+            prev_li      = int(req["prev_log_index"])
+            prev_lt      = int(req["prev_log_term"])
+            entries_raw  = req.get("entries", [])
+            ldr_commit   = int(req["leader_commit"])
+
+            if term > self.current_term:
+                self._step_down(term)
+
+            if term < self.current_term:
+                return {"term": self.current_term, "success": False}
+
+            # Valid AppendEntries from current leader
+            self._reset_deadline()
+            self.leader_url = leader_url
+            if self.role == self.CANDIDATE:
+                self.role = self.FOLLOWER
+
+            # Consistency check on prev entry
+            if prev_li >= 0:
+                if prev_li >= len(self.log):
+                    return {"term": self.current_term, "success": False}
+                if self.log[prev_li].term != prev_lt:
+                    self.log = self.log[:prev_li]
+                    return {"term": self.current_term, "success": False}
+
+            # Append new entries (detect and overwrite conflicts)
+            for i, raw in enumerate(entries_raw):
+                idx = prev_li + 1 + i
+                entry = LogEntry(int(raw["term"]), raw["command"], raw["payload"])
+                if idx < len(self.log):
+                    if self.log[idx].term != entry.term:
+                        self.log = self.log[:idx]
+                        self.log.append(entry)
+                else:
+                    self.log.append(entry)
+
+            if ldr_commit > self.commit_index:
+                self.commit_index = min(ldr_commit, len(self.log) - 1)
+                self._apply_up_to_commit()
+
+            return {"term": self.current_term, "success": True}
+
+    # ------------------------------------------------------------------ internals
+
+    def _tick_loop(self) -> None:
+        while self._running:
+            time.sleep(0.01)
+            with self._lock:
+                role     = self.role
+                deadline = self._election_deadline
+
+            if role == self.LEADER:
+                self._send_heartbeats()
+                time.sleep(self.HB_INTERVAL)
+            elif time.monotonic() > deadline:
+                self._start_election()
+
+    def _send_heartbeats(self) -> None:
+        with self._lock:
+            term         = self.current_term
+            ldr_commit   = self.commit_index
+            log_snapshot = list(self.log)
+
+        for peer in self.peers:
+            try:
+                self._send_append_entries(peer, term, ldr_commit, log_snapshot)
+            except Exception:
+                pass
+
+    def _replicate_to_peers(self, term: int) -> None:
+        """Send AppendEntries to all peers and update commit_index on majority ack."""
+        with self._lock:
+            ldr_commit   = self.commit_index
+            log_snapshot = list(self.log)
+
+        acks = 1  # leader counts as 1
+        for peer in self.peers:
+            try:
+                ok = self._send_append_entries(peer, term, ldr_commit, log_snapshot)
+                if ok:
+                    acks += 1
+            except Exception:
+                pass
+
+        # Advance commit if majority replicated
+        with self._lock:
+            if self.role != self.LEADER or self.current_term != term:
+                return
+            majority_index = len(self.log) - 1
+            while majority_index > self.commit_index:
+                if self.log[majority_index].term == self.current_term:
+                    count = 1 + sum(
+                        1 for p in self.peers
+                        if self._match_index.get(p, -1) >= majority_index
+                    )
+                    if count > (len(self.peers) + 1) / 2:
+                        self.commit_index = majority_index
+                        self._apply_up_to_commit()
+                        break
+                majority_index -= 1
+
+    def _send_append_entries(
+        self, peer: str, term: int, ldr_commit: int, log_snapshot: list[LogEntry]
+    ) -> bool:
+        with self._lock:
+            next_idx = self._next_index.get(peer, len(log_snapshot))
+
+        prev_li = next_idx - 1
+        prev_lt = log_snapshot[prev_li].term if 0 <= prev_li < len(log_snapshot) else 0
+        entries = [
+            {"term": e.term, "command": e.command, "payload": e.payload}
+            for e in log_snapshot[next_idx:]
+        ]
+
+        body = json.dumps({
+            "term": term,
+            "leader_url": self.self_url,
+            "prev_log_index": prev_li,
+            "prev_log_term": prev_lt,
+            "entries": entries,
+            "leader_commit": ldr_commit,
+        }).encode()
+        req = urllib.request.Request(
+            f"{peer}/v1/raft/append",
+            data=body,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        with urllib.request.urlopen(req, timeout=0.15) as r:
+            resp = json.loads(r.read())
+
+        with self._lock:
+            if resp.get("success"):
+                new_match = len(log_snapshot) - 1
+                self._match_index[peer] = max(self._match_index.get(peer, -1), new_match)
+                self._next_index[peer]  = new_match + 1
+                return True
+            else:
+                if resp.get("term", 0) > self.current_term:
+                    self._step_down(resp["term"])
+                else:
+                    self._next_index[peer] = max(0, self._next_index.get(peer, 1) - 1)
+                return False
+
+    def _start_election(self) -> None:
+        with self._lock:
+            self.current_term += 1
+            self.role = self.CANDIDATE
+            self.voted_for = self.self_url
+            term  = self.current_term
+            my_li = len(self.log) - 1
+            my_lt = self.log[-1].term if self.log else 0
+            self._reset_deadline()
+
+        votes = 1
+        for peer in self.peers:
+            try:
+                body = json.dumps({
+                    "term": term,
+                    "candidate_url": self.self_url,
+                    "last_log_index": my_li,
+                    "last_log_term": my_lt,
+                }).encode()
+                req = urllib.request.Request(
+                    f"{peer}/v1/raft/vote",
+                    data=body,
+                    headers={"Content-Type": "application/json"},
+                    method="POST",
+                )
+                with urllib.request.urlopen(req, timeout=0.1) as r:
+                    resp = json.loads(r.read())
+                if resp.get("vote_granted"):
+                    votes += 1
+                elif resp.get("term", 0) > term:
+                    with self._lock:
+                        self._step_down(resp["term"])
+                    return
+            except Exception:
+                pass
+
+        with self._lock:
+            if self.role == self.CANDIDATE and self.current_term == term:
+                total = len(self.peers) + 1
+                if votes > total / 2:
+                    self._become_leader()
+                else:
+                    self.role = self.FOLLOWER
+                    self._reset_deadline()
+
+    def _become_leader(self) -> None:
+        """Must be called with _lock held."""
+        self.role = self.LEADER
+        self.leader_url = self.self_url
+        for peer in self.peers:
+            self._next_index[peer]  = len(self.log)
+            self._match_index[peer] = -1
+        print(f"[raft] {self.self_url} became leader (term {self.current_term})", flush=True)
+
+    def _step_down(self, new_term: int) -> None:
+        """Must be called with _lock held."""
+        self.current_term = new_term
+        self.role = self.FOLLOWER
+        self.voted_for = None
+        self._reset_deadline()
+
+    def _apply_up_to_commit(self) -> None:
+        """Must be called with _lock held."""
+        while self.last_applied < self.commit_index:
+            self.last_applied += 1
+            entry = self.log[self.last_applied]
+            try:
+                self._apply_fn(entry.command, entry.payload)
+            except Exception:
+                pass
+
+    def _reset_deadline(self) -> None:
+        self._election_deadline = self._fresh_deadline()
+
+    @staticmethod
+    def _fresh_deadline() -> float:
+        return time.monotonic() + random.uniform(
+            RaftNode.ELECTION_MIN, RaftNode.ELECTION_MAX
+        )

packages/tracker/pyproject.toml

@@ -8,6 +8,10 @@ version = "0.1.0"
 description = "Distributed Inference Network node registry and route selection"
 requires-python = ">=3.10"
 
+dependencies = [
+    "websockets>=13",
+]
+
 [project.scripts]
 meshnet-tracker = "meshnet_tracker.cli:main"
 

scripts/ralph_progress.py

@@ -152,7 +152,7 @@ def _story_meta(
         parts.append(f"worktree: {wt}")
 
     # summary -------------------------------------------------------------
-    notes = story.get("completionNotes", "").strip()
+    notes = (story.get("completionNotes") or "").strip()
     if not notes and wt:
         notes = _story_last_commit(sid)
     if notes:

scripts/test_lan_inference.py

@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+"""
+End-to-end LAN inference test for meshnet distributed inference.
+
+Sends 3 chat-completion requests to a meshnet node, validates OpenAI-format
+responses, and prints token counts + latency per request.
+
+Usage:
+    python scripts/test_lan_inference.py \\
+        --tracker http://192.168.1.10:8080 \\
+        --gateway http://192.168.1.10:8001
+
+Exit 0 on success, 1 on any failure.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+import urllib.error
+import urllib.parse
+import urllib.request
+
+
+PROMPTS = [
+    {"role": "user", "content": "What is 7 × 8? Answer in one word."},
+    {"role": "user", "content": "Name the capital of France in one word."},
+    {"role": "user", "content": "Complete the sequence: 1, 1, 2, 3, 5, ___. Answer in one word."},
+]
+
+MODEL = "microsoft/Phi-3-medium-128k-instruct"
+
+
+def _get(url: str, timeout: float = 10.0) -> dict:
+    with urllib.request.urlopen(url, timeout=timeout) as r:
+        return json.loads(r.read())
+
+
+def _post(url: str, payload: dict, timeout: float = 60.0) -> dict:
+    data = json.dumps(payload).encode()
+    req = urllib.request.Request(
+        url, data=data, headers={"Content-Type": "application/json"}, method="POST"
+    )
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.loads(r.read())
+
+
+def discover_gateway(tracker_url: str) -> str:
+    """Return the first tracker-mode node endpoint for MODEL."""
+    nodes = _get(f"{tracker_url}/v1/nodes", timeout=5.0)
+    if isinstance(nodes, dict):
+        nodes = list(nodes.values())
+    tracker_nodes = [
+        n for n in nodes
+        if n.get("tracker_mode") and (
+            n.get("hf_repo") == MODEL or n.get("model") == MODEL.split("/")[-1]
+        )
+    ]
+    if not tracker_nodes:
+        raise RuntimeError(
+            f"No tracker-mode nodes found for {MODEL!r}. "
+            "Is the first-shard node running and registered?"
+        )
+    endpoint: str = tracker_nodes[0]["endpoint"]
+    return endpoint.rstrip("/")
+
+
+def check_route(tracker_url: str, gateway_url: str) -> list[str]:
+    """Return the full inference route for MODEL."""
+    url = f"{tracker_url}/v1/route?model={urllib.parse.quote(MODEL)}"
+    try:
+        resp = _get(url, timeout=5.0)
+        return resp.get("route", [])
+    except Exception as exc:
+        print(f"  Warning: could not fetch route: {exc}", file=sys.stderr)
+        return [gateway_url]
+
+
+def run_inference(gateway_url: str, messages: list[dict]) -> tuple[str, int, float]:
+    """Send one chat-completion request. Returns (content, tokens, elapsed_s)."""
+    t0 = time.monotonic()
+    resp = _post(
+        f"{gateway_url}/v1/chat/completions",
+        {"model": MODEL, "messages": messages, "stream": False},
+        timeout=120.0,
+    )
+    elapsed = time.monotonic() - t0
+
+    choices = resp.get("choices")
+    if not choices:
+        raise ValueError(f"No choices in response: {resp}")
+    content: str = choices[0].get("message", {}).get("content", "")
+    if not isinstance(content, str):
+        raise TypeError(f"Expected string content, got {type(content)}: {content}")
+
+    usage = resp.get("usage", {})
+    tokens: int = usage.get("completion_tokens", len(content.split()))
+
+    return content, tokens, elapsed
+
+
+def main(argv: list[str] | None = None) -> int:
+    p = argparse.ArgumentParser(description=__doc__)
+    p.add_argument("--tracker", required=True, help="Tracker URL, e.g. http://192.168.1.10:8080")
+    p.add_argument(
+        "--gateway",
+        default=None,
+        help="Inference entry point URL. Auto-discovered from tracker if omitted.",
+    )
+    args = p.parse_args(argv)
+
+    tracker_url = args.tracker.rstrip("/")
+
+    print(f"Tracker: {tracker_url}")
+
+    # Resolve gateway
+    gateway_url = args.gateway.rstrip("/") if args.gateway else None
+    if gateway_url is None:
+        try:
+            gateway_url = discover_gateway(tracker_url)
+            print(f"Gateway (auto-discovered): {gateway_url}")
+        except Exception as exc:
+            print(f"ERROR: {exc}", file=sys.stderr)
+            return 1
+    else:
+        print(f"Gateway: {gateway_url}")
+
+    # Show route
+    route = check_route(tracker_url, gateway_url)
+    print(f"Route: {route}")
+    if len(route) < 2:
+        print("  Warning: only one node in route — is the second-shard node registered?")
+    print()
+
+    failures = 0
+    for i, msg in enumerate(PROMPTS, start=1):
+        print(f"[{i}] Q: {msg['content']}")
+        try:
+            content, tokens, elapsed = run_inference(gateway_url, [msg])
+            tps = tokens / elapsed if elapsed > 0 else 0.0
+            print(f"     A: {content.strip()}")
+            print(f"     {tokens} tokens  {elapsed:.2f}s  {tps:.1f} t/s")
+        except urllib.error.HTTPError as exc:
+            body = exc.read().decode(errors="replace")
+            print(f"     ERROR {exc.code}: {body}", file=sys.stderr)
+            failures += 1
+        except Exception as exc:
+            print(f"     ERROR: {exc}", file=sys.stderr)
+            failures += 1
+        print()
+
+    if failures == 0:
+        print(f"All {len(PROMPTS)} requests completed successfully.")
+        print("Exit code: 0")
+        return 0
+    else:
+        print(f"{failures}/{len(PROMPTS)} requests failed.", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tests/test_gossip_and_relay.py

@@ -0,0 +1,461 @@
+"""Tests for US-017: P2P gossip, relay node, and TLS infrastructure."""
+
+from __future__ import annotations
+
+import json
+import threading
+import time
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+
+# ---------------------------------------------------------------------------
+# identity tests
+# ---------------------------------------------------------------------------
+
+def test_load_or_create_identity_generates_peer_id(tmp_path):
+    from meshnet_p2p.identity import load_or_create_identity
+
+    identity = load_or_create_identity(tmp_path / "identity.json")
+    assert len(identity["peer_id"]) == 16
+    assert "public_key_pem" in identity
+    assert "private_key_pem" in identity
+
+
+def test_identity_is_stable_across_loads(tmp_path):
+    from meshnet_p2p.identity import load_or_create_identity
+
+    path = tmp_path / "identity.json"
+    first = load_or_create_identity(path)
+    second = load_or_create_identity(path)
+    assert first["peer_id"] == second["peer_id"]
+    assert first["public_key_pem"] == second["public_key_pem"]
+
+
+def test_identity_different_for_different_paths(tmp_path):
+    from meshnet_p2p.identity import load_or_create_identity
+
+    a = load_or_create_identity(tmp_path / "a.json")
+    b = load_or_create_identity(tmp_path / "b.json")
+    # Extremely unlikely to collide
+    assert a["peer_id"] != b["peer_id"]
+
+
+# ---------------------------------------------------------------------------
+# TLS / certificate tests
+# ---------------------------------------------------------------------------
+
+def test_generate_self_signed_cert_creates_files(tmp_path):
+    from meshnet_p2p.tls import generate_self_signed_cert
+
+    cert_p, key_p = generate_self_signed_cert(
+        cert_path=tmp_path / "cert.pem",
+        key_path=tmp_path / "key.pem",
+        common_name="localhost",
+    )
+    assert cert_p.exists()
+    assert key_p.exists()
+    assert cert_p.stat().st_size > 100
+    assert key_p.stat().st_size > 100
+
+
+def test_generate_self_signed_cert_is_idempotent(tmp_path):
+    from meshnet_p2p.tls import generate_self_signed_cert
+
+    args = dict(cert_path=tmp_path / "cert.pem", key_path=tmp_path / "key.pem", common_name="test")
+    generate_self_signed_cert(**args)
+    mtime1 = (tmp_path / "cert.pem").stat().st_mtime
+
+    generate_self_signed_cert(**args)
+    mtime2 = (tmp_path / "cert.pem").stat().st_mtime
+    assert mtime1 == mtime2  # file not regenerated
+
+
+def test_cert_fingerprint_returns_sha256_prefix(tmp_path):
+    from meshnet_p2p.tls import generate_self_signed_cert, cert_fingerprint
+
+    cert_p, key_p = generate_self_signed_cert(
+        cert_path=tmp_path / "cert.pem",
+        key_path=tmp_path / "key.pem",
+        common_name="test",
+    )
+    fp = cert_fingerprint(cert_p)
+    assert fp.startswith("sha256:")
+    assert len(fp) == len("sha256:") + 64  # 32 bytes hex = 64 chars
+
+
+def test_make_server_ssl_context_loads_cert(tmp_path):
+    import ssl
+    from meshnet_p2p.tls import generate_self_signed_cert, make_server_ssl_context
+
+    cert_p, key_p = generate_self_signed_cert(
+        cert_path=tmp_path / "cert.pem",
+        key_path=tmp_path / "key.pem",
+        common_name="test",
+    )
+    ctx = make_server_ssl_context(cert_p, key_p)
+    assert isinstance(ctx, ssl.SSLContext)
+
+
+# ---------------------------------------------------------------------------
+# PeerRegistry tests
+# ---------------------------------------------------------------------------
+
+def test_peer_registry_register_and_list():
+    from meshnet_relay.peer_registry import PeerRegistry
+
+    reg = PeerRegistry()
+    ws_mock = MagicMock()
+    reg.register("peer1", "http://1.2.3.4:8001", ws_mock)
+
+    assert len(reg) == 1
+    peers = reg.list_peers()
+    assert len(peers) == 1
+    assert peers[0]["peer_id"] == "peer1"
+
+
+def test_peer_registry_all_except_excludes_sender():
+    from meshnet_relay.peer_registry import PeerRegistry
+
+    reg = PeerRegistry()
+    reg.register("a", "http://a:8001", MagicMock())
+    reg.register("b", "http://b:8001", MagicMock())
+    reg.register("c", "http://c:8001", MagicMock())
+
+    others = reg.all_except("a")
+    assert len(others) == 2
+    assert all(e.peer_id != "a" for e in others)
+
+
+def test_peer_registry_unregister_removes_peer():
+    from meshnet_relay.peer_registry import PeerRegistry
+
+    reg = PeerRegistry()
+    reg.register("x", "http://x:8001", MagicMock())
+    reg.unregister("x")
+    assert len(reg) == 0
+    assert reg.get("x") is None
+
+
+# ---------------------------------------------------------------------------
+# GossipClient + RelayServer integration test
+# ---------------------------------------------------------------------------
+
+def _start_relay(host="127.0.0.1", port=0):
+    from meshnet_relay.server import RelayServer
+    server = RelayServer(host=host, port=port)
+    actual_port = server.start()
+    return server, actual_port
+
+
+def test_gossip_fanout_through_relay():
+    """Node B publishes node-join; node A receives it within 2 seconds."""
+    from meshnet_p2p.gossip import GossipClient, TOPIC_NODE_JOIN
+
+    relay, port = _start_relay()
+    relay_url = f"ws://127.0.0.1:{port}/ws"
+
+    client_a = GossipClient(relay_url=relay_url, peer_id="peer_a")
+    client_b = GossipClient(relay_url=relay_url, peer_id="peer_b")
+
+    received = []
+    client_a.subscribe(TOPIC_NODE_JOIN, lambda env: received.append(env))
+
+    client_a.start()
+    client_b.start()
+
+    assert client_a.wait_connected(timeout=5), "client_a failed to connect to relay"
+    assert client_b.wait_connected(timeout=5), "client_b failed to connect to relay"
+
+    # Give both peers time to register with relay
+    time.sleep(0.2)
+
+    client_b.publish(TOPIC_NODE_JOIN, {"addr": "http://192.168.1.10:8001", "peer_id": "peer_b"})
+
+    deadline = time.monotonic() + 2.0
+    while time.monotonic() < deadline and not received:
+        time.sleep(0.05)
+
+    client_a.stop()
+    client_b.stop()
+    relay.stop()
+
+    assert received, "client_a did not receive node-join message from client_b"
+    assert received[0]["topic"] == TOPIC_NODE_JOIN
+    assert received[0]["from_peer"] == "peer_b"
+
+
+def test_gossip_dedup_prevents_processing_duplicate_message_ids():
+    """A message with a duplicate msg_id is only processed once."""
+    from meshnet_p2p.gossip import GossipClient, TOPIC_NODE_JOIN
+
+    relay, port = _start_relay()
+    relay_url = f"ws://127.0.0.1:{port}/ws"
+
+    client_a = GossipClient(relay_url=relay_url, peer_id="peer_a2")
+    client_b = GossipClient(relay_url=relay_url, peer_id="peer_b2")
+
+    received = []
+    client_a.subscribe(TOPIC_NODE_JOIN, lambda env: received.append(env))
+
+    client_a.start()
+    client_b.start()
+    client_a.wait_connected(5)
+    client_b.wait_connected(5)
+    time.sleep(0.2)
+
+    # Publish once
+    client_b.publish(TOPIC_NODE_JOIN, {"test": "dedup"})
+    time.sleep(0.5)
+
+    count = len(received)
+
+    client_a.stop()
+    client_b.stop()
+    relay.stop()
+
+    # Should have received exactly one message (not duplicated)
+    assert count <= 1
+
+
+def test_relay_server_peer_list_grows_on_connect():
+    """Relay registry grows when clients connect."""
+    from meshnet_p2p.gossip import GossipClient
+
+    relay, port = _start_relay()
+    relay_url = f"ws://127.0.0.1:{port}/ws"
+
+    client = GossipClient(relay_url=relay_url, peer_id="solo_peer")
+    client.start()
+    client.wait_connected(5)
+    time.sleep(0.3)  # let peer-register message process
+
+    peer_count = len(relay.registry)
+    client.stop()
+    relay.stop()
+
+    assert peer_count >= 1
+
+
+def test_relay_circuit_relay_proxies_message():
+    """A node behind NAT (client_a) receives a message via circuit relay from client_b."""
+    import websockets.sync.client  # type: ignore[import]
+    from meshnet_relay.server import RelayServer
+
+    relay = RelayServer(host="127.0.0.1", port=0)
+    port = relay.start()
+
+    # client_a connects to gossip hub and registers
+    received_via_relay = []
+    ready = threading.Event()
+
+    def run_nat_client():
+        import websockets.sync.client as wsc
+        with wsc.connect(f"ws://127.0.0.1:{port}/ws") as ws:
+            ws.send(json.dumps({
+                "topic": "peer-register",
+                "version": 1,
+                "from_peer": "nat_peer",
+                "msg_id": "reg-001",
+                "timestamp": "2026-06-29T00:00:00Z",
+                "ttl": 3,
+                "payload": {"peer_id": "nat_peer", "addr": ""},
+            }))
+            # consume peer-list response
+            ws.recv()
+            ready.set()
+            # wait for a relayed message
+            try:
+                msg = ws.recv(timeout=3)
+                received_via_relay.append(json.loads(msg))
+            except Exception:
+                pass
+
+    nat_thread = threading.Thread(target=run_nat_client, daemon=True)
+    nat_thread.start()
+    ready.wait(timeout=5)
+    time.sleep(0.1)  # ensure peer is in registry
+
+    # client_b connects via circuit relay path
+    def send_via_relay():
+        try:
+            import websockets.sync.client as wsc
+            with wsc.connect(f"ws://127.0.0.1:{port}/relay/nat_peer") as ws:
+                ws.send(json.dumps({"topic": "direct-relay-test", "payload": {"hi": "there"}}))
+                time.sleep(0.3)
+        except Exception:
+            pass
+
+    relay_thread = threading.Thread(target=send_via_relay, daemon=True)
+    relay_thread.start()
+    relay_thread.join(timeout=3)
+    nat_thread.join(timeout=3)
+    relay.stop()
+
+    assert received_via_relay, "NAT'd peer did not receive message via circuit relay"
+
+
+def test_relay_rpc_round_trips_http_request_to_peer():
+    """Relay /rpc/<peer> sends one HTTP-shaped request to a connected peer."""
+    import websockets.sync.client as wsc  # type: ignore[import]
+    from meshnet_relay.server import RelayServer
+
+    relay = RelayServer(host="127.0.0.1", port=0)
+    port = relay.start()
+    ready = threading.Event()
+
+    def run_peer():
+        with wsc.connect(f"ws://127.0.0.1:{port}/ws") as ws:
+            ws.send(json.dumps({
+                "topic": "peer-register",
+                "version": 1,
+                "from_peer": "rpc_peer",
+                "msg_id": "rpc-reg-001",
+                "payload": {"peer_id": "rpc_peer", "addr": ""},
+            }))
+            ws.recv()
+            ready.set()
+            envelope = json.loads(ws.recv(timeout=3))
+            payload = envelope["payload"]
+            ws.send(json.dumps({
+                "topic": "relay-http-response",
+                "version": 1,
+                "from_peer": "rpc_peer",
+                "payload": {
+                    "request_id": payload["request_id"],
+                    "status": 200,
+                    "headers": {"Content-Type": "application/json"},
+                    "body": json.dumps({"ok": True, "path": payload["path"]}),
+                },
+            }))
+
+    peer_thread = threading.Thread(target=run_peer, daemon=True)
+    peer_thread.start()
+    assert ready.wait(timeout=5)
+
+    with wsc.connect(f"ws://127.0.0.1:{port}/rpc/rpc_peer") as ws:
+        ws.send(json.dumps({
+            "request_id": "req-1",
+            "method": "POST",
+            "path": "/v1/chat/completions",
+            "headers": {"Content-Type": "application/json"},
+            "body": "{}",
+        }))
+        response = json.loads(ws.recv(timeout=5))
+
+    relay.stop()
+    peer_thread.join(timeout=3)
+
+    assert response["status"] == 200
+    assert json.loads(response["body"]) == {"ok": True, "path": "/v1/chat/completions"}
+
+
+# ---------------------------------------------------------------------------
+# Tracker gossip fields tests
+# ---------------------------------------------------------------------------
+
+def _start_tracker_and_register(extra_fields: dict) -> dict:
+    """Helper: start tracker, register node with extra gossip fields, return response."""
+    import http.server
+    import json as _json
+    import urllib.request
+
+    from meshnet_tracker.server import TrackerServer
+
+    tracker = TrackerServer(host="127.0.0.1", port=0)
+    port = tracker.start()
+    url = f"http://127.0.0.1:{port}"
+
+    payload = {
+        "endpoint": f"http://127.0.0.1:8001",
+        "shard_start": 0,
+        "shard_end": 7,
+        "model": "stub-model",
+        "hardware_profile": {},
+        "score": 1.0,
+        **extra_fields,
+    }
+    data = _json.dumps(payload).encode()
+    req = urllib.request.Request(
+        f"{url}/v1/nodes/register",
+        data=data,
+        headers={"Content-Type": "application/json"},
+        method="POST",
+    )
+    with urllib.request.urlopen(req, timeout=5) as r:
+        resp = _json.loads(r.read())
+
+    tracker.stop()
+    return resp
+
+
+def test_tracker_accepts_relay_addr_in_registration():
+    resp = _start_tracker_and_register({
+        "relay_addr": "ws://relay.meshnet.ai:8765/relay/abc123",
+        "cert_fingerprint": "sha256:deadbeef",
+        "peer_id": "abc123def456ef01",
+    })
+    assert "node_id" in resp
+
+
+def test_tracker_accepts_registration_without_gossip_fields():
+    """Existing registrations without P2P fields still work."""
+    resp = _start_tracker_and_register({})
+    assert "node_id" in resp
+
+
+def test_tracker_network_map_exposes_relay_and_registered_peer():
+    import json as _json
+    import urllib.request
+
+    from meshnet_tracker.server import TrackerServer
+
+    tracker = TrackerServer(host="127.0.0.1", port=0, relay_url="wss://ai.neuron.d-popov.com/ws")
+    port = tracker.start()
+    url = f"http://127.0.0.1:{port}"
+    try:
+        req = urllib.request.Request(
+            f"{url}/v1/nodes/register",
+            data=_json.dumps({
+                "endpoint": "http://192.0.2.10:7000",
+                "model": "stub-model",
+                "shard_start": 0,
+                "shard_end": 31,
+                "relay_addr": "wss://ai.neuron.d-popov.com/rpc/peer123",
+                "peer_id": "peer123",
+            }).encode(),
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        with urllib.request.urlopen(req, timeout=5):
+            pass
+        with urllib.request.urlopen(f"{url}/v1/network/map", timeout=5) as resp:
+            body = _json.loads(resp.read())
+    finally:
+        tracker.stop()
+
+    assert body["relay_url"] == "wss://ai.neuron.d-popov.com/ws"
+    assert body["nodes"][0]["relay_addr"] == "wss://ai.neuron.d-popov.com/rpc/peer123"
+    assert body["nodes"][0]["peer_id"] == "peer123"
+
+
+# ---------------------------------------------------------------------------
+# mDNS (no-op without zeroconf installed)
+# ---------------------------------------------------------------------------
+
+def test_mdns_discovery_is_available_flag():
+    from meshnet_p2p.mdns import MdnsDiscovery
+
+    disc = MdnsDiscovery(peer_id="test", port=8001)
+    # is_available() should be bool regardless of zeroconf install status
+    assert isinstance(disc.is_available(), bool)
+
+
+def test_mdns_start_and_stop_without_zeroconf(monkeypatch):
+    from meshnet_p2p import mdns as mdns_mod
+    monkeypatch.setattr(mdns_mod, "_HAS_ZEROCONF", False)
+    from meshnet_p2p.mdns import MdnsDiscovery
+
+    disc = MdnsDiscovery(peer_id="x", port=8001)
+    disc.start()  # should not raise
+    disc.stop()   # should not raise

tests/test_mining_cli.py

@@ -0,0 +1,467 @@
+"""Tests for US-016: mining-style node startup CLI + live dashboard."""
+
+from __future__ import annotations
+
+import json
+import socket
+import sys
+import types
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+
+# ---------------------------------------------------------------------------
+# model_catalog tests
+# ---------------------------------------------------------------------------
+
+def test_curated_models_list_is_non_empty():
+    from meshnet_node.model_catalog import CURATED_MODELS
+    assert len(CURATED_MODELS) >= 5
+
+
+def test_model_preset_vram_for_quant():
+    from meshnet_node.model_catalog import CURATED_MODELS
+
+    m = next(m for m in CURATED_MODELS if "Llama-3-70B" in m.name)
+    assert m.vram_for_quant("nf4") == m.vram_nf4
+    assert m.vram_for_quant("int8") == m.vram_int8
+    assert m.vram_for_quant("bf16") == m.vram_bf16
+    assert m.vram_for_quant("bfloat16") == m.vram_bf16  # alias
+
+
+def test_model_preset_fits_vram():
+    from meshnet_node.model_catalog import CURATED_MODELS
+
+    small = next(m for m in CURATED_MODELS if m.vram_nf4 < 10)
+    assert small.fits_vram(small.vram_nf4, "nf4")
+    assert not small.fits_vram(small.vram_nf4 - 1, "nf4")
+
+
+def test_recommended_quant_respects_vram():
+    from meshnet_node.model_catalog import CURATED_MODELS
+
+    m = next(m for m in CURATED_MODELS if "Llama-3-70B" in m.name)
+    # nf4=18, int8=40, bf16=140
+    assert m.recommended_quant(200) == "bf16"
+    assert m.recommended_quant(50) == "int8"
+    assert m.recommended_quant(20) == "nf4"
+    assert m.recommended_quant(5) is None
+
+
+def test_models_with_insufficient_vram_are_marked(monkeypatch):
+    from meshnet_node import wizard as wiz
+
+    # Simulate 6 GB GPU
+    gpus = [{"index": 0, "name": "RTX 3060", "vram_gb": 6.0, "backend": "cuda"}]
+    monkeypatch.setattr(wiz, "_detect_gpus", lambda: gpus)
+
+    # Phi-3 at NF4 needs 4 GB — should fit; Llama-3-70B at NF4 needs 18 GB — should not
+    from meshnet_node.model_catalog import CURATED_MODELS
+
+    phi = next(m for m in CURATED_MODELS if "Phi-3" in m.name)
+    llama = next(m for m in CURATED_MODELS if "Llama-3-70B" in m.name)
+
+    assert phi.fits_vram(6.0, "nf4")
+    assert not llama.fits_vram(6.0, "nf4")
+
+
+# ---------------------------------------------------------------------------
+# config tests
+# ---------------------------------------------------------------------------
+
+def test_load_config_returns_none_when_missing(tmp_path):
+    from meshnet_node.config import load_config
+    assert load_config(tmp_path / "nonexistent.json") is None
+
+
+def test_save_and_load_config_roundtrip(tmp_path):
+    from meshnet_node.config import save_config, load_config
+
+    cfg = {"model_hf_repo": "test/model", "quantization": "nf4", "tracker_url": "http://localhost:8080"}
+    cfg_path = tmp_path / "config.json"
+    save_config(cfg, cfg_path)
+
+    loaded = load_config(cfg_path)
+    assert loaded == cfg
+
+
+def test_save_config_creates_parent_dirs(tmp_path):
+    from meshnet_node.config import save_config, load_config
+
+    nested = tmp_path / "deep" / "nested" / "config.json"
+    save_config({"x": 1}, nested)
+    assert nested.exists()
+    assert load_config(nested) == {"x": 1}
+
+
+def test_merge_cli_overrides_applies_non_none_values():
+    from meshnet_node.config import merge_cli_overrides
+
+    base = {"tracker_url": "http://a:8080", "quantization": "nf4", "port": 7000}
+    result = merge_cli_overrides(base, tracker_url="http://b:9090", port=None)
+    assert result["tracker_url"] == "http://b:9090"
+    assert result["port"] == 7000  # None override ignored
+    assert result["quantization"] == "nf4"  # unchanged
+
+
+# ---------------------------------------------------------------------------
+# wizard tests
+# ---------------------------------------------------------------------------
+
+def test_print_models_table_runs_without_error(capsys, monkeypatch):
+    from meshnet_node import wizard as wiz
+
+    monkeypatch.setattr(wiz, "_detect_gpus", lambda: [{"index": 0, "name": "GPU", "vram_gb": 24.0, "backend": "cuda"}])
+    wiz.print_models_table()
+    out = capsys.readouterr().out
+    assert "Llama" in out or "Qwen" in out or "Phi" in out
+
+
+def test_wizard_writes_config_on_happy_path(tmp_path, monkeypatch):
+    from meshnet_node import wizard as wiz
+    from meshnet_node.config import load_config, save_config
+
+    # Fake GPU
+    gpus = [{"index": 0, "name": "RTX 4090", "vram_gb": 24.0, "backend": "cuda"}]
+    monkeypatch.setattr(wiz, "_detect_gpus", lambda: gpus)
+    # Tracker not reachable (stub)
+    monkeypatch.setattr(wiz, "_ping_tracker", lambda url: False)
+
+    # Simulate user selecting model 1 (Qwen2.5-0.5B), quant 1 (nf4), default dir, default tracker, default wallet
+    inputs = iter([
+        "1",          # pick Qwen2.5-0.5B-Instruct (index 1 in CURATED_MODELS)
+        "1",          # quant NF4
+        str(tmp_path / "models"),  # download dir
+        "http://localhost:8080",   # tracker
+        str(tmp_path / "wallet.json"),  # wallet
+    ])
+    monkeypatch.setattr("builtins.input", lambda prompt="": next(inputs))
+
+    cfg = wiz.run_wizard(config_path_override=tmp_path / "config.json")
+    assert cfg["model_hf_repo"] == "Qwen/Qwen2.5-0.5B-Instruct"
+    assert cfg["quantization"] == "nf4"
+    assert "download_dir" in cfg
+    assert cfg["tracker_url"] == "http://localhost:8080"
+
+
+def test_wizard_raises_keyboard_interrupt_on_ctrl_c(monkeypatch):
+    from meshnet_node import wizard as wiz
+
+    gpus = [{"index": 0, "name": "RTX 4090", "vram_gb": 24.0, "backend": "cuda"}]
+    monkeypatch.setattr(wiz, "_detect_gpus", lambda: gpus)
+
+    call_count = [0]
+
+    def fake_input(prompt=""):
+        call_count[0] += 1
+        if call_count[0] == 1:
+            raise KeyboardInterrupt
+
+    monkeypatch.setattr("builtins.input", fake_input)
+
+    import pytest
+    with pytest.raises(KeyboardInterrupt):
+        wiz.run_wizard()
+
+
+# ---------------------------------------------------------------------------
+# dashboard tests
+# ---------------------------------------------------------------------------
+
+def test_is_interactive_tty_false_when_not_tty(monkeypatch):
+    from meshnet_node import dashboard as dash
+
+    monkeypatch.setattr(sys.stdout, "isatty", lambda: False)
+    assert not dash.is_interactive_tty()
+
+
+def test_dashboard_plain_fallback_on_keyboard_interrupt(monkeypatch):
+    """Plain loop exits cleanly when Ctrl-C is raised."""
+    from meshnet_node import dashboard as dash
+
+    node = MagicMock()
+    node.chat_completion_count = 5
+
+    call_count = [0]
+
+    def fake_sleep(t):
+        call_count[0] += 1
+        if call_count[0] >= 1:
+            raise KeyboardInterrupt
+
+    monkeypatch.setattr(dash.time, "sleep", fake_sleep)
+    monkeypatch.setattr(dash, "_gpu_stats", lambda: [])
+    monkeypatch.setattr(sys.stdout, "isatty", lambda: False)
+
+    cfg = {"model_name": "test-model", "quantization": "nf4"}
+    # Should not raise
+    dash.run_dashboard(node, cfg, start_time=dash.time.monotonic())
+
+
+def test_ema_updates_correctly():
+    from meshnet_node.dashboard import _EMA
+
+    ema = _EMA(alpha=1.0)  # alpha=1.0 → always takes latest sample
+    ema.update(10.0)
+    assert ema.value == 10.0
+    ema.update(20.0)
+    assert ema.value == 20.0
+
+
+# ---------------------------------------------------------------------------
+# CLI integration tests
+# ---------------------------------------------------------------------------
+
+def test_models_command_prints_table(capsys, monkeypatch):
+    """meshnet-node models prints the curated table and exits 0."""
+    from meshnet_node import wizard as wiz
+
+    monkeypatch.setattr(wiz, "_detect_gpus", lambda: [])
+
+    from meshnet_node.cli import main
+    monkeypatch.setattr(sys, "argv", ["meshnet-node", "models"])
+
+    try:
+        main()
+    except SystemExit as exc:
+        assert exc.code == 0
+
+    out = capsys.readouterr().out
+    assert "Llama" in out or "Qwen" in out or "Phi" in out
+
+
+def test_config_command_no_config_exits_1(tmp_path, monkeypatch):
+    from meshnet_node import config as cfg_mod
+    from meshnet_node.cli import main
+
+    monkeypatch.setattr(cfg_mod, "_DEFAULT_CONFIG_FILE", tmp_path / "nonexistent.json")
+    monkeypatch.setattr(sys, "argv", ["meshnet-node", "config"])
+
+    with patch("meshnet_node.config.config_path", return_value=tmp_path / "nonexistent.json"):
+        try:
+            main()
+        except SystemExit as exc:
+            assert exc.code == 1
+
+
+def test_config_command_prints_saved_config(tmp_path, monkeypatch, capsys):
+    from meshnet_node import config as cfg_mod
+    from meshnet_node.config import save_config
+    from meshnet_node.cli import main
+
+    saved = {"model_hf_repo": "meta-llama/Meta-Llama-3-70B-Instruct", "quantization": "nf4"}
+    cfg_file = tmp_path / "config.json"
+    save_config(saved, cfg_file)
+
+    monkeypatch.setattr(sys, "argv", ["meshnet-node", "config"])
+
+    with patch("meshnet_node.config.config_path", return_value=cfg_file):
+        with patch("meshnet_node.config.load_config", return_value=saved):
+            try:
+                main()
+            except SystemExit as exc:
+                assert exc.code == 0
+
+    out = capsys.readouterr().out
+    data = json.loads(out.split("\n", 1)[1])  # skip the "Config: ..." header line
+    assert data["model_hf_repo"] == saved["model_hf_repo"]
+
+
+def test_detect_num_layers_returns_catalog_value_without_network(monkeypatch):
+    """detect_num_layers uses the curated catalog first — no network call."""
+    from meshnet_node.model_catalog import detect_num_layers
+
+    # Qwen2.5-0.5B is in the catalog with 24 layers
+    layers = detect_num_layers("Qwen/Qwen2.5-0.5B-Instruct")
+    assert layers == 24
+
+
+def test_detect_num_layers_returns_none_on_error(monkeypatch):
+    from meshnet_node.model_catalog import detect_num_layers
+
+    # Monkeypatch AutoConfig to raise
+    import meshnet_node.model_catalog as cat
+    monkeypatch.setattr(cat, "detect_num_layers", lambda repo: None if "bad" in repo else detect_num_layers(repo))
+    assert cat.detect_num_layers("bad/repo") is None
+
+
+def test_startup_auto_detects_shard_range(monkeypatch, tmp_path):
+    """When shard_start/end are None, startup reads layer count from catalog."""
+    from meshnet_node import startup as su
+    from meshnet_node.model_catalog import detect_num_layers
+
+    calls = []
+
+    def fake_detect(repo):
+        calls.append(repo)
+        return 24  # Qwen2.5-0.5B
+
+    monkeypatch.setattr(su, "_detect_num_layers", fake_detect)
+
+    # Fake hardware detection
+    monkeypatch.setattr(su, "detect_hardware", lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0})
+
+    # Fake wallet
+    monkeypatch.setattr(su, "load_or_create_wallet", lambda **kw: (None, None, "fake-wallet"))
+
+    # Fake TorchNodeServer
+    class FakeNode:
+        chat_completion_count = 0
+        def start(self): return 9999
+        def stop(self): pass
+
+    import meshnet_node.startup as su2
+    monkeypatch.setattr(su2, "TorchNodeServer", lambda **kw: FakeNode())
+
+    node = su.run_startup(
+        tracker_url="http://localhost:8080",
+        model_id="Qwen/Qwen2.5-0.5B-Instruct",
+        # shard_start and shard_end intentionally omitted
+        quantization="bfloat16",
+        host="127.0.0.1",
+    )
+    assert calls == ["Qwen/Qwen2.5-0.5B-Instruct"]
+    assert isinstance(node, FakeNode)
+
+
+def test_legacy_start_subcommand_accepted(monkeypatch):
+    """meshnet-node start --tracker http://... does not crash on arg parsing."""
+    from meshnet_node.cli import main
+
+    def fake_run_startup(*args, **kwargs):
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node", "start",
+        "--tracker", "http://localhost:8080",
+        "--model", "stub-model",
+        "--port", "0",
+    ])
+
+    raised = []
+
+    def fake_sleep(t):
+        raise KeyboardInterrupt
+
+    with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+        with patch("time.sleep", side_effect=fake_sleep):
+            try:
+                main()
+            except SystemExit as exc:
+                raised.append(exc.code)
+
+    # Exited (either 0 or via KeyboardInterrupt caught in _cmd_start)
+    # The important thing is no unhandled exception from arg parsing
+
+
+def test_legacy_start_treats_repo_model_as_model_id(monkeypatch):
+    """`meshnet-node start --model org/repo` enters the HF model startup path."""
+    from meshnet_node.cli import main
+
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node", "start",
+        "--tracker", "http://192.168.0.179:8081",
+        "--model", "Qwen/Qwen2.5-0.5B-Instruct",
+        "--port", "0",
+    ])
+
+    with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+        with patch("time.sleep", side_effect=KeyboardInterrupt):
+            try:
+                main()
+            except SystemExit as exc:
+                assert exc.code == 0
+
+    assert captured["model"] == "Qwen2.5-0.5B-Instruct"
+    assert captured["model_id"] == "Qwen/Qwen2.5-0.5B-Instruct"
+
+
+def test_legacy_start_without_port_uses_next_available_port(monkeypatch):
+    """Omitting --port skips an occupied default port before startup loads the model."""
+    from meshnet_node.cli import main
+
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    occupied = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    occupied.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    occupied.bind(("127.0.0.1", 7000))
+    occupied.listen(1)
+    try:
+        monkeypatch.setattr(sys, "argv", [
+            "meshnet-node", "start",
+            "--tracker", "http://192.168.0.179:8081",
+            "--model", "Qwen/Qwen2.5-0.5B-Instruct",
+            "--host", "127.0.0.1",
+        ])
+
+        with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+            with patch("time.sleep", side_effect=KeyboardInterrupt):
+                try:
+                    main()
+                except SystemExit as exc:
+                    assert exc.code == 0
+    finally:
+        occupied.close()
+
+    assert captured["port"] == 7001
+
+
+def test_default_cli_passes_advertise_host(monkeypatch):
+    """The documented no-subcommand LAN flag reaches startup."""
+    from meshnet_node.cli import main
+
+    saved = {
+        "model_hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+        "model_name": "Qwen2.5-0.5B-Instruct",
+        "quantization": "nf4",
+        "tracker_url": "http://localhost:8080",
+        "wallet_path": "",
+        "download_dir": "",
+        "port": 7000,
+        "host": "0.0.0.0",
+    }
+    captured = {}
+
+    def fake_run_startup(*args, **kwargs):
+        captured.update(kwargs)
+        class _FakeNode:
+            chat_completion_count = 0
+            def stop(self): pass
+        return _FakeNode()
+
+    monkeypatch.setattr(sys, "argv", [
+        "meshnet-node",
+        "--tracker", "http://192.168.0.179:8081",
+        "--advertise-host", "192.168.0.42",
+        "--no-tui",
+    ])
+
+    with patch("meshnet_node.config.load_config", return_value=saved):
+        with patch("meshnet_node.startup.run_startup", side_effect=fake_run_startup):
+            with patch("meshnet_node.dashboard.run_dashboard", side_effect=KeyboardInterrupt):
+                try:
+                    main()
+                except SystemExit as exc:
+                    assert exc.code == 0
+
+    assert captured["tracker_url"] == "http://192.168.0.179:8081"
+    assert captured["advertise_host"] == "192.168.0.42"

tests/test_node_startup.py

@@ -347,6 +347,50 @@ def test_tracker_assign_lists_peers_for_same_model_shard():
         tracker.stop()
 
 
+def test_real_model_startup_summary_shows_total_layers(tmp_path, monkeypatch, capsys):
+    """Real-model startup summary prints the shard range plus total model layers."""
+    import meshnet_node.startup as startup_mod
+
+    class FakeBackend:
+        total_layers = 24
+
+    class FakeTorchNodeServer:
+        def __init__(self, **kwargs):
+            self.kwargs = kwargs
+            self.backend = FakeBackend()
+            self.port = None
+
+        def start(self):
+            self.port = 8001
+            return self.port
+
+    monkeypatch.setattr(
+        startup_mod,
+        "detect_hardware",
+        lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0},
+    )
+    monkeypatch.setattr(startup_mod, "TorchNodeServer", FakeTorchNodeServer)
+    monkeypatch.setattr(
+        startup_mod,
+        "_post_json",
+        lambda _url, _payload, timeout=10.0: {"node_id": "node-test-123"},
+    )
+
+    node = run_startup(
+        tracker_url="http://127.0.0.1:8080",
+        model_id="Qwen/Qwen2.5-0.5B-Instruct",
+        shard_start=0,
+        shard_end=23,
+        wallet_path=tmp_path / "wallet.json",
+    )
+
+    assert node.backend.total_layers == 24
+    assert node.tracker_node_id == "node-test-123"
+    output = capsys.readouterr().out
+    assert "Shard:        layers 0–23; 24 of 24" in output
+    assert "Node ID:      node-test-123" in output
+
+
 # ---------------------------------------------------------------------------
 # Full startup integration test
 # ---------------------------------------------------------------------------
@@ -470,6 +514,163 @@ def test_second_node_downloads_same_shard_from_peer_without_huggingface(
         tracker.stop()
 
 
+def test_network_assign_gap_found_field():
+    """network/assign sets gap_found=True when a real gap exists, False when fully covered."""
+    import json as _json
+    import urllib.request as _ur
+
+    tracker = TrackerServer()
+    port = tracker.start()
+    try:
+        # Register a node covering only layers 0-11 of a 24-layer model.
+        data = _json.dumps({
+            "endpoint": "http://127.0.0.1:9200",
+            "model": "Qwen2.5-0.5B-Instruct",
+            "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+            "num_layers": 24,
+            "shard_start": 0,
+            "shard_end": 11,
+            "hardware_profile": {},
+            "score": 1.0,
+        }).encode()
+        req = _ur.Request(
+            f"http://127.0.0.1:{port}/v1/nodes/register",
+            data=data,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        with _ur.urlopen(req) as r:
+            r.read()
+
+        # A new node should be told there is a gap (layers 12-23).
+        resp = _get_json(
+            f"http://127.0.0.1:{port}/v1/network/assign?device=cpu&vram_mb=0"
+            "&hf_repo=Qwen/Qwen2.5-0.5B-Instruct"
+        )
+        assert resp["gap_found"] is True
+        assert resp["shard_start"] == 12, f"expected gap at 12, got {resp['shard_start']}"
+        assert resp["shard_end"] == 23
+
+        # Register the second node covering the gap.
+        data2 = _json.dumps({
+            "endpoint": "http://127.0.0.1:9201",
+            "model": "Qwen2.5-0.5B-Instruct",
+            "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+            "num_layers": 24,
+            "shard_start": 12,
+            "shard_end": 23,
+            "hardware_profile": {},
+            "score": 1.0,
+        }).encode()
+        req2 = _ur.Request(
+            f"http://127.0.0.1:{port}/v1/nodes/register",
+            data=data2,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        with _ur.urlopen(req2) as r:
+            r.read()
+
+        # Now fully covered — gap_found should be False.
+        resp2 = _get_json(
+            f"http://127.0.0.1:{port}/v1/network/assign?device=cpu&vram_mb=0"
+            "&hf_repo=Qwen/Qwen2.5-0.5B-Instruct"
+        )
+        assert resp2["gap_found"] is False
+    finally:
+        tracker.stop()
+
+
+def test_route_finds_hf_model_across_two_nodes():
+    """Tracker /v1/route returns ordered route for HF model even without a preset."""
+    import json as _json
+    import urllib.request as _ur
+
+    tracker = TrackerServer()
+    port = tracker.start()
+    try:
+        def register(endpoint, shard_start, shard_end):
+            data = _json.dumps({
+                "endpoint": endpoint,
+                "model": "Qwen2.5-0.5B-Instruct",
+                "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+                "num_layers": 24,
+                "shard_start": shard_start,
+                "shard_end": shard_end,
+                "hardware_profile": {},
+                "score": 1.0,
+            }).encode()
+            req = _ur.Request(
+                f"http://127.0.0.1:{port}/v1/nodes/register",
+                data=data,
+                headers={"Content-Type": "application/json"},
+                method="POST",
+            )
+            with _ur.urlopen(req) as r:
+                r.read()
+
+        register("http://127.0.0.1:9300", 0, 11)
+        register("http://127.0.0.1:9301", 12, 23)
+
+        # Route by hf_repo (full identifier).
+        resp = _get_json(
+            f"http://127.0.0.1:{port}/v1/route?model=Qwen/Qwen2.5-0.5B-Instruct"
+        )
+        assert resp["route"] == ["http://127.0.0.1:9300", "http://127.0.0.1:9301"]
+
+        # Route also works by short model name.
+        resp2 = _get_json(
+            f"http://127.0.0.1:{port}/v1/route?model=Qwen2.5-0.5B-Instruct"
+        )
+        assert resp2["route"] == ["http://127.0.0.1:9300", "http://127.0.0.1:9301"]
+    finally:
+        tracker.stop()
+
+
+def test_register_deduplicates_same_endpoint():
+    """Re-registering the same endpoint replaces the old entry, not duplicates it."""
+    import json as _json
+    import urllib.request as _ur
+
+    tracker = TrackerServer()
+    port = tracker.start()
+    try:
+        def register(shard_start, shard_end):
+            data = _json.dumps({
+                "endpoint": "http://127.0.0.1:9400",
+                "model": "Qwen2.5-0.5B-Instruct",
+                "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct",
+                "num_layers": 24,
+                "shard_start": shard_start,
+                "shard_end": shard_end,
+                "hardware_profile": {},
+                "score": 1.0,
+            }).encode()
+            req = _ur.Request(
+                f"http://127.0.0.1:{port}/v1/nodes/register",
+                data=data,
+                headers={"Content-Type": "application/json"},
+                method="POST",
+            )
+            with _ur.urlopen(req) as r:
+                return _json.loads(r.read())
+
+        register(0, 23)  # initial full-model registration
+        register(12, 23)  # re-register with corrected shard range
+
+        # After re-register, tracker should see only one node at 12-23 for this endpoint.
+        # If both were still registered, the gap scan would find no gap (0-23 still covers).
+        # With dedup, the old 0-23 is gone and a real gap 0-11 exists.
+        assign_resp = _get_json(
+            f"http://127.0.0.1:{port}/v1/network/assign?device=cpu&vram_mb=0"
+            "&hf_repo=Qwen/Qwen2.5-0.5B-Instruct"
+        )
+        assert assign_resp["gap_found"] is True
+        assert assign_resp["shard_start"] == 0, "old 0-23 entry should have been replaced"
+    finally:
+        tracker.stop()
+
+
 def test_startup_cpu_fallback(tmp_path, monkeypatch):
     """Node starts with CPU warning when no GPU is detected."""
     import meshnet_node.startup as startup_mod

tests/test_real_model_backend.py

@@ -12,6 +12,9 @@ import pytest
 from meshnet_node.model_backend import (
     InsufficientVRAMError,
     TensorPayload,
+    _call_layer,
+    _decoder_attention_mask,
+    _int_tensor_header,
     build_quantization_config,
     validate_quantization,
 )
@@ -52,6 +55,32 @@ class _FakeTailBackend(_FakeBackend):
         return " Paris"
 
 
+class _FakeFullBackend(_FakeBackend):
+    is_head = True
+    is_tail = True
+
+    def generate_text(
+        self,
+        messages: list[dict],
+        max_new_tokens: int = 16,
+        temperature: float = 1.0,
+        top_p: float = 1.0,
+    ) -> str:
+        assert messages == [{"role": "user", "content": "What is 7 times 8?"}]
+        assert max_new_tokens == 7
+        assert temperature == 1.0
+        assert top_p == 1.0
+        return "56"
+
+    def count_prompt_tokens(self, messages: list[dict]) -> int:
+        assert messages == [{"role": "user", "content": "What is 7 times 8?"}]
+        return 8
+
+    def count_text_tokens(self, text: str) -> int:
+        assert text == "56"
+        return 1
+
+
 def test_quantization_flag_validation():
     assert validate_quantization("bfloat16") == "bfloat16"
     assert validate_quantization("int8") == "int8"
@@ -145,6 +174,65 @@ def test_tail_forward_returns_text_completion_from_binary_activations():
         node.stop()
 
 
+def test_full_model_chat_completion_uses_generation_not_single_token_decode():
+    node = TorchNodeServer(backend=_FakeFullBackend())
+    port = node.start()
+    try:
+        payload = json.dumps({
+            "model": "fake-model",
+            "messages": [{"role": "user", "content": "What is 7 times 8?"}],
+            "max_tokens": 7,
+        }).encode()
+        req = urllib.request.Request(
+            f"http://127.0.0.1:{port}/v1/chat/completions",
+            data=payload,
+            headers={"Content-Type": "application/json"},
+            method="POST",
+        )
+        with urllib.request.urlopen(req, timeout=5) as resp:
+            body = json.loads(resp.read())
+
+        assert body["choices"][0]["message"]["content"] == "56"
+        assert body["usage"] == {"prompt_tokens": 8, "completion_tokens": 1, "total_tokens": 9}
+    finally:
+        node.stop()
+
+
+def test_int_tensor_header_serializes_torch_tensors():
+    torch = pytest.importorskip("torch")
+
+    header = _int_tensor_header(torch.tensor([[1, 2, 3]], dtype=torch.long))
+
+    assert header.startswith("1,3:")
+
+
+def test_decoder_attention_mask_is_causal_float_mask():
+    torch = pytest.importorskip("torch")
+
+    hidden_states = torch.zeros((1, 3, 8), dtype=torch.bfloat16)
+    mask = _decoder_attention_mask(torch.ones((1, 3), dtype=torch.long), hidden_states, torch)
+
+    assert mask.shape == (1, 1, 3, 3)
+    assert mask.dtype == torch.bfloat16
+    assert mask[0, 0, 0, 1] < 0
+    assert mask[0, 0, 2, 0] == 0
+
+
+def test_call_layer_passes_rotary_position_embeddings():
+    class NeedsPositionEmbeddings:
+        def __call__(self, hidden_states, **kwargs):
+            assert kwargs["position_embeddings"] == "rotary"
+            return hidden_states
+
+    assert _call_layer(
+        NeedsPositionEmbeddings(),
+        "hidden",
+        attention_mask=None,
+        position_ids="positions",
+        position_embeddings="rotary",
+    ) == "hidden"
+
+
 @pytest.mark.integration
 def test_two_node_gpt2_completion_is_deterministic():
     if os.environ.get("CI"):

tests/test_tracker_consensus.py

@@ -0,0 +1,255 @@
+"""US-019 integration tests: distributed tracker consensus (Raft + gossip).
+
+Three TrackerServer instances form a Raft cluster in-process.  Tests verify:
+- Leader election completes within 1 second
+- Registration forwarded from follower reaches all nodes
+- Killing the leader triggers a new election within 5 seconds
+- Heartbeat gossip propagates across nodes
+"""
+
+from __future__ import annotations
+
+import json
+import time
+import urllib.request
+
+import pytest
+
+from meshnet_tracker.server import TrackerServer
+
+
+# ------------------------------------------------------------------ helpers
+
+def _get(url: str, timeout: float = 5.0) -> dict:
+    with urllib.request.urlopen(url, timeout=timeout) as r:
+        return json.loads(r.read())
+
+
+def _post(url: str, payload: dict, timeout: float = 5.0) -> dict:
+    data = json.dumps(payload).encode()
+    req = urllib.request.Request(
+        url, data=data, headers={"Content-Type": "application/json"}, method="POST"
+    )
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.loads(r.read())
+
+
+def _register_node(tracker_url: str, port_hint: int) -> str:
+    resp = _post(f"{tracker_url}/v1/nodes/register", {
+        "endpoint": f"http://127.0.0.1:{port_hint}",
+        "model": "stub-model",
+        "shard_start": 0,
+        "shard_end": 15,
+        "hardware_profile": {},
+        "score": 1.0,
+    })
+    return resp["node_id"]
+
+
+def _raft_status(tracker_url: str) -> dict:
+    return _get(f"{tracker_url}/v1/raft/status")
+
+
+def _wait_for_leader(
+    urls: list[str], timeout: float = 5.0
+) -> tuple[str, list[str]]:
+    """Poll until exactly one leader is elected; return (leader_url, follower_urls)."""
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        leaders = []
+        for url in urls:
+            try:
+                s = _raft_status(url)
+                if s.get("role") == "leader":
+                    leaders.append(url)
+            except Exception:
+                pass
+        if len(leaders) == 1:
+            followers = [u for u in urls if u != leaders[0]]
+            return leaders[0], followers
+        time.sleep(0.05)
+    raise TimeoutError(f"No leader elected within {timeout}s")
+
+
+# ------------------------------------------------------------------ fixture
+
+@pytest.fixture
+def three_tracker_cluster():
+    """Start 3 TrackerServer instances as a Raft cluster.
+
+    Yields list of (TrackerServer, url) tuples — index 0 first to start.
+    """
+    # Use fixed starting port range for readability in logs; actual ports
+    # assigned by OS (port=0) to avoid conflicts.
+    trackers: list[TrackerServer] = []
+    ports: list[int] = []
+
+    # Phase 1: start three servers without cluster config to get ports
+    for _ in range(3):
+        t = TrackerServer(host="127.0.0.1", port=0)
+        ports.append(t.start())
+        trackers.append(t)
+
+    # Stop them — we need to restart with peer URLs now that we know ports
+    for t in trackers:
+        t.stop()
+
+    trackers = []
+    urls: list[str] = [f"http://127.0.0.1:{p}" for p in ports]
+
+    for i, port in enumerate(ports):
+        peers = [u for j, u in enumerate(urls) if j != i]
+        t = TrackerServer(
+            host="127.0.0.1",
+            port=port,
+            cluster_peers=peers,
+            cluster_self_url=urls[i],
+        )
+        actual_port = t.start()
+        assert actual_port == port, f"port mismatch: wanted {port}, got {actual_port}"
+        trackers.append(t)
+
+    yield list(zip(trackers, urls))
+
+    for t, _ in zip(trackers, urls):
+        try:
+            t.stop()
+        except Exception:
+            pass
+
+
+# ------------------------------------------------------------------ tests
+
+def test_leader_elected(three_tracker_cluster):
+    """Exactly one leader is elected within 1 second of cluster start."""
+    _, urls = zip(*three_tracker_cluster)
+    leader_url, followers = _wait_for_leader(list(urls), timeout=1.0)
+    assert leader_url in urls
+    assert len(followers) == 2
+
+
+def _wait_until_follower_knows_leader(follower_url: str, timeout: float = 2.0) -> None:
+    """Block until the follower has received a heartbeat and knows the leader URL."""
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        try:
+            s = _raft_status(follower_url)
+            if s.get("leader") and s.get("role") == "follower":
+                return
+        except Exception:
+            pass
+        time.sleep(0.05)
+    raise TimeoutError(f"{follower_url} still doesn't know the leader after {timeout}s")
+
+
+def test_registration_on_follower_visible_on_all_nodes(three_tracker_cluster):
+    """Registering via a follower propagates the entry to all tracker nodes."""
+    trackers_urls = three_tracker_cluster
+    trackers, urls = zip(*trackers_urls)
+    urls = list(urls)
+
+    leader_url, followers = _wait_for_leader(urls, timeout=1.0)
+
+    # Wait until the follower has received at least one heartbeat from the leader
+    follower = followers[0]
+    _wait_until_follower_knows_leader(follower, timeout=2.0)
+
+    # Register via a follower
+    node_id = _register_node(follower, port_hint=19999)
+
+    # Allow replication to propagate (Raft heartbeat interval is 50ms)
+    time.sleep(0.5)
+
+    # Every tracker node must know about this node
+    for url in urls:
+        # Use route endpoint as a proxy — if the node is replicated, the route
+        # will include it.  Alternatively, check /v1/raft/status log_length.
+        status = _raft_status(url)
+        assert status["log_length"] >= 1, (
+            f"{url} has log_length={status['log_length']}, expected ≥1 after replication"
+        )
+
+
+def test_follower_leader_status(three_tracker_cluster):
+    """All nodes agree on who the leader is after election."""
+    _, urls = zip(*three_tracker_cluster)
+    urls = list(urls)
+    _wait_for_leader(urls, timeout=1.0)
+    time.sleep(0.2)  # let heartbeats propagate leader_url to followers
+
+    statuses = [_raft_status(u) for u in urls]
+    leaders_reported = {s["leader"] for s in statuses if s.get("leader")}
+    # All nodes that have a leader opinion should agree on the same one
+    assert len(leaders_reported) <= 1 or len(leaders_reported) == len(
+        {s["leader"] for s in statuses if s["role"] == "leader"}
+    ), f"Nodes disagree on leader: {leaders_reported}"
+
+
+def test_new_leader_elected_after_kill(three_tracker_cluster):
+    """Killing the leader triggers a new election within 5 seconds."""
+    trackers, urls = zip(*three_tracker_cluster)
+    trackers = list(trackers)
+    urls = list(urls)
+
+    # Find the leader object
+    leader_url, followers = _wait_for_leader(urls, timeout=1.0)
+    leader_idx = urls.index(leader_url)
+    leader_tracker = trackers[leader_idx]
+
+    # Register a node before killing the leader (proves it works)
+    _register_node(leader_url, port_hint=19998)
+
+    # Kill the leader
+    leader_tracker.stop()
+    remaining_urls = [u for u in urls if u != leader_url]
+
+    # A new leader must be elected among the remaining 2 nodes
+    new_leader_url, _ = _wait_for_leader(remaining_urls, timeout=5.0)
+    assert new_leader_url in remaining_urls, f"New leader {new_leader_url!r} not in remaining {remaining_urls}"
+
+    # Registration must still work with the new leader
+    node_id = _register_node(new_leader_url, port_hint=19997)
+    assert node_id, "Registration after leader re-election returned no node_id"
+
+
+def test_registration_on_leader_visible_to_all(three_tracker_cluster):
+    """Registering with the leader replicates to all followers synchronously."""
+    _, urls = zip(*three_tracker_cluster)
+    urls = list(urls)
+
+    leader_url, followers = _wait_for_leader(urls, timeout=1.0)
+    node_id = _register_node(leader_url, port_hint=19996)
+
+    # Allow Raft heartbeat to replicate the entry
+    time.sleep(0.3)
+
+    for url in followers:
+        status = _raft_status(url)
+        assert status["log_length"] >= 1, (
+            f"Follower {url} log_length={status['log_length']}, expected ≥1"
+        )
+
+
+def test_gossip_propagates_heartbeat(three_tracker_cluster):
+    """Heartbeat recorded on one tracker propagates to others via gossip."""
+    trackers, urls = zip(*three_tracker_cluster)
+    trackers = list(trackers)
+    urls = list(urls)
+
+    _wait_for_leader(urls, timeout=1.0)
+    leader_url, _ = _wait_for_leader(urls, timeout=1.0)
+
+    # Register a node (so heartbeat makes sense)
+    node_id = _register_node(leader_url, port_hint=19995)
+
+    # Send a heartbeat directly to the leader
+    _post(f"{leader_url}/v1/nodes/{node_id}/heartbeat", {})
+
+    # Allow gossip to propagate to other nodes (gossip interval is 3s in prod,
+    # but we just want to verify the gossip table was updated locally)
+    leader_idx = urls.index(leader_url)
+    leader_gossip = trackers[leader_idx]._gossip
+    assert leader_gossip is not None, "Leader should have gossip enabled"
+    assert leader_gossip.last_seen(node_id) is not None, (
+        "Gossip table on leader should record the heartbeat"
+    )

tests/test_tracker_routing.py

@@ -27,6 +27,132 @@ def _get_json(url: str) -> dict:
         return json.loads(r.read())
 
 
+def test_tracker_send_json_ignores_broken_pipe_after_client_disconnect():
+    """A disconnected client must not dump a BrokenPipe traceback from the tracker."""
+    from meshnet_tracker.server import _TrackerHandler
+
+    class BrokenPipeWriter:
+        def write(self, _body):
+            raise BrokenPipeError
+
+    class DummyHandler:
+        wfile = BrokenPipeWriter()
+
+        def send_response(self, _status):
+            pass
+
+        def send_header(self, _name, _value):
+            pass
+
+        def end_headers(self):
+            pass
+
+    _TrackerHandler._send_json(DummyHandler(), 200, {"ok": True})
+
+
+def test_tracker_serves_health_while_proxy_request_is_in_flight():
+    """Long inference proxy requests must not block heartbeats/health checks."""
+
+    class SlowChatHandler(http.server.BaseHTTPRequestHandler):
+        def log_message(self, fmt, *args):
+            pass
+
+        def do_POST(self):
+            if self.path != "/v1/chat/completions":
+                self.send_response(404)
+                self.end_headers()
+                return
+            length = int(self.headers.get("Content-Length", 0))
+            self.rfile.read(length)
+            time.sleep(2.0)
+            body = json.dumps({"choices": [{"message": {"content": "ok"}}]}).encode()
+            self.send_response(200)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+
+    slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
+    slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
+    slow_thread.start()
+    tracker = TrackerServer(heartbeat_timeout=60.0)
+    tracker_port = tracker.start()
+    proxy_error = []
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": f"http://127.0.0.1:{slow_node.server_address[1]}",
+             "model": "slow-model", "hf_repo": "org/slow-model", "num_layers": 1,
+             "shard_start": 0, "shard_end": 0, "tracker_mode": True,
+             "hardware_profile": {}, "score": 1.0},
+        )
+
+        def call_proxy():
+            try:
+                _post_json(
+                    f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
+                    {"model": "slow-model", "messages": [{"role": "user", "content": "hi"}]},
+                )
+            except Exception as exc:
+                proxy_error.append(exc)
+
+        proxy_thread = threading.Thread(target=call_proxy)
+        proxy_thread.start()
+        time.sleep(0.2)
+
+        with urllib.request.urlopen(f"http://127.0.0.1:{tracker_port}/v1/health", timeout=1.0) as resp:
+            assert resp.status == 200
+
+        proxy_thread.join(timeout=3.0)
+        assert not proxy_thread.is_alive()
+        assert not proxy_error
+    finally:
+        tracker.stop()
+        slow_node.shutdown()
+        slow_node.server_close()
+        slow_thread.join(timeout=1.0)
+
+
+def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix():
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    wallet = "7j77FsPY1evV8tuf4Z73AVrWwxBEW1pvKwi4EvcRD3g"
+    try:
+        first = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9100", "model": "Qwen2.5-0.5B-Instruct",
+             "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct", "num_layers": 24,
+             "shard_start": 0, "shard_end": 21, "wallet_address": wallet,
+             "hardware_profile": {}, "score": 1.0},
+        )
+        second = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9100", "model": "Qwen2.5-0.5B-Instruct",
+             "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct", "num_layers": 24,
+             "shard_start": 0, "shard_end": 21, "wallet_address": wallet,
+             "hardware_profile": {}, "score": 1.0},
+        )
+        different_endpoint = _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9101", "model": "Qwen2.5-0.5B-Instruct",
+             "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct", "num_layers": 24,
+             "shard_start": 20, "shard_end": 23, "wallet_address": wallet,
+             "hardware_profile": {}, "score": 1.0},
+        )
+
+        assert first["node_id"].startswith("7j77FsPY-")
+        assert second["node_id"] == first["node_id"]
+        assert different_endpoint["node_id"].startswith("7j77FsPY-")
+        assert different_endpoint["node_id"] != first["node_id"]
+
+        route_resp = _get_json(
+            f"http://127.0.0.1:{tracker_port}/v1/route?model=Qwen/Qwen2.5-0.5B-Instruct"
+        )
+        assert route_resp["nodes"][0]["node_id"] == first["node_id"]
+    finally:
+        tracker.stop()
+
+
 def test_tracker_node_registration():
     """A node can register with the tracker and receives a node_id."""
     tracker = TrackerServer()
@@ -131,6 +257,64 @@ def test_tracker_coverage_endpoint_reports_uncovered_ranges():
         tracker.stop()
 
 
+def test_tracker_coverage_endpoint_accepts_registered_hf_repo_or_short_name():
+    """Coverage endpoint supports real HF models registered outside preset catalog."""
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9101", "model": "Qwen2.5-0.5B-Instruct",
+             "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct", "num_layers": 24,
+             "shard_start": 0, "shard_end": 21, "hardware_profile": {}, "score": 1.0},
+        )
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9102", "model": "Qwen2.5-0.5B-Instruct",
+             "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct", "num_layers": 24,
+             "shard_start": 20, "shard_end": 23, "hardware_profile": {}, "score": 1.0},
+        )
+
+        by_repo = _get_json(
+            f"http://127.0.0.1:{tracker_port}/v1/coverage/Qwen/Qwen2.5-0.5B-Instruct"
+        )
+        by_short_name = _get_json(
+            f"http://127.0.0.1:{tracker_port}/v1/coverage/Qwen2.5-0.5B-Instruct"
+        )
+
+        assert by_repo["model"] == "Qwen/Qwen2.5-0.5B-Instruct"
+        assert by_repo["coverage"] == [
+            {"start_layer": 0, "end_layer": 19, "node_count": 1},
+            {"start_layer": 20, "end_layer": 21, "node_count": 2},
+            {"start_layer": 22, "end_layer": 23, "node_count": 1},
+        ]
+        assert by_short_name["coverage"] == by_repo["coverage"]
+    finally:
+        tracker.stop()
+
+
+def test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias():
+    tracker = TrackerServer()
+    tracker_port = tracker.start()
+    try:
+        _post_json(
+            f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
+            {"endpoint": "http://127.0.0.1:9111", "model": "Qwen2.5-0.5B-Instruct",
+             "hf_repo": "Qwen/Qwen2.5-0.5B-Instruct", "num_layers": 24,
+             "shard_start": 0, "shard_end": 23, "hardware_profile": {}, "score": 1.0},
+        )
+
+        models_resp = _get_json(f"http://127.0.0.1:{tracker_port}/v1/models")
+
+        model = next(item for item in models_resp["data"] if item["id"] == "Qwen/Qwen2.5-0.5B-Instruct")
+        assert model["name"] == "Qwen2.5-0.5B-Instruct"
+        assert model["hf_repo"] == "Qwen/Qwen2.5-0.5B-Instruct"
+        assert model["aliases"] == ["Qwen/Qwen2.5-0.5B-Instruct", "Qwen2.5-0.5B-Instruct"]
+        assert model["shard_coverage_percentage"] == 100.0
+    finally:
+        tracker.stop()
+
+
 def test_tracker_auto_assigns_new_node_to_uncovered_range_first():
     """Capability-driven registration fills the first uncovered layer gap."""
     tracker = TrackerServer(model_presets={