feat(us-018): WSL2 install guide, two-machine LAN test docs, and test script

- docs/INSTALL_WINDOWS.md: step-by-step WSL2 + CUDA + meshnet-node install on Windows 11, including port-proxy setup and known issues - docs/TWO_MACHINE_TEST.md: two-machine LAN test procedure, start order, verification steps, latency reading, and Known Issues section - scripts/test_lan_inference.py: stdlib-only test script; sends 3 chat completions, validates OpenAI response format, prints tokens + latency, exits 0 on success; auto-discovers gateway from tracker if --gateway omitted Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 01:37:33 +03:00
parent d701ae9ba2
commit be37048145
4 changed files with 584 additions and 7 deletions
--- a/docs/TWO_MACHINE_TEST.md
+++ b/docs/TWO_MACHINE_TEST.md
@@ -0,0 +1,200 @@
+# Two-machine LAN inference test
+
+This guide proves that distributed inference works across two physical machines: a Linux rig (tracker + first shard) and a Windows 11 / WSL2 rig (second shard). A test script sends real inference requests and validates the output.
+
+## Network topology
+
+```
+[Linux machine — 192.168.1.10]
+  meshnet-tracker  :8080
+  meshnet-node A   :8001   shard 0–19  (tracker-mode, entry point)
+
+[Windows 11 / WSL2 — 192.168.1.20]
+  meshnet-node B   :8001   shard 20–39
+
+[Client — either machine]
+  scripts/test_lan_inference.py --tracker http://192.168.1.10:8080
+```
+
+Adjust the IPs and shard ranges to match your hardware. Use a model that fits (sharded) in both GPUs combined. The example uses `microsoft/Phi-3-medium-128k-instruct` (40 layers, BF16 ~15 GB each shard ~7.5 GB).
+
+---
+
+## Prerequisites
+
+**Both machines:**
+- Python 3.11+ with `meshnet-node` installed (see `docs/INSTALL_WINDOWS.md` for Windows)
+- Model weights already downloaded (pre-fetch prevents timeout on first startup)
+- LAN connectivity verified: `ping 192.168.1.10` from Windows, `ping 192.168.1.20` from Linux
+
+**Linux machine ports open:**
+
+```bash
+# ufw (skip if firewall is off)
+sudo ufw allow 8080/tcp   # tracker
+sudo ufw allow 8001/tcp   # node A
+```
+
+**Windows machine port forwarded (WSL2 only):**
+
+```powershell
+# Run in PowerShell as Administrator — redo after every WSL restart
+$wsl = (wsl hostname -I).Trim()
+netsh interface portproxy add v4tov4 listenport=8001 listenaddress=0.0.0.0 connectport=8001 connectaddress=$wsl
+New-NetFirewallRule -DisplayName "meshnet-node" -Direction Inbound -Protocol TCP -LocalPort 8001 -Action Allow
+```
+
+---
+
+## Start sequence
+
+**Always start in this order: tracker → node A → node B → test.**
+
+### Terminal 1 — Linux: tracker
+
+```bash
+meshnet-tracker --port 8080
+```
+
+Expected:
+
+```
+[tracker] listening on 0.0.0.0:8080
+```
+
+### Terminal 2 — Linux: node A (shard 0–19, tracker-mode)
+
+```bash
+meshnet-node \
+  --model microsoft/Phi-3-medium-128k-instruct \
+  --quantization bf16 \
+  --shard-start 0 --shard-end 19 \
+  --tracker http://localhost:8080 \
+  --port 8001 \
+  --host 0.0.0.0
+```
+
+`shard_start=0` auto-sets `tracker_mode=True` — this node accepts inference requests.
+
+Wait until you see `meshnet-node ready` before continuing.
+
+### Terminal 3 — Windows WSL2: node B (shard 20–39)
+
+```bash
+meshnet-node \
+  --model microsoft/Phi-3-medium-128k-instruct \
+  --quantization bf16 \
+  --shard-start 20 --shard-end 39 \
+  --tracker http://192.168.1.10:8080 \
+  --port 8001 \
+  --host 0.0.0.0 \
+  --advertise-host 192.168.1.20
+```
+
+`--advertise-host` must be the Windows **LAN IP** (not the WSL2 internal 172.x.x.x IP) so the Linux node can reach it.
+
+---
+
+## Verify nodes are registered
+
+From any machine with `curl`:
+
+```bash
+# List all registered nodes
+curl http://192.168.1.10:8080/v1/nodes
+
+# Check route for the model — should list both node endpoints in order
+curl "http://192.168.1.10:8080/v1/route?model=microsoft/Phi-3-medium-128k-instruct"
+```
+
+Expected route response:
+
+```json
+{
+  "route": [
+    "http://192.168.1.10:8001",
+    "http://192.168.1.20:8001"
+  ]
+}
+```
+
+If only one endpoint appears, node B hasn't registered yet — wait a few seconds and retry.
+
+---
+
+## Run the test script
+
+```bash
+# From any machine that can reach the tracker
+python3 scripts/test_lan_inference.py \
+  --tracker http://192.168.1.10:8080 \
+  --gateway http://192.168.1.10:8001
+```
+
+Expected output:
+
+```
+Inference endpoint: http://192.168.1.10:8001
+Tracker: http://192.168.1.10:8080
+
+Route: ['http://192.168.1.10:8001', 'http://192.168.1.20:8001']
+
+[1] Q: What is 7 × 8? Answer in one word.
+     A: 56
+     3 tokens  2.41s  1.2 t/s
+
+[2] Q: Name the capital of France in one word.
+     A: Paris
+     2 tokens  1.87s  1.1 t/s
+
+[3] Q: Complete the sequence: 1, 1, 2, 3, 5, ___
+     A: 8
+     2 tokens  1.93s  1.0 t/s
+
+All 3 requests completed successfully.
+Exit code: 0
+```
+
+The script exits 0 if all 3 requests complete with valid OpenAI-format responses.
+
+---
+
+## Reading latency from node logs
+
+The node logs show per-hop timing. On node A terminal look for:
+
+```
+[node] forwarding to downstream: http://192.168.1.20:8001  (took 1.23s)
+```
+
+Approximate breakdown:
+- **client → node A (encode + first shard):** full request latency minus the downstream time
+- **node A → node B (pipeline):** the `forwarding to downstream` duration
+- **node B → node A (tail decode + token):** included in downstream duration
+
+Full end-to-end latency = prompt encode + shard A forward + network transfer + shard B forward + decode.
+
+With LAN latency < 1 ms, the network transfer is negligible. Bottleneck is GPU compute.
+
+---
+
+## Known Issues
+
+**WSL2 IP changes after restart.**
+The `netsh portproxy` forwarding rule uses a fixed WSL2 IP. If Windows or WSL2 restarts, the IP changes and the rule breaks. Redo the `netsh` and `New-NetFirewallRule` commands. To automate this, add a Task Scheduler job on WSL start.
+
+**Node B registers with internal WSL2 IP (172.x.x.x) instead of LAN IP.**
+Symptom: route response lists `172.x.x.x` and node A cannot reach it.
+Fix: always pass `--advertise-host 192.168.1.20` (your Windows LAN IP) when starting node B.
+
+**Model download times out node registration.**
+If the model hasn't been pre-fetched, `transformers` downloads it during node startup, which can take 20+ minutes. The tracker heartbeat timeout (90s) will expire, and node A will deregister node B. Pre-download the model weights before starting the node (see `docs/INSTALL_WINDOWS.md` Step 6). Node B re-registers automatically via the heartbeat re-registration loop once it's up.
+
+**`bf16` unsupported on older NVIDIA GPUs.**
+GPUs before Ampere (RTX 30xx) have limited bfloat16 support. Use `--quantization int8` on RTX 20xx and earlier.
+
+**Windows Defender blocks inbound connection on WSL2.**
+Even with the firewall rule added, Windows Defender SmartScreen or a corporate security policy can block the connection. Verify by checking Windows Event Viewer → Security → Filtering Platform Connection for blocked connections on port 8001.
+
+**Route returns only one node.**
+If node B registers but the route only returns one endpoint, check that both nodes use the same `--model` string (full HuggingFace repo path). Route lookup matches on `hf_repo` — a short name vs. full path mismatch causes the node to be excluded.