neuron-tai/docs/issues/18-two-machine-lan-test.md

# US-018 — End-to-end two-machine LAN inference test

## Goal

Run real distributed inference across two physical machines: the Linux rig and a Windows 11 rig running WSL2. Document every setup step, firewall rule, and gotcha so this is repeatable. The test script exits 0 with token output and timing, proving the network works.

## Network topology for LAN test

```
[Linux machine]                           [Windows 11 / WSL2]
  meshnet-tracker  :8080                    meshnet-node (shard B)
  meshnet-node     :8001  (shard A, tracker-mode)
  meshnet-gateway  :8000  (optional, for OpenAI-compat)

Client (either machine):
  scripts/test_lan_inference.py --tracker http://192.168.1.10:8080
```

The Linux machine runs the tracker + the first-shard node (tracker-mode). The Windows/WSL2 machine runs the second-shard node. A small model (e.g. Phi-3-medium at BF16, fits on one GPU each) is split across both.

## WSL2 setup (Windows side)

`docs/INSTALL_WINDOWS.md` covers:

1. Enable WSL2: `wsl --install -d Ubuntu-24.04`
2. CUDA in WSL2: install NVIDIA driver on Windows (NOT inside WSL); WSL2 gets CUDA automatically
   - Verify: `nvidia-smi` inside WSL2 should show GPU
3. Install Python 3.11+ and pip inside WSL2
4. `pip install -e packages/node packages/p2p` (clone repo first)
5. Firewall: Windows Defender must allow inbound WSL2 → LAN on node port
   - PowerShell: `New-NetFirewallRule -DisplayName "meshnet-node" -Direction Inbound -Protocol TCP -LocalPort 8001 -Action Allow`
6. WSL2 IP: WSL2 has its own NAT'd IP (172.x.x.x); to expose to LAN, either:
   - Option A: `netsh interface portproxy add v4tov4 listenport=8001 listenaddress=0.0.0.0 connectport=8001 connectaddress=$(wsl hostname -I)`
   - Option B: use the relay node (US-017) — no port forwarding needed

## Linux setup

Standard install (already done after US-016). Firewall:

```bash
# If using ufw
sudo ufw allow 8080/tcp  # tracker
sudo ufw allow 8001/tcp  # node
sudo ufw allow 8000/tcp  # gateway (optional)
```

## Model split

For the test, use a model that has enough layers to split meaningfully but fits comfortably in memory. Phi-3-medium-128k-instruct (40 layers, BF16 15GB) works on a single 24GB GPU on each machine:

- Linux node: layers 0–19 (tracker-mode, owns tokenizer + embed_tokens)
- Windows/WSL2 node: layers 20–39

Start sequence:
```bash
# Terminal 1 (Linux) — tracker
meshnet-tracker --port 8080

# Terminal 2 (Linux) — first-shard node (tracker-mode auto-detected because shard_start=0)
meshnet-node --model microsoft/Phi-3-medium-128k-instruct \
             --quantization bf16 \
             --shard-start 0 --shard-end 19 \
             --tracker http://localhost:8080 \
             --port 8001

# Terminal 3 (Windows WSL2) — second-shard node
meshnet-node --model microsoft/Phi-3-medium-128k-instruct \
             --quantization bf16 \
             --shard-start 20 --shard-end 39 \
             --tracker http://192.168.1.10:8080 \
             --port 8001
```

## Test script

`scripts/test_lan_inference.py`:

```python
#!/usr/bin/env python3
"""
End-to-end LAN inference test.
Usage: python scripts/test_lan_inference.py --tracker http://192.168.1.10:8080
"""
import argparse, time, httpx, json

MESSAGES = [
    {"role": "user", "content": "What is 7 × 8? Answer in one word."},
    {"role": "user", "content": "Name the capital of France in one word."},
    {"role": "user", "content": "Complete the sequence: 1, 1, 2, 3, 5, ___"},
]

def run_test(tracker_url: str, gateway_url: str | None):
    # Discover inference entry point via tracker if gateway not given
    if not gateway_url:
        r = httpx.get(f"{tracker_url}/v1/tracker-nodes/phi-3-medium", timeout=5)
        r.raise_for_status()
        nodes = r.json()
        assert nodes, "No tracker-mode nodes registered — is the first-shard node running?"
        gateway_url = nodes[0]["url"]

    print(f"Inference endpoint: {gateway_url}")
    print(f"Tracker: {tracker_url}")
    print()

    for i, msg in enumerate(MESSAGES):
        t0 = time.monotonic()
        r = httpx.post(
            f"{gateway_url}/v1/chat/completions",
            json={"model": "phi-3-medium", "messages": [msg], "stream": False},
            timeout=60,
        )
        r.raise_for_status()
        data = r.json()
        elapsed = time.monotonic() - t0

        content = data["choices"][0]["message"]["content"]
        tokens = data["usage"]["completion_tokens"]
        tps = tokens / elapsed if elapsed > 0 else 0

        print(f"[{i+1}] Q: {msg['content']}")
        print(f"     A: {content}")
        print(f"     {tokens} tokens  {elapsed:.2f}s  {tps:.1f} t/s")
        print()

    print("✓ All 3 requests completed successfully")

if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--tracker", required=True)
    p.add_argument("--gateway", default=None)
    args = p.parse_args()
    run_test(args.tracker, args.gateway)
```

## Docs: TWO_MACHINE_TEST.md

`docs/TWO_MACHINE_TEST.md` must cover:

1. Prerequisites (models downloaded on both machines, same model ID, complementary shard ranges)
2. Start order: tracker first, then nodes, then test script
3. How to verify nodes are registered: `GET /v1/nodes` on tracker
4. How to verify coverage: `GET /v1/coverage/phi-3-medium` — all 40 layers must show node_count ≥ 1
5. How to run the test script
6. Expected output
7. Latency breakdown: how to read per-hop latency from node logs
8. **Known Issues** section — updated during actual test run with real gotchas

## Acceptance criteria

- `docs/INSTALL_WINDOWS.md` covers WSL2 + CUDA + meshnet-node install end-to-end
- `docs/TWO_MACHINE_TEST.md` covers the full two-machine setup and test procedure
- `scripts/test_lan_inference.py` exists and is executable
- When run against a real two-machine LAN setup: script exits 0, prints 3 valid answers with timing
- Coverage map shows 100% coverage (no gap) after both nodes register
- Known Issues section in TWO_MACHINE_TEST.md contains at least the issues encountered during this test run
- No new pytest failures from repo root (this story adds docs + a script, not new Python packages)
- Commit only this story's changes