neuron-tai/docs/INSTALL_WINDOWS.md

Installing meshnet-node on Windows 11 with WSL2

This guide covers setting up a meshnet-node on a Windows 11 machine using WSL2 with CUDA passthrough so it can join an existing inference network over LAN.

Prerequisites

• Windows 11 with WSL2 support (most systems with Windows 10 version 2004+ qualify)
• NVIDIA GPU with CUDA support (driver ≥ 525.x recommended for WSL2 CUDA)
• At least 8 GB RAM + enough VRAM for the model shard you intend to serve
• The Linux machine (other node) is reachable on your LAN

---

Step 1 — Enable WSL2 and install Ubuntu

Open PowerShell as Administrator and run:

wsl --install -d Ubuntu-24.04

This installs WSL2 with Ubuntu 24.04. Reboot when prompted.

After reboot, Ubuntu starts and asks you to create a UNIX username/password. Choose anything convenient.

Verify WSL version:

wsl -l -v

Output should show VERSION 2.

---

Step 2 — Install NVIDIA GPU driver on Windows (NOT inside WSL)

WSL2 CUDA passthrough works through the Windows host driver. Do not install CUDA inside WSL2.

1. Download the latest Game Ready or Studio driver for your GPU from https://www.nvidia.com/drivers
2. Install on Windows normally (standard installer).
3. Inside WSL2 (Ubuntu terminal), verify:

nvidia-smi

Expected output: your GPU name, driver version, CUDA version. If this command fails, the Windows driver is too old — update it.

Note: The cuda-toolkit package inside WSL2 is optional and only needed if you compile CUDA kernels. For inference with torch, the Windows host driver is sufficient.

---

Step 3 — Install Python 3.11+ inside WSL2

Ubuntu 24.04 ships Python 3.12. Confirm:

python3 --version

If it shows 3.10 or older:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.12 python3.12-venv python3.12-dev

Install pip:

curl -sS https://bootstrap.pypa.io/get-pip.py | python3

---

Step 4 — Clone the repository

Inside WSL2:

# Store the repo in the Linux filesystem (faster I/O than /mnt/c)
cd ~
git clone https://github.com/YOUR_ORG/d-popov.com.git
cd d-popov.com/AI

---

Step 5 — Create a virtualenv and install meshnet-node

python3 -m venv .venv
source .venv/bin/activate

# Install node + PyTorch (CUDA build)
pip install torch --index-url https://download.pytorch.org/whl/cu124
pip install -e "packages/node[torch]"

Verify the install:

meshnet-node --help
python -c "import transformers; print(transformers.__version__)"

transformers must be ≥ 5.12 for Qwen3.5/3.6-MoE models (older versions fail with 'Qwen3_5MoeConfig' object has no attribute 'vocab_size'). If you install into an existing conda/miniforge env instead of a fresh venv, run pip install -U transformers there. The startup warning about flash-linear-attention / causal-conv1d ("fast path is not available") is harmless on CPU — those are optional CUDA-only kernels.

If you run the node from native Windows instead of WSL2, install the Triton shim in the same environment:

python -m pip install triton-windows

Without it, Qwen3.5/3.6-MoE startup can fail with the misleading message Could not import module 'Qwen3_5MoeForCausalLM'.

---

Step 6 — Pre-download the model shard

Download the model before starting the node so the startup process doesn't time out on the tracker side:

python3 - <<'EOF'
from transformers import AutoConfig
AutoConfig.from_pretrained("microsoft/Phi-3-medium-128k-instruct")
EOF

For the full model weights (needed at runtime), transformers downloads them automatically on first meshnet-node start. If you want to pre-fetch:

python3 -c "
from transformers import AutoModelForCausalLM
AutoModelForCausalLM.from_pretrained('microsoft/Phi-3-medium-128k-instruct', device_map='cpu')
"

This can take 10–30 minutes on first run.

---

Step 7 — Expose the node port to your LAN

WSL2 runs behind a NAT with a virtual IP (typically 172.x.x.x). Your LAN sees the Windows host IP. You need to forward the node port.

Option A — Windows port proxy (recommended for simple setups):

In PowerShell as Administrator:

# Get the current WSL2 IP (changes on each WSL restart)
$wslIp = (wsl hostname -I).Trim()

# Forward Windows host port 8001 → WSL2 port 8001
netsh interface portproxy add v4tov4 `
  listenport=8001 listenaddress=0.0.0.0 `
  connectport=8001 connectaddress=$wslIp

# Allow inbound on Windows Firewall
New-NetFirewallRule -DisplayName "meshnet-node" `
  -Direction Inbound -Protocol TCP -LocalPort 8001 -Action Allow

Verify: from the Linux machine, curl http://WINDOWS_LAN_IP:8001/v1/health should return a response once the node is running.

Redo this after every WSL2 restart — the WSL2 IP changes.

Option B — P2P relay (US-017, no port forwarding needed):

Start a relay node on the Linux machine. The WSL2 node connects outbound through the relay. No firewall rules needed. See docs/TWO_MACHINE_TEST.md for details.

---

Step 8 — Start the node

Replace 192.168.1.10 with the actual LAN IP of the Linux machine running the tracker. Replace shard range with the complementary range to what the Linux node is serving.

source .venv/bin/activate

meshnet-node \
  --model microsoft/Phi-3-medium-128k-instruct \
  --quantization bf16 \
  --shard-start 20 --shard-end 39 \
  --tracker http://192.168.1.10:8080 \
  --port 8001 \
  --host 0.0.0.0 \
  --advertise-host WINDOWS_LAN_IP

The --advertise-host flag tells the tracker what IP the Linux machine should use to reach this node. Use your Windows machine's LAN IP (e.g. 192.168.1.20), not the WSL2 internal IP.

Expected startup output:

Detecting hardware...
  GPU: NVIDIA GeForce RTX 3080 (10240 MB VRAM)
Loading wallet...
  Wallet: 5K7r...
Loading real PyTorch model shard...
  Auto-detected 40 layers → shard 20–39
================================
meshnet-node ready
  Model ID:     microsoft/Phi-3-medium-128k-instruct
  Shard:        layers 20–39; 20 of 40
  Endpoint:     http://192.168.1.20:8001
  Hardware:     CUDA
================================

---

Known issues

• WSL2 IP changes on restart. Always re-run the netsh port-proxy command after restarting WSL2 or Windows.
• CUDA not visible in WSL2. If nvidia-smi fails inside WSL2, update the Windows host GPU driver to ≥ 525.x. Installing CUDA inside WSL2 will not fix it.
• Model download is slow. HuggingFace downloads happen over HTTPS. Pre-fetch the model before a timed test (see Step 6).
• Port 8001 already in use. Change --port to another value and update the firewall/portproxy rules accordingly.
• bf16 not supported on older GPUs. Use --quantization int8 on Turing (RTX 20xx) cards or earlier if bfloat16 ops fail.