diff --git a/QUICKSTART.md b/QUICKSTART.md index ffbca8f..e118474 100644 --- a/QUICKSTART.md +++ b/QUICKSTART.md @@ -48,13 +48,10 @@ python3 -m venv .venv If `.venv/bin/meshnet-node` is missing, the editable install step did not finish successfully. Re-run the `.venv/bin/pip install -e ...` command above inside WSL. -WSL2 is still useful for local development, but do not rely on it for the -"another machine connects back to this node" LAN case. WSL2 commonly sits behind -Windows NAT/port-proxy behavior and may not accept inbound traffic from other LAN -machines without extra host networking setup. We intentionally leave that unfixed -because it is useful for testing NAT/relay scenarios. If you just want to bring up -a Windows node that other machines can reach directly, run the node in native -Windows PowerShell instead. +WSL2 sits behind Windows NAT and is **not directly reachable** from other LAN machines. +Direct cross-host hops time out. The relay path (see below) solves this: the WSL2 node +opens an outbound WebSocket to the relay server and all traffic flows through that tunnel. +No firewall rules, no `--advertise-host` needed — just point at the public tracker URL. ### Native Windows PowerShell node (not WSL) @@ -126,17 +123,21 @@ $env:HF_HOME = "D:\DEV\models" --port 8005 ``` -One-line variants: +One-line variants (direct LAN — node must be reachable by IP from other machines): ```powershell -// only this works - when not behind NAT(wsl), and via IP. Revisit when relaying/RPC is implemented -.\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20 -.\.venv\Scripts\meshnet-node.exe start --tracker http://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20 +.\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20 +``` + +Via public hostname with relay (works from behind NAT, WSL2, 5G — no `--advertise-host` needed): + +```powershell +.\.venv\Scripts\meshnet-node.exe start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct ``` `--host 0.0.0.0` binds the node to all Windows interfaces. `--advertise-host` -is what the tracker gives to other nodes, so it must be the Windows LAN IP that -the tracker and peer nodes can actually reach. +is what the tracker gives to other nodes for direct connections; omit it when using +the relay path since all traffic flows through the relay tunnel instead. If you want verbose per-hop pipeline logs while debugging a split model, add `--debug`. Leave it off for normal runs; otherwise every generated token logs @@ -145,6 +146,7 @@ lines like: ```text [node] pipeline hop 0: http://127.0.0.1:8005 start_layer=22 [node] pipeline hop 0 returned text=' token' + [node] pipeline hop 1: wss://ai.neuron.d-popov.com/rpc/abc123 relay start_layer=12 ``` 6. From the tracker machine, verify Windows is reachable: @@ -158,27 +160,42 @@ connection reached the node process. If it times out or connection-refuses, chec the Windows Firewall rule, `--host 0.0.0.0`, the selected LAN IP, and that the node is still running. -### Public tracker + WSS relay +--- -For internet nodes, expose one public HTTPS host and proxy these paths: +## Public tracker + relay (internet / NAT nodes) -```text -/v1/* -> meshnet-tracker, for registration, heartbeats, routing, and OpenAI requests -/ws -> meshnet-relay, for outbound node gossip/bridge connections -/rpc/* -> meshnet-relay, for tracker-to-node relay requests +This setup lets nodes connect from anywhere — behind home NAT, 5G, WSL2, or +on a different continent — without opening firewall ports. + +### Architecture + +``` +Client → HTTPS → ai.neuron.d-popov.com (nginx) + ├─ /v1/* → meshnet-tracker :8081 + ├─ /ws → meshnet-relay :8765 (node persistent outbound WS) + └─ /rpc/* → meshnet-relay :8765 (caller opens WS per hop) ``` -Start the tracker with the public relay URL it should advertise: +### Start the relay and tracker (server side) ```bash +# Terminal 1 — relay (WebSocket hub) .venv/bin/meshnet-relay --host 0.0.0.0 --port 8765 + +# Terminal 2 — tracker (advertises relay URL to nodes) .venv/bin/meshnet-tracker start \ --host 0.0.0.0 \ --port 8081 \ --relay-url wss://ai.neuron.d-popov.com/ws ``` -Then a node only needs the public tracker address: +The `--relay-url` flag embeds the relay address in `/v1/network/map`. Every node +queries that endpoint on startup and auto-connects if a relay URL is present. + +### Start a node (any machine, any network) + +No `--advertise-host` needed. The node discovers the relay URL from the tracker +and opens a persistent outbound WebSocket: ```bash .venv/bin/meshnet-node start \ @@ -186,6 +203,84 @@ Then a node only needs the public tracker address: --model Qwen/Qwen2.5-0.5B-Instruct ``` +Expected startup output (relay path): + +``` + Auto-detected 24 layers → shard 0–23 + Relay connected — wss://ai.neuron.d-popov.com/rpc/abc1def2ef3f4567 +================================ +meshnet-node ready + Wallet:
+ Model ID: Qwen/Qwen2.5-0.5B-Instruct + Shard: layers 0–23; 24 of 24 + Quantization: bfloat16 + Endpoint: http://172.29.104.23:7001 + Node ID: + Hardware: CPU +================================ +``` + +The `Endpoint` shown is the local IP (unreachable from outside). Other nodes reach +this one via `wss://ai.neuron.d-popov.com/rpc/` instead. + +### How relay hops work + +When node A needs to forward activations to node B (behind NAT): + +1. Tracker injects `X-Meshnet-Route` with `relay_addr` for each behind-NAT hop. +2. Node A opens a WebSocket to `wss://relay/rpc/{peer_id_B}`. +3. Relay forwards the `relay-http-request` envelope to Node B's persistent connection. +4. Node B processes `/forward` locally, returns `relay-http-response`. +5. Relay sends the response back to Node A over the same WebSocket. +6. Node A closes the WebSocket and continues the pipeline. + +Binary activation tensors (bfloat16) are Base64-encoded through the relay JSON +protocol and decoded on both sides — no precision loss. + +If the relay hop fails (relay down, peer disconnected), the node logs a warning and +falls back to a direct HTTP attempt before returning an error. + +### Test from WSL2 using the public tracker + +In WSL2 (which gets a `172.x.x.x` virtual IP — unreachable from other machines): + +```bash +# WSL2 Terminal 1 — head node (layers 0–11, handles chat requests) +.venv/bin/meshnet-node start \ + --tracker https://ai.neuron.d-popov.com \ + --model Qwen/Qwen2.5-0.5B-Instruct \ + --shard-start 0 --shard-end 11 + +# WSL2 Terminal 2 — tail node (layers 12–23) +.venv/bin/meshnet-node start \ + --tracker https://ai.neuron.d-popov.com \ + --model Qwen/Qwen2.5-0.5B-Instruct \ + --shard-start 12 --shard-end 23 +``` + +Both nodes connect to the relay automatically. When a chat request arrives at Node A, +it forwards activations to Node B via `wss://ai.neuron.d-popov.com/rpc/{peer_id_B}`. + +Send inference through the tracker (which picks the head node and injects the route): + +```bash +curl -s https://ai.neuron.d-popov.com/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen2.5-0.5B-Instruct", + "messages": [{"role": "user", "content": "What is 7 times 8?"}], + "stream": false + }' | python3 -m json.tool +``` + +Or send directly to Node A's local port (within WSL): + +```bash +curl -s http://localhost:7001/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{"model": "Qwen/Qwen2.5-0.5B-Instruct", "messages": [{"role": "user", "content": "Hi"}]}' +``` + --- ## Step 1 — Start the tracker (Terminal 1) @@ -304,7 +399,7 @@ HF_HOME=/run/media/popov/d/DEV/models \ --port 8002 ``` -Send the request to Node A — it tokenizes, runs layers 0–13, passes binary +Send the request to Node A — it tokenizes, runs layers 0–11, passes binary activations to Node B, and streams the final response back. --- @@ -342,18 +437,6 @@ See `docs/TWO_MACHINE_TEST.md` (created by US-018). --- -## Start the relay node (for NAT traversal) - -```bash -.venv/bin/pip install -e packages/relay -.venv/bin/meshnet-relay --port 8765 -``` - -Nodes behind NAT connect to the relay and advertise their relay address to the -tracker. See `docs/adr/0010-p2p-gossip-and-nat-relay.md`. - ---- - ## Run all tests ```bash