popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: revise QUICKSTART with relay NAT/internet connectivity guide

Browse Source 
- Replace stale "only this works" PowerShell comment with accurate relay
  one-liner that works from behind NAT without --advertise-host
- Expand "Public tracker + WSS relay" into a full architecture section:
  nginx proxy paths, start commands, relay hop sequence, Base64 encoding
- Add WSL2 two-node relay test scenario with curl examples
- Document startup output when relay connects (Relay connected line)
- Remove "Start the relay node" stub; fold into the main relay section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dobromir Popov
2026-06-30 18:34:06 +03:00
parent ab6558d861
commit 4f00a37d72
1 changed files with 117 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

151
QUICKSTART.md
View File

@@ -48,13 +48,10 @@ python3 -m venv .venv
If `.venv/bin/meshnet-node` is missing, the editable install step did not finish If `.venv/bin/meshnet-node` is missing, the editable install step did not finish
successfully. Re-run the `.venv/bin/pip install -e ...` command above inside WSL. successfully. Re-run the `.venv/bin/pip install -e ...` command above inside WSL.
WSL2 is still useful for local development, but do not rely on it for the WSL2 sits behind Windows NAT and is **not directly reachable** from other LAN machines.
"another machine connects back to this node" LAN case. WSL2 commonly sits behind Direct cross-host hops time out. The relay path (see below) solves this: the WSL2 node
Windows NAT/port-proxy behavior and may not accept inbound traffic from other LAN opens an outbound WebSocket to the relay server and all traffic flows through that tunnel.
machines without extra host networking setup. We intentionally leave that unfixed No firewall rules, no `--advertise-host` needed — just point at the public tracker URL.
because it is useful for testing NAT/relay scenarios. If you just want to bring up
a Windows node that other machines can reach directly, run the node in native
Windows PowerShell instead.
### Native Windows PowerShell node (not WSL) ### Native Windows PowerShell node (not WSL)
@@ -126,17 +123,21 @@ $env:HF_HOME = "D:\DEV\models"
--port 8005 --port 8005
``` ```
One-line variants: One-line variants (direct LAN — node must be reachable by IP from other machines):
```powershell ```powershell
// only this works - when not behind NAT(wsl), and via IP. Revisit when relaying/RPC is implemented .\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
.\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20 ```
.\.venv\Scripts\meshnet-node.exe start --tracker http://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
Via public hostname with relay (works from behind NAT, WSL2, 5G — no `--advertise-host` needed):
```powershell
.\.venv\Scripts\meshnet-node.exe start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
``` ```
`--host 0.0.0.0` binds the node to all Windows interfaces. `--advertise-host` `--host 0.0.0.0` binds the node to all Windows interfaces. `--advertise-host`
is what the tracker gives to other nodes, so it must be the Windows LAN IP that is what the tracker gives to other nodes for direct connections; omit it when using
the tracker and peer nodes can actually reach. the relay path since all traffic flows through the relay tunnel instead.
If you want verbose per-hop pipeline logs while debugging a split model, add If you want verbose per-hop pipeline logs while debugging a split model, add
`--debug`. Leave it off for normal runs; otherwise every generated token logs `--debug`. Leave it off for normal runs; otherwise every generated token logs
@@ -145,6 +146,7 @@ lines like:
```text ```text
[node] pipeline hop 0: http://127.0.0.1:8005 start_layer=22 [node] pipeline hop 0: http://127.0.0.1:8005 start_layer=22
[node] pipeline hop 0 returned text=' token' [node] pipeline hop 0 returned text=' token'
[node] pipeline hop 1: wss://ai.neuron.d-popov.com/rpc/abc123 relay start_layer=12
``` ```
6. From the tracker machine, verify Windows is reachable: 6. From the tracker machine, verify Windows is reachable:
@@ -158,27 +160,42 @@ connection reached the node process. If it times out or connection-refuses, chec
the Windows Firewall rule, `--host 0.0.0.0`, the selected LAN IP, and that the the Windows Firewall rule, `--host 0.0.0.0`, the selected LAN IP, and that the
node is still running. node is still running.
### Public tracker + WSS relay ---
For internet nodes, expose one public HTTPS host and proxy these paths: ## Public tracker + relay (internet / NAT nodes)
```text This setup lets nodes connect from anywhere — behind home NAT, 5G, WSL2, or
/v1/* -> meshnet-tracker, for registration, heartbeats, routing, and OpenAI requests on a different continent — without opening firewall ports.
/ws -> meshnet-relay, for outbound node gossip/bridge connections
/rpc/* -> meshnet-relay, for tracker-to-node relay requests ### Architecture
```
Client → HTTPS → ai.neuron.d-popov.com (nginx)
├─ /v1/* → meshnet-tracker :8081
├─ /ws → meshnet-relay :8765 (node persistent outbound WS)
└─ /rpc/* → meshnet-relay :8765 (caller opens WS per hop)
``` ```
Start the tracker with the public relay URL it should advertise: ### Start the relay and tracker (server side)
```bash ```bash
# Terminal 1 — relay (WebSocket hub)
.venv/bin/meshnet-relay --host 0.0.0.0 --port 8765 .venv/bin/meshnet-relay --host 0.0.0.0 --port 8765
# Terminal 2 — tracker (advertises relay URL to nodes)
.venv/bin/meshnet-tracker start \ .venv/bin/meshnet-tracker start \
--host 0.0.0.0 \ --host 0.0.0.0 \
--port 8081 \ --port 8081 \
--relay-url wss://ai.neuron.d-popov.com/ws --relay-url wss://ai.neuron.d-popov.com/ws
``` ```
Then a node only needs the public tracker address: The `--relay-url` flag embeds the relay address in `/v1/network/map`. Every node
queries that endpoint on startup and auto-connects if a relay URL is present.
### Start a node (any machine, any network)
No `--advertise-host` needed. The node discovers the relay URL from the tracker
and opens a persistent outbound WebSocket:
```bash ```bash
.venv/bin/meshnet-node start \ .venv/bin/meshnet-node start \
@@ -186,6 +203,84 @@ Then a node only needs the public tracker address:
--model Qwen/Qwen2.5-0.5B-Instruct --model Qwen/Qwen2.5-0.5B-Instruct
``` ```
Expected startup output (relay path):
```
Auto-detected 24 layers → shard 0–23
Relay connected — wss://ai.neuron.d-popov.com/rpc/abc1def2ef3f4567
================================
meshnet-node ready
Wallet: <address>
Model ID: Qwen/Qwen2.5-0.5B-Instruct
Shard: layers 0–23; 24 of 24
Quantization: bfloat16
Endpoint: http://172.29.104.23:7001
Node ID: <id>
Hardware: CPU
================================
```
The `Endpoint` shown is the local IP (unreachable from outside). Other nodes reach
this one via `wss://ai.neuron.d-popov.com/rpc/<peer_id>` instead.
### How relay hops work
When node A needs to forward activations to node B (behind NAT):
1. Tracker injects `X-Meshnet-Route` with `relay_addr` for each behind-NAT hop.
2. Node A opens a WebSocket to `wss://relay/rpc/{peer_id_B}`.
3. Relay forwards the `relay-http-request` envelope to Node B's persistent connection.
4. Node B processes `/forward` locally, returns `relay-http-response`.
5. Relay sends the response back to Node A over the same WebSocket.
6. Node A closes the WebSocket and continues the pipeline.
Binary activation tensors (bfloat16) are Base64-encoded through the relay JSON
protocol and decoded on both sides — no precision loss.
If the relay hop fails (relay down, peer disconnected), the node logs a warning and
falls back to a direct HTTP attempt before returning an error.
### Test from WSL2 using the public tracker
In WSL2 (which gets a `172.x.x.x` virtual IP — unreachable from other machines):
```bash
# WSL2 Terminal 1 — head node (layers 0–11, handles chat requests)
.venv/bin/meshnet-node start \
--tracker https://ai.neuron.d-popov.com \
--model Qwen/Qwen2.5-0.5B-Instruct \
--shard-start 0 --shard-end 11
# WSL2 Terminal 2 — tail node (layers 12–23)
.venv/bin/meshnet-node start \
--tracker https://ai.neuron.d-popov.com \
--model Qwen/Qwen2.5-0.5B-Instruct \
--shard-start 12 --shard-end 23
```
Both nodes connect to the relay automatically. When a chat request arrives at Node A,
it forwards activations to Node B via `wss://ai.neuron.d-popov.com/rpc/{peer_id_B}`.
Send inference through the tracker (which picks the head node and injects the route):
```bash
curl -s https://ai.neuron.d-popov.com/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen2.5-0.5B-Instruct",
"messages": [{"role": "user", "content": "What is 7 times 8?"}],
"stream": false
}' | python3 -m json.tool
```
Or send directly to Node A's local port (within WSL):
```bash
curl -s http://localhost:7001/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "Qwen/Qwen2.5-0.5B-Instruct", "messages": [{"role": "user", "content": "Hi"}]}'
```
--- ---
## Step 1 — Start the tracker (Terminal 1) ## Step 1 — Start the tracker (Terminal 1)
@@ -304,7 +399,7 @@ HF_HOME=/run/media/popov/d/DEV/models \
--port 8002 --port 8002
``` ```
Send the request to Node A — it tokenizes, runs layers 0–13, passes binary Send the request to Node A — it tokenizes, runs layers 0–11, passes binary
activations to Node B, and streams the final response back. activations to Node B, and streams the final response back.
--- ---
@@ -342,18 +437,6 @@ See `docs/TWO_MACHINE_TEST.md` (created by US-018).
--- ---
## Start the relay node (for NAT traversal)
```bash
.venv/bin/pip install -e packages/relay
.venv/bin/meshnet-relay --port 8765
```
Nodes behind NAT connect to the relay and advertise their relay address to the
tracker. See `docs/adr/0010-p2p-gossip-and-nat-relay.md`.
---
## Run all tests ## Run all tests
```bash ```bash