Merge branch 'master' of https://git.d-popov.com/popov/neuron-tai
This commit is contained in:
105
QUICKSTART.md
105
QUICKSTART.md
@@ -135,6 +135,13 @@ Via public hostname with relay (works from behind NAT, WSL2, 5G — no `--advert
|
|||||||
.\.venv\Scripts\meshnet-node.exe start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
|
.\.venv\Scripts\meshnet-node.exe start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
|
||||||
```
|
```
|
||||||
|
|
||||||
|
WSL (same relay path — no `--advertise-host`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/meshnet-node start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
|
||||||
|
.venv/bin/meshnet-node start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct
|
||||||
|
```
|
||||||
|
|
||||||
`--host 0.0.0.0` binds the node to all Windows interfaces. `--advertise-host`
|
`--host 0.0.0.0` binds the node to all Windows interfaces. `--advertise-host`
|
||||||
is what the tracker gives to other nodes for direct connections; omit it when using
|
is what the tracker gives to other nodes for direct connections; omit it when using
|
||||||
the relay path since all traffic flows through the relay tunnel instead.
|
the relay path since all traffic flows through the relay tunnel instead.
|
||||||
@@ -176,26 +183,114 @@ Client → HTTPS → ai.neuron.d-popov.com (nginx)
|
|||||||
└─ /rpc/* → meshnet-relay :8765 (caller opens WS per hop)
|
└─ /rpc/* → meshnet-relay :8765 (caller opens WS per hop)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Start the relay and tracker (server side)
|
### Nginx Proxy Manager (Docker)
|
||||||
|
|
||||||
|
Use **one** proxy host for the domain. Do not create a second host for the same
|
||||||
|
domain to reach another port — path routing is done with **Custom locations** on
|
||||||
|
that same host.
|
||||||
|
|
||||||
|
**1. Details tab** (default `/` → tracker)
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|--------|
|
||||||
|
| Domain Names | `ai.neuron.d-popov.com` |
|
||||||
|
| Scheme | `http` |
|
||||||
|
| Forward Hostname / IP | LAN IP of the tracker machine (e.g. `192.168.0.179`) |
|
||||||
|
| Forward Port | `8081` |
|
||||||
|
| Websockets Support | ON |
|
||||||
|
|
||||||
|
This serves `/v1/network/map`, `/v1/chat/completions`, and the rest of the tracker API.
|
||||||
|
|
||||||
|
**2. Custom locations tab** (sub-paths → relay)
|
||||||
|
|
||||||
|
The Custom locations form has **no separate Websockets toggle** — only location,
|
||||||
|
scheme, forward host, optional sub-folder path, and port. Add **two** locations
|
||||||
|
(both pointing at the relay process on port `8765`). Leave **“Add a path for
|
||||||
|
sub-folder forwarding”** empty so the full URI reaches the relay
|
||||||
|
(`/ws`, `/rpc/<peer_id>`).
|
||||||
|
|
||||||
|
Location A — persistent node connections:
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|--------|
|
||||||
|
| Define location | `/ws` |
|
||||||
|
| Scheme | `http` |
|
||||||
|
| Forward Hostname / IP | `192.168.0.179` |
|
||||||
|
| Forward Port | `8765` |
|
||||||
|
| Sub-folder path | *(leave empty)* |
|
||||||
|
|
||||||
|
Location B — per-hop RPC:
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|--------|
|
||||||
|
| Define location | `/rpc` |
|
||||||
|
| Scheme | `http` |
|
||||||
|
| Forward Hostname / IP | `192.168.0.179` |
|
||||||
|
| Forward Port | `8765` |
|
||||||
|
| Sub-folder path | *(leave empty)* |
|
||||||
|
|
||||||
|
Nginx matches the longer prefixes first: `/ws` and `/rpc/…` go to relay; everything
|
||||||
|
else stays on `8081`.
|
||||||
|
|
||||||
|
**3. SSL tab**
|
||||||
|
|
||||||
|
Use your existing Let’s Encrypt certificate (unchanged).
|
||||||
|
|
||||||
|
**4. Advanced tab** (only if WebSocket upgrade fails on `/ws` or `/rpc`)
|
||||||
|
|
||||||
|
Custom locations do not expose a Websockets checkbox. If nodes show
|
||||||
|
`Relay configured but not connected yet` while `/v1/network/map` works, add this
|
||||||
|
snippet on the **proxy host** Advanced tab:
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection $http_connection;
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
```
|
||||||
|
|
||||||
|
**5. Verify routing**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Terminal 1 — relay (WebSocket hub)
|
# Tracker (8081 via default location)
|
||||||
|
curl -s https://ai.neuron.d-popov.com/v1/network/map | python3 -m json.tool
|
||||||
|
|
||||||
|
# Relay paths should not 502/404 from the tracker — check response headers/status
|
||||||
|
curl -sI https://ai.neuron.d-popov.com/ws
|
||||||
|
curl -sI https://ai.neuron.d-popov.com/rpc/test-peer
|
||||||
|
```
|
||||||
|
|
||||||
|
After NPM is correct, start relay and tracker on the LAN machine:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Terminal 1 — relay
|
||||||
.venv/bin/meshnet-relay --host 0.0.0.0 --port 8765
|
.venv/bin/meshnet-relay --host 0.0.0.0 --port 8765
|
||||||
|
|
||||||
# Terminal 2 — tracker (advertises relay URL to nodes)
|
# Terminal 2 — tracker (advertises relay to nodes)
|
||||||
.venv/bin/meshnet-tracker start \
|
.venv/bin/meshnet-tracker start \
|
||||||
--host 0.0.0.0 \
|
--host 0.0.0.0 \
|
||||||
--port 8081 \
|
--port 8081 \
|
||||||
--relay-url wss://ai.neuron.d-popov.com/ws
|
--relay-url wss://ai.neuron.d-popov.com/ws
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Nodes using `https://ai.neuron.d-popov.com` should then log:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Relay advertised by tracker — using outbound tunnel wss://ai.neuron.d-popov.com/ws
|
||||||
|
Relay connected — wss://ai.neuron.d-popov.com/rpc/<peer_id>
|
||||||
|
```
|
||||||
|
|
||||||
The `--relay-url` flag embeds the relay address in `/v1/network/map`. Every node
|
The `--relay-url` flag embeds the relay address in `/v1/network/map`. Every node
|
||||||
queries that endpoint on startup and auto-connects if a relay URL is present.
|
queries that endpoint on startup and auto-connects if a relay URL is present.
|
||||||
|
|
||||||
### Start a node (any machine, any network)
|
### Start a node (any machine, any network)
|
||||||
|
|
||||||
No `--advertise-host` needed. The node discovers the relay URL from the tracker
|
No `--advertise-host`, firewall rule, port forwarding, relay URL, or peer URL is
|
||||||
and opens a persistent outbound WebSocket:
|
needed on the node. The public tracker is the only bootstrap URL the user types.
|
||||||
|
The node queries the tracker for `/v1/network/map`, discovers the relay URL, and
|
||||||
|
opens a persistent outbound WebSocket. If the relay connection drops, the node
|
||||||
|
keeps retrying it.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
.venv/bin/meshnet-node start \
|
.venv/bin/meshnet-node start \
|
||||||
|
|||||||
@@ -35,13 +35,32 @@ def _get_json(url: str, timeout: float = 10.0) -> dict:
|
|||||||
return json.loads(r.read())
|
return json.loads(r.read())
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_relay_url_from_tracker(tracker_url: str) -> str | None:
|
||||||
|
"""Infer relay WebSocket URL from a public HTTPS tracker origin.
|
||||||
|
|
||||||
|
Public deployments colocate relay at /ws on the same host as the tracker API
|
||||||
|
(see QUICKSTART nginx layout). Local LAN trackers use a separate relay port
|
||||||
|
and must advertise relay_url explicitly via /v1/network/map.
|
||||||
|
"""
|
||||||
|
parsed = urllib.parse.urlparse(tracker_url)
|
||||||
|
if parsed.scheme != "https":
|
||||||
|
return None
|
||||||
|
host = parsed.hostname
|
||||||
|
if not host or host in ("127.0.0.1", "localhost"):
|
||||||
|
return None
|
||||||
|
return f"wss://{parsed.netloc}/ws"
|
||||||
|
|
||||||
|
|
||||||
def _discover_relay_url(tracker_url: str) -> str | None:
|
def _discover_relay_url(tracker_url: str) -> str | None:
|
||||||
|
relay_url: str | None = None
|
||||||
try:
|
try:
|
||||||
network_map = _get_json(f"{tracker_url}/v1/network/map", timeout=5.0)
|
network_map = _get_json(f"{tracker_url}/v1/network/map", timeout=5.0)
|
||||||
|
raw = network_map.get("relay_url")
|
||||||
|
if isinstance(raw, str) and raw:
|
||||||
|
relay_url = raw
|
||||||
except Exception:
|
except Exception:
|
||||||
return None
|
pass
|
||||||
relay_url = network_map.get("relay_url")
|
return relay_url or _infer_relay_url_from_tracker(tracker_url)
|
||||||
return relay_url if isinstance(relay_url, str) and relay_url else None
|
|
||||||
|
|
||||||
|
|
||||||
def _start_relay_bridge_if_available(
|
def _start_relay_bridge_if_available(
|
||||||
@@ -49,8 +68,9 @@ def _start_relay_bridge_if_available(
|
|||||||
wallet_address: str,
|
wallet_address: str,
|
||||||
local_base_url: str,
|
local_base_url: str,
|
||||||
advertised_endpoint: str,
|
advertised_endpoint: str,
|
||||||
|
relay_url: str | None = None,
|
||||||
) -> tuple[RelayHttpBridge | None, dict]:
|
) -> tuple[RelayHttpBridge | None, dict]:
|
||||||
relay_url = _discover_relay_url(tracker_url)
|
relay_url = relay_url or _discover_relay_url(tracker_url)
|
||||||
if not relay_url:
|
if not relay_url:
|
||||||
return None, {}
|
return None, {}
|
||||||
peer_id = peer_id_from_wallet(wallet_address)
|
peer_id = peer_id_from_wallet(wallet_address)
|
||||||
@@ -222,7 +242,8 @@ def _warn_virtual_network_ip(ip: str | None) -> None:
|
|||||||
f" This range is used by Docker, WSL2, and virtual machines and is\n"
|
f" This range is used by Docker, WSL2, and virtual machines and is\n"
|
||||||
f" NOT reachable from other physical machines on your LAN.\n"
|
f" NOT reachable from other physical machines on your LAN.\n"
|
||||||
f" Cross-host pipeline hops WILL time out.\n"
|
f" Cross-host pipeline hops WILL time out.\n"
|
||||||
f" Fix: pass --advertise-host <your-LAN-ip> (e.g. 192.168.x.x).\n",
|
f" Fix: use a public tracker with relay (wss://…/ws), or pass\n"
|
||||||
|
f" --advertise-host <your-LAN-ip> (e.g. 192.168.x.x).\n",
|
||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -260,6 +281,7 @@ def run_startup(
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
tracker_url = tracker_url.rstrip("/")
|
tracker_url = tracker_url.rstrip("/")
|
||||||
|
relay_url = _discover_relay_url(tracker_url)
|
||||||
|
|
||||||
# 1. Hardware detection
|
# 1. Hardware detection
|
||||||
if advertise_host is None and host == "0.0.0.0":
|
if advertise_host is None and host == "0.0.0.0":
|
||||||
@@ -275,6 +297,9 @@ def run_startup(
|
|||||||
except Exception:
|
except Exception:
|
||||||
advertise_host = socket.getfqdn()
|
advertise_host = socket.getfqdn()
|
||||||
|
|
||||||
|
if relay_url:
|
||||||
|
print(f"Relay advertised by tracker — using outbound tunnel {relay_url}", flush=True)
|
||||||
|
else:
|
||||||
_warn_virtual_network_ip(advertise_host)
|
_warn_virtual_network_ip(advertise_host)
|
||||||
|
|
||||||
print("Detecting hardware...", flush=True)
|
print("Detecting hardware...", flush=True)
|
||||||
@@ -361,6 +386,7 @@ def run_startup(
|
|||||||
address,
|
address,
|
||||||
local_base_url,
|
local_base_url,
|
||||||
endpoint,
|
endpoint,
|
||||||
|
relay_url=relay_url,
|
||||||
)
|
)
|
||||||
_attach_relay_bridge(node, relay_bridge)
|
_attach_relay_bridge(node, relay_bridge)
|
||||||
# Register with tracker so other nodes can auto-join this model.
|
# Register with tracker so other nodes can auto-join this model.
|
||||||
@@ -451,6 +477,7 @@ def run_startup(
|
|||||||
address,
|
address,
|
||||||
local_base_url,
|
local_base_url,
|
||||||
endpoint,
|
endpoint,
|
||||||
|
relay_url=relay_url,
|
||||||
)
|
)
|
||||||
_attach_relay_bridge(node, relay_bridge)
|
_attach_relay_bridge(node, relay_bridge)
|
||||||
auto_reg_payload = {
|
auto_reg_payload = {
|
||||||
@@ -548,6 +575,7 @@ def run_startup(
|
|||||||
address,
|
address,
|
||||||
local_base_url,
|
local_base_url,
|
||||||
endpoint,
|
endpoint,
|
||||||
|
relay_url=relay_url,
|
||||||
)
|
)
|
||||||
_attach_relay_bridge(node, relay_bridge)
|
_attach_relay_bridge(node, relay_bridge)
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import argparse
|
|||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
|
|
||||||
from .server import TrackerServer
|
from .server import TrackerServer, derive_relay_url_from_public_tracker_url
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
@@ -52,6 +52,7 @@ def main() -> None:
|
|||||||
|
|
||||||
if args.command in {None, "start"}:
|
if args.command in {None, "start"}:
|
||||||
cluster_peers = [u.strip() for u in args.cluster_peers.split(",") if u.strip()]
|
cluster_peers = [u.strip() for u in args.cluster_peers.split(",") if u.strip()]
|
||||||
|
relay_url = args.relay_url or derive_relay_url_from_public_tracker_url(args.self_url)
|
||||||
server = TrackerServer(
|
server = TrackerServer(
|
||||||
host=args.host,
|
host=args.host,
|
||||||
port=args.port,
|
port=args.port,
|
||||||
@@ -59,7 +60,7 @@ def main() -> None:
|
|||||||
cluster_peers=cluster_peers or None,
|
cluster_peers=cluster_peers or None,
|
||||||
cluster_self_url=args.self_url,
|
cluster_self_url=args.self_url,
|
||||||
stats_db=getattr(args, "stats_db", None),
|
stats_db=getattr(args, "stats_db", None),
|
||||||
relay_url=args.relay_url,
|
relay_url=relay_url,
|
||||||
)
|
)
|
||||||
port = server.start()
|
port = server.start()
|
||||||
print(f"meshnet-tracker listening on http://{args.host}:{port}", flush=True)
|
print(f"meshnet-tracker listening on http://{args.host}:{port}", flush=True)
|
||||||
|
|||||||
@@ -37,6 +37,19 @@ from .gossip import NodeGossip
|
|||||||
from .raft import RaftNode
|
from .raft import RaftNode
|
||||||
|
|
||||||
|
|
||||||
|
def derive_relay_url_from_public_tracker_url(url: str | None) -> str | None:
|
||||||
|
"""Return wss://host/ws when url is a public HTTPS tracker origin."""
|
||||||
|
if not url:
|
||||||
|
return None
|
||||||
|
parsed = urllib.parse.urlparse(url)
|
||||||
|
if parsed.scheme != "https":
|
||||||
|
return None
|
||||||
|
host = parsed.hostname
|
||||||
|
if not host or host in ("127.0.0.1", "localhost"):
|
||||||
|
return None
|
||||||
|
return f"wss://{parsed.netloc}/ws"
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_MODEL_PRESETS: dict[str, dict] = {
|
DEFAULT_MODEL_PRESETS: dict[str, dict] = {
|
||||||
"stub-model": {
|
"stub-model": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
@@ -1228,6 +1241,33 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
if auth:
|
if auth:
|
||||||
req.add_header("Authorization", auth)
|
req.add_header("Authorization", auth)
|
||||||
|
|
||||||
|
relay_headers = {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"X-Meshnet-Route": downstream_urls,
|
||||||
|
**({"Authorization": auth} if auth else {}),
|
||||||
|
}
|
||||||
|
|
||||||
|
if node.relay_addr:
|
||||||
|
print(
|
||||||
|
f"[tracker] proxy via relay {request_id}: {node.relay_addr} "
|
||||||
|
f"(direct endpoint {target_url})",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
relayed = _relay_http_request(
|
||||||
|
node.relay_addr,
|
||||||
|
path="/v1/chat/completions",
|
||||||
|
body=raw_body,
|
||||||
|
headers=relay_headers,
|
||||||
|
)
|
||||||
|
if relayed is not None:
|
||||||
|
self._send_relayed_response(relayed)
|
||||||
|
return
|
||||||
|
print(
|
||||||
|
f"[tracker] relay proxy failed {request_id}: {node.relay_addr}; "
|
||||||
|
f"trying direct {target_url}",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
upstream = urllib.request.urlopen(req, timeout=300.0)
|
upstream = urllib.request.urlopen(req, timeout=300.0)
|
||||||
print(f"[tracker] proxy connected {request_id}: {target_url}", flush=True)
|
print(f"[tracker] proxy connected {request_id}: {target_url}", flush=True)
|
||||||
@@ -1247,22 +1287,10 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
if node.relay_addr:
|
if node.relay_addr:
|
||||||
print(
|
print(
|
||||||
f"[tracker] direct proxy failed {request_id}: {target_url}: {exc}; "
|
f"[tracker] direct proxy failed {request_id}: {target_url}: {exc}; "
|
||||||
f"trying relay {node.relay_addr}",
|
f"relay already attempted for {node.relay_addr}",
|
||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
relayed = _relay_http_request(
|
else:
|
||||||
node.relay_addr,
|
|
||||||
path="/v1/chat/completions",
|
|
||||||
body=raw_body,
|
|
||||||
headers={
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
"X-Meshnet-Route": downstream_urls,
|
|
||||||
**({"Authorization": auth} if auth else {}),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
if relayed is not None:
|
|
||||||
self._send_relayed_response(relayed)
|
|
||||||
return
|
|
||||||
print(f"[tracker] proxy failed {request_id}: {target_url}: {exc}", flush=True)
|
print(f"[tracker] proxy failed {request_id}: {target_url}: {exc}", flush=True)
|
||||||
self._send_json(503, {"error": {
|
self._send_json(503, {"error": {
|
||||||
"message": f"upstream node unreachable: {exc}",
|
"message": f"upstream node unreachable: {exc}",
|
||||||
@@ -2040,6 +2068,11 @@ class TrackerServer:
|
|||||||
if self._server is not None:
|
if self._server is not None:
|
||||||
raise RuntimeError("TrackerServer is already running")
|
raise RuntimeError("TrackerServer is already running")
|
||||||
|
|
||||||
|
effective_relay_url = (
|
||||||
|
self._relay_url
|
||||||
|
or derive_relay_url_from_public_tracker_url(self._cluster_self_url)
|
||||||
|
)
|
||||||
|
|
||||||
# Start HTTP server first so we know our port
|
# Start HTTP server first so we know our port
|
||||||
self._server = _TrackerHTTPServer(
|
self._server = _TrackerHTTPServer(
|
||||||
(self._host, self._requested_port),
|
(self._host, self._requested_port),
|
||||||
@@ -2050,7 +2083,7 @@ class TrackerServer:
|
|||||||
self._model_presets,
|
self._model_presets,
|
||||||
self._contracts,
|
self._contracts,
|
||||||
self._minimum_stake,
|
self._minimum_stake,
|
||||||
relay_url=self._relay_url,
|
relay_url=effective_relay_url,
|
||||||
stats=self._stats,
|
stats=self._stats,
|
||||||
)
|
)
|
||||||
self.port = self._server.server_address[1]
|
self.port = self._server.server_address[1]
|
||||||
|
|||||||
@@ -350,6 +350,51 @@ def test_relay_rpc_round_trips_http_request_to_peer():
|
|||||||
assert json.loads(response["body"]) == {"ok": True, "path": "/v1/chat/completions"}
|
assert json.loads(response["body"]) == {"ok": True, "path": "/v1/chat/completions"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_node_relay_bridge_reconnects_after_failed_connection(monkeypatch):
|
||||||
|
"""Node-side relay bridge keeps retrying its outbound WebSocket connection."""
|
||||||
|
import websockets.sync.client as wsc # type: ignore[import]
|
||||||
|
from meshnet_node.relay_bridge import RelayHttpBridge
|
||||||
|
|
||||||
|
attempts = []
|
||||||
|
|
||||||
|
class FakeWebSocket:
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, exc_type, exc, tb):
|
||||||
|
return False
|
||||||
|
|
||||||
|
def send(self, _message):
|
||||||
|
pass
|
||||||
|
|
||||||
|
def recv(self, timeout=1):
|
||||||
|
raise TimeoutError
|
||||||
|
|
||||||
|
def fake_connect(url, open_timeout=5):
|
||||||
|
attempts.append((url, open_timeout))
|
||||||
|
if len(attempts) == 1:
|
||||||
|
raise OSError("temporary relay outage")
|
||||||
|
return FakeWebSocket()
|
||||||
|
|
||||||
|
monkeypatch.setattr(wsc, "connect", fake_connect)
|
||||||
|
|
||||||
|
bridge = RelayHttpBridge(
|
||||||
|
relay_url="ws://relay.example/ws",
|
||||||
|
peer_id="peer-reconnect",
|
||||||
|
local_base_url="http://127.0.0.1:8001",
|
||||||
|
advertised_addr="http://127.0.0.1:8001",
|
||||||
|
reconnect_interval=0.01,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
info = bridge.start()
|
||||||
|
assert info.relay_addr == "ws://relay.example/rpc/peer-reconnect"
|
||||||
|
assert bridge.wait_connected(timeout=1.0)
|
||||||
|
finally:
|
||||||
|
bridge.stop()
|
||||||
|
|
||||||
|
assert len(attempts) >= 2
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Tracker gossip fields tests
|
# Tracker gossip fields tests
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -389,6 +434,27 @@ def _start_tracker_and_register(extra_fields: dict) -> dict:
|
|||||||
return resp
|
return resp
|
||||||
|
|
||||||
|
|
||||||
|
def test_tracker_derives_relay_url_from_public_self_url():
|
||||||
|
from meshnet_tracker.server import TrackerServer, derive_relay_url_from_public_tracker_url
|
||||||
|
|
||||||
|
assert derive_relay_url_from_public_tracker_url("https://ai.neuron.d-popov.com") == (
|
||||||
|
"wss://ai.neuron.d-popov.com/ws"
|
||||||
|
)
|
||||||
|
assert derive_relay_url_from_public_tracker_url("http://127.0.0.1:8081") is None
|
||||||
|
|
||||||
|
tracker = TrackerServer(cluster_self_url="https://ai.neuron.d-popov.com")
|
||||||
|
port = tracker.start()
|
||||||
|
try:
|
||||||
|
import json as _json
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
with urllib.request.urlopen(f"http://127.0.0.1:{port}/v1/network/map", timeout=5) as resp:
|
||||||
|
body = _json.loads(resp.read())
|
||||||
|
assert body["relay_url"] == "wss://ai.neuron.d-popov.com/ws"
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
def test_tracker_accepts_relay_addr_in_registration():
|
def test_tracker_accepts_relay_addr_in_registration():
|
||||||
resp = _start_tracker_and_register({
|
resp = _start_tracker_and_register({
|
||||||
"relay_addr": "ws://relay.meshnet.ai:8765/relay/abc123",
|
"relay_addr": "ws://relay.meshnet.ai:8765/relay/abc123",
|
||||||
|
|||||||
@@ -11,7 +11,11 @@ import pytest
|
|||||||
|
|
||||||
from meshnet_node.downloader import download_shard, write_shard_archive
|
from meshnet_node.downloader import download_shard, write_shard_archive
|
||||||
from meshnet_node.hardware import detect_hardware
|
from meshnet_node.hardware import detect_hardware
|
||||||
from meshnet_node.startup import _probationary_status_line, run_startup
|
from meshnet_node.startup import (
|
||||||
|
_infer_relay_url_from_tracker,
|
||||||
|
_probationary_status_line,
|
||||||
|
run_startup,
|
||||||
|
)
|
||||||
from meshnet_node.wallet import _b58encode, load_or_create_wallet
|
from meshnet_node.wallet import _b58encode, load_or_create_wallet
|
||||||
from meshnet_contracts import LocalSolanaContracts
|
from meshnet_contracts import LocalSolanaContracts
|
||||||
from meshnet_tracker.server import TrackerServer
|
from meshnet_tracker.server import TrackerServer
|
||||||
@@ -347,6 +351,94 @@ def test_tracker_assign_lists_peers_for_same_model_shard():
|
|||||||
tracker.stop()
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def test_infer_relay_url_from_public_https_tracker():
|
||||||
|
assert _infer_relay_url_from_tracker("https://ai.neuron.d-popov.com") == (
|
||||||
|
"wss://ai.neuron.d-popov.com/ws"
|
||||||
|
)
|
||||||
|
assert _infer_relay_url_from_tracker("https://ai.neuron.d-popov.com/v1/network/map") == (
|
||||||
|
"wss://ai.neuron.d-popov.com/ws"
|
||||||
|
)
|
||||||
|
assert _infer_relay_url_from_tracker("http://192.168.0.179:8081") is None
|
||||||
|
assert _infer_relay_url_from_tracker("http://127.0.0.1:8081") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_public_https_tracker_infers_relay_when_network_map_omits_relay_url(
|
||||||
|
tmp_path,
|
||||||
|
monkeypatch,
|
||||||
|
capsys,
|
||||||
|
):
|
||||||
|
"""Nodes bootstrap relay from the tracker origin when map relay_url is null."""
|
||||||
|
import meshnet_node.startup as startup_mod
|
||||||
|
|
||||||
|
class FakeBackend:
|
||||||
|
total_layers = 24
|
||||||
|
|
||||||
|
class FakeTorchNodeServer:
|
||||||
|
def __init__(self, **kwargs):
|
||||||
|
self.backend = FakeBackend()
|
||||||
|
self.port = None
|
||||||
|
self.chat_completion_count = 0
|
||||||
|
self.total_requests = 0
|
||||||
|
self.failed_requests = 0
|
||||||
|
self.queue_depth = 0
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
self.port = 8001
|
||||||
|
return self.port
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
class FakeRelayHttpBridge:
|
||||||
|
def __init__(self, relay_url, peer_id, local_base_url, advertised_addr):
|
||||||
|
self.relay_url = relay_url
|
||||||
|
self.peer_id = peer_id
|
||||||
|
|
||||||
|
@property
|
||||||
|
def relay_addr(self):
|
||||||
|
return f"{self.relay_url.replace('/ws', '')}/rpc/{self.peer_id}"
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
return types.SimpleNamespace(peer_id=self.peer_id, relay_addr=self.relay_addr)
|
||||||
|
|
||||||
|
def wait_connected(self, timeout=5.0):
|
||||||
|
return True
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
startup_mod,
|
||||||
|
"detect_hardware",
|
||||||
|
lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(startup_mod, "_detect_num_layers", lambda _model_id: 24)
|
||||||
|
monkeypatch.setattr(startup_mod, "TorchNodeServer", FakeTorchNodeServer)
|
||||||
|
monkeypatch.setattr(startup_mod, "RelayHttpBridge", FakeRelayHttpBridge)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
startup_mod,
|
||||||
|
"_get_json",
|
||||||
|
lambda url, timeout=10.0: {"relay_url": None, "nodes": []},
|
||||||
|
)
|
||||||
|
|
||||||
|
tracker_url = "https://ai.neuron.d-popov.com"
|
||||||
|
node = run_startup(
|
||||||
|
tracker_url=tracker_url,
|
||||||
|
model_id="Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
advertise_host="172.29.104.23",
|
||||||
|
wallet_path=tmp_path / "wallet.json",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
node.stop()
|
||||||
|
|
||||||
|
output = capsys.readouterr().out
|
||||||
|
assert "Relay advertised by tracker" in output
|
||||||
|
assert "wss://ai.neuron.d-popov.com/ws" in output
|
||||||
|
assert "Cross-host pipeline hops WILL time out" not in output
|
||||||
|
|
||||||
|
|
||||||
def test_real_model_startup_summary_shows_total_layers(tmp_path, monkeypatch, capsys):
|
def test_real_model_startup_summary_shows_total_layers(tmp_path, monkeypatch, capsys):
|
||||||
"""Real-model startup summary prints the shard range plus total model layers."""
|
"""Real-model startup summary prints the shard range plus total model layers."""
|
||||||
import meshnet_node.startup as startup_mod
|
import meshnet_node.startup as startup_mod
|
||||||
@@ -394,6 +486,7 @@ def test_real_model_startup_summary_shows_total_layers(tmp_path, monkeypatch, ca
|
|||||||
def test_public_tracker_model_node_registers_relay_metadata_from_tracker_url_only(
|
def test_public_tracker_model_node_registers_relay_metadata_from_tracker_url_only(
|
||||||
tmp_path,
|
tmp_path,
|
||||||
monkeypatch,
|
monkeypatch,
|
||||||
|
capsys,
|
||||||
):
|
):
|
||||||
"""A node only needs the public tracker URL to discover relay metadata and register."""
|
"""A node only needs the public tracker URL to discover relay metadata and register."""
|
||||||
import meshnet_node.startup as startup_mod
|
import meshnet_node.startup as startup_mod
|
||||||
@@ -457,7 +550,6 @@ def test_public_tracker_model_node_registers_relay_metadata_from_tracker_url_onl
|
|||||||
node = run_startup(
|
node = run_startup(
|
||||||
tracker_url=tracker_url,
|
tracker_url=tracker_url,
|
||||||
model_id="Qwen/Qwen2.5-0.5B-Instruct",
|
model_id="Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
advertise_host="203.0.113.10",
|
|
||||||
wallet_path=tmp_path / "wallet.json",
|
wallet_path=tmp_path / "wallet.json",
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
@@ -471,9 +563,91 @@ def test_public_tracker_model_node_registers_relay_metadata_from_tracker_url_onl
|
|||||||
assert len(network_map["nodes"]) == 1
|
assert len(network_map["nodes"]) == 1
|
||||||
registered = network_map["nodes"][0]
|
registered = network_map["nodes"][0]
|
||||||
assert registered["hf_repo"] == "Qwen/Qwen2.5-0.5B-Instruct"
|
assert registered["hf_repo"] == "Qwen/Qwen2.5-0.5B-Instruct"
|
||||||
assert registered["endpoint"] == "http://203.0.113.10:8001"
|
assert registered["endpoint"].startswith("http://")
|
||||||
|
assert registered["endpoint"].endswith(":8001")
|
||||||
assert registered["relay_addr"].startswith("ws://public-relay.example/rpc/")
|
assert registered["relay_addr"].startswith("ws://public-relay.example/rpc/")
|
||||||
assert registered["peer_id"]
|
assert registered["peer_id"]
|
||||||
|
output = capsys.readouterr().out
|
||||||
|
assert "Relay advertised by tracker" in output
|
||||||
|
assert "Cross-host pipeline hops WILL time out" not in output
|
||||||
|
|
||||||
|
|
||||||
|
def test_public_tracker_relay_suppresses_virtual_ip_warning(
|
||||||
|
tmp_path,
|
||||||
|
monkeypatch,
|
||||||
|
capsys,
|
||||||
|
):
|
||||||
|
"""A WSL/Docker endpoint is acceptable when the tracker advertises relay RPC."""
|
||||||
|
import meshnet_node.startup as startup_mod
|
||||||
|
|
||||||
|
class FakeBackend:
|
||||||
|
total_layers = 24
|
||||||
|
|
||||||
|
class FakeTorchNodeServer:
|
||||||
|
def __init__(self, **kwargs):
|
||||||
|
self.backend = FakeBackend()
|
||||||
|
self.port = None
|
||||||
|
self.chat_completion_count = 0
|
||||||
|
self.total_requests = 0
|
||||||
|
self.failed_requests = 0
|
||||||
|
self.queue_depth = 0
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
self.port = 8001
|
||||||
|
return self.port
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
class FakeRelayHttpBridge:
|
||||||
|
def __init__(self, relay_url, peer_id, local_base_url, advertised_addr):
|
||||||
|
self.relay_url = relay_url
|
||||||
|
self.peer_id = peer_id
|
||||||
|
|
||||||
|
@property
|
||||||
|
def relay_addr(self):
|
||||||
|
return f"ws://public-relay.example/rpc/{self.peer_id}"
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
return types.SimpleNamespace(peer_id=self.peer_id, relay_addr=self.relay_addr)
|
||||||
|
|
||||||
|
def wait_connected(self, timeout=5.0):
|
||||||
|
return True
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
startup_mod,
|
||||||
|
"detect_hardware",
|
||||||
|
lambda: {"device": "cpu", "gpu_name": None, "vram_mb": 0},
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(startup_mod, "_detect_num_layers", lambda _model_id: 24)
|
||||||
|
monkeypatch.setattr(startup_mod, "TorchNodeServer", FakeTorchNodeServer)
|
||||||
|
monkeypatch.setattr(startup_mod, "RelayHttpBridge", FakeRelayHttpBridge)
|
||||||
|
|
||||||
|
tracker = TrackerServer(relay_url="ws://public-relay.example/ws")
|
||||||
|
tracker_port = tracker.start()
|
||||||
|
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
||||||
|
try:
|
||||||
|
node = run_startup(
|
||||||
|
tracker_url=tracker_url,
|
||||||
|
model_id="Qwen/Qwen2.5-0.5B-Instruct",
|
||||||
|
advertise_host="172.29.104.23",
|
||||||
|
wallet_path=tmp_path / "wallet.json",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
network_map = _get_json(f"{tracker_url}/v1/network/map")
|
||||||
|
finally:
|
||||||
|
node.stop()
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
|
||||||
|
assert network_map["nodes"][0]["endpoint"] == "http://172.29.104.23:8001"
|
||||||
|
assert network_map["nodes"][0]["relay_addr"].startswith("ws://public-relay.example/rpc/")
|
||||||
|
output = capsys.readouterr().out
|
||||||
|
assert "Relay advertised by tracker" in output
|
||||||
|
assert "Cross-host pipeline hops WILL time out" not in output
|
||||||
|
|
||||||
|
|
||||||
def test_later_node_auto_joins_existing_public_hf_model_with_only_tracker_url(
|
def test_later_node_auto_joins_existing_public_hf_model_with_only_tracker_url(
|
||||||
|
|||||||
Reference in New Issue
Block a user