# US-017 — P2P gossip, NAT-traversal relay node, and SSL/TLS ## Goal Nodes must work behind NAT (home routers, cloud VMs without public IPs) and must communicate securely. Implement: 1. **SSL/TLS everywhere** — all HTTP between nodes/tracker is HTTPS; all WebSocket gossip is WSS 2. **mDNS peer discovery** — nodes on the same LAN find each other automatically (no config) 3. **WebSocket gossip PubSub** — nodes propagate join/leave/coverage-update events in near-real-time 4. **Circuit relay node** — team-run public relay (`packages/relay`) that enables NAT traversal and bootstraps new nodes joining from the internet Architecture is designed to migrate to libp2p GossipSub + Kademlia DHT without breaking the message schema (topic names and payload formats are stable contracts). ## Gossip protocol ### Transport WebSocket (`wss://`) using the `websockets` Python library. Each node maintains persistent WSS connections to: - The relay node (always, bootstraps peer list) - Up to 8 direct peers (Kademlia-style target fanout; peers discovered via mDNS + relay peer list) ### Topics All messages are JSON with an envelope: ```json { "topic": "node-join", "version": 1, "from_peer": "", "timestamp": "", "payload": { ... } } ``` | Topic | Direction | Payload | |-------|-----------|---------| | `node-join` | broadcast | `{peer_id, addr, models: [{model_preset, shard_start, shard_end}], vram_gb, quant}` | | `node-leave` | broadcast | `{peer_id, reason}` | | `coverage-update` | broadcast | `{model_preset, coverage: [{start, end, count}]}` | | `heartbeat` | peer→relay | `{peer_id, addr, uptime_s, tokens_per_sec}` | | `peer-list` | relay→peer | `{peers: [{peer_id, addr}]}` | | `relay-announce` | relay→all | `{relay_id, relay_url, capacity}` | Gossip fanout: each node re-broadcasts received messages to all its peers (simple flooding with `seen_ids` dedup, TTL=3 hops). Migration to GossipSub mesh routing is a later ADR. ### Peer ID `peer_id = sha256(public_key)[:16].hex()` — generated on first run, stored in `~/.config/meshnet/identity.json`. The same keypair is used for TLS client certificates (mTLS) in future work. ## mDNS LAN discovery Use Python `zeroconf` library. Service type: `_meshnet._tcp.local.` ```python from zeroconf import ServiceInfo, Zeroconf info = ServiceInfo( "_meshnet._tcp.local.", f"{peer_id}._meshnet._tcp.local.", addresses=[socket.inet_aton(local_ip)], port=node_port, properties={"peer_id": peer_id, "version": "1"}, ) zc = Zeroconf() zc.register_service(info) ``` On startup, nodes also browse for `_meshnet._tcp.local.` to discover existing nodes. mDNS is LAN-only (does not traverse routers), which is correct for LAN discovery. ## NAT traversal: circuit relay ### How it works 1. Node A (behind NAT) cannot accept inbound TCP connections 2. Node A connects outbound to the public relay via WSS 3. Node A tells the tracker: `"effective_addr": "wss://relay.meshnet.ai/relay/{peer_id_A}"` 4. Node B (wants to call A) connects to the relay at the above URL 5. Relay proxies the TCP stream between A and B Hole-punching (direct connection via STUN) is attempted first (future work). Relay is the fallback. ### meshnet-relay `packages/relay/meshnet_relay/server.py` — a standalone aiohttp server: ``` GET /health → {status: ok} GET /v1/peers → [{peer_id, addr, last_seen}] POST /v1/gossip → receive a gossip message, fan out to connected peers WSS /ws → persistent gossip connection (subscribe to all topics) WSS /relay/{peer_id} → circuit relay proxy to that peer_id GET /v1/relay/capacity → {connected_peers: N, max_peers: 500} ``` CLI: ``` meshnet-relay [--port 8443] [--cert path/to/cert.pem] [--key path/to/key.pem] [--tracker-url http://...] [--max-peers 500] ``` The relay can optionally proxy to the tracker (so `relay.meshnet.ai` is the single internet-visible endpoint). ## SSL/TLS setup ### Node certificate (self-signed, auto-generated) On first run, `meshnet-node` generates a self-signed RSA-2048 cert valid for 10 years: ```python from cryptography import x509 from cryptography.hazmat.primitives import hashes, serialization from cryptography.hazmat.primitives.asymmetric import rsa ``` Cert saved to `~/.config/meshnet/node_cert.pem` + `node_key.pem`. Fingerprint stored in config and shared with tracker via heartbeat. Nodes connecting to each other validate the fingerprint (TOFU — trust on first use), not the CA chain. ### Relay certificate The relay uses a real Let's Encrypt cert (cert-bot or acme.sh). The relay cert is pinned in `packages/p2p/relay_bootstrap.json`: ```json { "relays": [ { "url": "wss://relay.meshnet.ai:8443", "cert_fingerprint": "sha256:", "operator": "meshnet-team" } ] } ``` ### All HTTP switched to HTTPS `meshnet-node` starts an HTTPS server using `ssl.SSLContext`. `meshnet-tracker` similarly. All outbound `httpx` / `aiohttp` calls use TLS verification against pinned fingerprints (not the system CA store — too many corporate proxies break this). ## Tracker changes Heartbeat payload gains new fields: ```json { "peer_id": "a1b2c3d4e5f6a1b2", "effective_addr": "https://192.168.1.42:8001", "relay_addr": "wss://relay.meshnet.ai:8443/relay/a1b2c3d4e5f6a1b2", "cert_fingerprint": "sha256:...", "gossip_peers": ["peer_id_1", "peer_id_2"] } ``` Tracker uses `effective_addr` (direct) or `relay_addr` (fallback) when building inference routes. ## Integration test ``` tests/test_gossip_and_relay.py scenario: 1. Start a local relay (localhost:18443) 2. Start node A (no inbound port — simulate NAT by binding to 127.0.0.1 only) 3. Start node B (public-reachable on localhost) 4. Both register with relay; relay peer-list includes both 5. Node B sends a gossip node-join message 6. Assert node A receives it within 500ms 7. Start tracker; confirm tracker's node registry includes node A via relay_addr 8. Send inference request; assert it routes through relay to node A ``` ## Package layout ``` packages/relay/ pyproject.toml meshnet_relay/ __init__.py server.py # aiohttp relay + gossip hub + circuit relay proxy cli.py # meshnet-relay entrypoint peer_registry.py # in-memory {peer_id: {addr, last_seen, ...}} circuit_relay.py # WSS proxy between two peers packages/p2p/ meshnet_p2p/ gossip.py # GossipClient — connect to relay + peers, pub/sub mdns.py # ZeroconfDiscovery — mDNS announce + browse identity.py # PeerIdentity — generate/load peer_id + keypair tls.py # cert generation, fingerprint, SSLContext helpers packages/node/meshnet_node/ gossip_integration.py # wires GossipClient into node lifecycle ``` ## Acceptance criteria - All node↔node and node↔tracker HTTP uses HTTPS; self-signed cert auto-generated on first run - `cert_fingerprint` included in heartbeat; tracker stores and logs it - mDNS: two nodes on the same LAN discover each other without manual tracker URL (test with two localhost processes using different mDNS names) - Relay: `meshnet-relay` starts, accepts WSS connections, fans out gossip messages to all connected peers - Circuit relay: node A (127.0.0.1-only) can receive a gossip message via the relay from node B - Tracker routes inference to node A using `relay_addr` when direct addr not reachable - `relay_bootstrap.json` exists in `packages/p2p/` with at least one entry (localhost for tests) - ADR-0010 documents the gossip architecture and libp2p migration path - `python -m pytest` passes from repo root - Commit only this story's changes