2b439e8a5f
Issue files (.scratch/issues/20-29): retrospective specs for all work done in the current sprint — hardening, route-timeout, start-layer protocol, heartbeat stats, availability map, rolling RPM, smart assignment, throughput routing, routing tests, relay outbound client. ADRs (docs/adr/0011-0014): 0011 — Auto-shard from memory budget and tracker network assignment 0012 — X-Meshnet-Start-Layer overlapping shard execution protocol 0013 — Rolling RPM statistics, smart assignment scoring, throughput routing 0014 — Relay outbound client for NAT/internet pipeline hops prd.json: US-020 through US-029 added, all marked done. ralph_progress.py now shows 29/29 complete (100%). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
45 lines
2.2 KiB
Markdown
45 lines
2.2 KiB
Markdown
# US-029 — Outbound relay client: NAT/internet pipeline hops
|
|
|
|
Status: done
|
|
Priority: Critical
|
|
Stage: Implemented
|
|
|
|
## Context
|
|
|
|
Nodes behind NAT (WSL2 with 172.x.x.x addresses, 5G mobile, home routers) register with the
|
|
tracker and include a `relay_addr` (`wss://relay/rpc/{peer_id}`). When the head node needs to
|
|
forward activations to such a peer, it currently fails because the direct HTTP endpoint is
|
|
unreachable.
|
|
|
|
The relay server (US-017) is already running and the node already opens a persistent outbound
|
|
WebSocket (`RelayHttpBridge`). What is missing is the *outbound caller side*: given a `relay_addr`,
|
|
open a per-hop WebSocket to the relay's `/rpc/{peer_id}` endpoint and send the activation through it.
|
|
|
|
## Protocol
|
|
|
|
```
|
|
Node A → WS connect wss://relay/rpc/{peer_id_B}
|
|
→ send JSON: {request_id, method, path, headers, body_base64}
|
|
Relay → forward as relay-http-request envelope to Node B's persistent WS
|
|
Node B → process /forward locally
|
|
→ send relay-http-response envelope back
|
|
Relay → resolve future, send response JSON to Node A
|
|
Node A ← {request_id, status, headers, body_base64}
|
|
```
|
|
|
|
Binary activations (bfloat16) are base64-encoded. No precision loss.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `_relay_hop(relay_addr, path, body, headers, timeout)` in `torch_server.py` — opens WS, sends, receives, returns `(status, headers_lower, body_bytes)`
|
|
- [ ] `_get_remaining_route` returns `list[dict]` with `relay_addr` field (was `list[tuple]`)
|
|
- [ ] `_run_downstream_pipeline` dispatches via `_relay_hop` when hop has `relay_addr`; falls back to direct HTTP if relay connection fails
|
|
- [ ] Tracker `_handle_proxy_chat` includes `relay_addr` in downstream hop dicts when node has one
|
|
- [ ] `relay_bridge._handle_request` decodes `body_base64`; response uses `body_base64` for `octet-stream` content
|
|
- [ ] All 157 tests pass (`python -m pytest`)
|
|
- [ ] QUICKSTART.md updated with relay NAT/internet architecture and test scenario
|
|
|
|
## WSL2 test scenario
|
|
|
|
Start two nodes in WSL2 pointing at the public tracker. Both get `172.x.x.x` endpoints (unreachable from outside). Both connect to relay automatically. Send an inference request through the tracker — activations flow via relay. No `--advertise-host` needed.
|