d1e75ddded
Move issues (01–29) and PRD from .scratch/distributed-inference-network/ into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path and rewrite docs/agents/issue-tracker.md to reflect the new layout. The distributed_inference_network.egg-info/docs/ mirror is a build artifact already covered by *.egg-info/ in .gitignore — not committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
45 lines
2.2 KiB
Markdown
45 lines
2.2 KiB
Markdown
# US-029 — Outbound relay client: NAT/internet pipeline hops
|
|
|
|
Status: done
|
|
Priority: Critical
|
|
Stage: Implemented
|
|
|
|
## Context
|
|
|
|
Nodes behind NAT (WSL2 with 172.x.x.x addresses, 5G mobile, home routers) register with the
|
|
tracker and include a `relay_addr` (`wss://relay/rpc/{peer_id}`). When the head node needs to
|
|
forward activations to such a peer, it currently fails because the direct HTTP endpoint is
|
|
unreachable.
|
|
|
|
The relay server (US-017) is already running and the node already opens a persistent outbound
|
|
WebSocket (`RelayHttpBridge`). What is missing is the *outbound caller side*: given a `relay_addr`,
|
|
open a per-hop WebSocket to the relay's `/rpc/{peer_id}` endpoint and send the activation through it.
|
|
|
|
## Protocol
|
|
|
|
```
|
|
Node A → WS connect wss://relay/rpc/{peer_id_B}
|
|
→ send JSON: {request_id, method, path, headers, body_base64}
|
|
Relay → forward as relay-http-request envelope to Node B's persistent WS
|
|
Node B → process /forward locally
|
|
→ send relay-http-response envelope back
|
|
Relay → resolve future, send response JSON to Node A
|
|
Node A ← {request_id, status, headers, body_base64}
|
|
```
|
|
|
|
Binary activations (bfloat16) are base64-encoded. No precision loss.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `_relay_hop(relay_addr, path, body, headers, timeout)` in `torch_server.py` — opens WS, sends, receives, returns `(status, headers_lower, body_bytes)`
|
|
- [ ] `_get_remaining_route` returns `list[dict]` with `relay_addr` field (was `list[tuple]`)
|
|
- [ ] `_run_downstream_pipeline` dispatches via `_relay_hop` when hop has `relay_addr`; falls back to direct HTTP if relay connection fails
|
|
- [ ] Tracker `_handle_proxy_chat` includes `relay_addr` in downstream hop dicts when node has one
|
|
- [ ] `relay_bridge._handle_request` decodes `body_base64`; response uses `body_base64` for `octet-stream` content
|
|
- [ ] All 157 tests pass (`python -m pytest`)
|
|
- [ ] QUICKSTART.md updated with relay NAT/internet architecture and test scenario
|
|
|
|
## WSL2 test scenario
|
|
|
|
Start two nodes in WSL2 pointing at the public tracker. Both get `172.x.x.x` endpoints (unreachable from outside). Both connect to relay automatically. Send an inference request through the tracker — activations flow via relay. No `--advertise-host` needed.
|