issues, chat FPS; optimisations

docs/adr/0014-relay-outbound-client.md

@@ -1,6 +1,6 @@
 # ADR-0014: Relay outbound client for NAT/internet pipeline hops
 
-## Status: Accepted
+## Status: Accepted, amended 2026-07-10
 
 ## Context
 
@@ -25,20 +25,22 @@ of connection setup matters.
 
 ## Options considered
 
-**A. Relay hop (WebSocket per hop, chosen)**  
-Node A opens a WebSocket to `wss://relay/rpc/{peer_id_B}`, sends the activation,
-receives the response, closes. The relay's `_handle_rpc` forwards it to B's persistent
-connection via the existing `relay-http-request` envelope mechanism.
+**A. Relay hop (persistent per Route Session, chosen)**
+Node A opens a WebSocket to `wss://relay/rpc/{peer_id_B}`, sends activation requests
+sequentially for the Route Session, then closes it when generation ends. The relay's
+`_handle_rpc` forwards each request to B's persistent connection via the existing
+`relay-http-request` envelope mechanism.
 
 Pros: reuses the existing relay server unchanged. Each hop is independent; failures don't
 affect other requests.
 
-Cons: WebSocket connection setup adds ~50–150 ms per hop on a fast relay. For
-autoregressive inference (N tokens × M hops), this adds up.
+The original implementation opened and closed this socket per token. It was amended
+to retain one requester socket per downstream relay address for the generation, so
+connection setup is amortized across all tokens.
 
-**B. Persistent per-session tunnel**  
-Node A opens a persistent WebSocket to the relay for the duration of an inference session
-and multiplexes all token hops over it.
+**B. Multiplexed persistent tunnel**
+Node A sends multiple concurrent Route Sessions over a shared WebSocket and demultiplexes
+responses by request id.
 
 Pros: amortises connection setup across tokens.
 
@@ -53,15 +55,16 @@ traffic through the tracker would saturate it. Rejected.
 
 ## Decision
 
-Option A — per-hop WebSocket relay. Simple, reuses existing infrastructure, correct.
-Option B is noted as a future optimization when activation-path latency becomes the
-bottleneck.
+Option A — one sequential WebSocket per relayed Activation Seam and Route Session.
+Each activation still has a unique request id for response correlation, while
+`X-Meshnet-Session` remains stable for KV state. Option B remains a possible
+connection-count optimization for high-concurrency workloads.
 
 ## Protocol
 
 ```
-Node A opens WS → wss://relay/rpc/{peer_id_B}
-Node A sends:
+Node A opens WS once → wss://relay/rpc/{peer_id_B}
+Node A sends repeatedly:
   {
     "request_id": "<hex>",
     "method": "POST",
@@ -81,8 +84,8 @@ Response:
     # OR
     "body": "<json string>"                    ← for text (last-hop decode)
   }
-Relay sends response JSON back to Node A.
-Node A decodes body_base64, continues pipeline.
+Relay sends each response back to Node A without closing the requester socket.
+Node A continues the pipeline and closes the socket when generation ends.
 ```
 
 ### Binary data through JSON: base64
@@ -115,6 +118,6 @@ The head node reads `relay_addr` from the injected `X-Meshnet-Route` header and
 
 - Nodes behind NAT (WSL2, 5G, home routers) can now participate in distributed pipeline inference without opening firewall ports
 - `relay_addr` is a stable registration field; nodes without a relay omit it and receive direct HTTP hops
-- Per-hop WebSocket setup adds latency proportional to relay RTT; acceptable for prototype, optimize later with persistent tunnels
+- WebSocket/TCP/TLS setup occurs once per relayed Activation Seam per Route Session, not once per generated token
 - Base64 encoding increases payload size by ~33%; acceptable for prototype
 - The relay server remains stateless and horizontally scalable; only the persistent per-peer `/ws` connections are stateful