Files
neuron-tai/.scratch/distributed-inference-performance/issues/02-relay-session-compatibility.md
2026-07-10 01:30:07 +03:00

1.2 KiB

Status: ready-for-agent

02 — Persistent relay compatibility hardening

What to build

Harden the persistent /rpc/<peer> connection used by one Route Session. Preserve unique request correlation while allowing sequential binary and JSON requests on one socket. Handle peer disconnects, requester cancellation, legacy one-request relays, timeout cleanup, and generation-end close without leaking pending RPC entries or accidentally replaying a model mutation.

Acceptance criteria

  • A cached decode session uses one requester connection per relay Activation Seam and sends one unique request id per activation.
  • Legacy relays that close after one response fail over clearly without corrupting the Route Session or replaying an uncertain request.
  • Relay and bridge cleanup removes pending request state on normal close, cancellation, timeout, and peer disconnect.
  • Concurrent Route Sessions do not share a non-thread-safe socket; responses remain matched by request id.
  • Tests cover two sequential binary requests, JSON compatibility, close, timeout, disconnect, cancellation, and no leaked pending entries.

Blocked by

  • 01 — Baseline and profiling harness.