neuron-tai/.scratch/distributed-inference-performance/issues/04-seam-telemetry.md

Status: ready-for-agent

04 — Activation Seam telemetry and bounded progress reporting

What to build

Expose structured timing and byte counters for each Activation Seam while keeping per-token progress overhead bounded. Report route/session, phase, hop/node, queue wait, model time, encode/decode time, compression time, wire bytes, response bytes, and connection reuse. Aggregate high-cardinality events instead of flushing a log line for every token.

Acceptance criteria

• Generation Telemetry includes prefill/decode seam latency and rolling tokens/sec without changing token output or cache behavior.
• Every request can be correlated by stable Route Session plus unique activation request id.
• Counters are sampled or aggregated so telemetry work is bounded and does not perform network I/O in the model hot loop.
• Logs summarize decode progress by session and retain actionable failure context.
• Tests verify counters, aggregation cadence, and cleanup at session close.

Blocked by

• 01 — Baseline and profiling harness.