neuron-tai/.scratch/distributed-inference-performance/issues/06-activation-framing-copies.md

Status: ready-for-agent

06 — Activation framing and copy reduction

What to build

Profile and reduce avoidable allocations while activation data crosses a seam: binary frame assembly, header JSON, base64 metadata, CPU/GPU conversion, and response decompression. Preserve the current binary wire contract and use zero-copy or pooled buffers only where ownership and lifetime are explicit.

Acceptance criteria

• The benchmark identifies copy/allocation cost separately from model and network time.
• Decode hidden-state conversion has no unnecessary float32 round trip.
• Binary framing avoids base64 for activation bodies and does not retain buffers after a request completes.
• Position/attention metadata is validated and encoded efficiently without changing semantic headers or cache positions.
• A focused test proves byte-for-byte wire compatibility and stable output tokens before and after the optimization.

Blocked by

• 01 — Baseline and profiling harness.