Files
neuron-tai/.scratch/distributed-inference-performance/issues/05-adaptive-compression.md
2026-07-10 01:30:07 +03:00

27 lines
953 B
Markdown

Status: ready-for-agent
# 05 — Trace-driven activation compression
## What to build
Make zstd decisions from activation size, measured compression ratio, CPU cost,
and route conditions. Keep small decode payloads on the fast path, avoid
compressing data that does not shrink, and expose enough counters to compare
wire savings against compression latency on LAN and relay routes.
## Acceptance criteria
- [ ] A compression policy is explicit and configurable for LAN, relay, and
benchmark environments.
- [ ] Bodies that do not meet the configured savings threshold are sent raw.
- [ ] Compression and decompression time plus input/output bytes are reported.
- [ ] Prefill and decode policies can differ; decode latency is not regressed by
compressing small one-step activations.
- [ ] Tests cover incompressible, compressible, threshold, malformed, and
legacy-uncompressed bodies.
## Blocked by
- 01 — Baseline and profiling harness.