28 lines
1014 B
Markdown
28 lines
1014 B
Markdown
Status: ready-for-agent
|
|
|
|
# 06 — Activation framing and copy reduction
|
|
|
|
## What to build
|
|
|
|
Profile and reduce avoidable allocations while activation data crosses a seam:
|
|
binary frame assembly, header JSON, base64 metadata, CPU/GPU conversion, and
|
|
response decompression. Preserve the current binary wire contract and use
|
|
zero-copy or pooled buffers only where ownership and lifetime are explicit.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] The benchmark identifies copy/allocation cost separately from model and
|
|
network time.
|
|
- [ ] Decode hidden-state conversion has no unnecessary float32 round trip.
|
|
- [ ] Binary framing avoids base64 for activation bodies and does not retain
|
|
buffers after a request completes.
|
|
- [ ] Position/attention metadata is validated and encoded efficiently without
|
|
changing semantic headers or cache positions.
|
|
- [ ] A focused test proves byte-for-byte wire compatibility and stable output
|
|
tokens before and after the optimization.
|
|
|
|
## Blocked by
|
|
|
|
- 01 — Baseline and profiling harness.
|
|
|