28 lines
1.0 KiB
Markdown
28 lines
1.0 KiB
Markdown
Status: ready-for-agent
|
|
|
|
# 01 — Baseline and profiling harness
|
|
|
|
## What to build
|
|
|
|
Create a deterministic stub-backed benchmark for a Route Session that measures
|
|
prefill and cached decode across direct and relay paths. Attribute time to model
|
|
execution, activation encoding/decoding, compression, connection setup, relay
|
|
queueing, local HTTP forwarding, and end-to-end seam latency. Record payload
|
|
sizes and connection counts without requiring a real model or external host.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] The harness runs a fixed prompt and fixed generated-token count through a
|
|
two-node route in direct and relay modes.
|
|
- [ ] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
|
|
compression ratio, connection attempts, and queue wait.
|
|
- [ ] It distinguishes prefill from decode and cached from stateless mode.
|
|
- [ ] It emits machine-readable JSON suitable for CI artifacts and a concise
|
|
human-readable summary.
|
|
- [ ] A test fixture can assert connection attempts and output token identity.
|
|
|
|
## Blocked by
|
|
|
|
None - can start immediately.
|
|
|