issues, chat FPS; optimisations
This commit is contained in:
@@ -0,0 +1,28 @@
|
||||
Status: ready-for-agent
|
||||
|
||||
# 04 — Activation Seam telemetry and bounded progress reporting
|
||||
|
||||
## What to build
|
||||
|
||||
Expose structured timing and byte counters for each Activation Seam while
|
||||
keeping per-token progress overhead bounded. Report route/session, phase,
|
||||
hop/node, queue wait, model time, encode/decode time, compression time, wire
|
||||
bytes, response bytes, and connection reuse. Aggregate high-cardinality events
|
||||
instead of flushing a log line for every token.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Generation Telemetry includes prefill/decode seam latency and rolling
|
||||
tokens/sec without changing token output or cache behavior.
|
||||
- [ ] Every request can be correlated by stable Route Session plus unique
|
||||
activation request id.
|
||||
- [ ] Counters are sampled or aggregated so telemetry work is bounded and does
|
||||
not perform network I/O in the model hot loop.
|
||||
- [ ] Logs summarize decode progress by session and retain actionable failure
|
||||
context.
|
||||
- [ ] Tests verify counters, aggregation cadence, and cleanup at session close.
|
||||
|
||||
## Blocked by
|
||||
|
||||
- 01 — Baseline and profiling harness.
|
||||
|
||||
Reference in New Issue
Block a user