issues, chat FPS; optimisations

This commit is contained in:
Dobromir Popov
2026-07-10 01:30:07 +03:00
parent 916f531e9d
commit f54ea100fb
17 changed files with 688 additions and 108 deletions

View File

@@ -0,0 +1,28 @@
Status: ready-for-agent
# 04 — Activation Seam telemetry and bounded progress reporting
## What to build
Expose structured timing and byte counters for each Activation Seam while
keeping per-token progress overhead bounded. Report route/session, phase,
hop/node, queue wait, model time, encode/decode time, compression time, wire
bytes, response bytes, and connection reuse. Aggregate high-cardinality events
instead of flushing a log line for every token.
## Acceptance criteria
- [ ] Generation Telemetry includes prefill/decode seam latency and rolling
tokens/sec without changing token output or cache behavior.
- [ ] Every request can be correlated by stable Route Session plus unique
activation request id.
- [ ] Counters are sampled or aggregated so telemetry work is bounded and does
not perform network I/O in the model hot loop.
- [ ] Logs summarize decode progress by session and retain actionable failure
context.
- [ ] Tests verify counters, aggregation cadence, and cleanup at session close.
## Blocked by
- 01 — Baseline and profiling harness.