popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

KC cache task

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-08 21:05:16 +02:00
parent 0ffd195fec
commit 436e872abe
3 changed files with 80 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
.scratch/alpha-hardening/README.md
View File

@@ -11,6 +11,8 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
**Resume task (2026-07-07):** [24 - Routing telemetry resume](./issues/24-routing-telemetry-resume.md) is `ready-for-agent`. Learned-routing commit `518c259` is already present; dirty tree contains current-request heartbeat/dashboard telemetry and a known import-time annotation crash in `server.py:1490`.
**Perf follow-up (2026-07-08):** [25 — Sharded per-node KV cache for distributed generation](./issues/25-per-node-kv-cache-distributed.md) is `ready-for-agent`. The ADR-0020 mixed-topology `start_layer` bug is fixed, but the distributed generation loop still has no KV cache at all — every step re-encodes the full sequence and re-runs every layer on every node, causing quadratic tps decay observed live (22.3 → 12.6 tps over one generation on a 2-node Qwen2.5-0.5B pipeline). Must be architecture-aware: Qwen3.6's hybrid linear-attention layers cache recurrent conv/delta state, not standard K/V.
## Artifacts
| Path | Status |
@@ -18,7 +20,7 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
| [research-verifiable-inference.md](./research-verifiable-inference.md) | Complete — SOTA research, §8 layered scheme, TOPLOC adopt |
| [handoff.md](./handoff.md) | Session handoff — locked decisions, env notes |
| [docs/adr/0016–0019](../../docs/adr/) | Alpha scope, auth, fraud, multi-tracker design |
| [issues/](./issues/) | 22 work items (Buckets 1–3) |
| [issues/](./issues/) | 25 work items (Buckets 1–3 + perf follow-ups) |
## ADRs (this feature)
@@ -78,6 +80,12 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status.md) (done) |
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
### Phase 5 — Distributed-inference performance (post-routing-fix)
| Issue | Depends on |
|---|---|
| [25 Sharded per-node KV cache](./issues/25-per-node-kv-cache-distributed.md) | ADR-0020 routing fix (done), [24 routing telemetry resume](./issues/24-routing-telemetry-resume.md) |
## First 3 to implement
1. **02 + 20** — Unified auth boundary + validator service token (shared helper and roles)