popov/neuron-tai
1
0
Fork 0
Code Issues 52 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: define implementation-ready distributed GGUF roadmap

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-16 22:19:50 +03:00
parent 02b3709311
commit 81b1fa6074
65 changed files with 5346 additions and 2158 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
.scratch/distributed-gguf-runtime/milestones.md
View File

@@ -1,35 +1,37 @@
# Distributed GGUF runtime milestones
# Distributed GGUF Runtime milestones
## Gate A — measured runtime value
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented by this materialization, no story has completion credit, and legacy files remain for the DGR-017 audit. `prd.json` is authoritative.
- DGR-001 locks the safetensors-versus-GGUF performance/fit/quality contract.
- DGR-002 can proceed independently and defines the battle-proven backend-neutral wire protocol.
- DGR-003 builds exact recipe identity on DGR-002.
- Expensive native llama.cpp work remains gated by DGR-001.
## M0 — Truth and contracts
## Gate B — minimal native execution seam
- **Stories:** `DGR-017..DGR-020`
- **Outcome:** Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline.
- DGR-004 creates the reproducible pinned fork boundary.
- DGR-005 implements dense-Llama range ownership.
- DGR-006 proves architecture-defined boundary parity.
## M1 — Protocol and native substrate
## Gate C — concurrent production worker
- **Stories:** `DGR-021..DGR-033`
- **Outcome:** Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker.
- DGR-007 isolates concurrent Hot KV State.
- DGR-008 exposes the native worker over gRPC.
- DGR-009 integrates the worker without replacing Meshnet's control plane.
- DGR-010 passes local real-model two-process acceptance.
## M2 — Dense vertical proof
## Gate D — real consumer-hardware route
- **Stories:** `DGR-034..DGR-043`
- **Outcome:** Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing.
- DGR-011 passes two-physical-machine execution.
- DGR-012 adds continuous batching and bounded admission.
- DGR-013 hardens failure and cancellation.
## M3 — DeepSeek V4 Flash alpha
## Gate E — product release decision
- **Stories:** `DGR-044..DGR-054`
- **Outcome:** Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off.
- DGR-014 compares distributed GGUF against the current distributed safetensors route under locked thresholds.
- DGR-015 adds Qwen3/Qwen3-MoE only after the dense runtime passes.
- DGR-016 prepares narrow upstream llama.cpp collaboration material.
## M4 — Performance and beta hardening
No later gate may be claimed from synthetic workers or documentation-only evidence.
- **Stories:** `DGR-055..DGR-067`
- **Outcome:** Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix.
## M5 — Release and maintenance
- **Stories:** `DGR-068..DGR-071`
- **Outcome:** Reproducible packages, upstream collaboration, beta decision, and sustainable recertification.
## Release semantics
Alpha is DGR-054 and requires correctness plus the immutable useful-speed contract; MTP remains off. Beta is DGR-070 and requires DGR-066 MTP evidence plus concurrency, recovery, long-context, 10+ scenario, compression/optimization where certified, hardware matrix, and packaging. Failed gates remain failed evidence; they are not relabeled as success.