docs: define distributed GGUF runtime plan

.scratch/distributed-gguf-runtime/README.md

@@ -1,63 +1,46 @@
-# Distributed GGUF runtime — planning index
+# Performant concurrent distributed GGUF runtime
 
-Status: draft scratch package.
+Status: active benchmark-gated implementation program.
 
-Goal: make the node network capable of serving large, high-quality open models by distributing GGUF/model artifacts over a torrent-style swarm while executing inference over a sticky multi-node route with per-shard local KV cache.
+## Objective
 
-This scratch supersedes the old assumption in [ADR-0001](../../docs/adr/0001-pytorch-over-llama-cpp.md) that llama.cpp is only a single-node leaf backend. That assumption was correct for the original llama.cpp RPC shape, but the target is now different: torrent-distributed GGUF artifacts plus an explicit route/KV protocol owned by this platform, ideally developed in collaboration with upstream llama.cpp.
+Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.
 
-## Artifacts
+## Critical path
 
-| Path | Purpose |
-|---|---|
-| [architecture.md](./architecture.md) | Proposed runtime architecture, data flow, session state, and failure model |
-| [technical-challenges.md](./technical-challenges.md) | Detailed challenge/solution register with acceptance tests |
-| [decision-framework.md](./decision-framework.md) | Grilling framework for open decisions and recommended answers |
-| [research-prior-art.md](./research-prior-art.md) | Prior-art notes for Petals, exo, Distributed Llama, prima.cpp, llama.cpp, DeepSeek-V4-Flash, GLM-5.2, and Ornith |
-| [ADR-0020-distributed-gguf-runtime.md](./ADR-0020-distributed-gguf-runtime.md) | Draft decision record for the GGUF/llama.cpp distributed runtime |
-| [PRD.md](./PRD.md) | Product/runtime requirements and acceptance criteria |
-| [milestones.md](./milestones.md) | Dependency-ordered implementation milestones |
-| [issues/](./issues/) | Implementation-ready tracer-bullet issue briefs |
+```text
+Meshnet control plane
+  -> versioned gRPC/Protobuf Shard protocol
+  -> project-owned standalone C++ worker
+  -> small pinned llama.cpp patch stack
+```
 
-## Decision Summary
+Transformers/safetensors remains the correctness baseline. vLLM remains an optional complete managed provider and a design donor; it is not forked into the public mesh.
 
-Adopt a hybrid runtime:
+## Planning artifacts
 
-- **Weights and artifacts**: distributed by torrent / content-addressed storage / optional CDN.
-- **Hot KV cache**: local to the node that owns the corresponding layer range.
-- **Prefix snapshots**: optionally persisted to cache servers for reuse, retry, and failover.
-- **Active route**: sticky for one request/session.
-- **Context cap**: 128K hard product limit for large models unless explicitly revised.
-- **Backends**: keep PyTorch for fast model-architecture coverage and validation; add llama.cpp/GGUF as the performance path for supported models.
-- **Client feedback**: stream token deltas when feasible; always expose Generation Telemetry.
-- **First serious target model**: DeepSeek-V4-Flash after a smaller GGUF protocol smoke test.
+- **[Mandatory Ralph context](RALPH-CONTEXT.md)** — read first in every fresh iteration
+- [Task evidence contract](evidence/README.md)
+- [Implementation strategy](implementation-strategy.md)
+- [Current architecture](architecture.md)
+- [PRD](PRD.md)
+- [Ralph backlog](prd.json)
+- [ADR-0020](ADR-0020-distributed-gguf-runtime.md)
+- [Milestones](milestones.md)
+- [Issues](issues/)
+- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)
+- [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md)
+- [vLLM assessment](../../docs/research/vllm-distributed-gguf-assessment.md)
 
-## What We Learned
+## Ralph execution
 
-- Our current full-model PyTorch path uses Transformers `generate()` and gets local KV cache.
-- Our current distributed PyTorch path disables cache and recomputes the full growing sequence per token.
-- The seam today carries hidden activations, not KV cache; at 128K this becomes impossible for serious models if repeated every decode token.
-- The missing capability is not "send KV across the network"; it is **stable per-session local KV cache per shard**.
-- GGUF distribution is solved enough at the artifact layer, but GGUF/llama.cpp needs explicit layer-boundary execution APIs for our route model.
+Use supervised one-story iterations for this high-risk runtime:
 
-## Recommended Order
+```bash
+ralph-tui run \
+  --prd .scratch/distributed-gguf-runtime/prd.json \
+  --agent claude --model opus \
+  --iterations 1 --no-tui --no-setup --verify
+```
 
-See [milestones.md](./milestones.md) for the full dependency map.
-
-1. [01 — Route Session lifecycle](./issues/01-route-session-lifecycle.md)
-2. [02 — Prefill/decode binary HTTP protocol](./issues/02-prefill-decode-binary-http.md)
-3. [03 — Generation Telemetry and streaming response contract](./issues/03-generation-telemetry-and-streaming.md)
-4. [04 — PyTorch distributed KV reference route](./issues/04-pytorch-distributed-kv-reference.md)
-5. [05 — Local llama.cpp/GGUF backend](./issues/05-local-llamacpp-gguf-backend.md)
-6. [06 — Model Artifact manifest and Shard advertisement](./issues/06-model-artifact-manifest.md)
-7. [07 — llama.cpp layer-boundary prototype](./issues/07-llamacpp-layer-boundary-prototype.md)
-8. [08 — Networked distributed GGUF route](./issues/08-networked-distributed-gguf-route.md)
-9. [09 — DeepSeek-V4-Flash support audit](./issues/09-deepseek-v4-flash-support-audit.md)
-10. [10 — GLM-5.2 and Ornith follow-up support audit](./issues/10-glm52-ornith-followup-audit.md)
-
-## Open Questions
-
-- Does upstream llama.cpp already expose enough internal API for arbitrary layer-range execution and hidden-state boundary I/O, or do we need an extension?
-- Can GGUF split metadata be made layer/tensor semantic enough for torrent placement and partial loading?
-- What is the minimum protocol needed for compressed KV formats such as GLM-5.2 DSA/MLA without exposing model-specific internals to the tracker?
-- How much reliability do we need in alpha: fail request on route loss, or support route repair with KV snapshots?
+Inspect the diff, run the story gates, and commit one verified story before the next iteration. Real-model stories require the explicit environment gate and mounted-drive model storage.