f0ddb69d33
# Conflicts: # .claude/memory/MEMORY.md # .scratch/distributed-gguf-runtime/PRD.md # .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md # .scratch/distributed-gguf-runtime/README.md # .scratch/distributed-gguf-runtime/architecture.md # .scratch/distributed-gguf-runtime/evidence/DGR-017/README.md # .scratch/distributed-gguf-runtime/implementation-strategy.md # .scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md # .scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md # .scratch/distributed-gguf-runtime/milestones.md # .scratch/distributed-gguf-runtime/prd.json # docs/issues/distributed-gguf-runtime/01-lock-the-safetensors-versus-gguf-performance-contract.md # docs/issues/distributed-gguf-runtime/02-adopt-the-versioned-grpc-shard-protocol.md # docs/issues/distributed-gguf-runtime/03-define-exact-artifact-and-runtime-recipe-identity.md # docs/issues/distributed-gguf-runtime/05-implement-dense-llama-range-aware-gguf-ownership.md # docs/issues/distributed-gguf-runtime/06-implement-architecture-defined-boundary-input-output.md
Distributed GGUF Runtime planning workspace
Specification status: planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit.
prd.jsonis authoritative.
Locked scope
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are not redesigned. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned
ShardEngineboundary. - llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
Target identities
- DeepSeek V4 official target SHA:
60d8d70770c6776ff598c94bb586a859a38244f1. - llama.cpp V4 support lineage began at PR 24162 / merge
8c146a8366304c871efc26057cc90370ccf58dad; DGR-027 later pins one exact validated current commit. - V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under
/home.
Navigation
prd.json— sole authoritative 55-story backlog, DGR-017..071.PRD.md— human-readable projection of goals, gates, and all stories.RALPH-CONTEXT.md— mandatory fresh-session context.architecture.md,implementation-strategy.md,milestones.md— design and execution sequence.issues/— generated story specs; files 01..16 are retained legacy artifacts pending DGR-017.evidence/— provenance and future per-story handoffs.