41 lines
4.0 KiB
Markdown
41 lines
4.0 KiB
Markdown
# Distributed GGUF Runtime implementation strategy
|
||
|
||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented by this materialization, no story has completion credit, and legacy files remain for the DGR-017 audit. `prd.json` is authoritative.
|
||
|
||
## Execution model
|
||
|
||
Execute one numerically ordered, dependency-ready story per fresh Ralph context. Read `RALPH-CONTEXT.md`, source issue, and dependency evidence first; use TDD/fixture-first verification; finish with exact evidence. `prd.json` is the only state authority.
|
||
|
||
## Sequence
|
||
|
||
1. **M0 DGR-017..020:** reconcile legacy reality, lock metadata/performance contracts, and run the independent whole-model baseline.
|
||
2. **M1 DGR-021..033:** protocol/lifecycle/codegen, exact identities and split artifacts, pinned upstream/patches, CPU then accelerator builds, `ShardEngine`, fixtures, fake worker.
|
||
3. **M2 DGR-034..043:** dense ranged ownership/boundary/parity/local state, worker integration, supervision/direct-relay, and measured GGUF inputs to unchanged routing.
|
||
4. **M3 DGR-044..054:** pin/inventory V4, adapt upstream boundary/local state/MoE/hash execution, pass parity and real 2–4 scenario, then enforce alpha with MTP off.
|
||
5. **M4 DGR-055..067:** batching/backpressure/failure/recovery/long-context, existing-routing 10+ certification, real scale, measured optimization/compression, MTP contract+implementation, hardware certification.
|
||
6. **M5 DGR-068..071:** packages, human upstream collaboration, beta gate (including MTP), and pin/patch/certification maintenance.
|
||
|
||
## Guardrails
|
||
|
||
|
||
## Locked scope
|
||
|
||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||
|
||
## Target identities
|
||
|
||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||
|
||
DGR-020 cannot use distributed results. DGR-054 does not depend on MTP. DGR-070 depends on DGR-066. Compile support and scenario success never imply general routability.
|