# Distributed GGUF Runtime technical challenges

> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.

## Key challenges and planned answers

1. **Truth/provenance:** old passes are unreliable. DGR-017 audits before preserving or cleaning anything.
2. **Baseline comparability:** lock lanes first (DGR-019), execute whole-model-only baseline next (DGR-020), and separate quant fit from runtime effects.
3. **Exact artifacts:** complete split manifests/hashes, resumable mounted-drive provisioning, no `/home` models, dynamic quant/topology (DGR-025..026).
4. **Fast-moving upstream:** exact manifest plus numbered patches in an ignored workspace; no vendoring/submodule/permanent fork (DGR-027..030, DGR-071).
5. **Stable project boundary:** gRPC/Protobuf and `ShardEngine` prevent upstream ABI leakage (DGR-021..033).
6. **V4 correctness:** llama.cpp owns mHC/MoE/attention/hash graphs and kernels; Meshnet adds ranged ownership and adapters, then validates whole-model parity (DGR-044..052).
7. **Boundary versus caches:** only mHC residual/positions/needed token IDs/schema expectations cross; KV and CSA/HCA/SWA/indexer/compressor state stay local/session-keyed (DGR-046, DGR-049).
8. **Recovery:** stale/missing local state causes cache miss and re-prefill/restart, never WAN migration (DGR-058..060).
9. **Scale without policy churn:** feed measured GGUF inputs into and certify existing backend-agnostic routing at 10+ candidates (DGR-043, DGR-061..062).
10. **Performance/concurrency:** bounded admission, continuous batching, fairness, long-context profiling, then evidence-selected optimization/compression (DGR-055..064).
11. **MTP:** define ownership/state first, keep off for alpha, implement/benchmark before beta (DGR-065..066).
12. **Hardware truth:** compile all five backend families but advertise only exact real-hardware-certified lanes (DGR-030, DGR-067).

## Locked scope

- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.

## Target identities

- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.