Ralph context: Distributed GGUF Runtime

Specification status: planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. prd.json is authoritative.

Mandatory startup for every fresh story

Read this file and authoritative prd.json completely.
Read the generated source issue named in the selected story description.
Read every dependency evidence README; legacy DGR-001..016 evidence is provenance only.
Read docs/adr/0024-distributed-gguf-runtime.md, root CONTEXT.md, .claude/memory/MEMORY.md, and relevant live source/tests.
Inspect git status; preserve unrelated work. Never infer implementation from planning text or old pass states.
If blocked or oversized, keep passes: false and write an honest BLOCKED.md/DECOMPOSITION.md; never weaken criteria or fabricate evidence.

Locked scope

Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are not redesigned. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned ShardEngine boundary.
llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.

Target identities

DeepSeek V4 official target SHA: 60d8d70770c6776ff598c94bb586a859a38244f1.
llama.cpp V4 support lineage began at PR 24162 / merge 8c146a8366304c871efc26057cc90370ccf58dad; DGR-027 later pins one exact validated current commit.
V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under /home.

Control/data-plane contract

Meshnet continues to own registration, coverage, existing route selection/load balancing, route epochs/sessions, direct/relay behavior, capability admission, cancellation, telemetry, billing, validation, and attribution. The GGUF adapter exposes measured inputs to those existing mechanisms. Direct seams use long-lived gRPC streams; relay seams carry byte-identical protobuf frames opaquely.

The project-owned ShardEngine hides llama.cpp internals. A worker loads one exact artifact/recipe/range identity. Default tests use fake/tiny fixtures. Real runs are opt-in, preserve raw metrics, and never download models under /home.

Gitea issue synchronization

Gitea is a projection of prd.json, never a competing source of truth. Before and after every supervised Ralph run, invoke:

python3 scripts/ralph_gitea_sync.py sync

For a complete Ralph invocation with automatic state reconciliation, use:

scripts/ralph-gitea-run.sh ralph-tui run --prd .scratch/distributed-gguf-runtime/prd.json --agent claude --model sonnet --iterations 1 --no-tui --no-setup --direct-merge --no-sandbox

The sync creates/reconciles one Gitea issue per DGR-* story, creates missing labels/milestones, closes issues whose passes is true, marks the selected next eligible story status:in-progress, and marks blocked stories status:blocked. gitea-issues.json is a derived mapping only.

Evidence and completion

Each story writes /run/media/popov/d/DEV/repos/d-popov.com/AI/.claude/worktrees/distributed-gguf-runtime/.scratch/distributed-gguf-runtime/evidence/<DGR-ID>/README.md with exact files, commands/results, limitations, identities, and dependent-story handoff. Only prd.json may record passes; DGR-017 and DGR-018 are complete and DGR-019 onward remain false. Generated Markdown and Gitea issues cannot override it. One scoped commit per story is expected during future execution.

5.0 KiB Raw Blame History Unescape Escape