58 lines
5.0 KiB
Markdown
58 lines
5.0 KiB
Markdown
# Ralph context: Distributed GGUF Runtime
|
||
|
||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||
|
||
## Mandatory startup for every fresh story
|
||
|
||
1. Read this file and authoritative `prd.json` completely.
|
||
2. Read the generated source issue named in the selected story description.
|
||
3. Read every dependency evidence README; legacy DGR-001..016 evidence is provenance only.
|
||
4. Read `docs/adr/0024-distributed-gguf-runtime.md`, root `CONTEXT.md`, `.claude/memory/MEMORY.md`, and relevant live source/tests.
|
||
5. Inspect `git status`; preserve unrelated work. Never infer implementation from planning text or old pass states.
|
||
6. If blocked or oversized, keep `passes: false` and write an honest `BLOCKED.md`/`DECOMPOSITION.md`; never weaken criteria or fabricate evidence.
|
||
|
||
## Locked scope
|
||
|
||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||
|
||
## Target identities
|
||
|
||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||
|
||
## Control/data-plane contract
|
||
|
||
Meshnet continues to own registration, coverage, existing route selection/load balancing, route epochs/sessions, direct/relay behavior, capability admission, cancellation, telemetry, billing, validation, and attribution. The GGUF adapter exposes measured inputs to those existing mechanisms. Direct seams use long-lived gRPC streams; relay seams carry byte-identical protobuf frames opaquely.
|
||
|
||
The project-owned `ShardEngine` hides llama.cpp internals. A worker loads one exact artifact/recipe/range identity. Default tests use fake/tiny fixtures. Real runs are opt-in, preserve raw metrics, and never download models under `/home`.
|
||
|
||
## Gitea issue synchronization
|
||
|
||
Gitea is a projection of `prd.json`, never a competing source of truth. Before and after every supervised Ralph run, invoke:
|
||
|
||
```bash
|
||
python3 scripts/ralph_gitea_sync.py sync
|
||
```
|
||
|
||
For a complete Ralph invocation with automatic state reconciliation, use:
|
||
|
||
```bash
|
||
scripts/ralph-gitea-run.sh ralph-tui run --prd .scratch/distributed-gguf-runtime/prd.json --agent claude --model sonnet --iterations 1 --no-tui --no-setup --direct-merge --no-sandbox
|
||
```
|
||
|
||
The sync creates/reconciles one Gitea issue per `DGR-*` story, creates missing labels/milestones, closes issues whose `passes` is true, marks the selected next eligible story `status:in-progress`, and marks blocked stories `status:blocked`. `gitea-issues.json` is a derived mapping only.
|
||
|
||
## Evidence and completion
|
||
|
||
Each story writes `/run/media/popov/d/DEV/repos/d-popov.com/AI/.claude/worktrees/distributed-gguf-runtime/.scratch/distributed-gguf-runtime/evidence/<DGR-ID>/README.md` with exact files, commands/results, limitations, identities, and dependent-story handoff. Only `prd.json` may record `passes`; DGR-017 and DGR-018 are complete and DGR-019 onward remain false. Generated Markdown and Gitea issues cannot override it. One scoped commit per story is expected during future execution.
|