docs: define distributed GGUF runtime plan

This commit is contained in:
Dobromir Popov
2026-07-13 15:09:27 +03:00
parent b5fa7245df
commit 4cae4a6c5c
42 changed files with 4913 additions and 691 deletions

View File

@@ -0,0 +1,61 @@
# 09 — Integrate the native worker with Meshnet
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
- This issue is `DGR-009` in [prd.json](../prd.json).
- Read the evidence README for every dependency listed below.
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
## Description
As the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.
## Expected durable outputs
- Meshnet GGUF backend adapter
- Registration, routing, relay, telemetry and billing tests
- evidence/DGR-009/README.md
## Acceptance criteria
- [ ] Implement the existing model-backend surface without changing Transformers behavior.
- [ ] Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.
- [ ] Tracker forms only complete compatible routes and keeps uncertified recipes dark.
- [ ] Direct routes use gRPC streams; relayed routes carry the same versioned protobuf frames as opaque binary through the existing relay seam.
- [ ] Existing request/work IDs, cancellation, Generation Telemetry, billing, and per-node attribution remain correlated.
- [ ] No vLLM, Nakshatra, prima.cpp, or custom-engine control plane becomes a core dependency.
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
## Dependency handoff
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
## Finish contract
- Create the task evidence directory and durable handoff required above.
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
- Change this issue to `Status: done` only after all criteria pass.
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
## References
- [Ralph execution context](../RALPH-CONTEXT.md)
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)