chore: clean superseded GGUF scaffolding
This commit is contained in:
Dobromir Popov
@@ -1,68 +0,0 @@
|
||||
# 01 — Lock the safetensors-versus-GGUF performance contract
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-001` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
|
||||
|
||||
## Baseline model target
|
||||
|
||||
Use the same model on both sides of the comparison, with the closest practical low-footprint precision pair:
|
||||
|
||||
- **safetensors:** `deepseek-ai/DeepSeek-V2-Lite-Chat` in **BF16**
|
||||
- **GGUF:** `second-state/DeepSeek-V2-Lite-Chat-GGUF` in **Q2_K** (~6.5GB)
|
||||
|
||||
Keep the benchmark matrix explicit for **CPU** and **GPU** runs. Reserve smaller non-DeepSeek fallback models only for loader plumbing smoke tests if needed; they do not count as the DGR-001 architecture-aligned baseline.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Benchmark harness and deterministic tests
|
||||
- evidence/DGR-001/performance-contract.json
|
||||
- Raw and summarized safetensors/GGUF benchmark evidence
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.
|
||||
- [ ] Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.
|
||||
- [ ] Report TTFT, prefill tok/s, decode tok/s, p50/p95 latency, aggregate throughput, RSS, VRAM, artifact size, failures, and output drift in machine-readable JSON.
|
||||
- [ ] Add concurrency levels 1 and 4 where memory permits.
|
||||
- [ ] Write a versioned performance contract consumed by later release gates, including an explicit stop condition when llama.cpp/GGUF has no meaningful speed or fit benefit.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-001/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- None. This story may start immediately.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,7 +1,7 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-017: Reconcile and clean the superseded DGR backlog
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Status / triage:** completed; `passes: true`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** None
|
||||
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
|
||||
- [ ] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
|
||||
- [ ] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
|
||||
- [ ] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
|
||||
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
|
||||
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
|
||||
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
# 02 — Adopt the versioned gRPC Shard protocol
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-002` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- packages/node/native/proto/shard_runtime.proto
|
||||
- Reproducible Python/C++ schema generation and build wiring
|
||||
- Protocol round-trip and compatibility tests
|
||||
- evidence/DGR-002/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
|
||||
- [x] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
|
||||
- [x] Define bounded chunking for prefill and a small decode fast path.
|
||||
- [x] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
|
||||
- [x] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
|
||||
- [x] Add generated-schema round-trip and compatibility tests in Python and C++.
|
||||
- [x] Targeted pytest tests pass
|
||||
- [x] python -m compileall packages tests passes for Python changes
|
||||
- [x] git diff --check passes
|
||||
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [x] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- None. This story may start immediately.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,57 +0,0 @@
|
||||
# 03 — Define exact Artifact and runtime recipe identity
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-003` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact runtime recipe/fingerprint implementation
|
||||
- Tracker/node fail-closed admission tests
|
||||
- evidence/DGR-003/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
|
||||
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,61 +0,0 @@
|
||||
# 04 — Create the reproducible pinned llama.cpp patch stack
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-004` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact llama.cpp upstream pin
|
||||
- Numbered minimal patch stack
|
||||
- Reproducible fetch/apply/build smoke
|
||||
- evidence/DGR-004/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
|
||||
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
|
||||
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
|
||||
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
|
||||
- [ ] Preserve upstream license and attribution notices.
|
||||
- [ ] Add a clean rebuild smoke test that does not download a model.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,61 +0,0 @@
|
||||
# 05 — Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-005` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Dense-Llama range-aware ownership implementation
|
||||
- Authoritative loaded-range introspection
|
||||
- Mapped/resident memory evidence
|
||||
- evidence/DGR-005/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Register and allocate only `blk.N.*` tensors in the assigned range.
|
||||
- [ ] Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.
|
||||
- [ ] Prefer range-aware mapping from one exact source GGUF; if derivative sub-GGUFs are used temporarily, verify source/slice hashes and avoid claiming final artifact semantics.
|
||||
- [ ] Report authoritative loaded range and endpoint ownership from the model, not operator CLI claims.
|
||||
- [ ] Demonstrate mapped/resident memory scales with owned tensors rather than full model size.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-005/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-004` must have `passes: true`; read `../evidence/DGR-004/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,61 +0,0 @@
|
||||
# 06 — Implement architecture-defined boundary input/output
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-006` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Architecture boundary adapter
|
||||
- Whole-model/two-range parity tests and results
|
||||
- evidence/DGR-006/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Head accepts token IDs and owns token embedding.
|
||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||
- [ ] The adapter interface fails closed for uncertified architectures.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-006/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
- `DGR-005` must have `passes: true`; read `../evidence/DGR-005/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,60 +0,0 @@
|
||||
# 07 — Add isolated concurrent local Hot KV State
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-007` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Concurrent local KV/session manager
|
||||
- Isolation, eviction, cancellation and cleanup tests
|
||||
- evidence/DGR-007/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Map `(Route Session ID, route epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Allocate KV only for owned layers.
|
||||
- [ ] Support prefill append, decode append, truncate, release, TTL/LRU eviction, and explicit cache-miss response.
|
||||
- [ ] Reject stale epochs and incompatible cache recipes.
|
||||
- [ ] At least four concurrent sessions on a small model complete without token or KV cross-talk.
|
||||
- [ ] Cancellation/release of one session leaves other sessions intact and memory returns to the configured budget.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-007/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,65 +0,0 @@
|
||||
# 08 — Build the standalone C++ gRPC Shard worker
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-008` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Standalone C++ gRPC worker
|
||||
- Fake-model Python/C++ integration tests
|
||||
- Lifecycle and bounded-failure evidence
|
||||
- evidence/DGR-008/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Worker exposes capability, health, session stream, release, cancellation, and metrics services from DGR-002.
|
||||
- [ ] Worker loads one exact Artifact/recipe/Shard identity and refuses mismatched requests.
|
||||
- [ ] Streaming path enforces bounded messages, flow control, deadlines, idempotency, and independent session cancellation.
|
||||
- [ ] Worker does not expose raw llama.cpp RPC or arbitrary GGML graph execution.
|
||||
- [ ] Graceful shutdown releases sessions; crash behavior is bounded and observable.
|
||||
- [ ] Python integration tests run against a fake model mode without model downloads.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-008/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-004` must have `passes: true`; read `../evidence/DGR-004/README.md` and verify its referenced files/commands.
|
||||
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
|
||||
- `DGR-007` must have `passes: true`; read `../evidence/DGR-007/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,61 +0,0 @@
|
||||
# 09 — Integrate the native worker with Meshnet
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-009` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Meshnet GGUF backend adapter
|
||||
- Registration, routing, relay, telemetry and billing tests
|
||||
- evidence/DGR-009/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Implement the existing model-backend surface without changing Transformers behavior.
|
||||
- [ ] Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.
|
||||
- [ ] Tracker forms only complete compatible routes and keeps uncertified recipes dark.
|
||||
- [ ] Direct routes use gRPC streams; relayed routes carry the same versioned protobuf frames as opaque binary through the existing relay seam.
|
||||
- [ ] Existing request/work IDs, cancellation, Generation Telemetry, billing, and per-node attribution remain correlated.
|
||||
- [ ] No vLLM, Nakshatra, prima.cpp, or custom-engine control plane becomes a core dependency.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,62 +0,0 @@
|
||||
# 10 — Pass local real-model two-process acceptance
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-010` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a release engineer, I need real local distributed parity before involving network variability.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Real local two-process commands and configuration
|
||||
- Raw parity, memory and performance results
|
||||
- evidence/DGR-010/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Two local worker processes open disjoint dense-Llama ranges from the certified Artifact.
|
||||
- [ ] Prefill and at least 32 greedy decode tokens match whole-model llama.cpp within the certified tolerance.
|
||||
- [ ] Each worker retains only its own tensors and Hot KV State.
|
||||
- [ ] Four concurrent Route Sessions pass isolation and cleanup checks.
|
||||
- [ ] Report TTFT, prefill/decode throughput, seam bytes/latency, worker RSS/VRAM, KV memory, batch size, and queue time.
|
||||
- [ ] Killing one worker produces a bounded structured failure rather than a deadlock.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-010/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,62 +0,0 @@
|
||||
# 11 — Pass a real heterogeneous two-machine route
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-011` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Two-machine hardware/network/runtime manifest
|
||||
- Raw real-route metrics and output evidence
|
||||
- evidence/DGR-011/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Tracker selects two physical nodes with disjoint Shards and one exact certified recipe/compatibility class.
|
||||
- [ ] Actual CPU/GPU execution occurs on both nodes; synthetic workers do not satisfy acceptance.
|
||||
- [ ] Prefill/decode, concurrent-session isolation, telemetry, cancellation, and cleanup pass over the real transport/relay path.
|
||||
- [ ] Exact hardware, network, backend, model hash, route, commands, and raw metrics are recorded.
|
||||
- [ ] A model or recipe larger than one participating node's admitted memory is exercised when available.
|
||||
- [ ] Output drift is measured and incompatible mixed backends fail closed.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-011/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-010` must have `passes: true`; read `../evidence/DGR-010/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,63 +0,0 @@
|
||||
# 12 — Implement continuous batching and bounded admission
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-012` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Continuous batching/admission scheduler
|
||||
- Concurrency 1/2/4/8 report
|
||||
- Queue, batch and KV-pressure evidence
|
||||
- evidence/DGR-012/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Node scheduler admits sessions against weight, KV, scratch, and queue budgets.
|
||||
- [ ] Compatible decode steps from multiple sessions form llama.cpp batches while preserving per-session positions and outputs.
|
||||
- [ ] Prefill does not starve decode; scheduling policy and bounds are explicit.
|
||||
- [ ] Backpressure prevents unbounded queued activations or KV growth.
|
||||
- [ ] Capability telemetry reports active sessions, queue depth, batch occupancy, KV pressure, prefill/decode rates, and rejected admissions.
|
||||
- [ ] Concurrency 1/2/4/8 benchmark identifies saturation and shows no cross-session corruption.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-012/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-007` must have `passes: true`; read `../evidence/DGR-007/README.md` and verify its referenced files/commands.
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
- `DGR-010` must have `passes: true`; read `../evidence/DGR-010/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,62 +0,0 @@
|
||||
# 13 — Harden failure, cancellation, and restart semantics
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-013` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Failure/cancel/restart test matrix
|
||||
- Resource cleanup and billing-state evidence
|
||||
- evidence/DGR-013/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Deadlines and heartbeat/health loss terminate blocked stream operations.
|
||||
- [ ] Cancellation propagates across every Shard and releases local KV and queued buffers.
|
||||
- [ ] Duplicate steps are idempotent; uncertain mutations are never replayed silently.
|
||||
- [ ] Alpha failover restarts from token zero on a newly compatible route rather than importing unverified KV.
|
||||
- [ ] Worker death, stream reset, malformed bundle, stale epoch, and cache miss tests pass.
|
||||
- [ ] Billing/work records distinguish completed, cancelled, failed, and unverified work.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-013/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,65 +0,0 @@
|
||||
# 14 — Enforce the GGUF-versus-safetensors release gate
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-014` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Immutable comparison against DGR-001 thresholds
|
||||
- Machine-readable final report
|
||||
- Ship/optimize/stop recommendation
|
||||
- evidence/DGR-014/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.
|
||||
- [ ] Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.
|
||||
- [ ] Evaluate against the DGR-001 performance contract without changing thresholds after seeing results.
|
||||
- [ ] Ship recommendation is one of: promote GGUF, optimize a measured bottleneck with a new bounded task, or stop the native track.
|
||||
- [ ] Results clearly separate quantization gains from transport/runtime gains.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-014/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
|
||||
- `DGR-011` must have `passes: true`; read `../evidence/DGR-011/README.md` and verify its referenced files/commands.
|
||||
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
|
||||
- `DGR-013` must have `passes: true`; read `../evidence/DGR-013/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,61 +0,0 @@
|
||||
# 15 — Add and certify a Qwen3/Qwen3-MoE adapter
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-015` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Qwen3-family architecture adapter
|
||||
- Architecture-specific parity/admission/performance results
|
||||
- evidence/DGR-015/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Implement explicit tensor ownership, router/top-k, expert/shared-expert, Q/K normalization, boundary bundle, and cache semantics for the selected Qwen3 family recipe.
|
||||
- [ ] Do not reuse the dense-Llama adapter through unchecked name substitutions.
|
||||
- [ ] Whole-model versus distributed prefill/decode parity passes the architecture-specific tolerance.
|
||||
- [ ] Expert memory ownership and communication are measured.
|
||||
- [ ] Real consumer-hardware acceptance and capability admission pass before the recipe becomes routable.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-015/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-014` must have `passes: true`; read `../evidence/DGR-014/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,60 +0,0 @@
|
||||
# 16 — Produce the upstream llama.cpp collaboration package
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-016` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Narrow upstream patches/tests
|
||||
- Generic API design note
|
||||
- Human-ready llama.cpp outreach package
|
||||
- evidence/DGR-016/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Separate generic llama.cpp hooks from Meshnet protocol/control-plane code.
|
||||
- [ ] Prepare minimal reproducible examples and tests for range-aware loading, boundary input/output, and layer-filtered KV.
|
||||
- [ ] Compare the proposal with Nakshatra and prima.cpp evidence and explain why the API is generally useful.
|
||||
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
||||
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-016/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-010` must have `passes: true`; read `../evidence/DGR-010/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
Reference in New Issue
Block a user