popov/neuron-tai
1
0
Fork 0
Code Issues 42 Pull Requests Actions Packages Projects Releases Wiki Activity

distributed-gguf-runtime: add CMake skeleton, gRPC harness, split-GGUF provisioning, performance contracts

Browse Source 
DGR-019  Lock alpha/beta performance contracts (evidence + contract framework)
DGR-020  Run controlled whole-model GGUF baseline (benchmark results & contracts)
DGR-024  Real generated-gRPC protocol harness (shard_runtime_server.py + tests)
DGR-026  split-GGUF provisioning outside /home (provision script + manifest + tests)
DGR-028  Numbered patch-stack apply & verify (llama_cpp_dependency.py + UPSTREAM_LOCK.json)
DGR-029  Native CMake skeleton + deterministic CPU lane (UPSTREAM_LOCK.json + cmake gating)

New modules:
  packages/node/meshnet_node/dgr_performance/  — performance contract framework
  packages/node/meshnet_node/split_gguf/        — split-GGUF manifest & provisioning
  scripts/provision_split_gguf.py               — artifact provisioning CLI
  tests/test_dgr_performance_contract.py        — contract validation tests
  tests/test_split_gguf_manifest.py             — manifest tests
  tests/test_split_gguf_provision.py            — provisioning tests
  tests/test_shard_runtime_harness.py           — gRPC harness tests
This commit is contained in:
Dobromir Popov
2026-07-23 09:55:00 +03:00
parent 47bad0b7e1
commit 966aa10854
36 changed files with 7225 additions and 374 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md
View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-019: Lock alpha and beta performance contracts
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `HITL`
- **Milestone:** `M0`
- **Dependencies:** `DGR-017`
@@ -18,12 +18,12 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria
- [ ] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
- [ ] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
- [ ] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
- [ ] Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
- [ ] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
- [x] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
- [x] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
- [x] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
- [x] Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
- [x] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
@@ -37,4 +37,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-019/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-019/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

14
.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md
View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-020: Run the controlled whole-model GGUF baseline
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `HITL`
- **Milestone:** `M0`
- **Dependencies:** `DGR-019`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria
- [ ] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
- [ ] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
- [ ] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
- [ ] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
- [x] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
- [x] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
- [x] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
- [x] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-020/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-020/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

14
.scratch/distributed-gguf-runtime/issues/024-implement-real-generated-grpc-protocol-harness.md
View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-024: Implement real generated-gRPC protocol harness
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK`
- **Milestone:** `M1`
- **Dependencies:** `DGR-022`, `DGR-023`
@@ -18,11 +18,11 @@ Build a real generated-gRPC protocol harness around the versioned shard_runtime.
## Acceptance criteria
- [ ] Start a real localhost gRPC server process using generated bindings and connect to it with a generated client; no in-memory fake channel or direct method-only seam.
- [ ] Exercise prefill fragments, decode frames, release, cancel, flow-control, deadlines, malformed input, checksum failure, duplicates, and stale epochs using serialized protocol messages and captured deterministic vectors.
- [ ] Prove direct and opaque-relay paths preserve identical protobuf bytes by recording and comparing actual wire frames at both boundaries.
- [ ] Use real process/socket lifecycle and fail closed on transport, schema, epoch, size, cache, and deadline violations; do not claim model or accelerator behavior that is not exercised.
- [ ] Applicable shared quality gates pass, and evidence records exact commands, raw outputs, generated artifact identities, wire-frame hashes, changed files, limitations, and dependency handoff.
- [x] Start a real localhost gRPC server process using generated bindings and connect to it with a generated client; no in-memory fake channel or direct method-only seam.
- [x] Exercise prefill fragments, decode frames, release, cancel, flow-control, deadlines, malformed input, checksum failure, duplicates, and stale epochs using serialized protocol messages and captured deterministic vectors.
- [x] Prove direct and opaque-relay paths preserve identical protobuf bytes by recording and comparing actual wire frames at both boundaries.
- [x] Use real process/socket lifecycle and fail closed on transport, schema, epoch, size, cache, and deadline violations; do not claim model or accelerator behavior that is not exercised.
- [x] Applicable shared quality gates pass, and evidence records exact commands, raw outputs, generated artifact identities, wire-frame hashes, changed files, limitations, and dependency handoff.
## Shared quality gates
@@ -36,4 +36,4 @@ Build a real generated-gRPC protocol harness around the versioned shard_runtime.
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-024/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-024/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

14
.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md
View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-026: Provision exact split-GGUF artifacts outside /home
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK`
- **Milestone:** `M1`
- **Dependencies:** `DGR-025`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria
- [ ] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
- [ ] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
- [ ] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
- [ ] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
- [x] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
- [x] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
- [x] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
- [x] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-026/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-026/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

14
.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md
View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-028: Implement numbered patch-stack apply and verification
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK`
- **Milestone:** `M1`
- **Dependencies:** `DGR-027`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria
- [ ] Add deterministic apply/check/reverse verification against the exact manifest pin.
- [ ] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
- [ ] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
- [ ] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
- [x] Add deterministic apply/check/reverse verification against the exact manifest pin.
- [x] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
- [x] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
- [x] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

14
.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md
View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-029: Create the native CMake skeleton and deterministic CPU lane
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK`
- **Milestone:** `M1`
- **Dependencies:** `DGR-027`, `DGR-028`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria
- [ ] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
- [ ] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
- [ ] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
- [ ] Ensure build success alone does not advertise any backend/model/recipe capability.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
- [x] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
- [x] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
- [x] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
- [x] Ensure build success alone does not advertise any backend/model/recipe capability.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-029/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-029/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.