122 Commits

Author SHA1 Message Date
Dobromir Popov
f0f9a0eed7 controller: record DGR-043 completion 2026-08-01 01:58:14 +03:00
Dobromir Popov
e6ad9fdca9 story: DGR-043 Expose GGUF compatibility and measured cost inputs to existing routing 2026-08-01 01:58:13 +03:00
Dobromir Popov
d53acb1145 controller: record DGR-042 completion 2026-08-01 01:52:43 +03:00
Dobromir Popov
fd10607033 story: DGR-042 Carry native frames through direct and existing relay seams 2026-08-01 01:52:42 +03:00
Dobromir Popov
eb986ddf10 controller: record DGR-041 completion 2026-08-01 01:47:29 +03:00
Dobromir Popov
f37c4352fe story: DGR-041 Register native Shard capabilities without redesigning Meshnet 2026-08-01 01:47:28 +03:00
Dobromir Popov
95f005f646 controller: record DGR-040 completion 2026-08-01 01:41:52 +03:00
Dobromir Popov
520ccb8266 story: DGR-040 Add node-side native worker supervision 2026-08-01 01:41:51 +03:00
Dobromir Popov
f4980491d2 controller: record DGR-039 completion 2026-08-01 01:35:16 +03:00
Dobromir Popov
3a67eea569 story: DGR-039 Pass local two-process dense acceptance 2026-08-01 01:35:14 +03:00
Dobromir Popov
4c6c78d837 controller: record DGR-038 completion 2026-08-01 01:32:48 +03:00
Dobromir Popov
49560b396f story: DGR-038 Implement isolated shard-local Hot KV State 2026-08-01 01:32:46 +03:00
Dobromir Popov
a1df87deb6 controller: record DGR-037 completion 2026-08-01 01:28:07 +03:00
Dobromir Popov
8217b4c4a2 story: DGR-037 Bind llama.cpp to the standalone worker 2026-08-01 01:28:06 +03:00
Dobromir Popov
dfa403adc6 controller: record DGR-036 completion 2026-08-01 01:20:17 +03:00
Dobromir Popov
6e8bf7a64d story: DGR-036 Prove dense fixture and real-model range parity 2026-08-01 01:20:16 +03:00
Dobromir Popov
6e88b3bd8f controller: record DGR-035 completion 2026-08-01 01:13:52 +03:00
Dobromir Popov
64c2046e5a story: DGR-035 Implement dense architecture boundary input/output 2026-08-01 01:13:50 +03:00
Dobromir Popov
79c9bbaf63 controller: record DGR-034 completion 2026-08-01 01:08:29 +03:00
Dobromir Popov
d339cfde25 story: DGR-034 Implement dense-Llama range-aware GGUF ownership 2026-08-01 01:08:28 +03:00
Dobromir Popov
27a0d89678 docs: align PRD source-of-truth status 2026-07-27 08:58:51 +03:00
Dobromir Popov
8c87fae1ac chore: restore canonical PRD metadata and projections 2026-07-27 08:53:16 +03:00
Dobromir Popov
4d530d702c chore: restore canonical DGR-033 metadata projection 2026-07-26 23:04:45 +03:00
Dobromir Popov
7473bb7e44 fix: DGR-033 repair native worker protocol per cross-review BLOCK
Address the Codex GPT-5.5 review of the standalone fake C++ gRPC Shard
worker. Four root protocol defects fixed:

- Fail closed before SessionOpen: a per-session `opened` flag gates
  chunk/decode so no activation bypasses lifecycle, cancellation, epoch
  or flow-control state (terminal ERROR_CODE_INTERNAL), even when an
  out-of-band Cancel created placeholder state.
- Strict flow-control negotiation: NegotiateFlow takes the strictest of
  peer-vs-worker bounds (mirrors codec.negotiate_flow_control) and the
  negotiated per-session max_chunk_bytes is enforced on every bundle
  instead of trusting the peer proposal.
- In-stream ReleaseSignal now erases session state immediately.
- SessionOpen rejects incompatible schema, fingerprint, and shard-range
  identity and reports the worker's own served fingerprint rather than
  echoing the caller.

Adds 9 regression tests (worker suite 18 -> 27). Real gates on the
rebuilt pinned-gRPC binary: cmake build exit 0; ctest 2/2; worker
pytest 27 passed; harness+protocol 63 passed; compileall 0; diff --check
clean; ldd/nm show 0 llama/ggml linkage. DGR-033 passes -> true.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 22:57:03 +03:00
Dobromir Popov
c073826374 chore: reblock DGR-033 after protocol review 2026-07-26 22:34:34 +03:00
Dobromir Popov
0c7d475335 chore: remove Ralph lane runtime artifacts 2026-07-26 22:33:37 +03:00
Dobromir Popov
84d75f4cd2 controller: record DGR-033 completion 2026-07-25 22:38:01 +03:00
Dobromir Popov
766e480ba5 story: DGR-033 Build a standalone fake C++ gRPC Shard worker 2026-07-25 22:38:00 +03:00
Dobromir Popov
25e53bfeab story: DGR-032 Implement deterministic fake ShardEngine 2026-07-23 11:09:16 +03:00
Dobromir Popov
c34ab059cc story: DGR-031 Introduce the project-owned ShardEngine interface 2026-07-23 11:00:33 +03:00
Dobromir Popov
fd742d35c0 story: DGR-030 Add accelerator build presets and native CI matrix 2026-07-23 10:51:08 +03:00
Dobromir Popov
254297660a add CLAUDE.md with full milestone map and task explanations 2026-07-23 10:17:55 +03:00
Dobromir Popov
966aa10854 distributed-gguf-runtime: add CMake skeleton, gRPC harness, split-GGUF provisioning, performance contracts
DGR-019  Lock alpha/beta performance contracts (evidence + contract framework)
DGR-020  Run controlled whole-model GGUF baseline (benchmark results & contracts)
DGR-024  Real generated-gRPC protocol harness (shard_runtime_server.py + tests)
DGR-026  split-GGUF provisioning outside /home (provision script + manifest + tests)
DGR-028  Numbered patch-stack apply & verify (llama_cpp_dependency.py + UPSTREAM_LOCK.json)
DGR-029  Native CMake skeleton + deterministic CPU lane (UPSTREAM_LOCK.json + cmake gating)

New modules:
  packages/node/meshnet_node/dgr_performance/  — performance contract framework
  packages/node/meshnet_node/split_gguf/        — split-GGUF manifest & provisioning
  scripts/provision_split_gguf.py               — artifact provisioning CLI
  tests/test_dgr_performance_contract.py        — contract validation tests
  tests/test_split_gguf_manifest.py             — manifest tests
  tests/test_split_gguf_provision.py            — provisioning tests
  tests/test_shard_runtime_harness.py           — gRPC harness tests
2026-07-23 09:55:00 +03:00
Dobromir Popov
47bad0b7e1 backlog updated 2026-07-21 21:56:08 +03:00
Dobromir Popov
aa148cc7aa fix(vscode): use dynamic interpreter path in launch.json for cross-machine debug
Configs hardcoded .venv-rocm/bin/python (this Linux box's ROCm venv), which
doesn't exist on the Windows dev machine. Switch to
${command:python.interpreterPath} so each machine resolves whatever
interpreter is selected in the VS Code Python extension locally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 14:09:45 +03:00
Dobromir Popov
505f37dd8d logs 2026-07-21 14:00:30 +03:00
Dobromir Popov
cd6b4d9d48 Merge DGR-024 (real generated-gRPC protocol harness) from ralph-terra-loop lane
# Conflicts:
#	.scratch/distributed-gguf-runtime/prd.json
2026-07-21 13:46:53 +03:00
Dobromir Popov
5177db25b0 feat: implement real generated-gRPC protocol harness (DGR-024)
Real ShardRuntimeServicer process bound to a real localhost socket, driven
by a generated ShardRuntimeStub over grpc.insecure_channel from a separately
spawned subprocess. Proves direct-hop and opaque-relay (exact captured
request bytes re-sent, no reinterpretation) produce byte-identical server
responses, cross-checked against an independent server-side wire capture.

Fails closed on the required negative paths: stale route epoch, expired
deadline, malformed/non-tiling fragments, checksum failure, exhausted
flow-control credit (with in-band top-up), duplicate idempotency steps
(acked, not re-applied), and cancel — both in-band CancelSignal (single
work item vs whole session) and the out-of-band unary Cancel RPC, including
a Cancel that races ahead of SessionOpen.

Supersedes the earlier in-memory fake-seam approach for this ticket, which
a policy audit rejected under the no-fake-data rule; that code is not
reintroduced. Evidence README rewritten to describe the actual files.

11 passed in tests/test_shard_runtime_harness.py.
2026-07-21 13:39:58 +03:00
Dobromir Popov
159284b3b5 Merge DGR-025 (certified-artifact-byte recipe identity) from ralph-fable-loop lane 2026-07-21 13:23:17 +03:00
Dobromir Popov
732ee9f91a Merge DGR-028 (numbered patch-stack apply/verify) from ralph-kimi-loop lane 2026-07-21 13:23:12 +03:00
Dobromir Popov
7da90ef475 feat: implement numbered patch-stack apply/verify enforcement (DGR-028)
Split the range-loader patch into single-concern patches 0002-0005 (loader,
filtered state report, boundary I/O endpoint guard, worker range-report
hook), add UPSTREAM-ASSUMPTIONS.json describing each patch's assumptions,
and enforce control-plane/license boundary checks plus first-incompatible-
patch reporting in scripts/llama_cpp_dependency.py apply/reverse/verify.

7 passed in tests/test_llama_cpp_dependency.py; SHA256SUMS verified against
all five patches; focused native CTest (test-meshnet-range-ownership 1/1)
recorded in evidence README (build/ dir not present in this environment to
independently reverify).
2026-07-21 13:22:55 +03:00
Dobromir Popov
03e97ca31a fix: bind recipe identity to certified artifact bytes (DGR-025)
Append +artifact.<sha256> to the llama.cpp runtime axis, computed from the
exact bytes read by attest_loaded_runtime, so a differently-built shared
object with copied lock values can no longer forge a certified runtime
identity. Node/tracker parsers require the suffix; new test proves a
byte-identical-lock but different-binary artifact produces a different
recipe fingerprint. Regenerates conformance vectors accordingly.

105 passed in tests/test_native_identity_emission.py,
tests/test_runtime_pin_identity.py, tests/test_runtime_recipe_identity.py.
2026-07-21 13:22:02 +03:00
Dobromir Popov
54d19f9a29 chore: replace fake protocol story with real harness 2026-07-19 00:22:03 +03:00
Dobromir Popov
377bc3475c chore: reconcile DGR-023 completion projection 2026-07-18 15:57:10 +03:00
Dobromir Popov
673830eac8 chore: reconcile DGR-023 completion projection 2026-07-18 15:56:59 +03:00
Dobromir Popov
902ecde363 [verified] feat: pin native protobuf and gRPC generation 2026-07-17 23:43:03 +03:00
Dobromir Popov
db59caa8e9 [verified] fix: enforce canonical native runtime pin 2026-07-17 23:20:03 +03:00
Dobromir Popov
ad66f7a4d8 feat: pin runtime identity to the exact llama.cpp patch stack (DGR-025)
Derive the recipe's runtime_version axis from the DGR-027 lock manifest
(exact upstream commit + ordered patch-stack byte digest) in new
meshnet_node.runtime_pin, failing closed on any lock/series/SHA256SUMS/
patch disagreement, and enforce pin discipline on runtime_version in both
the node and tracker identity implementations.

Also repair pre-existing backlog consistency: add missing DGR-022/DGR-027
completionNotes, regenerate the DGR-022/025/027 issue projections, and
relocate three pre-DGR legacy GLM issue files to issues/legacy/. Mark
DGR-025 passes=true with evidence at evidence/DGR-025/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 22:59:22 +03:00
Dobromir Popov
f83cf331c3 [verified] feat: harden llama.cpp provenance workspace 2026-07-17 16:24:46 +03:00
Dobromir Popov
ae51526e85 Merge remote-tracking branch 'origin/ralph/distributed-gguf-runtime' into ralph/distributed-gguf-runtime 2026-07-17 15:32:17 +03:00
Dobromir Popov
521a7b108a merge: close alternate DGR-001 maintenance history 2026-07-17 15:05:20 +03:00
Dobromir Popov
b7d40c5bcf fix: finish master branch integration compatibility 2026-07-17 14:35:05 +03:00
Dobromir Popov
8563d218c9 Merge branch 'ralph/distributed-gguf-runtime' of https://git.d-popov.com/popov/neuron-tai into ralph/distributed-gguf-runtime 2026-07-17 13:33:08 +02:00
Dobromir Popov
f0197bfa83 fix: reconcile merged runtime behavior 2026-07-17 14:29:09 +03:00
Dobromir Popov
6aced6a005 fix: reconcile legacy branch runtime with current GGUF 2026-07-17 14:21:02 +03:00
Dobromir Popov
66d9888a11 memory 2026-07-17 13:19:14 +02:00
Dobromir Popov
c758106a42 Merge branch 'archived_ralph/proxy-stream-cancellation' into merge/all-branches-into-master 2026-07-17 13:45:04 +03:00
Dobromir Popov
f0ddb69d33 Merge branch 'archived_ralph/dgr-001-performance-contract' into merge/all-branches-into-master
# Conflicts:
#	.claude/memory/MEMORY.md
#	.scratch/distributed-gguf-runtime/PRD.md
#	.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
#	.scratch/distributed-gguf-runtime/README.md
#	.scratch/distributed-gguf-runtime/architecture.md
#	.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
#	.scratch/distributed-gguf-runtime/implementation-strategy.md
#	.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md
#	.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md
#	.scratch/distributed-gguf-runtime/milestones.md
#	.scratch/distributed-gguf-runtime/prd.json
#	docs/issues/distributed-gguf-runtime/01-lock-the-safetensors-versus-gguf-performance-contract.md
#	docs/issues/distributed-gguf-runtime/02-adopt-the-versioned-grpc-shard-protocol.md
#	docs/issues/distributed-gguf-runtime/03-define-exact-artifact-and-runtime-recipe-identity.md
#	docs/issues/distributed-gguf-runtime/05-implement-dense-llama-range-aware-gguf-ownership.md
#	docs/issues/distributed-gguf-runtime/06-implement-architecture-defined-boundary-input-output.md
2026-07-17 13:44:52 +03:00
Dobromir Popov
9cb334cced Merge branch 'archived_ralph/deepseek-v4-flash-epic' into merge/all-branches-into-master 2026-07-17 13:44:12 +03:00
Dobromir Popov
ffe937678a Merge branch 'ralph/distributed-gguf-runtime' into merge/all-branches-into-master 2026-07-17 13:44:07 +03:00
Dobromir Popov
a35d86f343 chore: archive historical task programs 2026-07-17 12:41:46 +03:00
Dobromir Popov
9b257d9a1b feat: define shard lifecycle status contract 2026-07-17 11:58:50 +03:00
Dobromir Popov
3611b2cf9e Revert "fix: support headless Gitea credentials"
This reverts commit efd1cf4ef6.
2026-07-17 11:31:30 +03:00
Dobromir Popov
efd1cf4ef6 fix: support headless Gitea credentials 2026-07-17 10:47:15 +03:00
Dobromir Popov
ab466ce6b6 feat: add activation stream envelope 2026-07-17 02:36:24 +03:00
Dobromir Popov
989b55970b fix: reconcile Gitea Ralph status labels 2026-07-17 00:32:08 +03:00
Dobromir Popov
9e70b94417 feat: sync Ralph stories with Gitea issues 2026-07-16 23:37:03 +03:00
Dobromir Popov
9db036f91a feat: DGR-018 canonical Ralph metadata schema 2026-07-16 23:01:55 +03:00
Dobromir Popov
369b2072cc chore: clean superseded GGUF scaffolding 2026-07-16 22:32:37 +03:00
Dobromir Popov
81b1fa6074 docs: define implementation-ready distributed GGUF roadmap 2026-07-16 22:19:50 +03:00
Dobromir Popov
994546f78e Merge remote-tracking branch 'origin/master' into ralph/proxy-stream-cancellation 2026-07-16 18:06:54 +03:00
Dobromir Popov
6fd9d93e4b fix: flush direct SSE proxy frames before completion 2026-07-16 18:06:50 +03:00
Dobromir Popov
02b3709311 feat: checkpoint batching and release-gate stories 2026-07-16 17:24:56 +03:00
Dobromir Popov
737bade989 COLIBRI RESEARCH 2026-07-16 16:22:58 +02:00
Dobromir Popov
254627629b Merge commit '47b243cd98fd94da7918cacf5725373b099208e5' into ralph/distributed-gguf-runtime 2026-07-15 23:04:52 +02:00
Dobromir Popov
1fe31ef38d feat: checkpoint distributed gguf runtime stories 2026-07-15 23:42:58 +03:00
Dobromir Popov
47b243cd98 model loading, dash 2026-07-15 13:55:38 +02:00
Dobromir Popov
2852b1f80b loading more 2026-07-15 12:54:51 +02:00
Dobromir Popov
eaf00f6add test: record public relay smoke benchmark 2026-07-15 13:42:22 +03:00
Dobromir Popov
22f28bd69a fix model load/unload 2026-07-15 12:35:32 +02:00
Dobromir Popov
97e2784b37 node registration fixes 2026-07-15 10:34:41 +02:00
Dobromir Popov
c035bad5b7 feat: wire live benchmark CLI endpoints 2026-07-15 10:34:20 +03:00
Dobromir Popov
a508768e8a feat: add live endpoint benchmark runner 2026-07-14 22:46:11 +03:00
Dobromir Popov
e6f6782995 feat: add deterministic CPU/GPU benchmark runner slice 2026-07-14 21:39:13 +03:00
Dobromir Popov
ba7c656364 node metrics 2026-07-14 20:33:02 +02:00
Dobromir Popov
b661590ac7 log window bigger 2026-07-14 17:47:20 +02:00
Dobromir Popov
5b33bf8b99 feat: compare safetensors and gguf on cpu and gpu 2026-07-14 18:45:12 +03:00
Dobromir Popov
c7554ef7d8 feat: add DGR-001 performance contract 2026-07-14 18:13:54 +03:00
Dobromir Popov
21e6c86147 fix: let admin placement recover joined nodes 2026-07-14 16:37:42 +02:00
Dobromir Popov
def47f1a42 Merge branch 'master' of https://git.d-popov.com/popov/neuron-tai 2026-07-14 16:11:26 +02:00
Dobromir Popov
8cb00e951f feat: show admin node pool capacity 2026-07-14 16:11:18 +02:00
Dobromir Popov
7b3399760e chore: wrap up completed story metadata 2026-07-14 17:09:04 +03:00
Dobromir Popov
f102be1098 docs: retarget gguf epic to DeepSeek-V4-Flash 2026-07-14 16:24:39 +03:00
Dobromir Popov
cae7c2b171 chore: triage maintenance review and close completed stories 2026-07-14 14:33:09 +03:00
Dobromir Popov
1749f9b4ad chore: triage maintenance review and close completed stories 2026-07-14 14:33:09 +03:00
Dobromir Popov
64f83d4392 feat: MAINT-002 - Update evidence READMEs for all completed stories 2026-07-14 14:23:27 +03:00
Dobromir Popov
6516a92c04 feat: MAINT-002 - Update evidence READMEs for all completed stories 2026-07-14 14:23:27 +03:00
Dobromir Popov
454a681a50 feat: MAINT-001 - Fix Ruff violations across all Python source 2026-07-14 14:17:23 +03:00
Dobromir Popov
4eeec7fa7f feat: MAINT-001 - Fix Ruff violations across all Python source 2026-07-14 14:17:23 +03:00
Dobromir Popov
a0f28b5631 chore: preserve DGR-018 preflight scripts (postponed) 2026-07-14 14:02:10 +03:00
Dobromir Popov
7925e5253d feat: implement DGR-006 tensor bundle boundary 2026-07-14 13:52:57 +03:00
Dobromir Popov
91c450840d chore: DGR-005 evidence README 2026-07-14 13:33:20 +03:00
Dobromir Popov
d6b808dcf9 chore: mark DGR-005 passes:true in PRD 2026-07-14 13:29:54 +03:00
Dobromir Popov
31065c0e12 feat: distributed GGUF shard load integration test with TinyLlama 1.1B 2026-07-14 13:01:51 +03:00
Dobromir Popov
ec36290863 feat: emit native DGR-003 shard identity 2026-07-14 11:31:10 +03:00
Dobromir Popov
f844ae6567 feat: DGR-005B endpoint ownership and graph guard 2026-07-14 11:14:35 +03:00
Dobromir Popov
252d131e7d feat: DGR-005A dense Llama owned range loader 2026-07-14 11:01:28 +03:00
Dobromir Popov
3d8f93f4aa feat: DGR-005-003-CHAIN - DGR-005 + DGR-003-emission + anchor 2026-07-14 10:48:59 +03:00
Dobromir Popov
f9722e7b57 feat: DGR-004-CHAIN - Execute chained DGR-004/005/003-emission with anchor 2026-07-14 10:34:38 +03:00
Dobromir Popov
7b8e467c6b fix: harden DGR-003 identity trust boundary 2026-07-14 09:48:42 +03:00
Dobromir Popov
7364ed6731 fix: harden DGR-017 contract continuity 2026-07-14 01:10:33 +03:00
Dobromir Popov
ad2d17541c feat: DGR-003 - Define exact Artifact and runtime recipe identity 2026-07-14 01:10:07 +03:00
Dobromir Popov
e7c780a623 feat: DGR-017 - Lock the GLM-5.2 Max target and alpha contract 2026-07-14 00:19:16 +03:00
Dobromir Popov
9580ed643e docs: harden GLM alpha resource and protocol gates 2026-07-13 22:48:26 +03:00
Dobromir Popov
5ebce15d7a docs: target GLM-5.2 Max for distributed alpha 2026-07-13 22:32:14 +03:00
Dobromir Popov
ef2a9e67e8 feat: add signed ROCm diagnostic lane 2026-07-13 21:24:43 +03:00
Dobromir Popov
b1c9deeb01 fix: cryptographically bind DGR-001 evidence 2026-07-13 19:38:14 +03:00
Dobromir Popov
9e67b829e3 fix: harden DGR-001 performance contract evidence 2026-07-13 19:10:24 +03:00
Dobromir Popov
e24db7854f feat: DGR-001 - Lock the safetensors-versus-GGUF performance contract 2026-07-13 17:55:55 +03:00
Dobromir Popov
59f2486bf2 feat: DGR-001 - Lock the safetensors-versus-GGUF performance contract 2026-07-13 17:49:09 +03:00
Dobromir Popov
d904c40f66 fix: harden DGR-002 protocol bounds 2026-07-13 17:30:54 +03:00
Dobromir Popov
30dcf953fe feat: DGR-002 - Adopt the versioned gRPC Shard protocol 2026-07-13 16:00:49 +03:00
406 changed files with 61862 additions and 3405 deletions

View File

@@ -0,0 +1,29 @@
# DGR ROCm and llama.cpp setup
As of 2026-07-13:
- Project ROCm runtime: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm`
- ROCm/TheRock build: `7.13.0a20260513`, target `gfx1151`
- `rocm-sdk-devel` is installed. Its expanded SDK lives under the venv at
`site-packages/_rocm_sdk_devel`.
- The wheel's redundant packaged payload was relocated to
`/home/popov/.local/share/rocm-sdk/7.13.0a20260513/rocm_sdk_devel` and symlinked
back into the venv because installing both packaged and expanded forms filled
the mounted drive. Do not reinstall it blindly; the wheel expands beyond
20 GB.
- HIP llama.cpp source: `/run/media/popov/d/DEV/llamacpp/llama.cpp`, commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25` (version 9991).
- HIP build: `/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip`
- HIP `llama-server` SHA-256:
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`
- Verified device: `ROCm0: Radeon 8060S Graphics`, `gfx1151`.
- Model artifacts remain under `/run/media/popov/DATA/llm`; none were put under
`/home`.
DGR-001's immutable contract remains CPU-only. GPU evidence uses the distinct
signed `gpu-diagnostic` profile because llama-server process VRAM is not yet
measurable by the benchmark driver. The profile must capture measured
llama-server startup evidence for `ROCm0` and the actual offloaded/total layer
count; configured `device` and `n_gpu_layers` values alone are not evidence.
The accepted signer fingerprint is anchored in
`.scratch/distributed-gguf-runtime/trusted-evidence-signers.json`.

View File

@@ -8,6 +8,11 @@ metadata:
# Project Status (2026-07-13)
## Selected-node model placement (2026-07-14)
- Admin Model placement now opens a node selector for load and release; the control-plane accepts optional `node_id` and targets only that registry assignment. Multi-model serving remains supported through `ADD_SHARD` and `max_loaded_shards`.
- Total node pool resource values are rendered from `/v1/network/map`'s `node.capacity` contract. Route selection remains assignment/capability/throughput/queue based; capacity is used for placement and falls back to tracker defaults only if a node truly omits it.
## Distributed inference performance (2026-07-14)
`DIP-001` is done in `.scratch/distributed-inference-performance/`: the deterministic two-node Route Session stub benchmark covers direct/relay plus cached/stateless prefill and decode. Its JSON and concise summary explicitly attribute model execution, activation encode/decode, compression, connection setup, relay queueing, local HTTP forwarding, and end-to-end seam latency. `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` passed (7); the fixture assertion checks output-token identity and connection attempts.
@@ -39,6 +44,10 @@ Historical handoff note: `/mnt/c/Users/popov/Downloads/neuron-tai-alpha-handoff-
Planning is ready at `.scratch/node-capability-admission/` with five sequential Ralph stories and ADR-0023. The design is model-agnostic: a Node must validate its selected Model Artifact/shard with a bounded real forward before Tracker routing; Qwen3.6 is only an optional development fixture. P0 adds a versioned local recipe-manifest/report contract, `meshnet-node doctor`, fail-closed startup admission, and tracker route gating. It intentionally excludes dynamic recipe/dependency installation and the future signed Node updater.
## Gitea DGR sync (2026-07-17)
Gitea is ahead of the local Markdown backlog with open DGR-022..DGR-071. The first executable P0 dependency frontier is DGR-022 (Shard lifecycle and structured status RPCs), DGR-023 (reproducible protobuf generation), DGR-025 (artifact/runtime recipe identity), and DGR-027 (llama.cpp provenance manifest). DGR-021, the named-tensor stream envelope prerequisite for DGR-022/023/025, is closed. DGR-022 is the next dependency-ordered issue and blocks DGR-024, DGR-033, and DGR-037.
## Windows CUDA node (working as of 2026-07-01)
- miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118
- RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200

1405
.fuse_hidden0002bd66000001f0 Normal file

File diff suppressed because it is too large Load Diff

1521
.fuse_hidden0002bd66000001f9 Normal file

File diff suppressed because it is too large Load Diff

1
.gitignore vendored
View File

@@ -12,6 +12,7 @@ dist/
# Ralph local runtime state
.ralph-tui/*
!.ralph-tui/config.toml
.ralph-lane/
.env

5
.ralph-supervisor.log Normal file
View File

@@ -0,0 +1,5 @@
[2026-07-23 10:24:53] supervisor started, tailer pid=1460238
[2026-07-23 10:24:53] cycle 1: running ralph-tui resume (log starts at line 978)
[2026-07-23 10:25:59] ralph-tui exited without a recognized stop reason; retrying resume in 5 min
[2026-07-23 10:33:51] supervisor started, tailer pid=1465293
[2026-07-23 10:33:51] cycle 1: running ralph-tui run (log starts at line 1150)

1304
.ralph-tui-run.log Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,13 +1,2 @@
# Ralph TUI Configuration
# Generated by setup wizard
# See: ralph-tui config help
configVersion = "2.1"
tracker = "json"
agent = "codex"
model = "gpt-5.6-terra"
maxIterations = 0
autoCommit = true
[trackerOptions]
[agentOptions]
configVersion = "2.1"

View File

@@ -0,0 +1,384 @@
# GLM-5.2 Max distributed alpha roadmap
Status: proposed executable epic target
Last updated: 2026-07-13
## Executive decision
The alpha-release target is the exact open-weight model `zai-org/GLM-5.2` served with `reasoning_effort=max` from the smallest published Unsloth GGUF recipe, `UD-IQ1_S`, across multiple consumer machines.
“Max” is a reasoning mode selected by the chat template/API request. It is not a separate model checkpoint.
Alpha is earned only when the real target model:
1. cannot fit within any one participating node's admitted memory;
2. loads as contiguous layer Shards on at least two physical consumer machines;
3. performs real GLM-5.2 MoE + DSA + IndexShare computation on every selected node;
4. responds through the existing OpenAI-compatible Meshnet API with `reasoning_effort=max`;
5. passes locked parity, usefulness, performance, telemetry, cancellation, and cleanup checks; and
6. stores all model artifacts on configured mounted-drive storage, never under `/home`.
The shortest safe path is not “support every GGUF architecture.” Dense Llama remains a small structural fixture. GLM-5.2 moves onto the critical path immediately after exact recipe identity and the pinned llama.cpp boundary. Qwen expansion, 1M context certification, MTP/speculative decoding, broad concurrency, and automatic route repair are post-alpha.
## 1. Exact target contract
### 1.1 Source model
| Field | Locked target |
|---|---|
| Official repository | `zai-org/GLM-5.2` |
| Official revision observed 2026-07-13 | `b4734de4facf877f85769a911abafc5283eab3d9` |
| Model-weight license | MIT |
| Official code/documentation license | Apache-2.0 |
| Architecture | `glm_moe_dsa` / `GlmMoeDsaForCausalLM` |
| Official architecture label | 744B total / approximately 40B active per token |
| Exact stored checkpoint tensors | 753,329,940,480 parameters |
| Transformer layers | 78 backbone layers plus one shared NextN/MTP layer in the artifact |
| Layer types | first 3 dense; remaining 75 sparse MoE |
| Routed experts | 256 |
| Experts selected | 8 routed experts plus shared expert path |
| Hidden width | 6,144 |
| Attention | MLA under DSA, lightning indexer top-k 2,048 |
| IndexShare | indexer roles are encoded by `indexer_types`; consumers reuse prior Full-layer indices |
| Architectural maximum context | 1,048,576 tokens |
| Alpha reasoning mode | `reasoning_effort=max` |
The runtime must derive these values from the pinned artifact and fail closed on contradictory metadata. Marketing names are not compatibility identity.
### 1.2 Alpha GGUF artifact
| Field | Locked target |
|---|---|
| GGUF repository | `unsloth/GLM-5.2-GGUF` |
| GGUF revision observed 2026-07-13 | `abc55e72527792c6e77069c99b4cb7de16fa9f23` |
| Quantization | `UD-IQ1_S` |
| Files | six GGUF shards |
| Exact published bytes | 216,715,360,960 bytes |
| Binary GiB | 201.832 GiB |
| Published quality indication | about 76.2% top-1 agreement with the high-precision reference on Unsloth's quantization analysis |
| Mounted storage rule | configured mounted drive only; never `/home` |
Before downloading 216.7 GB, DGR-017 must generate a checked-in target manifest containing repository revisions, expected filenames, byte sizes, and resolved LFS SHA-256 values. Download is resumable and verified before route admission.
`UD-IQ1_M` (228,492,966,624 bytes / 212.801 GiB) is the first diagnostic fallback if `UD-IQ1_S` exposes a runtime or quality defect. It does not satisfy the explicit “lowest quantization” alpha target unless the target contract is changed by human review.
### 1.3 Runtime semantics required for alpha
Required:
- GGUF parsing and quantized kernels from one exact llama.cpp pin.
- Correct GLM-5.2 MoE routing, selected experts, and shared expert.
- Correct compressed MLA KV cache for locally owned layers.
- Native DSA lightning indexer and sparse attention.
- Correct IndexShare Full/Shared role execution from artifact metadata.
- Range-owned contiguous transformer layers; each owned layer keeps all of its experts local.
- Head-only embeddings and tail-only final norm/output head/sampling.
- Architecture-defined activation boundary and optional DSA index sideband.
- `reasoning_effort=max` chat-template behavior through the public API.
- F32 seam correctness lane and a separately certified production activation dtype.
Not required for alpha:
- MTP/speculative decoding. The trailing NextN tensors may be loaded or explicitly excluded according to a certified recipe, but cannot be silently misinterpreted.
- Full 1,048,576-token context.
- Continuous batching beyond one target session.
- Public-WAN tensor or expert parallel collectives.
- Automatic mid-generation repair or KV migration.
- Every CPU/GPU backend combination.
## 2. Minimum resource envelope
### 2.1 Weight and runtime memory
The smallest artifact occupies 201.832 GiB before KV, DSA indexer state, scratch buffers, backend workspaces, process memory, and the operating system. **224 GiB aggregate runtime-accessible memory is only the experimental hard-fit floor**, consistent with Unsloth's approximate 223 GB one-bit requirement. It is not a conservative operational envelope.
For admission, each node reserves:
```text
max(20% of physically usable memory, 8 GiB)
```
The remainder is the combined weight-plus-KV placement budget. Actual peak scratch is measured by backend/context and can force one extra node. Unified memory is counted once: integrated-GPU “VRAM” must not be added again to the same physical system RAM.
| Physical usable tier | Minimum reserve | Weight + KV placement budget | IQ1_S 16K arithmetic minimum | Operational position |
|---:|---:|---:|---:|---|
| 32 GiB | 8.0 GiB | 24.0 GiB | 9 nodes | use 10 if attempted; latency-heavy |
| 48 GiB | 9.6 GiB | 38.4 GiB | 6 nodes | possible; latency-heavy |
| 64 GiB | 12.8 GiB | 51.2 GiB | 4 nodes | hard minimum; **5 recommended** |
| 96 GiB | 19.2 GiB | 76.8 GiB | 3 nodes | recommended |
| 128 GiB unified/system | 25.6 GiB | 102.4 GiB | 2 nodes | arithmetic hard minimum; **3 recommended** |
The planner must use exact tensor byte ownership, not equal percentages. Embeddings, final head, dense versus MoE layers, shared experts, indexer tensors, quant block alignment, KV distribution, and backend workspace make equal layer counts unequal in memory.
Recommended first target route: **three 96/128-GiB-class physical machines** or **five 64-GiB-class machines**, on the same wired switch with mounted model storage. Four 64-GiB or two 128-GiB machines are fit probes only and qualify solely if exact placement and measured peak-memory evidence retain the required reserve with no swap/overcommit.
### 2.2 KV cache
GLM-5.2 MLA caches 576 latent/rope values per token per backbone layer. Correct DSA also caches 128-dimensional indexer keys: ideally only for the 21 Full indexer layers, while the current experimental implementation may allocate them across all 78 layers. Alpha locks **Q8_0 KV** for quality and budgets the conservative current-implementation layout.
| Context × concurrency | MLA-only Q8 | Optimized DSA Q8 | Conservative current-DSA Q8 | Conservative current-DSA F16 |
|---:|---:|---:|---:|---:|
| 16,384 × 1 | 0.73 GiB | 0.77 GiB | **0.89 GiB** | 1.68 GiB |
| 131,072 × 1 | 5.83 GiB | 6.18 GiB | **7.12 GiB** | 13.41 GiB |
| 1,048,576 × 1 | 46.62 GiB | 49.41 GiB | **56.98 GiB** | 107.25 GiB |
These are planning estimates, not admission truth. The runtime must report measured allocated/resident MLA and indexer cache by Shard. Alpha configures a 16,384-token window, Q8_0 KV, and one session. Longer contexts and lower-bit KV are separate quality/resource certification gates.
### 2.3 Activation seams and network
A BF16 hidden-state boundary is 6,144 elements = 12,288 bytes/token before framing.
- A 16,384-token prefill sends about 192 MiB per seam.
- One decode token sends about 12 KiB per seam.
- A 512-token decode sends about 6 MiB per seam.
- Four nodes imply three serial seams.
If a Shard boundary splits an IndexShare producer/consumer group, a 2,048-entry int32 top-k sideband can add up to 8 KiB/query before framing. The route planner should prefer boundaries that preserve complete IndexShare ownership groups. The protocol must still support and validate the named sideband because memory fit may force an internal group split.
Decode bandwidth is small, but every generated token crosses all seams serially, so node count and per-hop latency dominate. Alpha requires a same-switch wired route: **2.5 GbE minimum and 10 GbE recommended**, with measured one-way/RTT, serialization, and queue latency. A 1 GbE route may be retained as fit-only evidence but is not the recommended alpha topology. Alpha records per-seam bytes, p50/p95 transfer latency, retries, and checksum failures; no speed claim is inferred from link rate alone.
### 2.4 Storage
The shortest alpha path allows every node to hold the complete six-file source GGUF while mapping/allocating only owned tensors. This minimizes artifact-transformation risk but costs 216.7 GB disk per node.
Deterministic source-bound layer packages are a follow-up optimization. If needed before target fit, every package must retain:
- source repository/revision and source file hashes;
- exact owned tensor names, layer range, and endpoint role;
- tokenizer/config identity;
- deterministic package hash; and
- proof that composing all packages matches the source tensor inventory.
## 3. Current state and critical gaps
### Completed foundation
- DGR-001: immutable CPU contract plus separate signed ROCm diagnostic. CPU v1 remains `stop`; the GPU diagnostic establishes a viable fit/performance investigation lane but does not rewrite CPU evidence.
- DGR-002: versioned backend-neutral gRPC/Protobuf Shard protocol with bounded fragments and compatibility checks.
- Existing Meshnet: Tracker, contiguous Shards, Route Sessions/epochs, relay/direct transport, local Hot KV semantics in the reference backend, cancellation, telemetry, billing, and model-agnostic admission.
### Missing before target alpha
1. Exact GLM target/artifact manifest and memory-fit planner.
2. A current llama.cpp pin proven to load and generate with the exact `UD-IQ1_S` artifact.
3. A narrow decision on native GLM-5.2 DSA/IndexShare support. As observed 2026-07-13, merged llama.cpp PR #24770 loads GLM-5.2 through a dense-MLA compatibility path, while full IndexShare/DSA PR #25407 remains open and its generic sparse path can be slower than dense fallback. Generic CPU lightning-indexer support is merged; backend coverage remains uneven.
4. A decode protocol amendment. `ActivationChunk` carries `TensorBundle`, but the current `DecodeStep` fast path carries only one `NamedTensor`; it cannot transport a hidden state plus GLM top-k sideband. Tail token/logit and sampling behavior also needs an explicit typed result contract.
5. Correct range-owned GGUF loading and memory proof.
6. GLM-specific boundary/KV/IndexShare semantics.
7. Standalone native worker and Meshnet integration.
8. Real target hardware route with no node individually able to admit the whole model.
9. Locked target parity, usefulness, speed, failure, and cleanup evidence.
### Donor policy
`Mesh-LLM/mesh-llm` is a high-value test and patch donor. Its live GLM branch was observed with 261 llama.cpp patches, 167 named for GLM/DSA/MTP-related work. That is evidence of the problem's depth, not an acceptable maintained fork boundary.
Audit and selectively reproduce the smallest independently understood pieces for:
- GLM DSA graph semantics;
- lightning indexer and sparse-attention tests;
- IndexShare metadata/Full/Shared validation;
- top-k sideband shape and lifecycle;
- stage-local KV filtering; and
- target parity/performance fixtures.
Do not import Mesh-LLM routing, discovery, scheduler, public mesh, package manager, or full patch stack. Keep Meshnet as the sole control plane and collaborate narrowly upstream with llama.cpp/Mesh-LLM maintainers where practical.
## 4. Revised roadmap
## Phase 0 — lock the target before implementation
### DGR-017: lock GLM-5.2 Max target and alpha contract
Deliver:
- machine-readable target manifest for official and GGUF revisions;
- exact `UD-IQ1_S` file/size/hash inventory;
- architecture/config/chat-template snapshot;
- memory/KV/network planner with unified-memory de-duplication;
- immutable alpha acceptance thresholds from section 5; and
- current upstream/donor status report.
Exit: target identity and alpha requirements are reviewable without downloading the model.
## Phase 1 — establish a correct whole-model oracle
### DGR-003: exact runtime recipe identity
Extend the existing generic identity with GLM fields: DSA/IndexShare metadata, adapter version, reasoning template revision, activation bundle schema, KV dtype/layout, llama.cpp pin/patch hash, and target artifact manifest hash.
### DGR-004: reproducible llama.cpp pin and narrow patch boundary
Select a current exact upstream commit only after testing its stock GLM behavior. Add clean fetch/apply/build checks. Record every donor patch and whether it is adopted, rewritten, rejected, or waiting upstream.
### DGR-018: certify whole-model GLM-5.2 runtime semantics
On a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation, or a measured equivalent:
1. verify all six `UD-IQ1_S` shards;
2. load with a stock pinned runtime and capture tensor/metadata warnings;
3. prove whether DSA, IndexShare, shared expert, and Max template are actually active;
4. add the minimum correctness patches/tests required;
5. run deterministic prefill/decode and fixed Max-mode sentinel prompts; and
6. sign the oracle recipe, output, telemetry, and limitations.
Exit: one whole-model oracle exists for the same artifact/runtime semantics the distributed path will implement. “It emits text” is insufficient.
## Phase 2 — build the generic local seam using small fixtures
### DGR-005: range-owned GGUF tensors
Keep dense Llama as a cheap structural fixture. Implement authoritative owned-tensor registration/loading, head/tail ownership, and measured resident-memory scaling. Design tensor classification so GLM adds explicit rules rather than unchecked name substitution.
### DGR-006: architecture-defined boundary
Implement named boundary bundles, F32 correctness lane, bounded fragmentation, and optional sidebands. Amend the decode fast path so it carries a versioned `TensorBundle` rather than one `NamedTensor`, while preserving a small one-tensor encoding. Define an explicit typed tail result for logits/token output and bind sampling/chat-template parameters to the recipe/request. Regenerate Python/C++ schema code and compatibility goldens. Dense fixture parity proves the seam mechanism, not GLM certification.
Exit: two local processes can execute a small dense model with correct range ownership and boundary parity.
## Phase 3 — add GLM-5.2 as the product adapter
### DGR-019: implement and certify GLM-5.2 range/DSA/IndexShare semantics
Deliver explicit support for:
- 78 main layers and endpoint tensor ownership;
- 256-expert MoE routing/top-8 and shared expert;
- compressed MLA KV by owned layers;
- DSA lightning indexer and sparse attention;
- IndexShare metadata, Full producer, Shared consumer, and sideband behavior;
- NextN/MTP tensor policy with MTP disabled or enabled explicitly;
- shard-boundary planner aware of IndexShare ownership groups; and
- whole-model versus two-stage parity against DGR-018.
Exit: a same-host two-stage target run matches the locked oracle tolerance with real GLM computation in both stages. If the full target cannot fit on one host for this check, use a layer-reduced GLM architecture fixture for graph parity and defer full-artifact output parity to DGR-020; label the distinction explicitly.
## Phase 4 — worker, KV, and Meshnet route
Execute existing stories with GLM requirements included:
1. DGR-007 — isolated local Hot KV keyed by `(Route Session, epoch)`, including DSA/IndexShare state.
2. DGR-008 — standalone C++ gRPC worker.
3. DGR-009 — Meshnet backend, capability, relay/direct, cancellation, and telemetry integration.
4. DGR-010 — small-model local two-process acceptance.
5. DGR-011 — real two-physical-machine route and heterogeneous fail-closed behavior.
6. DGR-013 subset required by alpha — node loss, cancellation, stale epoch, restart, and memory/KV cleanup.
Continuous batching (DGR-012) is deliberately not an alpha dependency. The first target release supports one admitted GLM route session; concurrency follows after target correctness and fit.
## Phase 5 — target alpha gate
### DGR-020: pass real distributed GLM-5.2 Max alpha acceptance
Use at least two physical machines and enough aggregate usable memory to meet the locked target planner. No participating node may individually admit the complete target. All stages must report real compute and exact tensor ownership.
Run the complete acceptance matrix in section 5, preserve raw logs/metrics/output, sign the evidence, and publish an explicit `alpha` or `stop` verdict. Thresholds cannot be weakened after results are known.
## Phase 6 — post-alpha hardening
After DGR-020 passes:
1. DGR-012 — 1/2/4-session continuous batching and bounded admission.
2. DGR-014 — final distributed GGUF versus reference-route performance decision.
3. 32K, 128K, 200K, then 1M context certification with quantized KV.
4. MTP/speculative decoding.
5. Deterministic range packages to remove full-artifact replication.
6. Additional backend compatibility classes and route topologies.
7. DGR-016 — narrow upstream collaboration package, split by independently reviewable llama.cpp changes.
8. DGR-015 — Qwen3/Qwen3-MoE only as later architecture expansion, not as the GLM alpha target.
## 5. Locked alpha acceptance matrix
These thresholds are set before target execution.
### 5.1 Identity and fit
- Exact official and GGUF repository revisions match the target manifest.
- All six source GGUF sizes and LFS SHA-256 values verify.
- Every route node reports owned tensor names/bytes, layer range, endpoint role, backend, KV recipe, and patch fingerprint.
- Union of owned tensors equals the certified runtime-required tensor inventory; unintended overlap is zero.
- No node's weight-plus-KV placement budget can hold the complete recipe.
- Every node reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-KV placement; measured peak scratch must remain inside that reserve.
- Aggregate peak RSS/VRAM stays within physical budgets with no swap, overcommit, mmap-only, or double-counted unified-memory success claim.
- Arithmetic-minimum topologies require exact contiguous tensor placement evidence; recommended alpha topology is 5×64 GiB or 3×96/128 GiB.
- Unified RAM/VRAM is not double-counted.
### 5.2 Semantic correctness
- Logs and graph tests prove GLM MoE/shared-expert, DSA lightning indexer, sparse attention, and IndexShare Full/Shared paths are active; dense-attention compatibility fallback cannot satisfy alpha.
- `reasoning_effort=max` is observable in the rendered template/API recipe.
- F32 same-backend seam fixture: 32 greedy decode tokens exactly match the whole-model oracle and activation tolerance is locked by DGR-006.
- Production seam on the fixed prompt corpus: greedy token agreement is at least 0.90 and mean compared-state/logit cosine similarity is at least 0.999 versus DGR-018, with no malformed or non-finite tensors.
- Incompatible artifact, tokenizer, adapter, DSA metadata, boundary, activation, KV, backend class, or runtime patch fingerprints fail closed.
### 5.3 End-to-end target run
- Context configured to 16,384 tokens with Q8_0 MLA/indexer KV.
- Fixed 4,096-token prompt lane completes prefill.
- Route uses a same-switch wired network; 2.5 GbE is the alpha minimum and 10 GbE is recommended.
- One Max-mode request generates at least 512 output tokens or reaches a valid natural EOS after at least 128 tokens.
- Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable, relevant outputs; raw prompts and outputs are retained for review.
- OpenAI-compatible response includes stable model ID, finish reason, and token usage.
### 5.4 Minimum useful performance
On the declared minimum alpha topology after one warm-up:
- median decode throughput is at least 0.5 generated token/s for the fixed Max-mode lane;
- 4,096-token-prompt TTFT is at most 10 minutes;
- no unexplained stall exceeds 60 seconds without progress telemetry;
- per-stage compute, queue, KV, seam bytes/latency, RSS/VRAM, and backend timing are present; and
- results are labeled by hardware/topology and are not generalized to other consumer systems.
If output quality passes but the speed floor fails, verdict is `stop` for alpha and the evidence selects the next optimization target. It is not relabeled as success merely because the model loaded.
### 5.5 Reliability and security
- Two consecutive cold starts load, generate, release, and exit cleanly.
- Cancellation during prefill and decode releases every stage's queued buffers and KV lease.
- One worker loss aborts the route; alpha retries only from token zero on a new compatible route.
- Stale epochs and duplicate step IDs are rejected.
- Artifact paths stay outside `/home`; logs contain no secrets or unrestricted prompt payloads.
- Synthetic workers and layer-reduced fixtures are labeled unit/integration coverage and cannot satisfy target alpha.
## 6. First execution order
The next unattended work should run in this order:
1. DGR-017 — target contract, manifest, planner, and upstream status.
2. DGR-003 — exact recipe identity.
3. DGR-004 — current llama.cpp pin and minimal patch harness.
4. Run in parallel:
- DGR-018 — whole-model oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory.
- DGR-005 and DGR-006 — generic range/boundary seam on local small fixtures.
5. DGR-019 — GLM semantics and parity after both parallel lanes pass.
6. DGR-007 through DGR-011 — native worker and real transport route.
7. Required DGR-013 failure subset.
8. DGR-020 — real target alpha verdict.
The first external hardware blocker is DGR-018, but DGR-005/DGR-006 proceed locally while that host is sourced. Do not download the full model until DGR-017's exact manifest and storage preflight pass.
## 7. Sources checked on 2026-07-13
Authoritative or primary:
- Official model card and config: <https://huggingface.co/zai-org/GLM-5.2>
- Official release/architecture blog: <https://z.ai/blog/glm-5.2>
- Official code/documentation repository: <https://github.com/zai-org/GLM-5>
- Official source revision API: <https://huggingface.co/api/models/zai-org/GLM-5.2>
- Official GLM-5 technical report: <https://arxiv.org/abs/2602.15763>
- Unsloth GGUF repository: <https://huggingface.co/unsloth/GLM-5.2-GGUF>
- Unsloth local-run/quantization guide: <https://unsloth.ai/docs/models/glm-5.2>
- llama.cpp GLM-5.2 support issue: <https://github.com/ggml-org/llama.cpp/issues/24730>
- llama.cpp merged dense-MLA compatibility loader: <https://github.com/ggml-org/llama.cpp/pull/24770>
- llama.cpp open GLM-5.2 DSA/IndexShare implementation: <https://github.com/ggml-org/llama.cpp/pull/25407>
- llama.cpp merged generic CPU lightning indexer: <https://github.com/ggml-org/llama.cpp/pull/24231>
- llama.cpp 1M-context discussion: <https://github.com/ggml-org/llama.cpp/discussions/24622>
- IndexCache/IndexShare paper: <https://arxiv.org/abs/2603.12201>
Donor/current implementation evidence:
- Mesh-LLM repository: <https://github.com/Mesh-LLM/mesh-llm>
- Mesh-LLM GLM branch noted by llama.cpp collaborator in issue #24730: `feat/jianyang-glm-52`
Web/repository observations are pinned by date and must be refreshed in DGR-017 before implementation because upstream support is moving quickly.

View File

@@ -1,252 +1,714 @@
# PRD: Performant Concurrent Distributed GGUF Runtime
# PRD: Distributed GGUF Runtime
## Overview
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
Build one lean native GGUF execution path that lets an Inference Route combine consumer machines to serve models larger than any one node can hold. Reuse the existing Meshnet control plane and llama.cpp/GGML execution engine. Adopt gRPC/HTTP2 and Protocol Buffers for the native Shard worker data plane rather than inventing a transport.
## Goal
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
Deliver benchmark-gated, concurrent, distributed GGUF inference across consumer machines through existing Meshnet control-plane behavior and a standalone native worker around upstream llama.cpp. The accepted target is DeepSeek V4 Flash, not the superseded legacy target.
## Goals
## Locked scope
- Execute one GGUF model across independently addressable contiguous Shards.
- Retain Hot KV State locally for each Shard and isolate concurrent Route Sessions.
- Batch compatible decode steps across active sessions for aggregate throughput.
- Use consumer CPU, AMD, NVIDIA, Vulkan, Metal, and mixed routes only where a real certified forward passes.
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 24 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
## Quality Gates
## Target identities
Every story must:
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
- Run its targeted `pytest` tests.
- Run `python -m compileall packages tests` for Python changes.
- Run `git diff --check`.
- Keep default tests deterministic, model-download-free, API-credit-free, and GPU-free.
- Preserve existing Transformers/safetensors behavior unless the story explicitly changes a versioned compatibility contract.
## Quality gates
Stories touching the native worker must also:
The canonical gate groups live in `prd.json`. Every story explicitly requires its applicable shared gates: deterministic targeted tests, Python compile checks where applicable, native CMake/CTest and exact patch verification where applicable, real artifact/hardware provenance for opt-in lanes, `git diff --check`, and a durable evidence handoff.
- Build the pinned C++ target with CMake.
- Run focused C++/protocol tests through CTest or the documented equivalent.
- Verify the llama.cpp patch stack applies cleanly to the exact pinned commit.
## Milestones
Real-model/hardware stories must:
- **M0 — Truth and contracts (DGR-017..DGR-020):** Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline.
- **M1 — Protocol and native substrate (DGR-021..DGR-033):** Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker.
- **M2 — Dense vertical proof (DGR-034..DGR-043):** Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing.
- **M3 — DeepSeek V4 Flash alpha (DGR-044..DGR-054):** Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off.
- **M4 — Performance and beta hardening (DGR-055..DGR-067):** Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix.
- **M5 — Release and maintenance (DGR-068..DGR-071):** Reproducible packages, upstream collaboration, beta decision, and sustainable recertification.
- Require `MESHNET_ENABLE_REAL_INFERENCE_TESTS=1`.
- Use the machine-specific mounted-drive model path and the certified runtime environment; never place model artifacts under `/home`.
- Record exact model revision, artifact hash, runtime recipe, hardware, driver/backend, commands, raw JSON metrics, and output-quality result.
- Label synthetic tests as unit coverage rather than distributed acceptance.
## User stories
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
### DGR-017: Reconcile and clean the superseded DGR backlog
## User Stories
**Milestone:** M0 · **Mode:** AFK · **State:** complete · **Depends on:** none
### DGR-001: Lock the safetensors-versus-GGUF performance contract
**Description:** As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md`, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.
**Acceptance Criteria:**
- [ ] Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.
- [ ] Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.
- [ ] Report TTFT, prefill tok/s, decode tok/s, p50/p95 latency, aggregate throughput, RSS, VRAM, artifact size, failures, and output drift in machine-readable JSON.
- [ ] Add concurrency levels 1 and 4 where memory permits.
- [ ] Write a versioned performance contract consumed by later release gates, including an explicit stop condition when llama.cpp/GGUF has no meaningful speed or fit benefit.
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-002: Adopt the versioned gRPC Shard protocol
**Description:** As a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.
### DGR-018: Define canonical Ralph and Gitea metadata schema
**Acceptance Criteria:**
- [ ] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
- [ ] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
- [ ] Define bounded chunking for prefill and a small decode fast path.
- [ ] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
- [ ] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
- [ ] Add generated-schema round-trip and compatibility tests in Python and C++.
**Milestone:** M0 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-017
### DGR-003: Define exact Artifact and runtime recipe identity
**Description:** As the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make `prd.json` the validated source from which Markdown and Gitea issues can later be generated losslessly.
**Acceptance Criteria:**
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
- [ ] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
- [ ] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
- [ ] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
- [ ] Add deterministic model-free tests for parse, validation, and generation round trips.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-004: Create the reproducible pinned llama.cpp patch stack
**Description:** As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
### DGR-019: Lock alpha and beta performance contracts
**Acceptance Criteria:**
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
- [ ] Preserve upstream license and attribution notices.
- [ ] Add a clean rebuild smoke test that does not download a model.
**Milestone:** M0 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-017
### DGR-005: Implement dense-Llama range-aware GGUF ownership
**Description:** As a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.
**Acceptance Criteria:**
- [ ] Register and allocate only `blk.N.*` tensors in the assigned range.
- [ ] Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.
- [ ] Prefer range-aware mapping from one exact source GGUF; if derivative sub-GGUFs are used temporarily, verify source/slice hashes and avoid claiming final artifact semantics.
- [ ] Report authoritative loaded range and endpoint ownership from the model, not operator CLI claims.
- [ ] Demonstrate mapped/resident memory scales with owned tensors rather than full model size.
- [ ] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
- [ ] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
- [ ] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
- [ ] Treat quants and 24/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
- [ ] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-006: Implement architecture-defined boundary input/output
**Description:** As a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.
### DGR-020: Run the controlled whole-model GGUF baseline
**Acceptance Criteria:**
- [ ] Head accepts token IDs and owns token embedding.
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
- [ ] Tail emits logits or token output through an explicit sampling contract.
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
- [ ] The adapter interface fails closed for uncertified architectures.
**Milestone:** M0 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-019
### DGR-007: Add isolated concurrent local Hot KV State
**Description:** As a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.
**Acceptance Criteria:**
- [ ] Map `(Route Session ID, route epoch)` to an isolated llama sequence or bounded context.
- [ ] Allocate KV only for owned layers.
- [ ] Support prefill append, decode append, truncate, release, TTL/LRU eviction, and explicit cache-miss response.
- [ ] Reject stale epochs and incompatible cache recipes.
- [ ] At least four concurrent sessions on a small model complete without token or KV cross-talk.
- [ ] Cancellation/release of one session leaves other sessions intact and memory returns to the configured budget.
- [ ] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
- [ ] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
- [ ] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
- [ ] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-008: Build the standalone C++ gRPC Shard worker
**Description:** As a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.
### DGR-021: Define the versioned named-tensor stream envelope
**Acceptance Criteria:**
- [ ] Worker exposes capability, health, session stream, release, cancellation, and metrics services from DGR-002.
- [ ] Worker loads one exact Artifact/recipe/Shard identity and refuses mismatched requests.
- [ ] Streaming path enforces bounded messages, flow control, deadlines, idempotency, and independent session cancellation.
- [ ] Worker does not expose raw llama.cpp RPC or arbitrary GGML graph execution.
- [ ] Graceful shutdown releases sessions; crash behavior is bounded and observable.
- [ ] Python integration tests run against a fake model mode without model downloads.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-018
### DGR-009: Integrate the native worker with Meshnet
**Description:** As the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/021-define-the-versioned-named-tensor-stream-envelope.md`, and evidence READMEs for dependencies (DGR-018) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the backend-neutral protobuf envelope used by direct and relayed Shard activation traffic.
**Acceptance Criteria:**
- [ ] Implement the existing model-backend surface without changing Transformers behavior.
- [ ] Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.
- [ ] Tracker forms only complete compatible routes and keeps uncertified recipes dark.
- [ ] Direct routes use gRPC streams; relayed routes carry the same versioned protobuf frames as opaque binary through the existing relay seam.
- [ ] Existing request/work IDs, cancellation, Generation Telemetry, billing, and per-node attribution remain correlated.
- [ ] No vLLM, Nakshatra, prima.cpp, or custom-engine control plane becomes a core dependency.
- [ ] Define schema version, request/work ID, route session/epoch, shard range/effective start, phase, position, and idempotency step.
- [ ] Define named tensors with shape, dtype, byte order, bounded fragments, compression identity, and checksum.
- [ ] Reserve extensible fields for token-ID sidebands, architecture state, recurrent state, and MTP without claiming implementations.
- [ ] Add deterministic serialization, fragmentation, checksum, unknown-field, and size-limit tests.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-010: Pass local real-model two-process acceptance
**Description:** As a release engineer, I need real local distributed parity before involving network variability.
### DGR-022: Define Shard lifecycle and structured status RPCs
**Acceptance Criteria:**
- [ ] Two local worker processes open disjoint dense-Llama ranges from the certified Artifact.
- [ ] Prefill and at least 32 greedy decode tokens match whole-model llama.cpp within the certified tolerance.
- [ ] Each worker retains only its own tensors and Hot KV State.
- [ ] Four concurrent Route Sessions pass isolation and cleanup checks.
- [ ] Report TTFT, prefill/decode throughput, seam bytes/latency, worker RSS/VRAM, KV memory, batch size, and queue time.
- [ ] Killing one worker produces a bounded structured failure rather than a deadlock.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021
### DGR-011: Pass a real heterogeneous two-machine route
**Description:** As a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/022-define-shard-lifecycle-and-structured-status-rpcs.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Complete the gRPC contract for worker capability, health, sessions, cancellation, release, and metrics.
**Acceptance Criteria:**
- [ ] Tracker selects two physical nodes with disjoint Shards and one exact certified recipe/compatibility class.
- [ ] Actual CPU/GPU execution occurs on both nodes; synthetic workers do not satisfy acceptance.
- [ ] Prefill/decode, concurrent-session isolation, telemetry, cancellation, and cleanup pass over the real transport/relay path.
- [ ] Exact hardware, network, backend, model hash, route, commands, and raw metrics are recorded.
- [ ] A model or recipe larger than one participating node's admitted memory is exercised when available.
- [ ] Output drift is measured and incompatible mixed backends fail closed.
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-012: Implement continuous batching and bounded admission
**Description:** As a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.
### DGR-023: Make Python and C++ protobuf generation reproducible
**Acceptance Criteria:**
- [ ] Node scheduler admits sessions against weight, KV, scratch, and queue budgets.
- [ ] Compatible decode steps from multiple sessions form llama.cpp batches while preserving per-session positions and outputs.
- [ ] Prefill does not starve decode; scheduling policy and bounds are explicit.
- [ ] Backpressure prevents unbounded queued activations or KV growth.
- [ ] Capability telemetry reports active sessions, queue depth, batch occupancy, KV pressure, prefill/decode rates, and rejected admissions.
- [ ] Concurrency 1/2/4/8 benchmark identifies saturation and shows no cross-session corruption.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021
### DGR-013: Harden failure, cancellation, and restart semantics
**Description:** As a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Generate identical Python/C++ protocol bindings without manual copying or checked-in build debris.
**Acceptance Criteria:**
- [ ] Deadlines and heartbeat/health loss terminate blocked stream operations.
- [ ] Cancellation propagates across every Shard and releases local KV and queued buffers.
- [ ] Duplicate steps are idempotent; uncertain mutations are never replayed silently.
- [ ] Alpha failover restarts from token zero on a newly compatible route rather than importing unverified KV.
- [ ] Worker death, stream reset, malformed bundle, stale epoch, and cache miss tests pass.
- [ ] Billing/work records distinguish completed, cancelled, failed, and unverified work.
- [ ] Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
- [ ] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
- [ ] Add Python↔C++ round-trip and descriptor compatibility tests.
- [ ] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-014: Enforce the GGUF-versus-safetensors release gate
**Description:** As the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.
### DGR-024: Implement in-memory fake gRPC seam transport
**Acceptance Criteria:**
- [ ] Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.
- [ ] Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.
- [ ] Evaluate against the DGR-001 performance contract without changing thresholds after seeing results.
- [ ] Ship recommendation is one of: promote GGUF, optimize a measured bottleneck with a new bounded task, or stop the native track.
- [ ] Results clearly separate quantization gains from transport/runtime gains.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-023
### DGR-015: Add and certify a Qwen3/Qwen3-MoE adapter
**Description:** As a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md`, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.
**Acceptance Criteria:**
- [ ] Implement explicit tensor ownership, router/top-k, expert/shared-expert, Q/K normalization, boundary bundle, and cache semantics for the selected Qwen3 family recipe.
- [ ] Do not reuse the dense-Llama adapter through unchecked name substitutions.
- [ ] Whole-model versus distributed prefill/decode parity passes the architecture-specific tolerance.
- [ ] Expert memory ownership and communication are measured.
- [ ] Real consumer-hardware acceptance and capability admission pass before the recipe becomes routable.
- [ ] Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.
- [ ] Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.
- [ ] Verify direct and opaque-relay framing preserve identical protobuf bytes.
- [ ] Tests require no sockets outside localhost, model downloads, or native accelerator.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-016: Produce the upstream llama.cpp collaboration package
**Description:** As a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.
### DGR-025: Define exact artifact and runtime recipe identity
**Acceptance Criteria:**
- [ ] Separate generic llama.cpp hooks from Meshnet protocol/control-plane code.
- [ ] Prepare minimal reproducible examples and tests for range-aware loading, boundary input/output, and layer-filtered KV.
- [ ] Compare the proposal with Nakshatra and prima.cpp evidence and explain why the API is generally useful.
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-018, DGR-021
## Functional Requirements
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md`, and evidence READMEs for dependencies (DGR-018, DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure the tracker and worker only combine numerically and operationally compatible shards.
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
2. The native runtime uses llama.cpp/GGML; vLLM remains optional as a complete managed provider.
3. Native worker communication uses gRPC/HTTP2 and Protocol Buffers with one stable stream per Route Session Activation Seam.
4. Artifact identity, runtime recipe, boundary schema, activation dtype and cache layout must match exactly before routing.
5. Hot KV State remains local to the node serving the Shard.
6. Multiple Route Sessions must execute concurrently without shared-cache corruption.
7. Nodes batch compatible active decode steps and enforce bounded admission/backpressure.
8. Unsupported architectures and hardware recipes remain non-routable until real certification passes.
9. Default tests never download models or require GPUs; real tests are explicit and preserve artifacts off `/home`.
10. The release decision is based on measured performance, fit, quality, concurrency, and reliability relative to the safetensors baseline.
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant.
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Non-Goals
### DGR-026: Provision exact split-GGUF artifacts outside /home
- Forking vLLM or importing its PagedAttention/Torch distributed runtime.
- Adopting Nakshatra, prima.cpp, llama-gguf, LiGGUF, or GPUStack as the control plane.
- Public WAN tensor/expert parallel collectives.
- QUIC, WebRTC, or a custom socket protocol.
- Automatic KV migration or mid-generation route repair in the first release.
- Speculative decoding or disaggregated prefill before the core release gate.
- Supporting every GGUF architecture before dense Llama and Qwen3-family certification.
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-025
## Success Metrics
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md`, and evidence READMEs for dependencies (DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make exact split-GGUF inputs reproducibly available from mounted-drive storage without embedding a quantization or topology assumption in product code.
- A real model larger than one admitted node can execute across consumer machines when suitable hardware/artifacts are available.
- Four or more concurrent sessions complete without cross-talk; hardware-specific saturation is measured.
- Distributed GGUF passes the locked performance/fit contract against the existing safetensors route.
- Worker and Tracker recover all resources after completion, cancellation, malformed input, and node failure.
- The critical runtime remains Meshnet plus one standalone worker and a small auditable llama.cpp patch stack.
- [ ] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
- [ ] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
- [ ] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
- [ ] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Open Questions
### DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
- Exact benchmark model and quantization lanes are selected by DGR-001 from currently supported, legally redistributable artifacts.
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-017
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Pin llama.cpp exactly through an in-repo manifest while fetching source only into an ignored build workspace.
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
- [ ] Offline reuse is supported only after the cached trees exact identity is verified.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-028: Implement numbered patch-stack apply and verification
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-027
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md`, and evidence READMEs for dependencies (DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Maintain a minimal auditable llama.cpp delta with one numbered patch per concern.
- [ ] Add deterministic apply/check/reverse verification against the exact manifest pin.
- [ ] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
- [ ] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
- [ ] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-029: Create the native CMake skeleton and deterministic CPU lane
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-027, DGR-028
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md`, and evidence READMEs for dependencies (DGR-027, DGR-028) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish an out-of-tree standalone native build with a deterministic CPU lane before accelerator matrix work.
- [ ] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
- [ ] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
- [ ] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
- [ ] Ensure build success alone does not advertise any backend/model/recipe capability.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-030: Add accelerator build presets and native CI matrix
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-029
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/030-add-accelerator-build-presets-and-native-ci-matrix.md`, and evidence READMEs for dependencies (DGR-029) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Compile all required accelerator lanes reproducibly while keeping untested real-hardware capabilities registered-dark.
- [ ] Add isolated out-of-tree presets for CUDA, ROCm, Vulkan, and Metal without changing the deterministic CPU default.
- [ ] Add a native CI/build matrix that reports unavailable SDKs as explicit unavailable/skipped lanes rather than false success.
- [ ] Compile each available lane and preserve exact compiler, SDK, upstream pin, patch-stack, and build-option evidence.
- [ ] Keep every backend/model/recipe lane registered-dark until a separate real-hardware certification record exists.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-031: Introduce the project-owned `ShardEngine` interface
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-025
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/031-introduce-the-project-owned-shardengine-interface.md`, and evidence READMEs for dependencies (DGR-021, DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract.
- [ ] Define load, capabilities, prefill/decode, boundary/logits result, cancel, release, health, and metrics operations.
- [ ] Use project-owned request/result/state types; expose no `ggml_tensor`, llama context, scheduler, or ABI-owned structure.
- [ ] Reserve typed MTP and architecture auxiliary-state hooks without enabling them.
- [ ] Add contract tests proving fake and future llama implementations obey identical lifecycle semantics.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-032: Implement deterministic fake `ShardEngine`
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-031
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/032-implement-deterministic-fake-shardengine.md`, and evidence READMEs for dependencies (DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Provide an engine fixture that deterministically transforms typed boundary bundles and session state.
- [ ] Support head, middle, tail, prefill, decode, cancellation, and release with deterministic outputs.
- [ ] Model isolated session/epoch state and deterministic cache-miss/stale-epoch failures.
- [ ] Support configurable delay, memory pressure, malformed output, and crash injection.
- [ ] Contract tests distinguish fixture evidence from real-model certification.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-033: Build a standalone fake C++ gRPC Shard worker
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-024, DGR-032
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-024, DGR-032) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the standalone worker process, stream, lifecycle, and supervision shape before llama.cpp integration.
- [ ] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
- [ ] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
- [ ] Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
- [ ] The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-034: Implement dense-Llama range-aware GGUF ownership
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-028, DGR-029, DGR-031
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md`, and evidence READMEs for dependencies (DGR-028, DGR-029, DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Load only the dense-Llama tensors owned by a shard range.
- [ ] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
- [ ] Derive authoritative range and endpoint ownership from the loaded engine state.
- [ ] Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
- [ ] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-035: Implement dense architecture boundary input/output
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-031, DGR-034
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md`, and evidence READMEs for dependencies (DGR-021, DGR-031, DGR-034) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute dense head/middle/tail ranges through an architecture-defined residual boundary.
- [ ] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
- [ ] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
- [ ] Tail returns logits or sampled-token output under an explicit contract.
- [ ] Uncertified architectures and incompatible boundary schemas fail closed.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-036: Prove dense fixture and real-model range parity
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-033, DGR-035
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md`, and evidence READMEs for dependencies (DGR-033, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the engine/protocol seam before adding session concurrency.
- [ ] Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
- [ ] A small real dense GGUF passes whole-model versus two-range prefill parity.
- [ ] At least 32 greedy decode tokens match the locked tolerance.
- [ ] Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-037: Bind llama.cpp to the standalone worker
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-023, DGR-031, DGR-034, DGR-035
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-023, DGR-031, DGR-034, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Replace the fake engine with the pinned llama.cpp `ShardEngine` implementation without changing the public worker contract.
- [ ] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
- [ ] All execution passes through `ShardEngine`; llama.cpp implementation types remain private.
- [ ] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
- [ ] Graceful shutdown releases model/session resources; injected process death is observable and bounded.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-038: Implement isolated shard-local Hot KV State
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-037
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md`, and evidence READMEs for dependencies (DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Map each route session and epoch to isolated state containing only locally owned layers.
- [ ] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context.
- [ ] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
- [ ] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
- [ ] Release/eviction returns memory to the configured budget without affecting other sessions.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-039: Pass local two-process dense acceptance
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-036, DGR-037, DGR-038
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md`, and evidence READMEs for dependencies (DGR-036, DGR-037, DGR-038) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the first real, independently executing distributed GGUF route on one host.
- [ ] Two worker processes open disjoint dense ranges and both execute real prefill/decode work.
- [ ] Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass.
- [ ] Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics.
- [ ] Killing one worker returns a bounded structured failure rather than hanging.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-040: Add node-side native worker supervision
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-033, DGR-037
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md`, and evidence READMEs for dependencies (DGR-033, DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Let the existing node service safely start, probe, stop, and restart the standalone worker.
- [ ] Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination.
- [ ] Startup verifies worker binary, artifact identity, recipe, and range before registration.
- [ ] Crashes or health loss make the capability unavailable without corrupting the Transformers backend.
- [ ] Tests use the fake worker and deterministic crash injection.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-041: Register native Shard capabilities without redesigning Meshnet
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-025, DGR-040
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md`, and evidence READMEs for dependencies (DGR-025, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt native worker identity and capacity to existing backend-agnostic capability admission.
- [ ] Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status.
- [ ] Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic.
- [ ] Uncertified backend/model/recipe combinations are visible but unroutable.
- [ ] Existing Transformers registration and route tests remain unchanged in behavior.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-042: Carry native frames through direct and existing relay seams
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-024, DGR-040
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md`, and evidence READMEs for dependencies (DGR-024, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Use direct gRPC where available and the existing relay as opaque protobuf transport without redesigning relay behavior.
- [ ] Direct paths use the long-lived gRPC activation stream.
- [ ] Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract.
- [ ] Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths.
- [ ] Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-041
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md`, and evidence READMEs for dependencies (DGR-041) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt exact GGUF recipe/range compatibility and measured capacity, queue, seam-cost, health, and reliability data into the existing backend-agnostic routing inputs without redesigning routing or load balancing.
- [ ] Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts.
- [ ] Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms.
- [ ] Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics.
- [ ] Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-044: Pin the DeepSeek V4 Flash target contract
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-019, DGR-025, DGR-026, DGR-027
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/044-pin-the-deepseek-v4-flash-target-contract.md`, and evidence READMEs for dependencies (DGR-019, DGR-025, DGR-026, DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Fix the alpha/beta model identity and upstream maturity assumptions before implementing the adapter.
- [ ] Pin Hugging Face target SHA `60d8d70770c6776ff598c94bb586a859a38244f1`.
- [ ] Record the selected GGUF provenance and exact source binding without assuming one quant.
- [ ] Record that llama.cpp support began with PR 24162/commit `8c146a8` and remains maturing; the build manifest still pins one exact validated upstream commit.
- [ ] Record 43 main layers plus MTP and all known architecture-specific state obligations.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-045: Inventory V4 GGUF tensors and layer ownership
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-026, DGR-044
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/045-inventory-v4-gguf-tensors-and-layer-ownership.md`, and evidence READMEs for dependencies (DGR-026, DGR-044) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Derive exact head, repeated-layer, tail, expert, auxiliary-state, and MTP ownership from the pinned artifact.
- [ ] Produce a machine-readable inventory tied to the target SHA and exact GGUF artifact.
- [ ] Classify embeddings, 43 main layers, final head, MTP, mHC, attention variants, indexer/compressor, and MoE tensors.
- [ ] Validate half-open range ownership and fail on unclassified required tensors.
- [ ] Inventory tooling works across quants without embedding quant names in architecture logic.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-046: Define the V4 typed architecture boundary schema
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-045
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md`, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.
- [ ] Define a versioned named bundle for the mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.
- [ ] Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.
- [ ] Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.
- [ ] Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-047: Adapt the upstream V4 mHC boundary for ranged ownership
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-045, DGR-046
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.
- [ ] Represent and validate the upstream V4 4×4096 mHC boundary without flattening semantic axes.
- [ ] Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.
- [ ] Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.
- [ ] Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-048: Carry token-ID sideband through the first three hash-routed layers
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-046, DGR-047
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/048-carry-token-id-sideband-through-the-first-three-hash-routed-layers.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure any stage containing the first three hash-routed layers receives exact token identities.
- [ ] Head emits token IDs in the typed sideband for boundaries that require them.
- [ ] Intermediate stages preserve or consume the sideband according to their layer range.
- [ ] Routes splitting before, within, and after the first three layers pass deterministic partition tests.
- [ ] Missing or mismatched token IDs fail closed rather than silently selecting incorrect experts.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-049: Keep V4 attention and auxiliary state shard-local
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-046, DGR-047
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/049-keep-v4-attention-and-auxiliary-state-shard-local.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Bind V4 CSA/HCA/SWA/indexer/compressor and KV state to the owning shard session/epoch rather than transferring caches over the WAN.
- [ ] Map CSA, HCA, SWA, indexer, compressor, and KV ownership from the pinned upstream graph to each owned layer range.
- [ ] Key all such caches/state by route session and epoch with bounded lifecycle, memory accounting, stale-epoch rejection, release, and eviction.
- [ ] Prove cross-stage messages carry only the typed architecture boundary and cache expectations, never per-layer cache contents.
- [ ] On route change or state loss, use explicit cache miss and re-prefill/restart; do not migrate V4 caches across workers.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-050: Validate upstream V4 MoE and hash-routing execution under ranged ownership
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-045, DGR-047
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/050-validate-upstream-v4-moe-and-hash-routing-execution-under-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Validate the pinned upstream llama.cpp V4 MoE and hash-routing graph when layers are range-owned, without reimplementing routing or expert kernels in Meshnet.
- [ ] Load range-owned tensors for 256 routed plus one shared expert and validate upstream execution activates six routed experts where specified.
- [ ] Validate the first three upstream hash-routed layers receive exact token-ID sideband across splits before, within, and after them.
- [ ] Compare upstream whole-model and ranged router/expert outputs, shared-expert contribution, and malformed sideband failures.
- [ ] Document that llama.cpp owns V4 MoE, attention, hash routing, graph, and kernels; no expert-parallel WAN collective or Meshnet reimplementation is introduced.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-051: Assemble the DeepSeek V4 Flash `ShardEngine` adapter
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/051-assemble-the-deepseek-v4-flash-shardengine-adapter.md`, and evidence READMEs for dependencies (DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Assemble range ownership hooks and boundary/local-state adapters around upstream llama.cpp V4 execution behind the stable ShardEngine interface.
- [ ] Support valid head, middle, tail, and whole-model ranges over all 43 main layers using the pinned upstream llama.cpp V4 graph/kernels.
- [ ] Integrate exact range ownership, mHC boundary, positions, required token-ID sideband, and shard-local session/epoch state adapters.
- [ ] Report MTP reserved-but-not-executable for alpha and reject unsupported split points or cache expectations.
- [ ] Include exact adapter/schema/runtime/artifact identity and valid split restrictions without reimplementing V4 MoE, attention, mHC, or hash routing.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-052: Pass local V4 whole-model versus ranged parity
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-038, DGR-051
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/052-pass-local-v4-whole-model-versus-ranged-parity.md`, and evidence READMEs for dependencies (DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4 prefill and decode correctness locally before network certification.
- [ ] Compare pinned whole-model llama.cpp with at least two valid ranged partitions.
- [ ] Prefill intermediates and locked-length greedy decode pass architecture-specific tolerances.
- [ ] Test splits around the first three hash-routed layers and at least one CSA/HCA/SWA/indexer/compressor transition.
- [ ] Four concurrent sessions pass KV/state isolation and cleanup.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-053: Certify a real 24-stage V4 route
**Milestone:** M3 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-030, DGR-043, DGR-052
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md`, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.
- [ ] Run one documented 24-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.
- [ ] Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.
- [ ] Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.
- [ ] Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.
- [ ] Only the exact real-hardware lanes exercised become eligible for advertisement.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-054: Enforce the DeepSeek V4 Flash alpha gate
**Milestone:** M3 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/054-enforce-the-deepseek-v4-flash-alpha-gate.md`, and evidence READMEs for dependencies (DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Release alpha only when V4 is correct and clears the pre-locked useful-speed requirement.
- [ ] Evaluate correctness, model fit, TTFT, prefill/decode speed, seam cost, memory, and failure rate against DGF-004 without changing thresholds.
- [ ] Separate quant/model-fit gains from runtime/transport gains.
- [ ] Decision is `alpha`, `optimize measured bottleneck`, or `stop`; failures cannot be relabeled as certification.
- [ ] Alpha documentation states MTP is reserved but not implemented and identifies every dark hardware lane.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-055: Implement compatible continuous decode batching
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-038, DGR-040, DGR-054
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/055-implement-compatible-continuous-decode-batching.md`, and evidence READMEs for dependencies (DGR-038, DGR-040, DGR-054) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Batch compatible active sessions without violating route, recipe, position, or architecture state.
- [ ] Batch only sessions with compatible engine, recipe, phase, boundary schema, and state layout.
- [ ] Preserve per-session positions, sequence IDs, outputs, cancellation, and accounting.
- [ ] Decode receives bounded service while prefill remains schedulable.
- [ ] Deterministic 1/2/4/8 concurrency tests show no cross-session corruption.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-056: Enforce bounded admission, queues, and backpressure
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-055
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/056-enforce-bounded-admission-queues-and-backpressure.md`, and evidence READMEs for dependencies (DGR-055) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prevent weight, KV, state, scratch, and stream buffers from exceeding admitted capacity.
- [ ] Admission accounts separately for weights, KV/state, scratch, in-flight fragments, and queue slots.
- [ ] Queue and stream limits produce structured retryable or terminal responses.
- [ ] Prefill cannot starve decode and one route session cannot monopolize queue capacity.
- [ ] Telemetry reports active sessions, queue depth/time, batch occupancy, memory pressure, and rejects.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-057: Benchmark batching saturation and fairness
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-055, DGR-056
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/057-benchmark-batching-saturation-and-fairness.md`, and evidence READMEs for dependencies (DGR-055, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Identify useful concurrency and saturation points without assuming one global batch size.
- [ ] Run controlled concurrency 1/2/4/8 and record aggregate throughput, per-request p50/p95, TTFT, queue time, occupancy, and memory.
- [ ] Measure dense and V4 lanes separately where hardware permits.
- [ ] Verify bounded fairness and no prefill/decode starvation.
- [ ] Produce machine-readable saturation recommendations consumed by capability reporting, not hardcoded constants.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-058: Harden worker and stream failure semantics
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-040, DGR-042, DGR-056
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/058-harden-worker-and-stream-failure-semantics.md`, and evidence READMEs for dependencies (DGR-040, DGR-042, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure malformed input, process death, transport reset, and cancellation terminate cleanly.
- [ ] Test worker death, stream reset, malformed bundle, checksum failure, deadline, cache miss, stale epoch, and partial fragment loss.
- [ ] Cancellation propagates across all stages and releases KV/state, queue entries, and fragment buffers.
- [ ] Duplicate steps are idempotent; uncertain mutations are never silently replayed.
- [ ] Billing/work outcomes distinguish completed, cancelled, failed, and unverified work.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-059: Implement alpha-safe route recovery
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-043, DGR-058
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/059-implement-alpha-safe-route-recovery.md`, and evidence READMEs for dependencies (DGR-043, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Recover from route loss without importing or continuing unverified state.
- [ ] On route loss or local-state cache miss, restart from token zero or re-prefill on a newly compatible route according to the locked alpha contract.
- [ ] Never import KV, CSA, HCA, SWA, indexer, compressor, recurrent, or MTP cache/state across workers.
- [ ] Keep exact same-route duplicate delivery idempotent while cross-route continuation fails closed.
- [ ] Test direct/relay loss, cache miss, replacement selection, client cancellation, and billing reconciliation.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-060: Certify V4 long-context state correctness
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-051, DGR-056, DGR-058
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md`, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4s KV and auxiliary state remain correct and bounded at long contexts.
- [ ] Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.
- [ ] Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.
- [ ] Measure memory growth, fragment sizes, TTFT, decode degradation, cancellation latency, and cleanup.
- [ ] Cache mismatch, truncation, and context-limit behavior fail deterministically.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-061: Certify existing routing with 10+ GGUF stage candidates
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-043, DGR-057, DGR-058
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/061-certify-existing-routing-with-10-gguf-stage-candidates.md`, and evidence READMEs for dependencies (DGR-043, DGR-057, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Certify that existing backend-agnostic routing consumes GGUF compatibility and measured-cost inputs for a 10+ candidate scenario without algorithm or policy redesign.
- [ ] Generate deterministic pools with 10+ compatible GGUF stage candidates plus slower, overloaded, unhealthy, and incompatible alternatives.
- [ ] Run the existing routing implementation unchanged and verify complete coverage, compatibility, acyclicity, and use of measured inputs.
- [ ] Regression-test that no stage count, quant, fixed split, architecture, or backend sequence is encoded in production routing policy.
- [ ] Verify existing route recomputation reacts to measured load/failure while tracker, load-balancing, billing, relay, and provider semantics remain unchanged.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-062: Certify a dynamic real 10+ stage V4 scenario
**Milestone:** M4 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-053, DGR-060, DGR-061
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/062-certify-a-dynamic-real-10-stage-v4-scenario.md`, and evidence READMEs for dependencies (DGR-053, DGR-060, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Demonstrate that a large real route can be dynamically selected and usefully execute V4.
- [ ] Run one real 10+-stage V4 certification scenario; stage count, quant, and split remain scenario metadata only.
- [ ] Every selected stage proves real work, exact compatible identity, local state ownership, and bounded resources.
- [ ] Record end-to-end and per-stage TTFT, prefill/decode, seam cost, queueing, memory, failures, and cleanup.
- [ ] Inject at least one incompatible or degraded candidate and verify dynamic exclusion/reselection.
- [ ] The scenario must meet its pre-locked usefulness threshold or remain failed evidence.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-063: Profile and optimize the critical seam
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-057, DGR-061
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/063-profile-and-optimize-the-critical-seam.md`, and evidence READMEs for dependencies (DGR-057, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Optimize only the measured dominant compute, serialization, copy, or transport bottleneck.
- [ ] Profile worker compute, tensor conversion, copies, protobuf framing, compression, relay/direct transport, queueing, and synchronization.
- [ ] Select one dominant bottleneck from evidence and lock its before/after benchmark.
- [ ] Implement one bounded optimization without weakening correctness or protocol identity.
- [ ] Report whether the optimization changes TTFT, prefill, decode, throughput, or memory and preserve raw data.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-064: Add negotiated activation compression
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-060, DGR-063
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/064-add-negotiated-activation-compression.md`, and evidence READMEs for dependencies (DGR-060, DGR-063) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce V4 seam cost only when a certified compression recipe improves end-to-end performance acceptably.
- [ ] Negotiate compression through recipe identity; unnegotiated or mismatched compression fails closed.
- [ ] Preserve names, shapes, semantic axes, token-ID sideband, and checksums.
- [ ] Measure quality drift, encode/decode cost, bytes, TTFT, and throughput against uncompressed boundaries.
- [ ] Compression remains dark unless it meets pre-locked quality and speed thresholds.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-065: Define V4 MTP ownership and distributed state contract
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-051, DGR-054, DGR-060
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/065-define-v4-mtp-ownership-and-distributed-state-contract.md`, and evidence READMEs for dependencies (DGR-051, DGR-054, DGR-060) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Lock MTP tensor, execution, boundary, acceptance/rejection, and session-state ownership before implementation while keeping MTP disabled for alpha.
- [ ] Derive exact MTP tensor and execution ownership from the pinned V4 target and upstream llama.cpp graph, distinguishing main-layer, tail, and MTP responsibilities.
- [ ] Define typed MTP inputs/outputs plus accepted-token, rejection, rollback/truncation, position, cache, and session/epoch lifecycle semantics.
- [ ] Specify which MTP state remains shard-local and which typed boundary values cross a seam; prohibit WAN migration of KV or V4 auxiliary caches.
- [ ] Keep every MTP recipe disabled and unroutable for alpha, with fail-closed capability negotiation and deterministic contract fixtures.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-066: Implement and benchmark V4 MTP
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-054, DGR-060, DGR-064, DGR-065
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/066-implement-and-benchmark-v4-mtp.md`, and evidence READMEs for dependencies (DGR-054, DGR-060, DGR-064, DGR-065) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Implement the locked MTP contract around upstream llama.cpp before beta, without making MTP an alpha prerequisite.
- [ ] Implement the exact DGR-065 MTP ownership and typed state contract against the pinned upstream V4 graph while preserving shard-local caches.
- [ ] Pass whole-model versus ranged MTP correctness, accepted-token, rejection, rollback/truncation, and session-isolation tests.
- [ ] Benchmark MTP-off versus MTP-on quality, accepted-token rate, TTFT, decode speed, seam bytes, and memory on the locked lane.
- [ ] Distinguish implemented, certified, disabled, and unsupported MTP recipes in fail-closed capability advertisement.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-067: Certify the backend capability matrix
**Milestone:** M4 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-030, DGR-053, DGR-062, DGR-066
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/067-certify-the-backend-capability-matrix.md`, and evidence READMEs for dependencies (DGR-030, DGR-053, DGR-062, DGR-066) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Publish only backend/model/recipe lanes proven on actual CPU, CUDA, ROCm, Vulkan, or Metal hardware.
- [ ] Maintain independent build and certification status for CPU, CUDA, ROCm, Vulkan, and Metal.
- [ ] Each advertised lane includes real hardware/driver/runtime/model evidence for parity, concurrency, cancellation, cleanup, and useful speed.
- [ ] Build success alone never makes a lane routable; unavailable hardware remains explicitly dark.
- [ ] Tracker admission consumes signed/versioned certification records rather than backend-name allowlists.
- [ ] Mixed-backend routes require exact compatibility evidence and fail closed otherwise.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-068: Package reproducible native worker releases
**Milestone:** M5 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-030, DGR-054, DGR-067
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/068-package-reproducible-native-worker-releases.md`, and evidence READMEs for dependencies (DGR-030, DGR-054, DGR-067) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Produce installable worker packages that preserve exact source, patch, protocol, and certification identity.
- [ ] Package worker binaries/configuration for supported host/backend combinations without bundling an unverified llama.cpp tree.
- [ ] Embed protocol, engine, upstream pin, patch-stack, build-toolchain, and certification fingerprints.
- [ ] Installation/startup refuses mismatched artifacts, runtime recipes, or unsupported hardware lanes.
- [ ] Produce checksums, SBOM/license attribution, reproducible build instructions, and smoke tests.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-069: Prepare narrow upstream llama.cpp collaboration patches
**Milestone:** M5 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-028, DGR-035, DGR-038, DGR-051
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/069-prepare-narrow-upstream-llama-cpp-collaboration-patches.md`, and evidence READMEs for dependencies (DGR-028, DGR-035, DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce long-term patch burden through generic upstreamable hooks without asking llama.cpp to own Meshnet.
- [ ] Separate generic range loading, boundary I/O, and filtered state hooks from project worker/protocol code.
- [ ] Provide one scoped patch, focused test, and minimal reproducer per concern against the exact upstream pin.
- [ ] Compare the proposal with prior-art evidence and current upstream V4 support/maturity.
- [ ] Prepare human-ready design/outreach text; actual issue/PR submission remains a human action.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-070: Enforce the DeepSeek V4 Flash beta gate
**Milestone:** M5 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/070-enforce-the-deepseek-v4-flash-beta-gate.md`, and evidence READMEs for dependencies (DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Promote beta only after concurrency, resilience, long-context, MTP, scale, packaging, and speed gates pass.
- [ ] Evaluate all beta metrics against DGF-004 without changing thresholds after observing results.
- [ ] Require V4 parity, batching/fairness, bounded backpressure, failure recovery, long-context correctness, MTP evidence, and dynamic 10+ stage evidence.
- [ ] Publish the exact certified hardware/backend/recipe matrix; all other lanes remain dark.
- [ ] Decision is `beta`, `targeted optimization`, or `stop/rollback`, with unresolved failures listed explicitly.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
### DGR-071: Establish upstream pin, patch, and certification maintenance
**Milestone:** M5 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-069, DGR-070
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/071-establish-upstream-pin-patch-and-certification-maintenance.md`, and evidence READMEs for dependencies (DGR-069, DGR-070) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Keep beta reproducible and detect upstream/model/hardware drift before it reaches routable nodes.
- [ ] Add a documented update procedure for llama.cpp pin, numbered patches, protocol/engine versions, V4 target revision, and certification records.
- [ ] A candidate pin update must apply/build/test in isolation and cannot replace the production pin automatically.
- [ ] Any changed graph, tensor inventory, boundary, cache/state layout, backend behavior, or performance invalidates affected certification.
- [ ] Add scheduled model-free checks plus opt-in real-hardware recertification instructions and rollback procedure.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Non-goals
- Replacing or redesigning Tracker routing/load-balancing, billing, relay, or provider policy.
- Reimplementing DeepSeek V4 architecture graphs or kernels in Meshnet.
- Hardcoding a quant, stage count, node count, split, architecture, or backend sequence.
- WAN migration of KV or V4 auxiliary caches.
- Advertising a compile-only or fixture-only hardware lane.
- Making MTP an alpha dependency.

View File

@@ -1,309 +1,57 @@
# Ralph execution context: Performant Concurrent Distributed GGUF Runtime
# Ralph context: Distributed GGUF Runtime
Status: authoritative context for every fresh Ralph iteration
Last updated: 2026-07-13
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
## Mandatory startup sequence
## Mandatory startup for every fresh story
Before changing code, every Ralph agent must:
1. Read this file and authoritative `prd.json` completely.
2. Read the generated source issue named in the selected story description.
3. Read every dependency evidence README; legacy DGR-001..016 evidence is provenance only.
4. Read `docs/adr/0024-distributed-gguf-runtime.md`, root `CONTEXT.md`, `.claude/memory/MEMORY.md`, and relevant live source/tests.
5. Inspect `git status`; preserve unrelated work. Never infer implementation from planning text or old pass states.
6. If blocked or oversized, keep `passes: false` and write an honest `BLOCKED.md`/`DECOMPOSITION.md`; never weaken criteria or fabricate evidence.
1. Read this file completely.
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
3. Read `docs/adr/0024-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
6. Read the evidence/handoff directories for every declared dependency.
7. Inspect `git status` and preserve all pre-existing working-tree changes.
## Locked scope
A fresh Ralph iteration has no conversational memory. These files are the context contract.
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 24 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
## Story sizing and interruption rule
## Target identities
Each story is intended to fit one focused Ralph context. Before implementation, estimate whether every acceptance criterion can be completed and verified in the current iteration.
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
If the story is too large, an external dependency is unavailable, or the context/provider limit prevents completion:
## Control/data-plane contract
- Do not weaken criteria.
- Do not mark the issue done or set `passes: true`.
- Avoid leaving an unverified cross-cutting partial implementation when a smaller safe spike is possible.
- Write `evidence/<TASK-ID>/DECOMPOSITION.md` or `BLOCKED.md` with the exact blocker, current verified state, proposed child stories, dependency graph and rollback/continuation instructions.
- Stop for supervised review.
Meshnet continues to own registration, coverage, existing route selection/load balancing, route epochs/sessions, direct/relay behavior, capability admission, cancellation, telemetry, billing, validation, and attribution. The GGUF adapter exposes measured inputs to those existing mechanisms. Direct seams use long-lived gRPC streams; relay seams carry byte-identical protobuf frames opaquely.
If interrupted after code changes, record every changed file, command result and unresolved invariant so the next fresh loop can verify rather than guess.
The project-owned `ShardEngine` hides llama.cpp internals. A worker loads one exact artifact/recipe/range identity. Default tests use fake/tiny fixtures. Real runs are opt-in, preserve raw metrics, and never download models under `/home`.
## Product objective
## Gitea issue synchronization
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
Gitea is a projection of `prd.json`, never a competing source of truth. Before and after every supervised Ralph run, invoke:
A distributed demo is not success. The product must provide:
- Useful measured prefill and decode speed.
- Multiple concurrent Route Sessions.
- No KV/token cross-talk.
- Bounded memory, queues, cancellation and failures.
- Real execution on every participating node.
- A model-fit or performance advantage over the current Transformers/safetensors route.
## Critical-path architecture
```text
Existing Meshnet control plane
|
Versioned Protobuf over gRPC/HTTP2
|
Project-owned standalone C++ Shard worker
|
Small exact-commit llama.cpp patch stack
```bash
python3 scripts/ralph_gitea_sync.py sync
```
Meshnet remains the only control plane and owns:
For a complete Ralph invocation with automatic state reconciliation, use:
- Tracker registration, Coverage Map, route selection and route epochs.
- Route Sessions and Activation Seams.
- Direct/relay routing.
- Capability admission.
- Cancellation, Generation Telemetry and backpressure.
- Billing, validation and per-node work attribution.
Do not introduce another scheduler/control plane from vLLM, Nakshatra, prima.cpp, llama-gguf, GPUStack or another project.
## Runtime decisions that are not open
1. Public-network Shards are contiguous transformer layer ranges.
2. llama.cpp/GGML is the native GGUF execution substrate.
3. The project owns a small standalone worker and a narrow pinned llama.cpp patch stack.
4. The native Shard protocol is Protocol Buffers over gRPC/HTTP2.
5. One long-lived bidirectional stream serves one Route Session Activation Seam.
6. The public activation boundary is a versioned named-tensor bundle.
7. Hot KV State remains local to the node serving the Shard.
8. `(Route Session ID, route epoch)` maps to an isolated llama sequence or bounded context.
9. Concurrency uses continuous batching of compatible active sessions inside each node.
10. Transformers/safetensors remains the correctness and performance baseline.
11. vLLM may be an optional complete managed provider and concept donor; it is not forked into public Shards.
12. Tensor/expert collectives are deferred to a trusted composite provider, not public WAN routes.
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
## Performance discipline
GGUF performance is a hypothesis. Never write “GGUF is faster” without measurements.
DGR-001 locks controlled benchmark lanes and thresholds. DGR-014 enforces the final distributed comparison.
Always distinguish:
- Weight quantization from activation/compute/KV dtype.
- Runtime/kernel gains from quantization/model-fit gains.
- Single-request latency from aggregate concurrency throughput.
- Synthetic unit coverage from real distributed acceptance.
Required metrics where applicable:
```text
TTFT
prefill tokens/sec
decode tokens/sec
aggregate throughput
p50/p95 latency
seam bytes and latency
queue and batch occupancy
RSS and VRAM
KV pressure
output-quality drift
failures and cleanup
```bash
scripts/ralph-gitea-run.sh ralph-tui run --prd .scratch/distributed-gguf-runtime/prd.json --agent claude --model sonnet --iterations 1 --no-tui --no-setup --direct-merge --no-sandbox
```
Do not weaken or move performance thresholds after seeing implementation results.
The sync creates/reconciles one Gitea issue per `DGR-*` story, creates missing labels/milestones, closes issues whose `passes` is true, marks the selected next eligible story `status:in-progress`, and marks blocked stories `status:blocked`. `gitea-issues.json` is a derived mapping only.
## Transport discipline
## Evidence and completion
Do not invent a raw TCP protocol, new WebSocket protocol, QUIC layer or bespoke binary control format.
The `.proto` schema is the semantic contract. Direct transport uses gRPC. Existing relay infrastructure may carry the same serialized protobuf frames as opaque binary.
Protocol requirements:
- Schema/version negotiation.
- Request/work ID.
- Route Session ID and route epoch.
- Exact Model Artifact/runtime recipe fingerprint.
- Shard range and effective overlap-safe start.
- Prefill/decode/release/cancel phases.
- Position/token range and idempotency step.
- Named tensors with shape, dtype, byte order and bounded fragments.
- Compression/checksum.
- Cache expectation/result.
- Deadlines, cancellation, flow control and structured status.
Avoid per-token channel creation and unbounded unary payloads. Generated code and build tooling must be reproducible; do not require manual copying.
## Native runtime discipline
Reuse llama.cpp for GGUF, mmap, kernels, architecture graphs, tokenizer, KV, sequences and heterogeneous backends.
The project patch stack is limited to:
- Range-aware tensor registration/loading.
- Endpoint-specific embedding/final head ownership.
- Architecture-defined intermediate input/output.
- Intermediate output before final norm/head.
- Layer-filtered KV and session mapping.
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
Dense Llama-family is first. Qwen3/Qwen3-MoE is a separate adapter after the dense release gate. Do not generalize through unchecked tensor-name substitutions.
## Existing code seams to inspect first
- `packages/node/meshnet_node/model_backend.py` — backend abstraction.
- `packages/node/meshnet_node/torch_server.py` — reference ranged execution and session behavior.
- `packages/node/meshnet_node/activation_compression.py` — current activation framing/compression.
- `packages/node/meshnet_node/route_session_benchmark.py` — existing benchmark infrastructure.
- `packages/tracker/meshnet_tracker/server.py` — registration, route and proxy behavior.
- `packages/tracker/meshnet_tracker/capability.py` — fail-closed capability admission.
- `tests/test_real_model_backend.py` — real backend coverage.
- `tests/test_tracker_routing.py` — route/session behavior.
- `tests/test_tracker_capability_admission.py` — recipe admission.
- `tests/test_route_session_benchmark.py` and `tests/test_manual_route_benchmark.py` — benchmark patterns.
- `docs/adr/0008-binary-activation-wire-format.md` — existing wire compatibility.
- `docs/adr/0012-start-layer-overlapping-shards.md` — effective start semantics.
- `docs/adr/0022-sharded-per-node-kv-cache.md` — Hot KV State contract.
- `docs/adr/0023-model-agnostic-node-capability-admission.md` — certification/admission.
Do not edit generated `build/`, `__pycache__`, egg-info, Ralph logs or unrelated scratch features.
## Planned source layout
Use these paths unless current code inspection proves a better project-consistent location. If changed, document the reason in task evidence.
```text
packages/node/native/
proto/shard_runtime.proto
cmake/
llama/
UPSTREAM_COMMIT
patches/
gguf_worker/
tests/
packages/node/meshnet_node/
native_protocol/
gguf_backend.py
runtime_recipe.py
.scratch/distributed-gguf-runtime/evidence/<TASK-ID>/
README.md
commands.txt
results.json or other machine-readable evidence
```
Generated protobuf/C++ build outputs belong in build directories unless packaging explicitly requires checked-in generated Python modules. The story must document the generation command and version.
## Story output map
| Story | Required durable outputs |
|---|---|
| DGR-001 | benchmark harness/tests; `evidence/DGR-001/performance-contract.json`; raw/summary benchmark evidence |
| DGR-002 | `packages/node/native/proto/shard_runtime.proto`; reproducible Python/C++ generation/build wiring; protocol round-trip/compatibility tests; `evidence/DGR-002/` |
| DGR-003 | exact runtime-recipe/fingerprint implementation and admission tests; `evidence/DGR-003/` |
| DGR-004 | exact upstream pin, numbered patch series, reproducible fetch/apply/build smoke; `evidence/DGR-004/` |
| DGR-005 | dense-Llama range ownership loader and memory evidence; `evidence/DGR-005/` |
| DGR-006 | architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
| DGR-007 | concurrent session/KV manager, isolation/cleanup tests; `evidence/DGR-007/` |
| DGR-008 | standalone C++ gRPC worker, fake-model integration tests, lifecycle evidence; `evidence/DGR-008/` |
| DGR-009 | Meshnet backend/registration/relay integration and tests; `evidence/DGR-009/` |
| DGR-010 | real local two-process commands, raw metrics and parity report; `evidence/DGR-010/` |
| DGR-011 | two-machine configuration, commands, hardware/network manifest and raw results; `evidence/DGR-011/` |
| DGR-012 | continuous scheduler/admission implementation and 1/2/4/8 concurrency report; `evidence/DGR-012/` |
| DGR-013 | failure/cancel/restart test matrix and resource-cleanup evidence; `evidence/DGR-013/` |
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
## Dependency handoff rule
For every dependency listed by Ralph:
1. Confirm its `passes` state in `prd.json`.
2. Read `.scratch/distributed-gguf-runtime/evidence/<DEPENDENCY-ID>/README.md`.
3. Verify referenced source paths and commands still exist.
4. Do not repeat completed work unless verification exposes a concrete defect.
5. If dependency evidence is missing or contradictory, stop and repair the dependency instead of guessing.
## Testing and hardware rules
Default tests must be deterministic, GPU-free, model-download-free and API-credit-free.
Real model tests require:
```text
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1
```
On this machine:
- Use `.venv-rocm` for real Radeon 8060S ROCm execution.
- The default Python 3.14 `.venv` is unsuitable for real ROCm inference.
- Resolve model storage through the machine-specific `.env.<hostname>` configuration.
- Never download model artifacts under `/home`.
- Real acceptance must exercise actual Tracker-routed CPU/GPU computation; synthetic workers are only unit tests.
Record exact:
- Model/revision and Artifact hash.
- Quantization and runtime recipe.
- Host/hardware/backend/driver.
- Commands and environment names without secrets.
- Raw output and metrics.
- Whether the evidence is synthetic, local-real, or multi-machine-real.
## Worktree and commit discipline
This repository may contain pre-existing changes from research or another feature.
- Inspect `git status` before editing.
- Never reset, checkout over, stash, delete or reformat unrelated changes.
- Stage only files belonging to the selected story.
- Exclude `.ralph-tui`, iteration logs, caches, generated builds, FUSE artifacts and unrelated scratch work.
- Keep one scoped commit per completed story when the supervising loop requests commits.
- Do not modify `passes` for another story.
## Mandatory finish/handoff sequence
Before emitting `<promise>COMPLETE</promise>`:
1. Verify every acceptance criterion with real command output or file evidence.
2. Run story-specific gates and repository quality gates.
3. Write `.scratch/distributed-gguf-runtime/evidence/<TASK-ID>/README.md` containing:
- Summary of changes.
- Exact files changed.
- Commands run and their real results.
- Performance/correctness evidence.
- Known limitations and deferred work.
- Compatibility or migration notes.
- Clear handoff for dependent stories.
4. Save machine-readable evidence beside it when the story produces metrics or schemas.
5. Update the source issue status to `done` only after all gates pass.
6. Preserve failures honestly. Never fabricate model, benchmark, test or hardware output.
## Authoritative references
Active decisions:
- `.scratch/distributed-gguf-runtime/README.md`
- `.scratch/distributed-gguf-runtime/implementation-strategy.md`
- `.scratch/distributed-gguf-runtime/architecture.md`
- `docs/adr/0024-distributed-gguf-runtime.md`
- `.scratch/distributed-gguf-runtime/PRD.md`
- `.scratch/distributed-gguf-runtime/prd.json`
Source research:
- `docs/research/distributed-gguf-landscape.md`
- `docs/research/distributed-gguf-github-followup.md`
- `docs/research/vllm-distributed-gguf-assessment.md`
If historical notes conflict with these files, the active decisions above win.
Each story writes `/run/media/popov/d/DEV/repos/d-popov.com/AI/.claude/worktrees/distributed-gguf-runtime/.scratch/distributed-gguf-runtime/evidence/<DGR-ID>/README.md` with exact files, commands/results, limitations, identities, and dependent-story handoff. Only `prd.json` may record `passes`; DGR-017 and DGR-018 are complete and DGR-019 onward remain false. Generated Markdown and Gitea issues cannot override it. One scoped commit per story is expected during future execution.

View File

@@ -1,46 +1,32 @@
# Performant concurrent distributed GGUF runtime
# Distributed GGUF Runtime planning workspace
Status: active benchmark-gated implementation program.
> **Implementation status:** DGR-017 through DGR-033 have verified lane evidence, including a fixture-only standalone C++ gRPC worker. These lane checkpoints still require serialized integration and remote publication; they do not claim real model inference. `prd.json` is authoritative.
## Objective
Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.
## Locked scope
## Critical path
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 24 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
```text
Meshnet control plane
-> versioned gRPC/Protobuf Shard protocol
-> project-owned standalone C++ worker
-> small pinned llama.cpp patch stack
```
## Target identities
Transformers/safetensors remains the correctness baseline. vLLM remains an optional complete managed provider and a design donor; it is not forked into the public mesh.
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
## Planning artifacts
## Navigation
- **[Mandatory Ralph context](RALPH-CONTEXT.md)** — read first in every fresh iteration
- [Task evidence contract](evidence/README.md)
- [Implementation strategy](implementation-strategy.md)
- [Current architecture](architecture.md)
- [PRD](PRD.md)
- [Ralph backlog](prd.json)
- [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md)
- [Milestones](milestones.md)
- [Issues](issues/)
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)
- [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md)
- [vLLM assessment](../../docs/research/vllm-distributed-gguf-assessment.md)
## Ralph execution
Use supervised one-story iterations for this high-risk runtime:
```bash
ralph-tui run \
--prd .scratch/distributed-gguf-runtime/prd.json \
--agent claude --model opus \
--iterations 1 --no-tui --no-setup --verify
```
Inspect the diff, run the story gates, and commit one verified story before the next iteration. Real-model stories require the explicit environment gate and mounted-drive model storage.
- [`prd.json`](prd.json) — sole authoritative 55-story backlog, DGR-017..071.
- [`PRD.md`](PRD.md) — human-readable projection of goals, gates, and all stories.
- [`RALPH-CONTEXT.md`](RALPH-CONTEXT.md) — mandatory fresh-session context.
- [`architecture.md`](architecture.md), [`implementation-strategy.md`](implementation-strategy.md), [`milestones.md`](milestones.md) — design and execution sequence.
- [`issues/`](issues/) — generated story specs; files 01..16 are retained legacy artifacts pending DGR-017.
- [`evidence/`](evidence/) — provenance and future per-story handoffs.

View File

@@ -1,259 +1,45 @@
# Performant Concurrent Distributed GGUF Architecture
# Distributed GGUF Runtime architecture
Status: current target architecture
Last updated: 2026-07-13
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
## Product invariant
The system exists to serve high-quality models that exceed one consumer node's memory while retaining useful interactive speed and aggregate concurrency. A feature that only produces a distributed demo but is slower, globally serialized, or impossible to operate on consumer hardware is not complete.
## Locked scope
## Existing control plane
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 24 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
Meshnet remains the only public control plane:
## Target identities
- Tracker registration, Coverage Map, route scoring and assignment.
- Contiguous Shards and overlap-safe effective starts.
- Stable Route Sessions and route epochs.
- Local per-Shard Hot KV State in the reference backend.
- Direct/relay transport, cancellation and backpressure.
- Generation Telemetry, billing, validation and per-node attribution.
- Model-agnostic capability admission.
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
No external engine replaces these responsibilities.
## Runtime topology
## Topology
```text
OpenAI-compatible client
|
Gateway / Tracker Node
|
ordered Inference Route
|
+-- head Shard: tokenizer/embedding + early layers
| local weights and Hot KV State
|
+-- middle Shard(s): architecture boundary + owned layers
| local weights and Hot KV State
|
+-- tail Shard: final layers + norm/head/sampling
local weights and Hot KV State
existing Meshnet Tracker/control plane
-> existing backend-agnostic route/load-balancing decision
-> direct gRPC or existing opaque relay
-> project-owned standalone C++ Shard worker
-> project-owned ShardEngine
-> pinned upstream llama.cpp + numbered range/boundary/state hook patches
-> GGUF mmap, upstream V4 graph/kernels, local per-shard state
```
Weights never move in the per-request hot path. Every node opens and verifies its local Model Artifact before becoming routable.
A route is ordered contiguous half-open ranges. Head owns token embedding; tail owns final norm/head/sampling. Compatibility fingerprints bind source/split hashes, tokenizer, architecture adapter, typed boundary, runtime pin/patches, backend, quant, activation/compute/KV layout, range, and certification.
## Primary execution substrate
## V4 boundary and state
```text
project-owned C++ Shard worker
|
small exact-commit llama.cpp patch stack
|
GGUF mmap, quantized kernels, architecture graphs,
KV/sequence operations, CPU/CUDA/HIP/Vulkan/Metal backends
```
The inter-stage boundary is semantic and versioned: mHC 4×4096 residual, positions, token IDs only where the first three hash-routed layers require them, and cache/schema expectations. CSA/HCA/SWA/indexer/compressor/KV state belongs to upstream layer execution on the owning worker and is isolated by `(route_session_id, route_epoch)`. On loss, return cache miss and re-prefill/restart. Never serialize those caches into the WAN bundle.
The patch stack adds only the missing local execution seam:
## Concurrency, failure, and admission
1. Range-aware tensor registration/loading.
2. Endpoint-specific embedding and final head ownership.
3. Architecture-defined intermediate input.
4. Architecture-defined pre-tail boundary output.
5. Layer-filtered KV and external session mapping.
The worker owns protocol translation and process lifecycle. llama.cpp never receives Tracker, relay, billing or volunteer-network code.
## Shard data plane
Use Protocol Buffers and gRPC over HTTP/2.
### Service shape
- Unary capability and health.
- Bidirectional Route Session stream.
- Explicit release and cancellation.
- Metrics suitable for capability admission and route scoring.
### Session stream
One long-lived stream represents one Route Session Activation Seam. It amortizes connection setup and inherits HTTP/2 flow control. Every message carries enough identity to reject stale or incompatible work.
```text
schema version
request/work id
Route Session id
route epoch
Model Artifact hash
runtime recipe fingerprint
Shard begin/end and effective start
prefill/decode/release/cancel phase
position and token range
idempotency step id
cache expectation/result
named tensor bundle
compression/checksum
```
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small.
Direct nodes use gRPC. Nodes requiring the existing relay carry the same protobuf frames as opaque binary through the relay session. This preserves one semantic protocol instead of maintaining separate direct and relay payload contracts.
## Architecture boundary
The public boundary is a versioned named-tensor bundle:
```text
bundle schema/version
architecture adapter and boundary point
named tensors
per-tensor shape, dtype and byte order
payload fragments
compression/checksum
```
Dense Llama may use one residual tensor. Other adapters may require more. vLLM's Llama and Qwen3-MoE PP paths demonstrate a boundary with both `hidden_states` and `residual`; therefore the generic protocol must not assume one anonymous tensor.
Only the head owns token embedding. Only the tail owns final normalization, LM head and sampling. Middle Shards exchange the architecture-defined pre-tail boundary, not final normalized embeddings.
## Hot KV State and concurrency
```text
(Route Session id, route epoch)
-> local llama sequence or bounded context
-> KV for owned layers only
-> lease, memory accounting and lifecycle
```
Required operations:
- Prefill append.
- Decode append.
- Truncate after rejected speculative positions if later enabled.
- Explicit release.
- TTL/LRU eviction.
- Cache-miss response.
- Stale-epoch rejection.
A node must not clear global KV on a new stream or serialize all requests behind one logical serving sequence.
## Continuous batching
Autoregressive dependencies remain sequential inside one Route Session. Aggregate throughput comes from batching compatible decode steps across active sessions:
```text
time 0: session A token 1 + session B token 8 + session C token 3
-> one llama batch for this Shard
time 1: next ready positions from active sessions
-> next llama batch
```
The node scheduler:
- Admits work against weight, KV, scratch and queue budgets.
- Keeps per-session token positions and outputs separate.
- Prevents long prefill from starving decode.
- Applies bounded backpressure.
- Reports active sessions, queue depth, batch occupancy, KV pressure and throughput.
The initial deterministic gate is four concurrent sessions on a small model without cross-talk. Hardware-specific limits are measured and advertised through capability admission.
## Parallelism boundaries
| Mechanism | First-runtime use |
|---|---|
| Layer/pipeline parallelism | Public Inference Route across contiguous Shards |
| Continuous batching | Inside every node across active Route Sessions |
| Data parallelism | Multiple complete routes for independent requests |
| Tensor parallelism | Deferred to a trusted composite node/managed cluster |
| Expert parallelism | Deferred to a trusted composite node/managed cluster |
| Disaggregated prefill | Deferred until core route performance passes |
| Speculative decoding | Deferred optimization |
Public WAN tensor/expert collectives are rejected for the first runtime because their per-layer communication and static rank assumptions conflict with heterogeneous volunteer nodes.
## Optional providers
### Transformers/safetensors
Remains:
- Correctness/reference backend.
- Fallback for unsupported architectures.
- Baseline for performance and output quality.
### vLLM
May run unmodified as a complete model or managed TP/PP/EP cluster represented as one logical provider. Its internal ranks are not independently routed or rewarded.
Borrow only concepts such as named bundles, continuous batching, typed compatibility fingerprints, explicit transfer lifecycle and load telemetry.
### Whole-model llama.cpp
Provides a local proxy backend, correctness oracle and performance baseline. It is not the native distributed milestone.
## Artifact and recipe compatibility
A routable recipe identifies separately:
- Source Model Artifact hash and optional derivative/slice hash.
- Architecture and adapter version.
- Tokenizer revision and vocabulary.
- Weight quantization.
- Activation interchange dtype/schema.
- Backend compute dtype and backend implementation.
- KV dtype/layout.
- RoPE/context parameters.
- llama.cpp commit and project patch version.
- Shard range and endpoint ownership.
Compatibility fails closed. Similar quantization labels or model names are not enough.
## Admission and failure
A recipe becomes routable only after a real local and distributed forward passes. Synthetic tests remain unit coverage.
Alpha failure behavior:
- Deadline or node loss cancels the Route Session.
- Every node releases KV and queued buffers.
- Uncertain mutations are not replayed silently.
- Retry starts from token zero on a newly compatible route.
- No cross-node KV import is trusted until a later signed/compatible snapshot protocol exists.
## Performance release contract
Before native development proceeds, compare the current Transformers/safetensors backend with whole-model llama.cpp under controlled model/hardware/quality lanes.
Final release compares distributed GGUF with distributed safetensors using thresholds locked before seeing final results.
Required measurements:
- TTFT.
- Prefill and decode tokens/sec.
- Aggregate concurrency throughput.
- p50/p95 latency.
- Seam bytes and latency.
- Queue/batch occupancy.
- RSS, VRAM and KV pressure.
- Output-quality drift.
- Cancellation/failure cleanup.
The GGUF path ships only if it is faster at acceptable quality or enables a larger otherwise-unroutable model at useful measured speed.
## Implementation sequence
1. Lock benchmark/performance contract.
2. Define gRPC/protobuf and exact recipe identity.
3. Pin llama.cpp and create the minimal patch stack.
4. Implement dense-Llama range loading and boundary parity.
5. Implement concurrent local KV.
6. Build and integrate the standalone worker.
7. Pass local two-process real-model acceptance.
8. Pass real heterogeneous two-machine acceptance.
9. Add continuous batching and failure hardening.
10. Enforce the GGUF-versus-safetensors release gate.
11. Add Qwen3/Qwen3-MoE as a separately certified adapter.
12. Prepare narrow upstream collaboration patches/tests.
See [the Ralph backlog](prd.json) and [implementation strategy](implementation-strategy.md).
Compatible sessions may be continuously batched within a worker while retaining isolated positions/state. Admission bounds weights, local state/KV, scratch, fragments, and queues. Uncertain cross-route mutation is not replayed. Registration can show an uncertified lane, but existing admission keeps it unroutable until signed/versioned real-hardware evidence exists.

View File

@@ -1,270 +1,40 @@
# Distributed GGUF Decision Framework
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.
## Core Vocabulary
Use the existing domain terms this way:
- **Shard**: contiguous transformer layer range. This is the compute, routing, cache, and reward unit.
- **Shard Swarm**: storage/download group for artifacts needed by a shard.
- **Inference Route**: ordered node sequence that covers all layers for one request.
- **Route Session**: one active request bound to one inference route and stable session id.
- **Hot KV State**: live per-shard cache held by the route node during a route session.
- **Prefix Snapshot**: persisted route-session state used for reuse or failover, not the hot decode path.
- **Artifact Manifest**: canonical mapping from model artifacts to semantic model parts and runtime support.
- **Generation Telemetry**: realtime progress for a route session, including phase and tokens/sec, independent of whether token deltas are streamed.
## The Five Planes
### 1. Control Plane
Owner: Tracker.
Responsibilities:
- node registry
- coverage map
- route selection
- rebalance directives
- route-session creation
- health and telemetry
- client-visible Generation Telemetry
- billing/audit records
Must not do:
- serve hot KV during every token
- become the only place model artifacts can be fetched
### 2. Artifact Plane
Owner: Shard Swarms, local node storage, optional CDN/bootstrap mirrors.
Responsibilities:
- GGUF/safetensors/tokenizer download
- content-addressed verification
- local artifact inventory
- artifact-to-layer mapping
- cache eviction
Must not do:
- define execution order by file split alone
- imply that a downloaded file chunk equals a Shard
### 3. Execution Plane
Owner: active Inference Route.
Responsibilities:
- chunked prefill
- one-step decode
- hidden-state transfer across activation seams
- start-layer handling for overlapping shards
- backpressure
Must not do:
- resend full context activations during decode
- require cross-node tensor parallel all-reduce for public v1
### 4. Session State Plane
Owner: route nodes for hot KV; cache servers only for snapshots.
Responsibilities:
- per-shard local KV ownership
- cache allocation and eviction
- cache ABI compatibility
- session close/release
- optional prefix snapshots
Must not do:
- centralize hot KV in a remote service
- let a replacement node continue from incompatible state
### 5. Economics And Trust Plane
Owner: tracker plus settlement/validation components.
Responsibilities:
- distinguish storage/seeding work from inference work
- account for prefill and decode separately
- record route participation
- sample validation events
- slash proven fraud
Must not do:
- pay a node for merely holding files as if it generated tokens
- hide public-swarm privacy limits from clients
## Hard Invariants
These are the framework rules unless we deliberately write a new ADR:
1. Public-network Shards are contiguous layer ranges.
2. Hot KV State is local to the node serving that Shard in that Route Session.
3. Artifact distribution and route execution are separate systems.
4. Decode seam payload must be `O(hidden_size)`.
5. Prefill may be `O(sequence_length * hidden_size)`, but only in bounded chunks.
6. The tracker chooses routes; nodes do not negotiate route topology peer-to-peer.
7. Model/backend-specific cache internals stay behind backend capability reports.
8. PyTorch remains the correctness/reference backend while llama.cpp/GGUF becomes the performance backend.
9. Streaming responses are preferred when feasible; Generation Telemetry is always required.
## Resolved Gates
### Gate 1: Public Shard Semantics
Decision: public-network Shards are contiguous transformer layer ranges. Tensor-parallel or ring-style execution is allowed only inside one trusted node, one colocated pod, or a future composite node abstraction.
Rationale:
- Layer ranges match the existing `Shard`, `Coverage Map`, `Inference Route`, billing, and fraud vocabulary.
- Public volunteer nodes should not require cross-node all-reduce or tight per-layer synchronization in v1.
- Existing projects such as prima.cpp and Distributed Llama can still inform local-cluster/backend execution without becoming the public routing primitive.
Consequences:
- Artifact Manifests must map files/tensors to semantic layer ranges.
- Route selection remains ordered layer coverage.
- Rewards can be attributed to layer-range work.
- Hot KV State is naturally owned by the node serving that layer range for the Route Session.
### Gate 2: Hot KV Strategy
Decision: v1 rejects centralized hot KV. Hot KV State is local to the node serving the relevant Shard in the active Route Session. Cache servers may store Prefix Snapshots for reuse, retry, or failover, but they are not in the per-token decode path.
Rationale:
- Decode is the tight loop; adding remote cache I/O there makes latency and bandwidth worse at the worst point.
- Local KV naturally follows layer-range Shard ownership.
- Centralized hot KV increases privacy exposure and creates consistency problems.
- Prefix Snapshots preserve the useful part of central storage without making it mandatory for every generated token.
Consequences:
- Route Session must be sticky.
- Failover is limited in alpha unless a compatible Prefix Snapshot exists.
- Cache servers are optimization infrastructure, not required runtime infrastructure.
- Route repair requires compatible model revision, layer range, backend cache ABI, and snapshot position.
### Gate 3: First Runtime Proof
Decision: prove distributed Route Session and Hot KV State semantics in the existing PyTorch route before modifying llama.cpp/GGUF.
Rationale:
- PyTorch exposes model internals and cache objects more directly, so it is the fastest way to validate the distributed protocol.
- The current distributed PyTorch route already has the right high-level shape but disables cache and recomputes full prompts.
- Fixing that path gives us a reference implementation for correctness tests, telemetry, session lifecycle, and wire protocol behavior.
- llama.cpp/GGUF should receive a clear target ABI rather than becoming both the protocol experiment and the performance backend at once.
Consequences:
- Issue 02 precedes issue 05.
- llama.cpp collaboration has a concrete target ABI.
- The PyTorch route remains the architecture-coverage/reference backend even after GGUF becomes the preferred performance path.
- The first success metric is eliminating full-prompt recompute in distributed decode.
### Gate 3A: Client Feedback During Latency
Decision: streaming responses are preferred when feasible, and realtime Generation Telemetry is required regardless of streaming support.
Rationale:
- The product optimizes for access to large capable models, so some latency is acceptable.
- Users still need confidence that the route is alive and roughly how fast it is generating.
- Streaming token deltas give the best user experience when the backend exposes them cleanly.
- Tokens/sec remains useful during prefill, queueing, and any backend that cannot stream token deltas.
Consequences:
- The gateway should stream token deltas through an OpenAI-compatible response when possible.
- The gateway must expose progress through SSE, WebSocket, or polling.
- The final answer can be delivered after completion only as a fallback.
- Telemetry must include route phase, generated token count, and rolling tokens/sec.
- Non-streaming clients still need realtime telemetry.
### Gate 4: llama.cpp Collaboration Shape
Decision: target upstreamable `libllama`/ggml hooks instead of planning around a permanent fork.
Rationale:
- llama.cpp changes quickly across model support, quantization, kernels, and hardware backends.
- A permanent fork would become expensive to maintain and would lag upstream improvements.
- A short-lived prototype branch is acceptable if it proves the API and makes upstream collaboration concrete.
- Keeping tracker/routing logic outside llama.cpp makes the upstream ask smaller and cleaner.
Consequences:
- Need a minimal reproducible localhost demo before asking upstream to carry the design.
- Need to separate "what llama.cpp should expose" from "what our tracker does".
- Desired upstream surface is layer-range execution, hidden-state boundary I/O, partial loading/introspection, and per-session KV ownership.
- If upstream rejects the shape, we revisit whether to carry a narrow adapter fork or keep GGUF distributed execution as experimental.
### Gate 5: First Model Target
Decision: use a two-tier model target. Use a small, boring, llama.cpp-supported GGUF model for the first protocol smoke test. Use `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target. Keep GLM-5.2 and Ornith as later support audits.
Rationale:
- The first protocol proof should isolate route/session/KV bugs from model-architecture bugs.
- DeepSeek-V4-Flash is a strong first serious target because it is much smaller than 1.6T-class models while still being large enough to validate the product thesis.
- DeepSeek-V4-Flash still has architecture-specific risks, so it should not be the first smoke test.
- GLM-5.2 and Ornith remain valuable targets, but they add DSA/MLA/hybrid attention uncertainty.
Consequences:
- 128K cache accounting can be modeled now.
- The first "real" target-model audit is DeepSeek-V4-Flash support in PyTorch, vLLM/SGLang, and any available GGUF/llama.cpp quantization path.
- Production support waits for backend capability reports and exact cache ABI support.
### Gate 6: Failure Semantics
Decision: alpha fails Route Sessions on route-node loss instead of attempting automatic route repair.
Rationale:
- Route repair requires compatible Prefix Snapshots, cache ABI checks, replacement-node selection, billing correction, and client stream/error recovery.
- Local Hot KV State means a replacement node cannot continue unless it has compatible state at the same position.
- Fail-fast keeps the first implementation correct while the session/KV protocol is still being proven.
Consequences:
- Better observability and explicit errors are required.
- Snapshotting becomes a later feature, not a blocker for first inference.
- Generation Telemetry must report the last known phase and failure reason.
- Client or gateway retry starts a new Route Session from scratch.
### Gate 7: Transport
Decision: keep binary HTTP for v1 activation transfer instead of jumping immediately to QUIC, WebRTC, or a custom transport.
Rationale:
- ADR-0008 already defines binary activation bodies with HTTP headers.
- HTTP keeps the first implementation debuggable with the existing server stack and tooling.
- The core risk is route/session/KV correctness, not transport optimization.
- QUIC/WebRTC can be introduced later behind the same activation protocol once semantics are proven.
Consequences:
- Focus benchmark work on payload shape, chunking, and cache behavior first.
- QUIC/WebRTC can be introduced as an optimization behind the same activation protocol.
- v1 implementation can reuse the current HTTP routing, relay, and observability infrastructure.
- Transport abstraction should be kept narrow enough that HTTP can be replaced later without changing backend cache semantics.
## Grilling Progress
Gates 1, 2, 3, 3A, 4, 5, 6, and 7 are resolved. The remaining work is to convert the resolved framework into implementation-ready issue briefs and prototype milestones.
# Distributed GGUF Runtime decision framework
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
## Decision order
1. DGR-019 locks comparable lanes and thresholds before results.
2. DGR-020 runs safetensors and whole-model llama.cpp only, then returns `go`, `optimize baseline`, or `stop`.
3. Dense and V4 work must prove parity, independent per-stage execution, local-state isolation, bounded failure, and measured resources.
4. DGR-054 returns `alpha`, `optimize measured bottleneck`, or `stop`; MTP is explicitly off.
5. Post-alpha optimizations must be selected from profiles, not assumptions.
6. DGR-070 returns `beta`, `targeted optimization`, or `stop/rollback`, and requires MTP and the exact certified hardware/recipe matrix.
## Interpretation rules
- Quant/model-fit gains are separate from runtime/kernel/transport gains.
- Fixture, real-model, real-hardware, and release evidence are never interchangeable.
- 24 and 10+ stages are certification scenarios only.
- Existing routing policy is certified, not redesigned.
- Build success is not hardware certification; dark lanes remain unroutable.
- Route loss uses cache miss and re-prefill/restart, never WAN cache migration.
## Locked scope
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 24 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
## Target identities
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.

View File

@@ -0,0 +1,43 @@
# DGR-001 downstream stop-condition handoff
Status: **DGR-001 is complete; native-track promotion is blocked by the immutable v1 verdict.**
This is no longer an execution-prerequisite blocker. The required real benchmark
ran successfully, every recipe completed at concurrency 1 and 4, artifacts were
verified, and deterministic/full test gates passed.
## Locked result
`contract-evaluation.json` records:
```text
verdict: stop
quality_lane_pass: false
speed_benefit: true
fit_benefit: true
stop_condition_met: true
```
The exact-revision BF16 GGUF quality lane compared every prompt but achieved
`0.3333` exact match and `0.9471` mean similarity against the Transformers BF16
reference. V1 requires `0.90` and `0.97`. Quantized Q4_K_M had substantial speed
and fit benefits, but the contract explicitly forbids speed from redeeming a
failed near-lossless quality lane.
## Scope of this stop
The measured baseline is Qwen2.5-0.5B on CPU using a CPU-only llama.cpp build.
It is not a Radeon, large-model, distributed, or native-shard result. Therefore:
1. Do not silently mark v1 promoted or weaken its thresholds after observing the
data.
2. Do not let DGR-004 or later runtime stories treat DGR-001 completion as a
positive promotion signal.
3. A human may choose one of these explicit paths:
- stop the native GGUF track as v1 directs;
- diagnose and fix the BF16 runtime divergence, then rerun the exact v1 plan;
- authorize a separately versioned GPU/large-model contract whose scope and
workload are locked before its measurements.
All raw evidence, configuration, artifacts, hashes, and reproduction commands
are in this directory and `README.md`.

View File

@@ -0,0 +1,199 @@
# DGR-001 — Safetensors versus GGUF performance contract
Status: **complete; immutable v1 verdict is `stop`.**
DGR-001 successfully produced a controlled local-real CPU baseline. Completion
means the experiment and decision contract are durable and verified; it does
**not** mean the native GGUF track is approved to continue. The locked quality
gate failed, so dependent runtime work requires a human decision or a new,
explicitly versioned experiment/contract rather than silently weakening v1.
## Controlled workload
- Model: `Qwen/Qwen2.5-0.5B-Instruct`
- Exact source revision: `7ae557604adf67be50417f59c2c2f167def9a775`
- Machine: `fedora`, Linux `7.0.14-101.fc43.x86_64`, 32 logical CPUs
- Device: CPU for every recipe; VRAM is therefore correctly reported as zero
- Runtime reference: Transformers `5.13.0`, PyTorch
`2.10.0+rocm7.13.0a20260513`, BF16 safetensors
- GGUF runtime: llama.cpp version 9991, commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25`
- Workload: three fixed short/medium/long prompts, greedy sampling, 32 output
tokens, three repeats, two warmups, concurrency 1 and 4, 16 CPU threads
- Evidence class: `local-real`
All artifacts are beneath `/run/media/popov/DATA/llm/`; no model artifact was
created under `/home`.
## Recipes and exact artifacts
| Recipe | Artifact | SHA-256 |
|---|---|---|
| Transformers BF16 reference | complete mounted Hugging Face snapshot | `e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6` |
| llama.cpp BF16 quality lane | `Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf` | `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862` |
| llama.cpp Q4_K_M performance/fit lane | `Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf` | `a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5` |
The snapshot digest covers every sorted relative path, resolved size, and file
byte, so tokenizer/config drift is included. The BF16 GGUF was converted
directly from the exact snapshot while preserving BF16 weights. Q4_K_M was
quantized from an exact-revision F16 conversion with the pinned quantizer.
Runtime validation recomputes every declared digest before model loading.
## Real results
All recipes completed every request with zero failures.
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|---|---:|---:|---:|
| Decode tok/s, c=1 | 40.8 | 98.5 | 207.7 |
| Aggregate decode tok/s, c=4 | 46.5 | 222.8 | 195.7 |
| TTFT p50, c=1 | 40.0 ms | 15.1 ms | 21.6 ms |
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
| Failures | 0 | 0 | 0 |
Against the reference, the eligible Q4_K_M lane measured:
- single-request decode speedup: **5.10×**;
- concurrency-4 aggregate throughput speedup: **4.20×**;
- resident-memory ratio: **0.279×**;
- artifact-size ratio: **0.398×**.
The near-lossless BF16 quality lane compared all three prompts but measured:
- exact match: **0.3333** (v1 requires at least `0.90`);
- mean text similarity: **0.9471** (v1 requires at least `0.97`).
Tokenization and stopping were controlled: every runtime saw the same prompt
token counts and reported 31 post-TTFT decode tokens. The v1 mismatch is a
real greedy-output divergence on two prompts, not missing coverage or a
text-length artifact. Its root cause remains undetermined; no post-contract
logit-tie claim is acceptance evidence. Therefore `contract-evaluation.json`
records:
```text
verdict: stop
quality_lane_pass: false
speed_benefit: true
fit_benefit: true
stop_condition_met: true
```
Thresholds were not changed after observing these results.
## Post-contract parity and ROCm diagnostics
`summarize-quality-parity.py` verifies and separates two signed sources. The CPU
v1 row uses CPU kernels and a Transformers BF16 oracle; it remains at `0.3333`
exact match with an unexplained divergence. The ROCm row uses a different plan,
GPU kernels, and a Transformers float32 oracle. In that narrower diagnostic,
the same BF16 GGUF artifact matches all three 32-token sequences exactly (`1.0`
exact match and `1.0` similarity). No conversion corruption was observed in
that three-sequence ROCm sample; this does not prove global conversion
correctness or explain the CPU result.
A separate HIP build at commit `e920c523` was compiled for `gfx1151` and
measured `ROCm0: Radeon 8060S Graphics`; its `llama-server` SHA-256 is
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`.
A signed `gpu-diagnostic` profile measured zero failures:
| GPU metric | Transformers BF16 ROCm | llama.cpp Q4 ROCm | Q4 ratio |
|---|---:|---:|---:|
| Decode tok/s, c=1 | 81.12 | 251.25 | **3.10×** |
| Aggregate decode tok/s, c=4 | 91.24 | 511.33 | **5.60×** |
| TTFT p50, c=1 | 13.77 ms | 11.80 ms | **0.857×** |
The GPU report is signed under the distinct
`run_configured_gpu_diagnostic/v1` producer. The v1 evaluator rejects that
producer even when its signature is valid. llama-server process VRAM remains
unmeasured, so this diagnostic cannot replace or satisfy the immutable v1
contract. Its signed backend detail records the measured `ROCm0: Radeon 8060S
Graphics` device and `25/25` offloaded layers.
## Implementation
- `recipe_benchmark.py` provides the runtime-neutral measurement core, true
concurrency, continuous in-flight peak-memory sampling, percentile/throughput
aggregation, failures, and output drift.
- `recipe_drivers.py` provides opt-in Transformers and llama-server drivers,
mounted-drive confinement, exact artifact/runtime verification, equal
device/thread budgets, greedy-only validation, measured host provenance, a
CPU-only v1 guard until process VRAM can be measured honestly, and a distinct
signed GPU diagnostic profile that the v1 evaluator cannot accept.
- Peak RSS is runtime-scoped: Transformers reports growth above its pre-runtime
Python baseline, while llama.cpp reports its isolated server process tree.
Both are sampled continuously during in-flight requests.
- TTFT uses each runtime's prompt/first-token compute boundary; end-to-end HTTP,
scheduling, and queue overhead remains in latency and `queue_wait_ms`.
- The exact canonical plan SHA-256 locks prompts, model/revision, sampling,
output length, repeats, warmups, and concurrency. The evaluator also requires
equal prompt/decode token counts across recipes.
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
after it; the driver subtracts that token so decode throughput matches the
Transformers inter-token convention.
- `performance_contract.py` rejects wrong plans, unsigned or incorrectly signed
real evidence, wrong config/artifact/runtime/backend/host bindings, missing
recipes/concurrency, mixed model revisions, incomplete quality coverage, and
failed references.
- Every non-synthetic report is Ed25519-signed over the complete canonical JSON,
including raw outcomes and metrics. The contract pins the public key and exact
config SHA-256; the private key remains outside Git at mode `0600`.
- The signer fingerprint is independently anchored outside this evidence
directory in `../../trusted-evidence-signers.json` and checked by tests.
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
quality gate, and only performance-fit recipes can earn speed/fit benefits.
## Evidence files
- `performance-contract.json` — immutable v1 thresholds and stop condition
- `benchmark-config.json` — exact real-run plan, drivers, artifacts, and hashes
- `results.json` — raw machine-readable per-request and aggregate evidence
- `results.txt` — human-readable benchmark summary
- `baseline.json` — distilled measurements for later comparison
- `contract-evaluation.json` — fail-closed v1 verdict
- `quality-parity-diagnosis.json` / `.md` — run/device-scoped signed-evidence summary
- `summarize-quality-parity.py` — verifies both evidence chains and regenerates it
- `gpu-diagnostic-config.json` — exact ROCm diagnostic artifacts and runtimes
- `gpu-diagnostic-results.json` / `.txt` — signed GPU outcomes and summary
- `commands.txt` — reproducible conversion, benchmark, evaluation, and test commands
- `BLOCKED.md` — downstream stop-condition handoff
- `known-unrelated-failure.md` — clean-base reproduction of the tracker race
- `../../trusted-evidence-signers.json` — repository-reviewed signer fingerprint
## Verification
```text
Targeted: 28 passed (5/5 consecutive focused runs)
Latest full suite: 755 passed, 13 skipped
Earlier full suite: 751 passed, 13 skipped
Current cancellation retry matrix, DGR-001: 4/5 passed
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
compileall: passed
git diff --check: passed
Evidence JSON parse/integrity checks: passed
```
The intermittent tracker cancellation race reproduced at the same rate on the
clean base and is retained in `known-unrelated-failure.md`; the final full suite
completed green. DGR-001 changes no tracker/proxy files.
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
invalid: it used Hermes Agent's internal venv. Verification above used the
project `.venv`, which has the DGR-002-compatible runtime. Real inference used
`.venv-rocm` Python 3.12.
## Limitations and dependent-story handoff
- The immutable contract result is a **0.5B CPU baseline**. The separate Radeon
diagnostic is real local GPU evidence, but neither result covers a large
model, distributed execution, network transport, or a native shard worker.
- A separate `GGML_HIP=ON` llama.cpp build exists and produced GPU timings, but
llama-server process VRAM is not measurable by the current driver; GPU
memory/fit claims therefore remain ineligible for v1.
- Absolute timings are developer-machine measurements; locked ratios and raw
artifacts are provided for reproducibility.
- DGR-014 may consume v1 only with the exact plan/evidence requirements enforced
by `performance_contract.py`.
- DGR-004 and later native-runtime work must not treat DGR-001 completion as a
promotion. V1 says `stop`; proceeding requires a human decision backed by a
separately versioned GPU/large-model contract or a diagnosed quality fix.

View File

@@ -0,0 +1,169 @@
{
"artifact_sha256": {
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6"
},
"backend_detail": {
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16"
},
"evidence_class": "local-real",
"host": {
"accelerator_name": "Radeon 8060S Graphics",
"accelerator_runtime": "7.13.26183",
"benchmark_lane": "cpu-controlled-baseline",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"cpu_count": 32,
"cuda_available": true,
"hostname": "fedora",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_identities": {
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
}
},
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"platform": "Linux-7.0.14-101.fc43.x86_64-x86_64-with-glibc2.42",
"python": "3.12.13",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0"
},
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"provenance": {
"completed_at": "2026-07-13T16:27:19.647692Z",
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"producer": "meshnet_node.recipe_drivers.run_configured_benchmark/v1",
"run_id": "e4eedadf-22f6-4907-8990-985456961099",
"schema_version": 1,
"signature": "owev+/ToswP20C923G6E+srOCUBV5vrjmndVatr9CbTXakiFGqlHrTiEo+aymA4BcSwmG6KJTxlxO6WpLnpcAg==",
"signature_algorithm": "ed25519",
"signer_public_key_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
"started_at": "2026-07-13T16:26:22.361501Z"
},
"recipe_runtime": {
"llama-cpp-near-lossless-quality": {
"device": "cpu",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16"
},
"llama-cpp-quantized-performance-fit": {
"device": "cpu",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M"
},
"transformers-safetensors-reference": {
"device": "cpu",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16"
}
},
"recipes": {
"llama-cpp-near-lossless-quality": {
"artifact_bytes": 994156448,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 86.7339,
"decode_tokens_per_sec": 98.5178,
"failures": 0,
"latency_p50_ms": 333.023,
"latency_p95_ms": 383.0597,
"peak_rss_bytes": 1110728704,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1717.9451,
"ttft_p50_ms": 15.069,
"ttft_p95_ms": 63.766
},
"4": {
"aggregate_decode_tokens_per_sec": 222.788,
"decode_tokens_per_sec": 76.6297,
"failures": 0,
"latency_p50_ms": 490.8738,
"latency_p95_ms": 646.26,
"peak_rss_bytes": 1139466240,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 859.8985,
"ttft_p50_ms": 32.445,
"ttft_p95_ms": 218.387
}
},
"device": "cpu",
"lane": "quality"
},
"llama-cpp-quantized-performance-fit": {
"artifact_bytes": 397807520,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 139.2693,
"decode_tokens_per_sec": 207.712,
"failures": 0,
"latency_p50_ms": 168.3307,
"latency_p95_ms": 305.1338,
"peak_rss_bytes": 542081024,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 967.0195,
"ttft_p50_ms": 21.582,
"ttft_p95_ms": 147.859
},
"4": {
"aggregate_decode_tokens_per_sec": 195.6789,
"decode_tokens_per_sec": 76.9497,
"failures": 0,
"latency_p50_ms": 437.9196,
"latency_p95_ms": 885.5355,
"peak_rss_bytes": 573259776,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 572.4424,
"ttft_p50_ms": 48.127,
"ttft_p95_ms": 416.531
}
},
"device": "cpu",
"lane": "performance-fit"
},
"transformers-safetensors-reference": {
"artifact_bytes": 999586347,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 35.4722,
"decode_tokens_per_sec": 40.7545,
"failures": 0,
"latency_p50_ms": 818.3864,
"latency_p95_ms": 1258.0673,
"peak_rss_bytes": 1941458944,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 625.6467,
"ttft_p50_ms": 40.0018,
"ttft_p95_ms": 195.2551
},
"4": {
"aggregate_decode_tokens_per_sec": 46.5375,
"decode_tokens_per_sec": 12.9506,
"failures": 0,
"latency_p50_ms": 2481.8662,
"latency_p95_ms": 3365.8395,
"peak_rss_bytes": 2104832000,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 264.0101,
"ttft_p50_ms": 97.0403,
"ttft_p95_ms": 429.0665
}
},
"device": "cpu",
"lane": "quality"
}
},
"reference_recipe_id": "transformers-safetensors-reference"
}

View File

@@ -0,0 +1,118 @@
{
"artifact_storage_root": "/run/media/popov/DATA/llm",
"evidence_class": "local-real",
"host": {
"benchmark_lane": "cpu-controlled-baseline",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"transformers_version": "5.13.0"
},
"plan": {
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"prompts": [
{
"id": "short-fact",
"text": "The capital of France is",
"context_class": "short"
},
{
"id": "medium-code",
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
"context_class": "medium"
},
{
"id": "long-summary",
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
"context_class": "long"
}
],
"sampling": {
"temperature": 0.0,
"top_p": 1.0,
"top_k": 1,
"seed": 1234,
"max_output_tokens": 32
},
"concurrency_levels": [1, 4],
"repeats": 3,
"warmup_requests": 2
},
"recipes": [
{
"id": "transformers-safetensors-reference",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": true,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cpu",
"dtype": "bfloat16",
"threads": 16
}
},
{
"id": "llama-cpp-near-lossless-quality",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
},
{
"id": "llama-cpp-quantized-performance-fit",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"lane": "performance-fit",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
}
]
}

View File

@@ -0,0 +1,87 @@
# Exact source snapshot (already present on mounted storage)
SOURCE=/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775
LLAMA=/run/media/popov/d/DEV/llamacpp/llama.cpp
ROCM_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm/bin/python
PROJECT_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
OUT=/run/media/popov/DATA/llm/dgr-001
SIGNING_KEY=/home/popov/.config/neuron-tai/keys/dgr-001-evidence-ed25519.pem
# Private signing key is outside Git and must remain owner-only
stat -c '%a %n' "$SIGNING_KEY" # expected: 600
# Converter support check (no writes)
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" --dry-run
# Exact-revision near-lossless and performance-fit artifacts
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf"
$LLAMA/build/bin/llama-quantize "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf" Q4_K_M
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype bf16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf"
# Runtime and artifact identity
git -C "$LLAMA" rev-parse HEAD
$LLAMA/build/bin/llama-server --version
sha256sum "$LLAMA/build/bin/llama-server" "$LLAMA/convert_hf_to_gguf.py" "$LLAMA/build/bin/llama-quantize"
sha256sum "$SOURCE/model.safetensors" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf"
# Deterministic complete-snapshot digest used by benchmark-config.json
PYTHONPATH=packages/node $ROCM_PY - <<'PY'
from pathlib import Path
from meshnet_node.recipe_drivers import _artifact_sha256
print(_artifact_sha256(Path('/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775')))
PY
# Canonical opt-in local-real benchmark
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/benchmark-config.json \
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.json \
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.txt
# Distil the baseline and evaluate immutable v1
PYTHONPATH=packages/node $PROJECT_PY - <<'PY'
from pathlib import Path
import json
from meshnet_node.performance_contract import baseline_from_report, evaluate_contract, load_contract
root = Path('.scratch/distributed-gguf-runtime/evidence/DGR-001')
report = json.loads((root / 'results.json').read_text())
contract = load_contract(root / 'performance-contract.json')
(root / 'baseline.json').write_text(json.dumps(baseline_from_report(report), indent=2, sort_keys=True) + '\n')
(root / 'contract-evaluation.json').write_text(json.dumps(evaluate_contract(contract, report).to_dict(), indent=2, sort_keys=True) + '\n')
PY
# Optional ROCm GPU diagnostic (not eligible for immutable v1)
# The version-matched rocm[devel] wheel expands beyond 20 GB; ensure sufficient
# space or relocate its packaged payload before installation.
uv pip install --python "$ROCM_PY" --prerelease=allow \
--index-url https://rocm.nightlies.amd.com/v2/gfx1151/ \
'rocm[devel]==7.13.0a20260513'
ROCM_VENV=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm
ROCM_SDK="$ROCM_VENV/bin/rocm-sdk"
ROCM_ROOT="$($ROCM_SDK path --root)"
ROCM_BIN="$($ROCM_SDK path --bin)"
export PATH="$ROCM_VENV/bin:$ROCM_BIN:$PATH"
export ROCM_PATH="$ROCM_ROOT" HIP_PATH="$ROCM_ROOT"
export CMAKE_PREFIX_PATH="$($ROCM_SDK path --cmake):$ROCM_ROOT"
export LD_LIBRARY_PATH="$ROCM_ROOT/lib:$ROCM_ROOT/lib64:${LD_LIBRARY_PATH:-}"
$ROCM_VENV/bin/cmake -S /run/media/popov/d/DEV/llamacpp/llama.cpp \
-B /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip -G Ninja \
-DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
-DCMAKE_HIP_COMPILER="$ROCM_VENV/bin/amdclang++" \
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_BUILD_SERVER=ON
$ROCM_VENV/bin/cmake --build /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip \
--target llama-server llama-cli llama-bench -j 16
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
--profile gpu-diagnostic \
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-config.json \
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.json \
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.txt
PYTHONPATH=packages/node $PROJECT_PY \
.scratch/distributed-gguf-runtime/evidence/DGR-001/summarize-quality-parity.py
# Deterministic verification
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q tests/test_recipe_benchmark.py
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q
PYTHONPATH=packages/node $PROJECT_PY -m compileall -q packages tests
git diff --check

View File

@@ -0,0 +1,71 @@
{
"contract_version": 1,
"fit_benefit": true,
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"quality_lane_pass": false,
"rationale": [
"the near-lossless quality lane failed: the GGUF runtime disagrees with the safetensors reference beyond what near-lossless weights can explain",
"a meaningful speed benefit was measured",
"a meaningful fit benefit was measured"
],
"recipes": [
{
"comparable": true,
"failures": 0,
"fit_benefit": false,
"incomparable_reason": "",
"lane": "quality",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.7873,
"artifact_size_ratio": 0.9946,
"artifact_size_win": false,
"compared_prompts": 3,
"decode_speedup": 2.4173,
"exact_match_rate": 0.3333,
"expected_prompts": 3,
"failure_rate": 0.0,
"mean_similarity": 0.9471,
"resident_memory_ratio": 0.5721,
"ttft_ratio": 0.3767
},
"quality_pass": false,
"reasons": [
"single-request decode 2.42x reference (>= 1.25x) at TTFT ratio 0.38",
"aggregate throughput at concurrency 4 is 4.79x reference (>= 1.25x)",
"peak resident memory is 0.57x reference (<= 0.75x)",
"quality lane exact-match 0.33 / similarity 0.947 versus the reference (fail)"
],
"recipe_id": "llama-cpp-near-lossless-quality",
"speed_benefit": false
},
{
"comparable": true,
"failures": 0,
"fit_benefit": true,
"incomparable_reason": "",
"lane": "performance-fit",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.2048,
"artifact_size_ratio": 0.398,
"artifact_size_win": true,
"decode_speedup": 5.0967,
"failure_rate": 0.0,
"resident_memory_ratio": 0.2792,
"ttft_ratio": 0.5395
},
"quality_pass": null,
"reasons": [
"single-request decode 5.10x reference (>= 1.25x) at TTFT ratio 0.54",
"aggregate throughput at concurrency 4 is 4.20x reference (>= 1.25x)",
"peak resident memory is 0.28x reference (<= 0.75x)"
],
"recipe_id": "llama-cpp-quantized-performance-fit",
"speed_benefit": true
}
],
"speed_benefit": true,
"stop_condition_met": true,
"verdict": "stop"
}

View File

@@ -0,0 +1,143 @@
{
"artifact_storage_root": "/run/media/popov/DATA/llm",
"evidence_class": "local-real",
"host": {
"benchmark_lane": "rocm-gpu-diagnostic",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"transformers_version": "5.13.0",
"rocm_target": "gfx1151"
},
"plan": {
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"prompts": [
{
"id": "short-fact",
"text": "The capital of France is",
"context_class": "short"
},
{
"id": "medium-code",
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
"context_class": "medium"
},
{
"id": "long-summary",
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
"context_class": "long"
}
],
"sampling": {
"temperature": 0.0,
"top_p": 1.0,
"top_k": 1,
"seed": 1234,
"max_output_tokens": 32
},
"concurrency_levels": [
1,
4
],
"repeats": 3,
"warmup_requests": 2
},
"recipes": [
{
"id": "transformers-fp32-rocm-quality-oracle",
"runtime": "transformers-5.13.0-rocm-float32",
"weight_format": "safetensors",
"weight_quantization": "bfloat16-weights-float32-accumulation",
"lane": "quality",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": true,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cuda",
"dtype": "float32",
"threads": 16
}
},
{
"id": "llama-cpp-bf16-rocm-quality",
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"device": "cuda",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 99
}
},
{
"id": "transformers-bf16-rocm-throughput",
"runtime": "transformers-5.13.0-rocm-bfloat16",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"lane": "performance-fit",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cuda",
"dtype": "bfloat16",
"threads": 16
}
},
{
"id": "llama-cpp-q4-rocm-throughput",
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"lane": "performance-fit",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"device": "cuda",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 99
}
}
]
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,13 @@
Recipe benchmark dgr-001-rocm-gpu-diagnostic-v1 (local-real)
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
transformers-fp32-rocm-quality-oracle [quality ] c= 1 ttft p50/p95 11.0/ 35.5 ms; prefill 5746.7 tok/s; decode 35.7 tok/s; aggregate 29.6 tok/s; rss 1.39 GB; vram 2.26 GB; artifact 1.00 GB; failures 0
transformers-fp32-rocm-quality-oracle [quality ] c= 4 ttft p50/p95 27.5/ 80.4 ms; prefill 1985.4 tok/s; decode 9.4 tok/s; aggregate 35.4 tok/s; rss 1.39 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
llama-cpp-bf16-rocm-quality [quality ] c= 1 ttft p50/p95 13.2/ 83.4 ms; prefill 4154.4 tok/s; decode 148.0 tok/s; aggregate 127.4 tok/s; rss 0.84 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-bf16-rocm-quality [quality ] c= 4 ttft p50/p95 25.1/ 52.1 ms; prefill 2205.4 tok/s; decode 115.1 tok/s; aggregate 337.1 tok/s; rss 0.86 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
transformers-bf16-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 13.8/ 22.2 ms; prefill 4787.3 tok/s; decode 81.1 tok/s; aggregate 73.5 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
transformers-bf16-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 29.7/ 58.5 ms; prefill 2666.5 tok/s; decode 24.4 tok/s; aggregate 91.2 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
llama-cpp-q4-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 11.8/ 37.1 ms; prefill 4219.3 tok/s; decode 251.2 tok/s; aggregate 200.1 tok/s; rss 0.69 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
llama-cpp-q4-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 21.4/ 101.0 ms; prefill 2126.9 tok/s; decode 189.7 tok/s; aggregate 511.3 tok/s; rss 0.72 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
drift llama-cpp-bf16-rocm-quality vs transformers-fp32-rocm-quality-oracle exact 1.00; similarity 1.000 (gated)
drift transformers-bf16-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.33; similarity 0.946 (advisory)
drift llama-cpp-q4-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.00; similarity 0.628 (advisory)

View File

@@ -0,0 +1,55 @@
# Observed pre-existing intermittent tracker race
This file records an unrelated timing observation and its repeated reproduction;
it is **not** a DGR-001 benchmark/contract failure.
Test:
```text
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
```
One earlier full-suite run produced:
```text
1 failed, 745 passed, 13 skipped
```
A five-run isolated retry matrix reproduced the same rate repeatedly:
```text
current DGR-001 branch: 4/5 passed, 1/5 failed
clean d904c40: 4/5 passed, 1/5 failed
```
An earlier full-suite run on the signed-provenance DGR-001 state completed
green:
```text
751 passed, 13 skipped
```
Two full-suite runs after adding the isolated GPU diagnostic profile each hit
the same race and otherwise passed:
```text
1 failed, 750 passed, 13 skipped
```
The latest expanded hardening suite hit the same race and otherwise passed:
```text
1 failed, 754 passed, 13 skipped
```
The final hardened state subsequently completed a full green run:
```text
755 passed, 13 skipped
```
In each failure, the mock upstream's three-second release timeout completed the
stream before the cancel POST, so the request was already absent and the cancel
endpoint returned 404. No tracker/proxy file changed in DGR-001. The race is
therefore timing-sensitive, pre-existing, and unrelated to the benchmark,
provenance, or GPU-diagnostic code.

View File

@@ -0,0 +1,87 @@
{
"schema_version": 1,
"contract_version": 1,
"locked_at": "2026-07-13T00:00:00Z",
"locked_by": "DGR-001",
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"thresholds": {
"min_decode_speedup": 1.25,
"max_ttft_ratio": 1.25,
"min_aggregate_throughput_speedup": 1.25,
"max_resident_memory_ratio": 0.75,
"max_artifact_size_ratio": 0.6,
"min_quality_exact_match_rate": 0.9,
"min_quality_mean_similarity": 0.97,
"max_failure_rate": 0.0
},
"baseline": {
"status": "pending-real-evidence",
"required_evidence_class": "local-real",
"required_recipes": [
"transformers-safetensors-reference",
"llama-cpp-near-lossless-quality",
"llama-cpp-quantized-performance-fit"
],
"required_concurrency_levels": [
1,
4
],
"required_controlled_variables": [
"model architecture",
"model revision",
"machine and device",
"formatted prompts and context lengths",
"output length and greedy sampling policy"
],
"required_plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"minimum_prompt_count": 3,
"minimum_repeats": 3,
"minimum_output_tokens": 32,
"required_device": "cpu",
"required_config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"required_signer_public_key": "zQ/qRMwF/ydazzaxEI24Xvnrl5bZxzw16JYpP0bfRuI=",
"required_artifact_sha256": {
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5"
},
"required_recipe_runtime": {
"transformers-safetensors-reference": {
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"device": "cpu"
},
"llama-cpp-near-lossless-quality": {
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"device": "cpu"
},
"llama-cpp-quantized-performance-fit": {
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"device": "cpu"
}
},
"required_backend_detail": {
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16",
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0"
},
"required_host_identity": {
"python": "3.12.13",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0",
"llama_server_identities": {
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
}
}
}
},
"stop_condition": "Stop the native llama.cpp/GGUF track when, on the same machine and device as the Transformers/safetensors reference and under this plan, no performance-fit GGUF recipe delivers either a meaningful speed benefit (>=25% higher single-request decode tokens/sec without a >25% worse TTFT, or >=25% higher aggregate throughput under concurrency) or a meaningful fit benefit (>=25% lower peak resident memory), or when the near-lossless quality lane fails, which indicates a broken runtime rather than a quantization trade-off.",
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract. Non-synthetic evidence must be Ed25519-signed by the pinned key and match the exact locked config, artifacts, runtimes, backends, and host runtime identity."
}

View File

@@ -0,0 +1,47 @@
{
"conclusion": {
"conversion_corruption_observed_in_rocm_sample": false,
"cpu_bf16_divergence_explained": false,
"recommended_v2_design": "Predeclare a float32 quality oracle separately from the BF16 performance reference, with a larger prompt corpus and immutable thresholds.",
"scope": "The ROCm diagnostic establishes only that the same BF16 GGUF artifact matched the float32 oracle for three GPU sequences; it does not explain the CPU BF16 divergence or prove global conversion correctness.",
"v1_verdict_changed": false
},
"cpu_v1": {
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"contract_verdict": "stop",
"device": "cpu",
"exact_match_rate": 0.3333,
"mean_similarity": 0.9471,
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"quality_oracle": "Transformers BF16 safetensors",
"report": "results.json",
"report_sha256": "5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5",
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
"run_id": "e4eedadf-22f6-4907-8990-985456961099"
},
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"rocm_diagnostic": {
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"config_sha256": "b0f0c846c818f1307d034cee1f81daa311efc20985c32a4cdbbbd8ffe4153892",
"device": "cuda (ROCm)",
"exact_match_rate": 1.0,
"failures": 0,
"mean_similarity": 1.0,
"measured_backend_detail": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139; threads 16; parallel slots 4; ctx/slot 512; requested gpu layers 99; measured accelerator ROCm0: Radeon 8060S Graphics; measured offload 25/25 layers",
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
"plan_sha256": "dae8e40963588f71f5d201fd163d39bd762e392544b5603d483e90d21abee2e8",
"producer": "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1",
"quality_oracle": "Transformers float32 safetensors",
"report": "gpu-diagnostic-results.json",
"report_sha256": "527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d",
"run_id": "31bf44e7-ccd4-4277-84ac-c775dee65411",
"signer_fingerprint": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
"v1_eligible": false
},
"schema_version": 2
}

View File

@@ -0,0 +1,31 @@
# DGR-001 quality-parity evidence summary
This summary is generated by `summarize-quality-parity.py` from signed reports.
It contains no independent logit measurements or self-asserted verification flag.
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|---|---|---|---|---:|---:|---|
| CPU v1 (`e4eedadf-22f6-4907-8990-985456961099`) | CPU | Transformers BF16 | llama.cpp BF16 | 0.3333 | 0.9471 | immutable `stop` |
| ROCm diagnostic (`31bf44e7-ccd4-4277-84ac-c775dee65411`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | 1.0000 | 1.0000 | diagnostic only |
## Interpretation
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
matched the float32 oracle for all three GPU sequences with zero failures.
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
`25/25` layer offload.
No conversion corruption was observed in that three-sequence ROCm sample. This
does not prove global conversion correctness and does not retroactively change
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
separately from its BF16 performance reference and use a larger corpus.
## Reproduction and bindings
- CPU report SHA-256: `5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5`
- GPU report SHA-256: `527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d`
- BF16 GGUF SHA-256: `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862`
- Signer fingerprint: `8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de`
- Exact verification command: see `commands.txt`.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,10 @@
Recipe benchmark dgr-001-controlled-whole-model-baseline-v1 (local-real)
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 40.0/ 195.3 ms; prefill 625.6 tok/s; decode 40.8 tok/s; aggregate 35.5 tok/s; rss 1.94 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 97.0/ 429.1 ms; prefill 264.0 tok/s; decode 13.0 tok/s; aggregate 46.5 tok/s; rss 2.10 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 15.1/ 63.8 ms; prefill 1717.9 tok/s; decode 98.5 tok/s; aggregate 86.7 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 32.4/ 218.4 ms; prefill 859.9 tok/s; decode 76.6 tok/s; aggregate 222.8 tok/s; rss 1.14 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 21.6/ 147.9 ms; prefill 967.0 tok/s; decode 207.7 tok/s; aggregate 139.3 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 48.1/ 416.5 ms; prefill 572.4 tok/s; decode 76.9 tok/s; aggregate 195.7 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
drift llama-cpp-near-lossless-quality vs transformers-safetensors-reference exact 0.33; similarity 0.947 (gated)
drift llama-cpp-quantized-performance-fit vs transformers-safetensors-reference exact 0.00; similarity 0.456 (advisory)

View File

@@ -0,0 +1,261 @@
#!/usr/bin/env python3
"""Build the DGR-001 parity summary from cryptographically verified reports."""
from __future__ import annotations
import base64
import hashlib
import json
from pathlib import Path
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from meshnet_node.performance_contract import (
_canonical_sha256,
evaluate_contract,
load_contract,
report_signing_payload,
)
ROOT = Path(__file__).resolve().parent
def _read(name: str) -> dict:
return json.loads((ROOT / name).read_text(encoding="utf-8"))
def _file_sha256(name: str) -> str:
return hashlib.sha256((ROOT / name).read_bytes()).hexdigest()
def _drift(report: dict, recipe_id: str) -> dict:
return next(item for item in report["drift"] if item["recipe_id"] == recipe_id)
def _recipe(report: dict, recipe_id: str) -> dict:
return next(item for item in report["recipes"] if item["recipe"]["id"] == recipe_id)
def main() -> None:
contract = load_contract(ROOT / "performance-contract.json")
cpu_report = _read("results.json")
gpu_config = _read("gpu-diagnostic-config.json")
gpu_report = _read("gpu-diagnostic-results.json")
cpu_evaluation = evaluate_contract(contract, cpu_report)
if cpu_evaluation.verdict != "stop":
raise RuntimeError("immutable CPU v1 evidence no longer evaluates to stop")
public_key_bytes = base64.b64decode(contract.baseline["required_signer_public_key"])
public_key = Ed25519PublicKey.from_public_bytes(public_key_bytes)
public_key.verify(
base64.b64decode(gpu_report["provenance"]["signature"]),
report_signing_payload(gpu_report),
)
signer_fingerprint = hashlib.sha256(public_key_bytes).hexdigest()
if gpu_report["provenance"]["signer_public_key_sha256"] != signer_fingerprint:
raise RuntimeError("GPU report signer fingerprint does not match the contract trust key")
if gpu_report["provenance"]["config_sha256"] != _canonical_sha256(gpu_config):
raise RuntimeError("GPU report is not bound to gpu-diagnostic-config.json")
if gpu_report.get("schema_version") != 1 or gpu_report.get("evidence_class") != "local-real":
raise RuntimeError("GPU report must be schema-v1 local-real evidence")
expected_producer = "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1"
if gpu_report["provenance"].get("producer") != expected_producer:
raise RuntimeError("GPU report was not emitted by the canonical diagnostic producer")
if gpu_report.get("reference_recipe_id") != "transformers-fp32-rocm-quality-oracle":
raise RuntimeError("GPU report uses the wrong quality reference")
if gpu_report.get("host", {}).get("benchmark_lane") != "rocm-gpu-diagnostic":
raise RuntimeError("GPU report lacks the diagnostic host marker")
trusted = json.loads(
(ROOT.parents[1] / "trusted-evidence-signers.json").read_text(encoding="utf-8")
)
if not any(
signer.get("algorithm") == "ed25519"
and signer.get("fingerprint_sha256") == signer_fingerprint
and signer.get("status") == "active"
for signer in trusted.get("signers", ())
):
raise RuntimeError("GPU signer is not active in the trusted-signers registry")
for field in ("model_id", "model_revision"):
if gpu_report["plan"].get(field) != cpu_report["plan"].get(field):
raise RuntimeError(f"CPU and GPU reports do not share {field}")
if gpu_config["plan"].get(field) != gpu_report["plan"].get(field):
raise RuntimeError(f"GPU config and report do not share {field}")
expected_recipes = {
"transformers-fp32-rocm-quality-oracle": ("quality", "cuda"),
"llama-cpp-bf16-rocm-quality": ("quality", "cuda"),
"transformers-bf16-rocm-throughput": ("performance-fit", "cuda"),
"llama-cpp-q4-rocm-throughput": ("performance-fit", "cuda"),
}
actual_recipes = {
entry["recipe"]["id"]: (entry["recipe"]["lane"], entry["recipe"]["device"])
for entry in gpu_report["recipes"]
}
if actual_recipes != expected_recipes:
raise RuntimeError("GPU report recipe identities, lanes, or devices changed")
gpu_prompt_ids = {prompt["id"] for prompt in gpu_report["plan"]["prompts"]}
levels = {int(level) for level in gpu_report["plan"]["concurrency_levels"]}
repeats = int(gpu_report["plan"]["repeats"])
expected_outcomes = len(gpu_prompt_ids) * repeats * sum(levels)
for entry in gpu_report["recipes"]:
recipe_id = entry["recipe"]["id"]
if not entry.get("available") or len(entry.get("outcomes", ())) != expected_outcomes:
raise RuntimeError(f"GPU recipe {recipe_id!r} lacks complete outcomes")
if any(
not outcome.get("ok")
or outcome.get("recipe_id") != recipe_id
or outcome.get("prompt_id") not in gpu_prompt_ids
or int(outcome.get("concurrency", 0)) not in levels
or not 0 <= int(outcome.get("repeat", -1)) < repeats
for outcome in entry["outcomes"]
):
raise RuntimeError(f"GPU recipe {recipe_id!r} contains failed or invalid outcomes")
if {int(level) for level in entry["concurrency"]} != levels:
raise RuntimeError(f"GPU recipe {recipe_id!r} has wrong concurrency cells")
for prompt_id in gpu_prompt_ids:
for level in levels:
for repeat in range(repeats):
count = sum(
outcome["prompt_id"] == prompt_id
and int(outcome["concurrency"]) == level
and int(outcome["repeat"]) == repeat
for outcome in entry["outcomes"]
)
if count != level:
raise RuntimeError(
f"GPU recipe {recipe_id!r} lacks complete request coverage"
)
if any(
int(cell.get("failures", -1)) != 0
or int(cell.get("requests", -1))
!= len(
[
outcome
for outcome in entry["outcomes"]
if int(outcome["concurrency"]) == int(level)
]
)
for level, cell in entry["concurrency"].items()
):
raise RuntimeError(f"GPU recipe {recipe_id!r} aggregates do not match outcomes")
cpu_quality = _drift(cpu_report, "llama-cpp-near-lossless-quality")
gpu_quality = _drift(gpu_report, "llama-cpp-bf16-rocm-quality")
cpu_recipe = _recipe(cpu_report, "llama-cpp-near-lossless-quality")
gpu_recipe = _recipe(gpu_report, "llama-cpp-bf16-rocm-quality")
gpu_backend = gpu_recipe["load"]["backend_detail"]
if "measured accelerator ROCm0: Radeon 8060S Graphics" not in gpu_backend:
raise RuntimeError("GPU report lacks measured ROCm device evidence")
if "measured offload 25/25 layers" not in gpu_backend:
raise RuntimeError("GPU report lacks measured layer-offload evidence")
if cpu_recipe["recipe"]["artifact_sha256"] != gpu_recipe["recipe"]["artifact_sha256"]:
raise RuntimeError("CPU and GPU diagnostics use different BF16 GGUF artifacts")
if gpu_quality.get("compared_prompts") != len(gpu_prompt_ids):
raise RuntimeError("GPU quality drift lacks complete prompt coverage")
if {item["prompt_id"] for item in gpu_quality.get("per_prompt", ())} != gpu_prompt_ids:
raise RuntimeError("GPU quality drift prompt identities do not match the plan")
summary = {
"schema_version": 2,
"model_id": cpu_report["plan"]["model_id"],
"model_revision": cpu_report["plan"]["model_revision"],
"cpu_v1": {
"report": "results.json",
"report_sha256": _file_sha256("results.json"),
"run_id": cpu_report["provenance"]["run_id"],
"plan_id": cpu_report["plan"]["plan_id"],
"plan_sha256": _canonical_sha256(cpu_report["plan"]),
"config_sha256": cpu_report["provenance"]["config_sha256"],
"device": "cpu",
"quality_oracle": "Transformers BF16 safetensors",
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": cpu_recipe["recipe"]["artifact_sha256"],
"exact_match_rate": cpu_quality["exact_match_rate"],
"mean_similarity": cpu_quality["mean_similarity"],
"contract_verdict": cpu_evaluation.verdict,
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
},
"rocm_diagnostic": {
"report": "gpu-diagnostic-results.json",
"report_sha256": _file_sha256("gpu-diagnostic-results.json"),
"run_id": gpu_report["provenance"]["run_id"],
"producer": gpu_report["provenance"]["producer"],
"signer_fingerprint": signer_fingerprint,
"plan_id": gpu_report["plan"]["plan_id"],
"plan_sha256": _canonical_sha256(gpu_report["plan"]),
"config_sha256": gpu_report["provenance"]["config_sha256"],
"device": "cuda (ROCm)",
"quality_oracle": "Transformers float32 safetensors",
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": gpu_recipe["recipe"]["artifact_sha256"],
"measured_backend_detail": gpu_backend,
"exact_match_rate": gpu_quality["exact_match_rate"],
"mean_similarity": gpu_quality["mean_similarity"],
"failures": sum(
metrics["failures"]
for entry in gpu_report["recipes"]
for metrics in entry["concurrency"].values()
),
"v1_eligible": False,
},
"conclusion": {
"v1_verdict_changed": False,
"cpu_bf16_divergence_explained": False,
"conversion_corruption_observed_in_rocm_sample": False,
"scope": (
"The ROCm diagnostic establishes only that the same BF16 GGUF artifact "
"matched the float32 oracle for three GPU sequences; it does not explain "
"the CPU BF16 divergence or prove global conversion correctness."
),
"recommended_v2_design": (
"Predeclare a float32 quality oracle separately from the BF16 performance "
"reference, with a larger prompt corpus and immutable thresholds."
),
},
}
(ROOT / "quality-parity-diagnosis.json").write_text(
json.dumps(summary, indent=2, sort_keys=True) + "\n", encoding="utf-8"
)
md = f"""# DGR-001 quality-parity evidence summary
This summary is generated by `summarize-quality-parity.py` from signed reports.
It contains no independent logit measurements or self-asserted verification flag.
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|---|---|---|---|---:|---:|---|
| CPU v1 (`{summary['cpu_v1']['run_id']}`) | CPU | Transformers BF16 | llama.cpp BF16 | {summary['cpu_v1']['exact_match_rate']:.4f} | {summary['cpu_v1']['mean_similarity']:.4f} | immutable `stop` |
| ROCm diagnostic (`{summary['rocm_diagnostic']['run_id']}`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | {summary['rocm_diagnostic']['exact_match_rate']:.4f} | {summary['rocm_diagnostic']['mean_similarity']:.4f} | diagnostic only |
## Interpretation
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
matched the float32 oracle for all three GPU sequences with zero failures.
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
`25/25` layer offload.
No conversion corruption was observed in that three-sequence ROCm sample. This
does not prove global conversion correctness and does not retroactively change
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
separately from its BF16 performance reference and use a larger corpus.
## Reproduction and bindings
- CPU report SHA-256: `{summary['cpu_v1']['report_sha256']}`
- GPU report SHA-256: `{summary['rocm_diagnostic']['report_sha256']}`
- BF16 GGUF SHA-256: `{summary['rocm_diagnostic']['candidate_artifact_sha256']}`
- Signer fingerprint: `{signer_fingerprint}`
- Exact verification command: see `commands.txt`.
"""
(ROOT / "quality-parity-diagnosis.md").write_text(md, encoding="utf-8")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,242 @@
# DGR-002 — Adopt the versioned gRPC Shard protocol
Status: **done**. Every acceptance criterion is met with real command output.
Evidence class: **synthetic/unit** — this story defines a schema and proves both
languages agree on it. No model, GPU, network peer or benchmark is involved, and
none is claimed.
## 1. Summary
`packages/node/native/proto/shard_runtime.proto` is now the semantic contract for
the native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). Python
and C++ both generate from it, and a shared committed conformance vector proves
they encode it identically — byte for byte.
Design decisions worth carrying forward:
- **Everything gRPC gives you is *also* in the schema.** Deadline, cancellation,
identity and flow control are carried as fields, not left to HTTP/2 metadata,
because the existing relay carries these frames as **opaque binary**. A relayed
frame has no HTTP/2 context to inherit a deadline or a channel identity from.
If it is not in the schema, it does not survive the relay.
- **Cancellation is both in-band and out-of-band.** `CancelSignal` rides the
stream; `Cancel` is also a unary RPC. A cancel that can only travel down a
stream that flow control has wedged is not a cancel.
- **Checksums cover the uncompressed payload.** Compression is a per-hop
transport decision (reusing the existing `activation_compression` policies), so
a checksum over the compressed frame would be invalidated by a hop that merely
chose differently.
- **Application-level flow-control credits, not just HTTP/2 windows.** HTTP/2
bounds *bytes in flight*; it does not bound how much *work* a worker has queued,
and a relayed frame gets no window at all. Credits bound queue occupancy and KV
pressure, and negotiation takes the strictest bound of either peer so a sender
cannot talk a worker into unbounded queues.
## 2. Files changed
New:
| Path | What |
|---|---|
| `packages/node/native/proto/shard_runtime.proto` | The schema (sha256 `9e211660…`, see `protocol.json`) |
| `packages/node/native/CMakeLists.txt` | C++ generation + build wiring + ctest |
| `packages/node/native/tests/test_shard_protocol_conformance.cpp` | C++ conformance test |
| `packages/node/native/testdata/*.binpb` | Committed cross-language vectors |
| `packages/node/native/README.md` | How to regenerate and build |
| `packages/node/meshnet_node/native_protocol/__init__.py` | Public Python surface |
| `packages/node/meshnet_node/native_protocol/codec.py` | Bundle encode/decode, fragmentation, CRC32C, chunking, FC negotiation |
| `packages/node/meshnet_node/native_protocol/conformance.py` | Canonical vectors shared by both languages |
| `packages/node/meshnet_node/native_protocol/generated/` | Generated Python stubs (committed) |
| `scripts/generate_native_protocol.py` | Python generation, with `--check` |
| `scripts/generate_protocol_goldens.py` | Vector generation, with `--check` |
| `scripts/bootstrap_native_toolchain.sh` | Builds protobuf C++ from source |
| `tests/test_native_shard_protocol.py` | 45 Python tests |
Modified:
- `packages/node/pyproject.toml` — added runtime floors `grpcio>=1.82.1` and
`protobuf>=7.35.0`, matching the committed generated-code requirements; new
`proto` extra pinning `grpcio-tools==1.82.1`.
- `packages/node/meshnet_node/activation_compression.py` — optional bounded zstd
output for untrusted protocol frames; existing callers remain compatible.
- `packages/node/meshnet_node/native_protocol/__init__.py` — exports negotiated
bound constants and whole-session-message validation.
The canonical PRD marks only DGR-002 passed. `git status` before this story was clean.
## 3. Commands and real results
See `commands.txt` for the exact ordered list. Results:
```
python scripts/generate_native_protocol.py --check -> generated stubs are up to date
python scripts/generate_protocol_goldens.py --check -> conformance vectors are up to date
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
-- gRPC C++ not found: building message types only (sufficient for the conformance test)
cmake --build build/native -j -> Built target shard_protocol_conformance
ctest --test-dir build/native --output-on-failure -> 1/1 Test #1: shard_protocol_conformance ... Passed
100% tests passed out of 1
cmp build/native/cpp_roundtrip.binpb \
packages/node/native/testdata/session_request_golden.binpb -> identical (exit 0)
pytest -q tests/test_native_shard_protocol.py -> 45 passed
pytest -q tests/test_native_shard_protocol.py \
tests/test_activation_compression.py -> 51 passed
pytest -q (final full suite) -> 728 passed, 12 skipped
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
(after an earlier flaky full-suite failure) -> 1 passed, 1 passed, 1 passed
clean minimum-runtime import + codec smoke test -> passed
grpcio==1.82.1, protobuf==7.35.0
compileall -q packages tests -> OK (exit 0)
git diff --check -> clean (exit 0)
```
The C++ lane was rebuilt from scratch by Ralph (`rm -rf build/native`) using only
the documented commands, and reproduced the same result. During controller
review the user explicitly chose not to repeat the destructive build-directory
cleanup, so the independent controller relied on the recorded CMake/CTest run
while reproducing every Python/generation/full-suite gate.
### Controller review corrections
Independent controller review found and fixed two classes of issue before
integration:
1. Generated stubs required gRPC 1.82.1 and Protobuf 7.35.0, while the initial
package metadata allowed much older runtimes that could fail at import time.
2. Flow-control bounds were described but not enforced by the reference decoder.
Tensor declarations, shape rank/dimensions, fragment/tensor counts, fragments,
wire bodies, whole bundles, complete session messages (including envelope
overhead), and zstd window/output expansion are now fail-closed against the
negotiated/default bounds. Unspecified bundle versions, compression and
checksums are rejected rather than interpreted as valid data.
3. Negotiated initial credits could exceed `max_inflight_chunks`; credits are now
capped by the settled in-flight limit.
Controller results: protocol tests `45 passed`; protocol plus shared compression
tests `51 passed`; final full suite `728 passed, 12 skipped`. A clean environment
at the declared minimum gRPC/Protobuf runtime versions imported both generated
stub modules and round-tripped the codec. Generation checks, `compileall`, static
secret scan, and `git diff --check` all passed.
### Full-suite note — a pre-existing flaky test
`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy`
is **flaky on a clean tree, independent of this story**. Reproduction, run
*before any DGR-002 file existed* (working tree clean, `git status` empty):
```
pytest -q -> 1 failed, 682 passed, 12 skipped
FAILED tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# same test, three consecutive isolated runs on the same clean tree:
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 1 passed in 1.76s
-> 1 failed in 4.39s
-> 1 passed in 1.10s
```
It is a timing race in proxy cancellation (a 3-second in-flight generation raced
against the cancel assertion), not a deterministic failure, and it touches no code
this story changes. One controller full-suite run reported exactly that one failure
(`1 failed, 719 passed, 12 skipped`); three immediate isolated retries all passed
in 1.11 seconds, and the final exact-code full suite was green (`728 passed,
12 skipped`). It is flagged for whoever owns the tracker cancel path and is **not**
fixed here, since silently touching another story's code is out of scope.
## 4. Acceptance criteria
| Criterion | Where it is proven |
|---|---|
| Schema for capability, health, session stream, release, cancellation | `shard_runtime.proto` `service ShardRuntime`; `test_service_exposes_capability_health_session_release_and_cancel` |
| One long-lived bidi stream per Activation Seam, with deadlines, cancellation, flow control, structured errors | `rpc Session (stream) returns (stream)`; `test_session_is_one_long_lived_bidirectional_stream`; `Envelope.deadline_unix_nanos`, `CancelSignal` + unary `Cancel`, `FlowControl`, `ShardError` |
| Bounded chunking for prefill; small decode fast path | `ChunkInfo` + `plan_prefill_chunks` (128-token bound, ADR-0008); `DecodeStep`; `test_prefill_is_split_into_bounded_token_aligned_chunks`, `test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk` |
| Envelope carries schema version, work id, session id, epoch, fingerprint, range/effective start, phase, position, idempotency step, cache expectation, compression, checksum | `Envelope` + `NamedTensor`; `test_envelope_carries_every_field_the_protocol_promises` asserts against the **descriptor**, so deleting a field from the `.proto` fails the test |
| Versioned named-tensor bundle: name, shape, dtype, byte order, fragments | `TensorBundle`/`NamedTensor`/`TensorFragment`; `test_named_tensor_bundle_is_versioned_and_fully_described`, `test_bundle_round_trips_multiple_named_tensors` |
| Round-trip + compatibility tests in Python and C++ | 45 Python tests; C++ `ctest` 1/1; cross-language byte equality |
| Targeted pytest passes | 45 passed |
| `compileall packages tests` | exit 0 |
| `git diff --check` | exit 0 |
| Default tests deterministic, download-free, credit-free, GPU-free | Pure in-memory protobuf; no model, no network, no GPU |
| Full deterministic pytest passes, or pre-existing failure recorded | Final exact-code run: 728 passed, 12 skipped; earlier sole flaky failure documented with clean-tree reproduction and 3/3 passing retries |
## 5. How the cross-language claim is actually earned
Two codecs that each round-trip their own output prove only that each is
self-consistent. Instead:
1. Python builds the canonical `SessionRequest` and commits its bytes.
2. The C++ test parses **those** bytes, asserts every field, recomputes the CRC32C
**from the polynomial in independent C++ code**, reassembles the multi-fragment
tensor, and re-serializes to `cpp_roundtrip.binpb`.
3. `test_cpp_and_python_agree_byte_for_byte` asserts that file equals the golden.
Compatibility is tested in both languages: an unknown field from a newer peer
survives a parse/serialize hop (a Shard forwards activations — silently stripping
fields would corrupt a route it is merely a waypoint on), and a sparse message
from an older peer parses to proto3 defaults.
## 6. Limitations and deferred work
- **gRPC C++ was not built or linked.** The C++ lane verifies the *schema* (message
types), not a running gRPC C++ server, because this machine has no gRPC C++ stack
and building it is a large dependency the conformance test does not need.
`CMakeLists.txt` already generates and exports `shard_runtime_grpc` when
`find_package(gRPC)` succeeds. **DGR-008 must install gRPC C++ and extend
`scripts/bootstrap_native_toolchain.sh`.**
- **No wire is exercised.** No client, server, or stream lifecycle exists yet — no
deadline actually fires, no credit is actually consumed. This story defines and
proves the contract; DGR-008/DGR-009 implement it.
- The protobuf C++ toolchain used here was installed to `/tmp/pbsrc/install` (ephemeral).
`scripts/bootstrap_native_toolchain.sh` reproduces it; prefer a durable prefix such
as `build/native-toolchain`.
- `crc32c` has a pure-Python fallback (used here) and picks up `google_crc32c` when
present. The fallback is byte-exact but slow; a worker on the hot path should install
the native package. Not a correctness limitation.
- Compression on the wire is zstd-or-none only, matching the existing seam.
## 7. Compatibility and migration notes
- **This does not change the existing HTTP activation wire.** `X-Meshnet-Wire` stays
at `2` and the legacy `/forward` path is untouched. The native protocol is a
*separate* contract with its own `SchemaVersion`, starting at 1. Nothing in this
story is on any live request path — it is additive.
- Semantics are deliberately preserved from the existing ADRs so the two transports
mean the same thing: `effective_start_layer` (ADR-0012), `CacheMode`/`expected_past_len`
and `ERROR_CODE_CACHE_MISS` mapping to today's HTTP 409 `cache_miss` (ADR-0022),
bfloat16 boundary dtype and 128-token prefill chunks (ADR-0008), fingerprint/recipe
identity mirroring the capability report (ADR-0023).
- `TensorFragment` field 5 (`uncompressed_size`) is **reserved**: it was removed
because `NamedTensor.total_bytes` is the single source of truth. Never recycle it —
a recycled field number is the one schema change peers cannot detect, because the
bytes still parse.
- Committed Python stubs are guarded by `--check` in the test suite, so they cannot
drift from the schema unnoticed.
## 8. Handoff to dependent stories
- **DGR-003 (runtime recipe/fingerprint):** populate `Fingerprint`
(`model_artifact_digest`, `runtime_recipe_digest`, `recipe_id`, `recipe_version`,
`catalogue_version`). The mismatch outcome is already specified:
`ERROR_CODE_FINGERPRINT_MISMATCH`. Do not invent a second identity struct.
- **DGR-005/006 (range loading, architecture boundary):** the boundary payload is a
**named bundle**, not a bare tensor — a boundary needing more than one tensor is
already representable. Execute `[effective_start_layer, end_layer)`, never from
`start_layer`.
- **DGR-007 (concurrent sessions/KV):** isolate on `(route_session_id, route_epoch)`.
`CacheExpectation`/`CacheResult` and `ERROR_CODE_CACHE_MISS` are the contract; a
decode step whose `expected_past_len` does not match **must** miss, never fall back
to a silent stateless forward. `idempotency_step` means a retried step is
acknowledged (`Ack.duplicate`), not re-applied — re-applying advances the KV cache
twice and desynchronises the route.
- **DGR-008 (C++ worker):** link `shard_runtime_grpc` from `CMakeLists.txt`; you must
first install gRPC C++ (see limitations). Honour `FlowControl` credits and the
`max_chunk_bytes` bound. Use `packages/node/meshnet_node/native_protocol/codec.py`
as the reference for fragment reassembly and checksum validation.
- **DGR-009 (Meshnet integration):** the relay may carry these serialized frames as
opaque binary — that is exactly why deadline/cancel/identity are in-band. Do not add
a second control plane.
- **Anyone editing the schema:** run both `--check` scripts; if a vector legitimately
changes, regenerate it and say so, because the C++ test asserts those exact bytes.

View File

@@ -0,0 +1,45 @@
# DGR-002 — exact commands, in order. Run from the repository root.
# Interpreter: <repo>/.venv/bin/python (CPython 3.14.6). Deterministic, GPU-free,
# no model download, no API credits.
# --- toolchain (this machine had no protoc, no cmake, no protobuf C++ headers)
.venv/bin/python -m pip install grpcio-tools==1.82.1 grpcio==1.82.1 cmake==4.4.0
scripts/bootstrap_native_toolchain.sh /tmp/pbsrc/install # protobuf C++ 33.1 + abseil 20250814.1
# --- schema generation (Python stubs; committed)
.venv/bin/python scripts/generate_native_protocol.py
.venv/bin/python scripts/generate_native_protocol.py --check # -> "generated stubs are up to date"
# --- cross-language conformance vectors (committed)
.venv/bin/python scripts/generate_protocol_goldens.py
.venv/bin/python scripts/generate_protocol_goldens.py --check # -> "conformance vectors are up to date"
# --- C++ generation, build and conformance test
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
cmake --build build/native -j"$(nproc)"
ctest --test-dir build/native --output-on-failure # -> 1/1 Passed
cmp build/native/cpp_roundtrip.binpb packages/node/native/testdata/session_request_golden.binpb
# --- Python tests
.venv/bin/python -m pytest -q tests/test_native_shard_protocol.py # -> 29 passed
.venv/bin/python -m pytest -q # full suite
# --- repository gates
.venv/bin/python -m compileall -q packages tests
git diff --check
# --- independent controller review after Ralph
PYTHONPATH=packages/node .venv/bin/python -m pytest -q tests/test_native_shard_protocol.py
# -> 45 passed
PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
tests/test_native_shard_protocol.py tests/test_activation_compression.py
# -> 51 passed
PYTHONPATH=packages/node .venv/bin/python -m pytest -q
# -> final exact-code run: 728 passed, 12 skipped
for i in 1 2 3; do PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy; done
# -> 1 passed, 1 passed, 1 passed
# clean minimum-runtime venv: protobuf==7.35.0 grpcio==1.82.1
# generated pb2 + pb2_grpc imports and one-byte codec round trip -> passed
# The user chose to rely on Ralph's recorded successful C++ CMake/CTest run
# rather than repeat deletion of an isolated generated build directory.

View File

@@ -0,0 +1,95 @@
{
"schema_version": "SCHEMA_VERSION_1",
"bundle_version": 1,
"proto_path": "packages/node/native/proto/shard_runtime.proto",
"proto_sha256": "9e211660b3fcefc88bcdf3851c3571088c00349aacb5adc5ef45083c83d0cce2",
"protoc": "grpc_tools 1.82.1 (python) / protobuf 33.1 (C++)",
"service": {
"GetCapability": {
"client_streaming": false,
"server_streaming": false
},
"Health": {
"client_streaming": false,
"server_streaming": false
},
"Session": {
"client_streaming": true,
"server_streaming": true
},
"Release": {
"client_streaming": false,
"server_streaming": false
},
"Cancel": {
"client_streaming": false,
"server_streaming": false
}
},
"envelope_fields": [
"cache_expectation",
"chunk",
"deadline_unix_nanos",
"fingerprint",
"idempotency_step",
"phase",
"position",
"route_epoch",
"route_session_id",
"schema_version",
"shard_range",
"work_id"
],
"named_tensor_fields": [
"byte_order",
"checksum",
"compression",
"dtype",
"fragments",
"name",
"shape",
"total_bytes"
],
"phases": [
"PHASE_UNSPECIFIED",
"PHASE_PREFILL",
"PHASE_DECODE",
"PHASE_RELEASE",
"PHASE_CANCEL"
],
"error_codes": [
"ERROR_CODE_UNSPECIFIED",
"ERROR_CODE_SCHEMA_UNSUPPORTED",
"ERROR_CODE_FINGERPRINT_MISMATCH",
"ERROR_CODE_EPOCH_STALE",
"ERROR_CODE_SHARD_RANGE_MISMATCH",
"ERROR_CODE_CACHE_MISS",
"ERROR_CODE_RESOURCE_EXHAUSTED",
"ERROR_CODE_PAYLOAD_CORRUPT",
"ERROR_CODE_CANCELLED",
"ERROR_CODE_DEADLINE_EXCEEDED",
"ERROR_CODE_FLOW_CONTROL_VIOLATION",
"ERROR_CODE_INTERNAL"
],
"bounds": {
"max_prefill_chunk_tokens": 128,
"max_chunk_bytes": 4194304,
"max_fragment_bytes": 1048576,
"max_inflight_chunks": 8,
"max_fragments_per_tensor": 64,
"max_tensors_per_bundle": 64,
"max_tensor_rank": 8,
"max_tensor_dimension": 2147483647,
"whole_session_message_enforced": true
},
"golden_vectors": {
"session_request_golden.binpb": "c2c3df8a717ddeae7bd99624d2c7f34c09a518988de990237fe313b75cff0817",
"capability_report_golden.binpb": "71ac5f150775f398515b43a63596a5cbe8d2ad607e7e4de56bd44fbe7987080c"
},
"verification": {
"python_protocol_tests": "45 passed",
"python_protocol_and_compression_tests": "51 passed",
"full_suite": "728 passed, 12 skipped",
"minimum_runtime": "grpcio 1.82.1 / protobuf 7.35.0 passed import and codec smoke"
}
}

View File

@@ -0,0 +1,186 @@
# DGR-003 — exact Artifact and runtime recipe identity
Evidence class: deterministic offline/unit. No model payload, GPU, external API,
network node, or API credit is required or claimed.
## Result — delayed-review repair, 2026-07-14
DGR-003 defines and tests an exact, model-agnostic compatibility identity and
connects it to DGR-002's gRPC `Fingerprint` plus tracker parsing, admission,
route partitioning, and certification. It is **not complete**: the existing
production doctor/backend path still emits the legacy capability report without
constructing a `ShardIdentity` from authoritative loaded artifact/runtime state.
No exact recipe is therefore claimed live or routable from that path; supplied
exact identities remain dark until tracker-owned certification.
A matching digest proves canonical consistency, **not node authenticity or real
execution**. Tracker-owned certification of a fingerprint by a non-synthetic,
complete, multi-node distributed forward is the execution trust boundary.
## Implementation
- `ArtifactIdentity` binds artifact ID/revision, exact content digest,
architecture/config digest, layer count, and optional derivative binding.
- `DerivativeBinding` binds a split artifact to the exact source artifact digest
and its end-exclusive layer range. A Shard cannot advertise outside that range.
- `RuntimeRecipe` keeps these canonical axes separate rather than hiding them in
a backend label:
- weight quantization;
- activation and compute dtypes;
- KV dtype and layout;
- tokenizer revision;
- architecture adapter;
- backend and runtime version;
- boundary and protocol schema versions;
- recipe ID/version and catalogue version.
- `CompatibilityFingerprint` populates the existing DGR-002 Protobuf
`Fingerprint`; `check_session_open()` fails closed on schema, fingerprint,
advertised/effective range, non-empty route session, positive route epoch,
and (when supplied) exact tracker route-session/epoch assignment.
- Node and tracker implementations independently canonicalize the declaration.
This is intentional: the tracker must not trust a digest copied from a node,
and future native/C++ workers also need an independent implementation. Their
behavior is pinned by `tests/data/recipe_fingerprint_vectors.json`.
- Tracker admission cross-checks the exact identity against the capability
proof's model, range, recipe labels, backend, and weight quantization. Any
disagreement fails closed.
- `TrackerServer` owns the sole live certification ledger and passes it through
direct and replicated registration paths. A known exact recipe is
`uncertified` and dark for user traffic until the same exact fingerprint is
certified. Restart fails closed; durable/cluster-wide certification events
require the later real-forward control path and are not claimed here.
- Certification evidence is bound to the promoted fingerprint, requires at
least two distinct nodes, complete layer coverage, generated tokens, and
`synthetic=false`. Unknown or mismatched fingerprints cannot be promoted.
## Files changed
- `packages/node/meshnet_node/runtime_recipe.py`
- `packages/tracker/meshnet_tracker/recipe.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/data/recipe_fingerprint_vectors.json`
- `tests/test_runtime_recipe_identity.py`
- this evidence directory, issue state, and DGR-003 PRD state
A late review of dependency DGR-017 also found and fixed two genuine contract
continuity defects during delayed DGR-003 review: v1 now has an independently
trusted digest and recursively immutable parsed state. Those changes and tests
are recorded in DGR-017 evidence rather than claimed as DGR-003 functionality.
## Verification
Exact commands and outcomes are in `commands.txt`.
Observed final results:
- DGR-003 identity + node/tracker capability suites: **126 passed**.
- DGR-017 focused dependency repair suite: **99 passed**.
- Tracker routing suite: **93 passed**.
- First delayed-review integrated run: **898 passed, 13 skipped, 1 failed** on
the pre-existing tracker-cancellation race.
- Final delayed-review integrated rerun: **899 passed, 13 skipped** in
**253.64s**; Hermes controller acceptance rerun: **899 passed, 13 skipped**
in **252.66s**.
- `python -m compileall -q packages tests`: pass.
- `git diff --check`: pass.
- Ruff on the changed identity, capability, contract, and test modules: pass.
- `server.py` has 8 pre-existing Ruff findings at both pushed baseline and the
current tree; DGR-003 added no finding.
The first integrated full-suite run produced **871 passed, 13 skipped, 1 failed**
on the known unrelated
`test_tracker_dashboard_can_cancel_inflight_proxy` timing race. Its fixture
completed after three seconds just before cancellation, so the cancel endpoint
returned 404. In this delayed repair it again produced a 404 after the stream
finished (first integrated run: **898 passed, 13 skipped, 1 failed**); three
immediate isolated repeats passed before a fourth reproduced the same race.
No cancellation-test code was changed. The final complete integrated rerun
passed **899/899** tests.
## Limitations
- Certification state is process-local in this story. The same running tracker
reuses it across registrations, but durable/cluster-wide certification-event
persistence belongs with the later real distributed-forward control path.
Restart or failover therefore returns exact recipes to the safe dark state;
it never makes an unsupported recipe routable.
- The node module has no certification ledger or admission policy; it holds only
identity construction and handshake validation. The Tracker is the sole
promotion authority.
- **Completion blocker:** `doctor._validate_recipe()` calls
`build_capability_report()` without `identity=`, because the legacy
Transformers backend does not expose an immutable artifact-content pin and
full runtime recipe axes authoritative enough to build one. Adding a guessed
identity would weaken this contract. Production emission must be added with
the authoritative native worker/backend loading seam; until then the issue and
PRD deliberately remain incomplete.
- This story proves identity and admission behavior with deterministic fixtures.
It does not claim a real GLM forward or hardware certification.
## Compatibility
- Capability report identity is additive. Legacy reports without the new block
retain ADR-0023's explicit compatibility-policy behavior.
- Reports that opt into exact identity are held to it and fail closed on malformed,
inconsistent, unknown, dark, or mismatched declarations.
- No new wire identity was invented; DGR-002's `Fingerprint` remains the gRPC
representation.
## Handoff
DGR-004 and native workers must build `ShardIdentity` from the actual immutable
artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout,
schema versions, and owned range. At `SessionOpen`, compare its
`CompatibilityFingerprint` and return DGR-002's
`ERROR_CODE_FINGERPRINT_MISMATCH` on any mismatch.
A digest match is not certification. Only tracker-recorded evidence from the
same exact fingerprint and a real complete distributed forward can move that
recipe out of dark status.
## Native emission closure — 2026-07-14
Status: **done**. DGR-004/DGR-005's native loaded-artifact seam now reaches the
production capability-report path through `NativeWorkerBackendAdapter`.
### Files changed
- `packages/node/meshnet_node/native_backend.py` — immutable loaded-GGUF report,
immutable artifact and numerical pins, exact identity derivation, and the
SessionOpen boundary.
- `packages/node/meshnet_node/doctor.py` — includes exact identity only for the
native adapter and derives all matching capability-proof fields from it.
- `tests/test_native_identity_emission.py` — deterministic native report,
immutable-pin, SessionOpen, capability emission, legacy-dark, and
tracker-uncertified tests.
- This issue, `prd.json`, and this evidence directory.
### Correctness and trust boundary
The native report carries the end-exclusive owned range, mapped/resident/
registered bytes, GGUF architecture metadata digest, and layer count. The
adapter constructs `ShardIdentity` only from that report plus immutable artifact
pin, tokenizer revision, and numerical recipe inputs. It does not accept a
caller-supplied shard range.
`on_session_open()` calls `check_session_open()` before returning
`SessionAccepted`, preserving fingerprint, schema, range, tracker-session, and
epoch fail-closed behavior. The legacy Transformers backend is deliberately not
an adapter and its doctor report remains identity-free.
The tracker evaluates a self-consistent native report as `uncertified`: digest
equality is canonical consistency, not node authenticity. Only its owned
certification ledger can promote a real distributed forward.
### Verification
- Focused/adversarial DGR-003, node/tracker capability, doctor, and native
dependency suites: **171 passed, 1 skipped**.
- Native protocol CMake configure/build plus CTest: **1/1 passed**.
- `compileall`, Ruff, and `git diff --check`: pass.
- Full deterministic suite: **902 passed, 13 skipped** (255.01s).
No model payload, GPU, external API, network node, or real distributed forward
was run or claimed. The standalone gRPC process remains DGR-008 work; this
story supplies its exact native identity and fail-closed SessionOpen contract.

View File

@@ -0,0 +1,108 @@
# DGR-003 final verification — 2026-07-14
# Native emission closure — 2026-07-14
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_native_identity_emission.py tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py tests/test_node_doctor.py tests/test_llama_cpp_dependency.py
# result: 171 passed, 1 skipped in 7.07s
ruff check packages/node/meshnet_node/native_backend.py packages/node/meshnet_node/doctor.py tests/test_native_identity_emission.py
# result: All checks passed
git diff --check
# result: pass
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall packages tests
# result: pass
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake -S packages/node/native -B build/dgr-003-native-protocol -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake --build build/dgr-003-native-protocol -j2
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/ctest --test-dir build/dgr-003-native-protocol --output-on-failure
# result: configured and built shard_protocol_conformance; 1/1 CTest passed
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 902 passed, 13 skipped in 255.01s
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
# result: 99 passed in 4.76s
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
# result: 99 passed in 0.15s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# first integrated result: 871 passed, 13 skipped, 1 failed in 258.18s
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# fixture completed at ~3s before cancellation; cancel endpoint returned 404
for i in 1 2 3 4 5; do
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
done
# result: 5/5 passed (1.14s, 1.14s, 1.26s, 1.14s, 1.64s)
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# final integrated result: 872 passed, 13 skipped in 253.46s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
# result: pass
git diff --check
# result: pass
ruff check packages/node/meshnet_node/glm_alpha/contract.py packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_glm_alpha_target.py tests/test_runtime_recipe_identity.py
# result: All checks passed!
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
ruff check /tmp/dgr003-server-base.py
ruff check packages/tracker/meshnet_tracker/server.py
# result: both baseline and current server.py report the same 8 pre-existing findings
# ---------------------------------------------------------------------------
# Delayed-review repair continuation — 2026-07-14
# No model payload, GPU, external API, or real inference was run.
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
# result: 126 passed in 4.77s
# includes adversarial certification binding, unknown participant, mutation-atomicity,
# report/identity revision+config, route partition, golden-vector, and SessionOpen tests
PYTHONPATH=packages/node:packages/tracker /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python scripts/gen_recipe_fingerprint_vectors.py --check
# result: tests/data/recipe_fingerprint_vectors.json matches the identity implementation
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
# result: 99 passed in 0.11s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py
# result: 93 passed in 46.83s
# there is no separate tests/test_tracker_server.py in this repository
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
# result: pass
ruff check packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_runtime_recipe_identity.py scripts/gen_recipe_fingerprint_vectors.py
# result: All checks passed!
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
ruff check /tmp/dgr003-server-base.py
ruff check packages/tracker/meshnet_tracker/server.py
# result: baseline has 8 pre-existing findings; current has 7 because DGR-003 now
# uses the previously unused STATE_ADMITTED import. No new server.py finding.
git diff --check
# result: pass
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 898 passed, 13 skipped, 1 failed in 255.43s
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# the fixture completed its three-second stream before the cancel request, so cancel returned 404
for i in 1 2 3 4 5; do
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy || exit 1
done
# result: first 3 passed (1.16s, 1.65s, 1.64s); attempt 4 reproduced the same 404 race.
# The test was not modified because it is outside the current DGR-003 P1 repair.
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 899 passed, 13 skipped in 253.64s (0:04:13)
# Hermes controller acceptance rerun after agent completion
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 899 passed, 13 skipped in 252.66s (0:04:12)

View File

@@ -0,0 +1,42 @@
# DGR-004 verification blocker — 2026-07-14
## Verified state
The pre-existing DGR-004 boundary is present and its lock data is internally
consistent:
- `scripts/llama_cpp_dependency.py inspect` reports pin
`e920c523e3b8a0163fe498af5bf90df35ff51d25`, one patch, no model downloads,
and no semantic certification.
- The existing clean cached checkout at `build/dgr-004-final/source` is at the
locked commit/tree and contains only the expected staged patch changes.
- The existing `llama-gguf-hash --help` smoke binary runs successfully.
- `python -m compileall -q packages tests`, Ruff on the DGR-004 Python files,
and `git diff --check` pass.
## Blocker
The verification environment no longer contains the `.venv` recorded in
`commands.txt`, nor a `cmake` executable on `PATH`. The available global
pytest environment cannot import the native protocol because its protobuf
runtime is 6.33.6 while the checked-in generated code requires 7.35.0. This
causes both `tests/test_llama_cpp_dependency.py` and the native protocol suite
to fail during the repository-wide autouse fixture setup, before their tests
run.
This prevents the required fresh focused test and native CTest verification.
No DGR-004 completion state, commit, or push is claimed from this worktree.
## Continuation
1. Restore the project test environment used by the prior evidence (including
protobuf >= 7.35.0 and CMake), without changing DGR-004 source files.
2. Run the exact focused test command from `commands.txt` and the clean
`reproduce` command using the local llama.cpp object cache.
3. Re-run compileall, Ruff, diff check, and the deterministic full suite.
4. Only then apply the supervising engine's commit policy and unblock DGR-005.
## Dependency graph
`DGR-004 verification -> DGR-005 range-aware GGUF ownership -> DGR-003 live
ShardIdentity emission`. DGR-005 and DGR-003-emission were not modified.

View File

@@ -0,0 +1,31 @@
# DGR-004 — Reproducible pinned llama.cpp patch stack
Status: **done**. This is reproducible native-build infrastructure evidence, not model execution evidence.
## Delivered boundary
- Pin: `ggml-org/llama.cpp` at `e920c523e3b8a0163fe498af5bf90df35ff51d25` (tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1`).
- Ordered patch: `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch`, SHA-256 `1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75`.
- The sole patch adds an interface-library CMake marker. It adds no model execution/loading, networking, Tracker, relay, gRPC, billing, or authentication code.
- `scripts/llama_cpp_dependency.py` makes a fresh checkout, validates commit/tree/baseline blob, validates patch order/digests/context, applies the series, and verifies the exact resulting Git index tree. It rejects stale destinations, upstream drift, changed patches, untracked files, and local edits.
## Build and smoke result
The clean build cloned only the already-present exact Git object cache as a read-only source and did not trust its worktree. CMake 4.4.0 and GCC 15.2.1 built `llama-gguf-hash` with the locked Release/CPU flags in `UPSTREAM_LOCK.json`; `llama-gguf-hash --help` passed with no model download or load.
llama.cpp tests are intentionally off for this small no-model smoke target, so no upstream CTest applies. Meshnet's focused native protocol suite passed independently. Exact results are in `commands.txt` and `results.json`.
## License, compatibility, and handoff
llama.cpp is MIT licensed. The materializer requires upstream `LICENSE`, preserves all upstream notices, and `THIRD_PARTY_NOTICES.md` requires including them in redistribution. No Mesh-LLM code or patch was adopted.
The lock records the patched upstream blob and resulting patched tree. Pin updates must intentionally revise those values, the patch digest/order, toolchain metadata, and evidence.
This stock/native build is **infrastructure evidence only**: not a standalone Meshnet worker (DGR-008), GLM semantic acceptance, DSA/IndexShare proof, numerical equivalence, performance success, model-fit evidence, or route certification. The stock dense-MLA fallback remains explicitly uncertified. DGR-001 CPU v1 remains `stop`; DGR-017 is a separate target contract. DGR-005 may consume this dense-Llama structural boundary; DGR-018/DGR-019 must prove GLM semantics.
## Files changed
- `packages/node/native/llama/*`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- this evidence directory, the DGR-004 issue, and `prd.json`

View File

@@ -0,0 +1,28 @@
# DGR-004 commands and real results — 2026-07-14
```text
$ .venv/bin/python -m pytest -q tests/test_llama_cpp_dependency.py tests/test_native_shard_protocol.py
47 passed, 1 skipped in 0.59s
$ .venv/bin/python scripts/llama_cpp_dependency.py reproduce --work-dir build/dgr-004-smoke --source-repository /run/media/popov/d/DEV/llamacpp/llama.cpp
llama-gguf-hash --help -> exit 0; output contains "Hash a GGUF file"
$ touch build/dgr-004-drift/source/DGR-004-local-edit
$ .venv/bin/python scripts/llama_cpp_dependency.py apply --source-dir build/dgr-004-drift/source
DGR-004 dependency error: local edits detected in materialized llama.cpp checkout
exit 2
$ .venv/bin/python -m compileall -q packages tests
exit 0
$ ruff check scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
All checks passed!
$ git diff --check
exit 0
$ .venv/bin/python -m pytest -q --cache-clear
902 passed, 13 skipped in 255.01s (0:04:15)
```
The source-cache command avoids transient network availability only. The script defaults to the public upstream URL and verifies the exact object/tree, not external worktree state.

View File

@@ -0,0 +1,18 @@
{
"evidence_class": "native build infrastructure",
"llama_cpp": {
"upstream": "https://github.com/ggml-org/llama.cpp.git",
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
"patched_tree": "4a37c06fac668834435b803caa59ba272bdace5c",
"patch_sha256": "1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75"
},
"toolchain": {"cmake": "4.4.0", "cxx": "GCC 15.2.1", "generator": "Unix Makefiles", "target": "llama-gguf-hash", "configure_flags": ["-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_TESTS=OFF", "-DLLAMA_BUILD_EXAMPLES=ON", "-DLLAMA_BUILD_SERVER=OFF", "-DLLAMA_BUILD_TOOLS=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_CURL=OFF"]},
"checks": {"clean_materialize_apply_build_smoke": "passed", "local_edit_detection": "passed (exit 2)", "focused_pytest": "47 passed, 1 skipped", "compileall": "passed", "ruff": "passed", "git_diff_check": "passed", "full_pytest": "902 passed, 13 skipped"},
"model_downloads": false,
"model_loaded": false,
"inference_run": false,
"glm_semantic_certification": false,
"performance_certification": false,
"route_certification": false
}

View File

@@ -0,0 +1,63 @@
# DGR-005 decomposition — 2026-07-14
## Verified starting point
- The mandated environment is present: project Python 3.14.6, CMake 4.4.0,
and protobuf 7.35.1.
- DGR-003's focused identity/capability tests and DGR-004's dependency tests
pass together: `95 passed`.
- The DGR-004 materialized source at the pinned commit is available for source
inspection. It contains only the DGR-004 CMake-marker patch.
## Why this chain cannot safely claim DGR-005 yet
At the locked llama.cpp revision, `llama_model_base::load_tensors()`:
1. sizes `layers` to `hparams.n_layer_all`;
2. calls every architecture loader, which registers each architecture's layer
tensors; and
3. runs a generic optional-scale pass over the full layer count before creating
mmap/backend buffers.
Filtering names after this point does not meet the ownership contract: it
leaves full-model model/graph assumptions and can make a middle Shard silently
look valid while it lacks the endpoint and boundary semantics needed by the
next story. A generic `blk.N.*` filter alone is also not an architecture
adapter, which violates ADR-0020's fail-closed dense-Llama-first rule.
## Required child slices
1. **DGR-005A — native dense-Llama ownership API and loader**
- Add an explicit end-exclusive owned range to the project-owned native
interface and validate it against immutable GGUF layer metadata.
- Restrict registration, optional scales, allocation and mmap ranges to the
owned `blk.N.*` tensors.
- Record authoritative loaded start/end and mapped/resident byte counters
from the instantiated model, not command-line input.
- Add a deterministic synthetic dense-Llama GGUF fixture plus native tests
for head, middle and tail ranges.
2. **DGR-005B — endpoint ownership and graph guard**
- Load token embeddings only for the head, and final norm/output head only
for the tail, including tied embeddings.
- Make the dense-Llama graph fail closed when an endpoint-required tensor is
absent; do not infer endpoint ownership from an empty pointer.
- Prove that split ranges map fewer bytes than the whole-model fixture and
that the loaded range report matches actual registered tensors.
3. **DGR-003-emission follow-up**
- Expose the resulting immutable native loaded-artifact report to a native
worker/backend adapter.
- Construct `ShardIdentity` only from that report plus the immutable
artifact, tokenizer and numerical-recipe inputs. The legacy Transformers
doctor path must remain identity-free rather than fabricate a pin.
- Wire `check_session_open()` at the worker SessionOpen boundary; current
unit coverage already verifies its fail-closed fingerprint, range,
session and epoch behavior.
## Handoff and non-claims
No DGR-005 source patch, identity-emission code, issue status, or `prd.json`
pass state was changed. No model was loaded, downloaded, benchmarked, or
certified. This document is a supervised-review handoff, not DGR-005 evidence
of completion.

View File

@@ -0,0 +1,79 @@
# DGR-005 — dense-Llama range-aware GGUF ownership
Evidence class: deterministic offline/unit (synthetic fixture) plus
real-model integration (TinyLlama 1.1B, opt-in via MESHNET_ENABLE_REAL_INFERENCE_TESTS=1).
## Result
All six acceptance criteria pass:
1. **Range-aware tensor ownership**: native C++ patch (`0002-dense-llama-owned-range-loader.patch`,
169 lines as merged — DGR-005A's original 365-line version was slimmed by DGR-005B)
adds `llama_model_params.meshnet_owned_layer_start/end`, `llama_meshnet_range_report`,
and restricts `blk.N.*` registration to the owned range.
2. **Head/tail embedding loading**: head loads `token_embd.weight`; tail loads `output_norm`/`output`
(with tied-embedding dedup). Middle shards load zero endpoint tensors.
3. **Mapped/resident memory scales with owned tensors**: proven with TinyLlama 1.1B Q4_K_M.
4. **Targeted pytest tests**: `tests/test_llama_cpp_dependency.py` (3 tests — lock/patch
manifest consistency, offline dependency report, control-plane-code scan; re-verified
2026-07-14: `3 passed, 6 skipped` together with the opt-in integration file), native CTest
(`test-meshnet-range-ownership` synthetic fixture, added by the 0002 patch).
5. **compileall, ruff, git diff --check, full pytest**: all pass.
6. **Integration test**: `tests/test_gguf_distributed_load.py` (6/6, opt-in real model).
## Files changed (vs HEAD at DGR-004)
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch` — 169-line native patch (as merged)
- `packages/node/native/llama/patches/SHA256SUMS` — updated hash
- `packages/node/native/llama/patches/series` — added patch to series
- `packages/node/native/llama/UPSTREAM_LOCK.json` — updated patched_tree, serial number
- `scripts/llama_cpp_dependency.py``inspect` report for 2-patch stack
- `tests/test_llama_cpp_dependency.py` — patch_count 2
- `packages/node/native/llama/meshnet-range-loader.cpp` — C CLI wrapper
- `tests/test_gguf_distributed_load.py` — real-model integration test
## Commands
```text
# Build patched llama.cpp + range loader
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
-S build/dgr-004-final/source -B build/dgr-004-final/build \
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_EXAMPLES=ON \
-DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=OFF \
-DLLAMA_BUILD_TOOLS=ON -DLLAMA_CURL=OFF
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
--build build/dgr-004-final/build --target llama-simple -j$(nproc)
g++ -std=c++17 -Ibuild/dgr-004-final/source -Ibuild/dgr-004-final/source/include \
-Ibuild/dgr-004-final/source/ggml/include -Lbuild/dgr-004-final/build/bin \
packages/node/native/llama/meshnet-range-loader.cpp -lllama \
-Wl,-rpath,build/dgr-004-final/build/bin \
-o build/dgr-004-final/build/bin/meshnet-range-loader
# Focused tests (no model download)
PYTHONPATH=packages/node:packages/tracker:packages/contracts
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
-m pytest -q tests/test_llama_cpp_dependency.py
# Real-model integration test (opt-in, downloads ~670 MB)
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=... \
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
-m pytest -q tests/test_gguf_distributed_load.py
```
## Limitations
- Dense-Llama architecture only (LLM_ARCH_LLAMA). GLM/MoE/MLA is DGR-006+.
- Graph-level endpoint assertions (`has_token_embeddings`, `has_output_head`) were
simplified to only `start_layer`/`end_layer`/`mapped_bytes`/`resident_bytes` in
the patch as merged. Full endpoint tracking is available via the integration test
by observing which tensors are registered per shard.
- Loading the full `llama-simple` CLI requires reconfiguring with `-DLLAMA_BUILD_EXAMPLES=ON`.
The smoke-only build (`llama-gguf-hash`) is sufficient for patch verification.
- TinyLlama 1.1B is a baseline dense-Llama architecture only.
## Commits
- `252d131` feat: DGR-005A dense Llama owned range loader
- `f844ae6` feat: DGR-005B endpoint ownership and graph guard
- `31065c0` feat: distributed GGUF shard load integration test with TinyLlama 1.1B
- `d6b808d` chore: mark DGR-005 passes:true in PRD

View File

@@ -0,0 +1,71 @@
# DGR-006 — architecture-defined boundary input/output
Status: complete deterministic/offline contract and dense-fixture evidence.
## Result
The native protocol now carries a versioned `TensorBundle` on the decode fast
path. It includes explicit architecture and boundary-point metadata. Its legacy
`NamedTensor` field remains a compact one-tensor encoding for certified dense
boundaries; the writer deliberately selects it only for a one-tensor bundle and
new readers wrap that representation into a bundle. The bundle is authoritative
when present, allowing MoE/MLA sidebands without a second transport contract.
`architecture_boundary.py` is the fail-closed adapter boundary. Dense head
Shards accept token IDs and own embedding. Middle/tail Shards accept only a
validated bundle. Dense, MoE, and MLA route through explicit adapters; unknown
architectures are rejected. The dense F32 fixture proves whole-model versus
two-range boundary parity without model downloads or real inference.
Tail output is explicit in the schema: `TailResult` contains either logits or a
sampled token and binds sampling parameters plus request ID, runtime recipe,
chat template/version, reasoning mode, and architecture identity. The adapter
builds and validates the serialized protobuf result before returning it.
## Files changed
- `packages/node/native/proto/shard_runtime.proto`
- `packages/node/meshnet_node/native_protocol/{codec.py,__init__.py,conformance.py,generated/*}`
- `packages/node/native/testdata/decode_step_golden.binpb`
- `packages/node/native/tests/test_shard_protocol_conformance.cpp`
- `packages/node/meshnet_node/architecture_boundary.py`
- `tests/test_architecture_boundary.py`
- `tests/test_native_shard_protocol.py`
- `packages/node/native/README.md`
## Commands and results
All Python commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python`.
All native commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake`.
```text
python scripts/generate_native_protocol.py --check -> passed
python scripts/generate_protocol_goldens.py --check -> passed
pytest -q tests/test_architecture_boundary.py \
tests/test_native_shard_protocol.py tests/test_llama_cpp_dependency.py
-> 59 passed
cmake -S packages/node/native -B build/native \
-DCMAKE_PREFIX_PATH=/tmp/pbsrc/install -> configured
cmake --build build/native -j$(nproc) -> built shard_protocol_conformance
ctest --test-dir build/native --output-on-failure -> 1/1 passed
python -m compileall -q packages tests -> passed
git diff --check -> passed
pytest -q -> 917 passed, 18 skipped
```
## Compatibility and limitations
- Existing Nodes that send `DecodeStep.tensor` are accepted. New multi-tensor
Nodes require the versioned bundle and older Nodes safely preserve it as an
unknown field rather than interpreting it as a single tensor.
- The committed C++ conformance vector covers the multi-tensor decode path.
- The dense parity result is a deterministic F32 structural fixture, not real
GGUF inference or GLM certification. No real inference was run.
- MoE and MLA adapters define and validate their sideband contracts but are not
architecture certifications. DGR-019 owns GLM MoE/MLA/DSA/IndexShare semantics.
## Handoff
DGR-007 can key its Hot KV state to the validated decoded bundle. DGR-008 can
translate the generated `TailResult` and decode bundle over gRPC. DGR-019 must
replace the generic MoE/MLA sideband names with exact certified GLM semantics.

View File

@@ -0,0 +1,114 @@
# DGR-017 evidence — superseded backlog cleanup
**Completed:** 2026-07-16
**Branch:** `ralph/distributed-gguf-runtime`
**Planning checkpoint before cleanup:** `81b1fa6`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
## Outcome
The old DGR-001…016 completion claims and active artifacts were reconciled against the live branch. No old pass state transferred to the new implementation roadmap.
The active `packages/` and `tests/` trees were restored exactly to `origin/master`. The branch therefore no longer exposes a nominal GGUF startup path backed by unimplemented transport methods, a protobuf-only native scaffold, or isolated synthetic scheduler/cache/failure modules as if they were a working distributed GGUF runtime.
## Classification and disposition
### Retained
- Accepted ADRs and repository research, including `docs/research/colibri-implementation-audit.md`.
- The authoritative 55-story roadmap `DGR-017…071` and its generated issue specifications.
- The real public-relay smoke benchmark, moved with provenance to `legacy-public-relay-smoke-benchmark.json`.
- Git history containing the complete superseded implementation/reference work.
### Removed from the active tree
- Legacy issue specifications DGR-001…016 and their stale/blocked/synthetic evidence directories.
- The nonfunctional `gguf_backend` startup path whose gRPC execution methods raised not-implemented errors.
- Synthetic/reference-only boundary, Hot KV, scheduler, failure, recipe, ownership, and native-protocol modules that were not a real llama.cpp Shard runtime.
- The protobuf round-trip-only native scaffold, placeholder llama.cpp patch, generated bindings/build workspace, and associated tests.
- Tracker/admission/source modifications coupled to that superseded scaffold.
### Confirmed absent and still required
- Real standalone C++ gRPC Shard worker.
- Exact pinned llama.cpp manifest and verified patch stack.
- Range-aware GGUF tensor ownership and real ranged execution.
- Real Shard-local llama.cpp KV/V4 auxiliary state.
- DeepSeek V4 boundary adapter and ranged parity.
- Real multi-machine DeepSeek V4 alpha or beta acceptance.
These remain `passes: false` in DGR-018…071.
## Before-cleanup baseline
Command:
```bash
.venv-rocm/bin/python -m pytest -q \
tests/test_performance_contract.py tests/test_native_shard_protocol.py \
tests/test_gguf_ownership.py tests/test_boundary_adapter.py \
tests/test_hot_kv_state.py tests/test_gguf_backend.py \
tests/test_batch_scheduler.py tests/test_failure_semantics.py \
tests/test_llama_worker_build.py tests/test_node_admission.py \
tests/test_node_capability.py tests/test_tracker_capability_admission.py
```
Result:
```text
216 passed, 2 skipped, 1 failed, 1 warning
```
The failure was a synthetic capability-test helper `KeyError: 'compatibility_fingerprint'`. The warning was a pre-existing heartbeat-thread `SystemExit` warning.
## Cleanup verification
### Source equality
Command:
```bash
git diff --quiet origin/master -- packages tests
```
Result:
```text
packages_tests_match_origin_master=yes
```
The staged cleanup removes approximately 15.3k obsolete source/test/evidence lines from the active branch.
### Cleanup-relevant regression suite
Command:
```bash
.venv-rocm/bin/python -m pytest -q \
tests/test_node_admission.py tests/test_node_capability.py \
tests/test_tracker_capability_admission.py \
tests/test_kv_cache_distributed.py tests/test_real_distributed_inference.py
```
Result:
```text
119 passed, 2 skipped, 1 warning in 15.90s
```
The warning is the same pre-existing heartbeat-thread `SystemExit` warning.
### Known `origin/master` limitations
The wider routing run produced `210 passed, 2 skipped, 4 failed, 1 warning`. Each failure reproduced individually while `packages/` and `tests/` matched `origin/master` exactly:
- `test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`
- `test_torch_node_applies_tracker_load_shard_directive`
- `test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`
- `test_a_node_with_an_unusable_precision_covers_no_layers`
They are recorded as pre-existing baseline defects and were not repaired or hidden by this cleanup story.
## Dependency handoff
DGR-018 and later stories must start from the cleaned upstream-equivalent runtime tree. Reuse concepts from superseded commits only by explicitly porting the smallest verified slice under the new storys contracts, tests, and evidence gates. Git history is provenance, not completion evidence.

View File

@@ -0,0 +1,115 @@
# DGR-017 — exact commands and real results (2026-07-13)
# Project venv is used explicitly. NOTE: bare `pytest` on this machine resolves to
# Hermes Agent's internal venv (/home/popov/.hermes/...), which DGR-001 already
# recorded as the cause of a bogus "suite is blocked" claim. Always use $VP.
VP=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python # Python 3.14.6
# ---------------------------------------------------------------------------
# 1. Resolve the target from upstream metadata ONLY. No weight payload downloaded.
# Sizes and SHA-256 come from the HF LFS pointer metadata (paths-info), not the blobs.
# ---------------------------------------------------------------------------
curl -sS "https://huggingface.co/api/models/zai-org/GLM-5.2"
-> sha b4734de4facf877f85769a911abafc5283eab3d9 (matches the roadmap pin)
-> license mit, lastModified 2026-07-02T08:08:14.000Z
curl -sS "https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF"
-> sha abc55e72527792c6e77069c99b4cb7de16fa9f23 (matches the roadmap pin)
-> license mit, lastModified 2026-06-23T15:18:23.000Z
-> six UD-IQ1_S shards present
curl -sS -X POST -d '{"paths": [<6 UD-IQ1_S shards>]}' \
"https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF/paths-info/abc55e7..."
-> all six shards resolved with exact size + LFS oid (sha256)
-> sum = 216,715,360,960 bytes = 201.832 GiB = 216.715 GB
-> matches the roadmap's published byte total EXACTLY
-> UD-IQ1_M fallback = 228,492,966,624 bytes = 212.801 GiB (also matches)
curl -sS ".../resolve/b4734de4.../{config.json,chat_template.jinja,
generation_config.json,tokenizer_config.json}"
-> config.json 3732 B sha256 185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a
-> chat_template.jinja 5076 B sha256 172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679
-> generation_config.json 194 B sha256 ac76b43d8683d3b930126870fc8be73d8679308fe752fa1f381096d8354f6a55
-> tokenizer_config.json 761 B sha256 98b1271574f41abf89427ae2dda030d94dc9478f0edc5a8bd240db213c6fd5fc
# ---------------------------------------------------------------------------
# 2. Verify the checked-in pins still match live upstream (reproducible, no weights)
# ---------------------------------------------------------------------------
$VP scripts/refresh_glm_target_manifest.py --check
-> "target manifest and architecture snapshot match upstream"
-> exit 0
# ---------------------------------------------------------------------------
# 3. Upstream llama.cpp / donor status refresh (GitHub REST API, read-only)
# ---------------------------------------------------------------------------
curl -sS "https://api.github.com/repos/ggml-org/llama.cpp/issues/{24730,24770,25407,24231}"
-> #24730 issue OPEN "Feature Request: Support for GLM 5.2"
-> #24770 PR MERGED 2026-06-20 dense-MLA compatibility loader (DSA tensors optional)
-> #24231 PR MERGED 2026-07-11 generic GGML_OP_LIGHTNING_INDEXER [CHANGED since roadmap]
-> #25407 PR OPEN updated 2026-07-13, non-draft, 12 files, +414/-7 GLM 5.2 Indexer support
curl -sS "https://api.github.com/repos/Mesh-LLM/mesh-llm{,/branches/feat%2Fjianyang-glm-52}"
-> Apache-2.0, 2048 stars, branch head 9bd18f1509dff7fac21578635084035b3ba90a38 (2026-07-12)
-> recorded as donor only; nothing forked, nothing adopted
# ---------------------------------------------------------------------------
# 4. Seal the alpha contract (digest over its own canonical content)
# ---------------------------------------------------------------------------
$VP -c "seal_contract(...)" -> contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
# ---------------------------------------------------------------------------
# 5. Generate the machine-readable resource plan from the pinned artifact
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node $VP <generate resource-plan.json>
-> manifest_sha256 0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962
-> architecture_sha256 253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1
-> alpha_contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
-> tier arithmetic minimum 32:9 48:6 64:4 96:3 128:2 (reproduces the roadmap table)
-> tier recommended 32:10 48:6 64:5 96:3 128:3 (reproduces the roadmap table)
-> 5x64 GiB unified fits, +53.28 GiB headroom
-> 3x96 GiB unified fits, +27.68 GiB headroom
-> 2x128 / 4x64 (fit probes) fit with only +2.08 GiB headroom across the WHOLE route
-> 2x112 GiB (= 224 GiB, the hard-fit floor) DOES NOT FIT: -23.52 GiB
-> 3x64 GiB does not fit: -49.12 GiB
# ---------------------------------------------------------------------------
# 6. Quality gates (project .venv, deterministic, offline, GPU-free)
# ---------------------------------------------------------------------------
$VP -m pytest -q tests/test_glm_alpha_target.py
-> 97 passed in 0.12s
-> includes coordinated shard/config substitution, malformed telemetry, and
contract-ID reseal rejection tests added during controller review
$VP -m pip wheel --no-deps packages/node -w /tmp/dgr017-wheel
$VP -m pip install --no-deps --target /tmp/dgr017-install /tmp/dgr017-wheel/*.whl
$VP -I -c "... from meshnet_node.glm_alpha import load_locked_target ..."
-> INSTALLED_WHEEL_PASS
-> packaged alpha-contract.json, target-manifest.json, and architecture-snapshot.json
load and cross-bind successfully outside the source tree
$VP -m compileall -q packages tests
-> exit 0
git diff --check
-> exit 0
$VP -m pytest -q # full deterministic suite
-> first final run: 1 failed, 851 passed, 13 skipped; only the tracker cancellation
race already documented by DGR-001/DGR-002 failed
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 1 passed, repeated 5/5 in isolation
$VP -m pytest -q # integrated rerun
-> 852 passed, 13 skipped in 253.30s (0:04:13)
# ---------------------------------------------------------------------------
# 7. Late independent-review repair (2026-07-14)
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node $VP -m pytest -q tests/test_glm_alpha_target.py
-> 99 passed in 0.15s
-> adds trusted-v1-digest rejection after coordinated mutation + reseal
-> adds nested parsed-state immutability and isolated to_dict() coverage
$VP -m pytest -q # after DGR-003 integration and DGR-017 repair
-> first run: 871 passed, 13 skipped, 1 known cancellation-race failure
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 5/5 passed in isolation
$VP -m pytest -q # integrated rerun
-> 872 passed, 13 skipped in 253.46s (0:04:13)

View File

@@ -0,0 +1,83 @@
{
"schema_version": 1,
"executed_at_utc": "2026-07-15T10:41:14Z",
"test_kind": "public-relay-single-node-streaming-smoke-benchmark",
"target": {
"public_chat_endpoint": "https://meshnet.2.d-popov.com/v1/chat/completions",
"relay_url": "wss://meshnet.2.d-popov.com/ws",
"model": "qwen2.5-0.5b-instruct",
"quantization": "bfloat16"
},
"recovery": {
"problem": "The local node's capability proof had expired and its port-7000 HTTP server had wedged with CLOSE-WAIT sockets.",
"action": "Gracefully restarted the local public-tracker meshnet-node process on port 7000.",
"startup_validation": {
"device": "cuda",
"capability_proof_ms": 336,
"node_id": "7j77FsPY-b32476219492",
"relay_addr": "wss://meshnet.2.d-popov.com/rpc/7j77FsPY1evV8tuf-7000"
}
},
"tracker_admission_after_recovery": {
"node_id": "7j77FsPY-b32476219492",
"alive": true,
"status": "ready",
"capability_state": "admitted",
"routable": true,
"route_hops": 1
},
"client_measurements": {
"warmup": {
"http_status": 200,
"ttft_ms": 420.8,
"elapsed_ms": 610.23,
"response_text": "MeshNet Relay Benchmark Passed"
},
"runs": [
{
"run": 1,
"ttft_ms": 376.04,
"elapsed_ms": 458.65,
"response_text": "relay benchmark pass"
},
{
"run": 2,
"ttft_ms": 258.33,
"elapsed_ms": 336.71,
"response_text": "relay benchmark pass"
},
{
"run": 3,
"ttft_ms": 288.26,
"elapsed_ms": 363.2,
"response_text": "relay benchmark pass"
}
],
"p50_ttft_ms": 288.26,
"p50_elapsed_ms": 363.2
},
"tracker_relay_evidence": [
{
"status": 200,
"relay": true,
"node_id": "7j77FsPY-b32476219492",
"tokens": 11,
"elapsed_seconds": 0.1686,
"tokens_per_sec": 65.2541
},
{
"status": 200,
"relay": true,
"node_id": "7j77FsPY-b32476219492",
"tokens": 11,
"elapsed_seconds": 0.1891,
"tokens_per_sec": 58.1799
}
],
"scope_and_remaining_work": {
"validated": "Public HTTPS chat endpoint routed a streaming request through the tracker relay to the local CUDA node and completed with HTTP 200.",
"not_validated": "Two-node shard routing was not run because the remote node 5gMLrmyB-88f5cba044d0 still had an expired capability proof and was not routable.",
"next_gate": "Refresh the remote node capability proof, then load a multi-node-compatible assignment and repeat the benchmark through the public tracker relay."
},
"reproduction": "Use a valid bearer API key with the public /v1/chat/completions endpoint and stream a short qwen2.5-0.5b-instruct request. Do not connect directly to private node HTTP endpoints; the tracker relay is the required path."
}

View File

@@ -0,0 +1,255 @@
{
"generated_by": "DGR-017 meshnet_node.glm_alpha.planner",
"target": {
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
"quantization": "UD-IQ1_S",
"total_bytes": 216715360960,
"total_gib": 201.832,
"total_gb": 216.715
},
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962",
"architecture_snapshot_sha256": "253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1",
"alpha_contract_sha256": "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9",
"kv_assumptions": {
"dtype": "Q8_0",
"bytes_per_value": 1.0625,
"context_tokens": 16384,
"concurrency": 1,
"indexer_layout": "conservative",
"mla_values_per_token_per_layer": 576,
"backbone_layers": 78,
"indexer_full_layers": 21,
"note": "Alpha budgets indexer keys across all 78 layers (current experimental DSA layout), not only the 21 Full layers."
},
"kv_table_gib": {
"16384": {
"mla_only_q8_gib": 0.73,
"optimized_dsa_q8_gib": 0.77,
"conservative_dsa_q8_gib": 0.89,
"conservative_dsa_f16_gib": 1.68
},
"131072": {
"mla_only_q8_gib": 5.83,
"optimized_dsa_q8_gib": 6.18,
"conservative_dsa_q8_gib": 7.12,
"conservative_dsa_f16_gib": 13.41
},
"1048576": {
"mla_only_q8_gib": 46.62,
"optimized_dsa_q8_gib": 49.41,
"conservative_dsa_q8_gib": 56.98,
"conservative_dsa_f16_gib": 107.25
}
},
"aggregate_hard_fit_floor_gib": 224.0,
"aggregate_floor_class": "experimental_hard_fit_floor",
"placement_imbalance_factor": 1.1,
"tier_table": {
"32": {
"physical_usable_gib": 32.0,
"reserve_gib": 8.0,
"placement_budget_gib": 24.0,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 9,
"recommended_nodes": 10,
"imbalance_factor": 1.1
},
"48": {
"physical_usable_gib": 48.0,
"reserve_gib": 9.6,
"placement_budget_gib": 38.4,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 6,
"recommended_nodes": 6,
"imbalance_factor": 1.1
},
"64": {
"physical_usable_gib": 64.0,
"reserve_gib": 12.8,
"placement_budget_gib": 51.2,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 4,
"recommended_nodes": 5,
"imbalance_factor": 1.1
},
"96": {
"physical_usable_gib": 96.0,
"reserve_gib": 19.2,
"placement_budget_gib": 76.8,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 3,
"recommended_nodes": 3,
"imbalance_factor": 1.1
},
"128": {
"physical_usable_gib": 128.0,
"reserve_gib": 25.6,
"placement_budget_gib": 102.4,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 2,
"recommended_nodes": 3,
"imbalance_factor": 1.1
}
},
"routes": {
"recommended_5x64_unified": {
"node_count": 5,
"aggregate_usable_gib": 320.0,
"aggregate_placement_budget_gib": 256.0,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 53.278,
"reasons": []
},
"recommended_3x96_unified": {
"node_count": 3,
"aggregate_usable_gib": 288.0,
"aggregate_placement_budget_gib": 230.4,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 27.678,
"reasons": []
},
"recommended_3x128_unified": {
"node_count": 3,
"aggregate_usable_gib": 384.0,
"aggregate_placement_budget_gib": 307.2,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 104.478,
"reasons": []
},
"fit_probe_2x128_unified": {
"node_count": 2,
"aggregate_usable_gib": 256.0,
"aggregate_placement_budget_gib": 204.8,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 2.078,
"reasons": []
},
"fit_probe_4x64_unified": {
"node_count": 4,
"aggregate_usable_gib": 256.0,
"aggregate_placement_budget_gib": 204.8,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 2.078,
"reasons": []
},
"hard_fit_floor_2x112_unified": {
"node_count": 2,
"aggregate_usable_gib": 224.0,
"aggregate_placement_budget_gib": 179.2,
"required_placement_gib": 202.722,
"fits": false,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": -23.522,
"reasons": [
"aggregate placement budget 179.2 GiB is below the 202.7 GiB the target needs after each node's reserve"
]
},
"insufficient_3x64_unified": {
"node_count": 3,
"aggregate_usable_gib": 192.0,
"aggregate_placement_budget_gib": 153.6,
"required_placement_gib": 202.722,
"fits": false,
"meets_hard_fit_floor": false,
"no_single_node_can_admit_target": true,
"headroom_gib": -49.122,
"reasons": [
"aggregate placement budget 153.6 GiB is below the 202.7 GiB the target needs after each node's reserve",
"aggregate usable memory 192.0 GiB is below the 224 GiB experimental hard-fit floor"
]
}
},
"seams": {
"3_nodes_2.5gbe": {
"node_count": 3,
"seam_count": 2,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 2.5,
"meets_alpha_minimum": true,
"is_recommended_link": false,
"decode_serialization_ms_per_token": 0.0786,
"decode_latency_ms_per_token": 1.0,
"decode_bandwidth_share_ms_per_token": 0.0786,
"prefill_serialization_ms": 1288.49
},
"3_nodes_10.0gbe": {
"node_count": 3,
"seam_count": 2,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 10.0,
"meets_alpha_minimum": true,
"is_recommended_link": true,
"decode_serialization_ms_per_token": 0.0197,
"decode_latency_ms_per_token": 1.0,
"decode_bandwidth_share_ms_per_token": 0.0197,
"prefill_serialization_ms": 322.123
},
"5_nodes_2.5gbe": {
"node_count": 5,
"seam_count": 4,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 2.5,
"meets_alpha_minimum": true,
"is_recommended_link": false,
"decode_serialization_ms_per_token": 0.1573,
"decode_latency_ms_per_token": 2.0,
"decode_bandwidth_share_ms_per_token": 0.1573,
"prefill_serialization_ms": 2576.98
},
"5_nodes_10.0gbe": {
"node_count": 5,
"seam_count": 4,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 10.0,
"meets_alpha_minimum": true,
"is_recommended_link": true,
"decode_serialization_ms_per_token": 0.0393,
"decode_latency_ms_per_token": 2.0,
"decode_bandwidth_share_ms_per_token": 0.0393,
"prefill_serialization_ms": 644.245
}
}
}

View File

@@ -0,0 +1,89 @@
{
"observed_at": "2026-07-13",
"observed_by": "DGR-017",
"method": "GitHub REST API (api.github.com), read-only; no fork, no clone, no patch adopted",
"refresh_note": "The roadmap's 2026-07-13 observations were re-verified against live upstream. One item changed: PR #24231 is now MERGED (2026-07-11), which the roadmap already anticipated as 'generic CPU lightning-indexer support is merged'.",
"llama_cpp": {
"repo": "ggml-org/llama.cpp",
"items": [
{
"ref": "issue #24730",
"url": "https://github.com/ggml-org/llama.cpp/issues/24730",
"title": "Feature Request: Support for GLM 5.2",
"type": "issue",
"state": "open",
"updated_at": "2026-07-03T22:02:15Z",
"meaning": "The umbrella GLM-5.2 support request is still open. GLM-5.2 is not fully supported upstream."
},
{
"ref": "PR #24770",
"url": "https://github.com/ggml-org/llama.cpp/pull/24770",
"title": "model : glm-dsa load DSA indexer tensors as optional",
"type": "pull_request",
"state": "closed",
"merged_at": "2026-06-20T10:48:24Z",
"meaning": "MERGED. GLM-5.2 loads, but through a dense-MLA compatibility path with DSA indexer tensors treated as optional. This is the fallback the alpha contract explicitly refuses: it can produce text without performing DSA/IndexShare computation."
},
{
"ref": "PR #24231",
"url": "https://github.com/ggml-org/llama.cpp/pull/24231",
"title": "New GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer",
"type": "pull_request",
"state": "closed",
"merged_at": "2026-07-11T09:39:07Z",
"meaning": "MERGED since the roadmap was written. A generic lightning-indexer op now exists in GGML. Backend coverage beyond CPU remains uneven and must be verified per backend by DGR-018, not assumed."
},
{
"ref": "PR #25407",
"url": "https://github.com/ggml-org/llama.cpp/pull/25407",
"title": "GLM 5.2 Indexer support",
"type": "pull_request",
"state": "open",
"draft": false,
"mergeable_state": "unstable",
"head_sha": "8dedd06415f36f10fc6091241a39b23c1bf0ee11",
"base": "master",
"commits": 6,
"changed_files": 12,
"additions": 414,
"deletions": 7,
"updated_at": "2026-07-13T15:28:51Z",
"meaning": "OPEN and actively moving (updated today). This is the real DSA/IndexShare implementation alpha needs. It is narrow — 12 files, +414/-7 — which is the single most important finding for donor policy: the semantics alpha requires are reviewable and trackable upstream, not a 261-patch fork."
}
]
},
"capability_status_for_alpha": {
"gguf_load_of_UD-IQ1_S": "expected via merged #24770, unverified by this project; DGR-018 must prove it against the exact pinned artifact",
"moe_routing_and_shared_expert": "expected supported; unverified here",
"compressed_mla_kv": "supported via the dense-MLA compatibility path",
"dsa_lightning_indexer": "generic GGML op merged (#24231); GLM-5.2 wiring still open (#25407)",
"indexshare_full_shared_roles": "NOT upstream; only in open PR #25407",
"mtp_nextn": "not required for alpha; NextN tensors must be explicitly loaded or excluded, never silently reinterpreted",
"conclusion": "As of 2026-07-13 no released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare. A stock pin today would satisfy 'it emits text' via the dense fallback and would FAIL the alpha semantic-correctness contract. This is the gating technical risk for DGR-004 and DGR-018."
},
"donor": {
"repo": "Mesh-LLM/mesh-llm",
"url": "https://github.com/Mesh-LLM/mesh-llm",
"license": "Apache-2.0",
"stars_observed": 2048,
"pushed_at": "2026-07-13T06:45:51Z",
"glm_branch": "feat/jianyang-glm-52",
"glm_branch_head": "9bd18f1509dff7fac21578635084035b3ba90a38",
"glm_branch_head_date": "2026-07-12T06:37:43Z",
"policy": "TEST AND PATCH DONOR ONLY. Do not adopt the fork, its scheduler, discovery, routing, public mesh, or package manager. Meshnet remains the sole control plane (RALPH-CONTEXT runtime decision, ADR-0020).",
"focused_candidates": [
"GLM DSA graph semantics",
"lightning indexer and sparse-attention tests",
"IndexShare metadata and Full/Shared role validation",
"top-k sideband shape and lifecycle",
"stage-local KV filtering",
"target parity and performance fixtures"
],
"adoption_state": "none adopted in DGR-017. This story reads and records upstream state; it takes no patch and forks nothing."
},
"recommendation_for_dgr_004_and_dgr_018": [
"Track upstream PR #25407 rather than forking Mesh-LLM. At 12 files and +414/-7 it is small enough to review, reproduce, and carry as a numbered patch in the project's own pinned stack.",
"Any pin chosen before #25407 merges will load GLM-5.2 through the dense-MLA compatibility path. DGR-018 must therefore prove DSA/IndexShare are ACTIVE, not merely that the model emits text — the alpha contract already forbids the fallback.",
"Verify lightning-indexer backend coverage (#24231) on the specific backend the route will use. CPU support being merged says nothing about ROCm/HIP."
]
}

View File

@@ -0,0 +1,53 @@
# DGR-018 — BLOCKED: no 256-GiB-class oracle host
Recorded: 2026-07-14 (MAINT-003). Preflight scripts preserved at commit
`a0f28b5` ("chore: preserve DGR-018 preflight scripts (postponed)").
## Blocker
DGR-018 requires a 256-GiB-class host with at least **224 GiB
runtime-accessible memory** (the DGR-017 experimental hard-fit floor for the
whole-model `UD-IQ1_S` oracle) and **250 GB free storage** on one filesystem
outside `/home` (216.715 GB artifact plus resume/temp headroom). The available
development host fails both gates, so the whole-model oracle cannot be
established. Per the issue's finish contract, no smaller model may be
substituted.
DGR-019 (needs the DGR-018 oracle for parity certification) and DGR-020
(needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single
node admits the whole recipe) are blocked transitively.
## Exact preflight output
Command (offline; resolves everything from the pinned target manifest and
never contacts the network):
```
$ python scripts/glm_whole_model_preflight.py
target: UD-IQ1_S 216.715 GB, 6 shards @ abc55e725277
[FAIL] storage: need >= 250 GB free on one filesystem outside ['/home']; observed no eligible filesystem
[FAIL] memory: need >= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor); observed 124.9 GiB MemTotal
destination: NONE — no filesystem outside ['/home'] has 250 GB free
- /run/media/popov/DATA (ext4): 74.2 GB free
- / (ext4): 51.1 GB free
- /run/media/popov/Windows (fuseblk): 26.0 GB free
- /run/media/popov/d (fuseblk): 5.1 GB free
verdict: fail
$ echo $?
1
```
Host: Linux 7.0.14-101.fc43.x86_64 x86_64, `MemTotal: 130997376 kB`
(124.9 GiB). The full machine-readable report (including the ordered
download/verify plan against revision `abc55e72527792c6e77069c99b4cb7de16fa9f23`,
manifest SHA-256 `0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`)
is in [preflight.json](preflight.json).
## How to resume
1. On a qualifying host, run `python scripts/glm_whole_model_preflight.py`
(optionally `--dest DIR`); it must exit 0 with `verdict: pass`.
2. Download shards in the preflight's ordered plan; verify each with
`python scripts/verify_glm_shards.py` before the next transfer starts.
3. Proceed with the DGR-018 issue
(`.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md`).

View File

@@ -0,0 +1,204 @@
# DGR-018 evidence — canonical Ralph and Gitea metadata schema
**Completed:** 2026-07-16
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependency:** DGR-017 (`evidence/DGR-017/README.md`) — cleaned backlog reconciled to `origin/master`; no old pass state transferred.
## Objective
Make `prd.json` the validated source from which Markdown (and, later, Gitea) issues
can be generated losslessly, per
`.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`.
## Pre-existing state found (not caused by this story)
Before any change in this session, `git status` showed `.scratch/distributed-gguf-runtime/prd.json`
already modified in the working tree relative to `HEAD` (commit `369b207`), with no corresponding
progress-log entry. Diffing against `HEAD` showed the working copy had **dropped** prd.json's
top-level `sourceOfTruth`, `qualityGates`, `metadataSchema`, `milestones`, and `supersededStories`
objects, while `userStories` itself was byte-identical to `HEAD`. This looked like an abandoned,
uncommitted partial edit from a prior session, not intentional current work — those fields are
exactly the schema/quality-gate/audit-provenance content this story depends on, and their loss
wasn't explained by any acceptance criterion. They were restored (see "Changes" below) rather than
silently accepted or discarded, per the instruction to investigate unexplained working-tree state
before building on top of it.
## Changes
### `scripts/ralph_prd_schema.py` (new)
Single module providing:
- **Parse:** `load_prd(path)` — JSON load with clear `PrdValidationError`s for missing file /
invalid JSON / non-object document.
- **Canonical schema registry:** `STORY_FIELDS` (name → required/type), `EXECUTION_MODES`,
`EVIDENCE_CLASSES`, `HARDWARE_FLAGS`, `UPSTREAM_FLAGS`, `TRIAGE_VALUES`. Covers every field named
in the acceptance criteria: stable `id`/`title`, `labels`, `milestone`, derived `type`
(`derive_type`), `dependsOn`, derived `blocks`, `triage`, `evidenceClass`, and
`hardware`/`model`/`upstream` flags.
- **Structural validation:** `validate_schema(data)` — required fields, types, enum membership,
ID convention, `type:`/`priority:` label cardinality, non-empty `acceptanceCriteria`.
- **Semantic validation:** `validate_semantics(data)` — unique IDs, unique titles, `dependsOn`
resolves to known stories (no self-dependency), dependency graph is acyclic (with a reported
cycle path on failure), `blocks` matches the dependency graph exactly (sorted set equality, not
superset), and `evidencePath` matches the per-story convention.
- **Fresh vs. in-progress backlog:** `validate_fresh_backlog(data)` additionally requires every
story to start `passes: false` (for a backlog that hasn't started execution yet);
`validate_backlog(data)` is the composed check for a real, in-flight backlog where some stories
have legitimately completed.
- **Self-consistency check:** `validate_metadata_schema_consistency(data)` — when prd.json declares
its own `metadataSchema`/`qualityGates` (as this one now does), verifies that self-documentation
hasn't drifted from what the validator actually enforces (enum sets, required/optional field
lists, presence of `qualityGates` and `generatedArtifactDisclaimer`). This is a no-op for minimal
fixture PRDs that don't carry that documentation.
- **Generation (one-directional, prd.json → artifact):** `render_issue_markdown(story, data)`
renders the exact Markdown convention already used by
`.scratch/distributed-gguf-runtime/issues/*.md`, sourcing the "Shared quality gates" bullets from
`data["qualityGates"]` and the leading disclaimer from
`data["metadataSchema"]["generatedArtifactDisclaimer"]` (falling back to a module default only
when `data` omits them) — not from a duplicated Python string literal.
`to_gitea_issue_payload(story, data)` wraps the same body into a Gitea create-issue-shaped payload
(`title`, `body`, `labels`, `milestone`).
- **Authority guard:** `check_generated_markdown_authority(text, disclaimer=...)` rejects generated
Markdown that's missing the disclaimer or that contains a conflicting authority claim (e.g. "this
file is authoritative"). There is deliberately no Markdown → prd.json parser, so a generated
artifact structurally cannot feed `passes` (or anything else) back into the authoritative source.
- CLI: `python scripts/ralph_prd_schema.py validate <prd.json> [--fresh]` and
`... render <prd.json> <STORY-ID>`.
### `.scratch/distributed-gguf-runtime/prd.json`
- Restored the top-level `sourceOfTruth`, `qualityGates`, `milestones`, and `supersededStories`
objects to their `HEAD` content (see "Pre-existing state" above); `userStories` was already
identical to `HEAD` and is unchanged in content.
- Extended `metadataSchema` (previously incomplete for this story's own acceptance criteria) with:
`requiredStoryFields` now also lists `notes` and `blocks` (present on all 55 stories); new
`optionalStoryFields: ["completionNotes"]`; new `hardwareValues`/`upstreamValues` enums (`model`
is documented as an open convention, not a closed enum, since quantization/model targets are
dynamic recipe inputs per `RALPH-CONTEXT.md`); new `typeDerivation` and `labelConventions`
(reserved prefixes, cardinality); new `generatedArtifactDisclaimer` (the exact string generated
artifacts must start with); extended `dependencyRules`/`authorityRule` prose to match what the
validator enforces.
- Reworded `sourceOfTruth`'s stale "All stories are unimplemented ... passes=false" clause, which
was no longer accurate once DGR-017 completed.
- Marked `DGR-018.passes = true` with `completionNotes` recording this story's outcome.
### `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`
Regenerated via `render_issue_markdown` to reflect `passes: true` (checked acceptance criteria,
"completed" status line, "Verified evidence" handoff line) — matching the same convention DGR-017's
issue file already used for a completed story.
### `tests/test_ralph_prd_schema.py` (new)
108 deterministic, model-download-free, GPU-free tests:
- **Parse** (4 tests): real backlog parses to 55 stories; missing file, invalid JSON, and
non-object documents raise `PrdValidationError`.
- **Structural/semantic validation against the real backlog** (7 tests): passes `validate_schema`,
`validate_semantics`, and the composed `validate_backlog`; unique IDs/titles; all `dependsOn`
resolve; `blocks` matches the derived dependency graph for all 55 stories; no cycle; every
`passes: true` story carries `completionNotes` and an existing evidence README (a durable
invariant, not a hardcoded list of which stories have completed — that list will keep growing).
- **Structural/semantic failure-mode fixtures** (13 tests): missing required field, bad enum, wrong
type, empty `acceptanceCriteria`, multiple `type:` labels, duplicate ID, duplicate title, unknown
dependency, self-dependency, dependency cycle, mismatched `blocks`, bad `evidencePath`.
- **Fresh-backlog invariant** (3 tests): accepts all-`false`, rejects a premature `passes: true`,
and confirms `validate_backlog` (the in-progress variant) permits completed stories.
- **prd.json-as-source-of-truth for boilerplate** (9 tests): `qualityGates`/`metadataSchema`
self-consistency checks (no-op without them, catches a drifted enum, catches a missing
`qualityGates`), `quality_gate_bullets` flattening order, `authority_disclaimer` precedence and
fallback, and 3 tests asserting the real backlog's declared schema matches the code, its 7
quality-gate bullets are intact, and its disclaimer matches the module default.
- **`derive_type`** (4 tests): label-derived type, release-gate synthetic type for HITL gate
stories, `None` when absent, and confirmation that the real backlog's two release-gate stories
(`DGR-054`, `DGR-070`) derive `release-gate`.
- **Markdown generation round trips** (55 parametrized + 6 tests): `render_issue_markdown` for
every story `DGR-017`..`DGR-071` is byte-for-byte identical to the corresponding file already in
`.scratch/distributed-gguf-runtime/issues/`; determinism; leading disclaimer; `Blocks (derived)`
rendering (`None` vs. listed); checkbox reflects `passes`; filename convention.
- **Authority-claim rejection** (4 tests): accepts real generated text, rejects a missing
disclaimer, rejects an overriding claim, and confirms every committed issue file in
`.scratch/distributed-gguf-runtime/issues/` passes the check.
- **Gitea payload generation** (3 tests): payload shape, body carries no information beyond what's
in prd.json, and every real story's payload is well-formed and authority-clean.
## Commands and results
```bash
python3 -m pytest -q tests/test_ralph_prd_schema.py
```
```text
108 passed in 0.16s
```
```bash
python3 -m compileall -q packages tests
```
Exit code 0, no output (all files compile).
```bash
git diff --check
```
Exit code 0 (no whitespace errors).
```bash
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
OK: 55 stories validated.
```
```bash
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json --fresh
```
```text
ERROR: DGR-017: fresh backlog requires passes=false, got True
ERROR: DGR-018: fresh backlog requires passes=false, got True
2 validation error(s).
```
Expected: `--fresh` is the invariant for a backlog that hasn't started execution; this backlog has
legitimately completed two stories, so it correctly fails that stricter check while passing the
plain (in-progress) `validate` command above.
### Baseline: full repository suite (ad hoc `python3`, not a project venv)
```bash
python3 -m pytest -q
```
```text
20 failed, 776 passed, 13 skipped, 2 warnings in 244.17s (0:04:04)
```
None of the failures touch `scripts/ralph_prd_schema.py` or `tests/test_ralph_prd_schema.py`
(neither file existed before this story; this story adds no changes to `packages/`). Four of the
20 failures reproduce exactly the pre-existing baseline defects DGR-017's evidence already recorded
(`test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`,
`test_torch_node_applies_tracker_load_shard_directive`,
`test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`,
`test_a_node_with_an_unusable_precision_covers_no_layers`). The remaining 16 (activation
compression, dynamic routing, gossip/relay, manual route benchmark, openai gateway, TOPLoC
calibration dispatch, tracker control plane) include a `ModuleNotFoundError: langchain` failure,
indicating this ad hoc `python3` lacks the project's `dev` extras (`langchain-openai`, etc.) rather
than a real regression; this environment has no project virtualenv (e.g. no `.venv-rocm`) to run
against instead. Not investigated further as out of scope for this story.
## Limitations
- No real Gitea instance or API integration exists; `to_gitea_issue_payload` defines the payload
shape (title/body/labels/milestone) only. Creating issues against a live Gitea server is future
work, not claimed here.
- `model` is intentionally validated as an open string, not a closed enum, per
`RALPH-CONTEXT.md`'s "Quantization and placement are dynamic recipe inputs" constraint; the schema
documents (`metadataSchema.modelConvention`) but does not restrict its value set.
- Validation and generation were exercised only against this feature's `prd.json`
(`.scratch/distributed-gguf-runtime/prd.json`); `docs/prd.json` and other `.scratch/*/prd.json`
files in this repo use a materially different (simpler) shape and are out of scope.
## Dependency handoff
DGR-021 and DGR-025 (this story's derived `blocks`) may treat `prd.json`'s `metadataSchema`,
`qualityGates`, and this validator/generator as stable. Any future field addition to a story shape
must extend `STORY_FIELDS` in `scripts/ralph_prd_schema.py` and the corresponding
`metadataSchema.requiredStoryFields`/`optionalStoryFields` in `prd.json` together —
`validate_metadata_schema_consistency` fails closed if they drift apart.

View File

@@ -0,0 +1,140 @@
{
"generated_by": "scripts/glm_whole_model_preflight.py",
"target": {
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
"quantization": "UD-IQ1_S",
"shard_count": 6,
"total_bytes": 216715360960,
"total_gb": 216.715,
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962"
},
"forbidden_path_prefixes": [
"/home"
],
"mounts": [
{
"mountpoint": "/run/media/popov/DATA",
"fstype": "ext4",
"total_gb": 1208.8,
"free_gb": 74.2,
"free_bytes": 74201321472,
"forbidden": false,
"eligible": false
},
{
"mountpoint": "/",
"fstype": "ext4",
"total_gb": 217.7,
"free_gb": 51.1,
"free_bytes": 51073683456,
"forbidden": false,
"eligible": false
},
{
"mountpoint": "/run/media/popov/Windows",
"fstype": "fuseblk",
"total_gb": 434.9,
"free_gb": 26.0,
"free_bytes": 25964466176,
"forbidden": false,
"eligible": false
},
{
"mountpoint": "/run/media/popov/d",
"fstype": "fuseblk",
"total_gb": 161.1,
"free_gb": 5.1,
"free_bytes": 5148332032,
"forbidden": false,
"eligible": false
}
],
"chosen_destination": null,
"checks": [
{
"check": "storage",
"requirement": ">= 250 GB free on one filesystem outside ['/home']",
"observed": "no eligible filesystem",
"passes": false
},
{
"check": "memory",
"requirement": ">= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor)",
"observed": "124.9 GiB MemTotal",
"passes": false,
"waived": false
}
],
"download_authorized": false,
"storage_only": false,
"download_plan": [
{
"step": 1,
"shard_index": 1,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
"size_bytes": 9423744,
"size_gb": 0.009,
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\"",
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 1"
},
{
"step": 2,
"shard_index": 6,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
"size_bytes": 19171063136,
"size_gb": 19.171,
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\"",
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 6"
},
{
"step": 3,
"shard_index": 2,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
"size_bytes": 49208128256,
"size_gb": 49.208,
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\"",
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 2"
},
{
"step": 4,
"shard_index": 3,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
"size_bytes": 49684417024,
"size_gb": 49.684,
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\"",
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 3"
},
{
"step": 5,
"shard_index": 4,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
"size_bytes": 49396052864,
"size_gb": 49.396,
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\"",
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 4"
},
{
"step": 6,
"shard_index": 5,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
"size_bytes": 49246275936,
"size_gb": 49.246,
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\"",
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 5"
}
],
"verdict": "fail"
}

View File

@@ -0,0 +1,215 @@
# DGR-019 evidence — lock alpha and beta performance contracts
**Completed:** 2026-07-22
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependency:** DGR-017 (`evidence/DGR-017/README.md`) — cleaned backlog reconciled to `origin/master`; no old pass state transferred.
## Objective
Freeze useful-speed, correctness, memory-fit, and stop/go thresholds for the DeepSeek V4 Flash
distributed GGUF track *before* any distributed implementation produces a benchmark result, per
`.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`.
## Pre-existing state found (not caused by this story)
Before any change in this session, `git status` showed `.scratch/distributed-gguf-runtime/prd.json`
already modified in the working tree relative to `HEAD` (commit `47bad0b`), with no corresponding
progress-log entry. Diffing against `HEAD` showed the working copy had **dropped** prd.json's
top-level `sourceOfTruth`, `qualityGates`, `metadataSchema`, `milestones`, and `supersededStories`
objects (replacing them with only a bare `metadata: {"updatedAt": ...}` stamp), while `userStories`
itself was byte-identical to `HEAD`. Running `tests/test_ralph_prd_schema.py` against the
as-found working tree confirmed the damage: 56 of 108 tests failed (every
`test_render_issue_markdown_matches_committed_file[...]` parametrization, since
`quality_gate_bullets`/`authority_disclaimer` fall back to module defaults once `qualityGates`/
`metadataSchema` are absent, which no longer match the committed issue files).
This is the same shape of problem DGR-018's evidence documented and fixed: an abandoned,
unexplained edit that silently dropped the schema/gates/milestone/provenance content this and
future stories depend on, while `scripts/ralph_prd_schema.py validate` did not catch it (those
top-level sections are optional-if-absent by design, so the CLI reported `OK: 55 stories
validated.` even with them missing). The most likely cause is `ralph-tui`'s own read/write of
`prd.json` as its task source, which only round-trips the fields it models
(`name`/`description`/`branchName`/`userStories`) and stamps its own `metadata.updatedAt`,
dropping any project-specific extension fields it doesn't know about.
Per `RALPH-CONTEXT.md`'s instruction to inspect `git status` and preserve unrelated work rather
than build on top of unexplained state, and following the DGR-018 precedent, the dropped fields
were restored verbatim from `HEAD` (`git show HEAD:.scratch/distributed-gguf-runtime/prd.json`)
while keeping the current `userStories` content (identical) and the current `metadata.updatedAt`
stamp. `tests/test_ralph_prd_schema.py` returned to `108 passed` immediately after the restore,
before any DGR-019-specific change was made.
## Changes
### `packages/node/meshnet_node/dgr_performance/` (new package)
- **`data/alpha-beta-contract-v1.json`** — the locked, versioned, machine-readable contract.
`schema_version`/`contract_version`/`contract_id` (`dgr-alpha-beta-performance/v1`), sealed with
a `contract_sha256` digest over its own canonical content (the repository's existing digest
convention, shared with `meshnet_node.glm_alpha.contract`). Contents:
- `prompt_set` — four fixed prompts (`short-instruction`, `code-completion`,
`multi-step-reasoning`, `long-context-fill`) referenced by ID from every lane, so no lane can
quietly drift onto a different workload.
- `sampling` — greedy (`temperature=0`, `top_p=1`, `top_k=1`, `seed=1234`), matching
`meshnet_node.recipe_benchmark.SamplingPolicy` defaults.
- `lanes` — all four lanes named in the acceptance criteria. `controlled-safetensors` and
`whole-model-gguf` are marked `locked_elsewhere: true` and point at the pre-existing immutable
DGR-001 lock (`meshnet_node.performance_contract`, `contract_version=1`,
`ContractThresholds`) rather than re-defining or risking a conflicting duplicate. Only
`dense-distributed-gguf` and `v4-flash-distributed` are newly locked here, each with fixed
`prompt_ids`, `context_tokens`/`output_tokens` (alpha- and beta-scale for the V4 lane),
`concurrency_levels`, `hardware` (named certification-scenario topology, network class, device
class, MTP-off note), and a `metrics` list drawn from the existing
`recipe_benchmark`/`performance_contract`/`route_session_benchmark` metric vocabulary
(`ttft_p50_ms`, `decode_tokens_per_sec`, `seam_bytes`, `seam_latency_ms`, ...).
- `gain_attribution` — two disjoint metric sets, `quantization_model_fit_metrics` and
`runtime_transport_batching_kernel_metrics`, plus the rule that a speed/fit claim must cite
which axis moved it.
- `certification_scenarios``quantization` (`Q4_K_M`, `Q8_0`, `bf16-reference`) and
`stage_count` (`2-4-stage`, `10-plus-stage`) as named labels only, with an explicit rule that
no product/runtime code path may hardcode them.
- `alpha` — correctness thresholds (greedy token agreement, mean state cosine similarity,
nonfinite-tensor/fail-closed checks, no dense-attention-fallback credit) plus a `useful_speed`
block whose ratios (`1.25`/`0.75`-class, matching the already-locked DGR-001 25% convention)
carry an explicit `human_approval` sub-block (`required: true`, `approved: false`,
`approved_by: null`, `approved_at: null`). The ratio alone cannot satisfy alpha; DGR-054 must
fill in the approval against real evidence. `mtp.reserved=true`/`enabled_for_alpha=false` per
`RALPH-CONTEXT.md`. `verdicts: ["alpha", "optimize", "stop"]`.
- `beta` — adds exactly `concurrency`, `long_context`, `failure`, `sustained_throughput` axes
(16k-token long-context threshold matching the V4 lane's `beta_context_tokens`, no-silent-KV-
migration and no-synthetic-workers failure rules, 30-minute sustained-throughput floor).
`verdicts: ["beta", "targeted-optimization", "stop-rollback"]`.
- `amendment_policy` — thresholds may not be weakened/moved/reinterpreted after results are
known; a change requires a new `contract_id`/`contract_version` under human review.
- **`contract.py`** — loader/validator mirroring the proven
`meshnet_node.glm_alpha.contract` pattern: `parse_contract` recomputes the canonical-JSON SHA-256
over the document (excluding the digest field) and requires it match both the document's own
declared `contract_sha256` *and* a digest pinned independently in code
(`CONTRACT_V1_SHA256`), so neither an in-place edit nor a resealed mutation can pass silently.
Structural checks enforce all four required lanes, that the two referenced lanes actually
declare `locked_elsewhere`, that the two newly-locked lanes carry full benchmark-plan fields,
that `alpha.verdicts`/`beta.verdicts` are exactly the three-outcome sets the release gates use,
and — the one property with no analogue in `glm_alpha` — that
`alpha.useful_speed.human_approval.required` is `true`. `seal_contract()` is the only supported
way to produce a new digest, kept separate from load-time verification for the same reason
`glm_alpha` keeps it separate.
- **`__init__.py`** — re-exports the public API, documented as the contract DGR-020, DGR-044,
DGR-054, and DGR-070 are judged against.
### `tests/test_dgr_performance_contract.py` (new, 28 tests)
Deterministic, offline, GPU-free, model-download-free. Covers: packaged load and identity; digest
recomputation; all four lanes present; the two referenced lanes point at the real DGR-001 module
and its actual immutable thresholds (`min_decode_speedup == 1.25`, `max_resident_memory_ratio ==
0.75`); the two newly-locked lanes carry complete benchmark plans, fixed context/output/
concurrency; the shared prompt set and every lane's `prompt_ids`/`beta_prompt_ids` are a subset of
it; sampling is greedy; `gain_attribution`'s two metric sets are non-empty and disjoint;
certification-scenario names and rule text; **a structural test that greps every `.py` file under
`packages/node/meshnet_node` (excluding this contract's own module and data file) for the literal
strings `2-4-stage`/`10-plus-stage` and fails if any product module hardcodes them** — the concrete
form of "no product logic may hardcode them"; alpha verdicts/correctness/`human_approval`/MTP-off;
beta verdicts/axes/long-context/failure semantics; digest-mutation rejection (in-place and
resealed); missing-digest rejection; `load_contract` from an explicit path matches the packaged
load; `seal_contract` reproduces the pinned digest; amendment policy text.
### `.scratch/distributed-gguf-runtime/prd.json`
- Restored the top-level `sourceOfTruth`/`qualityGates`/`metadataSchema`/`milestones`/
`supersededStories` objects dropped by the pre-existing unrelated edit (see above); kept the
current `metadata.updatedAt` tooling stamp.
- Marked `DGR-019.passes = true` with `completionNotes` summarizing this outcome.
### `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`
Regenerated via `python scripts/ralph_prd_schema.py render` to reflect `passes: true` (checked
acceptance criteria, "completed" status line, "Verified evidence" handoff line), matching the
convention DGR-017/DGR-018's issue files already use.
## Commands and results
```bash
.venv-rocm/bin/python -m pytest -q tests/test_dgr_performance_contract.py
```
```text
28 passed in 0.14s
```
```bash
.venv-rocm/bin/python -m pytest -q tests/test_ralph_prd_schema.py tests/test_dgr_performance_contract.py \
tests/test_glm_alpha_target.py tests/test_recipe_benchmark.py tests/test_route_session_benchmark.py
```
```text
270 passed in 1.04s
```
```bash
.venv-rocm/bin/python -m compileall -q packages tests
```
Exit code 0, no output (all files compile).
```bash
git diff --check
```
Exit code 0 (no whitespace errors).
```bash
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
OK: 55 stories validated.
```
```bash
.venv-rocm/bin/python -m pytest -q tests/ -k "not integration" --ignore=tests/test_shard_runtime_harness.py
```
```text
7 failed, 1146 passed, 11 skipped, 4 deselected, 3 warnings in 261.71s (0:04:21)
```
This full sweep was launched in the background while `prd.json`/the evidence README below were
still being written, so it raced its own inputs: one of its 7 failures
(`test_ralph_prd_schema.py::test_real_backlog_passed_stories_have_completion_evidence`) was this
story's own `passes=true`/evidence-README edit landing mid-run, not a real defect — re-running
`tests/test_ralph_prd_schema.py` alone afterward, against the finalized tree, gives
`108 passed`. The other 6 failures (`test_billing_ledger.py::
test_tracker_enables_billing_with_default_db`, `test_dynamic_routing.py::
test_admin_can_replace_a_served_model_and_release_it`, `test_dynamic_routing.py::
test_models_list_does_not_duplicate_a_preset_registered_by_hf_repo`, three cache tests in
`test_real_model_backend.py`) are in files this story's `git diff` never touches (`git diff --stat
HEAD -- tests/test_billing_ledger.py tests/test_dynamic_routing.py tests/test_real_model_backend.py`
is empty) and none of them import `dgr_performance`, `performance_contract`, or `glm_alpha`; they
are pre-existing baseline defects, not regressions from this story, in the same spirit as the
known `origin/master` limitations DGR-017's evidence recorded.
## Known limitations
- `tests/test_shard_runtime_harness.py` fails to *collect* in this environment
(`ModuleNotFoundError: No module named 'grpc'`). This is a pre-existing environment gap from
DGR-024's real generated-gRPC protocol harness, not something this story touched or caused; it is
excluded from the sweep above rather than silently masked.
- Alpha's `useful_speed` ratios (`1.25`/`0.75`-class) are proposed thresholds held at the same
margin already locked for the whole-model contract (DGR-001/v1). They are locked numbers, but
`human_approval.required=true` means DGR-054 may not treat them as self-certifying from the
ratio alone — a human must approve the observed ratio against real evidence. This session did
not, and could not, supply that approval: no distributed benchmark evidence exists yet.
- `v4-flash-distributed`'s `reference_baseline` documents that a safetensors DeepSeek V4 Flash
distributed baseline may not yet be pinned (that is DGR-044's job); until then, comparisons must
fall back to `dense-distributed-gguf` runtime/transport overhead as an explicit, stated
limitation rather than a silent substitution.
- This is a specification-materialization story; per the shared quality gates, it is intentionally
left uncommitted for manual review rather than given the "one scoped story commit" other stories
get.
## Dependency handoff
DGR-020 (run the controlled whole-model baseline) consumes the DGR-001 lock referenced — not
redefined — by this contract's `controlled-safetensors`/`whole-model-gguf` lanes.
DGR-044 (pin the DeepSeek V4 Flash target contract) and DGR-054/DGR-070 (enforce the alpha/beta
gates) must load `meshnet_node.dgr_performance.load_contract()` and judge results against its
`dense-distributed-gguf`/`v4-flash-distributed` lanes and `alpha`/`beta` sections without changing
any threshold. DGR-054 specifically must populate `alpha.useful_speed.human_approval`
(`approved`/`approved_by`/`approved_at`) as part of publishing its verdict — a satisfied ratio
without a filled-in approval is not alpha certification. Any amendment must open a new
`contract_id`/`contract_version` under human review per `amendment_policy`; this document and its
digest are not editable in place.

View File

@@ -0,0 +1,243 @@
# DGR-020 evidence — run the controlled whole-model GGUF baseline
**Completed:** 2026-07-22
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependency:** DGR-019 (`evidence/DGR-019/README.md`) — locked the alpha/beta performance
contract, whose `controlled-safetensors` and `whole-model-gguf` lanes are `locked_elsewhere:
true` and point at the pre-existing immutable DGR-001 lock (`meshnet_node.performance_contract`,
`contract_id: dgr-001-controlled-whole-model-baseline-v1`) rather than redefining it.
## Objective
Per `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`:
execute the exact locked safetensors and whole-model llama.cpp lanes — with locked prompts,
lengths, sampling, concurrency, hardware, and artifact/runtime identities — and publish a
threshold-based decision, before any distributed-implementation benchmark result can influence
it. Because DGR-019 references DGR-001's lock rather than defining a new one, "the exact DGR-019
safetensors and whole-model llama.cpp benchmark lanes" *is* the DGR-001
`dgr-001-controlled-whole-model-baseline-v1` plan. This story re-executes that exact plan live,
on the current real machine, rather than reusing DGR-001's prior numbers as inherited completion
credit.
## Pre-existing state found (not caused by this story)
Before any change, `git status` showed `.scratch/distributed-gguf-runtime/prd.json` already
modified relative to `HEAD` (`47bad0b`). Diffing against `HEAD` showed the same corruption
DGR-018 and DGR-019 documented: the working copy had dropped the top-level `sourceOfTruth`,
`qualityGates`, `metadataSchema`, `milestones`, and `supersededStories` objects (most likely from
`ralph-tui`'s own read/write of `prd.json`, which round-trips only the fields it models). The only
legitimate `userStories` difference from `HEAD` was DGR-019's own (uncommitted) `passes: true`
edit. Restored the five dropped top-level objects verbatim from `HEAD` while keeping the current
`userStories` (including DGR-019's edit) and `metadata.updatedAt`. `tests/test_ralph_prd_schema.py`
went from 56 failed / 108 passed to 108 passed immediately after the restore, before any
DGR-020-specific change.
## Reproducibility verification before running
Every identity DGR-001/DGR-019 pinned was independently re-checked against the current real
machine before the benchmark ran — nothing was assumed from prior evidence:
| Identity | Pinned (DGR-001) | Measured now | Match |
|---|---|---|---|
| llama.cpp commit | `e920c523e3b8a0163fe498af5bf90df35ff51d25` | `e920c523e3b8a0163fe498af5bf90df35ff51d25` | yes |
| `llama-server` SHA-256 | `fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd` | same | yes |
| BF16 GGUF artifact SHA-256 | `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862` | same | yes |
| Q4_K_M GGUF artifact SHA-256 | `a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5` | same | yes |
| Torch / Transformers versions | `2.10.0+rocm7.13.0a20260513` / `5.13.0` | same | yes |
The safetensors snapshot, both GGUF artifacts, the pinned `llama-server` binary, and the pinned
Python runtime were all still present unmodified on `/run/media/popov/DATA/llm/`, so this session
reused them exactly rather than reconverting or requantizing (which would itself have been a
silent redefinition of an immutable artifact identity).
## Real results — fresh run on real hardware
`.scratch/distributed-gguf-runtime/evidence/DGR-020/benchmark-config.json` and
`performance-contract.json` are byte-identical copies of DGR-001's (same `plan_sha256`
`efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570` and `config_sha256`
`00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3`), so this is the same plan,
not a new one.
```bash
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 \
MESHNET_EVIDENCE_SIGNING_KEY=/home/popov/.config/neuron-tai/keys/dgr-001-evidence-ed25519.pem \
PYTHONPATH=packages/node .venv-rocm/bin/python -m meshnet_node.recipe_benchmark \
--config .scratch/distributed-gguf-runtime/evidence/DGR-020/benchmark-config.json \
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-020/results.json \
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-020/results.txt
```
All three recipes completed every request with zero failures, on CPU, `fedora`
`7.0.14-101.fc43.x86_64`, 32 logical CPUs:
| Metric | Transformers BF16 (ref) | llama.cpp BF16 | llama.cpp Q4_K_M | DGR-001 (prior run, same plan) |
|---|---:|---:|---:|---|
| Decode tok/s, c=1 | 50.8 | 102.5 | 213.1 | 40.8 / 98.5 / 207.7 |
| Aggregate decode tok/s, c=4 | 48.8 | 218.1 | 235.7 | 46.5 / 222.8 / 195.7 |
| TTFT p50, c=1 | 32.9 ms | 15.1 ms | 17.3 ms | 40.0 / 15.1 / 21.6 ms |
| Peak resident memory, c=1 | 1.93 GB | 1.11 GB | 0.54 GB | 1.94 / 1.11 / 0.54 GB |
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB | (identical, same artifacts) |
| Failures | 0 | 0 | 0 | 0 / 0 / 0 |
| Exact match vs reference | — | 0.3333 | 0.00 (advisory) | 0.3333 |
| Mean similarity vs reference | — | 0.9471 | 0.456 (advisory) | 0.9471 |
Per-recipe measurements against the reference (`baseline.json`, `contract-evaluation.json`):
- `llama-cpp-near-lossless-quality` (BF16, quality lane): decode speedup **2.02x**, aggregate
throughput speedup (c=4) **4.47x**, resident-memory ratio **0.574x**, TTFT ratio **0.459x**
but `quality_pass: false` (exact match 0.33 < required 0.90).
- `llama-cpp-quantized-performance-fit` (Q4_K_M, performance-fit lane): decode speedup **4.19x**,
aggregate throughput speedup (c=4) **4.83x**, resident-memory ratio **0.280x**, artifact-size
ratio **0.398x**, TTFT ratio **0.525x**; drift is advisory only for this lane (never read as
quantization/bf16 numerical-equivalence evidence).
The absolute numbers move by ordinary machine-load variance (single-digit-percent) from DGR-001's
prior run of the identical plan; every pass/fail threshold crossing is identical, and the drift
figures (`exact_match_rate=0.3333`, `mean_similarity=0.9471`) are bit-for-bit the same greedy
divergence DGR-001 recorded, on the same three fixed prompts. This is a genuine independent
reproduction, not a copy: `results.json`'s `provenance.run_id`
(`59b12968-c5d0-4391-90f4-0cd2aff77b21`), `started_at`/`completed_at` timestamps, and Ed25519
`signature` are all freshly generated by this session's run, signed with the same DGR-001 evidence
key (`signer_public_key_sha256` `8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de`,
matching the sole active entry in `../../trusted-evidence-signers.json`).
## Gain attribution — quantization/model-fit versus runtime/transport/kernel
Per DGR-019's `dgr_performance` contract `gain_attribution` rule ("a speed or fit claim must cite
which axis moved it"):
- **Quantization/model-fit metrics** (`resident_memory_ratio`, `artifact_size_ratio`,
`exact_match_rate`, `mean_similarity`): the Q4_K_M recipe's memory win (0.280x) and size win
(0.398x) are attributable to the *weight-format/quantization* change (GGUF Q4_K_M vs Transformers
BF16 safetensors), not to any runtime/kernel change — the BF16 GGUF recipe, which changes runtime
but keeps the same near-lossless bit width, still shows a real (smaller) memory win of 0.574x
purely from the GGUF container/runtime being lighter-weight than the Transformers/PyTorch process,
which separates "quantization" memory savings (BF16→Q4_K_M: 0.574x→0.280x) from "runtime/format"
memory savings (safetensors→BF16 GGUF: 1.0x→0.574x). The quality-lane failure
(`exact_match_rate=0.3333`) is on the *quantization/model-fit* axis by the contract's own metric
list, even though the affected recipe (BF16 GGUF) is near-lossless — i.e. this is evidence of an
unexplained GGUF-runtime/conversion divergence at the same bit width, not a quantization
trade-off, and DGR-001's evidence already recorded that its root cause is undetermined.
- **Runtime/transport/batching/kernel metrics** (`decode_speedup`, `ttft_ratio`,
`aggregate_throughput_speedup`, `prefill_tokens_per_sec`): both GGUF recipes' decode-speed and
prefill-speed wins over the Transformers reference (2.02x/4.19x decode, 1740/1181 tok/s prefill
vs 700 tok/s) are attributable to the *llama.cpp GGML kernel and server runtime*, not to
quantization — the BF16 GGUF recipe reproduces almost the same speedup pattern as Q4_K_M despite
carrying the same bit width as the Transformers reference, so the dominant single-request speed
win here is a runtime/kernel effect, and only the *additional* Q4_K_M-over-BF16-GGUF delta
(102.5→213.1 tok/s decode, ~2.08x) is attributable to quantization on top of that runtime effect.
No distributed-lane (`dense-distributed-gguf`, `v4-flash-distributed`) result exists yet and none
was consulted; this story measures single-node recipe swap only.
## Failed / unavailable lanes
None. All three configured recipes (`transformers-safetensors-reference`,
`llama-cpp-near-lossless-quality`, `llama-cpp-quantized-performance-fit`) completed every request
at both concurrency levels with zero failures; nothing is reported as available-but-degraded or
silently skipped. There is no fourth lane to run here: DGR-019's contract explicitly does not
re-define `controlled-safetensors`/`whole-model-gguf` as separate artifacts from DGR-001's plan, so
running "the exact DGR-019 lanes" is exactly this one three-recipe experiment.
## Decision
`contract-evaluation.json` (evaluated with the unmodified, immutable
`meshnet_node.performance_contract` v1 thresholds — `min_decode_speedup=1.25`,
`max_ttft_ratio=1.25`, `min_aggregate_throughput_speedup=1.25`, `max_resident_memory_ratio=0.75`,
`min_quality_exact_match_rate=0.90`, `min_quality_mean_similarity=0.97`, `max_failure_rate=0.0`)
records:
```text
speed_benefit: true
fit_benefit: true
quality_lane_pass: false
stop_condition_met: true
verdict: stop
```
Mapped to this story's `go` / `optimize baseline` / `stop` vocabulary: **stop**. A meaningful speed
benefit and a meaningful fit benefit were both measured and would ordinarily be sufficient to
`go`/`optimize`, but the immutable v1 stop condition is explicit that a failed near-lossless
quality lane overrides speed/fit benefits ("indicates a broken runtime rather than a quantization
trade-off"). This decision uses only the locked v1 thresholds and this session's freshly measured
metrics; no threshold was changed, and no distributed-implementation result (DGR-024's gRPC
harness or any other distributed-lane evidence) was read or ingested to produce it.
This reproduces DGR-001's original `stop` verdict on the same plan on the same real machine,
confirming that verdict is stable over time and not an artifact of a single run.
## Limitations
- This is a **0.5B CPU baseline** (`Qwen/Qwen2.5-0.5B-Instruct`), the same generic model DGR-001
and DGR-019's `locked_elsewhere` reference use — not DeepSeek V4 Flash. DGR-019's evidence
already recorded that a DeepSeek V4 Flash `controlled-safetensors`/`whole-model-gguf` baseline is
not yet pinned; that is separate future work (see DGR-019's `v4-flash-distributed.reference_
baseline` note), not something this story's acceptance criteria ask it to create — it asks only
to run the exact already-locked lanes, which are this DGR-001 plan.
- The `whole-model-gguf` quality-lane exact-match divergence (0.33 vs 0.90 required) reproduces
identically and remains unexplained; this story does not diagnose it further beyond confirming
it reproduces (DGR-001's `quality-parity-diagnosis.md` documents the CPU-vs-ROCm split already
known).
- Absolute timings are single-developer-machine measurements with ordinary run-to-run variance;
the locked ratios/ratios-vs-threshold crossings are the durable evidence, not the raw absolute
tok/s figures.
- No new GPU (ROCm) diagnostic was re-run in this session — DGR-001's existing GPU diagnostic is
cited as prior evidence only; it uses a distinct signed `run_configured_gpu_diagnostic/v1`
producer that the v1 evaluator does not accept, so it cannot itself change the `stop` verdict
above.
## Files changed
- `.scratch/distributed-gguf-runtime/evidence/DGR-020/benchmark-config.json` (new) — byte-identical
copy of DGR-001's locked plan.
- `.scratch/distributed-gguf-runtime/evidence/DGR-020/performance-contract.json` (new) —
byte-identical copy of DGR-001's immutable v1 thresholds.
- `.scratch/distributed-gguf-runtime/evidence/DGR-020/results.json` / `results.txt` (new) — raw
signed real evidence from this session's fresh run.
- `.scratch/distributed-gguf-runtime/evidence/DGR-020/baseline.json` / `contract-evaluation.json`
(new) — distilled baseline and fail-closed v1 verdict for this session's run.
- `.scratch/distributed-gguf-runtime/evidence/DGR-020/README.md` (new, this file).
- `.scratch/distributed-gguf-runtime/prd.json` — restored the dropped top-level
`sourceOfTruth`/`qualityGates`/`metadataSchema`/`milestones`/`supersededStories` objects (see
above); marked `DGR-020.passes = true` with `completionNotes`.
- `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`
regenerated via `python scripts/ralph_prd_schema.py render` to reflect `passes: true`.
No source or test files under `packages/` or `tests/` were changed by this story.
## Commands and results
```bash
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
OK: 55 stories validated.
```
```bash
.venv-rocm/bin/python -m pytest -q tests/test_recipe_benchmark.py tests/test_dgr_performance_contract.py tests/test_ralph_prd_schema.py
```
```text
164 passed in 0.69s
```
```bash
.venv-rocm/bin/python -m compileall -q packages tests
```
Exit code 0, no output (all files compile).
```bash
git diff --check
```
Exit code 0 (no whitespace errors).
## Dependency handoff
DGR-054 (enforce the alpha gate) may cite this evidence when it fills in
`alpha.useful_speed.human_approval` — this is fresh, independently-collected, signed real-hardware
evidence that the `controlled-safetensors`/`whole-model-gguf` v1 contract still holds `stop` on the
current machine, immediately before any distributed-lane result exists, but it is a 0.5B CPU
baseline, not the DeepSeek V4 Flash target; DGR-044 must still pin the V4 Flash reference baseline
separately before DGR-054/DGR-070 can judge `dense-distributed-gguf`/`v4-flash-distributed` against
it. No threshold in either `meshnet_node.performance_contract` or `meshnet_node.dgr_performance`
was changed by this story.

View File

@@ -0,0 +1,169 @@
{
"artifact_sha256": {
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6"
},
"backend_detail": {
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16"
},
"evidence_class": "local-real",
"host": {
"accelerator_name": "Radeon 8060S Graphics",
"accelerator_runtime": "7.13.26183",
"benchmark_lane": "cpu-controlled-baseline",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"cpu_count": 32,
"cuda_available": true,
"hostname": "fedora",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_identities": {
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
}
},
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"platform": "Linux-7.0.14-101.fc43.x86_64-x86_64-with-glibc2.42",
"python": "3.12.13",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0"
},
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"provenance": {
"completed_at": "2026-07-22T05:52:30.445799Z",
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"producer": "meshnet_node.recipe_drivers.run_configured_benchmark/v1",
"run_id": "59b12968-c5d0-4391-90f4-0cd2aff77b21",
"schema_version": 1,
"signature": "aExtG1Y0fWFaqlKEtUOOpXZrganVAxbLvpov2WVgm19eNJ50VheeI7CuRhlWx4SJX9OFto2WuLaVPhjwSA88Cw==",
"signature_algorithm": "ed25519",
"signer_public_key_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
"started_at": "2026-07-22T05:51:36.511891Z"
},
"recipe_runtime": {
"llama-cpp-near-lossless-quality": {
"device": "cpu",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16"
},
"llama-cpp-quantized-performance-fit": {
"device": "cpu",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M"
},
"transformers-safetensors-reference": {
"device": "cpu",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16"
}
},
"recipes": {
"llama-cpp-near-lossless-quality": {
"artifact_bytes": 994156448,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 89.2873,
"decode_tokens_per_sec": 102.5344,
"failures": 0,
"latency_p50_ms": 316.647,
"latency_p95_ms": 374.8515,
"peak_rss_bytes": 1110106112,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1740.0213,
"ttft_p50_ms": 15.067,
"ttft_p95_ms": 65.191
},
"4": {
"aggregate_decode_tokens_per_sec": 218.1128,
"decode_tokens_per_sec": 80.0623,
"failures": 0,
"latency_p50_ms": 403.9781,
"latency_p95_ms": 767.6557,
"peak_rss_bytes": 1139265536,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1064.6179,
"ttft_p50_ms": 36.611,
"ttft_p95_ms": 178.801
}
},
"device": "cpu",
"lane": "quality"
},
"llama-cpp-quantized-performance-fit": {
"artifact_bytes": 397807520,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 149.8675,
"decode_tokens_per_sec": 213.1452,
"failures": 0,
"latency_p50_ms": 161.7164,
"latency_p95_ms": 282.5491,
"peak_rss_bytes": 541663232,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1181.0842,
"ttft_p50_ms": 17.252,
"ttft_p95_ms": 130.529
},
"4": {
"aggregate_decode_tokens_per_sec": 235.6963,
"decode_tokens_per_sec": 94.7604,
"failures": 0,
"latency_p50_ms": 373.7211,
"latency_p95_ms": 759.3151,
"peak_rss_bytes": 571027456,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 567.7335,
"ttft_p50_ms": 42.086,
"ttft_p95_ms": 312.645
}
},
"device": "cpu",
"lane": "performance-fit"
},
"transformers-safetensors-reference": {
"artifact_bytes": 999586347,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 44.4625,
"decode_tokens_per_sec": 50.8327,
"failures": 0,
"latency_p50_ms": 701.9146,
"latency_p95_ms": 776.2706,
"peak_rss_bytes": 1933221888,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 699.7553,
"ttft_p50_ms": 32.8569,
"ttft_p95_ms": 173.7161
},
"4": {
"aggregate_decode_tokens_per_sec": 48.849,
"decode_tokens_per_sec": 13.4779,
"failures": 0,
"latency_p50_ms": 2503.1601,
"latency_p95_ms": 2600.6307,
"peak_rss_bytes": 2170908672,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 264.5822,
"ttft_p50_ms": 95.7502,
"ttft_p95_ms": 425.4973
}
},
"device": "cpu",
"lane": "quality"
}
},
"reference_recipe_id": "transformers-safetensors-reference"
}

View File

@@ -0,0 +1,118 @@
{
"artifact_storage_root": "/run/media/popov/DATA/llm",
"evidence_class": "local-real",
"host": {
"benchmark_lane": "cpu-controlled-baseline",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"transformers_version": "5.13.0"
},
"plan": {
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"prompts": [
{
"id": "short-fact",
"text": "The capital of France is",
"context_class": "short"
},
{
"id": "medium-code",
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
"context_class": "medium"
},
{
"id": "long-summary",
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
"context_class": "long"
}
],
"sampling": {
"temperature": 0.0,
"top_p": 1.0,
"top_k": 1,
"seed": 1234,
"max_output_tokens": 32
},
"concurrency_levels": [1, 4],
"repeats": 3,
"warmup_requests": 2
},
"recipes": [
{
"id": "transformers-safetensors-reference",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": true,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cpu",
"dtype": "bfloat16",
"threads": 16
}
},
{
"id": "llama-cpp-near-lossless-quality",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
},
{
"id": "llama-cpp-quantized-performance-fit",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"lane": "performance-fit",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
}
]
}

View File

@@ -0,0 +1,71 @@
{
"contract_version": 1,
"fit_benefit": true,
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"quality_lane_pass": false,
"rationale": [
"the near-lossless quality lane failed: the GGUF runtime disagrees with the safetensors reference beyond what near-lossless weights can explain",
"a meaningful speed benefit was measured",
"a meaningful fit benefit was measured"
],
"recipes": [
{
"comparable": true,
"failures": 0,
"fit_benefit": false,
"incomparable_reason": "",
"lane": "quality",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.465,
"artifact_size_ratio": 0.9946,
"artifact_size_win": false,
"compared_prompts": 3,
"decode_speedup": 2.0171,
"exact_match_rate": 0.3333,
"expected_prompts": 3,
"failure_rate": 0.0,
"mean_similarity": 0.9471,
"resident_memory_ratio": 0.5742,
"ttft_ratio": 0.4586
},
"quality_pass": false,
"reasons": [
"single-request decode 2.02x reference (>= 1.25x) at TTFT ratio 0.46",
"aggregate throughput at concurrency 4 is 4.46x reference (>= 1.25x)",
"peak resident memory is 0.57x reference (<= 0.75x)",
"quality lane exact-match 0.33 / similarity 0.947 versus the reference (fail)"
],
"recipe_id": "llama-cpp-near-lossless-quality",
"speed_benefit": false
},
{
"comparable": true,
"failures": 0,
"fit_benefit": true,
"incomparable_reason": "",
"lane": "performance-fit",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.825,
"artifact_size_ratio": 0.398,
"artifact_size_win": true,
"decode_speedup": 4.1931,
"failure_rate": 0.0,
"resident_memory_ratio": 0.2802,
"ttft_ratio": 0.5251
},
"quality_pass": null,
"reasons": [
"single-request decode 4.19x reference (>= 1.25x) at TTFT ratio 0.53",
"aggregate throughput at concurrency 4 is 4.83x reference (>= 1.25x)",
"peak resident memory is 0.28x reference (<= 0.75x)"
],
"recipe_id": "llama-cpp-quantized-performance-fit",
"speed_benefit": true
}
],
"speed_benefit": true,
"stop_condition_met": true,
"verdict": "stop"
}

View File

@@ -0,0 +1,87 @@
{
"schema_version": 1,
"contract_version": 1,
"locked_at": "2026-07-13T00:00:00Z",
"locked_by": "DGR-001",
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"thresholds": {
"min_decode_speedup": 1.25,
"max_ttft_ratio": 1.25,
"min_aggregate_throughput_speedup": 1.25,
"max_resident_memory_ratio": 0.75,
"max_artifact_size_ratio": 0.6,
"min_quality_exact_match_rate": 0.9,
"min_quality_mean_similarity": 0.97,
"max_failure_rate": 0.0
},
"baseline": {
"status": "pending-real-evidence",
"required_evidence_class": "local-real",
"required_recipes": [
"transformers-safetensors-reference",
"llama-cpp-near-lossless-quality",
"llama-cpp-quantized-performance-fit"
],
"required_concurrency_levels": [
1,
4
],
"required_controlled_variables": [
"model architecture",
"model revision",
"machine and device",
"formatted prompts and context lengths",
"output length and greedy sampling policy"
],
"required_plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"minimum_prompt_count": 3,
"minimum_repeats": 3,
"minimum_output_tokens": 32,
"required_device": "cpu",
"required_config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"required_signer_public_key": "zQ/qRMwF/ydazzaxEI24Xvnrl5bZxzw16JYpP0bfRuI=",
"required_artifact_sha256": {
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5"
},
"required_recipe_runtime": {
"transformers-safetensors-reference": {
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"device": "cpu"
},
"llama-cpp-near-lossless-quality": {
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"device": "cpu"
},
"llama-cpp-quantized-performance-fit": {
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"device": "cpu"
}
},
"required_backend_detail": {
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16",
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0"
},
"required_host_identity": {
"python": "3.12.13",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0",
"llama_server_identities": {
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
}
}
}
},
"stop_condition": "Stop the native llama.cpp/GGUF track when, on the same machine and device as the Transformers/safetensors reference and under this plan, no performance-fit GGUF recipe delivers either a meaningful speed benefit (>=25% higher single-request decode tokens/sec without a >25% worse TTFT, or >=25% higher aggregate throughput under concurrency) or a meaningful fit benefit (>=25% lower peak resident memory), or when the near-lossless quality lane fails, which indicates a broken runtime rather than a quantization trade-off.",
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract. Non-synthetic evidence must be Ed25519-signed by the pinned key and match the exact locked config, artifacts, runtimes, backends, and host runtime identity."
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,10 @@
Recipe benchmark dgr-001-controlled-whole-model-baseline-v1 (local-real)
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 32.9/ 173.7 ms; prefill 699.8 tok/s; decode 50.8 tok/s; aggregate 44.5 tok/s; rss 1.93 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 95.8/ 425.5 ms; prefill 264.6 tok/s; decode 13.5 tok/s; aggregate 48.8 tok/s; rss 2.17 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 15.1/ 65.2 ms; prefill 1740.0 tok/s; decode 102.5 tok/s; aggregate 89.3 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 36.6/ 178.8 ms; prefill 1064.6 tok/s; decode 80.1 tok/s; aggregate 218.1 tok/s; rss 1.14 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 17.3/ 130.5 ms; prefill 1181.1 tok/s; decode 213.1 tok/s; aggregate 149.9 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 42.1/ 312.6 ms; prefill 567.7 tok/s; decode 94.8 tok/s; aggregate 235.7 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
drift llama-cpp-near-lossless-quality vs transformers-safetensors-reference exact 0.33; similarity 0.947 (gated)
drift llama-cpp-quantized-performance-fit vs transformers-safetensors-reference exact 0.00; similarity 0.456 (advisory)

View File

@@ -0,0 +1,101 @@
# DGR-021 evidence — versioned named-tensor activation envelope
**Completed:** 2026-07-17
**Branch:** `distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependency:** DGR-018 (`evidence/DGR-018/README.md`) — canonical backlog schema / issue projection contract
## Objective
Establish the backend-neutral activation envelope used by direct and relayed Shard traffic, with stable versioning, named tensors, bounded fragmentation, checksum validation, and reserved extensibility for future state.
## Changes
### `packages/node/meshnet_node/protocol.py` (new)
Added a self-contained activation-envelope module with:
- `SCHEMA_NAME = "meshnet.activation-stream"` and `SCHEMA_VERSION = 1`
- `TensorFragment`
- bounded byte fragments with offset, compression tag, checksum, and extension preservation
- deterministic `to_dict()` / `from_dict()` round-trip
- `NamedTensor`
- named tensor metadata: `name`, `shape`, `dtype`, `byte_order`, `compression`, `checksum`, `fragments`
- fragmentation via `from_bytes(..., max_fragment_bytes=...)`
- checksum validation over reconstructed tensor bytes
- unknown-field preservation via `extensions`
- `ActivationEnvelope`
- top-level fields for `request_id`, `work_id`, `route_session`, `route_epoch`, `shard_start`, `effective_start`, `phase`, `position`, and `idempotency_step`
- reserved extension fields for `token_id_sideband`, `architecture_state`, `recurrent_state`, and `mtp`
- deterministic canonical serialization (`to_bytes`) and round-trip parsing (`from_bytes`)
- size-limit enforcement (`to_bytes(max_bytes=...)`)
- conversion from a live `TensorPayload` into the envelope and back again
### `packages/node/meshnet_node/model_backend.py`
Extended `TensorPayload` with envelope conversion helpers:
- `TensorPayload.to_envelope(...)`
- `TensorPayload.from_envelope(...)`
These keep the existing activation payload interface intact while exposing the new versioned envelope as the shared protocol layer.
### `tests/test_activation_envelope.py` (new)
Added focused deterministic tests covering:
- deterministic envelope serialization and round-trip parsing
- tensor fragmentation and checksum validation
- unknown-field preservation at both envelope and tensor levels
- size-limit rejection
- `TensorPayload` ↔ envelope round-trip
### `.scratch/distributed-gguf-runtime/prd.json`
Marked `DGR-021.passes = true` and added completion notes recording the envelope implementation and verification commands.
## Commands and results
```bash
pytest -q tests/test_activation_envelope.py
```
```text
5 passed in 0.06s
```
```bash
pytest -q tests/test_activation_envelope.py tests/test_kv_cache_distributed.py -k 'session_is_stable_and_decode_payloads_are_single_token or large_prefill_activation_survives_zstd_compressed_hop'
```
```text
.. [100%]
2 passed, 21 deselected in 1.84s
```
```bash
python3 -m compileall packages/node/meshnet_node tests/test_activation_envelope.py
```
```text
Listing 'packages/node/meshnet_node'...
Listing 'packages/node/meshnet_node/native_protocol'...
Compiling 'tests/test_activation_envelope.py'...
```
```bash
git diff --check
```
```text
No whitespace errors
```
## Limitations
- The envelope is implemented as a canonical deterministic JSON contract with dataclasses and conversion hooks, not generated `.proto` classes. The environment had `protobuf` available but not the `grpc_tools` generation toolchain, so I did not materialize a compiled proto artifact here.
- The direct/relayed HTTP/WebSocket transports remain byte-oriented; the envelope is the shared structured contract layered above those transports.
## Dependency handoff
DGR-022 and later shard-control stories can reuse the envelope contract and its `TensorPayload` conversion hooks as the stable activation metadata layer. Future work that requires generated protobuf code can replace the JSON serialization with a generated wire codec without changing the top-level field contract defined here.

View File

@@ -0,0 +1,46 @@
# DGR-022 evidence — Shard lifecycle and structured status RPC contract
**Completed:** 2026-07-17
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
## Outcome
Implemented the versioned, backend-neutral lifecycle/status contract consumed by a future generated gRPC binding. The contract keeps Meshnet routing, identity, authentication policy, billing, and llama.cpp ownership outside the worker contract.
## Implemented
- `packages/node/meshnet_node/shard_lifecycle.py`
- capability, health, session, cancellation, release, and metrics RPC names
- schema version negotiation and fail-closed unsupported-version handling
- structured status/error taxonomy with retryability and details
- lifecycle state machine for prefill/decode/cancel/release transitions
- monotonic idempotency-step enforcement and duplicate rejection
- bounded frame/byte flow control with cancellation-aware waits
- explicit cache expectation/result types
- deadline policy and TLS/auth transport hooks
- deterministic contract serialization round-trip
- `tests/test_shard_lifecycle.py`
- contract round-trip and RPC coverage
- unsupported-version rejection
- malformed transition and idempotency rejection
- cancellation/release behavior
- bounded flow-control behavior
- TLS hook and incomplete-contract fail-closed behavior
## Verification
```text
$ PYTHONPATH=packages/node pytest -q tests/test_shard_lifecycle.py tests/test_activation_envelope.py
17 passed in 0.10s
```
The existing DGR-021 activation-envelope tests remain green alongside DGR-022.
## Scope limitation
This story defines the lifecycle/status contract only. Generated Python/C++ protobuf bindings and the concrete `shard_runtime.proto` generation pipeline are DGR-023 and remain separate.
## Dependency handoff
DGR-023 may consume the RPC names, status taxonomy, version identity, deadlines, flow-control limits, and TLS/auth hooks when the canonical `.proto` schema and toolchain are provisioned.

View File

@@ -0,0 +1,126 @@
# DGR-023 evidence — reproducible Python and C++ protobuf/gRPC generation
**Status:** complete after controller verification and independent-review repairs on 2026-07-17.
**Authority:** live Gitea issue #7. The local PRD is a secondary projection.
## Implemented contract
- Python generation requires exactly `grpcio-tools==1.82.1`; the generator checks installed distribution metadata and rejects missing or different versions with an actionable exact install command.
- The C++ bootstrap builds one ignored toolchain prefix from exact inputs:
- Protobuf release `33.1` (`protobuf-config` version `33.1.0`);
- Abseil release `20250814.1`;
- gRPC C++ `1.82.1` at commit `acccf84c0df20487d64101f528e5d426541ca4e5`;
- gRPC's exact-commit submodules for c-ares, RE2, OpenSSL, and zlib.
- Protobuf is configured with local dependencies only after the exact Abseil build. gRPC uses the installed Protobuf/Abseil packages and commit-pinned module dependencies, avoiding unpinned system development packages and download fallbacks.
- CMake requires exact Protobuf `33.1.0` and gRPC `1.82.1`, requires the exported `gRPC::grpc_cpp_plugin` target, and always generates/builds both message and service stubs in the ignored build tree.
- Python bindings remain committed package output; `--check` regenerates into a temporary directory and compares output. C++ bindings are never committed.
- The C++ conformance test parses Python-produced vectors, validates fields/CRC32C, and emits `cpp_roundtrip.binpb`; Python compares that artifact byte-for-byte.
## Defects found and fixed
1. A relative bootstrap prefix was resolved after entering the temporary source directory, so successful output was deleted by cleanup. The script now canonicalizes the caller-relative destination first. The regression executes `--print-prefix` from a temporary working directory and validates the resulting path behavior.
2. The original native path omitted gRPC C++ and accepted any discoverable plugin. The bootstrap now builds exact gRPC/plugin sources, and CMake rejects absent/incompatible versions.
3. The Python script named the `grpcio-tools` pin but did not validate the installed distribution. It now refuses mismatched versions.
4. Protobuf ignored a stale provider option and attempted to download a different Abseil. The build was stopped; exact Abseil is now built first and Protobuf uses `LOCAL_DEPENDENCIES_ONLY`.
5. The host lacked OpenSSL development headers. Rather than add a floating system dependency, gRPC now uses the submodule pinned by its exact commit.
6. Documentation uses `bash scripts/bootstrap_native_toolchain.sh ...`, so a normal checkout does not depend on executable-mode preservation.
## Verified toolchain
```text
cmake version 4.4.0
c++ (GCC) 15.2.1 20260123 (Red Hat 15.2.1-7)
libprotoc 33.1
protobuf CMake package 33.1.0
grpcio-tools 1.82.1
grpcio 1.82.1
protobuf Python runtime 7.35.1
gRPC C++ 1.82.1
commit acccf84c0df20487d64101f528e5d426541ca4e5
grpc_cpp_plugin sha256 995ca8ac620fe83532b649a7c8c0a9341c7003da927fe0e4a8f821bfc579206d
```
The native toolchain and generated/build artifacts live under ignored mounted-drive `build/` paths; model/build artifacts were not stored under `/home`.
## Commands and results
```bash
bash scripts/bootstrap_native_toolchain.sh build/native-toolchain
```
```text
passed from a clean build directory
libprotoc 33.1
gRPC 1.82.1 commit acccf84c0df20487d64101f528e5d426541ca4e5
grpc_cpp_plugin sha256 995ca8ac620fe83532b649a7c8c0a9341c7003da927fe0e4a8f821bfc579206d
```
```bash
cmake -S packages/node/native -B build/native \
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain"
cmake --build build/native -j"$(nproc)"
test -f build/native/shard_runtime.grpc.pb.cc
test -f build/native/shard_runtime.grpc.pb.h
test -f build/native/libshard_runtime_grpc.a
ctest --test-dir build/native --output-on-failure
```
```text
Pinned gRPC 1.82.1: building ShardRuntime service stubs
shard_runtime_proto built
shard_runtime_grpc built
1/1 shard_protocol_conformance passed
```
```bash
python3 -m pytest -q tests/test_native_shard_protocol.py
```
```text
50 passed, 2 optional-path skips
```
All DGR-023-required checks were selected explicitly:
```bash
python3 -m pytest -q -rs tests/test_native_shard_protocol.py \
-k 'cpp_and_python_agree_byte_for_byte or generated_python_stubs_match_the_proto or native_toolchain_bootstrap or wrong_grpcio'
```
```text
4 passed, 48 deselected
```
```bash
python3 scripts/generate_native_protocol.py --check
python3 scripts/generate_protocol_goldens.py --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
python3 -m compileall -q packages tests
git diff --check
```
```text
generated stubs are up to date
conformance vectors are up to date
OK: 55 stories validated
compileall passed
git diff --check passed
```
## Changed files
- `scripts/bootstrap_native_toolchain.sh`
- `scripts/generate_native_protocol.py`
- `packages/node/native/CMakeLists.txt`
- `packages/node/native/README.md`
- `tests/test_native_shard_protocol.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-023/README.md`
- `.scratch/distributed-gguf-runtime/prd.json` (secondary completion projection only)
## Limitations and dependency handoff
- This story proves exact schema/message/service generation and cross-language conformance. It does not implement or run the standalone worker service itself; DGR-033/DGR-037 own worker behavior.
- The plugin SHA is evidence for this verified build. Reproducibility authority is the exact gRPC commit plus its submodule graph, not an assumption that different compilers produce byte-identical executables.
- No model, GPU, API credits, or model download was used.
- DGR-024 and DGR-037 may consume this completed generation dependency but must provide their own transport/worker evidence.

View File

@@ -0,0 +1,180 @@
# DGR-024 evidence — real generated-gRPC protocol harness
**Status:** independently re-verified in a fresh worktree/environment (this session); `prd.json` `DGR-024.passes` is now `true`.
**Authority:** live Gitea #8 (revised); the local PRD is a secondary projection.
## Policy history
An earlier iteration of this lane implemented `FakeShardSeam` /
`InMemoryGrpcChannel`, an in-memory fake transport. A subsequent policy audit
rejected that approach outright under the no-fake-data/no-demo-implementation
rule (see `prd.json`, `DGR-024.notes`): "the former in-memory fake/stub seam
task was invalid... Existing fake-seam work is preserved as unaccepted
historical material and must not be integrated." That code
(`fake_shard_seam.py`, `test_fake_shard_seam.py`) is **not present** in this
worktree and must not be resurrected. This document supersedes any earlier
evidence describing it.
## Outcome
A real `ShardRuntimeServicer` (`packages/node/meshnet_node/shard_runtime_server.py`)
runs as an actual OS process, bound to a real localhost TCP socket, speaking
the generated `shard_runtime_pb2`/`shard_runtime_pb2_grpc` stubs over real
gRPC/HTTP2 — no in-memory channel, no synthetic model output. A test harness
(`tests/test_shard_runtime_harness.py`) spawns that process with
`subprocess.Popen`, waits for its real "listening on" readiness line, and
drives it with a generated `ShardRuntimeStub` over `grpc.insecure_channel`.
## Implemented
- `GetCapability` / `Health` unary RPCs over the real socket.
- `Session` bidirectional stream: `SessionOpen` handshake → `SessionAccepted`,
then `ActivationChunk` prefill and compact `DecodeStep` decode frames, each
echoed back after a real bounded forward (a CRC32C checksum derived from the
bytes actually deserialized off the socket — `derive_checksum`).
- **Wire fidelity proof**: the harness performs a DIRECT localhost hop and then
an OPAQUE RELAY that re-sends the exact captured request bytes verbatim
(`identity_send=True`, no reinterpretation), and asserts the server's
responses are byte-identical between the two paths. A server-side
`WireCapture` independently persists the same request bytes to a JSON-lines
file, cross-checked against what the client believes it sent.
- **Fail-closed negative paths** (`ShardRuntimeServicer.Session`, per-
`route_session_id` `SessionState`):
- Stale route epoch on an `ActivationChunk``ERROR_CODE_EPOCH_STALE`.
- Expired `deadline_unix_nanos` (chunk or decode) → `ERROR_CODE_DEADLINE_EXCEEDED`.
- Fragment tiling gap/overlap or CRC32C checksum mismatch on an uncompressed
tensor (`_validate_bundle`) → `ERROR_CODE_PAYLOAD_CORRUPT`.
- Exhausted flow-control credit → `ERROR_CODE_FLOW_CONTROL_VIOLATION`
(`retryable=True`); an in-band `FlowControl` top-up message tops the
session's remaining credit back up (capped at `max_inflight_chunks`).
- Duplicate `idempotency_step``Ack(duplicate=True)` instead of
re-executing the step.
- In-band `CancelSignal` with a `work_id` cancels only that item (session
continues, non-terminal `ShardStatus`); an empty `work_id` cancels the
whole session (terminal). The out-of-band unary `Cancel` RPC reaches the
same shared, lock-guarded `SessionState`, including a race where `Cancel`
arrives before the matching `SessionOpen` — the eventual session for that
id still fails closed.
- `Release` and `Cancel` unary RPCs operate on real per-session state rather
than a hardcoded response (`released` reflects whether the session existed;
`cancelled_work_items` reflects whether cancellation was newly recorded).
## Verification
The previous evidence for this story predated an environment with `grpc`
importable (`tests/test_shard_runtime_harness.py` could not even *collect* on
the ambient interpreter — see `.ralph-tui/progress.md`'s DGR-019 entry). This
session built a real, disposable `uv`-managed `.venv` at the repo root and
installed only the protocol-relevant floors already pinned in
`packages/node/pyproject.toml` (`grpcio==1.82.1`, `grpcio-tools==1.82.1`,
`protobuf==7.35.1`) plus `pytest==9.1.1`, then reran the full harness for
real — this is not a re-statement of the earlier claim, it is an independent
execution:
```bash
uv pip install grpcio grpcio-tools==1.82.1 protobuf pytest
PYTHONPATH=packages/node:packages/tracker .venv/bin/python -m pytest -q tests/test_shard_runtime_harness.py -v -s
```
```text
collected 11 items
tests/test_shard_runtime_harness.py .wire-frame sha256: requests=0eeae5943363a7cb7b74f6d4d819254d841397bb89fc36639b79595ec799765e responses=beeb3408d5401e362b2ebd3b2b0f20fd17be94cd7ba587ad7f9deaa7060d8ab1
..........
11 passed in 3.56s
```
Covers: `test_native_protocol_not_drifted` (generated stubs match
`shard_runtime.proto` exactly — reran `scripts/generate_native_protocol.py
--check`, which now succeeds with `grpc_tools` installed: `generated stubs
are up to date`), `test_shard_runtime_real_subprocess_harness` (the real
subprocess/socket/direct-vs-relay byte-identity proof, now extended with the
wire-frame-hash assertions below), and 9 negative-path tests — stale epoch,
expired deadline, malformed fragment tiling, checksum failure, duplicate
idempotency step, flow-control violation + top-up, in-band cancel of one work
item vs. the whole session, and an out-of-band `Cancel` RPC racing ahead of
`SessionOpen`.
### Wire-frame hashes (new this session)
The prior evidence proved wire fidelity only by raw byte-equality assertions;
it recorded no hash. `WireCapture.to_dict()`
(`packages/node/meshnet_node/shard_runtime_server.py`) now also persists
`requests_sha256`/`responses_sha256` — SHA-256 over the concatenation of the
exact serialized frame bytes the server captured, independent of the client's
own view. `tests/test_shard_runtime_harness.py::test_shard_runtime_real_subprocess_harness`
asserts these server-persisted hashes equal independently-computed SHA-256
hashes over the client-side captured bytes, and that the DIRECT and OPAQUE
RELAY hashes are identical:
```text
wire-frame sha256: requests=0eeae5943363a7cb7b74f6d4d819254d841397bb89fc36639b79595ec799765e responses=beeb3408d5401e362b2ebd3b2b0f20fd17be94cd7ba587ad7f9deaa7060d8ab1
```
### Generated artifact identities
SHA-256 of the committed generated stubs this harness runs against (produced
by `grpcio-tools==1.82.1` from `packages/node/native/proto/shard_runtime.proto`;
confirmed not-drifted by `test_native_protocol_not_drifted` above):
```text
759026b11bbd659f2caed713044a0584809c44bee733359e80a197635cd0c362 packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.py
f16326da96991c2e9212c6ca7f113037a194d601533edfbff13a583dfafa1fc8 packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.pyi
2f96f9ecac7f7358ce64a330a573f6da8d531b5a56b0e2b1c527c9ba759e5dbe packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2_grpc.py
```
```bash
.venv/bin/python -m compileall -q packages/node/meshnet_node/shard_runtime_server.py tests/test_shard_runtime_harness.py
.venv/bin/python -m compileall -q packages tests
git diff --check
```
```text
compileall (targeted): exit 0
compileall (packages tests, universal gate wording): exit 0
git diff --check: exit 0
```
Also re-ran `tests/test_ralph_prd_schema.py` (108 passed) after restoring
`prd.json`'s top-level `sourceOfTruth`/`qualityGates`/`metadataSchema`/
`milestones`/`supersededStories` fields — a recurrence of the known
prd.json-field-drop bug (see `.ralph-tui/progress.md` Codebase Patterns and
the DGR-019/DGR-020 evidence for two earlier occurrences); `userStories`
content (including the not-yet-committed DGR-019/DGR-020 completions already
present in this working tree) was untouched by the restore.
The full repository suite was not rerun from this worktree in isolation in
this session; the prior merge-time full sweep (after this lane was merged
into the integration branch alongside DGR-025 and DGR-028) produced 3
failures unrelated to this change (pre-existing billing-default-db and
dynamic-routing expectations) against 1116 passing — see the integration
branch merge commits.
## Limitations and handoff
- This is a model-free protocol/transport harness: `GetCapability` reports a
fixed test fingerprint, not a real validated model artifact, and the
"bounded real forward" is a checksum-and-echo, not real tensor compute.
- Checksum/tiling enforcement only covers `CHECKSUM_ALGORITHM_CRC32C` +
`COMPRESSION_NONE` tensors; a compressed tensor's fragment tiling is not
independently re-verified here (would require a real zstd decompressor).
- Flow control is a simple per-session credit counter, not a full HTTP/2-aware
admission model; it demonstrates the required violate/top-up/recover cycle
but does not enforce `max_chunk_bytes`/`max_prefill_chunk_tokens` size
limits yet — a real worker (DGR-029+) should add those checks.
- `CacheExpectation`/`CacheResult`/`CACHE_MISS` handling is not exercised: the
echo server has no real KV/session cache to miss against. A real worker
implementation owns that.
- Session state lives in process memory for the life of the server process;
there is no persistence or multi-process sharing story, which is fine for a
single-worker protocol harness but not for a production worker.
## Changed files
- `packages/node/meshnet_node/shard_runtime_server.py` (this session: added
`requests_sha256`/`responses_sha256` to `WireCapture.to_dict()`)
- `tests/test_shard_runtime_harness.py` (this session: added wire-frame-hash
assertions and a printed hash line to `test_shard_runtime_real_subprocess_harness`)
- `.scratch/distributed-gguf-runtime/evidence/DGR-024/README.md` (this session: independent
re-verification record, wire-frame hashes, generated-artifact identities)
- `.scratch/distributed-gguf-runtime/prd.json` (this session: restored
dropped top-level fields; `DGR-024.passes` flipped to `true`)

View File

@@ -0,0 +1,447 @@
# DGR-025 evidence — exact artifact and runtime recipe identity
**Status:** in progress — controller gates pass; final independent P0/P1 re-review is pending.
**Branch:** fixed detached Claude Fable provider lane
**Authority:** live Gitea #9; the local PRD is a secondary projection.
**Dependencies:** DGR-018 (`evidence/DGR-018/README.md` — canonical backlog schema and
issue projection), DGR-021 (`evidence/DGR-021/README.md` — versioned activation
envelope). Both read before changing code.
## Objective
Ensure the tracker and worker only combine numerically and operationally
compatible shards: fingerprint every axis that moves the numbers, bind shards to
exact half-open ranges, fail closed on any mismatch, and keep uncertified
recipes registered-but-dark.
## What was found live (verified, not inherited)
Per RALPH-CONTEXT, legacy pass states were not trusted. The DGR-003-lineage
identity core was inspected and exercised live before any change:
- `packages/node/meshnet_node/runtime_recipe.py` — node-side identity:
domain-separated digests (`meshnet.model-artifact.v1`,
`meshnet.runtime-recipe.v1`, `meshnet.shard-binding.v1`) over the source
artifact SHA (`source_digest`, with split artifacts bound to their exact
source via `DerivativeBinding`), tokenizer revision (pin-enforced),
architecture adapter + architecture/config digest, boundary and protocol
schema versions, backend, weight quantization, activation/compute dtypes, and
KV dtype/layout (`RECIPE_AXES`). Shard ranges are half-open
(`shard_start`/`shard_end`, end-exclusive, protocol convention) with no
topology or quant constants anywhere; `check_route` accepts any tiling of
`[0, layer_count)`. Route, handshake (`check_handshake`), and session-open
(`check_session_open`) checks fail closed with structured `RouteMismatch`
reasons mapped to specific protocol error codes (`handshake_error`).
- `packages/tracker/meshnet_tracker/recipe.py` — deliberately independent
tracker re-derivation (no `meshnet_node` import); declared fingerprints are
recomputed, never trusted (`parse_identity`, `FingerprintMismatch`). The
`CertificationLedger` keeps every registered recipe dark until a real
distributed forward — at least 2 distinct nodes, whole-model coverage,
non-synthetic, tokens actually generated — certifies it; dark recipes may
route only to certify.
- The two implementations are pinned by committed conformance vectors
(`tests/data/recipe_fingerprint_vectors.json`).
Live verification of that pre-existing core before changes:
`PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_recipe_identity.py`
`45 passed`; plus `tests/test_native_identity_emission.py`,
`tests/test_tracker_capability_admission.py`, `tests/test_node_admission.py`
`59 passed`.
## Gap found and closed (this story's change)
**The `runtime_version` recipe axis was a label, not a pin.** It was an opaque
caller-supplied string: nothing derived it from the DGR-027 lock manifest, and
neither identity implementation rejected a moving reference (`"latest"` was
accepted), so two workers could run different llama.cpp pins or patch stacks
under one label and still agree on the recipe digest. The acceptance criterion
explicitly requires fingerprinting the "runtime pin/patch stack".
### Changed files
- `packages/node/meshnet_node/runtime_pin.py` (new) — derives the canonical
`runtime_version` axis value from the DGR-027 lock workspace
(`packages/node/native/llama`):
`<runtime>@<40-hex upstream commit>+patchstack.<sha256>` where the stack
digest commits, under the `meshnet.runtime-patch-stack.v1` domain, to the
*ordered* `(patch name, patch bytes sha256)` stack. Fails closed on: missing
or malformed `UPSTREAM_LOCK.json`, unknown schema version, non-40-hex/moving
commit, `UPSTREAM_COMMIT` disagreement, any disagreement among the lock's
`patch_series`, `patches/series`, and `patches/SHA256SUMS`, a missing patch
file, or a patch whose bytes don't match their recorded digest. Reads the
committed manifest only; fetching/patching stays with
`scripts/llama_cpp_dependency.py` (DGR-027).
- `packages/node/meshnet_node/runtime_recipe.py``runtime_version` is now
pin-enforced (`_require_pin`) exactly like `tokenizer_revision`; for the
llama.cpp backend it must also match the canonical
`llama.cpp@<40-hex>+patchstack.<64-hex>` grammar.
- `packages/node/meshnet_node/native_backend.py` — the production native
identity seam no longer accepts a caller-supplied runtime string. It derives
`runtime_version` directly through `load_runtime_pin()` from the committed
lock and rejects a non-llama backend at this llama.cpp-specific boundary.
- `packages/tracker/meshnet_tracker/recipe.py` — the independent tracker
implementation applies the same backend-specific grammar before re-deriving
the recipe digest, so forged operator labels cannot register or certify.
- `tests/test_runtime_pin_identity.py` and
`tests/test_native_identity_emission.py` — deterministic tests cover lock
derivation, production native emission, and node/tracker rejection of the
forged values from independent review. Conformance vectors were regenerated
through `scripts/gen_recipe_fingerprint_vectors.py` for the tightened wire
contract.
### Backlog-consistency repair (pre-existing damage, honestly recorded)
`tests/test_ralph_prd_schema.py` had 4 pre-existing failures before this story
touched anything, left by prior sessions and the alternate-history merge:
- DGR-022 and DGR-027 were marked `passes: true` without `completionNotes` and
without regenerated issue projections. Added their `completionNotes`
(explicitly labeled as added during this repair, content drawn from their own
evidence READMEs) and regenerated
`issues/022-…` / `issues/027-…` via `scripts/ralph_prd_schema.py render`.
- Three pre-DGR legacy GLM alpha issue files (`18-…`, `19-…`, `20-…`,
committed 2026-07-14, before DGR-018 established the generated-only
convention; they carry no authority disclaimer because they are *not*
generated from prd.json) were relocated via `git mv` to
`issues/legacy/` — preserved as provenance, out of the generated namespace.
### prd.json
Marked `DGR-025.passes = true` with `completionNotes`; regenerated
`issues/025-define-exact-artifact-and-runtime-recipe-identity.md`.
## Acceptance criteria → evidence
1. **Fingerprint all axes**`RECIPE_AXES` + `ArtifactIdentity` cover source
artifact SHA, tokenizer revision, architecture adapter/version (adapter axis
+ architecture/config digest), boundary schema (boundary + protocol schema
versions), backend, quant, activation/compute dtype, KV/state layout; the
runtime pin/patch stack is now committed via the derived `runtime_version`
axis (`runtime_pin.py`). Verified by `test_runtime_recipe_identity.py` and
`test_runtime_pin_identity.py`.
2. **Exact half-open range, no hardcoded topology/quant**`ShardIdentity`
end-exclusive ranges, `DerivativeBinding` coverage checks, `check_route`
tiling over arbitrary layouts; quant/dtype values are open strings
(dynamic recipe inputs). Verified by `test_runtime_recipe_identity.py`
(routes of 1, 2, and 5 shards; no product constants).
3. **Fail closed on any mismatch** — artifact, adapter, boundary/schema, cache
layout, backend, and runtime mismatches each produce structured
`RouteMismatch` reasons and protocol error codes; the tracker recomputes
digests and rejects inconsistent claims; moving runtime references are now
rejected on both sides.
4. **Registered-but-dark**`CertificationLedger`: unknown recipes cannot be
certified, registered recipes are dark, only a real ≥2-distinct-node
whole-model non-synthetic forward promotes; verified by
`test_runtime_recipe_identity.py` / `test_tracker_capability_admission.py`.
5. **Gates + this handoff** — below.
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_pin_identity.py
```
```text
23 passed in 0.15s
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py \
tests/test_native_identity_emission.py tests/test_tracker_capability_admission.py \
tests/test_node_admission.py tests/test_node_capability.py tests/test_recipe_benchmark.py
```
```text
202 passed, 1 warning in 5.38s
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_ralph_prd_schema.py
```
```text
108 passed
```
(4 failed before this story's backlog repair; 0 after.)
```bash
python3 -m compileall -q packages tests # exit 0
git diff --check # exit 0
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
# OK: 55 stories validated.
```
Default tests are model-download-free, API-credit-free, and GPU-free; no model
artifact was touched and nothing was written under `/home`.
## Limitations
- The production native identity seam now derives the manifest pin and cannot
accept an operator-supplied runtime label. It still cannot attest that the
running binary was built from those locked bytes. Embedding the patched-tree
hash at build time and echoing it through the DGR-022 status contract belongs
with DGR-028+/DGR-031; real distributed certification remains the final trust
boundary.
- The DGR-027-recorded blocker stands: `0002-dense-llama-owned-range-loader.patch`
does not apply cleanly against the pin (DGR-028). That does not affect this
story: the identity commits to the patch *bytes as committed*, which is
precisely what makes a later repaired patch a *different* runtime identity.
- No native/CMake change was made, so the native build/CTest gate is not
applicable; no llama.cpp patch content was changed, so apply/check/reverse
verification is not applicable (and is blocked by the DGR-028 defect anyway).
- Tracker routing, load balancing, billing, telemetry, and relay semantics are
untouched; the only behavior change outside the new module is the stricter
(fail-closed) rejection of moving `runtime_version` values.
## Dependency handoff
- **DGR-026** (split-GGUF provisioning): bind each provisioned split via
`DerivativeBinding` to the exact source digest recorded in its hashed
manifest; the per-split `shard_binding_digest` is what certification pins.
- **DGR-031** (`ShardEngine`): construct worker identity through
`shard_identity_from_native_report` and populate `runtime_version` from
`meshnet_node.runtime_pin.load_runtime_pin().runtime_version` — never from an
operator string. A build-time echo of the patched-tree hash through the
status contract would close the manifest-vs-binary gap noted above.
- **DGR-041** (capability registration): the tracker already re-derives and
fail-closes on presented identities (`parse_identity`); register recipes
through the `CertificationLedger` so they arrive dark.
- **DGR-044** (DeepSeek V4 Flash target): pin the target's artifact identity
the same way `glm_alpha_artifact` does — read locked manifests, never restate
digests — and note `layer_count` must count the routed transformer stack the
route tiles, excluding MTP (reserved for beta).
## Reopened P1 repair — 2026-07-18
The earlier evidence above is provenance only. Its stated limitation — that
the identity seam could not attest the executing runtime — was reproduced in
late review, along with the tokenizer-label weakness. This repair replaces
both claims at the production identity boundary.
### Changed files
- `packages/node/meshnet_node/runtime_recipe.py` — replaces the moving-ref
denylist with the sole valid `tokenizer.v1:<sha256>` form, derived from an
ordered map of named tokenizer/config byte digests. A label, tag, branch, or
symbolic ref cannot be a valid identity.
- `packages/tracker/meshnet_tracker/recipe.py` — independent tracker
derivation and validation of the same tokenizer byte identity; it does not
import node code.
- `packages/node/meshnet_node/runtime_pin.py` — adds patched source-tree and
numerically relevant build-recipe digest to the lock-derived runtime pin.
- `packages/node/meshnet_node/native_backend.py`
`NativeLoadedArtifactReport` now requires an executing-runtime attestation:
runtime/source-tree/patch-stack/build-recipe digests and boundary/protocol
ABI versions. `shard_identity_from_native_report` compares every field to
the lock/build-derived expectation before emitting an identity.
- `scripts/gen_recipe_fingerprint_vectors.py` and
`tests/data/recipe_fingerprint_vectors.json` — regenerate canonical vectors
for the strengthened wire contract.
- `tests/test_runtime_pin_identity.py`,
`tests/test_runtime_recipe_identity.py`, and
`tests/test_native_identity_emission.py` — cover mutable labels including
`origin/main`, `stable`, `release`, a tag, and `HEAD`; independent node and
tracker validation; distinct byte sets under one label; one-byte fingerprint
change; build-recipe change; and each executing-runtime attestation mismatch.
### Verification
```bash
PYTHONPATH=packages/node:packages/tracker python3 scripts/gen_recipe_fingerprint_vectors.py
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_runtime_pin_identity.py tests/test_native_identity_emission.py \
tests/test_runtime_recipe_identity.py
```
```text
92 passed
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py \
tests/test_native_identity_emission.py tests/test_node_admission.py \
tests/test_node_capability.py tests/test_recipe_benchmark.py
```
```text
188 passed, 1 pre-existing pytest thread warning
```
`python3 scripts/ralph_prd_schema.py validate
.scratch/distributed-gguf-runtime/prd.json`, `python3 -m compileall -q packages
tests`, and `git diff --check` each exit 0. The broader PRD pytest projection
suite has two unrelated existing DGR-023 failures: its `passes: true` entry has
no completion notes and its generated issue file is stale. The socket-backed
subset of `test_tracker_capability_admission.py` is additionally un-runnable in
this sandbox (`PermissionError: [Errno 1] Operation not permitted` creating an
AF_INET socket); its deterministic non-socket identity coverage is included in
the passing runs above.
### Remaining boundary (superseded 2026-07-18, same day — see below)
The attestation at this point was a native runtime report *contract*: a
Python dataclass the worker was trusted to populate. Late review reproduced
the obvious hole — `load_runtime_pin()` is world-readable, so any operator
could copy the lock's values into the dataclass and pass every comparison.
The section below closes that hole.
## Executing-artifact evidence binding — 2026-07-18 (this repair)
The executing native runtime's identity must not be forgeable by copying
repository lock values into a Python self-report. Attestation values are now
accepted only when *extracted from the native artifact itself*, through two
channels that must agree, and the seam fails closed until such native
evidence exists.
### The boundary
`meshnet_node.native_backend` now defines the attestation extraction
contract:
- **Static channel** — the artifact's bytes must embed exactly one
NUL-terminated `MESHNET-RUNTIME-ATTESTATION.v1:<canonical json>` marker.
The canonical payload (`attestation_payload` /
`expected_attestation_payload`) commits to runtime name, upstream commit,
patched tree, ordered patch-stack digest, build-recipe digest, and
boundary/protocol ABI versions; the DGR-027 CMake ABI-marker lane is where
a real native build bakes it in from the lock at configure time.
- **Dynamic channel** — the artifact must actually `dlopen`, and its exported
`llama_meshnet_runtime_attestation` symbol must return byte-identically the
embedded marker. A marker pasted into a plain file is not an executing
runtime.
- **Evidence capability** — `attest_loaded_runtime(artifact_path)` is the
only mint for `NativeArtifactEvidence` (module-private token). The evidence
records the artifact path, a sha256 over the artifact bytes
(`binary_digest`), and a sha256 over the extracted payload
(`payload_digest`). `NativeRuntimeAttestation` requires the evidence and
re-derives the canonical payload from its own field values on
construction: if the digest disagrees, construction fails — so
`dataclasses.replace`-style laundering of a mismatched runtime with copied
lock values also fails.
- `shard_identity_from_native_report` is unchanged downstream: it still
compares every attested field to the lock/build-derived expectation and
the `runtime_version` axis stays lock-derived, so the committed
conformance vectors are unchanged by this repair (regenerated and
byte-stable).
Fail-closed consequence: in a workspace with no built native artifact (this
one — the DGR-028 patch defect still blocks a native build), no attestation
and therefore no native identity can exist at all.
### Changed files
- `packages/node/meshnet_node/native_backend.py` — marker/symbol contract,
canonical payload encoding, `NativeArtifactEvidence` (token-guarded),
evidence-bound `NativeRuntimeAttestation`, `attest_loaded_runtime`
extractor with strict payload parsing (exact key set, types, canonical
re-encoding).
- `tests/test_native_identity_emission.py` — rewritten around real compiled
fixture artifacts: tests build tiny genuine/forged shared objects with
`cc -shared` at test time (skipped cleanly if no C compiler; one is
present here) and prove copied lock values alone cannot pass anywhere.
### Behavior tests proving copied lock values cannot pass
- Bare `NativeRuntimeAttestation(**lock_values)` (the pre-repair forgery) is
unconstructible; `evidence=None` and hand-authored/`object()`-token
`NativeArtifactEvidence` each raise.
- The true marker bytes written into a plain file fail (`not a loadable`).
- A loadable artifact with no marker, with conflicting markers, without the
exported symbol, whose symbol disagrees with its marker, or whose payload
is non-canonical (wrong keys, or right keys re-encoded with whitespace)
each fail closed.
- A self-consistent artifact built from the *wrong* values attests, then
fails identity emission per-field (runtime name, upstream commit, patched
tree, patch stack, build recipe, boundary/protocol ABI), and
`dataclasses.replace`-ing it with the lock's true values fails the
evidence binding (`edited after extraction`).
- The genuine path: an artifact embedding
`expected_attestation_payload(load_runtime_pin())` attests, emits the
lock-derived identity, and its evidence `binary_digest` equals the sha256
of the artifact bytes.
### Verification (all in this worktree, 2026-07-18)
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_native_identity_emission.py tests/test_runtime_pin_identity.py \
tests/test_runtime_recipe_identity.py
```
```text
104 passed
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_node_admission.py tests/test_node_capability.py \
tests/test_recipe_benchmark.py
```
```text
96 passed, 1 pre-existing pytest thread warning
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_tracker_capability_admission.py
```
```text
34 passed # socket-backed subset ran in this session's sandbox
```
`PYTHONPATH=packages/node:packages/tracker python3
scripts/gen_recipe_fingerprint_vectors.py` reproduces the committed vectors
byte-for-byte; `python3 -m compileall -q packages tests` and
`git diff --check` each exit 0.
A controller full-suite run (`python3 -m pytest -q`) was also executed and is
not represented as green: `13 failed, 1104 passed, 22 skipped, 2 warnings`.
The failures are outside the DGR-025 changed paths: unavailable optional
`zstandard`/`langchain_openai` dependencies, unrelated billing/dynamic-routing/
tracker expectations, and the already recorded stale DGR-023 local projection.
The exact DGR-025 identity suites and broader admission coverage remain green as
recorded above.
### Remaining boundary
What is now proven: no identity can be constructed, registered, admitted, or
certified without evidence extracted from an actual loadable native artifact
that both embeds and reports the attestation, and the extracted values cannot
be edited afterward. What is deliberately not claimed: a cross-compiler
bit-reproducible binary SHA, defense against an adversary who *builds* a
native artifact that embeds lock-true values while lying about its source
(a categorically higher bar than authoring a Python dict), an OS-level swap
of the artifact file between the byte read and the `dlopen` (documented
residual race), or in-process tampering below Python semantics. Real
distributed certification (the registered-but-dark ledger) remains the final
backstop behind this boundary; the DGR-028+ native build lane must embed the
marker via the reserved CMake ABI-marker hook.
## Executing-byte identity repair — 2026-07-18 controller follow-up
A later controller review rejected the preceding remaining-boundary claim as
insufficient for DGR-025: a separately built loadable shared object could copy
all public lock values into both marker channels and receive the same
`runtime_version` as a certified artifact. The repair now appends
`+artifact.<sha256>` to the llama.cpp runtime axis, where the digest is computed
from the exact bytes read by `attest_loaded_runtime`. Node and tracker parsers
independently require this suffix. Consequently, copying lock values into a
different loadable artifact produces a different recipe fingerprint; only the
same artifact bytes can retain the same identity, and every new binary remains
dark until certified.
`test_copying_public_lock_values_cannot_forge_the_certified_runtime_identity`
builds a second loadable artifact with byte-identical lock attestation but
different executable bytes, and proves both its `runtime_version` and recipe
digest differ from the accepted artifact. Conformance vectors were regenerated
for the strengthened wire identity.
Controller verification:
```text
python3 scripts/gen_recipe_fingerprint_vectors.py
python3 -m pytest -q tests/test_native_identity_emission.py \
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py
# 105 passed in 0.52s
python3 -m compileall -q packages/node/meshnet_node \
packages/tracker/meshnet_tracker tests scripts/gen_recipe_fingerprint_vectors.py
# exit 0
git diff --check
# exit 0
```

View File

@@ -0,0 +1,268 @@
# DGR-026 evidence — provision exact split-GGUF artifacts outside `/home`
**Status:** implemented and verified this session; live re-review, not inherited credit.
**Dependency:** DGR-025 (`evidence/DGR-025/README.md`) — read before changing code.
## Objective
Make exact split-GGUF inputs reproducibly available from mounted-drive
storage, bound by a hashed manifest that fingerprints the source artifact,
tokenizer/revision, and every split file, without embedding a quantization or
split-topology assumption anywhere in product code.
## What was found live (verified, not inherited)
Per RALPH-CONTEXT, legacy pass states were not trusted. No prior split-GGUF
manifest or provisioning module existed:
`grep -rln "provision\|mounted-drive" packages/ scripts/ tests/` found only
`packages/node/meshnet_node/recipe_drivers.py`'s existing
`artifact_storage_root` `/home` check (benchmark config validation, not
provisioning) and the RALPH-CONTEXT/prd.json prose itself. The pre-existing
`packages/node/meshnet_node/downloader.py` is a different mechanism entirely —
it fetches HuggingFace SafeTensors *layer* shards into `~/.cache/meshnet/shards`
(i.e. under `/home` by default) for the existing Tracker route/download flow,
with no manifest binding or split-GGUF concept; it was left untouched because
this story's provisioning target (mounted-drive-only, hash-manifest-bound
split-GGUF files) is a distinct concern from that peer/HF shard cache.
Two existing conventions were read and reused directly rather than
reinvented:
- `packages/node/meshnet_node/glm_alpha/manifest.py` (DGR-017) — the
per-shard identity manifest shape (name/size/sha256/revision, aggregate byte
cross-check) that this story's manifest schema follows for source/split
records.
- `packages/node/meshnet_node/runtime_recipe.py`'s `DerivativeBinding` (DGR-003)
— the half-open (`shard_start`, end-exclusive `shard_end`) range convention
a split is bound to its source under; this story's optional per-split range
fields use the same convention so a route already speaks the same layout
language.
- `packages/node/meshnet_node/recipe_drivers.py`'s `_validate_config` — the
exact `/home` rejection shape (`not root.is_absolute() or root ==
Path("/home") or Path("/home") in root.parents`) this story's
`reject_home_path` mirrors for provisioning destinations.
## What was built (this story's change)
### `packages/node/meshnet_node/split_gguf/` (new package)
- **`manifest.py`** — `SplitArtifactManifest`: binds a `SourceArtifact`
(artifact id, repo, 40-hex pinned revision, sha256, size), a `TokenizerRef`
(repo, 40-hex pinned revision, sha256), a free-form `quantization` string
(a recipe input, not a validated enum), and a tuple of `SplitFile` records —
each with `name`, `size_bytes`, `sha256`, `role`, optional `url`, and an
optional half-open (`shard_start`, `shard_end`) range. `total_bytes` is
cross-checked against the sum of split sizes (rejects a hand-edited "it fits
now" manifest, mirroring DGR-017's aggregate check); duplicate names and
duplicate content hashes are rejected; revisions must be full 40-hex commits
(a branch/tag/short-SHA is refused). Nothing in this module names a
quantization, shard count, or layout — `test_quantization_and_topology_are_manifest_data_not_constants`
parses a single-split, differently-quantized manifest to prove it.
- **`provision.py`** — `provision_split_artifact(manifest, dest_dir, fetch)`:
for each split, reuses an already-correct final file untouched (idempotent
re-run), discards and re-fetches a file with the wrong size/hash rather than
trusting it, stages fetches as `<name>.partial` so an interrupted run
resumes from the exact byte offset already on disk (a stale partial *larger*
than the manifest size is discarded and restarted, never trusted), and
promotes a partial to its final name only once its SHA-256 matches the
manifest exactly — a short, truncated, or hash-mismatched split is deleted
and raises `SplitProvisionError` rather than being silently accepted.
`verify_provisioned_split_artifact` is the standalone completeness/hash
check a downstream loader or a resumed run should call before trusting a
directory. `reject_home_path` is the fail-closed `/home` gate, called by
every entry point (provision, verify) before touching disk, and does not
require the destination to exist yet (provisioning creates it), unlike
`recipe_drivers.py`'s `strict=True` benchmark-root check. Two `SplitFetcher`
implementations are provided: `local_directory_fetcher` (byte-for-byte copy
with seek-based resume from a local directory — used by tests and for
splits already staged/mirrored on another local or mounted path) and
`http_split_fetcher` (Range-header resume over HTTP/HTTPS for real network
provisioning, with a fallback to a full restart if a server ignores
`Range`).
### `scripts/provision_split_gguf.py` (new)
A CLI wrapper: `--manifest`, `--dest`, optional `--source-dir` (uses
`local_directory_fetcher` instead of downloading each split's manifest `url`).
Manually smoke-tested end to end this session (see Commands below), including
a real `/home` destination rejection through the CLI, not just the library.
### Tests (new, deterministic, offline, GPU-free, download-free)
- `tests/test_split_gguf_manifest.py` (19 tests) — resolves source/tokenizer/
splits correctly; quantization/topology are manifest data, not constants
(single-split, differently-quantized manifest parses); digest stability;
rejects: split declaring only one of `shard_start`/`shard_end`, an empty
range, a missing required field, a duplicate split name, two splits sharing
one content hash, an inconsistent aggregate byte total, a shrunk split size,
a truncated SHA-256, a branch-name source/tokenizer revision, an unsupported
schema version, an empty `splits` array.
- `tests/test_split_gguf_provision.py` (12 tests) — covers exactly the four
scenarios the acceptance criteria name:
- **`/home` rejection** — a `/home/...` destination, `/home` itself, and a
nested `/home` subdirectory are refused by both `provision_split_artifact`
and `verify_provisioned_split_artifact`; a mounted-drive-style path is
accepted.
- **Interrupted download → resume** —
`test_an_interrupted_partial_download_resumes_from_its_exact_byte_offset`
plants a half-written `.partial` file, wraps the fetcher to record the
`resume_from_bytes` argument it's actually called with, and asserts
resume starts from the exact prior byte count (not 0) while an
unstarted split still starts from 0; a stale partial larger than the
manifest size is discarded and restarted from scratch.
- **Missing split** — a missing local source file raises
`SplitProvisionError` during provisioning; a split absent from an
already-provisioned destination is caught by
`verify_provisioned_split_artifact`.
- **Hash mismatch** — a same-size-but-wrong-content source file is rejected
(`SplitProvisionError`, and neither the corrupt final file nor its
`.partial` is left on disk); a destination file with the wrong hash (but
right size) is not trusted and is transparently replaced by a correct
re-fetch; a destination corrupted after a prior successful provisioning
run is caught by `verify_provisioned_split_artifact`.
- Also: idempotent no-op re-run over already-complete, correctly-hashed
splits (verified with the source files deleted, proving no re-fetch was
attempted).
## Acceptance criteria → evidence
1. **Exact manifest binding source artifact, tokenizer/revision, every split's
name/size/range-or-role/hash** — `SplitArtifactManifest`/`SourceArtifact`/
`TokenizerRef`/`SplitFile` in `manifest.py`; covered by
`test_split_gguf_manifest.py`.
2. **Resumable, hash-verifying provisioning targeting mounted-drive storage;
refuses `/home` and incomplete/mismatched splits** —
`provision_split_artifact`/`verify_provisioned_split_artifact`/
`reject_home_path` in `provision.py`; covered by
`test_split_gguf_provision.py` and the CLI smoke test below.
3. **Quantization/topology are manifest/recipe inputs, not hardcoded**
`quantization` is a free-form string; `SplitFile.shard_start`/`shard_end`
are optional per-split fields; no product module names a quant, node
count, or range constant. Verified by
`test_quantization_and_topology_are_manifest_data_not_constants` (a
single-split, differently-quantized manifest parses without any code
change).
4. **Deterministic model-download-free tests covering interrupted resume,
missing split, hash mismatch, `/home` rejection** — see the Tests section
above; all fixtures are in-memory or tiny `tmp_path` files, no network
access anywhere in the suite.
5. **Gates + this handoff** — below.
## Commands and results
```bash
python3 -m pytest -q tests/test_split_gguf_manifest.py tests/test_split_gguf_provision.py
```
```text
31 passed in 0.10s
```
```bash
python3 -m pytest -q tests/test_ralph_prd_schema.py
```
```text
108 passed
```
```bash
python3 -m compileall -q packages/node/meshnet_node/split_gguf tests scripts/provision_split_gguf.py
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
(compileall exit 0; git diff --check exit 0)
OK: 55 stories validated.
```
CLI smoke test (manual, not part of the automated suite — exercises the real
network-capable code path against tiny local files instead of a real model):
```bash
python3 scripts/provision_split_gguf.py \
--manifest /tmp/dgr026-smoke/manifest.json --dest /tmp/dgr026-smoke/dest \
--source-dir /tmp/dgr026-smoke/source
# -> "provisioned 2 split(s) to /tmp/dgr026-smoke/dest"
python3 scripts/provision_split_gguf.py \
--manifest /tmp/dgr026-smoke/manifest.json --dest /home/popov/should-fail \
--source-dir /tmp/dgr026-smoke/source
# -> "error: refusing to provision split-GGUF artifacts under /home/popov/should-fail: ..."
# exit 1
```
The scratch directory (`/tmp/dgr026-smoke`) was removed after the smoke test;
nothing from it is committed or referenced by the test suite.
Default tests are model-download-free, API-credit-free, and GPU-free; no model
artifact was downloaded and nothing product-relevant was written under
`/home` (the CLI smoke test's `/home` path was rejected before any write).
## Changed files
- `packages/node/meshnet_node/split_gguf/__init__.py` (new)
- `packages/node/meshnet_node/split_gguf/manifest.py` (new)
- `packages/node/meshnet_node/split_gguf/provision.py` (new)
- `scripts/provision_split_gguf.py` (new)
- `tests/test_split_gguf_manifest.py` (new)
- `tests/test_split_gguf_provision.py` (new)
- `.scratch/distributed-gguf-runtime/prd.json` (`DGR-026.passes = true` +
`completionNotes`; also restored the top-level `sourceOfTruth`/
`qualityGates`/`metadataSchema`/`milestones`/`supersededStories`/
`branchName` fields — see Gotcha below)
- `.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md`
(regenerated via `scripts/ralph_prd_schema.py render`)
- `.scratch/distributed-gguf-runtime/evidence/DGR-026/README.md` (new, this file)
## Gotcha reproduced (pre-existing, documented pattern)
Before touching anything, `.scratch/distributed-gguf-runtime/prd.json`'s
top-level `sourceOfTruth`/`qualityGates`/`metadataSchema`/`milestones`/
`supersededStories`/`branchName` fields were already missing in the working
tree at session start (this is the fourth documented occurrence of the
round-trip-drop bug noted in DGR-018/019/020/025's evidence — `userStories`
itself was unaffected, only these top-level fields). Restored them from
`git show HEAD:.scratch/distributed-gguf-runtime/prd.json` before making any
DGR-026 edit; `scripts/ralph_prd_schema.py validate` reported `OK` both before
and after the restoration, confirming (again) that this validator does not
catch the drop on its own.
## Limitations
- `http_split_fetcher` (the real network-download path) is exercised only by
manual code review and the CLI's argument wiring, not by an automated test —
by design, since the default suite must stay network-free. Its Range-header
resume logic shares the same `provision_split_artifact` byte/hash
verification as the tested `local_directory_fetcher` path, so the
fetcher-specific risk surface is the HTTP interaction itself (server Range
support, redirects, auth), not the resume/verify contract.
- No real DeepSeek V4 Flash split-GGUF manifest exists yet — this story
defines the manifest schema and provisioning tooling; DGR-044/DGR-045
(below) are what will populate a real manifest against the pinned target.
- `python3 -m pytest -q` (unscoped full-repo sweep) was not run this session;
DGR-019/DGR-020/DGR-025's evidence already recorded several pre-existing,
unrelated failures in that sweep (missing optional `zstandard`/
`langchain_openai` dependencies, unrelated billing/dynamic-routing/cache
tests, and `tests/test_shard_runtime_harness.py`'s `grpc` import
requirement). This story's own targeted suites, `test_ralph_prd_schema.py`,
`compileall`, and `git diff --check` are all green as recorded above.
- Tracker routing, load balancing, billing, telemetry, and relay semantics are
untouched; this story adds a new, isolated package and does not modify any
existing runtime/identity module.
## Dependency handoff
- **DGR-044** (DeepSeek V4 Flash target contract): when pinning the real
target's split-GGUF artifact, express it as a
`meshnet_node.split_gguf.manifest.SplitArtifactManifest``source.sha256`
is the whole-model artifact digest DGR-003's `ArtifactIdentity.source_digest`
compares against, and each `SplitFile`'s `shard_start`/`shard_end` should
match the exact ranges the route's `ShardIdentity`s claim.
- **DGR-045** (V4 GGUF tensor/layer-ownership inventory): once layer ownership
per split is derived, populate each `SplitFile.role` and
`shard_start`/`shard_end` from that inventory rather than restating them —
this manifest is meant to bind, not redefine, DGR-045's ownership finding.
- Any future story that actually provisions a real split-GGUF artifact onto
mounted-drive storage should call `provision_split_artifact` with
`http_split_fetcher` (or `local_directory_fetcher` if mirroring from another
local/mounted path) and must call `verify_provisioned_split_artifact` before
trusting a directory a prior run may have left partially populated.

View File

@@ -0,0 +1,72 @@
# DGR-027 evidence — exact llama.cpp provenance manifest and fetch workspace
**Completed implementation:** 2026-07-17
**Branch:** `ralph/dgr-small-terra`
**Authority:** live Gitea issue #11. The controller fetched and claimed the issue
through the Gitea API before launch; the isolated agent received that exact body.
## Changed files
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/PATCH-STACK.md`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`
## Provenance and retrieval contract
`UPSTREAM_LOCK.json` records the upstream Git URL, immutable 40-character
commit `e920c523e3b8a0163fe498af5bf90df35ff51d25`, expected Git tree
`6c91a11407a3a3fb160f5dac705f9c59718f54f1`, MIT license, and the sole
retrieval method: `git-clone-detached-commit` into `build/llama.cpp/source`.
`python3 scripts/llama_cpp_dependency.py fetch` has no branch, tag, ref, or
repository override. On a first fetch it clones the manifest URL, checks out
the detached commit, and verifies commit, tree, required upstream blobs,
license, and cleanliness. If the workspace already exists, it makes no network
request and accepts it only after the same verification. Dirty or mismatched
caches fail closed. The build directory is already ignored by `.gitignore`.
## Verification
| Command | Result |
| --- | --- |
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py` | `7 passed in 0.22s` |
| `python3 -m compileall packages tests` | passed |
| `git diff --check` | passed (no output) |
| `python3 scripts/llama_cpp_dependency.py inspect` | passed; reports exact commit/tree, retrieval workspace, MIT license, and two-patch stack |
| `python3 scripts/llama_cpp_dependency.py fetch --workspace /tmp/not-llama-workspace` | failed closed with status 2: workspace outside the locked ignored build root |
| symlinked workspace regression | passed; both a `build/` ancestor symlink and a final `source` symlink escaping the repository are refused |
| attached-branch cache regression | passed; an exact commit on a local branch is refused until checked out as detached HEAD |
| ignored/excluded injection regression | passed; a file hidden by `.git/info/exclude` is detected and refused |
| tracked injection regression | passed; modified tracked content hidden by both `assume-unchanged` and `skip-worktree` is content-hashed and refused |
| executable-mode regression | passed on the POSIX fixture for both index flags; the mounted project workspace has `core.filemode=false`, so its exact index tree is the canonical mode record and physical mode bits are not treated as meaningful |
| `git check-ignore -v build/llama.cpp/source` | passed; `.gitignore:6:build/` |
| `git diff --summary` and `git ls-files build packages/node/native/llama` | no source checkout or new submodule introduced; only manifest/docs/patches/native wrapper are tracked |
| `python3 scripts/llama_cpp_dependency.py fetch` (controller network lane) | passed; fetched the exact detached commit and verified HEAD `e920c523e3b8a0163fe498af5bf90df35ff51d25` and tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1` in the ignored workspace |
| `python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source` | failed on the pre-existing `0002-dense-llama-owned-range-loader.patch` as a corrupt patch at line 26; this is an explicit DGR-028 blocker and no native-build claim is made |
The targeted test suite creates a local Git fixture to prove offline cache reuse
after full identity verification, then proves a dirty cache is rejected. It
also proves the CLI rejects a repository/branch override and an arbitrary
workspace.
## Limitations
- The controller successfully materialized and verified the exact upstream
commit/tree, so the DGR-027 fetch and offline-cache boundary has real upstream
evidence rather than fixture-only evidence.
- The existing `0002-dense-llama-owned-range-loader.patch` is malformed and
cannot pass `git apply --check` against the exact pin. DGR-027 changes no patch
file; repairing and certifying the numbered patch stack belongs to DGR-028.
Until that story closes, the repository must not claim patched-tree, native
CMake/CTest, or reverse-apply certification.
- No model, API credits, GPU, or model artifact storage was used.
## Dependency handoff
DGR-028, DGR-029, and DGR-044 must invoke the manifest-owned `fetch` command
before touching llama.cpp source. They may use only the verified
`build/llama.cpp/source` checkout and must record any native build, CTest, and
patch apply/check/reverse evidence against the exact manifest pin. DGR-017's
cleanup remains provenance only and grants no inherited completion credit.

View File

@@ -0,0 +1,191 @@
# DGR-028 evidence — numbered llama.cpp patch-stack verification
**Status:** implementation complete; independently re-verified in a fresh Ralph session (2026-07-22) against live source and the real cached upstream checkout, per `RALPH-CONTEXT.md`'s "inspect live source/tests rather than trusting legacy pass states" mandate. `prd.json`'s `DGR-028.passes` is now `true`.
**Authority:** local `prd.json` is authoritative; live Gitea #12 is a projection.
**Upstream pin:** `e920c523e3b8a0163fe498af5bf90df35ff51d25` (`llama.cpp`)
## Implemented
- Replaced the stale non-applying range-loader patch with an ordered five-patch stack whose concerns are separated into build marker, dense-Llama owned-range loading, filtered state reporting, boundary I/O fail-closed guard, and worker range-report hook plus native fixture.
- Added `patches/UPSTREAM-ASSUMPTIONS.json`, binding each patch to the exact pre/post blob IDs and named upstream API assumptions for every touched file.
- Extended `scripts/llama_cpp_dependency.py` so `apply`, `reverse`, and `verify` validate patch digests, exact ordered coverage, assumptions, first-incompatible-patch behavior, pristine/patched Git trees, touched paths, license/attribution preservation, and exclusion of Meshnet control-plane concerns.
- `verify` performs the complete apply/check/reverse cycle and leaves the cached detached upstream checkout pristine.
- Updated the lock's exact patched tree and patch checksums. No model artifact was downloaded or created.
## Controller repairs during verification
The preserved Kimi output was not accepted from prose. Initial controller execution found and repaired:
1. a missing `_git` helper that made the dependency verifier raise `NameError`;
2. assumptions resolved relative to the repository root rather than the llama manifest directory;
3. the documented `verify`/`reverse` contract was not wired into the CLI or apply path;
4. assumptions and control-plane/license boundaries were defined but never enforced during apply;
5. a stale Python test hardcoded the old two-patch count;
6. the native fixture made an invalid strict resident-buffer-size comparison. Backend allocation granularity made a two-layer range and tail endpoint incomparable even though exact tensor ownership and mapped-byte behavior were correct. The assertion was narrowed to the deterministic mapped-byte invariant, and patch/blob/tree digests were regenerated.
## Verification
All commands below were re-executed in the continuation session on the exact
pin; results are from that run.
```text
cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS
# all five patches OK
python scripts/llama_cpp_dependency.py inspect
# exact commit/tree, MIT license, five-patch series, no model downloads
python scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
# reused verified offline cache; apply/check/reverse succeeded; source returned to clean detached HEAD
git -C build/llama.cpp/source status --short --branch --untracked-files=all
# ## HEAD (no branch)
python -m pytest -q tests/test_llama_cpp_dependency.py
# 7 passed in 0.27s
python -m compileall -q scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
# exit 0
python -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
```
Focused native gate against the patched exact pin (apply first because `verify`
intentionally restores the source checkout to pristine state, then reverse after
the test):
```text
python scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# patched index tree c0045714735ae5ee7b7334a480d8ac04e03e1b18 matches the lock
cmake -S build/llama.cpp/source -B build/llama.cpp/dgr028-build-verify \
-G 'Unix Makefiles' -DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=ON \
-DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_SERVER=OFF \
-DLLAMA_BUILD_TOOLS=OFF -DLLAMA_BUILD_APP=OFF -DLLAMA_CURL=OFF
cmake --build build/llama.cpp/dgr028-build-verify --target test-meshnet-range-ownership -j2
# [100%] Built target test-meshnet-range-ownership
ctest --test-dir build/llama.cpp/dgr028-build-verify \
-R '^test-meshnet-range-ownership$' --output-on-failure
# 1/1 Test #27: test-meshnet-range-ownership ..... Passed 0.01 sec
python scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
git -C build/llama.cpp/source status --short --branch --untracked-files=all
# ## HEAD (no branch); HEAD e920c523e3b8a0163fe498af5bf90df35ff51d25, tree 6c91a11407a3a3fb160f5dac705f9c59718f54f1
```
Build-directory note: `build/llama.cpp/dgr028-build` is a stale configure from
before the fixture repair and does not know the
`test-meshnet-range-ownership` target (`No rule to make target`); the working
configure lives in `build/llama.cpp/dgr028-build-verify` with the flag set
recorded above (verified against its `CMakeCache.txt`). Both directories are
derived artifacts under the ignored `build/` tree; no tracked work depends on
them.
A broad `cmake --build ... --target test` was also attempted after building only the focused target. It reported 52 unrelated tests as `Not Run` because their executables had not been built, and exposed the original focused-fixture assertion failure. It is not presented as a full-suite gate. After the fixture repair, the exact focused target was rebuilt and its CTest passed as shown above.
A controller Python full-suite run (`python3 -m pytest -q`) was also executed
and is not represented as green: `12 failed, 1072 passed, 22 skipped, 2
warnings`. The failures are outside the DGR-028 changed paths: unavailable
optional `zstandard`/`langchain_openai` dependencies, unrelated billing/
dynamic-routing/tracker expectations, and the stale DGR-023 local projection.
The exact dependency verifier, patch apply/check/reverse cycle, Python tests,
and focused native CTest remain green as recorded above.
## Changed files
- `packages/node/native/llama/PATCH-STACK.md`
- `packages/node/native/llama/THIRD_PARTY_NOTICES.md`
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/patches/series`
- `packages/node/native/llama/patches/SHA256SUMS`
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loading.patch`
- `packages/node/native/llama/patches/0003-owned-range-filtered-state-report.patch`
- `packages/node/native/llama/patches/0004-dense-boundary-io-endpoint-guard.patch`
- `packages/node/native/llama/patches/0005-worker-range-report-hook.patch`
- `packages/node/native/llama/patches/UPSTREAM-ASSUMPTIONS.json`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md`
The superseded `0002-dense-llama-owned-range-loader.patch` is removed.
## Limitations and handoff
- This is patch-stack and model-free native fixture evidence, not real-model correctness, memory-fit, performance, or route certification.
- The range loader remains dense-Llama scoped and deliberately fails partial-range graph execution closed until the typed DGR-035 boundary adapters exist.
- DGR-029 may use the now-verifiable exact patch stack for the deterministic native CPU build lane. DGR-034 owns real dense-Llama range behavior and memory evidence.
## Independent re-verification (2026-07-22, fresh Ralph session)
The prior evidence above was carried over from an earlier session that recorded
a focused native CMake/CTest build (`test-meshnet-range-ownership`) it could
not independently reverify because `build/` was not present at commit time
(see the DGR-028 commit message, `7da90ef`). This session re-ran the
Python/Git-level contract live and end to end, and is explicit about what
could and could not be re-checked:
```text
cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS
# all five patches: OK
python3 scripts/llama_cpp_dependency.py inspect
# exact commit e920c523e3b8a0163fe498af5bf90df35ff51d25, tree 6c91a114...,
# MIT license, five-patch series, no model downloads
python3 scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
# reused verified offline cache; apply -> assumption/boundary checks ->
# reverse succeeded; source left at pristine detached HEAD
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# git -C build/llama.cpp/source diff --cached --name-only ==
# CMakeLists.txt, cmake/meshnet-patch-stack.cmake, include/llama.h,
# src/llama-model.cpp, src/llama-model.h, src/models/llama.cpp,
# tests/CMakeLists.txt, tests/test-meshnet-range-ownership.cpp
# git -C build/llama.cpp/source write-tree ==
# c0045714735ae5ee7b7334a480d8ac04e03e1b18 (matches UPSTREAM_LOCK.json patched_tree)
python3 scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
# git -C build/llama.cpp/source status --short --branch --untracked-files=all
# -> ## HEAD (no branch)
# git -C build/llama.cpp/source rev-parse HEAD HEAD^{tree}
# -> e920c523e3b8a0163fe498af5bf90df35ff51d25 / 6c91a11407a3a3fb160f5dac705f9c59718f54f1
python3 -m pytest -q tests/test_llama_cpp_dependency.py tests/test_ralph_prd_schema.py
# 115 passed
python3 -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
```
`cmake` is not installed in this environment (`which cmake` fails), so the
native CMake/CTest build claim from the prior session (`test-meshnet-range-ownership`
1/1 Passed) could **not** be independently re-executed here; it is neither
re-confirmed nor retracted, just carried forward from `7da90ef` without a new
build-verified claim in this session. Everything at the Python/Git contract
level — patch digests, assumption-blob enforcement, apply/reverse against the
real cached upstream checkout, patched-tree identity, and pristine-restore —
was independently re-verified against live source in this fresh session.
## prd.json repair (unrelated to DGR-028 itself)
Before editing `DGR-028.passes`, `prd.json` was found with its top-level
`sourceOfTruth`/`qualityGates`/`metadataSchema`/`milestones`/`supersededStories`
fields silently dropped again (`branchName` was also missing but had already
been restored by a prior in-flight edit) — the same ralph-tui round-trip bug
documented for DGR-019/DGR-020. Unlike those occurrences, `userStories` in the
working tree was *not* unchanged: it already carried legitimate uncommitted
`passes: true`/`completionNotes` updates for DGR-019, DGR-020, DGR-024, and
DGR-026 from other stories' sessions. The missing top-level sections were
restored from `git show HEAD:.scratch/distributed-gguf-runtime/prd.json`
while preserving the current `userStories` array verbatim, then
`DGR-028.passes` was set `true` with `completionNotes` added, and
`.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md`
was regenerated via `scripts/ralph_prd_schema.py render` (which only prints;
the caller must redirect it into the issue file — it does not write in
place). `python3 scripts/ralph_prd_schema.py validate` and
`python3 -m pytest -q tests/test_ralph_prd_schema.py` (108 passed) both pass
against the repaired file.

View File

@@ -0,0 +1,198 @@
# DGR-029 evidence — native CMake skeleton and deterministic CPU lane
**Status:** implementation complete, live-verified in this session (2026-07-22).
**Authority:** local `prd.json` is authoritative; Gitea is a projection.
**Upstream pin:** `e920c523e3b8a0163fe498af5bf90df35ff51d25` (`llama.cpp`, from `UPSTREAM_LOCK.json`).
## What existed before this session
`scripts/llama_cpp_dependency.py` already had `build()`, `smoke()`, and `reproduce()`
functions and `UPSTREAM_LOCK.json` already had a `build` section (both landed as part
of DGR-028's commit `7da90ef`), but:
- No test in `tests/test_llama_cpp_dependency.py` ever exercised `build`/`smoke`/`reproduce`
— only `fetch`/`apply`/`reverse`/`inspect` had coverage.
- `cmake` was not installed in the DGR-028 session's environment ("`cmake` is not installed
in this environment," per its evidence), so this lane was never actually run end to end;
DGR-028's own live-verified CTest evidence used a one-off manual `cmake`/`ctest` invocation
with `-DLLAMA_BUILD_TESTS=ON` outside this driver, against a build directory that no longer
exists in this session.
- The locked `configure_flags` did not force CPU-only backend options (`GGML_CUDA`/`GGML_HIP`/
`GGML_VULKAN`/`GGML_METAL`/`GGML_BLAS`) — relying on upstream per-platform defaults (which
happen to default OFF on Linux, but are undocumented and platform-dependent), and
`LLAMA_BUILD_TESTS` was `OFF`, so no CTest lane existed at all — only a `--help` smoke check
against the unrelated stock `llama-gguf-hash` tool.
This session found and closed those three gaps rather than re-implementing from scratch.
## What changed in this session
- `packages/node/native/llama/UPSTREAM_LOCK.json`: the `build` section's `configure_flags` now
explicitly force `-DGGML_CPU=ON` and `-DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF
-DGGML_METAL=OFF -DGGML_BLAS=OFF`, so the CPU lane can never silently gain a GPU/BLAS backend
from a build machine's ambient toolchain. `-DLLAMA_BUILD_TESTS` flipped `OFF``ON` (required
so the `test-meshnet-range-ownership` CTest target exists at all — configuring
`LLAMA_BUILD_TESTS=ON` does not itself build every upstream test, only registers them; the
`native_targets` list still controls what actually gets compiled). Added `native_targets` entry
`test-meshnet-range-ownership` and a new `ctest_regex` field
(`"^test-meshnet-range-ownership$"`) naming the exact deterministic model-free fixture CTest
added by DGR-028's patch 0005.
- `scripts/llama_cpp_dependency.py`: factored `_cmake()`'s override/PATH/venv-sibling resolution
into a shared `_toolchain_binary(name, env_var)` and added `_ctest()` using the same resolution
(`CTEST` env override, PATH, or the sibling of the resolved `cmake` binary's environment). Added
`ctest_lane(build_dir)`, which loads the lock's `ctest_regex` and runs
`ctest --test-dir <build_dir> -R <regex> --output-on-failure`, printing output on success and
raising `DependencyError` (via the existing `_run` wrapper, which already attaches
stdout/stderr detail) on failure. Added a `ctest` CLI subcommand (`--build-dir`). Wired
`reproduce()` to run `fetch → apply → build → smoke → ctest_lane → reverse`, so a full
`reproduce` run leaves the cached upstream checkout pristine afterward (previously `reproduce()`
left the source permanently patched, which would have broken every *subsequent* `reproduce`/
`fetch` call's `require_clean=True` cleanliness check).
- `tests/test_llama_cpp_dependency.py`: added
`test_build_config_locks_an_explicit_cpu_only_deterministic_lane` (offline; asserts the lock's
`configure_flags` are CPU-only and that `ctest_regex`/`native_targets`/`smoke_binary` all agree
with each other and with `patched_paths`) and
`test_ctest_lane_raises_an_actionable_error_for_a_failing_named_test` (gated on `cmake`
availability via a `requires_cmake` marker mirroring `test_native_identity_emission.py`'s
`requires_cc` pattern; builds a tiny synthetic two-test CMake project — not the full llama.cpp
tree, so it runs in about a second — and proves `ctest_lane()` both passes silently on a passing
named test and raises `DependencyError` naming the failing test on a failing one).
## Toolchain note
Neither the ambient system Python nor `.venv-rocm` has `cmake`. This session installed `cmake`
(the PyPI wheel that bundles prebuilt binaries, version 4.4.0) into the pre-existing repo-root
`.venv` used by earlier DGR-024/DGR-026 sessions (`.venv/bin/cmake`, `.venv/bin/ctest`), which was
already on-disk from a prior session but had never had `cmake` installed into it. All commands
below were run with that `.venv/bin` prepended to `PATH`. This is the same "disposable venv for a
lightweight optional dependency" pattern DGR-024 used for `grpc`.
## Verification — full live `reproduce` run (fresh out-of-tree build)
```text
$ rm -rf build/llama.cpp/build
$ python3 scripts/llama_cpp_dependency.py reproduce
reused verified offline cache: .../build/llama.cpp/source
usage: .../build/llama.cpp/build/bin/llama-gguf-hash [options] GGUF_IN
Hash a GGUF file
options: ...
Test project .../build/llama.cpp/build
Start 27: test-meshnet-range-ownership
1/1 Test #27: test-meshnet-range-ownership ..... Passed 0.01 sec
100% tests passed out of 1
$ echo $?
0
```
Wall-clock: `real 2m16.227s` (fresh CPU compile of ggml/llama-common/llama plus the
`llama-gguf-hash` example and the `test-meshnet-range-ownership` fixture; no full `llama.cpp`
test suite or example set is built — only the two targets named in `native_targets`).
Post-run checks:
```text
$ ls build/llama.cpp/build/bin/*.so*
libggml-base.so libggml-base.so.0 libggml-base.so.0.16.0
libggml-cpu.so libggml-cpu.so.0 libggml-cpu.so.0.16.0
libggml.so libggml.so.0 libggml.so.0.16.0
libllama-common.so ... libllama.so ...
# no libggml-cuda*, libggml-hip*, libggml-vulkan*, or libggml-metal* — CPU-only backend built
$ grep -E '^GGML_(CPU|CUDA|HIP|VULKAN|METAL|BLAS):' build/llama.cpp/build/CMakeCache.txt
GGML_BLAS:BOOL=OFF
GGML_CPU:BOOL=ON
GGML_CUDA:BOOL=OFF
GGML_HIP:BOOL=OFF
GGML_METAL:BOOL=OFF
GGML_VULKAN:BOOL=OFF
$ cat build/llama.cpp/build/meshnet-build-metadata.json
{
"model_downloads": false,
"semantic_certification": false,
...
}
$ git -C build/llama.cpp/source status --short --branch --untracked-files=all
## HEAD (no branch)
$ git -C build/llama.cpp/source rev-parse HEAD HEAD^{tree}
e920c523e3b8a0163fe498af5bf90df35ff51d25
6c91a11407a3a3fb160f5dac705f9c59718f54f1
```
`reproduce()`'s final `reverse(source)` call restored the exact locked pin/tree — the cached
workspace is reusable for a subsequent `fetch`/`reproduce` without re-cloning.
## Verification — actionable toolchain failure (missing `cmake`)
```text
$ python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
$ env -i HOME="$HOME" PATH=/usr/bin:/bin python3 scripts/llama_cpp_dependency.py build \
--source-dir build/llama.cpp/source --build-dir /tmp/no-cmake-build
DGR-027 dependency error: cmake is unavailable; set CMAKE or activate the project toolchain
$ echo $?
2
$ python3 scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source # restore pristine
```
## Verification — targeted test suites and shared gates
| Command | Result |
| --- | --- |
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py` | `9 passed in 1.34s` (7 pre-existing + 2 new; the new gated CTest-wiring test ran for real, not skipped, since `cmake` is present in `.venv`) |
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py tests/test_ralph_prd_schema.py` | `117 passed` |
| `python3 -m compileall -q packages tests` | exit 0 |
| `git diff --check` | exit 0 (no output) |
## Ensuring build success does not advertise capability
- The locked `configure_flags` disable every accelerator backend explicitly
(`GGML_CUDA/HIP/VULKAN/METAL/BLAS=OFF`) rather than relying on per-platform defaults, so a
successful configure/build can only ever mean "the CPU reference backend compiled" — never an
accelerator claim, and never dependent on whether the build host happens to have a GPU SDK
installed.
- `meshnet-build-metadata.json` (written by `build()`) records `model_downloads: false` and
`semantic_certification: false` alongside the exact commit/patch/flag identities — the artifact
itself, not just prose, states this build proves toolchain compilation only.
- The two targets actually compiled are `llama-gguf-hash` (a stock upstream file-hashing utility;
no inference) and `test-meshnet-range-ownership` (a model-free fixture that writes a tiny
synthetic GGUF and asserts range-ownership bookkeeping — no real model, no generation, no
numerical/backend correctness claim). Neither exercises inference, MoE, attention, or any
DeepSeek V4 semantic path.
## Changed files
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- `.scratch/distributed-gguf-runtime/prd.json`
- `.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-029/README.md` (new)
## Limitations
- This is a toolchain/compile-and-link gate plus one model-free ownership-bookkeeping fixture —
it proves the CPU lane builds and the DGR-027/DGR-028 patch stack functions structurally on
CPU. It proves nothing about real-model correctness, memory-fit, performance, or any
backend/model/recipe certification; `stock_glm_limitations` in `UPSTREAM_LOCK.json` and DGR-028's
own limitations continue to apply unchanged.
- `cmake`/`ctest` are not installed system-wide or in `.venv-rocm` in this environment; they were
installed only into the pre-existing repo-root `.venv` for this session's verification (and for
the new gated pytest test, which is skipped in any environment lacking `cmake`). A future session
without that `.venv` (or without re-installing `cmake` into it) will see the same "cmake is
unavailable" actionable failure demonstrated above, not a silent pass.
- Only the two named targets are compiled (`llama-gguf-hash`, `test-meshnet-range-ownership`); a
broad `cmake --build ... --target test` / full upstream test suite is out of scope here, exactly
as DGR-028 recorded ("not presented as a full-suite gate").
- CUDA/ROCm/Vulkan/Metal compile lanes remain unimplemented; this story only establishes the CPU
lane "before accelerator matrix work," per its objective. Those lanes are separate future work.
## Dependency handoff
DGR-030 and DGR-034 (this story's declared blockers) may rely on: an out-of-tree, CPU-only,
explicit-backend-flag native build (`scripts/llama_cpp_dependency.py build`/`reproduce`) that
compiles the exact DGR-027/DGR-028 patched pin and runs a real CTest lane
(`test-meshnet-range-ownership`) proving the patch stack's range-ownership bookkeeping compiles
and passes on CPU. Any accelerator (CUDA/ROCm/Vulkan/Metal) lane, any real-model load, and any
backend/model/recipe capability certification remain unimplemented and must not be assumed from
this story's green build alone.

View File

@@ -0,0 +1,275 @@
# DGR-030 evidence — accelerator build presets and native CI/build matrix
**Status:** implementation complete, live-verified in this session (2026-07-23).
**Authority:** local `prd.json` is authoritative; Gitea is a projection.
**Upstream pin:** `e920c523e3b8a0163fe498af5bf90df35ff51d25` (`llama.cpp`, unchanged from DGR-027..029).
## What existed before this session
DGR-029 locked exactly one build lane — the deterministic CPU-only lane — in
`UPSTREAM_LOCK.json`'s `build` section, plus `scripts/llama_cpp_dependency.py`'s
`build()`/`smoke()`/`ctest_lane()`/`reproduce()`. There was no accelerator
preset, no SDK-availability probing, and no matrix runner: only the one CPU
lane existed, and there was no mechanism that could ever advertise a GPU
backend as compiled or capable.
## What changed in this session
- `packages/node/native/llama/UPSTREAM_LOCK.json`: added a new top-level
`accelerator_presets` object with one entry each for `cuda` (`GGML_CUDA`),
`rocm` (`GGML_HIP`), `vulkan` (`GGML_VULKAN`), and `metal` (`GGML_METAL`).
Each entry names only the one backend flag it flips and an `sdk_probe`
(a binary to resolve on `PATH`, an optional env-var override, and — for
Metal — a `platform_only: "darwin"` gate). **The existing `build` section
— the deterministic CPU default DGR-029 locked — is untouched.**
- `scripts/llama_cpp_dependency.py`:
- `_load_lock()` now calls a new `_verify_accelerator_presets()`, which
fail-closed-rejects any preset whose named backend flag is not `OFF` in
the CPU default's `configure_flags` — structurally guaranteeing a preset
can only ever *add* one backend on top of the untouched CPU baseline,
never redefine it.
- `accelerator_configure_flags(lock, name)` returns a **new** flag list —
the CPU default's own `configure_flags` list is never mutated — with
exactly the named preset's backend flag flipped `ON` and every other flag
(including `GGML_CPU=ON`, the fallback ops backend GPU builds still need)
left exactly as the CPU default declares it.
- `_sdk_probe(probe)` / `accelerator_status(name, lock)` resolve a lane's
SDK without ever raising: an absent SDK is returned as
`{"available": false, "reason": "<binary> is unavailable on PATH"}` (or
a platform-mismatch reason for Metal), so "unavailable" is data a caller
reports, never an exception a caller has to remember to catch.
- `accelerator_build(source, name, build_dir)` compiles one lane into its
own out-of-tree `build_dir` (an isolated directory, never DGR-029's CPU
`build_dir`), using the same patched-source verification and
`native_targets` as the CPU lane, then writes a
`meshnet-build-metadata.json` recording the exact `commit`/`commit_tree`,
per-patch SHA-256 digests, the lane's overridden `configure_flags`, the
resolved `cmake`/`cxx`/SDK-binary versions/paths, and explicit
`model_downloads: false`, `hardware_execution: false`,
`hardware_certified: false`, `semantic_certification: false` fields plus
a `note` stating the lane is registered-dark until a real-hardware
certification record exists. It **never** calls `smoke()`/`ctest_lane()`
— running a binary linked against a real accelerator backend would touch
real hardware, which this story deliberately keeps out of scope.
- Added `accelerator-status --name <lane>` and
`accelerator-build --name <lane> --source-dir --build-dir` CLI
subcommands, mirroring the existing `ctest`/`build` subcommand pattern.
- `scripts/native_accelerator_matrix.py` (new): the native CI/build matrix.
`run_matrix(workspace)` fetches and applies the locked pin/patch stack once,
runs the unchanged CPU lane (build → smoke → ctest, exactly DGR-029's
contract), then for each `accelerator_presets` entry either reports
`{"status": "skipped", "reason": ...}` (SDK absent) or compiles it via
`accelerator_build` and reports `{"status": "built", ...}` — never silently
treating a skip as a pass. Any `DependencyError` from a lane (CPU or
accelerator) is caught per-lane and reported as `{"status": "failed", ...}`
without aborting the remaining lanes or skipping cleanup. `reverse()` always
runs in a `finally`, restoring the exact pristine pin/tree regardless of
lane outcomes. The CLI prints a JSON report and exits non-zero only if any
lane actually `failed` (a `skipped` lane never fails the run).
- `tests/test_llama_cpp_dependency.py`: added 7 new tests —
`test_accelerator_presets_isolate_one_backend_without_touching_the_cpu_default`
(every preset flips exactly its own flag and the CPU default list is never
mutated), `test_accelerator_configure_flags_rejects_an_unknown_lane`,
`test_accelerator_status_reports_unavailable_sdks_without_raising` (asserts
the exact reason string for cuda/rocm/vulkan/metal absence),
`test_accelerator_status_honors_an_explicit_sdk_override`,
`test_accelerator_status_rejects_an_unknown_lane`,
`test_accelerator_build_refuses_to_compile_an_unavailable_lane` (asserts no
build directory is created), and a `requires_cmake`-gated
`test_accelerator_build_compiles_the_available_lane_with_isolated_evidence`,
which builds a tiny synthetic CMake project (not the full llama.cpp tree) to
prove `accelerator_build`'s "SDK present" path really configures with the
overridden flag, compiles, and writes the registered-dark metadata — in
about a second, without a real GPU SDK.
- `tests/test_native_accelerator_matrix.py` (new): 3 offline tests exercising
`run_matrix`'s orchestration with `llama_cpp_dependency`'s
fetch/apply/reverse/build/smoke/ctest_lane/accelerator_status/
accelerator_build stubbed out — proving unavailable SDKs are reported
`skipped` (never a false pass), an available accelerator lane is compiled
without ever calling `smoke`/`ctest_lane`, and a lane failure is reported
per-lane without aborting sibling lanes or skipping the `reverse()` cleanup.
## Toolchain note
As in DGR-029, neither the ambient system Python nor `.venv-rocm` has `cmake`;
this session's `.venv` also had no `cmake` (a prior session's install did not
persist). This session ran `.venv/bin/python3 -m ensurepip --upgrade` (no
`pip` was present in `.venv` either) and then
`.venv/bin/python3 -m pip install cmake`, landing the same PyPI wheel
(`cmake==4.4.0`) DGR-029 used, at `.venv/bin/cmake` / `.venv/bin/ctest`. All
commands below were run with that `.venv/bin` prepended to `PATH`. No CUDA,
ROCm, or Vulkan SDK (`nvcc`, `hipcc`, `glslc`) is installed in this
environment, and the host platform is Linux, not `darwin` — so all four
accelerator lanes are genuinely `skipped` in this environment's own live run
below, which is real evidence for AC2 ("unavailable SDKs ... explicit
unavailable/skipped lanes"), not a simulated one.
## Verification — live native CI/build matrix run
```text
$ rm -rf build/llama.cpp/build build/llama.cpp/build-cuda build/llama.cpp/build-rocm build/llama.cpp/build-vulkan build/llama.cpp/build-metal
$ python3 scripts/native_accelerator_matrix.py
reused verified offline cache: .../build/llama.cpp/source
usage: .../build/llama.cpp/build/bin/llama-gguf-hash [options] GGUF_IN
...
Test project .../build/llama.cpp/build
Start 27: test-meshnet-range-ownership
1/1 Test #27: test-meshnet-range-ownership ..... Passed 0.01 sec
100% tests passed out of 1
{
"failed_lanes": [],
"hardware_certified": false,
"lanes": [
{
"build_dir": ".../build/llama.cpp/build",
"lane": "cpu",
"metadata": {
"cmake": "cmake version 4.4.0",
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
"configure_flags": [
"-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_TESTS=ON",
"-DLLAMA_BUILD_EXAMPLES=ON", "-DLLAMA_BUILD_SERVER=OFF",
"-DLLAMA_BUILD_TOOLS=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_CURL=OFF",
"-DGGML_CPU=ON", "-DGGML_BLAS=OFF", "-DGGML_CUDA=OFF",
"-DGGML_HIP=OFF", "-DGGML_VULKAN=OFF", "-DGGML_METAL=OFF"
],
"cxx": "c++ (GCC) 15.2.1 20260123 (Red Hat 15.2.1-7)",
"model_downloads": false,
"patches": { "...": "... (5 entries, unchanged sha256 digests from DGR-029)" },
"semantic_certification": false
},
"status": "built"
},
{"lane": "cuda", "reason": "nvcc is unavailable on PATH", "status": "skipped"},
{"lane": "rocm", "reason": "hipcc is unavailable on PATH", "status": "skipped"},
{"lane": "vulkan", "reason": "glslc is unavailable on PATH", "status": "skipped"},
{"lane": "metal", "reason": "platform 'linux' is not 'darwin'", "status": "skipped"}
],
"note": "A `built` lane means it compiled with the exact recorded compiler/SDK/upstream-pin/patch-stack/build-option evidence — it never means an accelerator device was exercised. Every backend/model/recipe lane stays registered-dark until a separate real-hardware certification record exists."
}
$ echo $?
0
```
Wall-clock: `real 2m19.797s` — matches DGR-029's ~2m16s CPU-lane compile; no
accelerator lane actually compiled in this environment (all four SDKs are
genuinely absent), so this run's added cost over DGR-029's own CPU-only
`reproduce()` is just the four fast SDK probes.
Post-run checks (source checkout left pristine by the matrix's `reverse()`):
```text
$ git -C build/llama.cpp/source status --short --branch --untracked-files=all
## HEAD (no branch)
$ git -C build/llama.cpp/source rev-parse HEAD HEAD^{tree}
e920c523e3b8a0163fe498af5bf90df35ff51d25
6c91a11407a3a3fb160f5dac705f9c59718f54f1
$ ls build/llama.cpp/ | grep build
build
```
Only the CPU lane's `build/` directory was created — no `build-cuda`,
`build-rocm`, `build-vulkan`, or `build-metal` directory exists, because every
accelerator lane was genuinely skipped rather than attempted.
## Verification — targeted test suites and shared gates
| Command | Result |
| --- | --- |
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py tests/test_native_accelerator_matrix.py` | `19 passed` (9 pre-existing + 7 new accelerator-lane tests in `test_llama_cpp_dependency.py`, 3 new in `test_native_accelerator_matrix.py`; the `requires_cmake`-gated compile test ran for real, not skipped) |
| `python3 -m compileall -q packages tests` | exit 0 |
| `git diff --check -- packages/node/native/llama/UPSTREAM_LOCK.json scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py scripts/native_accelerator_matrix.py tests/test_native_accelerator_matrix.py` | exit 0 |
| `python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json` | `OK: 55 stories validated.` |
`git diff --check` against the full working tree separately reports one
pre-existing trailing-whitespace line in `.ralph-tui-run.log`, which was
already modified before this session started (see the session's initial
`git status`) and is unrelated to this story's scope; it is excluded above by
naming this story's own changed files explicitly.
`python3 -m pytest -q tests/test_ralph_prd_schema.py` reports `55 failed, 53
passed` in this session (all `test_render_issue_markdown_matches_committed_file`
drift between `prd.json` and committed issue Markdown for other stories,
e.g. `DGR-053`..`DGR-071`). `git stash`-ing this session's changes and rerunning
reproduces `56 failed, 52 passed` identically — the same 56 failures minus the
one this session's own `DGR-030` regeneration fixed, confirming the remaining
55 predate this story and are out of scope to fix here. This session did
regenerate `.scratch/distributed-gguf-runtime/issues/030-add-accelerator-
build-presets-and-native-ci-matrix.md` via
`python3 scripts/ralph_prd_schema.py render ... DGR-030` so DGR-030's own
generated issue Markdown matches `prd.json` byte-for-byte (confirmed by the
`test_render_issue_markdown_matches_committed_file[DGR-030]` case no longer
appearing in the failure list).
## Ensuring build success does not advertise capability
- Every accelerator lane's `meshnet-build-metadata.json` explicitly records
`hardware_execution: false`, `hardware_certified: false`, and
`semantic_certification: false`, plus a `note` stating the lane is
registered-dark until a separate real-hardware certification record exists
— the same "artifact states this, not just prose" pattern DGR-029 used for
the CPU lane's `model_downloads`/`semantic_certification` fields.
- `accelerator_build` never runs `smoke()` or `ctest_lane()`: it only
configures and compiles the exact `native_targets` DGR-029 already locked
(`llama-gguf-hash`, `test-meshnet-range-ownership`) — no binary linked
against a real accelerator backend is ever executed by this story's code.
- `_verify_accelerator_presets()` structurally refuses any preset whose
backend flag is not `OFF` in the locked CPU default, so a preset can never
be defined in a way that redefines (rather than adds one backend on top of)
DGR-029's deterministic CPU lane.
- The matrix's top-level report always carries `"hardware_certified": false`
regardless of how many lanes built, and its `note` field states this
explicitly for any consumer reading only the report, not the per-lane
metadata.
## Limitations
- This story proves accelerator lanes *compile* with correct, isolated
flags and preserves exact evidence when a lane's SDK is present. It proves
nothing about numerical correctness, performance, or any backend/model/
recipe capability on real accelerator hardware — that is explicitly
deferred to DGR-041 (capability registration), DGR-053 (real 2-4 stage
certification), and DGR-067 (capability matrix certification), all of which
remain unimplemented.
- No CUDA, ROCm, or Vulkan SDK, and no macOS/Metal toolchain, is available in
this session's environment, so the "compile an available accelerator lane"
path is proven end-to-end only via the `requires_cmake`-gated synthetic-
project unit test and the offline matrix-orchestration tests, not via a
live compile of the real llama.cpp tree under `GGML_CUDA=ON` (etc.). A
future session with a real SDK installed will exercise
`accelerator_build`'s real-lane path against the genuine llama.cpp source
for the first time; nothing in this story's design assumes that hasn't
happened yet.
- The accelerator lanes reuse the CPU lane's exact `native_targets`
(`llama-gguf-hash`, `test-meshnet-range-ownership`), so a passing
accelerator compile also proves the DGR-027/DGR-028 patch stack's
range-ownership code compiles under that backend flag combination — but,
per the point above, only structurally; it says nothing about GPU
execution correctness.
- `cmake`/`ctest` remain absent system-wide in this environment; this session
reinstalled them into `.venv` exactly as DGR-029 did, and that install does
not appear to persist across sessions (this session found `.venv` without
`cmake` despite DGR-029's evidence recording its earlier install). A future
session without a `cmake`-equipped `.venv` will see the same actionable
"cmake is unavailable" failure DGR-029 demonstrated, not a silent pass, and
the new `requires_cmake`-gated tests will be skipped rather than failing.
- `git diff --check` and `tests/test_ralph_prd_schema.py` both carry
pre-existing, out-of-scope failures unrelated to this story (see the gates
table above); this story's own changed files pass both checks cleanly.
## Dependency handoff
DGR-053 (real 2-4 stage certification), DGR-067 (capability matrix
certification), and DGR-068 (packaged releases) may rely on: four isolated,
out-of-tree accelerator build presets (`cuda`/`rocm`/`vulkan`/`metal`) in
`UPSTREAM_LOCK.json`'s `accelerator_presets`, each toggling exactly one
backend flag on top of DGR-029's unchanged CPU default; a native CI/build
matrix (`scripts/native_accelerator_matrix.py`) that compiles every
SDK-available lane with full compiler/SDK/upstream-pin/patch-stack/build-
option evidence and reports SDK-unavailable lanes as explicit `skipped`
lanes, never a false pass; and a compile-only contract (no lane here ever
runs a binary against real accelerator hardware). Real-hardware execution,
numerical correctness, performance measurement, and backend/model/recipe
certification for any accelerator remain entirely unimplemented and must not
be assumed from any lane's green compile.

View File

@@ -0,0 +1,237 @@
# DGR-031 evidence — the project-owned `ShardEngine` interface
**Completed:** 2026-07-23
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-021 (`evidence/DGR-021/README.md` — versioned activation
envelope, `NamedTensor`/`ActivationEnvelope` as the project-owned wire-envelope
layer), DGR-025 (`evidence/DGR-025/README.md` — exact artifact/runtime recipe
identity; both read before changing code).
## Objective
Isolate worker/protocol code from llama.cpp internals behind a stable
project-owned engine contract, so a fake fixture engine (DGR-032) and a real
llama.cpp-backed engine (DGR-037) are interchangeable subclasses of one
interface.
## What was found live before changing code
Per RALPH-CONTEXT, legacy pass states were not trusted; the live surrounding
contracts were read and exercised before designing this one:
- `packages/node/meshnet_node/shard_lifecycle.py` (DGR-022) already defines a
versioned RPC/session lifecycle contract — `StructuredStatus`, `StatusCode`,
`CacheExpectation`, `CacheResult`, `LifecycleState`, `SessionLifecycle` — but
it is explicitly the *wire RPC* contract "consumed by a future generated
gRPC binding," not an execution-engine boundary.
- `packages/node/meshnet_node/native_backend.py` (DGR-025) is the identity
boundary for the native GGUF artifact — it derives and attests a
`ShardIdentity`, but does not define an execution contract either.
- `packages/node/meshnet_node/protocol.py` (DGR-021) defines a project-owned
`NamedTensor`/`ActivationEnvelope` for activation traffic *between shard
hops over the network*, distinct from the generated-protobuf wire ABI in
`native_protocol`.
- `packages/node/meshnet_node/shard_runtime_server.py` (DGR-024) is today a
real gRPC servicer that proves wire fidelity by checksumming and echoing
bytes — it has no execution engine behind it yet; that seam is exactly
where `ShardEngine` plugs in for DGR-037.
- `packages/node/meshnet_node/architecture_boundary.py` established the
precedent this story follows for tail output: `TailOutput.sampled_token()`
never exposes raw logits, only a sampled token id.
- No `ShardEngine` (or `shard_engine`) symbol existed anywhere in the
repository prior to this story (confirmed by
`grep -rn -i "shardengine\|shard_engine"` across `.py`/`.md`, which returned
only planning-document prose naming it as future work).
Live verification of the pre-existing dependency contracts before adding new
code: `PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q
tests/test_shard_lifecycle.py tests/test_activation_envelope.py
tests/test_architecture_boundary.py tests/test_native_shard_protocol.py
tests/test_shard_runtime_harness.py``95 passed, 3 skipped`.
## What was added (this story's change)
### `packages/node/meshnet_node/shard_engine.py` (new)
The `ShardEngine` boundary: an `abc.ABC` with eight abstract operations —
`load`, `capabilities`, `prefill`, `decode`, `cancel`, `release`, `health`,
`metrics` — matching the acceptance criterion's list exactly (`prefill`/
`decode` share one operation family; their shared result type is what the
criterion calls the "boundary/logits result"). Every request/result type is a
frozen dataclass built from plain `str`/`int`/`bytes`/`Mapping` values:
- `EngineTensor` / `BoundaryBundle` — the project-owned named-tensor
activation crossing a shard boundary (head/middle/tail-in). Deliberately a
*new*, minimal type distinct from both `native_protocol.pb.TensorBundle`
(generated-protobuf ABI) and `protocol.NamedTensor`/`ActivationEnvelope`
(wire-framing/fragmentation concerns irrelevant to model execution) — a
fourth, execution-facing layer underneath the three that already existed.
- `TokenOutput` — a tail shard's sampled result: a token id (+ optional
decoded text), never a raw logits tensor.
- `MtpHook` — reserved multi-token-prediction hook; its own `__post_init__`
raises if constructed with `enabled=True`, so the type exists (fixing its
field shape for DGR-051/DGR-066) without any code path being able to turn it
on before DGR-066, matching RALPH-CONTEXT's "MTP is reserved and off for
alpha."
- `ArchitectureAuxStateHook` — reserved per-shard architecture auxiliary state
(V4 CSA/HCA/SWA/indexer/compressor and similar); has no wire encoding and is
never embedded in a `BoundaryBundle`, matching RALPH-CONTEXT's "remain local
... never carried over the WAN seam."
- `LoadRequest`/`LoadResult`, `EngineCapabilities`, `PrefillRequest`/
`DecodeRequest` (exactly one of `token_ids`/`token_id` (head) or `input`
(middle/tail) required — enforced in `__post_init__`), `StepResult` (a
successful result must carry an output; `cache_result` reuses
`shard_lifecycle.CacheResult`), `HealthResult`, `MetricsResult`.
- Status vocabulary is reused, not reinvented: `StructuredStatus`/
`StatusCode`/`CacheExpectation`/`CacheResult` are imported from
`shard_lifecycle` (already project-owned and version-stable) rather than a
parallel enum living alongside it.
- The module imports nothing from `native_protocol`, `grpc`, or `ctypes`
verified structurally, not just by convention (see tests below).
### `tests/shard_engine_contract.py` (new)
A reusable, non-`test_`-prefixed helper: `assert_shard_engine_contract(make_engine)`
takes a zero-arg engine factory and runs nine lifecycle checks — health before
load, load→capabilities range/MTP-off, prefill→decode determinism (byte-identical
output replayed on a fresh session), middle-shard boundary-bundle-in/out vs.
head/tail token-output, deterministic cache-miss on an unopened session,
stale-route-epoch rejection, cancel-then-decode rejection (+ cancel
idempotency), release-then-decode rejection (+ release idempotency), and
metrics reporting cancelled sessions. DGR-032's fixture and DGR-037's
llama.cpp binding are both expected to import this and pass it against their
own engine, proving identical lifecycle semantics without duplicating the
checks.
### `tests/test_shard_engine.py` (new)
- `_ReferenceEngine`: a minimal in-memory `ShardEngine` used only to prove the
shared contract is non-vacuous. It is explicitly *not* the DGR-032
deterministic fixture (no delay/memory-pressure/malformed/crash injection —
that is DGR-032's own, larger scope); the docstring says so to prevent this
story's evidence from being read as inherited completion credit for DGR-032.
- Dataclass validation tests: abstract-class instantiation refusal, tensor/
bundle/token-output field validation, MTP-hook enable refusal, exactly-one-
input-kind enforcement on `PrefillRequest`/`DecodeRequest`, `LoadRequest`
shard-range-vs-total-layers validation, `StepResult` output-required-on-OK.
- `test_shard_engine_module_imports_no_native_or_grpc_or_wire_abi_types`:
walks `vars(shard_engine_module)` and asserts no bound name's `__name__` is
`ctypes`, `grpc`, or `meshnet_node.native_protocol` — a structural check
(not a docstring-text grep, which produced a false positive on first draft
because the module's own docstring *names* `ggml_tensor` as an example of
what must never appear) that the ABI-isolation acceptance criterion holds.
### `.scratch/distributed-gguf-runtime/prd.json` / issue markdown
Marked `DGR-031.passes = true` with `completionNotes`; regenerated
`issues/031-introduce-the-project-owned-shardengine-interface.md` via
`scripts/ralph_prd_schema.py render` so it matches `prd.json` byte-for-byte.
## Acceptance criteria → evidence
1. **load/capabilities/prefill/decode/boundary-logits-result/cancel/release/
health/metrics** — `ShardEngine`'s eight abstract methods plus
`StepResult.output: BoundaryBundle | TokenOutput | None`. Verified by
`test_reference_engine_obeys_the_shared_shard_engine_contract` and the
middle-shard-vs-tail-shard assertion inside
`assert_shard_engine_contract`.
2. **No `ggml_tensor`/llama context/scheduler/ABI-owned structure** — every
type in `shard_engine.py` is a plain dataclass over `str`/`int`/`bytes`/
`Mapping`; no import of `native_protocol`, `grpc`, or `ctypes`. Verified by
`test_shard_engine_module_imports_no_native_or_grpc_or_wire_abi_types`.
3. **Reserved typed MTP/architecture-aux-state hooks, not enabled**
`MtpHook.__post_init__` raises on `enabled=True`; `ArchitectureAuxStateHook`
carries opaque shard-local state with no wire path. Verified by
`test_mtp_hook_is_reserved_and_refuses_to_enable` and
`test_architecture_aux_state_hook_carries_opaque_shard_local_state`, plus
`assert_shard_engine_contract`'s `caps.supports_mtp is False` check.
4. **Contract tests proving fake and future llama implementations obey
identical lifecycle semantics** — `tests/shard_engine_contract.py` is
written to be imported by DGR-032 and DGR-037 against their own engines;
`test_shard_engine.py` proves it is real by running it against
`_ReferenceEngine`.
5. **Gates + this handoff** — below.
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q tests/test_shard_engine.py
```
```text
12 passed in 0.13s
```
```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q \
tests/test_shard_engine.py tests/test_shard_lifecycle.py \
tests/test_architecture_boundary.py tests/test_activation_envelope.py \
tests/test_native_shard_protocol.py tests/test_shard_runtime_harness.py
```
```text
95 passed, 3 skipped in 3.65s
```
```bash
.venv/bin/python3 -m compileall packages/node/meshnet_node/shard_engine.py tests/shard_engine_contract.py tests/test_shard_engine.py
```
```text
Compiling 'packages/node/meshnet_node/shard_engine.py'...
Compiling 'tests/shard_engine_contract.py'...
Compiling 'tests/test_shard_engine.py'...
```
```bash
git diff --check
```
```text
(no output — clean)
```
## Limitations
- `tests/` as a whole does not collect cleanly in this environment: 27
pre-existing test modules fail to import for missing optional dependencies
(`cryptography`, etc.) unrelated to this story. Reproduced identically with
`git stash` before this session's change (`27 errors during collection`),
so this is pre-existing environment state, not a regression introduced
here. This story's own gates were run as the targeted, scoped test set
above per the shared quality gates' own wording ("Targeted deterministic
tests pass").
- The contract in `shard_engine_contract.py` proves *lifecycle* semantics
(gating, cache-miss/stale-epoch/cancel/release, boundary-vs-token output
shape) are identical across implementations. It does not — and cannot yet
— prove numerical parity between a fake and a real engine; that is
DGR-036's explicit job once DGR-032 and DGR-037 both exist.
- `_ReferenceEngine` in `test_shard_engine.py` is intentionally minimal
(no delay/memory-pressure/malformed-output/crash injection). DGR-032's
acceptance criteria require those independently; nothing here should be
read as satisfying them.
- No gRPC/CMake/native-build changes were needed or made — this story is
pure Python interface/type definition (`evidenceClass: model-free`,
`hardware: none`), so the native CMake/CTest and patch-stack gates in the
shared quality-gate list do not apply here (consistent with DGR-021/DGR-025,
which record the same non-applicability for non-native stories).
## Dependency handoff
- **DGR-032** (fake `ShardEngine`): subclass `ShardEngine`, add delay/memory-
pressure/malformed-output/crash injection, and pass the *same*
`assert_shard_engine_contract` from `tests/shard_engine_contract.py`
against it — no new contract vocabulary should be needed.
- **DGR-034/DGR-035** (range-aware GGUF ownership, boundary I/O): `LoadRequest`
already carries `shard_start`/`shard_end`/`total_layers`/`recipe`; `capabilities()`
reports the authoritative range via `EngineCapabilities.is_head`/`is_tail`.
`BoundaryBundle.token_id_sideband` is reserved for the first-three-hash-
routed-layers V4 requirement RALPH-CONTEXT documents.
- **DGR-037** (bind llama.cpp to the worker): implement `ShardEngine` as a
thin wrapper around the native artifact from `native_backend.py`/
`runtime_recipe.py`; `shard_runtime_server.py`'s `Session`/`GetCapability`/
`Health`/`Cancel`/`Release` handlers become the translation layer between
`pb.*` wire messages and this module's request/result types — this story
intentionally does not touch `shard_runtime_server.py` itself, since that
wiring is DGR-037's scope.
- **DGR-051** (V4 `ShardEngine` adapter): `MtpHook`/`ArchitectureAuxStateHook`
fix the field shape now so the V4 adapter does not need a breaking change
to enable MTP after DGR-066 or to carry CSA/HCA/SWA/indexer/compressor
state.

View File

@@ -0,0 +1,259 @@
# DGR-032 evidence — deterministic fake `ShardEngine`
**Completed:** 2026-07-23
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-031 (`evidence/DGR-031/README.md` — the project-owned
`ShardEngine` abstract contract, `tests/shard_engine_contract.py`'s
`assert_shard_engine_contract`, and its own dependency-handoff note that
DGR-032 should "subclass `ShardEngine`, add delay/memory-pressure/malformed-
output/crash injection, and pass the *same* `assert_shard_engine_contract`
... — no new contract vocabulary should be needed").
## Objective
Provide an engine fixture that deterministically transforms typed boundary
bundles and session state: head/middle/tail, prefill/decode, cancellation,
release, isolated per-session epoch state, deterministic cache-miss/stale-
epoch failures, and configurable delay/memory-pressure/malformed-output/
crash-injection fault surfaces — all without llama.cpp, a GPU, or any I/O.
## What was found live before changing code
- `packages/node/meshnet_node/shard_engine.py` (DGR-031): the abstract
`ShardEngine` with eight operations (`load`, `capabilities`, `prefill`,
`decode`, `cancel`, `release`, `health`, `metrics`) and its project-owned
dataclasses (`LoadRequest`, `EngineCapabilities`, `PrefillRequest`/
`DecodeRequest`, `StepResult`, `BoundaryBundle`/`EngineTensor`,
`TokenOutput`, `HealthResult`, `MetricsResult`).
- `tests/shard_engine_contract.py` (DGR-031): the reusable
`assert_shard_engine_contract(make_engine)` helper — nine lifecycle checks
any implementation must pass, explicitly designed to be imported by
DGR-032 and DGR-037 against their own engines.
- `tests/test_shard_engine.py` (DGR-031): its `_ReferenceEngine` is
explicitly documented as *not* the DGR-032 fixture ("no delay/memory-
pressure/malformed/crash injection... that is a separate, larger story") —
confirming this story starts from nothing, not inherited credit.
- `grep -rn -i "fakeshardengine\|fake_shard_engine"` across `.py`/`.md`
returned no prior matches — no fake engine existed before this story.
- No file in `packages/node/meshnet_node/` wires a `ShardEngine` into
`shard_runtime_server.py` yet (confirmed by grep for `ShardEngine`/
`shard_engine` in that file — no matches); that wiring is DGR-037's scope,
so this fixture is a standalone, importable engine only.
Live verification of the pre-existing dependency contract before adding new
code:
```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q tests/test_shard_engine.py
```
```text
12 passed in 0.13s
```
## What was added (this story's change)
### `packages/node/meshnet_node/fake_shard_engine.py` (new)
`FakeShardEngine(ShardEngine)` — a pure-Python, deterministic fixture:
- **Determinism.** Every `prefill`/`decode` output is `SHA-256(seed_bytes +
idempotency_step)`, where `seed_bytes` is derived from `token_ids` (head)
or the input `BoundaryBundle`'s tensor bytes plus any `token_id_sideband`
(middle/tail-in). Replaying identical inputs on a brand-new session
produces byte-identical output — proven by
`assert_shard_engine_contract`'s own determinism check and reused directly.
- **Head/middle/tail.** Tail shards (`shard_end >= total_layers - 1`) return
a `TokenOutput` sampled into `[0, TOKEN_ID_VOCAB_SIZE)`; head/middle shards
return a `BoundaryBundle` tagged `boundary_point="post_head_residual"` or
`"post_middle_residual"` respectively, so the three cases are
distinguishable in fixture output, not just in the load request. A middle
shard's `token_id_sideband` passes through unchanged from its input bundle
to its output bundle (the V4 first-three-hash-routed-layers requirement
RALPH-CONTEXT documents), never invented or dropped.
- **Isolated session/epoch state.** `_sessions: dict[str, _SessionState]`
keyed by `session_id`; each session tracks its own `epoch`/`cancelled`
flag. A stale epoch, cancel, or release on one session never touches
another's state (`test_session_state_is_isolated_between_two_concurrent_sessions`
proves a stale-epoch rejection and a cancel on session `"a"` leave session
`"b"` fully serviceable). Decoding an unopened session is a deterministic
`NOT_FOUND`/`CacheResult.MISS`, not an exception.
- **Configurable delay.** `FakeShardEngineConfig.step_delay_seconds` +
injectable `sleep` hook (defaults to `time.sleep`, overridable in tests so
they don't block wall-clock time) — invoked once per `prefill`/`decode`
call before computing the deterministic output.
- **Configurable memory pressure.** `FakeShardEngineConfig.memory_budget_bytes`
— the engine accumulates `_bytes_used` across every step's seed bytes;
once a step would push cumulative usage past the budget, that step
deterministically returns `StatusCode.RESOURCE_EXHAUSTED` (`retryable=True`)
with no output, instead of computing one.
- **Configurable malformed output.** `FakeShardEngineConfig.malformed_output`
— when set, the engine still reports `StatusCode.OK` (the point is a
buggy-but-"successful"-looking response, not a status-coded failure) but
the payload is structurally valid, semantically wrong: a tail `TokenOutput`
is pushed past `MALFORMED_TOKEN_ID_FLOOR` (outside the fixture's own
advertised vocab), and a head/middle `BoundaryBundle` gets an
`architecture` field prefixed `"malformed:"` and its tensor `data`
truncated to one byte — both structurally valid per `EngineTensor`'s and
`BoundaryBundle`'s own `__post_init__` validation (which does not
cross-check `data` length against `shape`/`dtype`), so a consumer must
actually check shape/semantics, not just status codes, to catch it.
- **Configurable crash injection.** `FakeShardEngineConfig.crash_after_calls`
+ `crash_exception_factory` — after the configured number of
`prefill`/`decode` calls, the engine raises an arbitrary exception (default
`RuntimeError`, injectable) directly out of the call instead of returning a
`StepResult`. This is deliberately *not* wrapped in `EngineError`/
`StructuredStatus`: it simulates a whole-process failure (what a worker
supervisor — DGR-040 — must catch and restart around), which is a
different failure mode from a graceful status-coded rejection.
- **Fixture-vs-real marker.** `FakeShardEngine.EVIDENCE_CLASS = "fixture"` —
a structural constant (not just docstring prose) so DGR-036's fixture-vs-
real-model parity check can assert programmatically that it is comparing a
fixture engine against a real one, never two fixtures.
- Every fault-injection knob defaults to off (`0`/`None`/`False`), so a bare
`FakeShardEngine()` passes `assert_shard_engine_contract` unmodified —
fault injection is opt-in, never a baseline behavior change.
### `tests/test_fake_shard_engine.py` (new)
- `test_fake_shard_engine_obeys_the_shared_shard_engine_contract` — runs the
full DGR-031 contract against a bare `FakeShardEngine`.
- `test_fake_shard_engine_declares_fixture_evidence_class` — pins the
`EVIDENCE_CLASS` marker DGR-036 will rely on.
- Head/middle/tail output-shape tests (`boundary_point`, token-id-sideband
pass-through, tail vocab range).
- `test_session_state_is_isolated_between_two_concurrent_sessions` — a
stale-epoch rejection and a cancel on one session leave a second,
concurrently open session fully serviceable.
- One test per fault-injection knob (delay hook invocation, memory-budget
trip, malformed tail/boundary-bundle output, crash-after-N-calls,
configurable crash exception type) plus `FakeShardEngineConfig`'s own
`__post_init__` validation (negative delay, negative budget, non-positive
`crash_after_calls`).
- `test_load_result_and_capabilities_report_recipe_architecture` — the
fixture threads `LoadRequest.recipe["architecture"]` through to both
`LoadResult.architecture` and `EngineCapabilities.architecture` rather than
hardcoding `"dense"`/`"fake"` everywhere, so a future V4 recipe is visible
in fixture output too.
### `.scratch/distributed-gguf-runtime/prd.json` / issue markdown
Marked `DGR-032.passes = true` with `completionNotes`; regenerated
`issues/032-implement-deterministic-fake-shardengine.md` via
`scripts/ralph_prd_schema.py render` so it matches `prd.json` byte-for-byte.
## Acceptance criteria → evidence
1. **Head, middle, tail, prefill, decode, cancellation, release with
deterministic outputs** — `FakeShardEngine`'s `_transform`, boundary-point
tagging, and `assert_shard_engine_contract`'s own determinism/cancel/
release checks. Verified by
`test_fake_shard_engine_obeys_the_shared_shard_engine_contract`,
`test_head_shard_returns_boundary_bundle_with_post_head_residual_point`,
`test_middle_shard_returns_boundary_bundle_and_passes_through_token_sideband`,
`test_tail_shard_returns_token_output_within_advertised_vocab`.
2. **Isolated session/epoch state and deterministic cache-miss/stale-epoch
failures** — `_sessions` dict keyed per session;
`test_session_state_is_isolated_between_two_concurrent_sessions` plus the
shared contract's own cache-miss/stale-epoch checks.
3. **Configurable delay, memory pressure, malformed output, crash
injection** — `FakeShardEngineConfig`; verified by
`test_step_delay_seconds_invokes_the_configured_sleep_hook`,
`test_memory_budget_bytes_trips_deterministic_resource_exhausted`,
`test_malformed_output_is_structurally_valid_but_semantically_wrong_for_tail`,
`test_malformed_output_is_structurally_valid_but_semantically_wrong_for_boundary_bundle`,
`test_crash_after_calls_raises_instead_of_returning_a_structured_status`,
`test_crash_exception_factory_is_configurable`,
`test_config_rejects_invalid_knob_values`.
4. **Contract tests distinguish fixture evidence from real-model
certification** — module docstring and this README are explicit that
this is FIXTURE evidence only (numeric parity is DGR-036 onward); the
`EVIDENCE_CLASS = "fixture"` constant makes that distinction structurally
checkable, not just prose, pinned by
`test_fake_shard_engine_declares_fixture_evidence_class`.
5. **Gates + this handoff** — below.
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q tests/test_fake_shard_engine.py tests/test_shard_engine.py
```
```text
26 passed in 0.17s
```
```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q \
tests/test_fake_shard_engine.py tests/test_shard_engine.py tests/test_shard_lifecycle.py \
tests/test_architecture_boundary.py tests/test_activation_envelope.py \
tests/test_native_shard_protocol.py tests/test_shard_runtime_harness.py
```
```text
109 passed, 3 skipped in 3.78s
```
```bash
.venv/bin/python3 -m compileall -q packages tests
```
```text
(no output — clean; exit 0)
```
```bash
git diff --check
```
```text
(no output — clean)
```
## Limitations
- `tests/` as a whole does not collect cleanly in this environment: the same
pre-existing collection errors DGR-031's evidence recorded (missing
optional dependencies such as `cryptography`) are still present and are
unrelated to this story. This story's own gates were run as the targeted,
scoped test set above per the shared quality gates' wording ("Targeted
deterministic tests pass").
- This is FIXTURE evidence only. `FakeShardEngine` proves lifecycle,
session/epoch isolation, and fault-injection semantics; it proves nothing
about numerical parity with a real model. That is DGR-036's explicit job
once DGR-037's real engine exists, and DGR-053/054 for V4 alpha
certification.
- `FakeShardEngine` is not wired into `shard_runtime_server.py` or any gRPC
surface — it is a standalone, importable engine only. Wiring a
`ShardEngine` (fake or real) into the gRPC servicer is DGR-037's scope for
the real engine; DGR-033 covers a C++ worker surface, which is a separate
native executable, not a consumer of this Python module.
- No gRPC/CMake/native-build changes were needed or made — this story is
pure Python fixture code (`evidenceClass: fixture`, `hardware: none`), so
the native CMake/CTest and patch-stack gates in the shared quality-gate
list do not apply here, consistent with DGR-031's own README recording the
same non-applicability.
## Dependency handoff
- **DGR-033** (standalone fake C++ gRPC Shard worker): its own issue
describes a native C++ executable serving the lifecycle/stream RPC
contract "using the fake engine" — that is a native analogue, not a
consumer of this Python module; DGR-033 should still read this README for
the exact deterministic-output/session-isolation/fault-injection semantics
its C++ fake engine needs to reproduce so both fakes behave identically
from a client's point of view.
- **DGR-034/DGR-035** (range-aware GGUF ownership, boundary I/O):
`FakeShardEngine` already demonstrates range-driven head/middle/tail
behavior purely from `LoadRequest.shard_start`/`shard_end`/`total_layers`;
no new range vocabulary was needed.
- **DGR-036** (fixture vs real-model parity): compare a `FakeShardEngine`
instance's `EVIDENCE_CLASS` (`"fixture"`) against DGR-037's real engine's
equivalent marker (expected `"real"`) to assert the parity check is
actually comparing two different implementations; reuse
`assert_shard_engine_contract` against both to prove lifecycle parity
before attempting numeric parity.
- **DGR-037** (bind llama.cpp to the worker): `FakeShardEngine` is the
reference implementation to diff a real engine's lifecycle behavior
against — same request/result types, same session/epoch model, no new
contract vocabulary.
- **DGR-040** (worker supervision): the crash-injection knob
(`crash_after_calls`/`crash_exception_factory`) exists specifically so
supervision/restart logic has a deterministic way to trigger and test an
unhandled engine failure distinct from a graceful `StructuredStatus`
rejection.

View File

@@ -0,0 +1,281 @@
# DGR-033 evidence — standalone fake C++ gRPC Shard worker
**Completed:** 2026-07-25 (initial); **repaired:** 2026-07-26 after Codex
GPT-5.5 cross-review BLOCK (see "Cross-review repair" below).
**Branch:** `ralph/distributed-gguf-opus`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-022 (lifecycle/status contract), DGR-024 (real generated
gRPC harness + `shard_runtime_server.py` reference semantics), DGR-032
(deterministic fake `ShardEngine` semantics).
## Objective
Prove the standalone worker process, stream, lifecycle, and supervision shape
before any llama.cpp integration: a real C++ executable that serves the whole
ShardRuntime lifecycle/stream contract over gRPC using a model-free fake engine,
driven end-to-end by Python integration tests over a real socket.
## What was found live before changing code
- `packages/node/native/proto/shard_runtime.proto` (DGR-021..023): the single
semantic contract. Its `ShardRuntime` service has exactly five RPCs —
`GetCapability`, `Health`, `Session` (bidi stream), `Release`, `Cancel`.
- `packages/node/meshnet_node/shard_runtime_server.py` (DGR-024): the reference
Python servicer. It performs a *bounded real forward* (a CRC over the received
bundle bytes) then echoes the chunk, and fails closed on stale epoch, expired
deadline, corrupt/mis-tiled fragments, exhausted flow-control credit, duplicate
idempotency step, and in-band/out-of-band cancellation, with per-`route_session_id`
state kept on the servicer so an out-of-band `Cancel` can reach a live session.
**Key finding:** despite the schema labelling the checksum `CRC32C`, this
runtime computes it with `zlib.crc32` (standard CRC-32, *not* Castagnoli). The
C++ worker mirrors `zlib.crc32` exactly so its checksum acceptance is
byte-identical to the existing Python surface (the committed C++ *conformance*
test, by contrast, uses true Castagnoli against separately-generated goldens —
the two are unrelated code paths).
- `packages/node/native/CMakeLists.txt` (DGR-029/030): configures against the
ignored `build/native-toolchain` prefix (pinned Protobuf 33.1 + gRPC 1.82.1),
always generates both message and service stubs, and registers a C++
conformance CTest. There was **no** worker executable and **no** Python
worker integration test before this story (confirmed by
`ls packages/node/native/worker` → absent, and grep for `shard_worker`).
- `packages/node/meshnet_node/fake_shard_engine.py` (DGR-032): the Python fake
engine, deliberately *not* wired into the gRPC surface. DGR-033's worker is
its native analogue — a separate executable, not a consumer of that module —
so both fakes present identical behaviour to a client (deterministic,
model-free bounded forward; per-session isolation; fail-closed lifecycle).
## What was added (this story's change)
### `packages/node/native/worker/fake_engine.h` (new)
`meshnet::worker::FakeShardEngine` — a header-only, model-free fixture engine.
Its only capability is to validate a `TensorBundle` (fragments tile exactly, the
uncompressed CRC-32 matches the declared checksum, the declared payload stays
within the negotiated `max_chunk_bytes`) and fold the fragment bytes through a
bounded forward. It links, loads, and dispatches to **nothing** — no llama.cpp,
no graph execution. Carries `kEvidenceClass = "fixture"` mirroring the Python
`FakeShardEngine.EVIDENCE_CLASS` for the later DGR-036 parity check.
### `packages/node/native/worker/shard_service.{h,cpp}` (new)
`ShardRuntimeServiceImpl : meshnet::shard::v1::ShardRuntime::Service` — a faithful
C++ port of the DGR-024 Python servicer: the same per-`route_session_id`
identity/credit/dedup state guarded by a mutex, the same fail-closed negative
paths, and the same lifecycle (open → prefill/decode → flow-control top-up →
release/cancel). Each per-request response is computed under the lock and written
*after* releasing it, so a blocking `Write` can never deadlock the out-of-band
`Cancel` RPC that needs the same lock. Bounded messages are enforced two ways: a
per-tensor `RESOURCE_EXHAUSTED` app check against `max_chunk_bytes`, plus a hard
transport receive ceiling.
### `packages/node/native/worker/shard_worker_main.cpp` (new)
The standalone `shard_worker` executable. Binds `MESHNET_SHARD_LISTEN_ADDR`
(or an `argv` address), prints one readiness line (`ShardRuntime worker listening
on <addr>`), and serves until `SIGTERM`/`SIGINT`. **Graceful shutdown** uses a
self-pipe: the async-signal-safe handler writes one byte, a drain thread reads it
and calls `server->Shutdown()`, so in-flight sessions finish and the process
exits `0` printing `ShardRuntime worker shut down cleanly`. A `--selftest` mode
binds an ephemeral port and self-drives capability/health/fragmented-prefill/
decode/release over a real loopback gRPC channel, giving a pure-C++ CTest that
needs no Python.
### `packages/node/native/CMakeLists.txt` (modified)
Adds the `shard_worker` executable (linking only `shard_runtime_grpc` +
`gRPC::grpc++` — no llama.cpp) and registers `shard_worker_selftest` as a CTest.
### `tests/test_native_shard_worker.py` (new)
18 integration tests that spawn the **real compiled binary** as a subprocess and
drive it with the committed generated stubs over a real localhost socket. When
the binary is not built they skip (the DGR-029/030 `requires_cmake` gating
pattern), locating it via `MESHNET_SHARD_WORKER_BIN` or `build/native/shard_worker`.
## Acceptance criteria → evidence
1. **Standalone C++ executable serves the complete lifecycle/stream contract
using the fake engine** — `shard_worker` builds and serves all five RPCs; the
`shard_worker_selftest` CTest drives open → fragmented prefill → decode →
release over real gRPC; the 18 Python tests cover the same against the
subprocess.
2. **Python integration tests cover startup, health, capability, fragmented
prefill, decode, release, cancellation, graceful shutdown** —
`test_worker_startup_and_health`, `test_worker_capability`,
`test_fragmented_prefill_echoes_reassembled_payload` (3-fragment tiling),
`test_decode_step_is_served`, `test_release_is_terminal`,
`test_in_band_cancel_of_single_work_item_does_not_end_stream`,
`test_in_band_cancel_of_whole_session_is_terminal`,
`test_out_of_band_cancel_rpc_races_ahead_of_open`,
`test_graceful_shutdown_on_sigterm` (SIGTERM → exit 0 + clean-shutdown line).
3. **Bounded messages, deadlines, flow control, independent session
cancellation enforced** — `test_bounded_message_is_rejected`
(`RESOURCE_EXHAUSTED` on an over-ceiling tensor),
`test_expired_deadline_is_rejected`, `test_flow_control_violation_and_topup`,
`test_independent_session_cancellation` (cancelling session A leaves session B
fully serviceable), plus `test_stale_route_epoch_is_rejected`,
`test_duplicate_idempotency_step_is_acked`,
`test_malformed_fragment_tiling_is_rejected`.
4. **Exposes neither llama.cpp RPC nor arbitrary graph execution**
`ldd build/native/shard_worker` shows no llama/ggml shared libs;
`nm -C build/native/shard_worker | grep -icE 'llama_|ggml_'``0`; the proto
exposes exactly one service with five lifecycle RPCs and no graph-exec entry.
5. **Gates + this handoff** — below.
## Commands and results
Toolchain (ignored `build/native-toolchain`, pinned Protobuf 33.1 + gRPC 1.82.1):
```bash
bash scripts/bootstrap_native_toolchain.sh "$PWD/build/native-toolchain"
# ... gRPC 1.82.1 commit acccf84c0df20487d64101f528e5d426541ca4e5
# grpc_cpp_plugin sha256 43705cf26ae9ce98bbcee76b3408f5e171eec746b50bf0dd42dd68d132c6a533
```
Focused out-of-tree CMake build + CTest:
```bash
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain"
cmake --build build/native -j"$(nproc)"
ctest --test-dir build/native --output-on-failure
```
```text
1/2 Test #1: shard_worker_selftest ............ Passed 0.01 sec
2/2 Test #2: shard_protocol_conformance ....... Passed 0.00 sec
100% tests passed out of 2
```
Python integration tests against the real binary:
```bash
PYTHONPATH=packages/node:packages/tracker python -m pytest -q tests/test_native_shard_worker.py
```
```text
18 passed in 3.96s
```
AC4 (no llama.cpp / no graph exec):
```bash
ldd build/native/shard_worker | grep -iE 'llama|ggml' # -> (no matches)
nm build/native/shard_worker | grep -icE 'llama_|ggml_' # -> 0
```
Shared gates + regression:
```bash
python -m compileall -q packages tests # exit 0
git diff --check -- packages/node/native tests/test_native_shard_worker.py # exit 0
PYTHONPATH=packages/node:packages/tracker python -m pytest -q \
tests/test_shard_runtime_harness.py tests/test_native_shard_protocol.py
# -> 61 passed, 2 skipped (DGR-024 harness + native protocol untouched)
```
Toolchain used: `cmake`/`ctest` from the `distributed-gguf-runtime` worktree's
`.venv` (PyPI `cmake==4.4.0` wheel — no system cmake exists here, same as
DGR-029/030); the Python client uses that venv's `grpcio==1.82.1`,
`grpcio-tools==1.82.1`, `protobuf`, `pytest`. `g++ (GCC) 15.2.1`.
## Limitations
- This is FIXTURE evidence only. The worker's "forward" is a CRC-over-wire-bytes
echo, not real tensor compute; it proves process/stream/lifecycle/supervision
shape, nothing about numerical correctness. Real engine binding is DGR-037 and
numeric parity is DGR-036/052.
- The worker checksum path mirrors the DGR-024 runtime's `zlib.crc32` (standard
CRC-32 under a `CRC32C` label). Compressed-tensor tiling/checksum is not
independently verified (no zstd decompressor in the fixture) — identical to the
DGR-024 limitation.
- Default `pytest` runs skip `tests/test_native_shard_worker.py` unless the
worker binary is built (or `MESHNET_SHARD_WORKER_BIN` is set); this session
built it and ran all 18 for real (results above). Building requires the pinned
gRPC C++ toolchain, which is not present by default and must be bootstrapped.
- No CUDA/ROCm/GPU, no model download, no network at test time — all default
tests are fixture-only and offline.
## Dependency handoff
- **DGR-036** (fixture vs real-model parity): the worker's `FakeShardEngine`
carries `kEvidenceClass = "fixture"`; diff it against DGR-037's real engine's
equivalent marker, and reuse the same lifecycle/stream contract this worker
serves to prove behavioural parity before numeric parity.
- **DGR-037** (bind llama.cpp): replace `FakeShardEngine`'s bounded forward with
the real engine behind the *same* `ShardRuntimeServiceImpl` surface; the
service's session/epoch/credit/dedup/cancel machinery and the graceful-shutdown
supervision shape are reusable as-is.
- **DGR-040** (worker supervision): `shard_worker` already provides the
supervision primitives — a readiness line for start detection, `SIGTERM`
graceful drain with a clean-exit line, and a `--selftest` liveness probe.
A supervisor can start/monitor/restart the process around these.
## Cross-review repair (2026-07-26)
An independent Codex GPT-5.5 review BLOCKED the initial implementation. Four
root protocol defects in the native worker were fixed in this worktree
(`.claude/worktrees/distributed-gguf-opus`); the fake-engine echo semantics and
supervision shape are unchanged.
### Defects fixed
1. **Activation before SessionOpen bypassed all state.** A chunk/decode whose
`route_session_id` had no opened session fell through every `if (state && ...)`
guard and was echoed — bypassing lifecycle, cancellation, epoch and
flow-control. `SessionState` now carries an `opened` flag set only by a valid
`SessionOpen`; chunk and decode fail closed with a terminal
`ERROR_CODE_INTERNAL` and end the stream when it is false. A placeholder state
created by an out-of-band `Cancel` that races `Open` has `opened == false`, so
it can never admit work either.
2. **Flow control blindly trusted the peer proposal.** `SessionOpen` copied the
proposed `credits/max_inflight/max_chunk_bytes` verbatim into session state and
the accepted reply. New `ShardRuntimeServiceImpl::NegotiateFlow` takes the
strictest bound of peer-vs-worker for every field (mirroring
`negotiate_flow_control` in `native_protocol/codec.py`), stores the negotiated
ceilings on the session, and enforces the negotiated per-session
`max_chunk_bytes` on every bundle (`FakeShardEngine::Validate` now takes the
ceiling as an argument instead of a fixed construction-time value).
3. **In-stream `ReleaseSignal` leaked session state.** The stream `release` arm
wrote a terminal status but never dropped the session. It now erases the
session under the lock before responding, so KV/credits/dedup are freed
immediately (the out-of-band `Release` RPC already erased).
4. **`SessionOpen` echoed caller identity instead of validating it.** The handshake
now rejects an incompatible `schema_version` (`SCHEMA_UNSUPPORTED`), a
mismatched model/recipe `Fingerprint` (`FINGERPRINT_MISMATCH`), and a
`ShardRange` outside the worker's served range (`SHARD_RANGE_MISMATCH`), each
terminal; `SessionAccepted` now reports the worker's own served fingerprint
rather than a copy of the caller's.
### Changed files (repair)
- `packages/node/native/worker/shard_service.h``opened` +
`max_prefill_chunk_tokens` on `SessionState`; `NegotiateFlow` decl; engine now
default-constructed.
- `packages/node/native/worker/shard_service.cpp` — worker-identity constants +
fill helpers; `NegotiateFlow`; `SessionOpen` validation/negotiation; fail-closed
chunk/decode; per-session `max_chunk_bytes`; in-stream release erase.
- `packages/node/native/worker/fake_engine.h``Validate(bundle, max_chunk_bytes)`.
- `tests/test_native_shard_worker.py` — extended `_open` (schema/fingerprint/range/
flow overrides); fixed `test_release_rpc_is_idempotent` for the new erase
semantics; added 9 regression tests (chunk/decode before open, flow-control
clamp, negotiated-ceiling cap, in-stream release erase, schema/fingerprint/range
rejection, worker-fingerprint-not-caller).
### Re-run gates (real, rebuilt binary)
Build driven through the pinned `cmake` (Unix Makefiles + `gmake`, gRPC 1.82.1):
```text
cmake --build build/native --parallel 8 -> BUILD_EXIT 0
ctest --test-dir build/native --output-on-failure -> 100% (2/2) passed
shard_worker_selftest ....... Passed
shard_protocol_conformance .. Passed
python -m pytest -q tests/test_native_shard_worker.py -> 27 passed
python -m pytest -q tests/test_shard_runtime_harness.py \
tests/test_native_shard_protocol.py -> 63 passed
python -m compileall -q packages tests -> exit 0
git diff --check -> clean
ldd build/native/shard_worker | grep -iE 'llama|ggml' -> NONE
nm -C build/native/shard_worker | grep -cE 'llama_|ggml_' -> 0
```
The worker integration suite grew from 18 to 27 tests; all pass against the
freshly compiled binary. No `.ralph-lane` runtime artifacts were touched.

View File

@@ -0,0 +1,94 @@
# DGR-034 evidence — dense-Llama range-aware GGUF ownership
**Status:** implemented and live-verified on 2026-08-01. `prd.json` remains
the authority for story state.
## What changed
- The pinned llama.cpp patch stack adds `meshnet_owned_layer_start/end` and
filters dense-Llama GGUF registration to `blk.N.*` for the requested
half-open range. `token_embd.weight` belongs to the head; `output_norm` and
`output.weight` (or the tied embedding) belong to the tail.
- The load state exposes a C range report derived from the registered model
buffers, and a project-owned `meshnet-range-report` tool audits the live
registered tensor map. It rejects empty, inverted, out-of-model, missing,
outside-range, unexpected, and endpoint-inconsistent loads.
- `meshnet_node.range_report` accepts only audited tool output. It makes the
range and endpoint flags authoritative from loaded state rather than caller
assertions, and fails closed on malformed ownership or byte counts.
## Real-model memory evidence
Artifact: `Magistral-Small-2509-Q4_K_M.gguf`, 14,333,911,104 bytes, SHA-256
`a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc`.
It stayed on the configured mounted drive; no artifact was downloaded or put
under `/home`.
The direct non-mmap lane proves resident storage tracks owned tensors:
| Range | Registered tensors | Resident bytes | Process peak RSS |
| --- | ---: | ---: | ---: |
| `[10, 20)` | 90 | 3,304,898,560 | 3,298,800 KiB |
| `[0, 40)` | 363 | 14,326,026,240 | 14,061,632 KiB |
Raw reports and timings are in `runs/default-mid-a.*` and
`runs/default-full-nommap.*`. The middle range is 23.1% of the full
resident allocation and owns 24.8% of the registered tensors.
## Commands and results
```text
python3 scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
python3 scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# apply/check/reverse succeeded against e920c523e3b8a0163fe498af5bf90df35ff51d25;
# the source was then applied for the focused native checks.
(cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS)
# all six patches: OK
/home/popov/.hermes/hermes-agent/venv/bin/ctest \
--test-dir build/llama.cpp/dgr034-check \
-R '^test-meshnet-range-ownership$' --output-on-failure
# 1/1 passed
PYTHONPATH=packages/node MESHNET_RANGE_REPORT_BIN="$PWD/build/llama.cpp/dgr034-check/bin/meshnet-range-report" \
/home/popov/.hermes/hermes-agent/venv/bin/pytest -q \
tests/test_range_report.py tests/test_meshnet_range_report_tool.py \
tests/test_llama_cpp_dependency.py
# 56 passed in 0.87s
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python \
-m compileall -q packages tests
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
git diff --check && git diff --cached --check
# all exit 0; PRD validation: 55 stories validated
```
The model commands used the same `meshnet-range-report` binary with
`--no-mmap --no-extra-bufts`, first for `[10,20)` and then `[0,40)`; both
returned `ok: true` and their exact output is retained above.
## Changed files
- `packages/node/native/llama/PATCH-STACK.md`
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/patches/{series,SHA256SUMS,UPSTREAM-ASSUMPTIONS.json,0006-meshnet-range-report-tool.patch}`
- `packages/node/meshnet_node/range_report.py`
- `tests/test_range_report.py`
- `tests/test_meshnet_range_report_tool.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-034/*`
## Limitations and dependency handoff
- The mmap loader can retain broad contiguous file spans when GGUF tensor
order places a tail endpoint near the beginning of the artifact; the direct
non-mmap lane is the certified resident-memory result. The raw mmap report
is retained in `runs/default-head.json` and must not be presented as a
physical-RSS saving.
- This story proves loading/ownership only. Partial-range graph execution
remains fail-closed until DGR-035 provides typed dense boundary adapters.
- DGR-037 can bind the worker to `llama_model_meshnet_range_report` or the
strict Python consumer; it must use the reported range, not requested range,
for capability publication. DGR-051 must add its V4-specific ownership
rules separately.

View File

@@ -0,0 +1 @@
a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc Magistral-Small-2509-Q4_K_M.gguf

View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 40],
"reported_range": [0, 40],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": true,
"has_output_head": true,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 14326026240,
"registered_tensors": 363,
"registered_bytes": 14326026240,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 14392061952,
"vm_rss_bytes": 14387003392,
"vm_hwm_bytes": 14399111168
}

View File

@@ -0,0 +1 @@
elapsed=0:02.48 maxrss_kib=14061632 exit=0

View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 10],
"reported_range": [0, 10],
"mmap": true,
"touched": false,
"use_extra_bufts": true,
"has_token_embeddings": true,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 6219366400,
"resident_bytes": 6219366400,
"registered_tensors": 91,
"registered_bytes": 3771596800,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 16942260224,
"vm_rss_bytes": 16937005056,
"vm_hwm_bytes": 16947953664
}

View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [10, 20],
"reported_range": [10, 20],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": false,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 3304898560,
"registered_tensors": 90,
"registered_bytes": 3304898560,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 3370934272,
"vm_rss_bytes": 3365814272,
"vm_hwm_bytes": 3377971200
}

View File

@@ -0,0 +1 @@
elapsed=0:00.82 maxrss_kib=3298800 exit=0

View File

@@ -0,0 +1,54 @@
# DGR-035 evidence — dense architecture boundary input/output
**Implemented:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
## What changed
- `DenseRangeBoundaryExecutor` is a strict execution-facing adapter for the certified `dense-llama` architecture. A head range accepts non-empty token IDs and owns the embedding callback. Middle/tail ranges reject token IDs and require the named `dense.residual.v1` `BoundaryBundle`.
- Non-tail execution returns exactly the raw `hidden_states` residual from its local layer callback. Its constructor rejects a final-norm/output callback, preventing final normalization, logits projection, sampling, and tail-only row pruning before the tail.
- Tail execution is the only path allowed to own final output and returns an explicit `TailOutput`: either validated logits or a sampled token. The existing wire `TypedTailResult` now serializes and validates both choices.
- Unknown architectures, wrong boundary points, and tensor bundles other than one named `hidden_states` tensor fail closed.
## Changed files
- `packages/node/meshnet_node/architecture_boundary.py`
- `tests/test_dense_range_boundary.py`
- `tests/test_architecture_boundary.py`
- `.ralph-tui/progress.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-035/README.md`
## Commands and results
```bash
TESTPY=/home/popov/.hermes/hermes-agent/venv/bin/python
PYTHONPATH=packages/node:packages/tracker "$TESTPY" -m pytest -q tests/test_dense_range_boundary.py tests/test_architecture_boundary.py tests/test_shard_engine.py tests/test_fake_shard_engine.py
```
```text
37 passed in 0.22s
```
```bash
"$TESTPY" -m ruff check packages/node/meshnet_node/architecture_boundary.py tests/test_dense_range_boundary.py tests/test_architecture_boundary.py
PYTHONPATH=packages/node "$TESTPY" -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
All checks passed!
OK: 55 stories validated.
```
## Limitations
- This story adds and proves the project-owned boundary contract with deterministic, model-download-free tests. It does not claim real-model range parity; DGR-036 owns that numerical certification.
- The llama.cpp graph remains fail-closed for partial owned ranges until DGR-037 binds its worker to this execution contract. No native source or patch-stack file was changed here, so native CMake/CTest and patch-cycle gates are not applicable to this Python contract change.
- `.venv/bin/python3` has no `pytest` module in this worktree. The available project validation interpreter above ran the exact targeted tests.
## Dependency handoff
- DGR-036 should use `DenseRangeBoundaryExecutor` with its real-engine bridge to compare whole-model and split residual/logits outputs, including prefill and decode.
- DGR-037 must adapt the pinned llama.cpp dense graph to `embed_tokens`, `run_layers`, and tail-only `tail_output`; it must preserve `dense.residual.v1` unnormalized and avoid row pruning until the tail.
- DGR-069 can propose only a generic residual-in/residual-out llama.cpp hook; architecture names and Meshnet wire/session semantics remain outside upstream.

View File

@@ -0,0 +1,12 @@
# DGR-036 real-model lane blocker
`DGR-036` cannot receive completion credit yet. The live standalone worker is DGR-033's `FakeShardEngine`, a CRC/echo fixture; DGR-037's real llama.cpp `ShardEngine` binding has not been implemented. Consequently no code path can execute a whole or ranged dense GGUF and no real prefill/logit or greedy-token parity result exists.
The deterministic two-process fake-worker regression is implemented in `tests/test_native_shard_worker.py`, but this sandbox cannot open loopback sockets (`PermissionError: [Errno 1] Operation not permitted`), so that runtime test needs host-side execution as well.
Unblock in this order:
1. Complete DGR-037's real ranged llama.cpp worker binding without changing the DGR-035 dense boundary contract.
2. Provision a small exact dense GGUF on mounted-drive storage and record its artifact/split hashes plus runtime/backend/hardware/network identity.
3. Run whole-model and two-range prefill comparison, then record at least 32 greedy token IDs against the locked tolerance and retain raw metrics.
4. Run the deterministic two-process test on a host with loopback sockets, then update `README.md` and only then set `prd.json` completion truth.

View File

@@ -0,0 +1,63 @@
# DGR-036 evidence — dense fixture and real-model range parity
**Status:** incomplete; `prd.json` remains authoritative and keeps `DGR-036.passes` as `false`.
## Deterministic fixture proof implemented
`tests/test_native_shard_worker.py` now contains `test_two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam`. It starts two separate DGR-033 `shard_worker` OS processes, opens disjoint requested ranges `[0, 16)` and `[16, 32)`, forwards the first worker's actual protobuf output to the second, and checks one prefill plus 32 sequential decode positions. The test tops up the worker's 16-credit flow-control window before decode positions 16 and 32, so all 32 positions are exercised.
This is deliberately **fixture evidence only**. The worker's `FakeShardEngine` validates a bundle and echoes its bytes; it has no dense graph, logits, sampler, or GGUF load. The assertions prove the two-process protocol/lifecycle seam and that bytes survive a disjoint-range handoff. They do not claim numerical model or greedy-token parity.
## Real-model lane: blocked honestly
DGR-037, which is still `passes: false`, is the story that binds llama.cpp to the standalone worker. The live DGR-033 worker remains the fake CRC/echo fixture, and no `ShardEngine` implementation can load/run a GGUF range. DGR-034 proves tensor ownership and memory reporting, while DGR-035 proves the Python boundary contract; neither supplies a real ranged execution engine. Therefore there is no truthful way to run a small dense GGUF whole-model versus two-range prefill comparison or to compare 32 greedy generated tokens yet.
The real-model proof must be run after DGR-037 with an exact small dense GGUF, the pinned llama.cpp/runtime identity, two loaded worker ranges, and a raw report containing artifact and split hashes, backend/driver/hardware/network, prefill tolerance, all 32 token IDs, and raw metrics. It must remain opt-in, use mounted-drive artifact storage, and never download an artifact under `/home`.
## Commands and results
```bash
TESTPY=/home/popov/.hermes/hermes-agent/venv/bin/python
PYTHONPATH=packages/node:packages/tracker "$TESTPY" -m pytest -q tests/test_dense_range_boundary.py tests/test_architecture_boundary.py tests/test_shard_engine.py tests/test_fake_shard_engine.py
```
```text
37 passed in 0.18s
```
```bash
"$TESTPY" -m ruff check tests/test_native_shard_worker.py
PYTHONPATH=packages/node "$TESTPY" -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
All checks passed!
OK: 55 stories validated.
```
Attempted two-process fixture command:
```bash
PYTHONPATH=packages/node:packages/tracker "$TESTPY" -m pytest -q tests/test_native_shard_worker.py -k two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam
```
```text
FAILED: PermissionError: [Errno 1] Operation not permitted at socket.socket(AF_INET, SOCK_STREAM)
```
This is the workspace sandbox's known localhost-socket restriction, before any worker is spawned; it is not a test assertion failure. Run that exact command on a host that permits loopback sockets after building `build/native/shard_worker`.
## Changed files
- `tests/test_native_shard_worker.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-036/README.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-036/BLOCKED.md`
- `.ralph-tui/progress.md`
## Dependency handoff
- DGR-033 supplies the process, lifecycle, generated gRPC surface, fake engine, and bounded-flow-control behaviour used by the deterministic test.
- DGR-035 supplies the strict dense residual boundary and tail-only output contract. DGR-037 must preserve that contract when it replaces the echo fake with a real engine.
- Once DGR-037 is complete, return here to run the opt-in numerical lane. Do not turn this fixture test into a claim that a real GGUF can execute ranges.

View File

@@ -0,0 +1,77 @@
# DGR-037 evidence — bind llama.cpp to the standalone worker
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false` until the opt-in real-model worker lane and native CMake/CTest lane run).
## Implemented
- Replaced the native worker's `FakeShardEngine` member with a private C++
`ShardEngine` implementation backed by the pinned, patched llama.cpp API.
`LlamaShardEngine` owns `llama_model` and backend lifetime; neither type is
visible to the gRPC service interface.
- Startup now requires one node-provided artifact path/digest, recipe digest,
recipe/catalogue identity, and half-open layer range. It loads the artifact
with the pinned range-loader parameters and rejects startup unless
`llama_model_meshnet_range_report` attests the same range.
- `GetCapability`, `Health`, and `SessionOpen` derive identity/range and
resident memory from the loaded engine. An open must name the exact loaded
range and compatible artifact/recipe digests; stream values cannot select a
different artifact or range.
- Prefill/decode validation and admitted execution route through
`ShardEngine::Validate` / `ShardEngine::Execute`; session release reaches the
engine and process shutdown releases the model/backend handles.
- Added the opt-in `MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS` test hook.
The worker exits `70` after the configured admitted operation so DGR-040's
supervisor can observe bounded process death without an in-process recovery
path.
## Changed files
- `packages/node/native/CMakeLists.txt`
- `packages/node/native/README.md`
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
- `packages/node/native/worker/shard_service.{h,cpp}`
- `packages/node/native/worker/shard_worker_main.cpp`
- `tests/test_llama_shard_worker_binding.py`
## Commands and results
```text
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
Applied the exact local DGR-027 patch stack; the resulting header exposed
meshnet_owned_layer_start/end and llama_model_meshnet_range_report.
c++ -std=c++17 -fsyntax-only [llama_shard_engine.cpp, shard_service.cpp, shard_worker_main.cpp]
All three translation units passed syntax checking. The gRPC toolchain emitted
only its existing deprecation warnings.
/home/popov/.hermes/hermes-agent/venv/bin/cmake -S packages/node/native -B build/native-dgr037 \
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain" \
-DMESHNET_LLAMA_SOURCE_DIR="$PWD/build/llama.cpp/source" \
-DMESHNET_LLAMA_LIBRARY_DIR="$PWD/build/llama.cpp/build/bin"
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
shard_worker built successfully; 1/1 shard_protocol_conformance passed.
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
53 passed, 2 skipped
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
compileall passed; diff check passed; OK: 55 stories validated
```
## Limitations and dependency handoff
- No model artifact was selected for this session, so no opt-in real-model
process run, process-death observation, or raw hardware metrics are claimed.
- The pinned API currently attests range ownership/loading. Its typed
dense-boundary graph bridge remains intentionally separated from generated
wire bytes; DGR-038 owns per-session local KV/context state and DGR-039 owns
the real two-process range-parity exercise.
- DGR-040 can supervise this worker using its readiness line, health identity,
clean SIGTERM shutdown, and deterministic exit-70 injection hook. DGR-038
must make `ReleaseSession` dispose of local llama sequence/KV resources.

View File

@@ -0,0 +1,66 @@
# DGR-038 evidence — isolated shard-local Hot KV State
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false` until the opt-in real-model concurrency lane runs).
## Implemented
- The native `LlamaShardEngine` now creates one bounded llama.cpp context and
assigns a distinct `llama_seq_id` to each `(route_session_id, route_epoch)`.
It never accepts remote KV data; the loaded, range-attested llama model owns
the local cache layout and layers.
- Prefill/decode append state tracks local positions and expected past length.
A re-prefill at an earlier position truncates only that sequence with
`llama_memory_seq_rm`; a discontinuity or past-length mismatch returns a
retryable `CACHE_MISS`. Older route epochs return `EPOCH_STALE`.
- The token-reservation budget is bounded by per-session context, total Hot KV
budget, maximum sequence count, TTL, and LRU. Release, superseding epoch,
TTL, and LRU remove only the victim sequence and return its token reservation
and sequence id to the worker.
- The gRPC service converts native cache/stale/resource results to the typed
protocol errors and does not consume idempotency/flow-control credit on a
rejected append. Release is epoch-specific, so a stale release cannot erase
the active epoch's service state.
- Added opt-in configuration: `MESHNET_HOT_KV_MAX_SESSIONS`,
`MESHNET_HOT_KV_CONTEXT_TOKENS`, `MESHNET_HOT_KV_BUDGET_TOKENS`, and
`MESHNET_HOT_KV_TTL_SECONDS`.
## Changed files
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
- `packages/node/native/worker/shard_service.cpp`
- `packages/node/native/worker/shard_worker_main.cpp`
- `tests/test_llama_shard_worker_binding.py`
## Commands and results
```text
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
54 passed, 2 skipped
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
shard_worker built successfully against the pinned, patched llama.cpp source.
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
1/1 shard_protocol_conformance passed.
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
compileall and diff check passed; OK: 55 stories validated.
```
## Limitations and dependency handoff
- DGR-037 supplied the range-attested native model/engine boundary. DGR-038
adds local sequence ownership without changing its artifact or range
identity contract.
- No mounted GGUF artifact was selected. Therefore no opt-in real-model
four-session run, actual llama KV byte measurement, or hardware metrics are
claimed. The default tests intentionally remain model-download-free and the
source `prd.json` remains `passes: false`.
- DGR-039 should exercise the real two-process range-parity lane with four
sessions and the Hot-KV environment bounds, recording actual cache memory
and cancellation isolation evidence.

View File

@@ -0,0 +1,48 @@
# DGR-039 is blocked: no real dense ranged executor exists
**Date:** 2026-08-01
`DGR-039` remains `passes: false` in the authoritative `prd.json`.
## Verified blocker
The live native worker can load and range-attest a GGUF, and it maintains
per-session llama.cpp KV bookkeeping. It cannot execute a dense model range:
- `LlamaShardEngine::Execute` in
`packages/node/native/worker/llama_shard_engine.cpp` deliberately does not
convert the `TensorBundle` into a llama.cpp/ggml graph, call graph compute,
return a residual, or return tail logits/token IDs. Its only successful
effect is advancing `session.past_len` and the local token reservation.
- `ShardRuntimeServiceImpl::Session` in
`packages/node/native/worker/shard_service.cpp` returns the incoming prefill
bundle verbatim (`*response.mutable_chunk() = chunk`) and builds the decode
response from the same received bundle. It therefore cannot demonstrate
that either range performed prefill/decode, compare whole-model parity, or
greedily generate 32 tokens.
- `tests/test_architecture_boundary.py` proves a pure-Python fixture contract,
while `tests/test_native_shard_worker.py` proves an echo seam. Neither is a
real GGUF execution route. There is also no local coordinator/harness that
drives a whole-model baseline, two range workers, four route sessions,
cancellation/cleanup, process death, and the required metrics collection.
The prerequisite evidence READMEs describe this limitation, but their current
`prd.json` completion flags do not alter the live implementation above.
## Required follow-on before this acceptance can run
1. Bind the DGR-035 dense boundary adapter to a native llama.cpp graph bridge:
head accepts token IDs and emits its real pre-tail residual; tail consumes
that residual and emits real logits/sampled token IDs. Use the exact pinned
API and preserve the `ShardEngine` privacy boundary.
2. Add a real-model-only two-worker harness which opens disjoint ranges against
one exact mounted-drive artifact, records the whole-model baseline and all
raw identity/hardware/metric fields, and does not run by default.
3. Make the harness enforce bounded RPC deadlines and translate a killed
worker to an observed structured failure; test four concurrent sessions,
cancellation, and release without cross-talk.
4. Run it on a host with loopback sockets and an explicitly selected GGUF.
This managed sandbox denies `socket(AF_INET, SOCK_STREAM)` before a worker
starts, so it cannot supply even the fixture process evidence.
No criterion is weakened and no real-model evidence is claimed.

View File

@@ -0,0 +1,97 @@
# DGR-039 evidence — local two-process dense acceptance
**Date:** 2026-08-01
**Status:** blocked; `prd.json` remains authoritative and keeps
`DGR-039.passes` as `false`.
## Result
The requested acceptance run cannot truthfully be executed from the current
source. This is not a missing-model-artifact-only limitation: the live
`LlamaShardEngine::Execute` has no llama.cpp graph/boundary execution and the
gRPC service returns received boundary bytes unchanged. Consequently, two
workers could only prove protocol/KV bookkeeping, not real prefill/decode,
whole-model parity, greedy tokens, or tail output.
See [BLOCKED.md](BLOCKED.md) for the exact live-source blocker and the required
implementation seam.
## Dependency review
- **DGR-036:** its two-process proof is explicitly a `FakeShardEngine` echo
fixture; its real-model lane was blocked pending DGR-037.
- **DGR-037:** it loads and range-attests a GGUF, but its own handoff says the
typed dense-boundary graph bridge remains separate.
- **DGR-038:** it provides bounded per-session llama sequence/KV bookkeeping,
but its own handoff says DGR-039 must supply the real concurrency and metric
run.
The live source confirms those limits: `llama_shard_engine.cpp` increments
`past_len` without computing a graph, and `shard_service.cpp` echoes both
prefill/decode bundles.
## Commands and results
```bash
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_architecture_boundary.py tests/test_llama_shard_worker_binding.py \
tests/test_native_shard_protocol.py
```
```text
61 passed, 2 skipped in 0.51s
```
```bash
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
OK: 55 stories validated.
```
```bash
/home/popov/.hermes/hermes-agent/venv/bin/python -m cmake --build build/native-dgr037 -j2
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
```
```text
shard_worker built successfully.
1/1 shard_protocol_conformance passed.
```
Attempted existing two-worker fixture:
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_shard_worker.py -k two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam
```
```text
FAILED before worker startup: PermissionError: [Errno 1] Operation not permitted
at socket.socket(AF_INET, SOCK_STREAM).
```
That is the managed sandbox's loopback restriction, not an assertion result.
Even on a socket-permitting host this test uses fake echo workers and does not
meet DGR-039's real-model acceptance criteria.
## Changed files
- `.scratch/distributed-gguf-runtime/evidence/DGR-039/README.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-039/BLOCKED.md`
- `.ralph-tui/progress.md`
## Limitations and dependency handoff
- No artifact was selected and no raw artifact/split hash, hardware/backend,
TTFT, prefill/decode rate, seam bytes/latency, RSS/VRAM, KV, queue, or
failure metric is claimed.
- No whole-model parity, 32-token greedy decode, four-session isolation,
cancellation/cleanup, or killed-worker structured-failure acceptance is
claimed.
- The next owner must first implement the native dense graph bridge and then
add/run the opt-in coordinator harness on a socket-permitting host. Keep
`DGR-039.passes` false until it has the required real run evidence.

View File

@@ -0,0 +1,89 @@
# DGR-040 evidence — node-side native worker supervision
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false`; this is fixture-only supervision evidence and does not claim a real
GGUF/gRPC process run in this sandbox).
## Implemented
- Added `NativeWorkerSupervisor`, the node-side owner of one standalone native
worker's process lifecycle. It verifies SHA-256-pinned executable and model
artifact bytes before `Popen`, passes the immutable artifact/recipe/range
identity through the worker's required environment, waits for the native
readiness line, and only then accepts a bounded capability/health probe whose
identity and half-open range exactly match the configured values.
- The default probe uses the generated gRPC `GetCapability` and `Health` RPCs.
The test seam accepts a model-free probe, so process supervision can be
proved without a mounted GGUF artifact or a listening socket.
- Both stdout and stderr are captured into a bounded in-memory log tail.
`stop()` sends SIGTERM to the owned process group, waits for graceful drain,
then sends SIGKILL only after the configured timeout. `restart()` withdraws
availability, stops the old child, and proves a new child before making it
available again.
- A monitor detects process exit and failed health probes, withdraws only the
native capability through an `on_unavailable` callback, and leaves existing
Transformers startup/server objects untouched. DGR-041 owns connecting those
callbacks to backend-agnostic tracker registration.
- Added deterministic fake-worker tests. The fake recognizes
`MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS` and exits 70 once, matching
DGR-037's production crash-injection exit code; the supervisor observes the
withdrawal and successfully restarts it.
## Changed files
- `packages/node/meshnet_node/native_worker_supervisor.py`
- `tests/test_native_worker_supervisor.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-040/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_worker_supervisor.py tests/test_llama_shard_worker_binding.py \
tests/test_native_shard_protocol.py
# 60 passed, 2 skipped in 0.97s
python3 -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/native_worker_supervisor.py \
tests/test_native_worker_supervisor.py
# All checks passed!
```
The system Python and repository `.venv` did not contain pytest; the existing
Hermes Python environment above supplied pytest 9.0.3 and grpc for the focused
checks. No model was downloaded, no GPU/API credits were used, and no native
source/patch changed, so an out-of-tree CMake/CTest or patch-apply gate was not
applicable to this story's Python-only change.
## Limitations
- The real worker requires a mounted GGUF artifact and a pinned native runtime;
this fixture run did not exercise the default socket-based gRPC probe. It
exercises the same identity and state transitions through an injected probe.
- Availability callbacks deliberately do not perform tracker registration or
deregistration yet. That integration is DGR-041; direct/relay stream handling
remains DGR-042.
- The supervisor exposes explicit restart rather than an automatic retry loop.
Retry policy/backoff and stream failure semantics belong to DGR-058, so this
story cannot accidentally re-advertise a repeatedly crashing capability.
## Dependency handoff
- DGR-033 supplied the readiness line and SIGTERM-clean-shutdown contract used
here. The supervisor captures both lines and bounds escalation if SIGTERM does
not complete.
- DGR-037 supplied startup identity environment names, range reporting via
capability/health, and deterministic exit-70 injection. The supervisor now
verifies all of those before availability and after failure.
- DGR-041 can use `on_available` only after `start()` returns a verified probe,
and must use `on_unavailable` to withdraw the native backend without changing
Transformers registration. DGR-042 can receive the verified native listen
address after DGR-041 publishes the capability.

View File

@@ -0,0 +1,99 @@
# DGR-041 evidence — backend-agnostic native Shard registration
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false`; this is model-free integration evidence, not a real hardware
certification).
## Implemented
- Added the optional, backend-neutral `ExecutionCapacity` capability-report
block: memory capacity in bytes, Hot-KV capacity in tokens, and maximum
concurrent Route Sessions. Existing Transformers reports omit it and keep
their previous serialized shape.
- Added `NativeShardRegistration`, which accepts only an exact `ShardIdentity`,
DGR-040 startup spec, and verified worker probe that all agree on artifact
digest, recipe fingerprint, recipe labels, and half-open range. It emits the
existing tracker registration payload and uses the capability report for
backend, capacity, and exact identity facts.
- Added `NativeCapabilityRegistrar.bind()` and additive supervisor callbacks:
publish happens only after DGR-040 has verified availability; a worker health
loss invokes caller-owned withdrawal. The adapter owns neither tracker HTTP
nor routing, billing, telemetry, relay, or provider policy.
- Tracker capability parsing/network state now preserves the three optional
capacity facts. Its existing `CertificationLedger` still registers the exact
native recipe as `dark` / `uncertified`, making it visible but unroutable.
No backend-name allowlist or routing special case was added.
## Changed files
- `packages/node/meshnet_node/capability.py`
- `packages/node/meshnet_node/native_registration.py`
- `packages/node/meshnet_node/native_worker_supervisor.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `tests/test_native_registration.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-041/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_registration.py tests/test_native_worker_supervisor.py \
tests/test_node_capability.py tests/test_runtime_recipe_identity.py
```
```text
101 passed in 0.71s
```
```bash
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/capability.py \
packages/node/meshnet_node/native_registration.py \
packages/node/meshnet_node/native_worker_supervisor.py \
packages/tracker/meshnet_tracker/capability.py \
tests/test_native_registration.py
```
```text
All checks passed!
```
```bash
python3 -m compileall -q packages tests
git diff --check
```
```text
Both exit 0.
```
The default focused tests are model-download-free, API-credit-free, and
GPU-free. No model artifact was touched and nothing was written under `/home`.
## Limitations
- The full HTTP tracker-registration route suite could not run in this sandbox:
`PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q tests/test_tracker_capability_admission.py`
produced `25 passed, 9 failed`; every failure is the known sandbox
`PermissionError: [Errno 1] Operation not permitted` while creating an AF_INET
listening socket. The model-free direct tracker admission path is exercised
by `test_native_registration.py` and the existing identity suite.
- No native source/protobuf/patch changed, so an out-of-tree CMake/CTest build
and pin patch apply/check/reverse gates are not applicable.
- The registrar deliberately takes caller-owned register/withdraw callbacks.
DGR-042 owns the native direct/relay activation endpoint; deployment wiring
must provide its existing tracker transport rather than invent another one.
- No real backend/model/recipe combination is certified by this change.
`prd.json` remains false until the authoritative execution process grants
completion credit.
## Dependency handoff
- **DGR-025:** `ShardIdentity` and the tracker-owned `CertificationLedger` are
used directly; do not substitute labels for the fingerprint or promote a
recipe in node code.
- **DGR-040:** construct this registration from the post-`start()` verified
probe and call `NativeCapabilityRegistrar.bind(supervisor)` before startup.
Its unavailable callback must withdraw only the native capability.
- **DGR-042:** consume the registration's verified native endpoint through the
existing direct/relay route mechanism; keep its protobuf transport opaque to
tracker admission.

View File

@@ -0,0 +1,73 @@
# DGR-042 evidence — native frames through direct and relay seams
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`.
## Implemented
- Added `NativeActivationSeam`, a Route-Session-scoped adapter with exactly two
selectable transports. Direct traffic calls the generated
`ShardRuntimeStub.Session()` once and keeps its bidirectional gRPC stream
open for the session. Its request and response hand-off queues are bounded.
- Relay traffic calls the existing persistent relay request shape with
`POST /native/session`, `application/x-protobuf`, and the exact
`SessionRequest.SerializeToString()` body. It parses only the returned
`SessionResponse`; neither the adapter nor the relay contract rewrites a
protobuf frame. Relay failure is explicitly uncertain and is never retried.
- `NativeFrameContext` validates Route Session, epoch, work, and deadline
fields against the versioned protobuf request before either path sends it.
The unchanged existing relay header contract receives request/billing ID,
node attribution, route, work, and deadline copies for control-plane
telemetry/billing correlation. `NativeSeamTelemetry` reports per-node,
per-request seam byte/latency observations without interpreting frames.
- Deterministic fake-worker tests cover a single direct stream, byte-identical
relay request frames, relay disconnect/no replay, cancellation, correlation
headers, telemetry, and bounded direct buffering.
## Changed files
- `packages/node/meshnet_node/native_activation_seam.py`
- `tests/test_native_activation_seam.py`
- `.scratch/distributed-gguf-runtime/prd.json`
- `.scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-042/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_activation_seam.py tests/test_native_shard_protocol.py \
tests/test_native_worker_supervisor.py tests/test_native_registration.py \
tests/test_ralph_prd_schema.py
# 172 passed, 2 skipped in 2.01s
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/native_activation_seam.py tests/test_native_activation_seam.py
# All checks passed!
python3 -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
```
No model download, GPU, API credit, native worker build, or upstream patch was
required. Native CMake/CTest and patch-stack gates do not apply to this
Python-only transport adapter.
## Limitations and dependency handoff
- Relay is deliberately a sequence of opaque existing relay RPC bodies, not a
gRPC tunnel. The direct path alone is a long-lived gRPC stream; this avoids
changing relay behavior while preserving native frame bytes.
- This fixture lane uses an injected generated-stub-shaped fake worker and an
injected existing-relay-client-shaped callable. DGR-054/DGR-058 must use the
adapter with certified workers and add real route-loss/restart policy; they
must retain the no-replay rule after an uncertain relay send.
- DGR-024 supplied the versioned generated `Session` protocol and the prior
raw-frame identity proof. DGR-040 supplied the verified worker lifecycle;
its published native listen address is the direct endpoint for this seam.
- Existing Transformer HTTP routes and relay routing, load balancing, billing,
and peer behavior were not changed.

View File

@@ -0,0 +1,69 @@
# DGR-043 evidence — GGUF inputs through existing tracker routing
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false`; this is model-free integration evidence, not a hardware certification).
## Implemented
- Added optional backend-neutral `RoutingMeasurements` to the existing capability report. It carries measured tokens/second, queue depth, seam latency, health, and reliability; reports that omit it retain their exact previous serialized shape.
- Extended the trackers existing sanitized `CapabilityState` and network-map capability view to retain the routing measurements with exact recipe, artifact/runtime fingerprint, half-open-range-derived coverage, capacity, backend, and certification facts.
- `NativeShardRegistration` now accepts this generic measurement block and adapts throughput and queue depth to the existing registration/heartbeat scoring inputs. The tracker continues to apply its established queue-adjusted throughput selection; no GGUF routing, balancing, billing, relay, provider, quantization, topology, or architecture branch was added.
- Added deterministic coverage tests showing that existing route formation excludes a dark candidate, forms a complete route only from matching exact fingerprints, and rejects a range otherwise covered only by a mismatched recipe.
## Changed files
- `packages/node/meshnet_node/capability.py`
- `packages/node/meshnet_node/native_registration.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/test_native_registration.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-043/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_registration.py tests/test_node_capability.py \
tests/test_runtime_recipe_identity.py
```
```text
96 passed in 0.23s
```
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_dgr_performance_contract.py tests/test_native_activation_seam.py \
tests/test_native_worker_supervisor.py tests/test_native_registration.py \
tests/test_ralph_prd_schema.py
```
```text
151 passed in 1.78s
```
```bash
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/capability.py \
packages/node/meshnet_node/native_registration.py \
packages/tracker/meshnet_tracker/capability.py \
packages/tracker/meshnet_tracker/server.py tests/test_native_registration.py
python3 -m compileall -q packages tests
git diff --check
```
```text
All checks passed; both remaining commands exited 0.
```
Default tests were model-download-free, API-credit-free, and GPU-free. No native source, protobuf, patch, model artifact, or mounted-drive content was changed; therefore native CMake/CTest, patch-stack, and real-hardware gates do not apply to this Python-only adapter.
## Limitations
- The full HTTP tracker/admission and tracker-routing suites cannot bind an AF_INET listener in this sandbox. The attempted focused suite had 132 passes and 14 failures, all `PermissionError: [Errno 1] Operation not permitted` during socket creation. Model-free direct tracker parsing and route-formation tests cover this change; HTTP/billing/relay regression suites must be rerun in an environment that permits localhost sockets.
- Measurements are inputs, not self-certification. An exact native recipe remains `dark` until the existing tracker-owned certification ledger admits it, and worker health loss continues to withdraw the native capability.
- Seam latency is retained as a measured tracker capability input. Existing route latency learning remains the tracker-owned mechanism for end-to-end seam cost; this story intentionally does not alter its scoring algorithm.
## Dependency handoff
- **DGR-041:** `NativeShardRegistration`, `ExecutionCapacity`, exact `ShardIdentity`, and the tracker certification ledger remain the only registration/admission path. Supply `RoutingMeasurements` from verified worker/telemetry observations; do not infer values from backend names, quantization labels, architecture, or stage topology.
- **DGR-053/DGR-061:** use the exposed opaque measurements and existing tracker routing mechanisms for real certified routes. Any real-run evidence must add artifact/split hashes, worker/upstream pins, backend/driver, hardware/network details, commands, and raw metrics.

View File

@@ -0,0 +1,62 @@
# Maintenance review handoff — distributed GGUF runtime
Date: 2026-07-14
Scope: close the maintenance review, preserve the hard blockers, and hand off the remaining implementation work to the next model.
## What is complete
- Completed stories are now recorded in `docs/issues/distributed-gguf-runtime/`.
- The PRD and milestone docs were updated to reflect the closed set and the blocked set.
- The DGR-018 preflight scripts were preserved at commit `a0f28b5`.
- The current feature line has delivered DGR-001 through DGR-006 and DGR-017.
## Hard / unsolved issues for later
### 1) DGR-018 requires hardware we do not have
DGR-018 is blocked because the whole-model GLM-5.2 UD-IQ1_S oracle requires:
- a **256-GiB-class host**,
- at least **224 GiB runtime-accessible memory**,
- at least **250 GB free storage on one filesystem outside `/home`**.
The current development host reports only **124.9 GiB MemTotal** and has no eligible filesystem with 250 GB free.
The authoritative blocker evidence is in `evidence/DGR-018/BLOCKED.md` and `evidence/DGR-018/preflight.json`.
### 2) DGR-019 and DGR-020 are transitively blocked
- **DGR-019** needs the DGR-018 oracle for parity certification.
- **DGR-020** needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single node can admit the whole recipe.
No smaller model may be substituted for these stories.
### 3) The remainder of the graph stays blocked unless replanned
The current graph makes **DGR-007 depend on DGR-019**, which means:
- DGR-007 through DGR-016 are also blocked transitively.
- Unblocking the dense pipeline without the 256-GiB host would require an explicit replanning decision to relax the DGR-007 → DGR-019 dependency.
- That replanning decision has **not** been made.
### 4) Maintenance-only tasks should stay separate from feature implementation
The review uncovered that the codebase now has a clean closed-story split, but further work should avoid mixing:
- maintenance cleanup,
- blocked-hardware preparation,
- and actual distributed GLM implementation.
The next model should treat the maintenance pass as closed and only pick up real implementation work that is not hardware-blocked.
## Recommended next move
Use the next model to continue on the **non-blocked implementation queue** only.
Priority candidates are whatever is still actionable without the GLM oracle host; if a story depends on DGR-018, keep it deferred.
## Reference files
- `docs/issues/distributed-gguf-runtime/README.md`
- `.scratch/distributed-gguf-runtime/PRD.md`
- `.scratch/distributed-gguf-runtime/milestones.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/preflight.json`

View File

@@ -1,15 +1,15 @@
# Ralph task evidence
# Distributed GGUF Runtime evidence
Each completed story creates `evidence/<TASK-ID>/README.md`. Fresh dependent iterations must read it before coding.
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
Required README sections:
## Authority and classes
1. Summary and acceptance decision.
2. Exact files changed.
3. Commands run and real exit/results.
4. Correctness, performance and hardware evidence classification.
5. Known limitations and deferred work.
6. Compatibility/migration notes.
7. Explicit handoff for each dependent story.
Evidence supports but never overrides `prd.json`. Valid classes are `model-free`, `fixture`, `real-model`, `real-hardware`, and `release`; lower classes cannot satisfy higher-class acceptance. Legacy DGR-001..016 directories remain unchanged for DGR-017 provenance audit and confer no completion credit.
Store raw machine-readable metrics, manifests and protocol artifacts beside the README. Never store secrets, model weights, build outputs or Ralph iteration logs here.
## Future story layout
Each DGR-017..071 story writes `evidence/<ID>/README.md` with summary, exact changed files, exact commands and real outputs, limitations, compatibility/migration notes, and dependent-story handoff. Machine-readable contracts, manifests, metrics, and raw logs live beside it. Never fabricate output.
Real runs record exact model SHA and all split hashes, tokenizer, quant/recipe, llama.cpp pin+patch identity, backend/driver/toolchain, host/hardware/network, commands/environment (without secrets), raw parity/performance/resource results, and evidence class. Models live on configured mounted-drive storage, never `/home`.
Routing certification records prove only the exact exercised backend/model/recipe lane. Compile-only, fixture, failed, or unavailable lanes remain registered-dark. V4 cache/state evidence must show KV and CSA/HCA/SWA/indexer/compressor data remain shard-local/session-keyed; route recovery evidence must show cache miss plus re-prefill/restart, not migration.

View File

@@ -0,0 +1,335 @@
{
"repository": "https://git.d-popov.com/popov/neuron-tai",
"stories": {
"DGR-017": {
"number": 1,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/1",
"state": "closed",
"status": "completed"
},
"DGR-018": {
"number": 2,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/2",
"state": "closed",
"status": "completed"
},
"DGR-019": {
"number": 3,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/3",
"state": "closed",
"status": "completed"
},
"DGR-020": {
"number": 4,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/4",
"state": "closed",
"status": "completed"
},
"DGR-021": {
"number": 5,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/5",
"state": "closed",
"status": "completed"
},
"DGR-022": {
"number": 6,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/6",
"state": "closed",
"status": "completed"
},
"DGR-023": {
"number": 7,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/7",
"state": "closed",
"status": "completed"
},
"DGR-024": {
"number": 8,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/8",
"state": "closed",
"status": "completed"
},
"DGR-025": {
"number": 9,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/9",
"state": "closed",
"status": "completed"
},
"DGR-026": {
"number": 10,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/10",
"state": "closed",
"status": "completed"
},
"DGR-027": {
"number": 11,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/11",
"state": "closed",
"status": "completed"
},
"DGR-028": {
"number": 12,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/12",
"state": "closed",
"status": "completed"
},
"DGR-029": {
"number": 13,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/13",
"state": "closed",
"status": "completed"
},
"DGR-030": {
"number": 14,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/14",
"state": "open",
"status": "in-progress"
},
"DGR-031": {
"number": 15,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/15",
"state": "open",
"status": "ready"
},
"DGR-032": {
"number": 16,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/16",
"state": "open",
"status": "blocked"
},
"DGR-033": {
"number": 17,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/17",
"state": "open",
"status": "blocked"
},
"DGR-034": {
"number": 18,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/18",
"state": "open",
"status": "blocked"
},
"DGR-035": {
"number": 19,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/19",
"state": "open",
"status": "blocked"
},
"DGR-036": {
"number": 20,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/20",
"state": "open",
"status": "blocked"
},
"DGR-037": {
"number": 21,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/21",
"state": "open",
"status": "blocked"
},
"DGR-038": {
"number": 22,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/22",
"state": "open",
"status": "blocked"
},
"DGR-039": {
"number": 23,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/23",
"state": "open",
"status": "blocked"
},
"DGR-040": {
"number": 24,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/24",
"state": "open",
"status": "blocked"
},
"DGR-041": {
"number": 25,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/25",
"state": "open",
"status": "blocked"
},
"DGR-042": {
"number": 26,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/26",
"state": "open",
"status": "blocked"
},
"DGR-043": {
"number": 27,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/27",
"state": "open",
"status": "blocked"
},
"DGR-044": {
"number": 28,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/28",
"state": "open",
"status": "ready"
},
"DGR-045": {
"number": 29,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/29",
"state": "open",
"status": "blocked"
},
"DGR-046": {
"number": 30,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/30",
"state": "open",
"status": "blocked"
},
"DGR-047": {
"number": 31,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/31",
"state": "open",
"status": "blocked"
},
"DGR-048": {
"number": 32,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/32",
"state": "open",
"status": "blocked"
},
"DGR-049": {
"number": 33,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/33",
"state": "open",
"status": "blocked"
},
"DGR-050": {
"number": 34,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/34",
"state": "open",
"status": "blocked"
},
"DGR-051": {
"number": 35,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/35",
"state": "open",
"status": "blocked"
},
"DGR-052": {
"number": 36,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/36",
"state": "open",
"status": "blocked"
},
"DGR-053": {
"number": 37,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/37",
"state": "open",
"status": "blocked"
},
"DGR-054": {
"number": 38,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/38",
"state": "open",
"status": "blocked"
},
"DGR-055": {
"number": 39,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/39",
"state": "open",
"status": "blocked"
},
"DGR-056": {
"number": 40,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/40",
"state": "open",
"status": "blocked"
},
"DGR-057": {
"number": 41,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/41",
"state": "open",
"status": "blocked"
},
"DGR-058": {
"number": 42,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/42",
"state": "open",
"status": "blocked"
},
"DGR-059": {
"number": 43,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/43",
"state": "open",
"status": "blocked"
},
"DGR-060": {
"number": 44,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/44",
"state": "open",
"status": "blocked"
},
"DGR-061": {
"number": 45,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/45",
"state": "open",
"status": "blocked"
},
"DGR-062": {
"number": 46,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/46",
"state": "open",
"status": "blocked"
},
"DGR-063": {
"number": 47,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/47",
"state": "open",
"status": "blocked"
},
"DGR-064": {
"number": 48,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/48",
"state": "open",
"status": "blocked"
},
"DGR-065": {
"number": 49,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/49",
"state": "open",
"status": "blocked"
},
"DGR-066": {
"number": 50,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/50",
"state": "open",
"status": "blocked"
},
"DGR-067": {
"number": 51,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/51",
"state": "open",
"status": "blocked"
},
"DGR-068": {
"number": 52,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/52",
"state": "open",
"status": "blocked"
},
"DGR-069": {
"number": 53,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/53",
"state": "open",
"status": "blocked"
},
"DGR-070": {
"number": 54,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/54",
"state": "open",
"status": "blocked"
},
"DGR-071": {
"number": 55,
"url": "https://git.d-popov.com/popov/neuron-tai/issues/55",
"state": "open",
"status": "blocked"
}
}
}

View File

@@ -1,241 +1,40 @@
# Focused implementation strategy: performant concurrent distributed inference
# Distributed GGUF Runtime implementation strategy
Status: Accepted planning direction
Last updated: 2026-07-13
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
## Product objective
## Execution model
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
Execute one numerically ordered, dependency-ready story per fresh Ralph context. Read `RALPH-CONTEXT.md`, source issue, and dependency evidence first; use TDD/fixture-first verification; finish with exact evidence. `prd.json` is the only state authority.
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
## Sequence
1. Models larger than one node's RAM/VRAM.
2. Useful interactive decode speed on consumer CPU, AMD, NVIDIA, Vulkan, and mixed routes where certified.
3. Multiple concurrent Route Sessions without cache corruption or global serialization.
4. A lean runtime with one control plane and one primary GGUF engine.
5. Measured improvement over the existing Transformers/safetensors implementation.
1. **M0 DGR-017..020:** reconcile legacy reality, lock metadata/performance contracts, and run the independent whole-model baseline.
2. **M1 DGR-021..033:** protocol/lifecycle/codegen, exact identities and split artifacts, pinned upstream/patches, CPU then accelerator builds, `ShardEngine`, fixtures, fake worker.
3. **M2 DGR-034..043:** dense ranged ownership/boundary/parity/local state, worker integration, supervision/direct-relay, and measured GGUF inputs to unchanged routing.
4. **M3 DGR-044..054:** pin/inventory V4, adapt upstream boundary/local state/MoE/hash execution, pass parity and real 24 scenario, then enforce alpha with MTP off.
5. **M4 DGR-055..067:** batching/backpressure/failure/recovery/long-context, existing-routing 10+ certification, real scale, measured optimization/compression, MTP contract+implementation, hardware certification.
6. **M5 DGR-068..071:** packages, human upstream collaboration, beta gate (including MTP), and pin/patch/certification maintenance.
## Current reality
## Guardrails
The existing project already owns the differentiating distributed control plane:
- Tracker-selected contiguous Shards.
- Stable Route Sessions.
- Local per-Shard Hot KV State in the Transformers reference backend.
- Binary Activation Seams.
- Relay/direct routing, cancellation, telemetry, billing, and capability admission.
- Persistent relay and direct transport optimizations.
## Locked scope
The missing production path is a native GGUF execution worker that can load and execute only an assigned layer range while retaining local Hot KV State for concurrent Route Sessions.
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
- Quantization and placement are dynamic recipe inputs. The 24 and 10+ stage layouts are certification scenarios, never product constants.
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
Whole-model llama.cpp, vLLM, and existing Transformers serving remain baselines or optional route kinds. They are not substitutes for native distributed Shards.
## Target identities
## Performance hypothesis—not an assumption
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kernels, memory layout, mmap, backend scheduling, and reduced working set.
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
Before expensive native work, establish controlled lanes:
- Same model architecture and upstream revision.
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
- Transformers/safetensors BF16 or the current production recipe.
- llama.cpp GGUF F16/BF16 or Q8 correctness lane where available.
- Q4_K_M or selected production quantization performance/fit lane.
- TTFT, prefill tok/s, decode tok/s, p50/p95 latency, RSS, VRAM, artifact size, energy where available, and output-quality drift.
The program proceeds only if llama.cpp/GGUF provides at least one meaningful advantage recorded in a machine-readable performance contract:
- Better decode or aggregate throughput at acceptable quality; or
- Materially lower memory that makes the target model routable while preserving useful throughput.
## Parallelism we will use
### Public Inference Route: layer/pipeline parallelism
Each node independently executes one contiguous Shard. Activations cross seams; weights and Hot KV State remain local.
This is the only public cross-machine model-parallel primitive in the first runtime.
### Per-node continuous batching
Autoregressive tokens remain sequential within one generation. Throughput comes from batching decode steps from multiple active Route Sessions inside each node using llama.cpp batches and sequence IDs or bounded context pools.
This is essential. A worker that globally serializes sessions is not production-ready.
### Multiple complete routes: data parallelism
The Tracker may select multiple complete routes for independent requests. This increases network throughput and availability without requiring collectives between routes.
### Trusted composite node: optional tensor/expert parallelism
Tensor parallelism and expert parallelism require frequent collectives and tight compatibility. They may be used later inside one operator-controlled composite node or managed cluster exposed as one logical provider. They are not public WAN routing primitives.
### Deferred mechanisms
- Disaggregated prefill and KV transfer.
- Speculative decoding.
- Cross-route prefix snapshots.
- Route repair with KV migration.
- Public tensor/expert parallel collectives.
They remain out of the critical path until the native layer route passes performance and concurrency gates.
## Reuse decisions
### llama.cpp/GGML: primary runtime substrate
Reuse:
- GGUF parsing and mmap.
- Quantized kernels.
- CPU, CUDA, HIP/ROCm, Vulkan, Metal, and other supported backends.
- Tokenizer and model architecture implementations.
- KV and sequence operations.
- Backend scheduler and graph execution.
Maintain a small exact-commit fork only for the missing local seam:
- Range-aware tensor ownership/loading.
- Architecture-defined boundary input/output.
- Intermediate boundary output without tail normalization.
- Layer-filtered KV and sequence mapping.
Keep networking, Tracker logic, billing, and public protocol outside llama.cpp. Upstream generic hooks where possible.
### vLLM: concepts and optional managed backend
Use unmodified vLLM only as:
- A whole-model node backend.
- A managed TP/PP/EP cluster represented as one logical provider.
- A performance/correctness baseline.
Adapt concepts, not runtime code:
- Named intermediate tensor bundles.
- Continuous batching and request-owner maps.
- Versioned KV-transfer compatibility fingerprints.
- Explicit send/receive/abort/failure lifecycle.
- Load telemetry and unbiased route selection.
Do not fork vLLM for public Shards and do not transplant PagedAttention, Torch process groups, or GGUF-plugin kernels into the llama.cpp worker.
### Nakshatra, prima.cpp, llama-gguf, LiGGUF, GPUStack
Use as source and test donors only:
- Nakshatra: partial-GGUF patches, daemon concepts, replay cases.
- prima.cpp: selected tensor ownership and local-layer KV evidence.
- llama-gguf: small protocol and integration-test patterns.
- LiGGUF: Q8 activation transport and tensor-reduction reference.
- historical GPUStack: resource preflight and role-oriented placement.
Do not adopt or fork their repositories wholesale.
## Battle-proven transport decision
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
Why:
- Mature Python and C++ implementations.
- Bidirectional streaming.
- HTTP/2 flow control and connection reuse.
- Deadlines, cancellation, status codes, TLS, authentication interceptors, and generated schemas.
- Avoids inventing a socket protocol.
Scope boundary:
- OpenAI-compatible client/Gateway APIs remain HTTP/SSE.
- Tracker/control APIs remain existing project interfaces.
- One long-lived bidirectional gRPC stream serves one Route Session Activation Seam.
- Existing relay/WebSocket infrastructure may carry the same versioned protobuf frames as opaque binary when direct gRPC reachability is unavailable.
- Large prefill tensors are chunked into bounded frames; decode bundles stay small.
- No QUIC/WebRTC/custom transport in this milestone.
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`.
Minimum identity:
```text
schema version
request/work id
Route Session id and route epoch
Model Artifact and runtime recipe fingerprint
Shard range and effective start
phase: prefill/decode/release/cancel
position/token range
named tensors with shape/dtype/byte order
compression and checksum
idempotency step id
cache expectation/result
```
## Concurrency model
A native worker must not use one global serving sequence or one lock around all model execution.
Required ownership:
```text
(Route Session id, route epoch)
-> local sequence/context
-> Shard-local Hot KV State
-> bounded lease and memory accounting
```
The node scheduler:
- Admits sessions against model memory and KV budget.
- Forms compatible decode batches from active sessions.
- Preserves per-session position and route order.
- Applies bounded queues and backpressure.
- Cancels/releases independently.
- Reports queue, batch, KV, prefill, decode, and seam telemetry.
Initial deterministic gate: at least four concurrent sessions on a small certified model with no token/KV cross-talk. Final concurrency targets are hardware/recipe-specific and recorded by capability admission rather than hardcoded globally.
## Stage gates
### Gate A: performance hypothesis
Controlled safetensors-versus-GGUF benchmark produces a signed/reproducible report and locks thresholds. Stop native work if there is no meaningful speed or fit benefit.
### Gate B: local range parity
Two local processes own disjoint GGUF ranges and match whole-model llama.cpp within the certified numerical tolerance for prefill and greedy decode.
### Gate C: concurrent KV
Multiple Route Sessions prefill/decode concurrently with isolated local KV, bounded memory, cancellation, and release.
### Gate D: real distributed route
Two physical machines execute one model that uses both Shards. Synthetic activation tests do not satisfy this gate.
### Gate E: consumer-hardware performance
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
### Gate F: architecture expansion
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
## Scope discipline
The following do not block the first production candidate:
- New cryptocurrency/economics work.
- New artifact P2P protocol.
- QUIC or WebRTC.
- vLLM fork.
- Whole-repository Nakshatra/prima adoption.
- Every GGUF architecture.
- Automatic route repair.
- Prefix snapshot migration.
- Speculative decoding.
- A large-model marketing demo before small-model parity and concurrency pass.
Every optimization must preserve output contract, session isolation, cancellation, resource cleanup, capability admission, and per-node attribution.
DGR-020 cannot use distributed results. DGR-054 does not depend on MTP. DGR-070 depends on DGR-066. Compile support and scenario success never imply general routability.

View File

@@ -1,59 +0,0 @@
# 01 — Lock the safetensors-versus-GGUF performance contract
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
- This issue is `DGR-001` in [prd.json](../prd.json).
- Read the evidence README for every dependency listed below.
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
## Description
As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
## Expected durable outputs
- Benchmark harness and deterministic tests
- evidence/DGR-001/performance-contract.json
- Raw and summarized safetensors/GGUF benchmark evidence
## Acceptance criteria
- [ ] Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.
- [ ] Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.
- [ ] Report TTFT, prefill tok/s, decode tok/s, p50/p95 latency, aggregate throughput, RSS, VRAM, artifact size, failures, and output drift in machine-readable JSON.
- [ ] Add concurrency levels 1 and 4 where memory permits.
- [ ] Write a versioned performance contract consumed by later release gates, including an explicit stop condition when llama.cpp/GGUF has no meaningful speed or fit benefit.
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-001/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
## Dependency handoff
- None. This story may start immediately.
## Finish contract
- Create the task evidence directory and durable handoff required above.
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
- Change this issue to `Status: done` only after all criteria pass.
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
## References
- [Ralph execution context](../RALPH-CONTEXT.md)
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -0,0 +1,39 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-017: Reconcile and clean the superseded DGR backlog
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK`
- **Milestone:** `M0`
- **Dependencies:** None
- **Blocks (derived):** `DGR-018`, `DGR-019`, `DGR-027`, `DGR-054`
- **Labels:** `area:provenance`, `area:cleanup`, `type:audit`, `priority:p0`, `ready-for-agent`
- **Evidence class:** `model-free`
- **Hardware:** `none`
- **Model:** `generic`
- **Upstream:** `no`
## Objective / description
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md`, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.
## Acceptance criteria
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
- `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -0,0 +1,39 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-018: Define canonical Ralph and Gitea metadata schema
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK`
- **Milestone:** `M0`
- **Dependencies:** `DGR-017`
- **Blocks (derived):** `DGR-021`, `DGR-025`
- **Labels:** `area:planning`, `area:gitea`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
- **Evidence class:** `model-free`
- **Hardware:** `none`
- **Model:** `generic`
- **Upstream:** `no`
## Objective / description
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make `prd.json` the validated source from which Markdown and Gitea issues can later be generated losslessly.
## Acceptance criteria
- [x] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
- [x] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
- [x] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
- [x] Add deterministic model-free tests for parse, validation, and generation round trips.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
- `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -0,0 +1,40 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-019: Lock alpha and beta performance contracts
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `HITL`
- **Milestone:** `M0`
- **Dependencies:** `DGR-017`
- **Blocks (derived):** `DGR-020`, `DGR-044`, `DGR-054`
- **Labels:** `area:performance`, `type:contract`, `priority:p0`, `gate:hitl`, `ready-for-human`
- **Evidence class:** `release`
- **Hardware:** `required`
- **Model:** `generic+deepseek-v4`
- **Upstream:** `no`
## Objective / description
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.
## Acceptance criteria
- [x] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
- [x] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
- [x] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
- [x] Treat quants and 24/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
- [x] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
- `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-019/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,59 +0,0 @@
# 02 — Adopt the versioned gRPC Shard protocol
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
- This issue is `DGR-002` in [prd.json](../prd.json).
- Read the evidence README for every dependency listed below.
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
## Description
As a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.
## Expected durable outputs
- packages/node/native/proto/shard_runtime.proto
- Reproducible Python/C++ schema generation and build wiring
- Protocol round-trip and compatibility tests
- evidence/DGR-002/README.md
## Acceptance criteria
- [ ] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
- [ ] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
- [ ] Define bounded chunking for prefill and a small decode fast path.
- [ ] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
- [ ] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
- [ ] Add generated-schema round-trip and compatibility tests in Python and C++.
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
## Dependency handoff
- None. This story may start immediately.
## Finish contract
- Create the task evidence directory and durable handoff required above.
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
- Change this issue to `Status: done` only after all criteria pass.
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
## References
- [Ralph execution context](../RALPH-CONTEXT.md)
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -0,0 +1,39 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-020: Run the controlled whole-model GGUF baseline
- **Status / triage:** completed; `passes: true`
- **Execution mode:** `HITL`
- **Milestone:** `M0`
- **Dependencies:** `DGR-019`
- **Blocks (derived):** `DGR-054`
- **Labels:** `area:performance`, `type:benchmark`, `priority:p0`, `gate:hitl`, `ready-for-human`
- **Evidence class:** `real-hardware`
- **Hardware:** `required`
- **Model:** `generic`
- **Upstream:** `no`
## Objective / description
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.
## Acceptance criteria
- [x] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
- [x] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
- [x] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
- [x] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
- `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-020/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

Some files were not shown because too many files have changed in this diff Show More