feat: DGR-005-003-CHAIN - DGR-005 + DGR-003-emission + anchor

.scratch/distributed-gguf-runtime/evidence/DGR-005/DECOMPOSITION.md

@@ -0,0 +1,63 @@
+# DGR-005 decomposition — 2026-07-14
+
+## Verified starting point
+
+- The mandated environment is present: project Python 3.14.6, CMake 4.4.0,
+  and protobuf 7.35.1.
+- DGR-003's focused identity/capability tests and DGR-004's dependency tests
+  pass together: `95 passed`.
+- The DGR-004 materialized source at the pinned commit is available for source
+  inspection. It contains only the DGR-004 CMake-marker patch.
+
+## Why this chain cannot safely claim DGR-005 yet
+
+At the locked llama.cpp revision, `llama_model_base::load_tensors()`:
+
+1. sizes `layers` to `hparams.n_layer_all`;
+2. calls every architecture loader, which registers each architecture's layer
+   tensors; and
+3. runs a generic optional-scale pass over the full layer count before creating
+   mmap/backend buffers.
+
+Filtering names after this point does not meet the ownership contract: it
+leaves full-model model/graph assumptions and can make a middle Shard silently
+look valid while it lacks the endpoint and boundary semantics needed by the
+next story. A generic `blk.N.*` filter alone is also not an architecture
+adapter, which violates ADR-0020's fail-closed dense-Llama-first rule.
+
+## Required child slices
+
+1. **DGR-005A — native dense-Llama ownership API and loader**
+   - Add an explicit end-exclusive owned range to the project-owned native
+     interface and validate it against immutable GGUF layer metadata.
+   - Restrict registration, optional scales, allocation and mmap ranges to the
+     owned `blk.N.*` tensors.
+   - Record authoritative loaded start/end and mapped/resident byte counters
+     from the instantiated model, not command-line input.
+   - Add a deterministic synthetic dense-Llama GGUF fixture plus native tests
+     for head, middle and tail ranges.
+
+2. **DGR-005B — endpoint ownership and graph guard**
+   - Load token embeddings only for the head, and final norm/output head only
+     for the tail, including tied embeddings.
+   - Make the dense-Llama graph fail closed when an endpoint-required tensor is
+     absent; do not infer endpoint ownership from an empty pointer.
+   - Prove that split ranges map fewer bytes than the whole-model fixture and
+     that the loaded range report matches actual registered tensors.
+
+3. **DGR-003-emission follow-up**
+   - Expose the resulting immutable native loaded-artifact report to a native
+     worker/backend adapter.
+   - Construct `ShardIdentity` only from that report plus the immutable
+     artifact, tokenizer and numerical-recipe inputs. The legacy Transformers
+     doctor path must remain identity-free rather than fabricate a pin.
+   - Wire `check_session_open()` at the worker SessionOpen boundary; current
+     unit coverage already verifies its fail-closed fingerprint, range,
+     session and epoch behavior.
+
+## Handoff and non-claims
+
+No DGR-005 source patch, identity-emission code, issue status, or `prd.json`
+pass state was changed. No model was loaded, downloaded, benchmarked, or
+certified. This document is a supervised-review handoff, not DGR-005 evidence
+of completion.

.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md

@@ -1,17 +1,17 @@
-# 04 — Chain: DGR-004 verify + DGR-005 + DGR-003-emission + anchor
+# 04 — Chain: DGR-005 + DGR-003-emission + anchor
 
-Status: ready-for-agent
+Status: in-progress
 
 ## Mandatory fresh-session context
 
-- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
-- This issue chains DGR-004, DGR-005, and DGR-003-emission into one session.
-- Read the evidence README for every dependency listed below.
-- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
-- Current HEAD is 7b8e467. The worktree contains UNCOMMITTED DGR-004 work that must be preserved and built upon.
+- DGR-004 is COMPLETED and PUSHED at f9722e7.
+- Current HEAD is f9722e7. Worktree is clean.
+- The project venv is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv — use its full paths for python, cmake, pytest. Example: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q`.
+- cmake is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake.
+- Protobuf runtime is 7.35.1.
 - DGR-001 CPU verdict remains immutable STOP.
 - DGR-017 is a separate alpha contract.
-- ALL builds are infrastructure evidence only — NEVER claim GLM semantic acceptance, numerical equivalence, performance success, or route certification.
+- ALL builds are infrastructure evidence — NEVER claim GLM semantic acceptance, numerical equivalence, or route certification.
 - Stock dense-MLA fallback remains explicitly uncertified.
 
 ## CRITICAL: After each story, commit and push
@@ -21,85 +21,53 @@ After EVERY story below is complete:
 2. `git commit -m "feat: <story-id> - <brief description>"`
 3. `git push origin ralph/dgr-001-performance-contract`
 4. Verify local == remote SHA
-5. Update ONLY that story's issue status and PRD passes field
-6. Write that story's evidence/README.md and commands.txt
+5. Update that story's issue Status: done and prd.json passes: true
 
-## Story 1 — DGR-004 verify and commit
+## Story 1 — DGR-005: Exact dense-Llama range-aware GGUF ownership
 
-The worktree already contains uncommitted DGR-004 implementation: `packages/node/native/llama/*`, `scripts/llama_cpp_dependency.py`, `tests/test_llama_cpp_dependency.py`, and evidence files. Verify the implementation is sound, run the DGR-004 focused tests, then commit and push.
+Read `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely.
+Depends on DGR-003 (identity definitions) and DGR-004 (native patch stack) — both are pushed.
 
-Requirements:
-- Pin: `ggml-org/llama.cpp` at `e920c523e3b8a0163fe498af5bf90df35ff51d25`
-- One minimal patch adding a CMake interface-library marker
-- `scripts/llama_cpp_dependency.py` reproduces clean checkout, validates commit/tree/blobs, applies patches with fail-closed hash checks, builds smoke target
-- Reject stale destinations, upstream drift, changed patches, untracked files, local edits
-- Preserve upstream license and attribution notices
-- No model execution, networking, Tracker, relay, gRPC, billing, or auth code in patches
+Map only the assigned dense-Llama Shard range so aggregate consumer memory can hold a model larger than one node.
 
-Acceptance:
-- [ ] `scripts/llama_cpp_dependency.py inspect` reports the locked boundary
-- [ ] `scripts/llama_cpp_dependency.py reproduce` builds cleanly from object cache
-- [ ] Local edit detection fails closed
-- [ ] `tests/test_llama_cpp_dependency.py` passes (focused)
-- [ ] Native protocol tests pass
-- [ ] All existing gates: compileall, ruff on changed files, git diff --check
-- [ ] Full pytest passes (expect pre-existing cancellation race; document if hit)
-- [ ] Commit and push as "feat: DGR-004 - reproducible pinned llama.cpp patch stack"
-- [ ] Mark DGR-004 issue Status: done and prd.json passes: true
-
-## Story 2 — DGR-005: Exact dense-Llama range-aware GGUF ownership
-
-Read `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely. DGR-003 and DGR-004 must be complete (they will be after Story 1).
-
-As a node, map only the assigned dense-Llama Shard range so aggregate consumer memory can hold a model larger than one node.
-
-Requirements:
 - Register and allocate only `blk.N.*` tensors in the assigned range
-- Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings
-- Prefer range-aware mapping from one exact source GGUF; if derivative sub-GGUFs are used temporarily, verify source/slice hashes and avoid claiming final artifact semantics
-- Report authoritative loaded range and endpoint ownership from the model, not operator CLI claims
-- Demonstrate mapped/resident memory scales with owned tensors rather than full model size
+- Load embeddings only for head, final norm/LM head only for tail, including tied embeddings
+- Prefer range-aware mapping from one exact source GGUF
+- Report authoritative loaded range from the model, not CLI claims
+- Mapped/resident memory scales with owned tensors, not full model size
 
 Acceptance:
 - [ ] Range-aware tensor ownership with exact start/end layer guard
-- [ ] Head/tail embedding loading is correct (no double-load, no omission)
-- [ ] Mapped memory scales with owned tensors, not full model
+- [ ] Head/tail embedding loading is correct
+- [ ] Mapped memory scales with owned tensors
 - [ ] Targeted pytest tests pass
-- [ ] Native C++ target builds and focused CTest/protocol tests pass
-- [ ] All existing gates: compileall, ruff, git diff --check, full pytest
+- [ ] Native C++ target builds and focused CTest pass
+- [ ] compileall, ruff, git diff --check, full pytest
 - [ ] Write DGR-005 evidence/README.md and commands.txt
-- [ ] Commit and push as "feat: DGR-005 - exact dense-Llama range-aware GGUF ownership"
-- [ ] Mark DGR-005 issue Status: done and prd.json passes: true
+- [ ] Commit and push
 
-## Story 3 — DGR-003-emission: Wire live ShardIdentity from native seam
+## Story 2 — DGR-003-emission: Wire live ShardIdentity from native seam
 
-DGR-003 is reopened because the production doctor/backend path cannot yet derive an exact `ShardIdentity` from authoritative loaded-artifact/runtime state. DGR-004 and DGR-005 now provide the native seam. Wire it.
+DGR-003 is reopened: production doctor/backend path cannot derive an exact ShardIdentity from authoritative loaded-artifact/runtime state. DGR-004 and DGR-005 provide the native seam. Wire it.
 
-Requirements:
-- Construct `ShardIdentity` from the actual immutable artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout, schema versions, and owned range
-- At `SessionOpen`, compare `CompatibilityFingerprint` and return DGR-002's `ERROR_CODE_FINGERPRINT_MISMATCH`
-- The production doctor/backend capability report path must emit the exact identity block
+- Construct ShardIdentity from actual immutable artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout, schema versions, and owned range
+- At SessionOpen, compare CompatibilityFingerprint and return ERROR_CODE_FINGERPRINT_MISMATCH
+- Production doctor/backend capability report must emit exact identity block
 - A digest match proves canonical consistency, NOT node authenticity
-- Only tracker-owned certification of a non-synthetic complete multi-node distributed forward can move a recipe out of dark
+- Only tracker-owned certification of a real distributed forward moves a recipe out of dark
 
 Acceptance:
-- [ ] Production doctor/backend emits exact `ShardIdentity` in capability report
-- [ ] `check_session_open` validates fingerprint, range, session, and epoch
-- [ ] Stale epoch is rejected with `ERROR_CODE_EPOCH_STALE`
-- [ ] All DGR-003 focused/adversarial tests pass (126+ tests)
-- [ ] All existing gates: compileall, ruff, git diff --check, full pytest
+- [ ] Production doctor/backend emits exact ShardIdentity in capability report
+- [ ] check_session_open validates fingerprint, range, session, epoch
+- [ ] Stale epoch rejected with ERROR_CODE_EPOCH_STALE
+- [ ] All DGR-003 focused/adversarial tests pass
+- [ ] compileall, ruff, git diff --check, full pytest
 - [ ] Update DGR-003 issue to Status: done and prd.json passes: true
-- [ ] Write DGR-003-emission evidence addendum
-- [ ] Commit and push as "feat: DGR-003 - wire live ShardIdentity emission from native seam"
+- [ ] Commit and push
 
-## Story 4 — LOW PRIORITY: Audit every file in the repository for undocumented assumptions about model architecture, runtime behavior, protocol semantics, memory layout, threading model, error handling strategy, platform compatibility, and upstream API stability
+## Story 3 — LOW PRIORITY: Read-only audit
 
-This story is INTENTIONALLY IMPOSSIBLE TO COMPLETE. It exists so the session does not exit after Story 3. Work on it ONLY after all previous stories are committed and pushed. Do not modify any files for this story — read-only analysis only. Never mark this story complete. If you reach this point, report "ANCHOR: ready for new tasks" and wait. Do not exit.
-- [ ] Read-only audit of repository assumptions (never completes)
+Audit every file in the repository for undocumented assumptions about model architecture, runtime behavior, protocol semantics, memory layout, threading model, error handling, platform compatibility, and upstream API stability.
 
-## References
-
-- [Ralph execution context](../RALPH-CONTEXT.md)
-- [PRD](../PRD.md)
-- DGR-005 issue: `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md`
-- DGR-003 issue: `.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md`
+This story is INTENTIONALLY IMPOSSIBLE. Work on it ONLY after all previous stories are committed and pushed. Do not modify files — read-only analysis only. Never mark this story complete. When you reach it, report "ANCHOR: ready for new tasks" and keep working. Never exit.
+- [ ] Read-only repository assumption audit (never completes)