6 Commits

Author SHA1 Message Date
Dobromir Popov
db59caa8e9 [verified] fix: enforce canonical native runtime pin 2026-07-17 23:20:03 +03:00
Dobromir Popov
ad66f7a4d8 feat: pin runtime identity to the exact llama.cpp patch stack (DGR-025)
Derive the recipe's runtime_version axis from the DGR-027 lock manifest
(exact upstream commit + ordered patch-stack byte digest) in new
meshnet_node.runtime_pin, failing closed on any lock/series/SHA256SUMS/
patch disagreement, and enforce pin discipline on runtime_version in both
the node and tracker identity implementations.

Also repair pre-existing backlog consistency: add missing DGR-022/DGR-027
completionNotes, regenerate the DGR-022/025/027 issue projections, and
relocate three pre-DGR legacy GLM issue files to issues/legacy/. Mark
DGR-025 passes=true with evidence at evidence/DGR-025/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 22:59:22 +03:00
Dobromir Popov
f83cf331c3 [verified] feat: harden llama.cpp provenance workspace 2026-07-17 16:24:46 +03:00
Dobromir Popov
ae51526e85 Merge remote-tracking branch 'origin/ralph/distributed-gguf-runtime' into ralph/distributed-gguf-runtime 2026-07-17 15:32:17 +03:00
Dobromir Popov
8563d218c9 Merge branch 'ralph/distributed-gguf-runtime' of https://git.d-popov.com/popov/neuron-tai into ralph/distributed-gguf-runtime 2026-07-17 13:33:08 +02:00
Dobromir Popov
66d9888a11 memory 2026-07-17 13:19:14 +02:00
23 changed files with 1237 additions and 77 deletions

View File

@@ -44,6 +44,10 @@ Historical handoff note: `/mnt/c/Users/popov/Downloads/neuron-tai-alpha-handoff-
Planning is ready at `.scratch/node-capability-admission/` with five sequential Ralph stories and ADR-0023. The design is model-agnostic: a Node must validate its selected Model Artifact/shard with a bounded real forward before Tracker routing; Qwen3.6 is only an optional development fixture. P0 adds a versioned local recipe-manifest/report contract, `meshnet-node doctor`, fail-closed startup admission, and tracker route gating. It intentionally excludes dynamic recipe/dependency installation and the future signed Node updater. Planning is ready at `.scratch/node-capability-admission/` with five sequential Ralph stories and ADR-0023. The design is model-agnostic: a Node must validate its selected Model Artifact/shard with a bounded real forward before Tracker routing; Qwen3.6 is only an optional development fixture. P0 adds a versioned local recipe-manifest/report contract, `meshnet-node doctor`, fail-closed startup admission, and tracker route gating. It intentionally excludes dynamic recipe/dependency installation and the future signed Node updater.
## Gitea DGR sync (2026-07-17)
Gitea is ahead of the local Markdown backlog with open DGR-022..DGR-071. The first executable P0 dependency frontier is DGR-022 (Shard lifecycle and structured status RPCs), DGR-023 (reproducible protobuf generation), DGR-025 (artifact/runtime recipe identity), and DGR-027 (llama.cpp provenance manifest). DGR-021, the named-tensor stream envelope prerequisite for DGR-022/023/025, is closed. DGR-022 is the next dependency-ordered issue and blocks DGR-024, DGR-033, and DGR-037.
## Windows CUDA node (working as of 2026-07-01) ## Windows CUDA node (working as of 2026-07-01)
- miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118 - miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118
- RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200 - RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200

View File

@@ -0,0 +1,210 @@
# DGR-025 evidence — exact artifact and runtime recipe identity
**Completed:** 2026-07-17
**Branch:** `ralph/fable-architecture-loop` (Claude Fable architecture lane)
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-018 (`evidence/DGR-018/README.md` — canonical backlog schema and
issue projection), DGR-021 (`evidence/DGR-021/README.md` — versioned activation
envelope). Both read before changing code.
## Objective
Ensure the tracker and worker only combine numerically and operationally
compatible shards: fingerprint every axis that moves the numbers, bind shards to
exact half-open ranges, fail closed on any mismatch, and keep uncertified
recipes registered-but-dark.
## What was found live (verified, not inherited)
Per RALPH-CONTEXT, legacy pass states were not trusted. The DGR-003-lineage
identity core was inspected and exercised live before any change:
- `packages/node/meshnet_node/runtime_recipe.py` — node-side identity:
domain-separated digests (`meshnet.model-artifact.v1`,
`meshnet.runtime-recipe.v1`, `meshnet.shard-binding.v1`) over the source
artifact SHA (`source_digest`, with split artifacts bound to their exact
source via `DerivativeBinding`), tokenizer revision (pin-enforced),
architecture adapter + architecture/config digest, boundary and protocol
schema versions, backend, weight quantization, activation/compute dtypes, and
KV dtype/layout (`RECIPE_AXES`). Shard ranges are half-open
(`shard_start`/`shard_end`, end-exclusive, protocol convention) with no
topology or quant constants anywhere; `check_route` accepts any tiling of
`[0, layer_count)`. Route, handshake (`check_handshake`), and session-open
(`check_session_open`) checks fail closed with structured `RouteMismatch`
reasons mapped to specific protocol error codes (`handshake_error`).
- `packages/tracker/meshnet_tracker/recipe.py` — deliberately independent
tracker re-derivation (no `meshnet_node` import); declared fingerprints are
recomputed, never trusted (`parse_identity`, `FingerprintMismatch`). The
`CertificationLedger` keeps every registered recipe dark until a real
distributed forward — at least 2 distinct nodes, whole-model coverage,
non-synthetic, tokens actually generated — certifies it; dark recipes may
route only to certify.
- The two implementations are pinned by committed conformance vectors
(`tests/data/recipe_fingerprint_vectors.json`).
Live verification of that pre-existing core before changes:
`PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_recipe_identity.py`
`45 passed`; plus `tests/test_native_identity_emission.py`,
`tests/test_tracker_capability_admission.py`, `tests/test_node_admission.py`
`59 passed`.
## Gap found and closed (this story's change)
**The `runtime_version` recipe axis was a label, not a pin.** It was an opaque
caller-supplied string: nothing derived it from the DGR-027 lock manifest, and
neither identity implementation rejected a moving reference (`"latest"` was
accepted), so two workers could run different llama.cpp pins or patch stacks
under one label and still agree on the recipe digest. The acceptance criterion
explicitly requires fingerprinting the "runtime pin/patch stack".
### Changed files
- `packages/node/meshnet_node/runtime_pin.py` (new) — derives the canonical
`runtime_version` axis value from the DGR-027 lock workspace
(`packages/node/native/llama`):
`<runtime>@<40-hex upstream commit>+patchstack.<sha256>` where the stack
digest commits, under the `meshnet.runtime-patch-stack.v1` domain, to the
*ordered* `(patch name, patch bytes sha256)` stack. Fails closed on: missing
or malformed `UPSTREAM_LOCK.json`, unknown schema version, non-40-hex/moving
commit, `UPSTREAM_COMMIT` disagreement, any disagreement among the lock's
`patch_series`, `patches/series`, and `patches/SHA256SUMS`, a missing patch
file, or a patch whose bytes don't match their recorded digest. Reads the
committed manifest only; fetching/patching stays with
`scripts/llama_cpp_dependency.py` (DGR-027).
- `packages/node/meshnet_node/runtime_recipe.py``runtime_version` is now
pin-enforced (`_require_pin`) exactly like `tokenizer_revision`; for the
llama.cpp backend it must also match the canonical
`llama.cpp@<40-hex>+patchstack.<64-hex>` grammar.
- `packages/node/meshnet_node/native_backend.py` — the production native
identity seam no longer accepts a caller-supplied runtime string. It derives
`runtime_version` directly through `load_runtime_pin()` from the committed
lock and rejects a non-llama backend at this llama.cpp-specific boundary.
- `packages/tracker/meshnet_tracker/recipe.py` — the independent tracker
implementation applies the same backend-specific grammar before re-deriving
the recipe digest, so forged operator labels cannot register or certify.
- `tests/test_runtime_pin_identity.py` and
`tests/test_native_identity_emission.py` — deterministic tests cover lock
derivation, production native emission, and node/tracker rejection of the
forged values from independent review. Conformance vectors were regenerated
through `scripts/gen_recipe_fingerprint_vectors.py` for the tightened wire
contract.
### Backlog-consistency repair (pre-existing damage, honestly recorded)
`tests/test_ralph_prd_schema.py` had 4 pre-existing failures before this story
touched anything, left by prior sessions and the alternate-history merge:
- DGR-022 and DGR-027 were marked `passes: true` without `completionNotes` and
without regenerated issue projections. Added their `completionNotes`
(explicitly labeled as added during this repair, content drawn from their own
evidence READMEs) and regenerated
`issues/022-…` / `issues/027-…` via `scripts/ralph_prd_schema.py render`.
- Three pre-DGR legacy GLM alpha issue files (`18-…`, `19-…`, `20-…`,
committed 2026-07-14, before DGR-018 established the generated-only
convention; they carry no authority disclaimer because they are *not*
generated from prd.json) were relocated via `git mv` to
`issues/legacy/` — preserved as provenance, out of the generated namespace.
### prd.json
Marked `DGR-025.passes = true` with `completionNotes`; regenerated
`issues/025-define-exact-artifact-and-runtime-recipe-identity.md`.
## Acceptance criteria → evidence
1. **Fingerprint all axes**`RECIPE_AXES` + `ArtifactIdentity` cover source
artifact SHA, tokenizer revision, architecture adapter/version (adapter axis
+ architecture/config digest), boundary schema (boundary + protocol schema
versions), backend, quant, activation/compute dtype, KV/state layout; the
runtime pin/patch stack is now committed via the derived `runtime_version`
axis (`runtime_pin.py`). Verified by `test_runtime_recipe_identity.py` and
`test_runtime_pin_identity.py`.
2. **Exact half-open range, no hardcoded topology/quant**`ShardIdentity`
end-exclusive ranges, `DerivativeBinding` coverage checks, `check_route`
tiling over arbitrary layouts; quant/dtype values are open strings
(dynamic recipe inputs). Verified by `test_runtime_recipe_identity.py`
(routes of 1, 2, and 5 shards; no product constants).
3. **Fail closed on any mismatch** — artifact, adapter, boundary/schema, cache
layout, backend, and runtime mismatches each produce structured
`RouteMismatch` reasons and protocol error codes; the tracker recomputes
digests and rejects inconsistent claims; moving runtime references are now
rejected on both sides.
4. **Registered-but-dark**`CertificationLedger`: unknown recipes cannot be
certified, registered recipes are dark, only a real ≥2-distinct-node
whole-model non-synthetic forward promotes; verified by
`test_runtime_recipe_identity.py` / `test_tracker_capability_admission.py`.
5. **Gates + this handoff** — below.
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_pin_identity.py
```
```text
23 passed in 0.15s
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py \
tests/test_native_identity_emission.py tests/test_tracker_capability_admission.py \
tests/test_node_admission.py tests/test_node_capability.py tests/test_recipe_benchmark.py
```
```text
202 passed, 1 warning in 5.38s
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_ralph_prd_schema.py
```
```text
108 passed
```
(4 failed before this story's backlog repair; 0 after.)
```bash
python3 -m compileall -q packages tests # exit 0
git diff --check # exit 0
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
# OK: 55 stories validated.
```
Default tests are model-download-free, API-credit-free, and GPU-free; no model
artifact was touched and nothing was written under `/home`.
## Limitations
- The production native identity seam now derives the manifest pin and cannot
accept an operator-supplied runtime label. It still cannot attest that the
running binary was built from those locked bytes. Embedding the patched-tree
hash at build time and echoing it through the DGR-022 status contract belongs
with DGR-028+/DGR-031; real distributed certification remains the final trust
boundary.
- The DGR-027-recorded blocker stands: `0002-dense-llama-owned-range-loader.patch`
does not apply cleanly against the pin (DGR-028). That does not affect this
story: the identity commits to the patch *bytes as committed*, which is
precisely what makes a later repaired patch a *different* runtime identity.
- No native/CMake change was made, so the native build/CTest gate is not
applicable; no llama.cpp patch content was changed, so apply/check/reverse
verification is not applicable (and is blocked by the DGR-028 defect anyway).
- Tracker routing, load balancing, billing, telemetry, and relay semantics are
untouched; the only behavior change outside the new module is the stricter
(fail-closed) rejection of moving `runtime_version` values.
## Dependency handoff
- **DGR-026** (split-GGUF provisioning): bind each provisioned split via
`DerivativeBinding` to the exact source digest recorded in its hashed
manifest; the per-split `shard_binding_digest` is what certification pins.
- **DGR-031** (`ShardEngine`): construct worker identity through
`shard_identity_from_native_report` and populate `runtime_version` from
`meshnet_node.runtime_pin.load_runtime_pin().runtime_version` — never from an
operator string. A build-time echo of the patched-tree hash through the
status contract would close the manifest-vs-binary gap noted above.
- **DGR-041** (capability registration): the tracker already re-derives and
fail-closes on presented identities (`parse_identity`); register recipes
through the `CertificationLedger` so they arrive dark.
- **DGR-044** (DeepSeek V4 Flash target): pin the target's artifact identity
the same way `glm_alpha_artifact` does — read locked manifests, never restate
digests — and note `layer_count` must count the routed transformer stack the
route tiles, excluding MTP (reserved for beta).

View File

@@ -0,0 +1,72 @@
# DGR-027 evidence — exact llama.cpp provenance manifest and fetch workspace
**Completed implementation:** 2026-07-17
**Branch:** `ralph/dgr-small-terra`
**Authority:** live Gitea issue #11. The controller fetched and claimed the issue
through the Gitea API before launch; the isolated agent received that exact body.
## Changed files
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/PATCH-STACK.md`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`
## Provenance and retrieval contract
`UPSTREAM_LOCK.json` records the upstream Git URL, immutable 40-character
commit `e920c523e3b8a0163fe498af5bf90df35ff51d25`, expected Git tree
`6c91a11407a3a3fb160f5dac705f9c59718f54f1`, MIT license, and the sole
retrieval method: `git-clone-detached-commit` into `build/llama.cpp/source`.
`python3 scripts/llama_cpp_dependency.py fetch` has no branch, tag, ref, or
repository override. On a first fetch it clones the manifest URL, checks out
the detached commit, and verifies commit, tree, required upstream blobs,
license, and cleanliness. If the workspace already exists, it makes no network
request and accepts it only after the same verification. Dirty or mismatched
caches fail closed. The build directory is already ignored by `.gitignore`.
## Verification
| Command | Result |
| --- | --- |
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py` | `7 passed in 0.22s` |
| `python3 -m compileall packages tests` | passed |
| `git diff --check` | passed (no output) |
| `python3 scripts/llama_cpp_dependency.py inspect` | passed; reports exact commit/tree, retrieval workspace, MIT license, and two-patch stack |
| `python3 scripts/llama_cpp_dependency.py fetch --workspace /tmp/not-llama-workspace` | failed closed with status 2: workspace outside the locked ignored build root |
| symlinked workspace regression | passed; both a `build/` ancestor symlink and a final `source` symlink escaping the repository are refused |
| attached-branch cache regression | passed; an exact commit on a local branch is refused until checked out as detached HEAD |
| ignored/excluded injection regression | passed; a file hidden by `.git/info/exclude` is detected and refused |
| tracked injection regression | passed; modified tracked content hidden by both `assume-unchanged` and `skip-worktree` is content-hashed and refused |
| executable-mode regression | passed on the POSIX fixture for both index flags; the mounted project workspace has `core.filemode=false`, so its exact index tree is the canonical mode record and physical mode bits are not treated as meaningful |
| `git check-ignore -v build/llama.cpp/source` | passed; `.gitignore:6:build/` |
| `git diff --summary` and `git ls-files build packages/node/native/llama` | no source checkout or new submodule introduced; only manifest/docs/patches/native wrapper are tracked |
| `python3 scripts/llama_cpp_dependency.py fetch` (controller network lane) | passed; fetched the exact detached commit and verified HEAD `e920c523e3b8a0163fe498af5bf90df35ff51d25` and tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1` in the ignored workspace |
| `python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source` | failed on the pre-existing `0002-dense-llama-owned-range-loader.patch` as a corrupt patch at line 26; this is an explicit DGR-028 blocker and no native-build claim is made |
The targeted test suite creates a local Git fixture to prove offline cache reuse
after full identity verification, then proves a dirty cache is rejected. It
also proves the CLI rejects a repository/branch override and an arbitrary
workspace.
## Limitations
- The controller successfully materialized and verified the exact upstream
commit/tree, so the DGR-027 fetch and offline-cache boundary has real upstream
evidence rather than fixture-only evidence.
- The existing `0002-dense-llama-owned-range-loader.patch` is malformed and
cannot pass `git apply --check` against the exact pin. DGR-027 changes no patch
file; repairing and certifying the numbered patch stack belongs to DGR-028.
Until that story closes, the repository must not claim patched-tree, native
CMake/CTest, or reverse-apply certification.
- No model, API credits, GPU, or model artifact storage was used.
## Dependency handoff
DGR-028, DGR-029, and DGR-044 must invoke the manifest-owned `fetch` command
before touching llama.cpp source. They may use only the verified
`build/llama.cpp/source` checkout and must record any native build, CTest, and
patch apply/check/reverse evidence against the exact manifest pin. DGR-017's
cleanup remains provenance only and grants no inherited completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-022: Define Shard lifecycle and structured status RPCs # DGR-022: Define Shard lifecycle and structured status RPCs
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-021` - **Dependencies:** `DGR-021`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs. - [x] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy. - [x] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker. - [x] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions. - [x] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-022/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-022/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-025: Define exact artifact and runtime recipe identity # DGR-025: Define exact artifact and runtime recipe identity
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-018`, `DGR-021` - **Dependencies:** `DGR-018`, `DGR-021`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout. - [x] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant. - [x] Bind each shard to an exact half-open range without hardcoding a topology or quant.
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch. - [x] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them. - [x] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-027: Add exact llama.cpp provenance manifest and fetch workspace # DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-017` - **Dependencies:** `DGR-017`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method. - [x] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag. - [x] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced. - [x] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
- [ ] Offline reuse is supported only after the cached trees exact identity is verified. - [x] Offline reuse is supported only after the cached trees exact identity is verified.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -489,7 +489,8 @@
"DGR-024", "DGR-024",
"DGR-033", "DGR-033",
"DGR-037" "DGR-037"
] ],
"completionNotes": "Completed 2026-07-17. Implemented the versioned backend-neutral Shard lifecycle/status contract in packages/node/meshnet_node/shard_lifecycle.py (RPC names, schema-version negotiation with fail-closed unsupported versions, structured status/error taxonomy, lifecycle state machine, monotonic idempotency enforcement, bounded flow control, cache expectation/result types, deadline policy and TLS/auth hooks) with deterministic tests in tests/test_shard_lifecycle.py (17 passed alongside the DGR-021 envelope tests). Generated protobuf bindings remain DGR-023. Evidence: evidence/DGR-022/README.md. These completionNotes were added during the DGR-025 backlog-consistency repair; the DGR-022 session omitted them."
}, },
{ {
"id": "DGR-023", "id": "DGR-023",
@@ -597,14 +598,15 @@
"Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.", "Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-026", "DGR-026",
"DGR-031", "DGR-031",
"DGR-041", "DGR-041",
"DGR-044" "DGR-044"
] ],
"completionNotes": "Completed 2026-07-17. Verified the live DGR-003-lineage identity core against every criterion: node packages/node/meshnet_node/runtime_recipe.py and the independent tracker packages/tracker/meshnet_tracker/recipe.py (pinned together by tests/data/recipe_fingerprint_vectors.json) fingerprint the source artifact SHA, tokenizer pin, architecture adapter and config digest, boundary/protocol schema versions, backend, weight quant, activation/compute dtypes, and KV dtype/layout under domain-separated digests; shards bind to exact half-open ranges with no topology or quant constants; route/handshake/session checks fail closed with structured mismatch reasons; recipes stay registered-but-dark in the tracker CertificationLedger until a real >=2-distinct-node whole-model distributed forward certifies them. Closed the one open criterion gap (runtime pin/patch stack): new packages/node/meshnet_node/runtime_pin.py derives the runtime_version axis from the DGR-027 lock manifest — exact upstream commit plus a digest over the ordered patch-stack bytes — failing closed on any UPSTREAM_LOCK.json/UPSTREAM_COMMIT/series/SHA256SUMS/patch-byte disagreement, and both identity implementations now reject a moving runtime_version reference. Tests: tests/test_runtime_pin_identity.py (17 passed) plus 196 passing impacted identity/admission/native-emission tests; python -m compileall and git diff --check clean. Also repaired backlog consistency left by prior sessions: added the missing DGR-022/DGR-027 completionNotes, regenerated the DGR-022/025/027 issue projections, and relocated three pre-DGR legacy GLM alpha issue files to issues/legacy/."
}, },
{ {
"id": "DGR-026", "id": "DGR-026",
@@ -673,13 +675,14 @@
"Offline reuse is supported only after the cached trees exact identity is verified.", "Offline reuse is supported only after the cached trees exact identity is verified.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md; prd.json is authoritative.", "notes": "Completed from Gitea #11 via the Terra Ralph lane; independently reviewed after fail-closed cache identity hardening. Verified 7 focused tests, exact real-cache reuse, compileall, and diff-check. The pre-existing malformed 0002 patch is handed to DGR-028.",
"blocks": [ "blocks": [
"DGR-028", "DGR-028",
"DGR-029", "DGR-029",
"DGR-044" "DGR-044"
] ],
"completionNotes": "Completed 2026-07-17. Added the exact llama.cpp provenance manifest packages/node/native/llama/UPSTREAM_LOCK.json pinning commit e920c523e3b8a0163fe498af5bf90df35ff51d25 (tree 6c91a11407a3a3fb160f5dac705f9c59718f54f1) with the sole git-clone-detached-commit retrieval into the ignored build/llama.cpp workspace, fail-closed dirty/mismatched-cache verification, PATCH-STACK.md, scripts/llama_cpp_dependency.py, and tests/test_llama_cpp_dependency.py (7 passed). Known blocker recorded for DGR-028: 0002-dense-llama-owned-range-loader.patch fails git apply --check against the pin, so no patched-tree/native-build claim is made. Evidence: evidence/DGR-027/README.md. These completionNotes were added during the DGR-025 backlog-consistency repair; the DGR-027 session omitted them."
}, },
{ {
"id": "DGR-028", "id": "DGR-028",

View File

@@ -12,6 +12,7 @@ from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb
from .runtime_pin import load_runtime_pin
from .runtime_recipe import ( from .runtime_recipe import (
ArtifactIdentity, ArtifactIdentity,
DerivativeBinding, DerivativeBinding,
@@ -72,7 +73,6 @@ class NativeNumericalRecipe:
kv_layout: str kv_layout: str
architecture_adapter: str architecture_adapter: str
backend_id: str backend_id: str
runtime_version: str
recipe_id: str recipe_id: str
recipe_version: str recipe_version: str
catalogue_version: str catalogue_version: str
@@ -95,6 +95,11 @@ def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIden
report = inputs.loaded_artifact report = inputs.loaded_artifact
pin = inputs.artifact_pin pin = inputs.artifact_pin
recipe = inputs.numerical_recipe recipe = inputs.numerical_recipe
if recipe.backend_id.strip().lower() not in {"llama.cpp", "llama-cpp"}:
raise RecipeIdentityError(
"native llama.cpp identity requires backend_id 'llama.cpp' or 'llama-cpp'"
)
runtime_version = load_runtime_pin().runtime_version
artifact = ArtifactIdentity( artifact = ArtifactIdentity(
artifact_id=pin.artifact_id, artifact_id=pin.artifact_id,
revision=pin.revision, revision=pin.revision,
@@ -115,7 +120,7 @@ def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIden
tokenizer_revision=inputs.tokenizer_revision, tokenizer_revision=inputs.tokenizer_revision,
architecture_adapter=recipe.architecture_adapter, architecture_adapter=recipe.architecture_adapter,
backend_id=recipe.backend_id, backend_id=recipe.backend_id,
runtime_version=recipe.runtime_version, runtime_version=runtime_version,
boundary_schema_version=recipe.boundary_schema_version, boundary_schema_version=recipe.boundary_schema_version,
protocol_schema_version=recipe.protocol_schema_version, protocol_schema_version=recipe.protocol_schema_version,
recipe_id=recipe.recipe_id, recipe_id=recipe.recipe_id,

View File

@@ -0,0 +1,221 @@
"""Canonical runtime pin identity for the recipe fingerprint (DGR-025).
The recipe digest (:mod:`meshnet_node.runtime_recipe`) commits to a
``runtime_version`` axis, but a string the operator typed is a label, not a
pin: two workers could run different patch stacks under the same label and
still hash to the same recipe. The DGR-027 lock manifest
(``packages/node/native/llama``) already records the one exact upstream commit
and the ordered patch stack the native runtime is built from, so the axis
value is *derived* from that manifest, never asserted.
The derived value has three load-bearing parts, and each is separately fatal
to compatibility: the runtime name (from the upstream URL), the exact
40-character upstream commit, and a digest over the ordered patch-stack bytes.
A different upstream pin, a reordered stack, or a single changed patch byte
each produce a different axis value, which produces a different recipe digest,
which partitions the route — exactly the fail-closed behavior DGR-025 asks
for.
Every consistency check here fails closed. The manifest keeps three records of
the stack — ``UPSTREAM_LOCK.json``'s ``patch_series``, ``patches/series``, and
``patches/SHA256SUMS`` — plus the ``UPSTREAM_COMMIT`` convenience file, and a
disagreement between any two of them means the workspace's identity is
unknowable, not "probably fine". This module reads the committed manifest
only; fetching and patching the actual source tree stays with
``scripts/llama_cpp_dependency.py`` (DGR-027).
"""
from __future__ import annotations
import hashlib
import json
import re
from dataclasses import dataclass
from pathlib import Path
# Domain separation, matching the runtime_recipe digest convention: a patch
# stack digest can never be confused with an artifact or recipe digest.
PATCH_STACK_DIGEST_DOMAIN = "meshnet.runtime-patch-stack.v1"
# The UPSTREAM_LOCK.json layout this reader understands (DGR-027 schema).
RUNTIME_PIN_SCHEMA_VERSION = 1
# The committed DGR-027 manifest for the llama.cpp runtime.
DEFAULT_LOCK_DIR = Path(__file__).resolve().parent.parent / "native" / "llama"
_HEX40 = re.compile(r"^[0-9a-f]{40}$")
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
class RuntimePinError(ValueError):
"""The lock workspace is missing, malformed, or internally inconsistent."""
def _canonical_sha256(value: object) -> str:
payload = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
@dataclass(frozen=True)
class RuntimePin:
"""One exact runtime: a name, an upstream commit, and an ordered patch stack."""
runtime_name: str
upstream_commit: str
patch_series: tuple[str, ...]
patch_digests: tuple[str, ...]
@property
def patch_stack_digest(self) -> str:
"""A digest over the ordered (name, bytes-digest) stack.
Order is digested deliberately: applying the same patches in a
different order can produce a different tree, so a reordered stack is
a different runtime.
"""
return _canonical_sha256(
{
"domain": PATCH_STACK_DIGEST_DOMAIN,
"body": {
"patches": [
[name, digest]
for name, digest in zip(self.patch_series, self.patch_digests)
]
},
}
)
@property
def runtime_version(self) -> str:
"""The exact ``runtime_version`` recipe axis value for this pin."""
return (
f"{self.runtime_name}@{self.upstream_commit}"
f"+patchstack.{self.patch_stack_digest}"
)
def _read_text(path: Path, what: str) -> str:
try:
return path.read_text(encoding="utf-8")
except FileNotFoundError:
raise RuntimePinError(f"{what} not found at {path}") from None
except OSError as exc:
raise RuntimePinError(f"{what} at {path} is unreadable: {exc}") from exc
def _read_series_file(path: Path) -> list[str]:
lines = _read_text(path, "patches/series").splitlines()
return [line.strip() for line in lines if line.strip() and not line.startswith("#")]
def _read_sums_file(path: Path) -> list[tuple[str, str]]:
entries: list[tuple[str, str]] = []
for line in _read_text(path, "patches/SHA256SUMS").splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
parts = line.split(None, 1)
if len(parts) != 2 or not _HEX64.match(parts[0]):
raise RuntimePinError(
"patches/SHA256SUMS contains a line that is not "
"'<sha256> <patch name>'"
)
entries.append((parts[0], parts[1].strip()))
return entries
def load_runtime_pin(lock_dir: Path = DEFAULT_LOCK_DIR) -> RuntimePin:
"""Derive the exact runtime pin from a DGR-027 lock workspace, or refuse.
Refuses — rather than guessing — on a missing or malformed lock, a moving
commit reference, a disagreement between the lock's ``patch_series``, the
``patches/series`` file, ``patches/SHA256SUMS``, or the actual patch
bytes, and on an ``UPSTREAM_COMMIT`` file that names a different commit.
"""
lock_path = lock_dir / "UPSTREAM_LOCK.json"
raw = _read_text(lock_path, "UPSTREAM_LOCK.json")
try:
lock = json.loads(raw)
except json.JSONDecodeError as exc:
raise RuntimePinError(f"UPSTREAM_LOCK.json is not valid JSON: {exc}") from exc
if not isinstance(lock, dict):
raise RuntimePinError("UPSTREAM_LOCK.json must be a JSON object")
schema = lock.get("schema_version")
if schema != RUNTIME_PIN_SCHEMA_VERSION:
raise RuntimePinError(
f"UPSTREAM_LOCK.json declares schema version {schema!r}; this reader "
f"understands version {RUNTIME_PIN_SCHEMA_VERSION}"
)
upstream = lock.get("upstream")
if not isinstance(upstream, str) or not upstream.strip():
raise RuntimePinError("UPSTREAM_LOCK.json is missing the upstream URL")
runtime_name = upstream.rstrip("/").rsplit("/", 1)[-1]
if runtime_name.endswith(".git"):
runtime_name = runtime_name[: -len(".git")]
if not runtime_name:
raise RuntimePinError("the upstream URL does not name a runtime")
commit = lock.get("commit")
if not isinstance(commit, str) or not _HEX40.match(commit):
raise RuntimePinError(
f"UPSTREAM_LOCK.json commit {commit!r} is not an exact 40-character "
"hexadecimal object id; a moving reference is not a pin"
)
commit_file = _read_text(lock_dir / "UPSTREAM_COMMIT", "UPSTREAM_COMMIT")
recorded = commit_file.strip().splitlines()[0].strip() if commit_file.strip() else ""
if recorded != commit:
raise RuntimePinError(
"UPSTREAM_COMMIT and UPSTREAM_LOCK.json disagree on the pinned commit"
)
lock_series = lock.get("patch_series")
if not isinstance(lock_series, list) or not all(
isinstance(name, str) and name.strip() for name in lock_series
):
raise RuntimePinError(
"UPSTREAM_LOCK.json patch_series must be a list of patch file names"
)
if len(set(lock_series)) != len(lock_series):
raise RuntimePinError("UPSTREAM_LOCK.json patch_series contains a duplicate")
series = _read_series_file(lock_dir / "patches" / "series")
if series != lock_series:
raise RuntimePinError(
"patches/series and UPSTREAM_LOCK.json patch_series disagree on the "
"ordered patch stack"
)
sums = _read_sums_file(lock_dir / "patches" / "SHA256SUMS")
if [name for _, name in sums] != lock_series:
raise RuntimePinError(
"patches/SHA256SUMS does not record exactly the ordered patch stack "
"named by UPSTREAM_LOCK.json"
)
digests: list[str] = []
for (expected_digest, name) in sums:
patch_path = lock_dir / "patches" / name
try:
body = patch_path.read_bytes()
except FileNotFoundError:
raise RuntimePinError(f"patch file {name} is named but missing") from None
except OSError as exc:
raise RuntimePinError(f"patch file {name} is unreadable: {exc}") from exc
actual = hashlib.sha256(body).hexdigest()
if actual != expected_digest:
raise RuntimePinError(
f"patch file {name} does not match its patches/SHA256SUMS digest"
)
digests.append(actual)
return RuntimePin(
runtime_name=runtime_name,
upstream_commit=commit,
patch_series=tuple(lock_series),
patch_digests=tuple(digests),
)

View File

@@ -125,6 +125,10 @@ _AXIS_MISMATCH: Mapping[str, str] = {
} }
_HEX64 = re.compile(r"^[0-9a-f]{64}$") _HEX64 = re.compile(r"^[0-9a-f]{64}$")
_LLAMA_CPP_RUNTIME_PIN = re.compile(
r"^llama\.cpp@[0-9a-f]{40}\+patchstack\.[0-9a-f]{64}$"
)
_LLAMA_CPP_BACKEND_IDS = frozenset({"llama.cpp", "llama-cpp"})
# A revision that can move is not a pin. DGR-017 learned this on the artifact; # A revision that can move is not a pin. DGR-017 learned this on the artifact;
# it is just as true of a tokenizer. # it is just as true of a tokenizer.
@@ -194,6 +198,17 @@ def _require_pin(value: Any, what: str) -> str:
return text return text
def _require_runtime_pin(value: Any, backend_id: Any) -> str:
text = _require_pin(value, "recipe.runtime_version")
backend = _require_text(backend_id, "recipe.backend_id").strip().lower()
if backend in _LLAMA_CPP_BACKEND_IDS and not _LLAMA_CPP_RUNTIME_PIN.fullmatch(text):
raise RecipeIdentityError(
"'recipe.runtime_version' for llama.cpp must be "
"'llama.cpp@<40-hex commit>+patchstack.<64-hex digest>'"
)
return text
def _as_mapping(value: Any, what: str) -> Mapping[str, Any]: def _as_mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping): if not isinstance(value, Mapping):
raise RecipeIdentityError( raise RecipeIdentityError(
@@ -374,6 +389,11 @@ class RuntimeRecipe:
one in fp16, produce different logits from the same bytes. Keeping the axes one in fp16, produce different logits from the same bytes. Keeping the axes
apart is the entire safety property; see :data:`RECIPE_AXES`. apart is the entire safety property; see :data:`RECIPE_AXES`.
`tokenizer_revision` and `runtime_version` must be exact pins, never moving
references. For the native runtime the canonical `runtime_version` value —
committing to the exact upstream commit *and* the ordered patch stack — is
derived from the DGR-027 lock manifest by :mod:`meshnet_node.runtime_pin`.
The three label fields are diagnosis only and are not digested. The three label fields are diagnosis only and are not digested.
""" """
@@ -400,6 +420,7 @@ class RuntimeRecipe:
else: else:
_require_text(value, f"recipe.{axis}") _require_text(value, f"recipe.{axis}")
_require_pin(self.tokenizer_revision, "recipe.tokenizer_revision") _require_pin(self.tokenizer_revision, "recipe.tokenizer_revision")
_require_runtime_pin(self.runtime_version, self.backend_id)
_require_text(self.recipe_id, "recipe.recipe_id") _require_text(self.recipe_id, "recipe.recipe_id")
_require_text(self.recipe_version, "recipe.recipe_version") _require_text(self.recipe_version, "recipe.recipe_version")
_require_text(self.catalogue_version, "recipe.catalogue_version") _require_text(self.catalogue_version, "recipe.catalogue_version")

View File

@@ -15,11 +15,19 @@ loading, endpoint ownership, architecture-defined intermediate boundaries, and
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay, layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
billing, authentication, and telemetry must remain outside this directory. billing, authentication, and telemetry must remain outside this directory.
`scripts/llama_cpp_dependency.py` verifies the exact commit/tree and baseline `scripts/llama_cpp_dependency.py fetch` reads the in-repo manifest and checks out
blobs, validates every patch digest and context with `git apply --check`, then only its exact commit as detached HEAD in `build/llama.cpp/source`, an ignored
applies the series in `patches/series` order. It refuses a dirty source tree, build workspace. It verifies the exact commit/tree and baseline blobs before use. A
wrong commit/tree/blob, changed patch digest, reordered series, or an existing later offline `fetch` may reuse that cache only after the same clean identity
destination/work directory. verification; an attached branch, tag/repository override, arbitrary destination,
symlinked workspace, dirty checkout, ignored injected file, or tracked-file
modification hidden by Git index flags is refused. The tool
validates every patch digest and context
with `git apply --check`, then applies the series in `patches/series` order.
Tracked executable modes are checked physically when Git reports
`core.filemode=true`; on mounted filesystems without POSIX mode fidelity, the
locked index tree remains the canonical mode record while every working-file
blob is independently re-hashed.
## Current semantic boundary ## Current semantic boundary

View File

@@ -3,6 +3,13 @@
"upstream": "https://github.com/ggml-org/llama.cpp.git", "upstream": "https://github.com/ggml-org/llama.cpp.git",
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25", "commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1", "commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
"expected_source": {
"git_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1"
},
"retrieval": {
"method": "git-clone-detached-commit",
"workspace": "build/llama.cpp"
},
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452", "patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
"upstream_license": "MIT", "upstream_license": "MIT",
"patch_series": [ "patch_series": [

View File

@@ -70,6 +70,10 @@ STATUS_CERTIFIED = "certified"
MIN_CERTIFYING_NODES = 2 MIN_CERTIFYING_NODES = 2
_HEX64 = re.compile(r"^[0-9a-f]{64}$") _HEX64 = re.compile(r"^[0-9a-f]{64}$")
_LLAMA_CPP_RUNTIME_PIN = re.compile(
r"^llama\.cpp@[0-9a-f]{40}\+patchstack\.[0-9a-f]{64}$"
)
_LLAMA_CPP_BACKEND_IDS = frozenset({"llama.cpp", "llama-cpp"})
_MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"}) _MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"})
@@ -128,6 +132,17 @@ def _pin(value: Any, what: str) -> str:
return text return text
def _runtime_pin(value: Any, backend_id: Any) -> str:
text = _pin(value, "recipe.runtime_version")
backend = _text(backend_id, "recipe.backend_id").strip().lower()
if backend in _LLAMA_CPP_BACKEND_IDS and not _LLAMA_CPP_RUNTIME_PIN.fullmatch(text):
raise RecipeIdentityError(
"'recipe.runtime_version' for llama.cpp must bind a 40-hex commit "
"and a 64-hex ordered patch-stack digest"
)
return text
def _mapping(value: Any, what: str) -> Mapping[str, Any]: def _mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping): if not isinstance(value, Mapping):
raise RecipeIdentityError(f"{what!r} must be a JSON object") raise RecipeIdentityError(f"{what!r} must be a JSON object")
@@ -333,6 +348,7 @@ def parse_identity(data: Any) -> PresentedIdentity:
else: else:
axes[axis] = _text(value, f"recipe.{axis}") axes[axis] = _text(value, f"recipe.{axis}")
_pin(axes["tokenizer_revision"], "recipe.tokenizer_revision") _pin(axes["tokenizer_revision"], "recipe.tokenizer_revision")
_runtime_pin(axes["runtime_version"], axes["backend_id"])
identity = PresentedIdentity( identity = PresentedIdentity(
artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"), artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"),

View File

@@ -49,7 +49,7 @@ _RECIPE = RuntimeRecipe(
tokenizer_revision="0123456789abcdef", tokenizer_revision="0123456789abcdef",
architecture_adapter="llama/range-v1", architecture_adapter="llama/range-v1",
backend_id="llama.cpp", backend_id="llama.cpp",
runtime_version="llama.cpp@deadbeef+meshnet.1", runtime_version="llama.cpp@" + "d" * 40 + "+patchstack." + "e" * 64,
recipe_id="example-gguf", recipe_id="example-gguf",
recipe_version="1", recipe_version="1",
catalogue_version="2026.07.1", catalogue_version="2026.07.1",

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Materialize, verify, build, and smoke-test DGR-004's llama.cpp pin. """Fetch, verify, build, and smoke-test DGR-027's exact llama.cpp pin.
This tool deliberately owns only a source dependency boundary. It never This tool deliberately owns only a source dependency boundary. It never
downloads a model, invokes inference, or interprets generated text. downloads a model, invokes inference, or interprets generated text.
@@ -12,6 +12,7 @@ import hashlib
import json import json
import os import os
import pathlib import pathlib
import re
import shutil import shutil
import subprocess import subprocess
import sys import sys
@@ -67,14 +68,27 @@ def _load_lock() -> dict[str, Any]:
raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error
required = { required = {
"upstream", "commit", "commit_tree", "patched_tree", "patch_series", "upstream", "commit", "commit_tree", "patched_tree", "patch_series",
"required_upstream_blobs", "patched_paths", "build", "required_upstream_blobs", "patched_paths", "build", "upstream_license",
"expected_source", "retrieval",
} }
missing = sorted(required - lock.keys()) missing = sorted(required - lock.keys())
if missing: if missing:
raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}") raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}")
commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip() commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
if lock["commit"] != commit_file or len(commit_file) != 40: object_ids = [commit_file, lock["commit"], lock["commit_tree"], lock["patched_tree"]]
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on a full commit") if lock["commit"] != commit_file or not all(
isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value)
for value in object_ids
):
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on full hexadecimal object IDs")
if lock["expected_source"] != {"git_tree": lock["commit_tree"]}:
raise DependencyError("expected_source must record the locked git tree")
retrieval = lock["retrieval"]
if retrieval != {
"method": "git-clone-detached-commit",
"workspace": "build/llama.cpp",
}:
raise DependencyError("retrieval must use the locked detached-commit build workspace")
return lock return lock
@@ -103,6 +117,56 @@ def _git(source: pathlib.Path, *args: str) -> str:
return _run("git", "-C", str(source), *args) return _run("git", "-C", str(source), *args)
def _verify_tracked_content(source: pathlib.Path, lock: dict[str, Any]) -> None:
if _git(source, "write-tree") != lock["commit_tree"]:
raise DependencyError("materialized checkout index differs from the locked tree")
filemode_trusted = _git(source, "config", "--bool", "core.filemode") == "true"
records = _git(source, "ls-files", "-s", "-z").split("\0")
paths: list[str] = []
expected: list[str] = []
for record in records:
if not record:
continue
metadata, path = record.split("\t", 1)
mode, blob, stage = metadata.split()
if stage != "0" or mode not in {"100644", "100755", "120000"}:
raise DependencyError(f"unsupported tracked entry in materialized checkout: {record!r}")
if "\n" in path:
raise DependencyError(f"newline-bearing tracked path is unsupported: {path!r}")
candidate = source / path
cursor = source
for part in pathlib.Path(path).parts[:-1]:
cursor = cursor / part
if cursor.is_symlink():
raise DependencyError(f"tracked path traverses a symlink: {path!r}")
if mode == "120000":
if not candidate.is_symlink():
raise DependencyError(f"tracked symlink type differs from the locked tree: {path!r}")
else:
if candidate.is_symlink() or not candidate.is_file():
raise DependencyError(f"tracked file type differs from the locked tree: {path!r}")
executable = bool(candidate.stat().st_mode & 0o111)
if filemode_trusted and executable != (mode == "100755"):
raise DependencyError(f"tracked executable mode differs from the locked tree: {path!r}")
paths.append(path)
expected.append(blob)
try:
completed = subprocess.run(
["git", "hash-object", "--stdin-paths"],
cwd=source,
input="".join(f"{path}\n" for path in paths),
check=True,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
except subprocess.CalledProcessError as error:
raise DependencyError(f"unable to hash tracked source content: {error.stderr.strip()}") from error
actual = completed.stdout.splitlines()
if len(actual) != len(expected) or actual != expected:
raise DependencyError("tracked source content differs from the locked tree")
def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None: def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None:
if not (source / ".git").exists(): if not (source / ".git").exists():
raise DependencyError(f"not a materialized git checkout: {source}") raise DependencyError(f"not a materialized git checkout: {source}")
@@ -110,8 +174,15 @@ def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean:
raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit") raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit")
if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]: if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]:
raise DependencyError("upstream drift: checkout tree does not equal the locked tree") raise DependencyError("upstream drift: checkout tree does not equal the locked tree")
if require_clean and _git(source, "status", "--porcelain"): if _git(source, "rev-parse", "--abbrev-ref", "HEAD") != "HEAD":
raise DependencyError("local edits detected in materialized llama.cpp checkout") raise DependencyError("materialized llama.cpp checkout must have a detached HEAD")
if require_clean:
_verify_tracked_content(source, lock)
tracked = _git(source, "status", "--porcelain", "--untracked-files=no")
untracked = _git(source, "ls-files", "--others", "--exclude-standard")
ignored = _git(source, "ls-files", "--others", "--ignored", "--exclude-standard")
if tracked or untracked or ignored:
raise DependencyError("local edits or unmanifested files detected in materialized llama.cpp checkout")
for relative, expected in lock["required_upstream_blobs"].items(): for relative, expected in lock["required_upstream_blobs"].items():
actual = _git(source, "rev-parse", f"HEAD:{relative}") actual = _git(source, "rev-parse", f"HEAD:{relative}")
if actual != expected: if actual != expected:
@@ -122,15 +193,43 @@ def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean:
raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution") raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution")
def materialize(source: pathlib.Path, repository: str) -> None: def _workspace_source(workspace: pathlib.Path, lock: dict[str, Any]) -> pathlib.Path:
relative = pathlib.Path(lock["retrieval"]["workspace"])
expected = (ROOT / relative).absolute()
supplied = workspace.absolute()
if supplied != expected:
raise DependencyError(f"--workspace must equal the locked ignored build root: {expected}")
cursor = ROOT
for part in relative.parts:
cursor = cursor / part
if cursor.is_symlink():
raise DependencyError(f"locked build workspace may not traverse a symlink: {cursor}")
resolved = workspace.resolve()
try:
resolved.relative_to(ROOT.resolve())
except ValueError as error:
raise DependencyError("locked build workspace escapes the repository root") from error
source = resolved / "source"
if source.is_symlink():
raise DependencyError(f"locked source checkout may not be a symlink: {source}")
return source
def fetch(workspace: pathlib.Path) -> pathlib.Path:
"""Fetch once, or verify an exact clean cached checkout for offline reuse."""
lock = _load_lock() lock = _load_lock()
_patches(lock) _patches(lock)
source = _workspace_source(workspace, lock)
if source.exists(): if source.exists():
raise DependencyError(f"destination already exists; refusing to reuse possibly edited source: {source}") _verify_source(source, lock, require_clean=True)
print(f"reused verified offline cache: {source}")
return source
source.parent.mkdir(parents=True, exist_ok=True) source.parent.mkdir(parents=True, exist_ok=True)
_run("git", "clone", "--no-checkout", repository, str(source)) _run("git", "clone", "--no-checkout", lock["upstream"], str(source))
_git(source, "checkout", "--detach", lock["commit"]) _git(source, "checkout", "--detach", lock["commit"])
_verify_source(source, lock, require_clean=True) _verify_source(source, lock, require_clean=True)
print(f"fetched and verified exact source: {source}")
return source
def apply(source: pathlib.Path) -> None: def apply(source: pathlib.Path) -> None:
@@ -198,18 +297,11 @@ def smoke(binary: pathlib.Path) -> None:
print(output) print(output)
def reproduce(work_dir: pathlib.Path, repository: str) -> None: def reproduce(workspace: pathlib.Path) -> None:
resolved = work_dir.resolve() source = fetch(workspace)
build_root = (ROOT / "build").resolve() build_dir = workspace.resolve() / "build"
if build_root not in resolved.parents: if build_dir.exists():
raise DependencyError(f"--work-dir must be below {build_root}: {resolved}") raise DependencyError(f"build directory already exists; refusing to erase possible local edits: {build_dir}")
if resolved.exists():
raise DependencyError(
f"work directory already exists; refusing to erase possible local edits: {resolved}"
)
source = resolved / "source"
build_dir = resolved / "build"
materialize(source, repository)
apply(source) apply(source)
smoke(build(source, build_dir)) smoke(build(source, build_dir))
@@ -219,6 +311,9 @@ def inspect() -> None:
patches = _patches(lock) patches = _patches(lock)
print(json.dumps({ print(json.dumps({
"commit": lock["commit"], "commit": lock["commit"],
"commit_tree": lock["commit_tree"],
"retrieval": lock["retrieval"],
"upstream_license": lock["upstream_license"],
"patch_count": len(patches), "patch_count": len(patches),
"patches": [patch.name for patch in patches], "patches": [patch.name for patch in patches],
"model_downloads": False, "model_downloads": False,
@@ -231,9 +326,8 @@ def main() -> int:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
subcommands = parser.add_subparsers(dest="command", required=True) subcommands = parser.add_subparsers(dest="command", required=True)
subcommands.add_parser("inspect") subcommands.add_parser("inspect")
materialize_parser = subcommands.add_parser("materialize") fetch_parser = subcommands.add_parser("fetch")
materialize_parser.add_argument("--source-dir", type=pathlib.Path, required=True) fetch_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
materialize_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
apply_parser = subcommands.add_parser("apply") apply_parser = subcommands.add_parser("apply")
apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True) apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
build_parser = subcommands.add_parser("build") build_parser = subcommands.add_parser("build")
@@ -242,14 +336,13 @@ def main() -> int:
smoke_parser = subcommands.add_parser("smoke") smoke_parser = subcommands.add_parser("smoke")
smoke_parser.add_argument("--binary", type=pathlib.Path, required=True) smoke_parser.add_argument("--binary", type=pathlib.Path, required=True)
reproduce_parser = subcommands.add_parser("reproduce") reproduce_parser = subcommands.add_parser("reproduce")
reproduce_parser.add_argument("--work-dir", type=pathlib.Path, default=ROOT / "build/dgr-004-smoke") reproduce_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
reproduce_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
args = parser.parse_args() args = parser.parse_args()
try: try:
if args.command == "inspect": if args.command == "inspect":
inspect() inspect()
elif args.command == "materialize": elif args.command == "fetch":
materialize(args.source_dir, args.source_repository) fetch(args.workspace)
elif args.command == "apply": elif args.command == "apply":
apply(args.source_dir) apply(args.source_dir)
elif args.command == "build": elif args.command == "build":
@@ -257,9 +350,9 @@ def main() -> int:
elif args.command == "smoke": elif args.command == "smoke":
smoke(args.binary) smoke(args.binary)
else: else:
reproduce(args.work_dir, args.source_repository) reproduce(args.workspace)
except DependencyError as error: except DependencyError as error:
print(f"DGR-004 dependency error: {error}", file=sys.stderr) print(f"DGR-027 dependency error: {error}", file=sys.stderr)
return 2 return 2
return 0 return 0

View File

@@ -8,9 +8,9 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31", "fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240363330303165306566656164613562393766326633353632353632646330666432643962643839303464633662376239396137316439386633653933386264301a0c6578616d706c652d676775662201312a09323032362e30372e31",
"identity": { "identity": {
"artifact": { "artifact": {
"architecture": "dense-llama", "architecture": "dense-llama",
@@ -26,7 +26,7 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"recipe": { "recipe": {
"activation_dtype": "bfloat16", "activation_dtype": "bfloat16",
@@ -40,7 +40,7 @@
"protocol_schema_version": 1, "protocol_schema_version": 1,
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1", "runtime_version": "llama.cpp@dddddddddddddddddddddddddddddddddddddddd+patchstack.eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
"tokenizer_revision": "0123456789abcdef", "tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M" "weight_quantization": "Q4_K_M"
}, },
@@ -58,9 +58,9 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31", "fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240363330303165306566656164613562393766326633353632353632646330666432643962643839303464633662376239396137316439386633653933386264301a0c6578616d706c652d676775662201312a09323032362e30372e31",
"identity": { "identity": {
"artifact": { "artifact": {
"architecture": "dense-llama", "architecture": "dense-llama",
@@ -80,7 +80,7 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"recipe": { "recipe": {
"activation_dtype": "bfloat16", "activation_dtype": "bfloat16",
@@ -94,7 +94,7 @@
"protocol_schema_version": 1, "protocol_schema_version": 1,
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1", "runtime_version": "llama.cpp@dddddddddddddddddddddddddddddddddddddddd+patchstack.eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
"tokenizer_revision": "0123456789abcdef", "tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M" "weight_quantization": "Q4_K_M"
}, },

View File

@@ -1,8 +1,9 @@
"""Offline guards for DGR-004's pinned llama.cpp dependency boundary.""" """Offline guards for DGR-027's pinned llama.cpp dependency boundary."""
from __future__ import annotations from __future__ import annotations
import hashlib import hashlib
import importlib.util
import json import json
import pathlib import pathlib
import subprocess import subprocess
@@ -33,6 +34,10 @@ def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
assert commit == lock["commit"] assert commit == lock["commit"]
assert len(commit) == 40 assert len(commit) == 40
assert lock["retrieval"]["method"] == "git-clone-detached-commit"
assert lock["retrieval"]["workspace"] == "build/llama.cpp"
assert lock["expected_source"]["git_tree"] == lock["commit_tree"]
assert lock["upstream_license"] == "MIT"
assert patches == lock["patch_series"] assert patches == lock["patch_series"]
assert patches == sorted(patches) assert patches == sorted(patches)
assert patches assert patches
@@ -42,6 +47,188 @@ def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
assert "Subject: [PATCH" in patch.read_text() assert "Subject: [PATCH" in patch.read_text()
def test_fetch_refuses_a_workspace_outside_the_ignored_build_root(tmp_path: pathlib.Path) -> None:
completed = subprocess.run(
[sys.executable, str(SCRIPT), "fetch", "--workspace", str(tmp_path / "llama.cpp")],
cwd=ROOT,
capture_output=True,
text=True,
)
assert completed.returncode == 2
assert "--workspace must equal" in completed.stderr
def test_workspace_refuses_a_symlinked_build_ancestor(tmp_path: pathlib.Path, monkeypatch) -> None:
spec = importlib.util.spec_from_file_location("llama_cpp_dependency_symlink", SCRIPT)
assert spec and spec.loader
dependency = importlib.util.module_from_spec(spec)
spec.loader.exec_module(dependency)
root = tmp_path / "repo"
outside = tmp_path / "outside"
root.mkdir()
outside.mkdir()
(root / "build").symlink_to(outside, target_is_directory=True)
monkeypatch.setattr(dependency, "ROOT", root)
try:
dependency._workspace_source(
root / "build/llama.cpp",
{"retrieval": {"workspace": "build/llama.cpp"}},
)
except dependency.DependencyError as error:
assert "may not traverse a symlink" in str(error)
else:
raise AssertionError("symlinked build ancestor must be refused")
(root / "build").unlink()
workspace = root / "build/llama.cpp"
workspace.mkdir(parents=True)
(workspace / "source").symlink_to(outside, target_is_directory=True)
try:
dependency._workspace_source(
workspace,
{"retrieval": {"workspace": "build/llama.cpp"}},
)
except dependency.DependencyError as error:
assert "source checkout may not be a symlink" in str(error)
else:
raise AssertionError("symlinked source checkout must be refused")
def test_fetch_refuses_a_branch_or_repository_override() -> None:
completed = subprocess.run(
[sys.executable, str(SCRIPT), "fetch", "--source-repository", "main"],
cwd=ROOT,
capture_output=True,
text=True,
)
assert completed.returncode == 2
assert "unrecognized arguments" in completed.stderr
def test_fetch_reuses_only_a_verified_cached_tree_offline(
tmp_path: pathlib.Path, monkeypatch
) -> None:
spec = importlib.util.spec_from_file_location("llama_cpp_dependency", SCRIPT)
assert spec and spec.loader
dependency = importlib.util.module_from_spec(spec)
spec.loader.exec_module(dependency)
root = tmp_path / "repo"
source = root / "build/llama.cpp/source"
upstream = tmp_path / "upstream"
upstream.mkdir()
subprocess.run(["git", "init", "-q", str(upstream)], check=True)
subprocess.run(["git", "-C", str(upstream), "config", "user.email", "test@example.invalid"], check=True)
subprocess.run(["git", "-C", str(upstream), "config", "user.name", "test"], check=True)
(upstream / "CMakeLists.txt").write_text("cmake_minimum_required(VERSION 3.14)\n")
(upstream / "LICENSE").write_text("MIT\n")
(upstream / "tool.sh").write_text("#!/bin/sh\nexit 0\n")
(upstream / "tool.sh").chmod(0o755)
subprocess.run(["git", "-C", str(upstream), "add", "."], check=True)
subprocess.run(["git", "-C", str(upstream), "commit", "-qm", "fixture"], check=True)
commit = subprocess.run(
["git", "-C", str(upstream), "rev-parse", "HEAD"], check=True, capture_output=True, text=True
).stdout.strip()
tree = subprocess.run(
["git", "-C", str(upstream), "rev-parse", "HEAD^{tree}"], check=True, capture_output=True, text=True
).stdout.strip()
blob = subprocess.run(
["git", "-C", str(upstream), "rev-parse", "HEAD:CMakeLists.txt"],
check=True,
capture_output=True,
text=True,
).stdout.strip()
source.parent.mkdir(parents=True)
subprocess.run(["git", "clone", "-q", str(upstream), str(source)], check=True)
llama_dir = root / "packages/node/native/llama"
patch_dir = llama_dir / "patches"
patch_dir.mkdir(parents=True)
(llama_dir / "UPSTREAM_COMMIT").write_text(f"{commit}\n")
(patch_dir / "series").write_text("0001-fixture.patch\n")
patch = patch_dir / "0001-fixture.patch"
patch.write_text("fixture patch\n")
(patch_dir / "SHA256SUMS").write_text(f"{_sha256(patch)} {patch.name}\n")
(llama_dir / "UPSTREAM_LOCK.json").write_text(json.dumps({
"upstream": str(upstream),
"commit": commit,
"commit_tree": tree,
"expected_source": {"git_tree": tree},
"retrieval": {"method": "git-clone-detached-commit", "workspace": "build/llama.cpp"},
"patched_tree": tree,
"patch_series": [patch.name],
"required_upstream_blobs": {"CMakeLists.txt": blob},
"patched_paths": [],
"build": {},
"upstream_license": "MIT",
}))
monkeypatch.setattr(dependency, "ROOT", root)
monkeypatch.setattr(dependency, "LLAMA_DIR", llama_dir)
monkeypatch.setattr(dependency, "LOCK_PATH", llama_dir / "UPSTREAM_LOCK.json")
monkeypatch.setattr(dependency, "PATCH_DIR", patch_dir)
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "detached HEAD" in str(error)
else:
raise AssertionError("attached branch cache must be refused")
subprocess.run(["git", "-C", str(source), "checkout", "--detach", "-q", commit], check=True)
assert dependency.fetch(root / "build/llama.cpp") == source
tracked_path = source / "CMakeLists.txt"
for flag, clear_flag in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
subprocess.run(["git", "-C", str(source), "update-index", flag, "CMakeLists.txt"], check=True)
tracked_path.write_text("injected tracked build input\n")
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "tracked source content differs" in str(error)
else:
raise AssertionError(f"tracked cache injection hidden by {flag} must be refused")
subprocess.run(["git", "-C", str(source), "update-index", clear_flag, "CMakeLists.txt"], check=True)
subprocess.run(["git", "-C", str(source), "checkout", "--", "CMakeLists.txt"], check=True)
executable_path = source / "tool.sh"
for flag, clear_flag in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
subprocess.run(["git", "-C", str(source), "update-index", flag, "tool.sh"], check=True)
executable_path.chmod(0o644)
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "executable mode differs" in str(error)
else:
raise AssertionError(f"tracked mode change hidden by {flag} must be refused")
subprocess.run(["git", "-C", str(source), "update-index", clear_flag, "tool.sh"], check=True)
subprocess.run(["git", "-C", str(source), "checkout", "--", "tool.sh"], check=True)
(source / "untracked.txt").write_text("edited\n")
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "local edits" in str(error)
else:
raise AssertionError("dirty cached source must be refused")
(source / "untracked.txt").unlink()
(source / ".git/info/exclude").write_text("injected.cmake\n")
(source / "injected.cmake").write_text("unmanifested input\n")
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "unmanifested files" in str(error)
else:
raise AssertionError("ignored cached source input must be refused")
def test_dependency_script_reports_the_locked_boundary_without_network() -> None: def test_dependency_script_reports_the_locked_boundary_without_network() -> None:
completed = subprocess.run( completed = subprocess.run(
[sys.executable, str(SCRIPT), "inspect"], [sys.executable, str(SCRIPT), "inspect"],

View File

@@ -15,6 +15,7 @@ from meshnet_node.native_backend import (
shard_identity_from_native_report, shard_identity_from_native_report,
) )
from meshnet_node.native_protocol import SCHEMA_VERSION, pb from meshnet_node.native_protocol import SCHEMA_VERSION, pb
from meshnet_node.runtime_pin import load_runtime_pin
from meshnet_node.recipe_manifest import parse_recipe_manifest from meshnet_node.recipe_manifest import parse_recipe_manifest
from meshnet_tracker.capability import STATE_UNCERTIFIED, evaluate_report from meshnet_tracker.capability import STATE_UNCERTIFIED, evaluate_report
@@ -42,7 +43,6 @@ def _inputs(**changes: object) -> NativeIdentityInputs:
kv_layout="llama-kv-v1", kv_layout="llama-kv-v1",
architecture_adapter="dense-llama-v1", architecture_adapter="dense-llama-v1",
backend_id="llama-cpp", backend_id="llama-cpp",
runtime_version="llama.cpp:e920c523",
recipe_id="native", recipe_id="native",
recipe_version="1", recipe_version="1",
catalogue_version="2026.07.1", catalogue_version="2026.07.1",
@@ -84,6 +84,7 @@ def test_native_identity_uses_loaded_report_not_a_caller_range():
assert (identity.shard_start, identity.shard_end) == (2, 6) assert (identity.shard_start, identity.shard_end) == (2, 6)
assert identity.artifact.architecture == "llama" assert identity.artifact.architecture == "llama"
assert identity.artifact.layer_count == 8 assert identity.artifact.layer_count == 8
assert identity.recipe.runtime_version == load_runtime_pin().runtime_version
def test_native_identity_requires_an_immutable_pin_and_gguf_range(): def test_native_identity_requires_an_immutable_pin_and_gguf_range():

View File

@@ -0,0 +1,312 @@
"""DGR-025: the recipe's runtime axis commits to the exact pin and patch stack.
The fingerprint (DGR-003 lineage) digests a ``runtime_version`` string, but a
string an operator typed is a label, not a pin: two workers could run different
patch stacks under the same label and still agree on the digest. These tests
pin the axis to the DGR-027 lock manifest — the exact upstream commit plus a
digest over the ordered patch-stack bytes — and prove both identity
implementations reject a moving runtime reference.
"""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import pytest
from meshnet_node.runtime_pin import (
DEFAULT_LOCK_DIR,
RuntimePinError,
load_runtime_pin,
)
from meshnet_node.runtime_recipe import RecipeIdentityError, RuntimeRecipe
from meshnet_tracker.recipe import (
RecipeIdentityError as TrackerRecipeIdentityError,
parse_identity,
)
REPO_LOCK_DIR = (
Path(__file__).resolve().parent.parent / "packages" / "node" / "native" / "llama"
)
def _recipe(**changes: object) -> RuntimeRecipe:
fields: dict[str, object] = {
"weight_quantization": "Q4_K_M",
"activation_dtype": "bfloat16",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"tokenizer_revision": "0123456789abcdef",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"runtime_version": "llama.cpp@" + "d" * 40 + "+patchstack." + "e" * 64,
"recipe_id": "example-gguf",
"recipe_version": "1",
"catalogue_version": "2026.07.1",
}
fields.update(changes)
return RuntimeRecipe(**fields) # type: ignore[arg-type]
def _write_fixture_workspace(
root: Path,
*,
commit: str = "e" * 40,
patches: dict[str, bytes] | None = None,
lock_series: list[str] | None = None,
series_lines: list[str] | None = None,
sums_lines: list[str] | None = None,
schema_version: int = 1,
upstream: str = "https://github.com/ggml-org/llama.cpp.git",
upstream_commit_file: str | None = None,
) -> Path:
"""A minimal DGR-027-shaped lock workspace; overrides create disagreement."""
if patches is None:
patches = {
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ d\n",
}
names = list(patches)
digests = {
name: hashlib.sha256(body).hexdigest() for name, body in patches.items()
}
patch_dir = root / "patches"
patch_dir.mkdir(parents=True)
for name, body in patches.items():
(patch_dir / name).write_bytes(body)
(root / "UPSTREAM_LOCK.json").write_text(
json.dumps(
{
"schema_version": schema_version,
"upstream": upstream,
"commit": commit,
"patch_series": names if lock_series is None else lock_series,
}
),
encoding="utf-8",
)
(root / "UPSTREAM_COMMIT").write_text(
(commit if upstream_commit_file is None else upstream_commit_file) + "\n",
encoding="utf-8",
)
(patch_dir / "series").write_text(
"\n".join(names if series_lines is None else series_lines) + "\n",
encoding="utf-8",
)
if sums_lines is None:
sums_lines = ["# ordered digests"] + [
f"{digests[name]} {name}" for name in names
]
(patch_dir / "SHA256SUMS").write_text(
"\n".join(sums_lines) + "\n", encoding="utf-8"
)
return root
# --- the committed manifest is the identity source -------------------------
def test_default_lock_dir_is_the_committed_manifest():
assert DEFAULT_LOCK_DIR == REPO_LOCK_DIR
def test_committed_manifest_derives_a_deterministic_runtime_pin():
pin = load_runtime_pin(REPO_LOCK_DIR)
again = load_runtime_pin(REPO_LOCK_DIR)
assert pin == again
lock = json.loads(
(REPO_LOCK_DIR / "UPSTREAM_LOCK.json").read_text(encoding="utf-8")
)
assert pin.upstream_commit == lock["commit"]
assert list(pin.patch_series) == lock["patch_series"]
# The axis value names the runtime, the exact commit, and the stack digest,
# so changing any of the three changes every downstream recipe digest.
assert pin.runtime_version == (
f"llama.cpp@{lock['commit']}+patchstack.{pin.patch_stack_digest}"
)
assert len(pin.patch_stack_digest) == 64
def test_derived_axis_value_is_a_valid_recipe_pin():
pin = load_runtime_pin(REPO_LOCK_DIR)
recipe = _recipe(runtime_version=pin.runtime_version)
assert recipe.runtime_version == pin.runtime_version
assert len(recipe.runtime_recipe_digest) == 64
def test_patch_byte_change_changes_the_runtime_identity(tmp_path):
baseline = load_runtime_pin(_write_fixture_workspace(tmp_path / "a"))
changed = load_runtime_pin(
_write_fixture_workspace(
tmp_path / "b",
patches={
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ DIFFERENT\n",
},
)
)
assert baseline.patch_stack_digest != changed.patch_stack_digest
assert baseline.runtime_version != changed.runtime_version
def test_patch_order_is_part_of_the_stack_identity(tmp_path):
patches = {
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ d\n",
}
forward = load_runtime_pin(
_write_fixture_workspace(tmp_path / "a", patches=patches)
)
names = list(patches)
reversed_names = list(reversed(names))
digests = {
name: hashlib.sha256(body).hexdigest() for name, body in patches.items()
}
swapped = load_runtime_pin(
_write_fixture_workspace(
tmp_path / "b",
patches=patches,
lock_series=reversed_names,
series_lines=reversed_names,
sums_lines=[f"{digests[name]} {name}" for name in reversed_names],
)
)
assert forward.patch_stack_digest != swapped.patch_stack_digest
# --- every manifest disagreement fails closed ------------------------------
def test_missing_lock_file_fails_closed(tmp_path):
with pytest.raises(RuntimePinError, match="UPSTREAM_LOCK.json"):
load_runtime_pin(tmp_path)
def test_malformed_lock_json_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path)
(tmp_path / "UPSTREAM_LOCK.json").write_text("{not json", encoding="utf-8")
with pytest.raises(RuntimePinError, match="not valid JSON"):
load_runtime_pin(tmp_path)
def test_unknown_lock_schema_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path, schema_version=2)
with pytest.raises(RuntimePinError, match="schema"):
load_runtime_pin(tmp_path)
def test_moving_commit_reference_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path, commit="master")
with pytest.raises(RuntimePinError, match="exact 40"):
load_runtime_pin(tmp_path)
def test_upstream_commit_file_disagreement_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path, upstream_commit_file="f" * 40)
with pytest.raises(RuntimePinError, match="UPSTREAM_COMMIT"):
load_runtime_pin(tmp_path)
def test_series_file_disagreement_fails_closed(tmp_path):
_write_fixture_workspace(
tmp_path, series_lines=["0002-second.patch", "0001-first.patch"]
)
with pytest.raises(RuntimePinError, match="series"):
load_runtime_pin(tmp_path)
def test_missing_patch_file_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path)
(tmp_path / "patches" / "0002-second.patch").unlink()
with pytest.raises(RuntimePinError, match="0002-second.patch"):
load_runtime_pin(tmp_path)
def test_checksum_disagreement_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path)
patch = tmp_path / "patches" / "0002-second.patch"
patch.write_bytes(patch.read_bytes() + b"tampered\n")
with pytest.raises(RuntimePinError, match="SHA256SUMS"):
load_runtime_pin(tmp_path)
def test_sums_entry_missing_fails_closed(tmp_path):
patches = {
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ d\n",
}
digest = hashlib.sha256(patches["0001-first.patch"]).hexdigest()
_write_fixture_workspace(
tmp_path,
patches=patches,
sums_lines=[f"{digest} 0001-first.patch"],
)
with pytest.raises(RuntimePinError, match="SHA256SUMS"):
load_runtime_pin(tmp_path)
def test_empty_patch_series_requires_empty_series_files(tmp_path):
# An unpatched runtime is a legal pin; a lock that *hides* patches is not.
_write_fixture_workspace(tmp_path, lock_series=[])
with pytest.raises(RuntimePinError, match="series"):
load_runtime_pin(tmp_path)
# --- both identity implementations reject a moving runtime -----------------
def test_node_recipe_rejects_a_moving_runtime_version():
with pytest.raises(RecipeIdentityError, match="moving reference"):
_recipe(runtime_version="latest")
def test_tracker_rejects_a_moving_runtime_version():
vectors = json.loads(
(Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json").read_text(
encoding="utf-8"
)
)
doc = json.loads(json.dumps(vectors["vectors"][0]["identity"]))
doc["recipe"]["runtime_version"] = "latest"
doc.pop("fingerprint", None)
with pytest.raises(TrackerRecipeIdentityError, match="moving reference"):
parse_identity(doc)
@pytest.mark.parametrize(
"forged",
[
"llama.cpp@master+patchstack.not-a-digest",
"llama.cpp@e920c523+patchstack.forged",
"release-that-operator-typed",
],
)
def test_node_recipe_rejects_noncanonical_llama_runtime_pins(forged):
with pytest.raises(RecipeIdentityError, match="40-hex commit"):
_recipe(runtime_version=forged)
@pytest.mark.parametrize(
"forged",
[
"llama.cpp@master+patchstack.not-a-digest",
"llama.cpp@e920c523+patchstack.forged",
"release-that-operator-typed",
],
)
def test_tracker_rejects_noncanonical_llama_runtime_pins(forged):
vectors = json.loads(
(Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json").read_text(
encoding="utf-8"
)
)
doc = json.loads(json.dumps(vectors["vectors"][0]["identity"]))
doc["recipe"]["runtime_version"] = forged
doc.pop("fingerprint", None)
with pytest.raises(TrackerRecipeIdentityError, match="40-hex commit"):
parse_identity(doc)

View File

@@ -65,7 +65,7 @@ def _recipe(**changes: object) -> RuntimeRecipe:
"tokenizer_revision": "0123456789abcdef", "tokenizer_revision": "0123456789abcdef",
"architecture_adapter": "llama/range-v1", "architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp", "backend_id": "llama.cpp",
"runtime_version": "llama.cpp@deadbeef+meshnet.1", "runtime_version": "llama.cpp@" + "d" * 40 + "+patchstack." + "e" * 64,
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"catalogue_version": "2026.07.1", "catalogue_version": "2026.07.1",
@@ -241,7 +241,7 @@ def test_committed_vectors_cover_a_whole_model_and_a_derivative_shard():
("tokenizer_revision", "fedcba9876543210"), ("tokenizer_revision", "fedcba9876543210"),
("architecture_adapter", "llama/range-v2"), ("architecture_adapter", "llama/range-v2"),
("backend_id", "other-backend"), ("backend_id", "other-backend"),
("runtime_version", "llama.cpp@other+meshnet.1"), ("runtime_version", "llama.cpp@" + "c" * 40 + "+patchstack." + "b" * 64),
("boundary_schema_version", 2), ("boundary_schema_version", 2),
("protocol_schema_version", 2), ("protocol_schema_version", 2),
], ],