9 Commits

Author SHA1 Message Date
Dobromir Popov
54d19f9a29 chore: replace fake protocol story with real harness 2026-07-19 00:22:03 +03:00
Dobromir Popov
377bc3475c chore: reconcile DGR-023 completion projection 2026-07-18 15:57:10 +03:00
Dobromir Popov
902ecde363 [verified] feat: pin native protobuf and gRPC generation 2026-07-17 23:43:03 +03:00
Dobromir Popov
db59caa8e9 [verified] fix: enforce canonical native runtime pin 2026-07-17 23:20:03 +03:00
Dobromir Popov
ad66f7a4d8 feat: pin runtime identity to the exact llama.cpp patch stack (DGR-025)
Derive the recipe's runtime_version axis from the DGR-027 lock manifest
(exact upstream commit + ordered patch-stack byte digest) in new
meshnet_node.runtime_pin, failing closed on any lock/series/SHA256SUMS/
patch disagreement, and enforce pin discipline on runtime_version in both
the node and tracker identity implementations.

Also repair pre-existing backlog consistency: add missing DGR-022/DGR-027
completionNotes, regenerate the DGR-022/025/027 issue projections, and
relocate three pre-DGR legacy GLM issue files to issues/legacy/. Mark
DGR-025 passes=true with evidence at evidence/DGR-025/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 22:59:22 +03:00
Dobromir Popov
f83cf331c3 [verified] feat: harden llama.cpp provenance workspace 2026-07-17 16:24:46 +03:00
Dobromir Popov
ae51526e85 Merge remote-tracking branch 'origin/ralph/distributed-gguf-runtime' into ralph/distributed-gguf-runtime 2026-07-17 15:32:17 +03:00
Dobromir Popov
8563d218c9 Merge branch 'ralph/distributed-gguf-runtime' of https://git.d-popov.com/popov/neuron-tai into ralph/distributed-gguf-runtime 2026-07-17 13:33:08 +02:00
Dobromir Popov
66d9888a11 memory 2026-07-17 13:19:14 +02:00
31 changed files with 1582 additions and 155 deletions

View File

@@ -44,6 +44,10 @@ Historical handoff note: `/mnt/c/Users/popov/Downloads/neuron-tai-alpha-handoff-
Planning is ready at `.scratch/node-capability-admission/` with five sequential Ralph stories and ADR-0023. The design is model-agnostic: a Node must validate its selected Model Artifact/shard with a bounded real forward before Tracker routing; Qwen3.6 is only an optional development fixture. P0 adds a versioned local recipe-manifest/report contract, `meshnet-node doctor`, fail-closed startup admission, and tracker route gating. It intentionally excludes dynamic recipe/dependency installation and the future signed Node updater. Planning is ready at `.scratch/node-capability-admission/` with five sequential Ralph stories and ADR-0023. The design is model-agnostic: a Node must validate its selected Model Artifact/shard with a bounded real forward before Tracker routing; Qwen3.6 is only an optional development fixture. P0 adds a versioned local recipe-manifest/report contract, `meshnet-node doctor`, fail-closed startup admission, and tracker route gating. It intentionally excludes dynamic recipe/dependency installation and the future signed Node updater.
## Gitea DGR sync (2026-07-17)
Gitea is ahead of the local Markdown backlog with open DGR-022..DGR-071. The first executable P0 dependency frontier is DGR-022 (Shard lifecycle and structured status RPCs), DGR-023 (reproducible protobuf generation), DGR-025 (artifact/runtime recipe identity), and DGR-027 (llama.cpp provenance manifest). DGR-021, the named-tensor stream envelope prerequisite for DGR-022/023/025, is closed. DGR-022 is the next dependency-ordered issue and blocks DGR-024, DGR-033, and DGR-037.
## Windows CUDA node (working as of 2026-07-01) ## Windows CUDA node (working as of 2026-07-01)
- miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118 - miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118
- RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200 - RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200

View File

@@ -0,0 +1,126 @@
# DGR-023 evidence — reproducible Python and C++ protobuf/gRPC generation
**Status:** complete after controller verification and independent-review repairs on 2026-07-17.
**Authority:** live Gitea issue #7. The local PRD is a secondary projection.
## Implemented contract
- Python generation requires exactly `grpcio-tools==1.82.1`; the generator checks installed distribution metadata and rejects missing or different versions with an actionable exact install command.
- The C++ bootstrap builds one ignored toolchain prefix from exact inputs:
- Protobuf release `33.1` (`protobuf-config` version `33.1.0`);
- Abseil release `20250814.1`;
- gRPC C++ `1.82.1` at commit `acccf84c0df20487d64101f528e5d426541ca4e5`;
- gRPC's exact-commit submodules for c-ares, RE2, OpenSSL, and zlib.
- Protobuf is configured with local dependencies only after the exact Abseil build. gRPC uses the installed Protobuf/Abseil packages and commit-pinned module dependencies, avoiding unpinned system development packages and download fallbacks.
- CMake requires exact Protobuf `33.1.0` and gRPC `1.82.1`, requires the exported `gRPC::grpc_cpp_plugin` target, and always generates/builds both message and service stubs in the ignored build tree.
- Python bindings remain committed package output; `--check` regenerates into a temporary directory and compares output. C++ bindings are never committed.
- The C++ conformance test parses Python-produced vectors, validates fields/CRC32C, and emits `cpp_roundtrip.binpb`; Python compares that artifact byte-for-byte.
## Defects found and fixed
1. A relative bootstrap prefix was resolved after entering the temporary source directory, so successful output was deleted by cleanup. The script now canonicalizes the caller-relative destination first. The regression executes `--print-prefix` from a temporary working directory and validates the resulting path behavior.
2. The original native path omitted gRPC C++ and accepted any discoverable plugin. The bootstrap now builds exact gRPC/plugin sources, and CMake rejects absent/incompatible versions.
3. The Python script named the `grpcio-tools` pin but did not validate the installed distribution. It now refuses mismatched versions.
4. Protobuf ignored a stale provider option and attempted to download a different Abseil. The build was stopped; exact Abseil is now built first and Protobuf uses `LOCAL_DEPENDENCIES_ONLY`.
5. The host lacked OpenSSL development headers. Rather than add a floating system dependency, gRPC now uses the submodule pinned by its exact commit.
6. Documentation uses `bash scripts/bootstrap_native_toolchain.sh ...`, so a normal checkout does not depend on executable-mode preservation.
## Verified toolchain
```text
cmake version 4.4.0
c++ (GCC) 15.2.1 20260123 (Red Hat 15.2.1-7)
libprotoc 33.1
protobuf CMake package 33.1.0
grpcio-tools 1.82.1
grpcio 1.82.1
protobuf Python runtime 7.35.1
gRPC C++ 1.82.1
commit acccf84c0df20487d64101f528e5d426541ca4e5
grpc_cpp_plugin sha256 995ca8ac620fe83532b649a7c8c0a9341c7003da927fe0e4a8f821bfc579206d
```
The native toolchain and generated/build artifacts live under ignored mounted-drive `build/` paths; model/build artifacts were not stored under `/home`.
## Commands and results
```bash
bash scripts/bootstrap_native_toolchain.sh build/native-toolchain
```
```text
passed from a clean build directory
libprotoc 33.1
gRPC 1.82.1 commit acccf84c0df20487d64101f528e5d426541ca4e5
grpc_cpp_plugin sha256 995ca8ac620fe83532b649a7c8c0a9341c7003da927fe0e4a8f821bfc579206d
```
```bash
cmake -S packages/node/native -B build/native \
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain"
cmake --build build/native -j"$(nproc)"
test -f build/native/shard_runtime.grpc.pb.cc
test -f build/native/shard_runtime.grpc.pb.h
test -f build/native/libshard_runtime_grpc.a
ctest --test-dir build/native --output-on-failure
```
```text
Pinned gRPC 1.82.1: building ShardRuntime service stubs
shard_runtime_proto built
shard_runtime_grpc built
1/1 shard_protocol_conformance passed
```
```bash
python3 -m pytest -q tests/test_native_shard_protocol.py
```
```text
50 passed, 2 optional-path skips
```
All DGR-023-required checks were selected explicitly:
```bash
python3 -m pytest -q -rs tests/test_native_shard_protocol.py \
-k 'cpp_and_python_agree_byte_for_byte or generated_python_stubs_match_the_proto or native_toolchain_bootstrap or wrong_grpcio'
```
```text
4 passed, 48 deselected
```
```bash
python3 scripts/generate_native_protocol.py --check
python3 scripts/generate_protocol_goldens.py --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
python3 -m compileall -q packages tests
git diff --check
```
```text
generated stubs are up to date
conformance vectors are up to date
OK: 55 stories validated
compileall passed
git diff --check passed
```
## Changed files
- `scripts/bootstrap_native_toolchain.sh`
- `scripts/generate_native_protocol.py`
- `packages/node/native/CMakeLists.txt`
- `packages/node/native/README.md`
- `tests/test_native_shard_protocol.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-023/README.md`
- `.scratch/distributed-gguf-runtime/prd.json` (secondary completion projection only)
## Limitations and dependency handoff
- This story proves exact schema/message/service generation and cross-language conformance. It does not implement or run the standalone worker service itself; DGR-033/DGR-037 own worker behavior.
- The plugin SHA is evidence for this verified build. Reproducibility authority is the exact gRPC commit plus its submodule graph, not an assumption that different compilers produce byte-identical executables.
- No model, GPU, API credits, or model download was used.
- DGR-024 and DGR-037 may consume this completed generation dependency but must provide their own transport/worker evidence.

View File

@@ -0,0 +1,210 @@
# DGR-025 evidence — exact artifact and runtime recipe identity
**Completed:** 2026-07-17
**Branch:** `ralph/fable-architecture-loop` (Claude Fable architecture lane)
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-018 (`evidence/DGR-018/README.md` — canonical backlog schema and
issue projection), DGR-021 (`evidence/DGR-021/README.md` — versioned activation
envelope). Both read before changing code.
## Objective
Ensure the tracker and worker only combine numerically and operationally
compatible shards: fingerprint every axis that moves the numbers, bind shards to
exact half-open ranges, fail closed on any mismatch, and keep uncertified
recipes registered-but-dark.
## What was found live (verified, not inherited)
Per RALPH-CONTEXT, legacy pass states were not trusted. The DGR-003-lineage
identity core was inspected and exercised live before any change:
- `packages/node/meshnet_node/runtime_recipe.py` — node-side identity:
domain-separated digests (`meshnet.model-artifact.v1`,
`meshnet.runtime-recipe.v1`, `meshnet.shard-binding.v1`) over the source
artifact SHA (`source_digest`, with split artifacts bound to their exact
source via `DerivativeBinding`), tokenizer revision (pin-enforced),
architecture adapter + architecture/config digest, boundary and protocol
schema versions, backend, weight quantization, activation/compute dtypes, and
KV dtype/layout (`RECIPE_AXES`). Shard ranges are half-open
(`shard_start`/`shard_end`, end-exclusive, protocol convention) with no
topology or quant constants anywhere; `check_route` accepts any tiling of
`[0, layer_count)`. Route, handshake (`check_handshake`), and session-open
(`check_session_open`) checks fail closed with structured `RouteMismatch`
reasons mapped to specific protocol error codes (`handshake_error`).
- `packages/tracker/meshnet_tracker/recipe.py` — deliberately independent
tracker re-derivation (no `meshnet_node` import); declared fingerprints are
recomputed, never trusted (`parse_identity`, `FingerprintMismatch`). The
`CertificationLedger` keeps every registered recipe dark until a real
distributed forward — at least 2 distinct nodes, whole-model coverage,
non-synthetic, tokens actually generated — certifies it; dark recipes may
route only to certify.
- The two implementations are pinned by committed conformance vectors
(`tests/data/recipe_fingerprint_vectors.json`).
Live verification of that pre-existing core before changes:
`PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_recipe_identity.py`
`45 passed`; plus `tests/test_native_identity_emission.py`,
`tests/test_tracker_capability_admission.py`, `tests/test_node_admission.py`
`59 passed`.
## Gap found and closed (this story's change)
**The `runtime_version` recipe axis was a label, not a pin.** It was an opaque
caller-supplied string: nothing derived it from the DGR-027 lock manifest, and
neither identity implementation rejected a moving reference (`"latest"` was
accepted), so two workers could run different llama.cpp pins or patch stacks
under one label and still agree on the recipe digest. The acceptance criterion
explicitly requires fingerprinting the "runtime pin/patch stack".
### Changed files
- `packages/node/meshnet_node/runtime_pin.py` (new) — derives the canonical
`runtime_version` axis value from the DGR-027 lock workspace
(`packages/node/native/llama`):
`<runtime>@<40-hex upstream commit>+patchstack.<sha256>` where the stack
digest commits, under the `meshnet.runtime-patch-stack.v1` domain, to the
*ordered* `(patch name, patch bytes sha256)` stack. Fails closed on: missing
or malformed `UPSTREAM_LOCK.json`, unknown schema version, non-40-hex/moving
commit, `UPSTREAM_COMMIT` disagreement, any disagreement among the lock's
`patch_series`, `patches/series`, and `patches/SHA256SUMS`, a missing patch
file, or a patch whose bytes don't match their recorded digest. Reads the
committed manifest only; fetching/patching stays with
`scripts/llama_cpp_dependency.py` (DGR-027).
- `packages/node/meshnet_node/runtime_recipe.py``runtime_version` is now
pin-enforced (`_require_pin`) exactly like `tokenizer_revision`; for the
llama.cpp backend it must also match the canonical
`llama.cpp@<40-hex>+patchstack.<64-hex>` grammar.
- `packages/node/meshnet_node/native_backend.py` — the production native
identity seam no longer accepts a caller-supplied runtime string. It derives
`runtime_version` directly through `load_runtime_pin()` from the committed
lock and rejects a non-llama backend at this llama.cpp-specific boundary.
- `packages/tracker/meshnet_tracker/recipe.py` — the independent tracker
implementation applies the same backend-specific grammar before re-deriving
the recipe digest, so forged operator labels cannot register or certify.
- `tests/test_runtime_pin_identity.py` and
`tests/test_native_identity_emission.py` — deterministic tests cover lock
derivation, production native emission, and node/tracker rejection of the
forged values from independent review. Conformance vectors were regenerated
through `scripts/gen_recipe_fingerprint_vectors.py` for the tightened wire
contract.
### Backlog-consistency repair (pre-existing damage, honestly recorded)
`tests/test_ralph_prd_schema.py` had 4 pre-existing failures before this story
touched anything, left by prior sessions and the alternate-history merge:
- DGR-022 and DGR-027 were marked `passes: true` without `completionNotes` and
without regenerated issue projections. Added their `completionNotes`
(explicitly labeled as added during this repair, content drawn from their own
evidence READMEs) and regenerated
`issues/022-…` / `issues/027-…` via `scripts/ralph_prd_schema.py render`.
- Three pre-DGR legacy GLM alpha issue files (`18-…`, `19-…`, `20-…`,
committed 2026-07-14, before DGR-018 established the generated-only
convention; they carry no authority disclaimer because they are *not*
generated from prd.json) were relocated via `git mv` to
`issues/legacy/` — preserved as provenance, out of the generated namespace.
### prd.json
Marked `DGR-025.passes = true` with `completionNotes`; regenerated
`issues/025-define-exact-artifact-and-runtime-recipe-identity.md`.
## Acceptance criteria → evidence
1. **Fingerprint all axes**`RECIPE_AXES` + `ArtifactIdentity` cover source
artifact SHA, tokenizer revision, architecture adapter/version (adapter axis
+ architecture/config digest), boundary schema (boundary + protocol schema
versions), backend, quant, activation/compute dtype, KV/state layout; the
runtime pin/patch stack is now committed via the derived `runtime_version`
axis (`runtime_pin.py`). Verified by `test_runtime_recipe_identity.py` and
`test_runtime_pin_identity.py`.
2. **Exact half-open range, no hardcoded topology/quant**`ShardIdentity`
end-exclusive ranges, `DerivativeBinding` coverage checks, `check_route`
tiling over arbitrary layouts; quant/dtype values are open strings
(dynamic recipe inputs). Verified by `test_runtime_recipe_identity.py`
(routes of 1, 2, and 5 shards; no product constants).
3. **Fail closed on any mismatch** — artifact, adapter, boundary/schema, cache
layout, backend, and runtime mismatches each produce structured
`RouteMismatch` reasons and protocol error codes; the tracker recomputes
digests and rejects inconsistent claims; moving runtime references are now
rejected on both sides.
4. **Registered-but-dark**`CertificationLedger`: unknown recipes cannot be
certified, registered recipes are dark, only a real ≥2-distinct-node
whole-model non-synthetic forward promotes; verified by
`test_runtime_recipe_identity.py` / `test_tracker_capability_admission.py`.
5. **Gates + this handoff** — below.
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_pin_identity.py
```
```text
23 passed in 0.15s
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py \
tests/test_native_identity_emission.py tests/test_tracker_capability_admission.py \
tests/test_node_admission.py tests/test_node_capability.py tests/test_recipe_benchmark.py
```
```text
202 passed, 1 warning in 5.38s
```
```bash
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_ralph_prd_schema.py
```
```text
108 passed
```
(4 failed before this story's backlog repair; 0 after.)
```bash
python3 -m compileall -q packages tests # exit 0
git diff --check # exit 0
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
# OK: 55 stories validated.
```
Default tests are model-download-free, API-credit-free, and GPU-free; no model
artifact was touched and nothing was written under `/home`.
## Limitations
- The production native identity seam now derives the manifest pin and cannot
accept an operator-supplied runtime label. It still cannot attest that the
running binary was built from those locked bytes. Embedding the patched-tree
hash at build time and echoing it through the DGR-022 status contract belongs
with DGR-028+/DGR-031; real distributed certification remains the final trust
boundary.
- The DGR-027-recorded blocker stands: `0002-dense-llama-owned-range-loader.patch`
does not apply cleanly against the pin (DGR-028). That does not affect this
story: the identity commits to the patch *bytes as committed*, which is
precisely what makes a later repaired patch a *different* runtime identity.
- No native/CMake change was made, so the native build/CTest gate is not
applicable; no llama.cpp patch content was changed, so apply/check/reverse
verification is not applicable (and is blocked by the DGR-028 defect anyway).
- Tracker routing, load balancing, billing, telemetry, and relay semantics are
untouched; the only behavior change outside the new module is the stricter
(fail-closed) rejection of moving `runtime_version` values.
## Dependency handoff
- **DGR-026** (split-GGUF provisioning): bind each provisioned split via
`DerivativeBinding` to the exact source digest recorded in its hashed
manifest; the per-split `shard_binding_digest` is what certification pins.
- **DGR-031** (`ShardEngine`): construct worker identity through
`shard_identity_from_native_report` and populate `runtime_version` from
`meshnet_node.runtime_pin.load_runtime_pin().runtime_version` — never from an
operator string. A build-time echo of the patched-tree hash through the
status contract would close the manifest-vs-binary gap noted above.
- **DGR-041** (capability registration): the tracker already re-derives and
fail-closes on presented identities (`parse_identity`); register recipes
through the `CertificationLedger` so they arrive dark.
- **DGR-044** (DeepSeek V4 Flash target): pin the target's artifact identity
the same way `glm_alpha_artifact` does — read locked manifests, never restate
digests — and note `layer_count` must count the routed transformer stack the
route tiles, excluding MTP (reserved for beta).

View File

@@ -0,0 +1,72 @@
# DGR-027 evidence — exact llama.cpp provenance manifest and fetch workspace
**Completed implementation:** 2026-07-17
**Branch:** `ralph/dgr-small-terra`
**Authority:** live Gitea issue #11. The controller fetched and claimed the issue
through the Gitea API before launch; the isolated agent received that exact body.
## Changed files
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/PATCH-STACK.md`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`
## Provenance and retrieval contract
`UPSTREAM_LOCK.json` records the upstream Git URL, immutable 40-character
commit `e920c523e3b8a0163fe498af5bf90df35ff51d25`, expected Git tree
`6c91a11407a3a3fb160f5dac705f9c59718f54f1`, MIT license, and the sole
retrieval method: `git-clone-detached-commit` into `build/llama.cpp/source`.
`python3 scripts/llama_cpp_dependency.py fetch` has no branch, tag, ref, or
repository override. On a first fetch it clones the manifest URL, checks out
the detached commit, and verifies commit, tree, required upstream blobs,
license, and cleanliness. If the workspace already exists, it makes no network
request and accepts it only after the same verification. Dirty or mismatched
caches fail closed. The build directory is already ignored by `.gitignore`.
## Verification
| Command | Result |
| --- | --- |
| `python3 -m pytest -q tests/test_llama_cpp_dependency.py` | `7 passed in 0.22s` |
| `python3 -m compileall packages tests` | passed |
| `git diff --check` | passed (no output) |
| `python3 scripts/llama_cpp_dependency.py inspect` | passed; reports exact commit/tree, retrieval workspace, MIT license, and two-patch stack |
| `python3 scripts/llama_cpp_dependency.py fetch --workspace /tmp/not-llama-workspace` | failed closed with status 2: workspace outside the locked ignored build root |
| symlinked workspace regression | passed; both a `build/` ancestor symlink and a final `source` symlink escaping the repository are refused |
| attached-branch cache regression | passed; an exact commit on a local branch is refused until checked out as detached HEAD |
| ignored/excluded injection regression | passed; a file hidden by `.git/info/exclude` is detected and refused |
| tracked injection regression | passed; modified tracked content hidden by both `assume-unchanged` and `skip-worktree` is content-hashed and refused |
| executable-mode regression | passed on the POSIX fixture for both index flags; the mounted project workspace has `core.filemode=false`, so its exact index tree is the canonical mode record and physical mode bits are not treated as meaningful |
| `git check-ignore -v build/llama.cpp/source` | passed; `.gitignore:6:build/` |
| `git diff --summary` and `git ls-files build packages/node/native/llama` | no source checkout or new submodule introduced; only manifest/docs/patches/native wrapper are tracked |
| `python3 scripts/llama_cpp_dependency.py fetch` (controller network lane) | passed; fetched the exact detached commit and verified HEAD `e920c523e3b8a0163fe498af5bf90df35ff51d25` and tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1` in the ignored workspace |
| `python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source` | failed on the pre-existing `0002-dense-llama-owned-range-loader.patch` as a corrupt patch at line 26; this is an explicit DGR-028 blocker and no native-build claim is made |
The targeted test suite creates a local Git fixture to prove offline cache reuse
after full identity verification, then proves a dirty cache is rejected. It
also proves the CLI rejects a repository/branch override and an arbitrary
workspace.
## Limitations
- The controller successfully materialized and verified the exact upstream
commit/tree, so the DGR-027 fetch and offline-cache boundary has real upstream
evidence rather than fixture-only evidence.
- The existing `0002-dense-llama-owned-range-loader.patch` is malformed and
cannot pass `git apply --check` against the exact pin. DGR-027 changes no patch
file; repairing and certifying the numbered patch stack belongs to DGR-028.
Until that story closes, the repository must not claim patched-tree, native
CMake/CTest, or reverse-apply certification.
- No model, API credits, GPU, or model artifact storage was used.
## Dependency handoff
DGR-028, DGR-029, and DGR-044 must invoke the manifest-owned `fetch` command
before touching llama.cpp source. They may use only the verified
`build/llama.cpp/source` checkout and must record any native build, CTest, and
patch apply/check/reverse evidence against the exact manifest pin. DGR-017's
cleanup remains provenance only and grants no inherited completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-022: Define Shard lifecycle and structured status RPCs # DGR-022: Define Shard lifecycle and structured status RPCs
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-021` - **Dependencies:** `DGR-021`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs. - [x] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy. - [x] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker. - [x] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions. - [x] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-022/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-022/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-023: Make Python and C++ protobuf generation reproducible # DGR-023: Make Python and C++ protobuf generation reproducible
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-021` - **Dependencies:** `DGR-021`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Pin protoc, gRPC, and plugin versions or declare a verified compatible range. - [x] Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
- [ ] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands. - [x] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
- [ ] Add Python↔C++ round-trip and descriptor compatibility tests. - [x] Add Python↔C++ round-trip and descriptor compatibility tests.
- [ ] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error. - [x] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-023/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-023/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -0,0 +1,39 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-024: Implement real generated-gRPC protocol harness
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
- **Execution mode:** `AFK`
- **Milestone:** `M1`
- **Dependencies:** `DGR-022`, `DGR-023`
- **Blocks (derived):** `DGR-033`, `DGR-042`
- **Labels:** `area:protocol`, `area:testing`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
- **Evidence class:** `model-free`
- **Hardware:** `none`
- **Model:** `none`
- **Upstream:** `no`
## Objective / description
Build a real generated-gRPC protocol harness around the versioned shard_runtime.proto contract. Use generated Python and C++ stubs over an actual localhost transport and real process lifecycle; exercise captured deterministic protocol vectors and serialized protobuf bytes before a real model worker exists. Do not implement an in-memory fake transport, synthetic model outputs, or a production-looking stub/demo service.
## Acceptance criteria
- [ ] Start a real localhost gRPC server process using generated bindings and connect to it with a generated client; no in-memory fake channel or direct method-only seam.
- [ ] Exercise prefill fragments, decode frames, release, cancel, flow-control, deadlines, malformed input, checksum failure, duplicates, and stale epochs using serialized protocol messages and captured deterministic vectors.
- [ ] Prove direct and opaque-relay paths preserve identical protobuf bytes by recording and comparing actual wire frames at both boundaries.
- [ ] Use real process/socket lifecycle and fail closed on transport, schema, epoch, size, cache, and deadline violations; do not claim model or accelerator behavior that is not exercised.
- [ ] Applicable shared quality gates pass, and evidence records exact commands, raw outputs, generated artifact identities, wire-frame hashes, changed files, limitations, and dependency handoff.
## Shared quality gates
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
- `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-024/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-025: Define exact artifact and runtime recipe identity # DGR-025: Define exact artifact and runtime recipe identity
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-018`, `DGR-021` - **Dependencies:** `DGR-018`, `DGR-021`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout. - [x] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant. - [x] Bind each shard to an exact half-open range without hardcoding a topology or quant.
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch. - [x] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them. - [x] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-027: Add exact llama.cpp provenance manifest and fetch workspace # DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-017` - **Dependencies:** `DGR-017`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method. - [x] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag. - [x] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced. - [x] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
- [ ] Offline reuse is supported only after the cached trees exact identity is verified. - [x] Offline reuse is supported only after the cached trees exact identity is verified.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -489,7 +489,8 @@
"DGR-024", "DGR-024",
"DGR-033", "DGR-033",
"DGR-037" "DGR-037"
] ],
"completionNotes": "Completed 2026-07-17. Implemented the versioned backend-neutral Shard lifecycle/status contract in packages/node/meshnet_node/shard_lifecycle.py (RPC names, schema-version negotiation with fail-closed unsupported versions, structured status/error taxonomy, lifecycle state machine, monotonic idempotency enforcement, bounded flow control, cache expectation/result types, deadline policy and TLS/auth hooks) with deterministic tests in tests/test_shard_lifecycle.py (17 passed alongside the DGR-021 envelope tests). Generated protobuf bindings remain DGR-023. Evidence: evidence/DGR-022/README.md. These completionNotes were added during the DGR-025 backlog-consistency repair; the DGR-022 session omitted them."
}, },
{ {
"id": "DGR-023", "id": "DGR-023",
@@ -521,8 +522,9 @@
"A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.", "A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md; prd.json is authoritative.", "notes": "Completed from Gitea #7 after controller provisioned and exercised the exact Python/C++ toolchains. Verified deterministic generation, native CMake/CTest, Python↔C++ byte parity, compileall, and diff checks; fixed relative bootstrap prefix resolution.",
"completionNotes": "Verified exact grpcio-tools 1.82.1, Protobuf 33.1, Abseil 20250814.1, and gRPC C++ 1.82.1 at commit acccf84c0df20487d64101f528e5d426541ca4e5. Mandatory Python/C++ message and service generation, native CTest, deterministic regeneration, and byte-for-byte Python/C++ parity passed; see evidence/DGR-023/README.md.",
"blocks": [ "blocks": [
"DGR-024", "DGR-024",
"DGR-037" "DGR-037"
@@ -530,7 +532,7 @@
}, },
{ {
"id": "DGR-024", "id": "DGR-024",
"title": "Implement in-memory fake gRPC seam transport", "title": "Implement real generated-gRPC protocol harness",
"priority": 8, "priority": 8,
"milestone": "M1", "milestone": "M1",
"executionMode": "AFK", "executionMode": "AFK",
@@ -541,26 +543,26 @@
"priority:p0", "priority:p0",
"ready-for-agent" "ready-for-agent"
], ],
"evidenceClass": "fixture", "evidenceClass": "model-free",
"evidencePath": ".scratch/distributed-gguf-runtime/evidence/DGR-024/README.md", "evidencePath": ".scratch/distributed-gguf-runtime/evidence/DGR-024/README.md",
"hardware": "none", "hardware": "none",
"model": "fake", "model": "none",
"upstream": "no", "upstream": "no",
"dependsOn": [ "dependsOn": [
"DGR-022", "DGR-022",
"DGR-023" "DGR-023"
], ],
"triage": "ready-for-agent", "triage": "ready-for-agent",
"description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md`, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.", "description": "Build a real generated-gRPC protocol harness around the versioned shard_runtime.proto contract. Use generated Python and C++ stubs over an actual localhost transport and real process lifecycle; exercise captured deterministic protocol vectors and serialized protobuf bytes before a real model worker exists. Do not implement an in-memory fake transport, synthetic model outputs, or a production-looking stub/demo service.",
"acceptanceCriteria": [ "acceptanceCriteria": [
"Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.", "Start a real localhost gRPC server process using generated bindings and connect to it with a generated client; no in-memory fake channel or direct method-only seam.",
"Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.", "Exercise prefill fragments, decode frames, release, cancel, flow-control, deadlines, malformed input, checksum failure, duplicates, and stale epochs using serialized protocol messages and captured deterministic vectors.",
"Verify direct and opaque-relay framing preserve identical protobuf bytes.", "Prove direct and opaque-relay paths preserve identical protobuf bytes by recording and comparing actual wire frames at both boundaries.",
"Tests require no sockets outside localhost, model downloads, or native accelerator.", "Use real process/socket lifecycle and fail closed on transport, schema, epoch, size, cache, and deadline violations; do not claim model or accelerator behavior that is not exercised.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates pass, and evidence records exact commands, raw outputs, generated artifact identities, wire-frame hashes, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": false,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md; prd.json is authoritative.", "notes": "Revised by policy audit: the former in-memory fake/stub seam task was invalid under the no-fake-data/no-demo-implementation rule. Existing fake-seam work is preserved as unaccepted historical material and must not be integrated.",
"blocks": [ "blocks": [
"DGR-033", "DGR-033",
"DGR-042" "DGR-042"
@@ -597,14 +599,15 @@
"Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.", "Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-026", "DGR-026",
"DGR-031", "DGR-031",
"DGR-041", "DGR-041",
"DGR-044" "DGR-044"
] ],
"completionNotes": "Completed 2026-07-17. Verified the live DGR-003-lineage identity core against every criterion: node packages/node/meshnet_node/runtime_recipe.py and the independent tracker packages/tracker/meshnet_tracker/recipe.py (pinned together by tests/data/recipe_fingerprint_vectors.json) fingerprint the source artifact SHA, tokenizer pin, architecture adapter and config digest, boundary/protocol schema versions, backend, weight quant, activation/compute dtypes, and KV dtype/layout under domain-separated digests; shards bind to exact half-open ranges with no topology or quant constants; route/handshake/session checks fail closed with structured mismatch reasons; recipes stay registered-but-dark in the tracker CertificationLedger until a real >=2-distinct-node whole-model distributed forward certifies them. Closed the one open criterion gap (runtime pin/patch stack): new packages/node/meshnet_node/runtime_pin.py derives the runtime_version axis from the DGR-027 lock manifest — exact upstream commit plus a digest over the ordered patch-stack bytes — failing closed on any UPSTREAM_LOCK.json/UPSTREAM_COMMIT/series/SHA256SUMS/patch-byte disagreement, and both identity implementations now reject a moving runtime_version reference. Tests: tests/test_runtime_pin_identity.py (17 passed) plus 196 passing impacted identity/admission/native-emission tests; python -m compileall and git diff --check clean. Also repaired backlog consistency left by prior sessions: added the missing DGR-022/DGR-027 completionNotes, regenerated the DGR-022/025/027 issue projections, and relocated three pre-DGR legacy GLM alpha issue files to issues/legacy/."
}, },
{ {
"id": "DGR-026", "id": "DGR-026",
@@ -673,13 +676,14 @@
"Offline reuse is supported only after the cached trees exact identity is verified.", "Offline reuse is supported only after the cached trees exact identity is verified.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md; prd.json is authoritative.", "notes": "Completed from Gitea #11 via the Terra Ralph lane; independently reviewed after fail-closed cache identity hardening. Verified 7 focused tests, exact real-cache reuse, compileall, and diff-check. The pre-existing malformed 0002 patch is handed to DGR-028.",
"blocks": [ "blocks": [
"DGR-028", "DGR-028",
"DGR-029", "DGR-029",
"DGR-044" "DGR-044"
] ],
"completionNotes": "Completed 2026-07-17. Added the exact llama.cpp provenance manifest packages/node/native/llama/UPSTREAM_LOCK.json pinning commit e920c523e3b8a0163fe498af5bf90df35ff51d25 (tree 6c91a11407a3a3fb160f5dac705f9c59718f54f1) with the sole git-clone-detached-commit retrieval into the ignored build/llama.cpp workspace, fail-closed dirty/mismatched-cache verification, PATCH-STACK.md, scripts/llama_cpp_dependency.py, and tests/test_llama_cpp_dependency.py (7 passed). Known blocker recorded for DGR-028: 0002-dense-llama-owned-range-loader.patch fails git apply --check against the pin, so no patched-tree/native-build claim is made. Evidence: evidence/DGR-027/README.md. These completionNotes were added during the DGR-025 backlog-consistency repair; the DGR-027 session omitted them."
}, },
{ {
"id": "DGR-028", "id": "DGR-028",

View File

@@ -12,6 +12,7 @@ from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb
from .runtime_pin import load_runtime_pin
from .runtime_recipe import ( from .runtime_recipe import (
ArtifactIdentity, ArtifactIdentity,
DerivativeBinding, DerivativeBinding,
@@ -72,7 +73,6 @@ class NativeNumericalRecipe:
kv_layout: str kv_layout: str
architecture_adapter: str architecture_adapter: str
backend_id: str backend_id: str
runtime_version: str
recipe_id: str recipe_id: str
recipe_version: str recipe_version: str
catalogue_version: str catalogue_version: str
@@ -95,6 +95,11 @@ def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIden
report = inputs.loaded_artifact report = inputs.loaded_artifact
pin = inputs.artifact_pin pin = inputs.artifact_pin
recipe = inputs.numerical_recipe recipe = inputs.numerical_recipe
if recipe.backend_id.strip().lower() not in {"llama.cpp", "llama-cpp"}:
raise RecipeIdentityError(
"native llama.cpp identity requires backend_id 'llama.cpp' or 'llama-cpp'"
)
runtime_version = load_runtime_pin().runtime_version
artifact = ArtifactIdentity( artifact = ArtifactIdentity(
artifact_id=pin.artifact_id, artifact_id=pin.artifact_id,
revision=pin.revision, revision=pin.revision,
@@ -115,7 +120,7 @@ def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIden
tokenizer_revision=inputs.tokenizer_revision, tokenizer_revision=inputs.tokenizer_revision,
architecture_adapter=recipe.architecture_adapter, architecture_adapter=recipe.architecture_adapter,
backend_id=recipe.backend_id, backend_id=recipe.backend_id,
runtime_version=recipe.runtime_version, runtime_version=runtime_version,
boundary_schema_version=recipe.boundary_schema_version, boundary_schema_version=recipe.boundary_schema_version,
protocol_schema_version=recipe.protocol_schema_version, protocol_schema_version=recipe.protocol_schema_version,
recipe_id=recipe.recipe_id, recipe_id=recipe.recipe_id,

View File

@@ -0,0 +1,221 @@
"""Canonical runtime pin identity for the recipe fingerprint (DGR-025).
The recipe digest (:mod:`meshnet_node.runtime_recipe`) commits to a
``runtime_version`` axis, but a string the operator typed is a label, not a
pin: two workers could run different patch stacks under the same label and
still hash to the same recipe. The DGR-027 lock manifest
(``packages/node/native/llama``) already records the one exact upstream commit
and the ordered patch stack the native runtime is built from, so the axis
value is *derived* from that manifest, never asserted.
The derived value has three load-bearing parts, and each is separately fatal
to compatibility: the runtime name (from the upstream URL), the exact
40-character upstream commit, and a digest over the ordered patch-stack bytes.
A different upstream pin, a reordered stack, or a single changed patch byte
each produce a different axis value, which produces a different recipe digest,
which partitions the route — exactly the fail-closed behavior DGR-025 asks
for.
Every consistency check here fails closed. The manifest keeps three records of
the stack — ``UPSTREAM_LOCK.json``'s ``patch_series``, ``patches/series``, and
``patches/SHA256SUMS`` — plus the ``UPSTREAM_COMMIT`` convenience file, and a
disagreement between any two of them means the workspace's identity is
unknowable, not "probably fine". This module reads the committed manifest
only; fetching and patching the actual source tree stays with
``scripts/llama_cpp_dependency.py`` (DGR-027).
"""
from __future__ import annotations
import hashlib
import json
import re
from dataclasses import dataclass
from pathlib import Path
# Domain separation, matching the runtime_recipe digest convention: a patch
# stack digest can never be confused with an artifact or recipe digest.
PATCH_STACK_DIGEST_DOMAIN = "meshnet.runtime-patch-stack.v1"
# The UPSTREAM_LOCK.json layout this reader understands (DGR-027 schema).
RUNTIME_PIN_SCHEMA_VERSION = 1
# The committed DGR-027 manifest for the llama.cpp runtime.
DEFAULT_LOCK_DIR = Path(__file__).resolve().parent.parent / "native" / "llama"
_HEX40 = re.compile(r"^[0-9a-f]{40}$")
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
class RuntimePinError(ValueError):
"""The lock workspace is missing, malformed, or internally inconsistent."""
def _canonical_sha256(value: object) -> str:
payload = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
@dataclass(frozen=True)
class RuntimePin:
"""One exact runtime: a name, an upstream commit, and an ordered patch stack."""
runtime_name: str
upstream_commit: str
patch_series: tuple[str, ...]
patch_digests: tuple[str, ...]
@property
def patch_stack_digest(self) -> str:
"""A digest over the ordered (name, bytes-digest) stack.
Order is digested deliberately: applying the same patches in a
different order can produce a different tree, so a reordered stack is
a different runtime.
"""
return _canonical_sha256(
{
"domain": PATCH_STACK_DIGEST_DOMAIN,
"body": {
"patches": [
[name, digest]
for name, digest in zip(self.patch_series, self.patch_digests)
]
},
}
)
@property
def runtime_version(self) -> str:
"""The exact ``runtime_version`` recipe axis value for this pin."""
return (
f"{self.runtime_name}@{self.upstream_commit}"
f"+patchstack.{self.patch_stack_digest}"
)
def _read_text(path: Path, what: str) -> str:
try:
return path.read_text(encoding="utf-8")
except FileNotFoundError:
raise RuntimePinError(f"{what} not found at {path}") from None
except OSError as exc:
raise RuntimePinError(f"{what} at {path} is unreadable: {exc}") from exc
def _read_series_file(path: Path) -> list[str]:
lines = _read_text(path, "patches/series").splitlines()
return [line.strip() for line in lines if line.strip() and not line.startswith("#")]
def _read_sums_file(path: Path) -> list[tuple[str, str]]:
entries: list[tuple[str, str]] = []
for line in _read_text(path, "patches/SHA256SUMS").splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
parts = line.split(None, 1)
if len(parts) != 2 or not _HEX64.match(parts[0]):
raise RuntimePinError(
"patches/SHA256SUMS contains a line that is not "
"'<sha256> <patch name>'"
)
entries.append((parts[0], parts[1].strip()))
return entries
def load_runtime_pin(lock_dir: Path = DEFAULT_LOCK_DIR) -> RuntimePin:
"""Derive the exact runtime pin from a DGR-027 lock workspace, or refuse.
Refuses — rather than guessing — on a missing or malformed lock, a moving
commit reference, a disagreement between the lock's ``patch_series``, the
``patches/series`` file, ``patches/SHA256SUMS``, or the actual patch
bytes, and on an ``UPSTREAM_COMMIT`` file that names a different commit.
"""
lock_path = lock_dir / "UPSTREAM_LOCK.json"
raw = _read_text(lock_path, "UPSTREAM_LOCK.json")
try:
lock = json.loads(raw)
except json.JSONDecodeError as exc:
raise RuntimePinError(f"UPSTREAM_LOCK.json is not valid JSON: {exc}") from exc
if not isinstance(lock, dict):
raise RuntimePinError("UPSTREAM_LOCK.json must be a JSON object")
schema = lock.get("schema_version")
if schema != RUNTIME_PIN_SCHEMA_VERSION:
raise RuntimePinError(
f"UPSTREAM_LOCK.json declares schema version {schema!r}; this reader "
f"understands version {RUNTIME_PIN_SCHEMA_VERSION}"
)
upstream = lock.get("upstream")
if not isinstance(upstream, str) or not upstream.strip():
raise RuntimePinError("UPSTREAM_LOCK.json is missing the upstream URL")
runtime_name = upstream.rstrip("/").rsplit("/", 1)[-1]
if runtime_name.endswith(".git"):
runtime_name = runtime_name[: -len(".git")]
if not runtime_name:
raise RuntimePinError("the upstream URL does not name a runtime")
commit = lock.get("commit")
if not isinstance(commit, str) or not _HEX40.match(commit):
raise RuntimePinError(
f"UPSTREAM_LOCK.json commit {commit!r} is not an exact 40-character "
"hexadecimal object id; a moving reference is not a pin"
)
commit_file = _read_text(lock_dir / "UPSTREAM_COMMIT", "UPSTREAM_COMMIT")
recorded = commit_file.strip().splitlines()[0].strip() if commit_file.strip() else ""
if recorded != commit:
raise RuntimePinError(
"UPSTREAM_COMMIT and UPSTREAM_LOCK.json disagree on the pinned commit"
)
lock_series = lock.get("patch_series")
if not isinstance(lock_series, list) or not all(
isinstance(name, str) and name.strip() for name in lock_series
):
raise RuntimePinError(
"UPSTREAM_LOCK.json patch_series must be a list of patch file names"
)
if len(set(lock_series)) != len(lock_series):
raise RuntimePinError("UPSTREAM_LOCK.json patch_series contains a duplicate")
series = _read_series_file(lock_dir / "patches" / "series")
if series != lock_series:
raise RuntimePinError(
"patches/series and UPSTREAM_LOCK.json patch_series disagree on the "
"ordered patch stack"
)
sums = _read_sums_file(lock_dir / "patches" / "SHA256SUMS")
if [name for _, name in sums] != lock_series:
raise RuntimePinError(
"patches/SHA256SUMS does not record exactly the ordered patch stack "
"named by UPSTREAM_LOCK.json"
)
digests: list[str] = []
for (expected_digest, name) in sums:
patch_path = lock_dir / "patches" / name
try:
body = patch_path.read_bytes()
except FileNotFoundError:
raise RuntimePinError(f"patch file {name} is named but missing") from None
except OSError as exc:
raise RuntimePinError(f"patch file {name} is unreadable: {exc}") from exc
actual = hashlib.sha256(body).hexdigest()
if actual != expected_digest:
raise RuntimePinError(
f"patch file {name} does not match its patches/SHA256SUMS digest"
)
digests.append(actual)
return RuntimePin(
runtime_name=runtime_name,
upstream_commit=commit,
patch_series=tuple(lock_series),
patch_digests=tuple(digests),
)

View File

@@ -125,6 +125,10 @@ _AXIS_MISMATCH: Mapping[str, str] = {
} }
_HEX64 = re.compile(r"^[0-9a-f]{64}$") _HEX64 = re.compile(r"^[0-9a-f]{64}$")
_LLAMA_CPP_RUNTIME_PIN = re.compile(
r"^llama\.cpp@[0-9a-f]{40}\+patchstack\.[0-9a-f]{64}$"
)
_LLAMA_CPP_BACKEND_IDS = frozenset({"llama.cpp", "llama-cpp"})
# A revision that can move is not a pin. DGR-017 learned this on the artifact; # A revision that can move is not a pin. DGR-017 learned this on the artifact;
# it is just as true of a tokenizer. # it is just as true of a tokenizer.
@@ -194,6 +198,17 @@ def _require_pin(value: Any, what: str) -> str:
return text return text
def _require_runtime_pin(value: Any, backend_id: Any) -> str:
text = _require_pin(value, "recipe.runtime_version")
backend = _require_text(backend_id, "recipe.backend_id").strip().lower()
if backend in _LLAMA_CPP_BACKEND_IDS and not _LLAMA_CPP_RUNTIME_PIN.fullmatch(text):
raise RecipeIdentityError(
"'recipe.runtime_version' for llama.cpp must be "
"'llama.cpp@<40-hex commit>+patchstack.<64-hex digest>'"
)
return text
def _as_mapping(value: Any, what: str) -> Mapping[str, Any]: def _as_mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping): if not isinstance(value, Mapping):
raise RecipeIdentityError( raise RecipeIdentityError(
@@ -374,6 +389,11 @@ class RuntimeRecipe:
one in fp16, produce different logits from the same bytes. Keeping the axes one in fp16, produce different logits from the same bytes. Keeping the axes
apart is the entire safety property; see :data:`RECIPE_AXES`. apart is the entire safety property; see :data:`RECIPE_AXES`.
`tokenizer_revision` and `runtime_version` must be exact pins, never moving
references. For the native runtime the canonical `runtime_version` value —
committing to the exact upstream commit *and* the ordered patch stack — is
derived from the DGR-027 lock manifest by :mod:`meshnet_node.runtime_pin`.
The three label fields are diagnosis only and are not digested. The three label fields are diagnosis only and are not digested.
""" """
@@ -400,6 +420,7 @@ class RuntimeRecipe:
else: else:
_require_text(value, f"recipe.{axis}") _require_text(value, f"recipe.{axis}")
_require_pin(self.tokenizer_revision, "recipe.tokenizer_revision") _require_pin(self.tokenizer_revision, "recipe.tokenizer_revision")
_require_runtime_pin(self.runtime_version, self.backend_id)
_require_text(self.recipe_id, "recipe.recipe_id") _require_text(self.recipe_id, "recipe.recipe_id")
_require_text(self.recipe_version, "recipe.recipe_version") _require_text(self.recipe_version, "recipe.recipe_version")
_require_text(self.catalogue_version, "recipe.catalogue_version") _require_text(self.catalogue_version, "recipe.catalogue_version")

View File

@@ -4,10 +4,8 @@
# never committed. A C++ consumer already needs a toolchain, so committing # never committed. A C++ consumer already needs a toolchain, so committing
# generated C++ would only create a second copy of the schema that can rot. # generated C++ would only create a second copy of the schema that can rot.
# #
# gRPC C++ is optional here on purpose. The conformance test only needs message # Protobuf and gRPC C++ are required together so message and service bindings are
# types, so the schema can be verified on a machine that has protobuf but not # generated by one exact toolchain. The ignored bootstrap prefix supplies both.
# the gRPC C++ stack. When gRPC *is* found, the service stubs are generated too
# and exported as `shard_runtime_grpc` for the worker (DGR-008) to link.
# #
# Build: # Build:
# cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=<protobuf-install> # cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=<protobuf-install>
@@ -23,8 +21,17 @@ project(meshnet_shard_protocol CXX)
set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(protobuf CONFIG REQUIRED) # Protobuf and gRPC are one pinned generation toolchain. Configure only against
find_package(gRPC CONFIG QUIET) # the ignored prefix produced by scripts/bootstrap_native_toolchain.sh; accepting
# an arbitrary system plugin would make generated service bindings host-dependent.
set(MESHNET_PROTOBUF_VERSION "33.1.0")
set(MESHNET_GRPC_VERSION "1.82.1")
find_package(protobuf ${MESHNET_PROTOBUF_VERSION} EXACT CONFIG REQUIRED)
find_package(gRPC ${MESHNET_GRPC_VERSION} EXACT CONFIG REQUIRED)
if(NOT TARGET gRPC::grpc_cpp_plugin)
message(FATAL_ERROR "pinned gRPC package does not export grpc_cpp_plugin")
endif()
set(SHARD_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/proto/shard_runtime.proto") set(SHARD_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/proto/shard_runtime.proto")
@@ -39,24 +46,19 @@ protobuf_generate(
PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}" PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}"
) )
# Service stubs: only when the gRPC C++ stack is present. # Service stubs are part of the reproducibility contract, not an optional branch.
if(gRPC_FOUND) add_library(shard_runtime_grpc STATIC "${SHARD_PROTO}")
add_library(shard_runtime_grpc STATIC "${SHARD_PROTO}") target_link_libraries(shard_runtime_grpc PUBLIC shard_runtime_proto gRPC::grpc++)
target_link_libraries(shard_runtime_grpc PUBLIC shard_runtime_proto gRPC::grpc++) target_include_directories(shard_runtime_grpc PUBLIC "${CMAKE_CURRENT_BINARY_DIR}")
target_include_directories(shard_runtime_grpc PUBLIC "${CMAKE_CURRENT_BINARY_DIR}") protobuf_generate(
protobuf_generate(
TARGET shard_runtime_grpc TARGET shard_runtime_grpc
LANGUAGE grpc LANGUAGE grpc
GENERATE_EXTENSIONS .grpc.pb.h .grpc.pb.cc GENERATE_EXTENSIONS .grpc.pb.h .grpc.pb.cc
PLUGIN "protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>" PLUGIN "protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>"
IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto" IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto"
PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}" PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}"
) )
message(STATUS "gRPC C++ found: building ShardRuntime service stubs") message(STATUS "Pinned gRPC ${gRPC_VERSION}: building ShardRuntime service stubs")
else()
message(STATUS "gRPC C++ not found: building message types only "
"(sufficient for the conformance test)")
endif()
enable_testing() enable_testing()

View File

@@ -46,7 +46,7 @@ sampled token to request/recipe identity and sampling/template/reasoning inputs.
If the machine has no protobuf C++ toolchain: If the machine has no protobuf C++ toolchain:
```bash ```bash
scripts/bootstrap_native_toolchain.sh build/native-toolchain bash scripts/bootstrap_native_toolchain.sh build/native-toolchain
``` ```
Then: Then:
@@ -58,10 +58,10 @@ cmake --build build/native -j
ctest --test-dir build/native --output-on-failure ctest --test-dir build/native --output-on-failure
``` ```
gRPC C++ is optional: without it, CMake builds the message types only, which is The bootstrap pins and builds Protobuf `33.1`, gRPC C++ `1.82.1`, and the
all the conformance test needs. When gRPC C++ *is* found, the `ShardRuntime` matching `grpc_cpp_plugin` into one ignored prefix. CMake requires those exact
service stubs are built too and exported as `shard_runtime_grpc` for the worker package versions and always generates both message and service stubs; it does
(DGR-008) to link. not fall back to an arbitrary system plugin.
## How the cross-language check actually proves something ## How the cross-language check actually proves something

View File

@@ -15,11 +15,19 @@ loading, endpoint ownership, architecture-defined intermediate boundaries, and
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay, layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
billing, authentication, and telemetry must remain outside this directory. billing, authentication, and telemetry must remain outside this directory.
`scripts/llama_cpp_dependency.py` verifies the exact commit/tree and baseline `scripts/llama_cpp_dependency.py fetch` reads the in-repo manifest and checks out
blobs, validates every patch digest and context with `git apply --check`, then only its exact commit as detached HEAD in `build/llama.cpp/source`, an ignored
applies the series in `patches/series` order. It refuses a dirty source tree, build workspace. It verifies the exact commit/tree and baseline blobs before use. A
wrong commit/tree/blob, changed patch digest, reordered series, or an existing later offline `fetch` may reuse that cache only after the same clean identity
destination/work directory. verification; an attached branch, tag/repository override, arbitrary destination,
symlinked workspace, dirty checkout, ignored injected file, or tracked-file
modification hidden by Git index flags is refused. The tool
validates every patch digest and context
with `git apply --check`, then applies the series in `patches/series` order.
Tracked executable modes are checked physically when Git reports
`core.filemode=true`; on mounted filesystems without POSIX mode fidelity, the
locked index tree remains the canonical mode record while every working-file
blob is independently re-hashed.
## Current semantic boundary ## Current semantic boundary

View File

@@ -3,6 +3,13 @@
"upstream": "https://github.com/ggml-org/llama.cpp.git", "upstream": "https://github.com/ggml-org/llama.cpp.git",
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25", "commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1", "commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
"expected_source": {
"git_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1"
},
"retrieval": {
"method": "git-clone-detached-commit",
"workspace": "build/llama.cpp"
},
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452", "patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
"upstream_license": "MIT", "upstream_license": "MIT",
"patch_series": [ "patch_series": [

View File

@@ -70,6 +70,10 @@ STATUS_CERTIFIED = "certified"
MIN_CERTIFYING_NODES = 2 MIN_CERTIFYING_NODES = 2
_HEX64 = re.compile(r"^[0-9a-f]{64}$") _HEX64 = re.compile(r"^[0-9a-f]{64}$")
_LLAMA_CPP_RUNTIME_PIN = re.compile(
r"^llama\.cpp@[0-9a-f]{40}\+patchstack\.[0-9a-f]{64}$"
)
_LLAMA_CPP_BACKEND_IDS = frozenset({"llama.cpp", "llama-cpp"})
_MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"}) _MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"})
@@ -128,6 +132,17 @@ def _pin(value: Any, what: str) -> str:
return text return text
def _runtime_pin(value: Any, backend_id: Any) -> str:
text = _pin(value, "recipe.runtime_version")
backend = _text(backend_id, "recipe.backend_id").strip().lower()
if backend in _LLAMA_CPP_BACKEND_IDS and not _LLAMA_CPP_RUNTIME_PIN.fullmatch(text):
raise RecipeIdentityError(
"'recipe.runtime_version' for llama.cpp must bind a 40-hex commit "
"and a 64-hex ordered patch-stack digest"
)
return text
def _mapping(value: Any, what: str) -> Mapping[str, Any]: def _mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping): if not isinstance(value, Mapping):
raise RecipeIdentityError(f"{what!r} must be a JSON object") raise RecipeIdentityError(f"{what!r} must be a JSON object")
@@ -333,6 +348,7 @@ def parse_identity(data: Any) -> PresentedIdentity:
else: else:
axes[axis] = _text(value, f"recipe.{axis}") axes[axis] = _text(value, f"recipe.{axis}")
_pin(axes["tokenizer_revision"], "recipe.tokenizer_revision") _pin(axes["tokenizer_revision"], "recipe.tokenizer_revision")
_runtime_pin(axes["runtime_version"], axes["backend_id"])
identity = PresentedIdentity( identity = PresentedIdentity(
artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"), artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"),

View File

@@ -1,18 +1,13 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build a protobuf C++ toolchain for the native Shard protocol. # Build a protobuf C++ toolchain for the native Shard protocol.
# #
# The Python side needs nothing beyond `pip install grpcio-tools` — it bundles # The Python side uses the exact grpcio-tools pin declared below. The C++ side
# protoc. The C++ side needs libprotobuf headers and a protoc binary, and a # builds exact Protobuf, Abseil, and gRPC source revisions so `protoc`,
# machine that has neither (no protobuf-devel, no cmake, no system protoc) can # `grpc_cpp_plugin`, headers, and libraries all come from one ignored prefix.
# still get a working one from source with this script. It is the exact recipe # No system Protobuf/gRPC installation is accepted by the documented build.
# DGR-002 used to build and run the C++ conformance test.
#
# gRPC C++ is deliberately NOT built here. The conformance test only needs
# message types, so verifying the schema does not require the whole gRPC stack.
# The worker (DGR-008) will need gRPC C++ and should extend this script then.
# #
# Usage: # Usage:
# scripts/bootstrap_native_toolchain.sh [install-prefix] # bash scripts/bootstrap_native_toolchain.sh [install-prefix]
# #
# Then: # Then:
# cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=<prefix> # cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=<prefix>
@@ -21,7 +16,17 @@
set -euo pipefail set -euo pipefail
PREFIX="${1:-${PWD}/build/native-toolchain}" resolve_prefix() {
local candidate="${1:-${PWD}/build/native-toolchain}"
realpath -m -- "${candidate}"
}
if [[ "${1:-}" == "--print-prefix" ]]; then
resolve_prefix "${2:-}"
exit 0
fi
PREFIX="$(resolve_prefix "${1:-}")"
WORK="$(mktemp -d)" WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT trap 'rm -rf "${WORK}"' EXIT
@@ -29,11 +34,15 @@ trap 'rm -rf "${WORK}"' EXIT
# that stub is allowed to use, so these are exact, not floating. # that stub is allowed to use, so these are exact, not floating.
PROTOBUF_VERSION="33.1" PROTOBUF_VERSION="33.1"
ABSEIL_VERSION="20250814.1" ABSEIL_VERSION="20250814.1"
GRPC_VERSION="1.82.1"
GRPC_COMMIT="acccf84c0df20487d64101f528e5d426541ca4e5"
command -v cmake >/dev/null || { for tool in cmake curl git realpath sha256sum tar; do
echo "cmake is required (pip install cmake==4.4.0)" >&2 command -v "${tool}" >/dev/null || {
echo "${tool} is required" >&2
exit 1 exit 1
} }
done
echo "--- fetching protobuf ${PROTOBUF_VERSION} and abseil ${ABSEIL_VERSION}" echo "--- fetching protobuf ${PROTOBUF_VERSION} and abseil ${ABSEIL_VERSION}"
cd "${WORK}" cd "${WORK}"
@@ -41,26 +50,77 @@ curl -sfL -o protobuf.tar.gz \
"https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protobuf-${PROTOBUF_VERSION}.tar.gz" "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protobuf-${PROTOBUF_VERSION}.tar.gz"
tar xzf protobuf.tar.gz tar xzf protobuf.tar.gz
# The protobuf release tarball ships utf8_range but not abseil, and its default
# CMake provider expects abseil as a submodule, so vendor it into place.
curl -sfL -o abseil.tar.gz \ curl -sfL -o abseil.tar.gz \
"https://github.com/abseil/abseil-cpp/releases/download/${ABSEIL_VERSION}/abseil-cpp-${ABSEIL_VERSION}.tar.gz" "https://github.com/abseil/abseil-cpp/releases/download/${ABSEIL_VERSION}/abseil-cpp-${ABSEIL_VERSION}.tar.gz"
tar xzf abseil.tar.gz tar xzf abseil.tar.gz
rm -rf "protobuf-${PROTOBUF_VERSION}/third_party/abseil-cpp"
mv "abseil-cpp-${ABSEIL_VERSION}" "protobuf-${PROTOBUF_VERSION}/third_party/abseil-cpp"
echo "--- building protobuf into ${PREFIX}" echo "--- building abseil ${ABSEIL_VERSION} into ${PREFIX}"
cmake -S "protobuf-${PROTOBUF_VERSION}" -B build \ cmake -S "abseil-cpp-${ABSEIL_VERSION}" -B abseil-build \
-DCMAKE_BUILD_TYPE=Release \ -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="${PREFIX}" \ -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \ -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-Dprotobuf_ABSL_PROVIDER=module \ -DABSL_ENABLE_INSTALL=ON \
-Dprotobuf_BUILD_TESTS=OFF \ -DABSL_BUILD_TESTING=OFF \
-Dprotobuf_BUILD_SHARED_LIBS=OFF \
-DABSL_PROPAGATE_CXX_STD=ON -DABSL_PROPAGATE_CXX_STD=ON
cmake --build build -j"$(nproc)" cmake --build abseil-build -j"$(nproc)"
cmake --install build cmake --install abseil-build
echo "--- building protobuf ${PROTOBUF_VERSION} into ${PREFIX}"
cmake -S "protobuf-${PROTOBUF_VERSION}" -B protobuf-build \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="${PREFIX}" \
-DCMAKE_PREFIX_PATH="${PREFIX}" \
-Dabsl_DIR="${PREFIX}/lib64/cmake/absl" \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-Dprotobuf_LOCAL_DEPENDENCIES_ONLY=ON \
-Dprotobuf_BUILD_TESTS=OFF \
-Dprotobuf_BUILD_SHARED_LIBS=OFF
cmake --build protobuf-build -j"$(nproc)"
cmake --install protobuf-build
echo "--- fetching gRPC ${GRPC_VERSION} at ${GRPC_COMMIT}"
git init -q grpc-source
git -C grpc-source remote add origin https://github.com/grpc/grpc.git
git -C grpc-source fetch --depth 1 origin "${GRPC_COMMIT}"
git -C grpc-source checkout --detach FETCH_HEAD
git -C grpc-source submodule update --init --recursive --depth 1
[[ "$(git -C grpc-source rev-parse HEAD)" == "${GRPC_COMMIT}" ]] || {
echo "gRPC checkout identity mismatch" >&2
exit 1
}
echo "--- building gRPC ${GRPC_VERSION} and grpc_cpp_plugin into ${PREFIX}"
cmake -S grpc-source -B grpc-build \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="${PREFIX}" \
-DCMAKE_PREFIX_PATH="${PREFIX}" \
-DProtobuf_DIR="${PREFIX}/lib64/cmake/protobuf" \
-Dabsl_DIR="${PREFIX}/lib64/cmake/absl" \
-DgRPC_INSTALL=ON \
-DgRPC_BUILD_TESTS=OFF \
-DgRPC_PROTOBUF_PROVIDER=package \
-DgRPC_ABSL_PROVIDER=package \
-DgRPC_CARES_PROVIDER=module \
-DgRPC_RE2_PROVIDER=module \
-DgRPC_SSL_PROVIDER=module \
-DgRPC_ZLIB_PROVIDER=module \
-DgRPC_BUILD_GRPC_CPP_PLUGIN=ON \
-DgRPC_BUILD_GRPC_CSHARP_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_NODE_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_PHP_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_RUBY_PLUGIN=OFF
cmake --build grpc-build -j"$(nproc)"
cmake --install grpc-build
echo "--- done" echo "--- done"
"${PREFIX}/bin/protoc" --version "${PREFIX}/bin/protoc" --version
[[ -x "${PREFIX}/bin/grpc_cpp_plugin" ]] || {
echo "grpc_cpp_plugin was not installed" >&2
exit 1
}
printf 'gRPC %s commit %s\n' "${GRPC_VERSION}" "${GRPC_COMMIT}"
printf 'grpc_cpp_plugin sha256 '
sha256sum "${PREFIX}/bin/grpc_cpp_plugin" | cut -d' ' -f1
echo "configure the protocol build with: -DCMAKE_PREFIX_PATH=${PREFIX}" echo "configure the protocol build with: -DCMAKE_PREFIX_PATH=${PREFIX}"

View File

@@ -49,7 +49,7 @@ _RECIPE = RuntimeRecipe(
tokenizer_revision="0123456789abcdef", tokenizer_revision="0123456789abcdef",
architecture_adapter="llama/range-v1", architecture_adapter="llama/range-v1",
backend_id="llama.cpp", backend_id="llama.cpp",
runtime_version="llama.cpp@deadbeef+meshnet.1", runtime_version="llama.cpp@" + "d" * 40 + "+patchstack." + "e" * 64,
recipe_id="example-gguf", recipe_id="example-gguf",
recipe_version="1", recipe_version="1",
catalogue_version="2026.07.1", catalogue_version="2026.07.1",

View File

@@ -20,6 +20,7 @@ already requires a toolchain and nothing is gained by committing them.
from __future__ import annotations from __future__ import annotations
import argparse import argparse
from importlib import metadata
import pathlib import pathlib
import shutil import shutil
import subprocess import subprocess
@@ -38,15 +39,32 @@ REQUIRED_GRPCIO_TOOLS = "1.82.1"
_HEADER = "# Generated by scripts/generate_native_protocol.py. Do not edit.\n" _HEADER = "# Generated by scripts/generate_native_protocol.py. Do not edit.\n"
def _generate(into: pathlib.Path) -> None: def _require_grpcio_tools_version() -> None:
"""Run protoc, writing generated modules into `into`."""
try: try:
from grpc_tools import protoc actual = metadata.version("grpcio-tools")
except ImportError: # pragma: no cover - exercised only without the toolchain except metadata.PackageNotFoundError:
sys.exit( sys.exit(
"grpc_tools is required to generate stubs:\n" "grpc_tools is required to generate stubs:\n"
f" pip install grpcio-tools=={REQUIRED_GRPCIO_TOOLS}" f" pip install grpcio-tools=={REQUIRED_GRPCIO_TOOLS}"
) )
if actual != REQUIRED_GRPCIO_TOOLS:
sys.exit(
"wrong grpcio-tools version for deterministic generation: "
f"found {actual}, require {REQUIRED_GRPCIO_TOOLS}\n"
f" pip install --upgrade grpcio-tools=={REQUIRED_GRPCIO_TOOLS}"
)
def _generate(into: pathlib.Path) -> None:
"""Run the exactly pinned protoc, writing generated modules into `into`."""
_require_grpcio_tools_version()
try:
from grpc_tools import protoc
except ImportError: # pragma: no cover - inconsistent/broken installation
sys.exit(
"grpcio-tools metadata exists but grpc_tools cannot be imported; reinstall it:\n"
f" pip install --force-reinstall grpcio-tools=={REQUIRED_GRPCIO_TOOLS}"
)
into.mkdir(parents=True, exist_ok=True) into.mkdir(parents=True, exist_ok=True)
# grpc_tools bundles protoc and the well-known types, so generation needs no # grpc_tools bundles protoc and the well-known types, so generation needs no

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Materialize, verify, build, and smoke-test DGR-004's llama.cpp pin. """Fetch, verify, build, and smoke-test DGR-027's exact llama.cpp pin.
This tool deliberately owns only a source dependency boundary. It never This tool deliberately owns only a source dependency boundary. It never
downloads a model, invokes inference, or interprets generated text. downloads a model, invokes inference, or interprets generated text.
@@ -12,6 +12,7 @@ import hashlib
import json import json
import os import os
import pathlib import pathlib
import re
import shutil import shutil
import subprocess import subprocess
import sys import sys
@@ -67,14 +68,27 @@ def _load_lock() -> dict[str, Any]:
raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error
required = { required = {
"upstream", "commit", "commit_tree", "patched_tree", "patch_series", "upstream", "commit", "commit_tree", "patched_tree", "patch_series",
"required_upstream_blobs", "patched_paths", "build", "required_upstream_blobs", "patched_paths", "build", "upstream_license",
"expected_source", "retrieval",
} }
missing = sorted(required - lock.keys()) missing = sorted(required - lock.keys())
if missing: if missing:
raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}") raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}")
commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip() commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
if lock["commit"] != commit_file or len(commit_file) != 40: object_ids = [commit_file, lock["commit"], lock["commit_tree"], lock["patched_tree"]]
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on a full commit") if lock["commit"] != commit_file or not all(
isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value)
for value in object_ids
):
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on full hexadecimal object IDs")
if lock["expected_source"] != {"git_tree": lock["commit_tree"]}:
raise DependencyError("expected_source must record the locked git tree")
retrieval = lock["retrieval"]
if retrieval != {
"method": "git-clone-detached-commit",
"workspace": "build/llama.cpp",
}:
raise DependencyError("retrieval must use the locked detached-commit build workspace")
return lock return lock
@@ -103,6 +117,56 @@ def _git(source: pathlib.Path, *args: str) -> str:
return _run("git", "-C", str(source), *args) return _run("git", "-C", str(source), *args)
def _verify_tracked_content(source: pathlib.Path, lock: dict[str, Any]) -> None:
if _git(source, "write-tree") != lock["commit_tree"]:
raise DependencyError("materialized checkout index differs from the locked tree")
filemode_trusted = _git(source, "config", "--bool", "core.filemode") == "true"
records = _git(source, "ls-files", "-s", "-z").split("\0")
paths: list[str] = []
expected: list[str] = []
for record in records:
if not record:
continue
metadata, path = record.split("\t", 1)
mode, blob, stage = metadata.split()
if stage != "0" or mode not in {"100644", "100755", "120000"}:
raise DependencyError(f"unsupported tracked entry in materialized checkout: {record!r}")
if "\n" in path:
raise DependencyError(f"newline-bearing tracked path is unsupported: {path!r}")
candidate = source / path
cursor = source
for part in pathlib.Path(path).parts[:-1]:
cursor = cursor / part
if cursor.is_symlink():
raise DependencyError(f"tracked path traverses a symlink: {path!r}")
if mode == "120000":
if not candidate.is_symlink():
raise DependencyError(f"tracked symlink type differs from the locked tree: {path!r}")
else:
if candidate.is_symlink() or not candidate.is_file():
raise DependencyError(f"tracked file type differs from the locked tree: {path!r}")
executable = bool(candidate.stat().st_mode & 0o111)
if filemode_trusted and executable != (mode == "100755"):
raise DependencyError(f"tracked executable mode differs from the locked tree: {path!r}")
paths.append(path)
expected.append(blob)
try:
completed = subprocess.run(
["git", "hash-object", "--stdin-paths"],
cwd=source,
input="".join(f"{path}\n" for path in paths),
check=True,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
except subprocess.CalledProcessError as error:
raise DependencyError(f"unable to hash tracked source content: {error.stderr.strip()}") from error
actual = completed.stdout.splitlines()
if len(actual) != len(expected) or actual != expected:
raise DependencyError("tracked source content differs from the locked tree")
def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None: def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None:
if not (source / ".git").exists(): if not (source / ".git").exists():
raise DependencyError(f"not a materialized git checkout: {source}") raise DependencyError(f"not a materialized git checkout: {source}")
@@ -110,8 +174,15 @@ def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean:
raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit") raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit")
if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]: if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]:
raise DependencyError("upstream drift: checkout tree does not equal the locked tree") raise DependencyError("upstream drift: checkout tree does not equal the locked tree")
if require_clean and _git(source, "status", "--porcelain"): if _git(source, "rev-parse", "--abbrev-ref", "HEAD") != "HEAD":
raise DependencyError("local edits detected in materialized llama.cpp checkout") raise DependencyError("materialized llama.cpp checkout must have a detached HEAD")
if require_clean:
_verify_tracked_content(source, lock)
tracked = _git(source, "status", "--porcelain", "--untracked-files=no")
untracked = _git(source, "ls-files", "--others", "--exclude-standard")
ignored = _git(source, "ls-files", "--others", "--ignored", "--exclude-standard")
if tracked or untracked or ignored:
raise DependencyError("local edits or unmanifested files detected in materialized llama.cpp checkout")
for relative, expected in lock["required_upstream_blobs"].items(): for relative, expected in lock["required_upstream_blobs"].items():
actual = _git(source, "rev-parse", f"HEAD:{relative}") actual = _git(source, "rev-parse", f"HEAD:{relative}")
if actual != expected: if actual != expected:
@@ -122,15 +193,43 @@ def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean:
raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution") raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution")
def materialize(source: pathlib.Path, repository: str) -> None: def _workspace_source(workspace: pathlib.Path, lock: dict[str, Any]) -> pathlib.Path:
relative = pathlib.Path(lock["retrieval"]["workspace"])
expected = (ROOT / relative).absolute()
supplied = workspace.absolute()
if supplied != expected:
raise DependencyError(f"--workspace must equal the locked ignored build root: {expected}")
cursor = ROOT
for part in relative.parts:
cursor = cursor / part
if cursor.is_symlink():
raise DependencyError(f"locked build workspace may not traverse a symlink: {cursor}")
resolved = workspace.resolve()
try:
resolved.relative_to(ROOT.resolve())
except ValueError as error:
raise DependencyError("locked build workspace escapes the repository root") from error
source = resolved / "source"
if source.is_symlink():
raise DependencyError(f"locked source checkout may not be a symlink: {source}")
return source
def fetch(workspace: pathlib.Path) -> pathlib.Path:
"""Fetch once, or verify an exact clean cached checkout for offline reuse."""
lock = _load_lock() lock = _load_lock()
_patches(lock) _patches(lock)
source = _workspace_source(workspace, lock)
if source.exists(): if source.exists():
raise DependencyError(f"destination already exists; refusing to reuse possibly edited source: {source}") _verify_source(source, lock, require_clean=True)
print(f"reused verified offline cache: {source}")
return source
source.parent.mkdir(parents=True, exist_ok=True) source.parent.mkdir(parents=True, exist_ok=True)
_run("git", "clone", "--no-checkout", repository, str(source)) _run("git", "clone", "--no-checkout", lock["upstream"], str(source))
_git(source, "checkout", "--detach", lock["commit"]) _git(source, "checkout", "--detach", lock["commit"])
_verify_source(source, lock, require_clean=True) _verify_source(source, lock, require_clean=True)
print(f"fetched and verified exact source: {source}")
return source
def apply(source: pathlib.Path) -> None: def apply(source: pathlib.Path) -> None:
@@ -198,18 +297,11 @@ def smoke(binary: pathlib.Path) -> None:
print(output) print(output)
def reproduce(work_dir: pathlib.Path, repository: str) -> None: def reproduce(workspace: pathlib.Path) -> None:
resolved = work_dir.resolve() source = fetch(workspace)
build_root = (ROOT / "build").resolve() build_dir = workspace.resolve() / "build"
if build_root not in resolved.parents: if build_dir.exists():
raise DependencyError(f"--work-dir must be below {build_root}: {resolved}") raise DependencyError(f"build directory already exists; refusing to erase possible local edits: {build_dir}")
if resolved.exists():
raise DependencyError(
f"work directory already exists; refusing to erase possible local edits: {resolved}"
)
source = resolved / "source"
build_dir = resolved / "build"
materialize(source, repository)
apply(source) apply(source)
smoke(build(source, build_dir)) smoke(build(source, build_dir))
@@ -219,6 +311,9 @@ def inspect() -> None:
patches = _patches(lock) patches = _patches(lock)
print(json.dumps({ print(json.dumps({
"commit": lock["commit"], "commit": lock["commit"],
"commit_tree": lock["commit_tree"],
"retrieval": lock["retrieval"],
"upstream_license": lock["upstream_license"],
"patch_count": len(patches), "patch_count": len(patches),
"patches": [patch.name for patch in patches], "patches": [patch.name for patch in patches],
"model_downloads": False, "model_downloads": False,
@@ -231,9 +326,8 @@ def main() -> int:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
subcommands = parser.add_subparsers(dest="command", required=True) subcommands = parser.add_subparsers(dest="command", required=True)
subcommands.add_parser("inspect") subcommands.add_parser("inspect")
materialize_parser = subcommands.add_parser("materialize") fetch_parser = subcommands.add_parser("fetch")
materialize_parser.add_argument("--source-dir", type=pathlib.Path, required=True) fetch_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
materialize_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
apply_parser = subcommands.add_parser("apply") apply_parser = subcommands.add_parser("apply")
apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True) apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
build_parser = subcommands.add_parser("build") build_parser = subcommands.add_parser("build")
@@ -242,14 +336,13 @@ def main() -> int:
smoke_parser = subcommands.add_parser("smoke") smoke_parser = subcommands.add_parser("smoke")
smoke_parser.add_argument("--binary", type=pathlib.Path, required=True) smoke_parser.add_argument("--binary", type=pathlib.Path, required=True)
reproduce_parser = subcommands.add_parser("reproduce") reproduce_parser = subcommands.add_parser("reproduce")
reproduce_parser.add_argument("--work-dir", type=pathlib.Path, default=ROOT / "build/dgr-004-smoke") reproduce_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
reproduce_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
args = parser.parse_args() args = parser.parse_args()
try: try:
if args.command == "inspect": if args.command == "inspect":
inspect() inspect()
elif args.command == "materialize": elif args.command == "fetch":
materialize(args.source_dir, args.source_repository) fetch(args.workspace)
elif args.command == "apply": elif args.command == "apply":
apply(args.source_dir) apply(args.source_dir)
elif args.command == "build": elif args.command == "build":
@@ -257,9 +350,9 @@ def main() -> int:
elif args.command == "smoke": elif args.command == "smoke":
smoke(args.binary) smoke(args.binary)
else: else:
reproduce(args.work_dir, args.source_repository) reproduce(args.workspace)
except DependencyError as error: except DependencyError as error:
print(f"DGR-004 dependency error: {error}", file=sys.stderr) print(f"DGR-027 dependency error: {error}", file=sys.stderr)
return 2 return 2
return 0 return 0

View File

@@ -8,9 +8,9 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31", "fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240363330303165306566656164613562393766326633353632353632646330666432643962643839303464633662376239396137316439386633653933386264301a0c6578616d706c652d676775662201312a09323032362e30372e31",
"identity": { "identity": {
"artifact": { "artifact": {
"architecture": "dense-llama", "architecture": "dense-llama",
@@ -26,7 +26,7 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"recipe": { "recipe": {
"activation_dtype": "bfloat16", "activation_dtype": "bfloat16",
@@ -40,7 +40,7 @@
"protocol_schema_version": 1, "protocol_schema_version": 1,
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1", "runtime_version": "llama.cpp@dddddddddddddddddddddddddddddddddddddddd+patchstack.eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
"tokenizer_revision": "0123456789abcdef", "tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M" "weight_quantization": "Q4_K_M"
}, },
@@ -58,9 +58,9 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31", "fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240363330303165306566656164613562393766326633353632353632646330666432643962643839303464633662376239396137316439386633653933386264301a0c6578616d706c652d676775662201312a09323032362e30372e31",
"identity": { "identity": {
"artifact": { "artifact": {
"architecture": "dense-llama", "architecture": "dense-llama",
@@ -80,7 +80,7 @@
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b", "model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa" "runtime_recipe_digest": "63001e0efeada5b97f2f3562562dc0fd2d9bd8904dc6b7b99a71d98f3e938bd0"
}, },
"recipe": { "recipe": {
"activation_dtype": "bfloat16", "activation_dtype": "bfloat16",
@@ -94,7 +94,7 @@
"protocol_schema_version": 1, "protocol_schema_version": 1,
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1", "runtime_version": "llama.cpp@dddddddddddddddddddddddddddddddddddddddd+patchstack.eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
"tokenizer_revision": "0123456789abcdef", "tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M" "weight_quantization": "Q4_K_M"
}, },

View File

@@ -1,8 +1,9 @@
"""Offline guards for DGR-004's pinned llama.cpp dependency boundary.""" """Offline guards for DGR-027's pinned llama.cpp dependency boundary."""
from __future__ import annotations from __future__ import annotations
import hashlib import hashlib
import importlib.util
import json import json
import pathlib import pathlib
import subprocess import subprocess
@@ -33,6 +34,10 @@ def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
assert commit == lock["commit"] assert commit == lock["commit"]
assert len(commit) == 40 assert len(commit) == 40
assert lock["retrieval"]["method"] == "git-clone-detached-commit"
assert lock["retrieval"]["workspace"] == "build/llama.cpp"
assert lock["expected_source"]["git_tree"] == lock["commit_tree"]
assert lock["upstream_license"] == "MIT"
assert patches == lock["patch_series"] assert patches == lock["patch_series"]
assert patches == sorted(patches) assert patches == sorted(patches)
assert patches assert patches
@@ -42,6 +47,188 @@ def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
assert "Subject: [PATCH" in patch.read_text() assert "Subject: [PATCH" in patch.read_text()
def test_fetch_refuses_a_workspace_outside_the_ignored_build_root(tmp_path: pathlib.Path) -> None:
completed = subprocess.run(
[sys.executable, str(SCRIPT), "fetch", "--workspace", str(tmp_path / "llama.cpp")],
cwd=ROOT,
capture_output=True,
text=True,
)
assert completed.returncode == 2
assert "--workspace must equal" in completed.stderr
def test_workspace_refuses_a_symlinked_build_ancestor(tmp_path: pathlib.Path, monkeypatch) -> None:
spec = importlib.util.spec_from_file_location("llama_cpp_dependency_symlink", SCRIPT)
assert spec and spec.loader
dependency = importlib.util.module_from_spec(spec)
spec.loader.exec_module(dependency)
root = tmp_path / "repo"
outside = tmp_path / "outside"
root.mkdir()
outside.mkdir()
(root / "build").symlink_to(outside, target_is_directory=True)
monkeypatch.setattr(dependency, "ROOT", root)
try:
dependency._workspace_source(
root / "build/llama.cpp",
{"retrieval": {"workspace": "build/llama.cpp"}},
)
except dependency.DependencyError as error:
assert "may not traverse a symlink" in str(error)
else:
raise AssertionError("symlinked build ancestor must be refused")
(root / "build").unlink()
workspace = root / "build/llama.cpp"
workspace.mkdir(parents=True)
(workspace / "source").symlink_to(outside, target_is_directory=True)
try:
dependency._workspace_source(
workspace,
{"retrieval": {"workspace": "build/llama.cpp"}},
)
except dependency.DependencyError as error:
assert "source checkout may not be a symlink" in str(error)
else:
raise AssertionError("symlinked source checkout must be refused")
def test_fetch_refuses_a_branch_or_repository_override() -> None:
completed = subprocess.run(
[sys.executable, str(SCRIPT), "fetch", "--source-repository", "main"],
cwd=ROOT,
capture_output=True,
text=True,
)
assert completed.returncode == 2
assert "unrecognized arguments" in completed.stderr
def test_fetch_reuses_only_a_verified_cached_tree_offline(
tmp_path: pathlib.Path, monkeypatch
) -> None:
spec = importlib.util.spec_from_file_location("llama_cpp_dependency", SCRIPT)
assert spec and spec.loader
dependency = importlib.util.module_from_spec(spec)
spec.loader.exec_module(dependency)
root = tmp_path / "repo"
source = root / "build/llama.cpp/source"
upstream = tmp_path / "upstream"
upstream.mkdir()
subprocess.run(["git", "init", "-q", str(upstream)], check=True)
subprocess.run(["git", "-C", str(upstream), "config", "user.email", "test@example.invalid"], check=True)
subprocess.run(["git", "-C", str(upstream), "config", "user.name", "test"], check=True)
(upstream / "CMakeLists.txt").write_text("cmake_minimum_required(VERSION 3.14)\n")
(upstream / "LICENSE").write_text("MIT\n")
(upstream / "tool.sh").write_text("#!/bin/sh\nexit 0\n")
(upstream / "tool.sh").chmod(0o755)
subprocess.run(["git", "-C", str(upstream), "add", "."], check=True)
subprocess.run(["git", "-C", str(upstream), "commit", "-qm", "fixture"], check=True)
commit = subprocess.run(
["git", "-C", str(upstream), "rev-parse", "HEAD"], check=True, capture_output=True, text=True
).stdout.strip()
tree = subprocess.run(
["git", "-C", str(upstream), "rev-parse", "HEAD^{tree}"], check=True, capture_output=True, text=True
).stdout.strip()
blob = subprocess.run(
["git", "-C", str(upstream), "rev-parse", "HEAD:CMakeLists.txt"],
check=True,
capture_output=True,
text=True,
).stdout.strip()
source.parent.mkdir(parents=True)
subprocess.run(["git", "clone", "-q", str(upstream), str(source)], check=True)
llama_dir = root / "packages/node/native/llama"
patch_dir = llama_dir / "patches"
patch_dir.mkdir(parents=True)
(llama_dir / "UPSTREAM_COMMIT").write_text(f"{commit}\n")
(patch_dir / "series").write_text("0001-fixture.patch\n")
patch = patch_dir / "0001-fixture.patch"
patch.write_text("fixture patch\n")
(patch_dir / "SHA256SUMS").write_text(f"{_sha256(patch)} {patch.name}\n")
(llama_dir / "UPSTREAM_LOCK.json").write_text(json.dumps({
"upstream": str(upstream),
"commit": commit,
"commit_tree": tree,
"expected_source": {"git_tree": tree},
"retrieval": {"method": "git-clone-detached-commit", "workspace": "build/llama.cpp"},
"patched_tree": tree,
"patch_series": [patch.name],
"required_upstream_blobs": {"CMakeLists.txt": blob},
"patched_paths": [],
"build": {},
"upstream_license": "MIT",
}))
monkeypatch.setattr(dependency, "ROOT", root)
monkeypatch.setattr(dependency, "LLAMA_DIR", llama_dir)
monkeypatch.setattr(dependency, "LOCK_PATH", llama_dir / "UPSTREAM_LOCK.json")
monkeypatch.setattr(dependency, "PATCH_DIR", patch_dir)
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "detached HEAD" in str(error)
else:
raise AssertionError("attached branch cache must be refused")
subprocess.run(["git", "-C", str(source), "checkout", "--detach", "-q", commit], check=True)
assert dependency.fetch(root / "build/llama.cpp") == source
tracked_path = source / "CMakeLists.txt"
for flag, clear_flag in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
subprocess.run(["git", "-C", str(source), "update-index", flag, "CMakeLists.txt"], check=True)
tracked_path.write_text("injected tracked build input\n")
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "tracked source content differs" in str(error)
else:
raise AssertionError(f"tracked cache injection hidden by {flag} must be refused")
subprocess.run(["git", "-C", str(source), "update-index", clear_flag, "CMakeLists.txt"], check=True)
subprocess.run(["git", "-C", str(source), "checkout", "--", "CMakeLists.txt"], check=True)
executable_path = source / "tool.sh"
for flag, clear_flag in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
subprocess.run(["git", "-C", str(source), "update-index", flag, "tool.sh"], check=True)
executable_path.chmod(0o644)
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "executable mode differs" in str(error)
else:
raise AssertionError(f"tracked mode change hidden by {flag} must be refused")
subprocess.run(["git", "-C", str(source), "update-index", clear_flag, "tool.sh"], check=True)
subprocess.run(["git", "-C", str(source), "checkout", "--", "tool.sh"], check=True)
(source / "untracked.txt").write_text("edited\n")
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "local edits" in str(error)
else:
raise AssertionError("dirty cached source must be refused")
(source / "untracked.txt").unlink()
(source / ".git/info/exclude").write_text("injected.cmake\n")
(source / "injected.cmake").write_text("unmanifested input\n")
try:
dependency.fetch(root / "build/llama.cpp")
except dependency.DependencyError as error:
assert "unmanifested files" in str(error)
else:
raise AssertionError("ignored cached source input must be refused")
def test_dependency_script_reports_the_locked_boundary_without_network() -> None: def test_dependency_script_reports_the_locked_boundary_without_network() -> None:
completed = subprocess.run( completed = subprocess.run(
[sys.executable, str(SCRIPT), "inspect"], [sys.executable, str(SCRIPT), "inspect"],

View File

@@ -15,6 +15,7 @@ from meshnet_node.native_backend import (
shard_identity_from_native_report, shard_identity_from_native_report,
) )
from meshnet_node.native_protocol import SCHEMA_VERSION, pb from meshnet_node.native_protocol import SCHEMA_VERSION, pb
from meshnet_node.runtime_pin import load_runtime_pin
from meshnet_node.recipe_manifest import parse_recipe_manifest from meshnet_node.recipe_manifest import parse_recipe_manifest
from meshnet_tracker.capability import STATE_UNCERTIFIED, evaluate_report from meshnet_tracker.capability import STATE_UNCERTIFIED, evaluate_report
@@ -42,7 +43,6 @@ def _inputs(**changes: object) -> NativeIdentityInputs:
kv_layout="llama-kv-v1", kv_layout="llama-kv-v1",
architecture_adapter="dense-llama-v1", architecture_adapter="dense-llama-v1",
backend_id="llama-cpp", backend_id="llama-cpp",
runtime_version="llama.cpp:e920c523",
recipe_id="native", recipe_id="native",
recipe_version="1", recipe_version="1",
catalogue_version="2026.07.1", catalogue_version="2026.07.1",
@@ -84,6 +84,7 @@ def test_native_identity_uses_loaded_report_not_a_caller_range():
assert (identity.shard_start, identity.shard_end) == (2, 6) assert (identity.shard_start, identity.shard_end) == (2, 6)
assert identity.artifact.architecture == "llama" assert identity.artifact.architecture == "llama"
assert identity.artifact.layer_count == 8 assert identity.artifact.layer_count == 8
assert identity.recipe.runtime_version == load_runtime_pin().runtime_version
def test_native_identity_requires_an_immutable_pin_and_gguf_range(): def test_native_identity_requires_an_immutable_pin_and_gguf_range():

View File

@@ -612,6 +612,27 @@ def test_a_peer_still_sending_the_retired_field_does_not_corrupt_the_tensor():
assert decode_tensor(pb.NamedTensor.FromString(wire)) == b"\xaa" * 32 assert decode_tensor(pb.NamedTensor.FromString(wire)) == b"\xaa" * 32
def test_native_toolchain_bootstrap_resolves_relative_prefix_before_temp_chdir(tmp_path):
script = REPO_ROOT / "scripts/bootstrap_native_toolchain.sh"
result = subprocess.run(
["bash", str(script), "--print-prefix", "relative/toolchain"],
cwd=tmp_path,
capture_output=True,
text=True,
)
assert result.returncode == 0, result.stdout + result.stderr
assert pathlib.Path(result.stdout.strip()) == tmp_path / "relative/toolchain"
def test_python_generator_rejects_the_wrong_grpcio_tools_version(monkeypatch):
from scripts import generate_native_protocol
monkeypatch.setattr(generate_native_protocol.metadata, "version", lambda _: "0.0.0")
with pytest.raises(SystemExit, match="found 0.0.0, require 1.82.1"):
generate_native_protocol._require_grpcio_tools_version()
def test_generated_python_stubs_match_the_proto(): def test_generated_python_stubs_match_the_proto():
pytest.importorskip("grpc_tools", reason="protoc toolchain is required to verify") pytest.importorskip("grpc_tools", reason="protoc toolchain is required to verify")
result = subprocess.run( result = subprocess.run(

View File

@@ -0,0 +1,312 @@
"""DGR-025: the recipe's runtime axis commits to the exact pin and patch stack.
The fingerprint (DGR-003 lineage) digests a ``runtime_version`` string, but a
string an operator typed is a label, not a pin: two workers could run different
patch stacks under the same label and still agree on the digest. These tests
pin the axis to the DGR-027 lock manifest — the exact upstream commit plus a
digest over the ordered patch-stack bytes — and prove both identity
implementations reject a moving runtime reference.
"""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import pytest
from meshnet_node.runtime_pin import (
DEFAULT_LOCK_DIR,
RuntimePinError,
load_runtime_pin,
)
from meshnet_node.runtime_recipe import RecipeIdentityError, RuntimeRecipe
from meshnet_tracker.recipe import (
RecipeIdentityError as TrackerRecipeIdentityError,
parse_identity,
)
REPO_LOCK_DIR = (
Path(__file__).resolve().parent.parent / "packages" / "node" / "native" / "llama"
)
def _recipe(**changes: object) -> RuntimeRecipe:
fields: dict[str, object] = {
"weight_quantization": "Q4_K_M",
"activation_dtype": "bfloat16",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"tokenizer_revision": "0123456789abcdef",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"runtime_version": "llama.cpp@" + "d" * 40 + "+patchstack." + "e" * 64,
"recipe_id": "example-gguf",
"recipe_version": "1",
"catalogue_version": "2026.07.1",
}
fields.update(changes)
return RuntimeRecipe(**fields) # type: ignore[arg-type]
def _write_fixture_workspace(
root: Path,
*,
commit: str = "e" * 40,
patches: dict[str, bytes] | None = None,
lock_series: list[str] | None = None,
series_lines: list[str] | None = None,
sums_lines: list[str] | None = None,
schema_version: int = 1,
upstream: str = "https://github.com/ggml-org/llama.cpp.git",
upstream_commit_file: str | None = None,
) -> Path:
"""A minimal DGR-027-shaped lock workspace; overrides create disagreement."""
if patches is None:
patches = {
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ d\n",
}
names = list(patches)
digests = {
name: hashlib.sha256(body).hexdigest() for name, body in patches.items()
}
patch_dir = root / "patches"
patch_dir.mkdir(parents=True)
for name, body in patches.items():
(patch_dir / name).write_bytes(body)
(root / "UPSTREAM_LOCK.json").write_text(
json.dumps(
{
"schema_version": schema_version,
"upstream": upstream,
"commit": commit,
"patch_series": names if lock_series is None else lock_series,
}
),
encoding="utf-8",
)
(root / "UPSTREAM_COMMIT").write_text(
(commit if upstream_commit_file is None else upstream_commit_file) + "\n",
encoding="utf-8",
)
(patch_dir / "series").write_text(
"\n".join(names if series_lines is None else series_lines) + "\n",
encoding="utf-8",
)
if sums_lines is None:
sums_lines = ["# ordered digests"] + [
f"{digests[name]} {name}" for name in names
]
(patch_dir / "SHA256SUMS").write_text(
"\n".join(sums_lines) + "\n", encoding="utf-8"
)
return root
# --- the committed manifest is the identity source -------------------------
def test_default_lock_dir_is_the_committed_manifest():
assert DEFAULT_LOCK_DIR == REPO_LOCK_DIR
def test_committed_manifest_derives_a_deterministic_runtime_pin():
pin = load_runtime_pin(REPO_LOCK_DIR)
again = load_runtime_pin(REPO_LOCK_DIR)
assert pin == again
lock = json.loads(
(REPO_LOCK_DIR / "UPSTREAM_LOCK.json").read_text(encoding="utf-8")
)
assert pin.upstream_commit == lock["commit"]
assert list(pin.patch_series) == lock["patch_series"]
# The axis value names the runtime, the exact commit, and the stack digest,
# so changing any of the three changes every downstream recipe digest.
assert pin.runtime_version == (
f"llama.cpp@{lock['commit']}+patchstack.{pin.patch_stack_digest}"
)
assert len(pin.patch_stack_digest) == 64
def test_derived_axis_value_is_a_valid_recipe_pin():
pin = load_runtime_pin(REPO_LOCK_DIR)
recipe = _recipe(runtime_version=pin.runtime_version)
assert recipe.runtime_version == pin.runtime_version
assert len(recipe.runtime_recipe_digest) == 64
def test_patch_byte_change_changes_the_runtime_identity(tmp_path):
baseline = load_runtime_pin(_write_fixture_workspace(tmp_path / "a"))
changed = load_runtime_pin(
_write_fixture_workspace(
tmp_path / "b",
patches={
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ DIFFERENT\n",
},
)
)
assert baseline.patch_stack_digest != changed.patch_stack_digest
assert baseline.runtime_version != changed.runtime_version
def test_patch_order_is_part_of_the_stack_identity(tmp_path):
patches = {
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ d\n",
}
forward = load_runtime_pin(
_write_fixture_workspace(tmp_path / "a", patches=patches)
)
names = list(patches)
reversed_names = list(reversed(names))
digests = {
name: hashlib.sha256(body).hexdigest() for name, body in patches.items()
}
swapped = load_runtime_pin(
_write_fixture_workspace(
tmp_path / "b",
patches=patches,
lock_series=reversed_names,
series_lines=reversed_names,
sums_lines=[f"{digests[name]} {name}" for name in reversed_names],
)
)
assert forward.patch_stack_digest != swapped.patch_stack_digest
# --- every manifest disagreement fails closed ------------------------------
def test_missing_lock_file_fails_closed(tmp_path):
with pytest.raises(RuntimePinError, match="UPSTREAM_LOCK.json"):
load_runtime_pin(tmp_path)
def test_malformed_lock_json_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path)
(tmp_path / "UPSTREAM_LOCK.json").write_text("{not json", encoding="utf-8")
with pytest.raises(RuntimePinError, match="not valid JSON"):
load_runtime_pin(tmp_path)
def test_unknown_lock_schema_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path, schema_version=2)
with pytest.raises(RuntimePinError, match="schema"):
load_runtime_pin(tmp_path)
def test_moving_commit_reference_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path, commit="master")
with pytest.raises(RuntimePinError, match="exact 40"):
load_runtime_pin(tmp_path)
def test_upstream_commit_file_disagreement_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path, upstream_commit_file="f" * 40)
with pytest.raises(RuntimePinError, match="UPSTREAM_COMMIT"):
load_runtime_pin(tmp_path)
def test_series_file_disagreement_fails_closed(tmp_path):
_write_fixture_workspace(
tmp_path, series_lines=["0002-second.patch", "0001-first.patch"]
)
with pytest.raises(RuntimePinError, match="series"):
load_runtime_pin(tmp_path)
def test_missing_patch_file_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path)
(tmp_path / "patches" / "0002-second.patch").unlink()
with pytest.raises(RuntimePinError, match="0002-second.patch"):
load_runtime_pin(tmp_path)
def test_checksum_disagreement_fails_closed(tmp_path):
_write_fixture_workspace(tmp_path)
patch = tmp_path / "patches" / "0002-second.patch"
patch.write_bytes(patch.read_bytes() + b"tampered\n")
with pytest.raises(RuntimePinError, match="SHA256SUMS"):
load_runtime_pin(tmp_path)
def test_sums_entry_missing_fails_closed(tmp_path):
patches = {
"0001-first.patch": b"--- a\n+++ b\n",
"0002-second.patch": b"--- c\n+++ d\n",
}
digest = hashlib.sha256(patches["0001-first.patch"]).hexdigest()
_write_fixture_workspace(
tmp_path,
patches=patches,
sums_lines=[f"{digest} 0001-first.patch"],
)
with pytest.raises(RuntimePinError, match="SHA256SUMS"):
load_runtime_pin(tmp_path)
def test_empty_patch_series_requires_empty_series_files(tmp_path):
# An unpatched runtime is a legal pin; a lock that *hides* patches is not.
_write_fixture_workspace(tmp_path, lock_series=[])
with pytest.raises(RuntimePinError, match="series"):
load_runtime_pin(tmp_path)
# --- both identity implementations reject a moving runtime -----------------
def test_node_recipe_rejects_a_moving_runtime_version():
with pytest.raises(RecipeIdentityError, match="moving reference"):
_recipe(runtime_version="latest")
def test_tracker_rejects_a_moving_runtime_version():
vectors = json.loads(
(Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json").read_text(
encoding="utf-8"
)
)
doc = json.loads(json.dumps(vectors["vectors"][0]["identity"]))
doc["recipe"]["runtime_version"] = "latest"
doc.pop("fingerprint", None)
with pytest.raises(TrackerRecipeIdentityError, match="moving reference"):
parse_identity(doc)
@pytest.mark.parametrize(
"forged",
[
"llama.cpp@master+patchstack.not-a-digest",
"llama.cpp@e920c523+patchstack.forged",
"release-that-operator-typed",
],
)
def test_node_recipe_rejects_noncanonical_llama_runtime_pins(forged):
with pytest.raises(RecipeIdentityError, match="40-hex commit"):
_recipe(runtime_version=forged)
@pytest.mark.parametrize(
"forged",
[
"llama.cpp@master+patchstack.not-a-digest",
"llama.cpp@e920c523+patchstack.forged",
"release-that-operator-typed",
],
)
def test_tracker_rejects_noncanonical_llama_runtime_pins(forged):
vectors = json.loads(
(Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json").read_text(
encoding="utf-8"
)
)
doc = json.loads(json.dumps(vectors["vectors"][0]["identity"]))
doc["recipe"]["runtime_version"] = forged
doc.pop("fingerprint", None)
with pytest.raises(TrackerRecipeIdentityError, match="40-hex commit"):
parse_identity(doc)

View File

@@ -65,7 +65,7 @@ def _recipe(**changes: object) -> RuntimeRecipe:
"tokenizer_revision": "0123456789abcdef", "tokenizer_revision": "0123456789abcdef",
"architecture_adapter": "llama/range-v1", "architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp", "backend_id": "llama.cpp",
"runtime_version": "llama.cpp@deadbeef+meshnet.1", "runtime_version": "llama.cpp@" + "d" * 40 + "+patchstack." + "e" * 64,
"recipe_id": "example-gguf", "recipe_id": "example-gguf",
"recipe_version": "1", "recipe_version": "1",
"catalogue_version": "2026.07.1", "catalogue_version": "2026.07.1",
@@ -241,7 +241,7 @@ def test_committed_vectors_cover_a_whole_model_and_a_derivative_shard():
("tokenizer_revision", "fedcba9876543210"), ("tokenizer_revision", "fedcba9876543210"),
("architecture_adapter", "llama/range-v2"), ("architecture_adapter", "llama/range-v2"),
("backend_id", "other-backend"), ("backend_id", "other-backend"),
("runtime_version", "llama.cpp@other+meshnet.1"), ("runtime_version", "llama.cpp@" + "c" * 40 + "+patchstack." + "b" * 64),
("boundary_schema_version", 2), ("boundary_schema_version", 2),
("protocol_schema_version", 2), ("protocol_schema_version", 2),
], ],