feat: pin runtime identity to the exact llama.cpp patch stack (DGR-025)
Derive the recipe's runtime_version axis from the DGR-027 lock manifest (exact upstream commit + ordered patch-stack byte digest) in new meshnet_node.runtime_pin, failing closed on any lock/series/SHA256SUMS/ patch disagreement, and enforce pin discipline on runtime_version in both the node and tracker identity implementations. Also repair pre-existing backlog consistency: add missing DGR-022/DGR-027 completionNotes, regenerate the DGR-022/025/027 issue projections, and relocate three pre-DGR legacy GLM issue files to issues/legacy/. Mark DGR-025 passes=true with evidence at evidence/DGR-025/README.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Dobromir Popov
206
.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md
Normal file
206
.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md
Normal file
@@ -0,0 +1,206 @@
|
||||
# DGR-025 evidence — exact artifact and runtime recipe identity
|
||||
|
||||
**Completed:** 2026-07-17
|
||||
**Branch:** `ralph/fable-architecture-loop` (Claude Fable architecture lane)
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
**Dependencies:** DGR-018 (`evidence/DGR-018/README.md` — canonical backlog schema and
|
||||
issue projection), DGR-021 (`evidence/DGR-021/README.md` — versioned activation
|
||||
envelope). Both read before changing code.
|
||||
|
||||
## Objective
|
||||
|
||||
Ensure the tracker and worker only combine numerically and operationally
|
||||
compatible shards: fingerprint every axis that moves the numbers, bind shards to
|
||||
exact half-open ranges, fail closed on any mismatch, and keep uncertified
|
||||
recipes registered-but-dark.
|
||||
|
||||
## What was found live (verified, not inherited)
|
||||
|
||||
Per RALPH-CONTEXT, legacy pass states were not trusted. The DGR-003-lineage
|
||||
identity core was inspected and exercised live before any change:
|
||||
|
||||
- `packages/node/meshnet_node/runtime_recipe.py` — node-side identity:
|
||||
domain-separated digests (`meshnet.model-artifact.v1`,
|
||||
`meshnet.runtime-recipe.v1`, `meshnet.shard-binding.v1`) over the source
|
||||
artifact SHA (`source_digest`, with split artifacts bound to their exact
|
||||
source via `DerivativeBinding`), tokenizer revision (pin-enforced),
|
||||
architecture adapter + architecture/config digest, boundary and protocol
|
||||
schema versions, backend, weight quantization, activation/compute dtypes, and
|
||||
KV dtype/layout (`RECIPE_AXES`). Shard ranges are half-open
|
||||
(`shard_start`/`shard_end`, end-exclusive, protocol convention) with no
|
||||
topology or quant constants anywhere; `check_route` accepts any tiling of
|
||||
`[0, layer_count)`. Route, handshake (`check_handshake`), and session-open
|
||||
(`check_session_open`) checks fail closed with structured `RouteMismatch`
|
||||
reasons mapped to specific protocol error codes (`handshake_error`).
|
||||
- `packages/tracker/meshnet_tracker/recipe.py` — deliberately independent
|
||||
tracker re-derivation (no `meshnet_node` import); declared fingerprints are
|
||||
recomputed, never trusted (`parse_identity`, `FingerprintMismatch`). The
|
||||
`CertificationLedger` keeps every registered recipe dark until a real
|
||||
distributed forward — at least 2 distinct nodes, whole-model coverage,
|
||||
non-synthetic, tokens actually generated — certifies it; dark recipes may
|
||||
route only to certify.
|
||||
- The two implementations are pinned by committed conformance vectors
|
||||
(`tests/data/recipe_fingerprint_vectors.json`).
|
||||
|
||||
Live verification of that pre-existing core before changes:
|
||||
`PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_recipe_identity.py`
|
||||
→ `45 passed`; plus `tests/test_native_identity_emission.py`,
|
||||
`tests/test_tracker_capability_admission.py`, `tests/test_node_admission.py`
|
||||
→ `59 passed`.
|
||||
|
||||
## Gap found and closed (this story's change)
|
||||
|
||||
**The `runtime_version` recipe axis was a label, not a pin.** It was an opaque
|
||||
caller-supplied string: nothing derived it from the DGR-027 lock manifest, and
|
||||
neither identity implementation rejected a moving reference (`"latest"` was
|
||||
accepted), so two workers could run different llama.cpp pins or patch stacks
|
||||
under one label and still agree on the recipe digest. The acceptance criterion
|
||||
explicitly requires fingerprinting the "runtime pin/patch stack".
|
||||
|
||||
### Changed files
|
||||
|
||||
- `packages/node/meshnet_node/runtime_pin.py` (new) — derives the canonical
|
||||
`runtime_version` axis value from the DGR-027 lock workspace
|
||||
(`packages/node/native/llama`):
|
||||
`<runtime>@<40-hex upstream commit>+patchstack.<sha256>` where the stack
|
||||
digest commits, under the `meshnet.runtime-patch-stack.v1` domain, to the
|
||||
*ordered* `(patch name, patch bytes sha256)` stack. Fails closed on: missing
|
||||
or malformed `UPSTREAM_LOCK.json`, unknown schema version, non-40-hex/moving
|
||||
commit, `UPSTREAM_COMMIT` disagreement, any disagreement among the lock's
|
||||
`patch_series`, `patches/series`, and `patches/SHA256SUMS`, a missing patch
|
||||
file, or a patch whose bytes don't match their recorded digest. Reads the
|
||||
committed manifest only; fetching/patching stays with
|
||||
`scripts/llama_cpp_dependency.py` (DGR-027).
|
||||
- `packages/node/meshnet_node/runtime_recipe.py` — `runtime_version` is now
|
||||
pin-enforced (`_require_pin`) exactly like `tokenizer_revision`; docstring
|
||||
points at the canonical derivation.
|
||||
- `packages/tracker/meshnet_tracker/recipe.py` — the independent tracker
|
||||
implementation applies the same pin rule in `parse_identity`, keeping the two
|
||||
implementations in step.
|
||||
- `tests/test_runtime_pin_identity.py` (new, TDD — written first and observed
|
||||
failing) — 17 deterministic tests: the committed manifest derives a
|
||||
deterministic pin whose axis value is a valid recipe pin; a changed patch
|
||||
byte and a reordered stack each change the runtime identity; every manifest
|
||||
disagreement above fails closed; and both node and tracker reject a moving
|
||||
`runtime_version`.
|
||||
|
||||
### Backlog-consistency repair (pre-existing damage, honestly recorded)
|
||||
|
||||
`tests/test_ralph_prd_schema.py` had 4 pre-existing failures before this story
|
||||
touched anything, left by prior sessions and the alternate-history merge:
|
||||
|
||||
- DGR-022 and DGR-027 were marked `passes: true` without `completionNotes` and
|
||||
without regenerated issue projections. Added their `completionNotes`
|
||||
(explicitly labeled as added during this repair, content drawn from their own
|
||||
evidence READMEs) and regenerated
|
||||
`issues/022-…` / `issues/027-…` via `scripts/ralph_prd_schema.py render`.
|
||||
- Three pre-DGR legacy GLM alpha issue files (`18-…`, `19-…`, `20-…`,
|
||||
committed 2026-07-14, before DGR-018 established the generated-only
|
||||
convention; they carry no authority disclaimer because they are *not*
|
||||
generated from prd.json) were relocated via `git mv` to
|
||||
`issues/legacy/` — preserved as provenance, out of the generated namespace.
|
||||
|
||||
### prd.json
|
||||
|
||||
Marked `DGR-025.passes = true` with `completionNotes`; regenerated
|
||||
`issues/025-define-exact-artifact-and-runtime-recipe-identity.md`.
|
||||
|
||||
## Acceptance criteria → evidence
|
||||
|
||||
1. **Fingerprint all axes** — `RECIPE_AXES` + `ArtifactIdentity` cover source
|
||||
artifact SHA, tokenizer revision, architecture adapter/version (adapter axis
|
||||
+ architecture/config digest), boundary schema (boundary + protocol schema
|
||||
versions), backend, quant, activation/compute dtype, KV/state layout; the
|
||||
runtime pin/patch stack is now committed via the derived `runtime_version`
|
||||
axis (`runtime_pin.py`). Verified by `test_runtime_recipe_identity.py` and
|
||||
`test_runtime_pin_identity.py`.
|
||||
2. **Exact half-open range, no hardcoded topology/quant** — `ShardIdentity`
|
||||
end-exclusive ranges, `DerivativeBinding` coverage checks, `check_route`
|
||||
tiling over arbitrary layouts; quant/dtype values are open strings
|
||||
(dynamic recipe inputs). Verified by `test_runtime_recipe_identity.py`
|
||||
(routes of 1, 2, and 5 shards; no product constants).
|
||||
3. **Fail closed on any mismatch** — artifact, adapter, boundary/schema, cache
|
||||
layout, backend, and runtime mismatches each produce structured
|
||||
`RouteMismatch` reasons and protocol error codes; the tracker recomputes
|
||||
digests and rejects inconsistent claims; moving runtime references are now
|
||||
rejected on both sides.
|
||||
4. **Registered-but-dark** — `CertificationLedger`: unknown recipes cannot be
|
||||
certified, registered recipes are dark, only a real ≥2-distinct-node
|
||||
whole-model non-synthetic forward promotes; verified by
|
||||
`test_runtime_recipe_identity.py` / `test_tracker_capability_admission.py`.
|
||||
5. **Gates + this handoff** — below.
|
||||
|
||||
## Commands and results
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_runtime_pin_identity.py
|
||||
```
|
||||
```text
|
||||
17 passed in 0.11s
|
||||
```
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
|
||||
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py \
|
||||
tests/test_native_identity_emission.py tests/test_tracker_capability_admission.py \
|
||||
tests/test_node_admission.py tests/test_node_capability.py tests/test_recipe_benchmark.py
|
||||
```
|
||||
```text
|
||||
196 passed, 1 warning in 5.35s
|
||||
```
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q tests/test_ralph_prd_schema.py
|
||||
```
|
||||
```text
|
||||
108 passed
|
||||
```
|
||||
(4 failed before this story's backlog repair; 0 after.)
|
||||
|
||||
```bash
|
||||
python3 -m compileall -q packages tests # exit 0
|
||||
git diff --check # exit 0
|
||||
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||
# OK: 55 stories validated.
|
||||
```
|
||||
|
||||
Default tests are model-download-free, API-credit-free, and GPU-free; no model
|
||||
artifact was touched and nothing was written under `/home`.
|
||||
|
||||
## Limitations
|
||||
|
||||
- `runtime_pin.py` proves what the *manifest* pins; it does not prove the
|
||||
running binary was built from that manifest. Binding the built native worker
|
||||
to the pin it reports (e.g. embedding the patched-tree hash at build time and
|
||||
echoing it through the DGR-022 status contract) belongs with the native
|
||||
worker stories (DGR-028+/DGR-031); until then `runtime_version` is exactly as
|
||||
trustworthy as the rest of the declared axes — a claim the tracker digests,
|
||||
with real distributed certification as the trust boundary (unchanged design).
|
||||
- The DGR-027-recorded blocker stands: `0002-dense-llama-owned-range-loader.patch`
|
||||
does not apply cleanly against the pin (DGR-028). That does not affect this
|
||||
story: the identity commits to the patch *bytes as committed*, which is
|
||||
precisely what makes a later repaired patch a *different* runtime identity.
|
||||
- No native/CMake change was made, so the native build/CTest gate is not
|
||||
applicable; no llama.cpp patch content was changed, so apply/check/reverse
|
||||
verification is not applicable (and is blocked by the DGR-028 defect anyway).
|
||||
- Tracker routing, load balancing, billing, telemetry, and relay semantics are
|
||||
untouched; the only behavior change outside the new module is the stricter
|
||||
(fail-closed) rejection of moving `runtime_version` values.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- **DGR-026** (split-GGUF provisioning): bind each provisioned split via
|
||||
`DerivativeBinding` to the exact source digest recorded in its hashed
|
||||
manifest; the per-split `shard_binding_digest` is what certification pins.
|
||||
- **DGR-031** (`ShardEngine`): construct worker identity through
|
||||
`shard_identity_from_native_report` and populate `runtime_version` from
|
||||
`meshnet_node.runtime_pin.load_runtime_pin().runtime_version` — never from an
|
||||
operator string. A build-time echo of the patched-tree hash through the
|
||||
status contract would close the manifest-vs-binary gap noted above.
|
||||
- **DGR-041** (capability registration): the tracker already re-derives and
|
||||
fail-closes on presented identities (`parse_identity`); register recipes
|
||||
through the `CertificationLedger` so they arrive dark.
|
||||
- **DGR-044** (DeepSeek V4 Flash target): pin the target's artifact identity
|
||||
the same way `glm_alpha_artifact` does — read locked manifests, never restate
|
||||
digests — and note `layer_count` must count the routed transformer stack the
|
||||
route tiles, excluding MTP (reserved for beta).
|
||||
Reference in New Issue
Block a user