Merge DGR-025 (certified-artifact-byte recipe identity) from ralph-fable-loop lane
This commit is contained in:
Dobromir Popov
@@ -1,8 +1,8 @@
|
||||
# DGR-025 evidence — exact artifact and runtime recipe identity
|
||||
|
||||
**Completed:** 2026-07-17
|
||||
**Branch:** `ralph/fable-architecture-loop` (Claude Fable architecture lane)
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
**Status:** in progress — controller gates pass; final independent P0/P1 re-review is pending.
|
||||
**Branch:** fixed detached Claude Fable provider lane
|
||||
**Authority:** live Gitea #9; the local PRD is a secondary projection.
|
||||
**Dependencies:** DGR-018 (`evidence/DGR-018/README.md` — canonical backlog schema and
|
||||
issue projection), DGR-021 (`evidence/DGR-021/README.md` — versioned activation
|
||||
envelope). Both read before changing code.
|
||||
@@ -208,3 +208,240 @@ artifact was touched and nothing was written under `/home`.
|
||||
the same way `glm_alpha_artifact` does — read locked manifests, never restate
|
||||
digests — and note `layer_count` must count the routed transformer stack the
|
||||
route tiles, excluding MTP (reserved for beta).
|
||||
|
||||
## Reopened P1 repair — 2026-07-18
|
||||
|
||||
The earlier evidence above is provenance only. Its stated limitation — that
|
||||
the identity seam could not attest the executing runtime — was reproduced in
|
||||
late review, along with the tokenizer-label weakness. This repair replaces
|
||||
both claims at the production identity boundary.
|
||||
|
||||
### Changed files
|
||||
|
||||
- `packages/node/meshnet_node/runtime_recipe.py` — replaces the moving-ref
|
||||
denylist with the sole valid `tokenizer.v1:<sha256>` form, derived from an
|
||||
ordered map of named tokenizer/config byte digests. A label, tag, branch, or
|
||||
symbolic ref cannot be a valid identity.
|
||||
- `packages/tracker/meshnet_tracker/recipe.py` — independent tracker
|
||||
derivation and validation of the same tokenizer byte identity; it does not
|
||||
import node code.
|
||||
- `packages/node/meshnet_node/runtime_pin.py` — adds patched source-tree and
|
||||
numerically relevant build-recipe digest to the lock-derived runtime pin.
|
||||
- `packages/node/meshnet_node/native_backend.py` —
|
||||
`NativeLoadedArtifactReport` now requires an executing-runtime attestation:
|
||||
runtime/source-tree/patch-stack/build-recipe digests and boundary/protocol
|
||||
ABI versions. `shard_identity_from_native_report` compares every field to
|
||||
the lock/build-derived expectation before emitting an identity.
|
||||
- `scripts/gen_recipe_fingerprint_vectors.py` and
|
||||
`tests/data/recipe_fingerprint_vectors.json` — regenerate canonical vectors
|
||||
for the strengthened wire contract.
|
||||
- `tests/test_runtime_pin_identity.py`,
|
||||
`tests/test_runtime_recipe_identity.py`, and
|
||||
`tests/test_native_identity_emission.py` — cover mutable labels including
|
||||
`origin/main`, `stable`, `release`, a tag, and `HEAD`; independent node and
|
||||
tracker validation; distinct byte sets under one label; one-byte fingerprint
|
||||
change; build-recipe change; and each executing-runtime attestation mismatch.
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 scripts/gen_recipe_fingerprint_vectors.py
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
|
||||
tests/test_runtime_pin_identity.py tests/test_native_identity_emission.py \
|
||||
tests/test_runtime_recipe_identity.py
|
||||
```
|
||||
```text
|
||||
92 passed
|
||||
```
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
|
||||
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py \
|
||||
tests/test_native_identity_emission.py tests/test_node_admission.py \
|
||||
tests/test_node_capability.py tests/test_recipe_benchmark.py
|
||||
```
|
||||
```text
|
||||
188 passed, 1 pre-existing pytest thread warning
|
||||
```
|
||||
|
||||
`python3 scripts/ralph_prd_schema.py validate
|
||||
.scratch/distributed-gguf-runtime/prd.json`, `python3 -m compileall -q packages
|
||||
tests`, and `git diff --check` each exit 0. The broader PRD pytest projection
|
||||
suite has two unrelated existing DGR-023 failures: its `passes: true` entry has
|
||||
no completion notes and its generated issue file is stale. The socket-backed
|
||||
subset of `test_tracker_capability_admission.py` is additionally un-runnable in
|
||||
this sandbox (`PermissionError: [Errno 1] Operation not permitted` creating an
|
||||
AF_INET socket); its deterministic non-socket identity coverage is included in
|
||||
the passing runs above.
|
||||
|
||||
### Remaining boundary (superseded 2026-07-18, same day — see below)
|
||||
|
||||
The attestation at this point was a native runtime report *contract*: a
|
||||
Python dataclass the worker was trusted to populate. Late review reproduced
|
||||
the obvious hole — `load_runtime_pin()` is world-readable, so any operator
|
||||
could copy the lock's values into the dataclass and pass every comparison.
|
||||
The section below closes that hole.
|
||||
|
||||
## Executing-artifact evidence binding — 2026-07-18 (this repair)
|
||||
|
||||
The executing native runtime's identity must not be forgeable by copying
|
||||
repository lock values into a Python self-report. Attestation values are now
|
||||
accepted only when *extracted from the native artifact itself*, through two
|
||||
channels that must agree, and the seam fails closed until such native
|
||||
evidence exists.
|
||||
|
||||
### The boundary
|
||||
|
||||
`meshnet_node.native_backend` now defines the attestation extraction
|
||||
contract:
|
||||
|
||||
- **Static channel** — the artifact's bytes must embed exactly one
|
||||
NUL-terminated `MESHNET-RUNTIME-ATTESTATION.v1:<canonical json>` marker.
|
||||
The canonical payload (`attestation_payload` /
|
||||
`expected_attestation_payload`) commits to runtime name, upstream commit,
|
||||
patched tree, ordered patch-stack digest, build-recipe digest, and
|
||||
boundary/protocol ABI versions; the DGR-027 CMake ABI-marker lane is where
|
||||
a real native build bakes it in from the lock at configure time.
|
||||
- **Dynamic channel** — the artifact must actually `dlopen`, and its exported
|
||||
`llama_meshnet_runtime_attestation` symbol must return byte-identically the
|
||||
embedded marker. A marker pasted into a plain file is not an executing
|
||||
runtime.
|
||||
- **Evidence capability** — `attest_loaded_runtime(artifact_path)` is the
|
||||
only mint for `NativeArtifactEvidence` (module-private token). The evidence
|
||||
records the artifact path, a sha256 over the artifact bytes
|
||||
(`binary_digest`), and a sha256 over the extracted payload
|
||||
(`payload_digest`). `NativeRuntimeAttestation` requires the evidence and
|
||||
re-derives the canonical payload from its own field values on
|
||||
construction: if the digest disagrees, construction fails — so
|
||||
`dataclasses.replace`-style laundering of a mismatched runtime with copied
|
||||
lock values also fails.
|
||||
- `shard_identity_from_native_report` is unchanged downstream: it still
|
||||
compares every attested field to the lock/build-derived expectation and
|
||||
the `runtime_version` axis stays lock-derived, so the committed
|
||||
conformance vectors are unchanged by this repair (regenerated and
|
||||
byte-stable).
|
||||
|
||||
Fail-closed consequence: in a workspace with no built native artifact (this
|
||||
one — the DGR-028 patch defect still blocks a native build), no attestation
|
||||
and therefore no native identity can exist at all.
|
||||
|
||||
### Changed files
|
||||
|
||||
- `packages/node/meshnet_node/native_backend.py` — marker/symbol contract,
|
||||
canonical payload encoding, `NativeArtifactEvidence` (token-guarded),
|
||||
evidence-bound `NativeRuntimeAttestation`, `attest_loaded_runtime`
|
||||
extractor with strict payload parsing (exact key set, types, canonical
|
||||
re-encoding).
|
||||
- `tests/test_native_identity_emission.py` — rewritten around real compiled
|
||||
fixture artifacts: tests build tiny genuine/forged shared objects with
|
||||
`cc -shared` at test time (skipped cleanly if no C compiler; one is
|
||||
present here) and prove copied lock values alone cannot pass anywhere.
|
||||
|
||||
### Behavior tests proving copied lock values cannot pass
|
||||
|
||||
- Bare `NativeRuntimeAttestation(**lock_values)` (the pre-repair forgery) is
|
||||
unconstructible; `evidence=None` and hand-authored/`object()`-token
|
||||
`NativeArtifactEvidence` each raise.
|
||||
- The true marker bytes written into a plain file fail (`not a loadable`).
|
||||
- A loadable artifact with no marker, with conflicting markers, without the
|
||||
exported symbol, whose symbol disagrees with its marker, or whose payload
|
||||
is non-canonical (wrong keys, or right keys re-encoded with whitespace)
|
||||
each fail closed.
|
||||
- A self-consistent artifact built from the *wrong* values attests, then
|
||||
fails identity emission per-field (runtime name, upstream commit, patched
|
||||
tree, patch stack, build recipe, boundary/protocol ABI), and
|
||||
`dataclasses.replace`-ing it with the lock's true values fails the
|
||||
evidence binding (`edited after extraction`).
|
||||
- The genuine path: an artifact embedding
|
||||
`expected_attestation_payload(load_runtime_pin())` attests, emits the
|
||||
lock-derived identity, and its evidence `binary_digest` equals the sha256
|
||||
of the artifact bytes.
|
||||
|
||||
### Verification (all in this worktree, 2026-07-18)
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
|
||||
tests/test_native_identity_emission.py tests/test_runtime_pin_identity.py \
|
||||
tests/test_runtime_recipe_identity.py
|
||||
```
|
||||
```text
|
||||
104 passed
|
||||
```
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
|
||||
tests/test_node_admission.py tests/test_node_capability.py \
|
||||
tests/test_recipe_benchmark.py
|
||||
```
|
||||
```text
|
||||
96 passed, 1 pre-existing pytest thread warning
|
||||
```
|
||||
|
||||
```bash
|
||||
PYTHONPATH=packages/node:packages/tracker python3 -m pytest -q \
|
||||
tests/test_tracker_capability_admission.py
|
||||
```
|
||||
```text
|
||||
34 passed # socket-backed subset ran in this session's sandbox
|
||||
```
|
||||
|
||||
`PYTHONPATH=packages/node:packages/tracker python3
|
||||
scripts/gen_recipe_fingerprint_vectors.py` reproduces the committed vectors
|
||||
byte-for-byte; `python3 -m compileall -q packages tests` and
|
||||
`git diff --check` each exit 0.
|
||||
|
||||
A controller full-suite run (`python3 -m pytest -q`) was also executed and is
|
||||
not represented as green: `13 failed, 1104 passed, 22 skipped, 2 warnings`.
|
||||
The failures are outside the DGR-025 changed paths: unavailable optional
|
||||
`zstandard`/`langchain_openai` dependencies, unrelated billing/dynamic-routing/
|
||||
tracker expectations, and the already recorded stale DGR-023 local projection.
|
||||
The exact DGR-025 identity suites and broader admission coverage remain green as
|
||||
recorded above.
|
||||
|
||||
### Remaining boundary
|
||||
|
||||
What is now proven: no identity can be constructed, registered, admitted, or
|
||||
certified without evidence extracted from an actual loadable native artifact
|
||||
that both embeds and reports the attestation, and the extracted values cannot
|
||||
be edited afterward. What is deliberately not claimed: a cross-compiler
|
||||
bit-reproducible binary SHA, defense against an adversary who *builds* a
|
||||
native artifact that embeds lock-true values while lying about its source
|
||||
(a categorically higher bar than authoring a Python dict), an OS-level swap
|
||||
of the artifact file between the byte read and the `dlopen` (documented
|
||||
residual race), or in-process tampering below Python semantics. Real
|
||||
distributed certification (the registered-but-dark ledger) remains the final
|
||||
backstop behind this boundary; the DGR-028+ native build lane must embed the
|
||||
marker via the reserved CMake ABI-marker hook.
|
||||
|
||||
## Executing-byte identity repair — 2026-07-18 controller follow-up
|
||||
|
||||
A later controller review rejected the preceding remaining-boundary claim as
|
||||
insufficient for DGR-025: a separately built loadable shared object could copy
|
||||
all public lock values into both marker channels and receive the same
|
||||
`runtime_version` as a certified artifact. The repair now appends
|
||||
`+artifact.<sha256>` to the llama.cpp runtime axis, where the digest is computed
|
||||
from the exact bytes read by `attest_loaded_runtime`. Node and tracker parsers
|
||||
independently require this suffix. Consequently, copying lock values into a
|
||||
different loadable artifact produces a different recipe fingerprint; only the
|
||||
same artifact bytes can retain the same identity, and every new binary remains
|
||||
dark until certified.
|
||||
|
||||
`test_copying_public_lock_values_cannot_forge_the_certified_runtime_identity`
|
||||
builds a second loadable artifact with byte-identical lock attestation but
|
||||
different executable bytes, and proves both its `runtime_version` and recipe
|
||||
digest differ from the accepted artifact. Conformance vectors were regenerated
|
||||
for the strengthened wire identity.
|
||||
|
||||
Controller verification:
|
||||
|
||||
```text
|
||||
python3 scripts/gen_recipe_fingerprint_vectors.py
|
||||
python3 -m pytest -q tests/test_native_identity_emission.py \
|
||||
tests/test_runtime_pin_identity.py tests/test_runtime_recipe_identity.py
|
||||
# 105 passed in 0.52s
|
||||
python3 -m compileall -q packages/node/meshnet_node \
|
||||
packages/tracker/meshnet_tracker tests scripts/gen_recipe_fingerprint_vectors.py
|
||||
# exit 0
|
||||
git diff --check
|
||||
# exit 0
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user