9 Commits

Author SHA1 Message Date
Dobromir Popov
31065c0e12 feat: distributed GGUF shard load integration test with TinyLlama 1.1B 2026-07-14 13:01:51 +03:00
Dobromir Popov
ec36290863 feat: emit native DGR-003 shard identity 2026-07-14 11:31:10 +03:00
Dobromir Popov
f844ae6567 feat: DGR-005B endpoint ownership and graph guard 2026-07-14 11:14:35 +03:00
Dobromir Popov
252d131e7d feat: DGR-005A dense Llama owned range loader 2026-07-14 11:01:28 +03:00
Dobromir Popov
3d8f93f4aa feat: DGR-005-003-CHAIN - DGR-005 + DGR-003-emission + anchor 2026-07-14 10:48:59 +03:00
Dobromir Popov
f9722e7b57 feat: DGR-004-CHAIN - Execute chained DGR-004/005/003-emission with anchor 2026-07-14 10:34:38 +03:00
Dobromir Popov
7b8e467c6b fix: harden DGR-003 identity trust boundary 2026-07-14 09:48:42 +03:00
Dobromir Popov
7364ed6731 fix: harden DGR-017 contract continuity 2026-07-14 01:10:33 +03:00
Dobromir Popov
ad2d17541c feat: DGR-003 - Define exact Artifact and runtime recipe identity 2026-07-14 01:10:07 +03:00
38 changed files with 4934 additions and 130 deletions

View File

@@ -9,3 +9,4 @@
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.
- **DGR-004 llama.cpp boundary** — `packages/node/native/llama/` locks `e920c523e3b8a0163fe498af5bf90df35ff51d25`, with a one-patch CMake marker and fail-closed clean materialize/apply/build/smoke harness. This is infrastructure only; stock GLM dense fallback remains uncertified.

View File

@@ -0,0 +1,186 @@
# DGR-003 — exact Artifact and runtime recipe identity
Evidence class: deterministic offline/unit. No model payload, GPU, external API,
network node, or API credit is required or claimed.
## Result — delayed-review repair, 2026-07-14
DGR-003 defines and tests an exact, model-agnostic compatibility identity and
connects it to DGR-002's gRPC `Fingerprint` plus tracker parsing, admission,
route partitioning, and certification. It is **not complete**: the existing
production doctor/backend path still emits the legacy capability report without
constructing a `ShardIdentity` from authoritative loaded artifact/runtime state.
No exact recipe is therefore claimed live or routable from that path; supplied
exact identities remain dark until tracker-owned certification.
A matching digest proves canonical consistency, **not node authenticity or real
execution**. Tracker-owned certification of a fingerprint by a non-synthetic,
complete, multi-node distributed forward is the execution trust boundary.
## Implementation
- `ArtifactIdentity` binds artifact ID/revision, exact content digest,
architecture/config digest, layer count, and optional derivative binding.
- `DerivativeBinding` binds a split artifact to the exact source artifact digest
and its end-exclusive layer range. A Shard cannot advertise outside that range.
- `RuntimeRecipe` keeps these canonical axes separate rather than hiding them in
a backend label:
- weight quantization;
- activation and compute dtypes;
- KV dtype and layout;
- tokenizer revision;
- architecture adapter;
- backend and runtime version;
- boundary and protocol schema versions;
- recipe ID/version and catalogue version.
- `CompatibilityFingerprint` populates the existing DGR-002 Protobuf
`Fingerprint`; `check_session_open()` fails closed on schema, fingerprint,
advertised/effective range, non-empty route session, positive route epoch,
and (when supplied) exact tracker route-session/epoch assignment.
- Node and tracker implementations independently canonicalize the declaration.
This is intentional: the tracker must not trust a digest copied from a node,
and future native/C++ workers also need an independent implementation. Their
behavior is pinned by `tests/data/recipe_fingerprint_vectors.json`.
- Tracker admission cross-checks the exact identity against the capability
proof's model, range, recipe labels, backend, and weight quantization. Any
disagreement fails closed.
- `TrackerServer` owns the sole live certification ledger and passes it through
direct and replicated registration paths. A known exact recipe is
`uncertified` and dark for user traffic until the same exact fingerprint is
certified. Restart fails closed; durable/cluster-wide certification events
require the later real-forward control path and are not claimed here.
- Certification evidence is bound to the promoted fingerprint, requires at
least two distinct nodes, complete layer coverage, generated tokens, and
`synthetic=false`. Unknown or mismatched fingerprints cannot be promoted.
## Files changed
- `packages/node/meshnet_node/runtime_recipe.py`
- `packages/tracker/meshnet_tracker/recipe.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/data/recipe_fingerprint_vectors.json`
- `tests/test_runtime_recipe_identity.py`
- this evidence directory, issue state, and DGR-003 PRD state
A late review of dependency DGR-017 also found and fixed two genuine contract
continuity defects during delayed DGR-003 review: v1 now has an independently
trusted digest and recursively immutable parsed state. Those changes and tests
are recorded in DGR-017 evidence rather than claimed as DGR-003 functionality.
## Verification
Exact commands and outcomes are in `commands.txt`.
Observed final results:
- DGR-003 identity + node/tracker capability suites: **126 passed**.
- DGR-017 focused dependency repair suite: **99 passed**.
- Tracker routing suite: **93 passed**.
- First delayed-review integrated run: **898 passed, 13 skipped, 1 failed** on
the pre-existing tracker-cancellation race.
- Final delayed-review integrated rerun: **899 passed, 13 skipped** in
**253.64s**; Hermes controller acceptance rerun: **899 passed, 13 skipped**
in **252.66s**.
- `python -m compileall -q packages tests`: pass.
- `git diff --check`: pass.
- Ruff on the changed identity, capability, contract, and test modules: pass.
- `server.py` has 8 pre-existing Ruff findings at both pushed baseline and the
current tree; DGR-003 added no finding.
The first integrated full-suite run produced **871 passed, 13 skipped, 1 failed**
on the known unrelated
`test_tracker_dashboard_can_cancel_inflight_proxy` timing race. Its fixture
completed after three seconds just before cancellation, so the cancel endpoint
returned 404. In this delayed repair it again produced a 404 after the stream
finished (first integrated run: **898 passed, 13 skipped, 1 failed**); three
immediate isolated repeats passed before a fourth reproduced the same race.
No cancellation-test code was changed. The final complete integrated rerun
passed **899/899** tests.
## Limitations
- Certification state is process-local in this story. The same running tracker
reuses it across registrations, but durable/cluster-wide certification-event
persistence belongs with the later real distributed-forward control path.
Restart or failover therefore returns exact recipes to the safe dark state;
it never makes an unsupported recipe routable.
- The node module has no certification ledger or admission policy; it holds only
identity construction and handshake validation. The Tracker is the sole
promotion authority.
- **Completion blocker:** `doctor._validate_recipe()` calls
`build_capability_report()` without `identity=`, because the legacy
Transformers backend does not expose an immutable artifact-content pin and
full runtime recipe axes authoritative enough to build one. Adding a guessed
identity would weaken this contract. Production emission must be added with
the authoritative native worker/backend loading seam; until then the issue and
PRD deliberately remain incomplete.
- This story proves identity and admission behavior with deterministic fixtures.
It does not claim a real GLM forward or hardware certification.
## Compatibility
- Capability report identity is additive. Legacy reports without the new block
retain ADR-0023's explicit compatibility-policy behavior.
- Reports that opt into exact identity are held to it and fail closed on malformed,
inconsistent, unknown, dark, or mismatched declarations.
- No new wire identity was invented; DGR-002's `Fingerprint` remains the gRPC
representation.
## Handoff
DGR-004 and native workers must build `ShardIdentity` from the actual immutable
artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout,
schema versions, and owned range. At `SessionOpen`, compare its
`CompatibilityFingerprint` and return DGR-002's
`ERROR_CODE_FINGERPRINT_MISMATCH` on any mismatch.
A digest match is not certification. Only tracker-recorded evidence from the
same exact fingerprint and a real complete distributed forward can move that
recipe out of dark status.
## Native emission closure — 2026-07-14
Status: **done**. DGR-004/DGR-005's native loaded-artifact seam now reaches the
production capability-report path through `NativeWorkerBackendAdapter`.
### Files changed
- `packages/node/meshnet_node/native_backend.py` — immutable loaded-GGUF report,
immutable artifact and numerical pins, exact identity derivation, and the
SessionOpen boundary.
- `packages/node/meshnet_node/doctor.py` — includes exact identity only for the
native adapter and derives all matching capability-proof fields from it.
- `tests/test_native_identity_emission.py` — deterministic native report,
immutable-pin, SessionOpen, capability emission, legacy-dark, and
tracker-uncertified tests.
- This issue, `prd.json`, and this evidence directory.
### Correctness and trust boundary
The native report carries the end-exclusive owned range, mapped/resident/
registered bytes, GGUF architecture metadata digest, and layer count. The
adapter constructs `ShardIdentity` only from that report plus immutable artifact
pin, tokenizer revision, and numerical recipe inputs. It does not accept a
caller-supplied shard range.
`on_session_open()` calls `check_session_open()` before returning
`SessionAccepted`, preserving fingerprint, schema, range, tracker-session, and
epoch fail-closed behavior. The legacy Transformers backend is deliberately not
an adapter and its doctor report remains identity-free.
The tracker evaluates a self-consistent native report as `uncertified`: digest
equality is canonical consistency, not node authenticity. Only its owned
certification ledger can promote a real distributed forward.
### Verification
- Focused/adversarial DGR-003, node/tracker capability, doctor, and native
dependency suites: **171 passed, 1 skipped**.
- Native protocol CMake configure/build plus CTest: **1/1 passed**.
- `compileall`, Ruff, and `git diff --check`: pass.
- Full deterministic suite: **902 passed, 13 skipped** (255.01s).
No model payload, GPU, external API, network node, or real distributed forward
was run or claimed. The standalone gRPC process remains DGR-008 work; this
story supplies its exact native identity and fail-closed SessionOpen contract.

View File

@@ -0,0 +1,108 @@
# DGR-003 final verification — 2026-07-14
# Native emission closure — 2026-07-14
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_native_identity_emission.py tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py tests/test_node_doctor.py tests/test_llama_cpp_dependency.py
# result: 171 passed, 1 skipped in 7.07s
ruff check packages/node/meshnet_node/native_backend.py packages/node/meshnet_node/doctor.py tests/test_native_identity_emission.py
# result: All checks passed
git diff --check
# result: pass
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall packages tests
# result: pass
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake -S packages/node/native -B build/dgr-003-native-protocol -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake --build build/dgr-003-native-protocol -j2
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/ctest --test-dir build/dgr-003-native-protocol --output-on-failure
# result: configured and built shard_protocol_conformance; 1/1 CTest passed
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 902 passed, 13 skipped in 255.01s
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
# result: 99 passed in 4.76s
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
# result: 99 passed in 0.15s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# first integrated result: 871 passed, 13 skipped, 1 failed in 258.18s
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# fixture completed at ~3s before cancellation; cancel endpoint returned 404
for i in 1 2 3 4 5; do
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
done
# result: 5/5 passed (1.14s, 1.14s, 1.26s, 1.14s, 1.64s)
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# final integrated result: 872 passed, 13 skipped in 253.46s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
# result: pass
git diff --check
# result: pass
ruff check packages/node/meshnet_node/glm_alpha/contract.py packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_glm_alpha_target.py tests/test_runtime_recipe_identity.py
# result: All checks passed!
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
ruff check /tmp/dgr003-server-base.py
ruff check packages/tracker/meshnet_tracker/server.py
# result: both baseline and current server.py report the same 8 pre-existing findings
# ---------------------------------------------------------------------------
# Delayed-review repair continuation — 2026-07-14
# No model payload, GPU, external API, or real inference was run.
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
# result: 126 passed in 4.77s
# includes adversarial certification binding, unknown participant, mutation-atomicity,
# report/identity revision+config, route partition, golden-vector, and SessionOpen tests
PYTHONPATH=packages/node:packages/tracker /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python scripts/gen_recipe_fingerprint_vectors.py --check
# result: tests/data/recipe_fingerprint_vectors.json matches the identity implementation
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
# result: 99 passed in 0.11s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py
# result: 93 passed in 46.83s
# there is no separate tests/test_tracker_server.py in this repository
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
# result: pass
ruff check packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_runtime_recipe_identity.py scripts/gen_recipe_fingerprint_vectors.py
# result: All checks passed!
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
ruff check /tmp/dgr003-server-base.py
ruff check packages/tracker/meshnet_tracker/server.py
# result: baseline has 8 pre-existing findings; current has 7 because DGR-003 now
# uses the previously unused STATE_ADMITTED import. No new server.py finding.
git diff --check
# result: pass
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 898 passed, 13 skipped, 1 failed in 255.43s
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# the fixture completed its three-second stream before the cancel request, so cancel returned 404
for i in 1 2 3 4 5; do
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy || exit 1
done
# result: first 3 passed (1.16s, 1.65s, 1.64s); attempt 4 reproduced the same 404 race.
# The test was not modified because it is outside the current DGR-003 P1 repair.
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 899 passed, 13 skipped in 253.64s (0:04:13)
# Hermes controller acceptance rerun after agent completion
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# result: 899 passed, 13 skipped in 252.66s (0:04:12)

View File

@@ -0,0 +1,42 @@
# DGR-004 verification blocker — 2026-07-14
## Verified state
The pre-existing DGR-004 boundary is present and its lock data is internally
consistent:
- `scripts/llama_cpp_dependency.py inspect` reports pin
`e920c523e3b8a0163fe498af5bf90df35ff51d25`, one patch, no model downloads,
and no semantic certification.
- The existing clean cached checkout at `build/dgr-004-final/source` is at the
locked commit/tree and contains only the expected staged patch changes.
- The existing `llama-gguf-hash --help` smoke binary runs successfully.
- `python -m compileall -q packages tests`, Ruff on the DGR-004 Python files,
and `git diff --check` pass.
## Blocker
The verification environment no longer contains the `.venv` recorded in
`commands.txt`, nor a `cmake` executable on `PATH`. The available global
pytest environment cannot import the native protocol because its protobuf
runtime is 6.33.6 while the checked-in generated code requires 7.35.0. This
causes both `tests/test_llama_cpp_dependency.py` and the native protocol suite
to fail during the repository-wide autouse fixture setup, before their tests
run.
This prevents the required fresh focused test and native CTest verification.
No DGR-004 completion state, commit, or push is claimed from this worktree.
## Continuation
1. Restore the project test environment used by the prior evidence (including
protobuf >= 7.35.0 and CMake), without changing DGR-004 source files.
2. Run the exact focused test command from `commands.txt` and the clean
`reproduce` command using the local llama.cpp object cache.
3. Re-run compileall, Ruff, diff check, and the deterministic full suite.
4. Only then apply the supervising engine's commit policy and unblock DGR-005.
## Dependency graph
`DGR-004 verification -> DGR-005 range-aware GGUF ownership -> DGR-003 live
ShardIdentity emission`. DGR-005 and DGR-003-emission were not modified.

View File

@@ -0,0 +1,31 @@
# DGR-004 — Reproducible pinned llama.cpp patch stack
Status: **done**. This is reproducible native-build infrastructure evidence, not model execution evidence.
## Delivered boundary
- Pin: `ggml-org/llama.cpp` at `e920c523e3b8a0163fe498af5bf90df35ff51d25` (tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1`).
- Ordered patch: `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch`, SHA-256 `1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75`.
- The sole patch adds an interface-library CMake marker. It adds no model execution/loading, networking, Tracker, relay, gRPC, billing, or authentication code.
- `scripts/llama_cpp_dependency.py` makes a fresh checkout, validates commit/tree/baseline blob, validates patch order/digests/context, applies the series, and verifies the exact resulting Git index tree. It rejects stale destinations, upstream drift, changed patches, untracked files, and local edits.
## Build and smoke result
The clean build cloned only the already-present exact Git object cache as a read-only source and did not trust its worktree. CMake 4.4.0 and GCC 15.2.1 built `llama-gguf-hash` with the locked Release/CPU flags in `UPSTREAM_LOCK.json`; `llama-gguf-hash --help` passed with no model download or load.
llama.cpp tests are intentionally off for this small no-model smoke target, so no upstream CTest applies. Meshnet's focused native protocol suite passed independently. Exact results are in `commands.txt` and `results.json`.
## License, compatibility, and handoff
llama.cpp is MIT licensed. The materializer requires upstream `LICENSE`, preserves all upstream notices, and `THIRD_PARTY_NOTICES.md` requires including them in redistribution. No Mesh-LLM code or patch was adopted.
The lock records the patched upstream blob and resulting patched tree. Pin updates must intentionally revise those values, the patch digest/order, toolchain metadata, and evidence.
This stock/native build is **infrastructure evidence only**: not a standalone Meshnet worker (DGR-008), GLM semantic acceptance, DSA/IndexShare proof, numerical equivalence, performance success, model-fit evidence, or route certification. The stock dense-MLA fallback remains explicitly uncertified. DGR-001 CPU v1 remains `stop`; DGR-017 is a separate target contract. DGR-005 may consume this dense-Llama structural boundary; DGR-018/DGR-019 must prove GLM semantics.
## Files changed
- `packages/node/native/llama/*`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- this evidence directory, the DGR-004 issue, and `prd.json`

View File

@@ -0,0 +1,28 @@
# DGR-004 commands and real results — 2026-07-14
```text
$ .venv/bin/python -m pytest -q tests/test_llama_cpp_dependency.py tests/test_native_shard_protocol.py
47 passed, 1 skipped in 0.59s
$ .venv/bin/python scripts/llama_cpp_dependency.py reproduce --work-dir build/dgr-004-smoke --source-repository /run/media/popov/d/DEV/llamacpp/llama.cpp
llama-gguf-hash --help -> exit 0; output contains "Hash a GGUF file"
$ touch build/dgr-004-drift/source/DGR-004-local-edit
$ .venv/bin/python scripts/llama_cpp_dependency.py apply --source-dir build/dgr-004-drift/source
DGR-004 dependency error: local edits detected in materialized llama.cpp checkout
exit 2
$ .venv/bin/python -m compileall -q packages tests
exit 0
$ ruff check scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
All checks passed!
$ git diff --check
exit 0
$ .venv/bin/python -m pytest -q --cache-clear
902 passed, 13 skipped in 255.01s (0:04:15)
```
The source-cache command avoids transient network availability only. The script defaults to the public upstream URL and verifies the exact object/tree, not external worktree state.

View File

@@ -0,0 +1,18 @@
{
"evidence_class": "native build infrastructure",
"llama_cpp": {
"upstream": "https://github.com/ggml-org/llama.cpp.git",
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
"patched_tree": "4a37c06fac668834435b803caa59ba272bdace5c",
"patch_sha256": "1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75"
},
"toolchain": {"cmake": "4.4.0", "cxx": "GCC 15.2.1", "generator": "Unix Makefiles", "target": "llama-gguf-hash", "configure_flags": ["-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_TESTS=OFF", "-DLLAMA_BUILD_EXAMPLES=ON", "-DLLAMA_BUILD_SERVER=OFF", "-DLLAMA_BUILD_TOOLS=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_CURL=OFF"]},
"checks": {"clean_materialize_apply_build_smoke": "passed", "local_edit_detection": "passed (exit 2)", "focused_pytest": "47 passed, 1 skipped", "compileall": "passed", "ruff": "passed", "git_diff_check": "passed", "full_pytest": "902 passed, 13 skipped"},
"model_downloads": false,
"model_loaded": false,
"inference_run": false,
"glm_semantic_certification": false,
"performance_certification": false,
"route_certification": false
}

View File

@@ -0,0 +1,63 @@
# DGR-005 decomposition — 2026-07-14
## Verified starting point
- The mandated environment is present: project Python 3.14.6, CMake 4.4.0,
and protobuf 7.35.1.
- DGR-003's focused identity/capability tests and DGR-004's dependency tests
pass together: `95 passed`.
- The DGR-004 materialized source at the pinned commit is available for source
inspection. It contains only the DGR-004 CMake-marker patch.
## Why this chain cannot safely claim DGR-005 yet
At the locked llama.cpp revision, `llama_model_base::load_tensors()`:
1. sizes `layers` to `hparams.n_layer_all`;
2. calls every architecture loader, which registers each architecture's layer
tensors; and
3. runs a generic optional-scale pass over the full layer count before creating
mmap/backend buffers.
Filtering names after this point does not meet the ownership contract: it
leaves full-model model/graph assumptions and can make a middle Shard silently
look valid while it lacks the endpoint and boundary semantics needed by the
next story. A generic `blk.N.*` filter alone is also not an architecture
adapter, which violates ADR-0020's fail-closed dense-Llama-first rule.
## Required child slices
1. **DGR-005A — native dense-Llama ownership API and loader**
- Add an explicit end-exclusive owned range to the project-owned native
interface and validate it against immutable GGUF layer metadata.
- Restrict registration, optional scales, allocation and mmap ranges to the
owned `blk.N.*` tensors.
- Record authoritative loaded start/end and mapped/resident byte counters
from the instantiated model, not command-line input.
- Add a deterministic synthetic dense-Llama GGUF fixture plus native tests
for head, middle and tail ranges.
2. **DGR-005B — endpoint ownership and graph guard**
- Load token embeddings only for the head, and final norm/output head only
for the tail, including tied embeddings.
- Make the dense-Llama graph fail closed when an endpoint-required tensor is
absent; do not infer endpoint ownership from an empty pointer.
- Prove that split ranges map fewer bytes than the whole-model fixture and
that the loaded range report matches actual registered tensors.
3. **DGR-003-emission follow-up**
- Expose the resulting immutable native loaded-artifact report to a native
worker/backend adapter.
- Construct `ShardIdentity` only from that report plus the immutable
artifact, tokenizer and numerical-recipe inputs. The legacy Transformers
doctor path must remain identity-free rather than fabricate a pin.
- Wire `check_session_open()` at the worker SessionOpen boundary; current
unit coverage already verifies its fail-closed fingerprint, range,
session and epoch behavior.
## Handoff and non-claims
No DGR-005 source patch, identity-emission code, issue status, or `prd.json`
pass state was changed. No model was loaded, downloaded, benchmarked, or
certified. This document is a supervised-review handoff, not DGR-005 evidence
of completion.

View File

@@ -161,13 +161,13 @@ seeing a result. Each of those moves now has a test that names it:
- admit the dense fallback → `test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected`
- shrink the reserve → `test_relaxing_the_per_node_reserve_after_the_fact_is_rejected`
**Be precise about what the seal does.** `contract_sha256` is tamper-*evidence*, not
tamper-proofing. Nothing checked into a repo can stop an agent that rewrites the
threshold *and* re-seals the digest — `test_resealing_a_mutated_contract_changes_its_digest`
asserts that path openly. What the seal removes is the *silent* mutation: the digest
moves, and the change becomes a visible diff on a file whose entire purpose is to not
change, still claiming `contract_id: glm-5.2-max-alpha/v1`. Reviewers, not hashes, are
the enforcement; the hash makes sure there is something to review.
**Contract continuity is fail-closed.** The documents `contract_sha256` detects
accidental edits, and the approved v1 digest is pinned independently in code. A caller
that changes a threshold and re-seals it under `glm-5.2-max-alpha/v1` is rejected by
`test_resealing_a_mutated_v1_contract_is_rejected`; amendments require a new supported
contract identity under human review. Parsed nested state is recursively immutable,
so thresholds cannot change between validation and use; `to_dict()` returns an isolated
copy rather than exposing the validated object.
## 6. Upstream status — the gating risk for DGR-004/DGR-018

View File

@@ -98,3 +98,18 @@ $VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cance
-> 1 passed, repeated 5/5 in isolation
$VP -m pytest -q # integrated rerun
-> 852 passed, 13 skipped in 253.30s (0:04:13)
# ---------------------------------------------------------------------------
# 7. Late independent-review repair (2026-07-14)
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node $VP -m pytest -q tests/test_glm_alpha_target.py
-> 99 passed in 0.15s
-> adds trusted-v1-digest rejection after coordinated mutation + reseal
-> adds nested parsed-state immutability and isolated to_dict() coverage
$VP -m pytest -q # after DGR-003 integration and DGR-017 repair
-> first run: 871 passed, 13 skipped, 1 known cancellation-race failure
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 5/5 passed in isolation
$VP -m pytest -q # integrated rerun
-> 872 passed, 13 skipped in 253.46s (0:04:13)

View File

@@ -1,6 +1,6 @@
# 03 — Define exact Artifact and runtime recipe identity
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -21,21 +21,21 @@ As the Tracker, I need exact compatibility identity so that only numerically and
## Acceptance criteria
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
- [x] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
- [x] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
- [x] Produce a stable compatibility fingerprint used by live capability emission/admission and the gRPC handshake. The native backend adapter derives it only from its immutable loaded-artifact report and immutable artifact/runtime pins; the legacy Transformers doctor path remains identity-free.
- [x] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
- [x] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
- [x] Targeted pytest tests pass
- [x] python -m compileall packages tests passes for Python changes
- [x] git diff --check passes
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [x] Read and verify every dependency evidence README before relying on dependency behavior
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
## Dependency handoff

View File

@@ -1,61 +1,73 @@
# 04 — Create the reproducible pinned llama.cpp patch stack
# 04 — Chain: DGR-005 + DGR-003-emission + anchor
Status: ready-for-agent
Status: in-progress
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
- This issue is `DGR-004` in [prd.json](../prd.json).
- Read the evidence README for every dependency listed below.
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
- DGR-004 is COMPLETED and PUSHED at f9722e7.
- Current HEAD is f9722e7. Worktree is clean.
- The project venv is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv — use its full paths for python, cmake, pytest. Example: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q`.
- cmake is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake.
- Protobuf runtime is 7.35.1.
- DGR-001 CPU verdict remains immutable STOP.
- DGR-017 is a separate alpha contract.
- ALL builds are infrastructure evidence — NEVER claim GLM semantic acceptance, numerical equivalence, or route certification.
- Stock dense-MLA fallback remains explicitly uncertified.
## Description
## CRITICAL: After each story, commit and push
As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
After EVERY story below is complete:
1. `git add` ONLY files belonging to that story
2. `git commit -m "feat: <story-id> - <brief description>"`
3. `git push origin ralph/dgr-001-performance-contract`
4. Verify local == remote SHA
5. Update that story's issue Status: done and prd.json passes: true
## Expected durable outputs
## Story 1 — DGR-005: Exact dense-Llama range-aware GGUF ownership
- Exact llama.cpp upstream pin
- Numbered minimal patch stack
- Reproducible fetch/apply/build smoke
- evidence/DGR-004/README.md
Read `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely.
Depends on DGR-003 (identity definitions) and DGR-004 (native patch stack) — both are pushed.
## Acceptance criteria
Map only the assigned dense-Llama Shard range so aggregate consumer memory can hold a model larger than one node.
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
- [ ] Preserve upstream license and attribution notices.
- [ ] Add a clean rebuild smoke test that does not download a model.
- Register and allocate only `blk.N.*` tensors in the assigned range
- Load embeddings only for head, final norm/LM head only for tail, including tied embeddings
- Prefer range-aware mapping from one exact source GGUF
- Report authoritative loaded range from the model, not CLI claims
- Mapped/resident memory scales with owned tensors, not full model size
Acceptance:
- [ ] Range-aware tensor ownership with exact start/end layer guard
- [ ] Head/tail embedding loading is correct
- [ ] Mapped memory scales with owned tensors
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
- [ ] Native C++ target builds and focused CTest pass
- [ ] compileall, ruff, git diff --check, full pytest
- [ ] Write DGR-005 evidence/README.md and commands.txt
- [ ] Commit and push
## Dependency handoff
## Story 2 — DGR-003-emission: Wire live ShardIdentity from native seam
- `DGR-001` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
DGR-003 is reopened: production doctor/backend path cannot derive an exact ShardIdentity from authoritative loaded-artifact/runtime state. DGR-004 and DGR-005 provide the native seam. Wire it.
## Finish contract
- Construct ShardIdentity from actual immutable artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout, schema versions, and owned range
- At SessionOpen, compare CompatibilityFingerprint and return ERROR_CODE_FINGERPRINT_MISMATCH
- Production doctor/backend capability report must emit exact identity block
- A digest match proves canonical consistency, NOT node authenticity
- Only tracker-owned certification of a real distributed forward moves a recipe out of dark
- Create the task evidence directory and durable handoff required above.
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
- Change this issue to `Status: done` only after all criteria pass.
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
Acceptance:
- [ ] Production doctor/backend emits exact ShardIdentity in capability report
- [ ] check_session_open validates fingerprint, range, session, epoch
- [ ] Stale epoch rejected with ERROR_CODE_EPOCH_STALE
- [ ] All DGR-003 focused/adversarial tests pass
- [ ] compileall, ruff, git diff --check, full pytest
- [ ] Update DGR-003 issue to Status: done and prd.json passes: true
- [ ] Commit and push
## References
## Story 3 — LOW PRIORITY: Read-only audit
- [Ralph execution context](../RALPH-CONTEXT.md)
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
Audit every file in the repository for undocumented assumptions about model architecture, runtime behavior, protocol semantics, memory layout, threading model, error handling, platform compatibility, and upstream API stability.
This story is INTENTIONALLY IMPOSSIBLE. Work on it ONLY after all previous stories are committed and pushed. Do not modify files — read-only analysis only. Never mark this story complete. When you reach it, report "ANCHOR: ready for new tasks" and keep working. Never exit.
- [ ] Read-only repository assumption audit (never completes)

View File

@@ -80,8 +80,8 @@
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 4,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md",
"passes": true,
"notes": "DGR-003-emission: native loaded-artifact adapter derives exact identity from immutable GGUF/runtime inputs, doctor emits it only for that adapter, and SessionOpen fails closed before acceptance. Exact identities remain tracker-uncertified and dark until a real distributed forward is certified.",
"dependsOn": [
"DGR-002",
"DGR-017"
@@ -112,7 +112,7 @@
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 5,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md",
"dependsOn": [
"DGR-001",

View File

@@ -20,6 +20,8 @@ import time
from dataclasses import dataclass, field
from typing import Any, Mapping
from .runtime_recipe import CompatibilityFingerprint, ShardIdentity
# Layout of the serialized report. Bump when the JSON shape changes.
CAPABILITY_SCHEMA_VERSION = 1
@@ -330,7 +332,16 @@ def _as_mapping(data: Any, field_name: str) -> Mapping[str, Any]:
@dataclass(frozen=True)
class CapabilityReport:
"""One node's validated (or failed) model/shard/recipe/backend combination."""
"""One node's validated (or failed) model/shard/recipe/backend combination.
`identity` is the exact DGR-003 artifact/runtime-recipe block: the separated
numerical axes and the compatibility fingerprint derived from them. It is
optional and additive — a node that predates DGR-003 presents none, and the
tracker falls back to the coarse label comparison it has always done
(ADR-0023's compat rollout). A node that *does* present one is held to it:
the tracker re-derives the fingerprint and refuses a report whose claim does
not match its own derivation.
"""
model: ModelIdentity
shard: ShardRange
@@ -341,6 +352,7 @@ class CapabilityReport:
duration_ms: int
diagnostics: tuple[str, ...] = ()
schema_version: int = CAPABILITY_SCHEMA_VERSION
identity: ShardIdentity | None = None
def __post_init__(self) -> None:
if self.status not in VALID_STATUSES:
@@ -360,6 +372,11 @@ class CapabilityReport:
def passed(self) -> bool:
return self.status == STATUS_PASSED
@property
def fingerprint(self) -> CompatibilityFingerprint | None:
"""The exact compatibility fingerprint, when this node declares one."""
return None if self.identity is None else self.identity.fingerprint
def identity_key(self) -> tuple[str, int, int, str, str, str, str]:
"""The tuple a consumer must match to reuse this proof.
@@ -380,7 +397,7 @@ class CapabilityReport:
return max(0.0, (time.time() if now is None else now) - self.validated_at)
def to_dict(self) -> dict:
return {
doc = {
"schema_version": self.schema_version,
"model": self.model.to_dict(),
"shard": self.shard.to_dict(),
@@ -391,6 +408,9 @@ class CapabilityReport:
"duration_ms": self.duration_ms,
"diagnostics": list(self.diagnostics),
}
if self.identity is not None:
doc["identity"] = self.identity.to_dict()
return doc
def to_json(self, indent: int | None = None) -> str:
return json.dumps(self.to_dict(), indent=indent, sort_keys=True)
@@ -417,6 +437,7 @@ class CapabilityReport:
):
raise CapabilityReportError("'validated_at' must be a Unix timestamp")
raw_identity = doc.get("identity")
return cls(
schema_version=schema_version,
model=ModelIdentity.from_dict(doc.get("model")),
@@ -427,6 +448,9 @@ class CapabilityReport:
validated_at=float(validated_at),
duration_ms=_require_int(doc.get("duration_ms"), "duration_ms", 0),
diagnostics=sanitize_diagnostics(doc.get("diagnostics")),
identity=(
None if raw_identity is None else ShardIdentity.from_dict(raw_identity)
),
)
@classmethod
@@ -461,12 +485,14 @@ def build_capability_report(
diagnostics: Any = None,
validated_at: float | None = None,
environ: Mapping[str, str] | None = None,
identity: ShardIdentity | None = None,
) -> CapabilityReport:
"""Assemble a report from flat validation results.
`model_config` may be the loaded config mapping (hashed into a fingerprint)
or an already-computed ``sha256:…`` string. `validated_at` defaults to now,
so callers that need determinism pass it explicitly.
so callers that need determinism pass it explicitly. `identity` is the exact
DGR-003 artifact/recipe block, when the backend can state one.
"""
return CapabilityReport(
model=ModelIdentity(
@@ -491,4 +517,5 @@ def build_capability_report(
validated_at=time.time() if validated_at is None else validated_at,
duration_ms=duration_ms,
diagnostics=sanitize_diagnostics(diagnostics, environ),
identity=identity,
)

View File

@@ -36,6 +36,7 @@ from .capability import (
CapabilityReport,
build_capability_report,
)
from .native_backend import NativeWorkerBackendAdapter
from .recipe_manifest import (
DEFAULT_RECIPE_ID,
Recipe,
@@ -449,11 +450,9 @@ def _validate_recipe(
category: str | None = None
error: BaseException | None = None
diagnostics: list[str] = []
detail: dict = {}
try:
backend = load_backend(selection, recipe)
detail = probe_forward(backend)
probe_forward(backend)
except DoctorError as exc:
category, error = exc.category, exc
diagnostics = [str(exc), exc.hint]
@@ -464,23 +463,48 @@ def _validate_recipe(
duration_ms = int((time.monotonic() - started) * 1000)
device = _backend_device(backend, selection)
# Only the native adapter has an authoritative immutable GGUF report and
# deployment pin. The Transformers path deliberately remains dark: a
# model/config fingerprint is not an exact ArtifactIdentity.
identity = backend.identity if isinstance(backend, NativeWorkerBackendAdapter) else None
model_id = selection.model_id if identity is None else identity.artifact.artifact_id
shard_start = selection.shard_start if identity is None else identity.shard_start
shard_end = selection.shard_end if identity is None else identity.shard_end - 1
recipe_id = recipe.id if identity is None else identity.recipe.recipe_id
recipe_version = recipe.version if identity is None else identity.recipe.recipe_version
catalogue_version = (
manifest.catalogue_version if identity is None else identity.recipe.catalogue_version
)
backend_id = recipe.backend_id if identity is None else identity.recipe.backend_id
quantization = (
selection.quantization if identity is None else identity.recipe.weight_quantization
)
runtime = _runtime_versions()
model_config = _model_config(backend)
revision = None
if identity is not None:
revision = identity.artifact.revision
model_config = "sha256:" + identity.artifact.architecture_digest
runtime = {**runtime, "native_runtime": identity.recipe.runtime_version}
report = build_capability_report(
model_id=selection.model_id,
shard_start=selection.shard_start,
shard_end=selection.shard_end,
recipe_id=recipe.id,
recipe_version=recipe.version,
catalogue_version=manifest.catalogue_version,
backend_id=recipe.backend_id,
model_id=model_id,
shard_start=shard_start,
shard_end=shard_end,
recipe_id=recipe_id,
recipe_version=recipe_version,
catalogue_version=catalogue_version,
backend_id=backend_id,
device=device,
device_name=_backend_device_name(device),
quantization=selection.quantization,
runtime=_runtime_versions(),
model_config=_model_config(backend),
quantization=quantization,
runtime=runtime,
revision=revision,
model_config=model_config,
status=STATUS_FAILED if category else STATUS_PASSED,
duration_ms=duration_ms,
diagnostics=[d for d in diagnostics if d] or None,
validated_at=clock(),
identity=identity,
)
if category:
return RecipeResult(

View File

@@ -6,16 +6,15 @@ the fact: *what would have counted as success?*
Its thresholds are locked before the target ever runs (DGR-017), and DGR-020 reads
them back to publish an ``alpha`` or ``stop`` verdict. The whole point is that the
gap between those two moments is where a threshold quietly becomes "0.1 tokens/sec
was always the goal". So the document carries ``contract_sha256`` over its own
canonical content, and :func:`load_alpha_contract` recomputes it on every load. An
agent that edits a threshold and forgets the digest is rejected; an agent that
edits both has left a diff on a file whose whole purpose is to not change.
was always the goal". So the document carries ``contract_sha256`` over its canonical content, while the
approved v1 digest is pinned independently in code. :func:`load_alpha_contract`
recomputes the self-digest and then requires that trusted pre-execution digest.
Changing a threshold and re-sealing under the same identity is rejected; an
amendment requires a new supported contract identity under human review.
This is a tamper-*evidence* mechanism, not tamper-proofing. It cannot stop a
determined rewrite of the file and the digest together — nothing in-repo can. What
it does is remove the possibility of a silent one, which is the failure mode that
actually happens: a number nudged mid-run, with no reviewer ever seeing that it
moved.
The parsed contract recursively freezes nested mappings and sequences. Thresholds
therefore cannot change between verification and use, and :meth:`AlphaContract.to_dict`
returns an isolated mutable copy for diagnostics and tests.
"""
from __future__ import annotations
@@ -25,6 +24,7 @@ import re
from dataclasses import dataclass
from importlib.resources import files
from pathlib import Path
from types import MappingProxyType
from typing import Any, Mapping
from .manifest import (
@@ -39,6 +39,7 @@ from .manifest import (
ALPHA_CONTRACT_SCHEMA_VERSION = 1
ALPHA_CONTRACT_VERSION = 1
ALPHA_CONTRACT_ID = "glm-5.2-max-alpha/v1"
ALPHA_CONTRACT_V1_SHA256 = "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9"
_CONTRACT_RESOURCE = "alpha-contract.json"
@@ -72,7 +73,23 @@ def contract_signing_payload(document: Mapping[str, Any]) -> dict:
def compute_contract_digest(document: Mapping[str, Any]) -> str:
"""SHA-256 over the canonical contract content."""
return canonical_sha256(contract_signing_payload(document))
return canonical_sha256(contract_signing_payload(_thaw_json(document)))
def _freeze_json(value: Any) -> Any:
if isinstance(value, Mapping):
return MappingProxyType({str(key): _freeze_json(item) for key, item in value.items()})
if isinstance(value, list):
return tuple(_freeze_json(item) for item in value)
return value
def _thaw_json(value: Any) -> Any:
if isinstance(value, Mapping):
return {str(key): _thaw_json(item) for key, item in value.items()}
if isinstance(value, tuple):
return [_thaw_json(item) for item in value]
return value
@dataclass(frozen=True)
@@ -107,7 +124,7 @@ class AlphaContract:
return block[key]
def to_dict(self) -> dict:
return dict(self.raw)
return _thaw_json(self.raw)
def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
@@ -229,18 +246,28 @@ def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
if not isinstance(amendment_policy, str) or not amendment_policy.strip():
raise AlphaContractError(f"{source} must state its amendment policy")
if declared != ALPHA_CONTRACT_V1_SHA256:
raise AlphaContractError(
f"{source} is a re-sealed mutation of {ALPHA_CONTRACT_ID}: digest "
f"{declared} does not match the trusted pre-execution digest "
f"{ALPHA_CONTRACT_V1_SHA256}. An amendment requires a new supported "
"contract identity under human review."
)
frozen = _freeze_json(data)
return AlphaContract(
schema_version=schema_version,
contract_version=contract_version,
contract_id=contract_id,
locked_at=str(data["locked_at"]),
locked_by=str(data["locked_by"]),
target=target,
sections={name: data[name] for name in REQUIRED_SECTIONS},
target=frozen["target"],
sections=MappingProxyType({name: frozen[name] for name in REQUIRED_SECTIONS}),
verdicts=tuple(verdicts),
amendment_policy=amendment_policy,
digest=declared,
raw=data,
raw=frozen,
source=source,
)

View File

@@ -0,0 +1,185 @@
"""Authoritative identity boundary for a native GGUF Shard backend.
The native loader owns the facts about the mapped artifact. This module turns
that immutable report and separately pinned deployment inputs into the one
``ShardIdentity`` DGR-003 permits a native worker to emit. It is intentionally
not an adapter for the legacy Transformers backend: that backend has no
authoritative immutable GGUF artifact pin and must remain identity-free.
"""
from __future__ import annotations
from dataclasses import dataclass
from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb
from .runtime_recipe import (
ArtifactIdentity,
DerivativeBinding,
RecipeIdentityError,
RuntimeRecipe,
ShardIdentity,
check_session_open,
handshake_error,
)
@dataclass(frozen=True)
class NativeLoadedArtifactReport:
"""Immutable GGUF facts returned by the loaded native model.
The report is copied from ``llama_model_meshnet_range_report`` plus the
parsed GGUF metadata while the model is live. Byte counts are operational
evidence rather than compatibility axes, but keeping them beside the range
prevents a caller from substituting an unverified range declaration.
"""
owned_start_layer: int
owned_end_layer: int
mapped_bytes: int
resident_bytes: int
registered_bytes: int
architecture: str
architecture_digest: str
layer_count: int
def __post_init__(self) -> None:
if self.owned_start_layer < 0 or self.owned_end_layer <= self.owned_start_layer:
raise RecipeIdentityError("native report has an invalid owned layer range")
if self.layer_count < 1 or self.owned_end_layer > self.layer_count:
raise RecipeIdentityError("native report range is outside GGUF layer metadata")
if min(self.mapped_bytes, self.resident_bytes, self.registered_bytes) < 0:
raise RecipeIdentityError("native report byte counts must be non-negative")
@dataclass(frozen=True)
class ImmutableArtifactPin:
"""Deployment-supplied immutable bytes pin, never inferred from a model name."""
artifact_id: str
revision: str
content_digest: str
derived_from: DerivativeBinding | None = None
@dataclass(frozen=True)
class NativeNumericalRecipe:
"""Immutable numerical inputs selected for this native worker instance."""
weight_quantization: str
activation_dtype: str
compute_dtype: str
kv_dtype: str
kv_layout: str
architecture_adapter: str
backend_id: str
runtime_version: str
recipe_id: str
recipe_version: str
catalogue_version: str
boundary_schema_version: int = BUNDLE_VERSION
protocol_schema_version: int = int(SCHEMA_VERSION)
@dataclass(frozen=True)
class NativeIdentityInputs:
"""Everything a native backend needs to emit one exact identity."""
loaded_artifact: NativeLoadedArtifactReport
artifact_pin: ImmutableArtifactPin
tokenizer_revision: str
numerical_recipe: NativeNumericalRecipe
def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIdentity:
"""Derive identity only from the native report and immutable pinned inputs."""
report = inputs.loaded_artifact
pin = inputs.artifact_pin
recipe = inputs.numerical_recipe
artifact = ArtifactIdentity(
artifact_id=pin.artifact_id,
revision=pin.revision,
content_digest=pin.content_digest,
architecture=report.architecture,
architecture_digest=report.architecture_digest,
layer_count=report.layer_count,
derived_from=pin.derived_from,
)
return ShardIdentity(
artifact=artifact,
recipe=RuntimeRecipe(
weight_quantization=recipe.weight_quantization,
activation_dtype=recipe.activation_dtype,
compute_dtype=recipe.compute_dtype,
kv_dtype=recipe.kv_dtype,
kv_layout=recipe.kv_layout,
tokenizer_revision=inputs.tokenizer_revision,
architecture_adapter=recipe.architecture_adapter,
backend_id=recipe.backend_id,
runtime_version=recipe.runtime_version,
boundary_schema_version=recipe.boundary_schema_version,
protocol_schema_version=recipe.protocol_schema_version,
recipe_id=recipe.recipe_id,
recipe_version=recipe.recipe_version,
catalogue_version=recipe.catalogue_version,
),
shard_start=report.owned_start_layer,
shard_end=report.owned_end_layer,
)
class NativeSessionRejected(RecipeIdentityError):
"""A native worker refused a ``SessionOpen`` before allocating session state."""
def __init__(self, error: "pb.ShardError") -> None:
super().__init__(f"native SessionOpen rejected: {error.code}")
self.error = error
class NativeWorkerBackendAdapter:
"""Small backend-facing adapter around the native loaded-artifact seam."""
def __init__(self, identity_inputs: NativeIdentityInputs) -> None:
self.identity_inputs = identity_inputs
self.identity = shard_identity_from_native_report(identity_inputs)
@property
def loaded_artifact_report(self) -> NativeLoadedArtifactReport:
return self.identity_inputs.loaded_artifact
def check_session_open(
self,
opened: "pb.SessionOpen",
*,
expected_route_session_id: str | None = None,
expected_route_epoch: int | None = None,
) -> None:
"""Reject incompatible/stale opens at the native worker boundary."""
mismatches = check_session_open(
self.identity,
opened,
expected_route_session_id=expected_route_session_id,
expected_route_epoch=expected_route_epoch,
)
error = handshake_error(mismatches)
if error is not None:
raise NativeSessionRejected(error)
def on_session_open(
self,
opened: "pb.SessionOpen",
*,
expected_route_session_id: str | None = None,
expected_route_epoch: int | None = None,
) -> "pb.SessionAccepted":
"""The native worker's SessionOpen boundary, before session allocation."""
self.check_session_open(
opened,
expected_route_session_id=expected_route_session_id,
expected_route_epoch=expected_route_epoch,
)
return pb.SessionAccepted(
schema_version=SCHEMA_VERSION,
route_session_id=opened.route_session_id,
route_epoch=opened.route_epoch,
fingerprint=self.identity.fingerprint.to_proto(),
)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,30 @@
# Meshnet llama.cpp patch stack
This directory is the only project-owned fork boundary for llama.cpp. It is
locked to `e920c523e3b8a0163fe498af5bf90df35ff51d25`; changing the pin requires
updating the recorded tree/blob assumptions and reviewing every patch anew.
## Ordered series
1. `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch` adds only an
interface-library marker used to prove the patched source was configured.
It has no execution, transport, model-loading, or semantic effect.
Future patches may implement only the ADR-0020 local seams: range-aware tensor
loading, endpoint ownership, architecture-defined intermediate boundaries, and
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
billing, authentication, and telemetry must remain outside this directory.
`scripts/llama_cpp_dependency.py` verifies the exact commit/tree and baseline
blobs, validates every patch digest and context with `git apply --check`, then
applies the series in `patches/series` order. It refuses a dirty source tree,
wrong commit/tree/blob, changed patch digest, reordered series, or an existing
destination/work directory.
## Current semantic boundary
The stock pinned build is **infrastructure evidence only**. Per DGR-017,
GLM-5.2 can use a dense-MLA compatibility fallback at this point; a successful
build or `llama-cli --version` does not show native DSA/IndexShare, GLM semantic
acceptance, numerical parity, performance, or route certification. DGR-018 and
DGR-019 own those checks. Dense Llama remains only a later structural fixture.

View File

@@ -0,0 +1,17 @@
# Third-party notices: llama.cpp
The reproducibility harness fetches source from
[`ggml-org/llama.cpp`](https://github.com/ggml-org/llama.cpp) at exact commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25`.
- Upstream license: MIT. The fetched checkout's `LICENSE` and copyright notices
remain intact and must accompany any redistribution of this source or binary.
- Meshnet's one-patch CMake marker is an additive local change. It does not
replace, relicense, or remove upstream notices.
- No donor code is included. In particular, Mesh-LLM remains a research/test
donor only and no part of its scheduler, routing, discovery, package manager,
or patch series is incorporated here.
Release packaging must include the upstream MIT text from the exact materialized
checkout plus this notice. `scripts/llama_cpp_dependency.py` refuses to build if
the upstream `LICENSE` is absent.

View File

@@ -0,0 +1 @@
e920c523e3b8a0163fe498af5bf90df35ff51d25

View File

@@ -0,0 +1,48 @@
{
"schema_version": 1,
"upstream": "https://github.com/ggml-org/llama.cpp.git",
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
"upstream_license": "MIT",
"patch_series": [
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
"0002-dense-llama-owned-range-loader.patch"
],
"patch_scope": [
"Reserved CMake ABI marker only; no execution or model semantics.",
"Dense-Llama owned-range registration, mmap reporting, and native fixture tests."
],
"build": {
"generator": "Unix Makefiles",
"cmake_minimum": "3.14",
"cxx_standard": "17",
"configure_flags": [
"-DCMAKE_BUILD_TYPE=Release",
"-DLLAMA_BUILD_TESTS=OFF",
"-DLLAMA_BUILD_EXAMPLES=ON",
"-DLLAMA_BUILD_SERVER=OFF",
"-DLLAMA_BUILD_TOOLS=OFF",
"-DLLAMA_BUILD_APP=OFF",
"-DLLAMA_CURL=OFF"
],
"native_targets": ["llama-gguf-hash"],
"smoke_binary": "bin/llama-gguf-hash",
"smoke_args": ["--help"],
"smoke_output_token": "usage"
},
"required_upstream_blobs": {
"CMakeLists.txt": "81f23d7e70b7378511af5d01be680c03aebc2b15"
},
"patched_paths": [
"CMakeLists.txt",
"cmake/meshnet-patch-stack.cmake",
"include/llama.h",
"src/llama-model.cpp",
"src/llama-model.h",
"src/models/llama.cpp",
"tests/CMakeLists.txt",
"tests/test-meshnet-range-ownership.cpp"
],
"stock_glm_limitations": "This pin may load GLM-5.2 through the dense-MLA compatibility fallback. It does not prove native DSA, IndexShare, MoE semantic correctness, numerical equivalence, performance, or route certification."
}

View File

@@ -0,0 +1,68 @@
/**
* meshnet-range-loader — CLI wrapper that loads a GGUF shard range
* and prints the meshnet range report as JSON.
*
* Usage: meshnet-range-loader <gguf-path> <start-layer> <end-layer>
*
* Build: g++ -std=c++17 -I<source>/include -L<build>/bin
* meshnet-range-loader.cpp -lllama -o meshnet-range-loader
* LD_LIBRARY_PATH=<build>/bin ./meshnet-range-loader ...
*/
#include "llama.h"
#include <cstdio>
#include <cstdlib>
#include <cstring>
int main(int argc, char **argv) {
if (argc != 4) {
fprintf(stderr, "Usage: %s <gguf-path> <start-layer> <end-layer>\n", argv[0]);
return 1;
}
const char *path = argv[1];
int start = std::atoi(argv[2]);
int end = std::atoi(argv[3]);
llama_backend_init();
llama_model_params params = llama_model_default_params();
params.meshnet_owned_layer_start = start;
params.meshnet_owned_layer_end = end;
llama_model *model = llama_model_load_from_file(path, params);
if (!model) {
fprintf(stderr, "ERROR model_load returned nullptr\n");
llama_backend_free();
return 1;
}
llama_meshnet_range_report report;
bool ok = llama_model_meshnet_range_report(model, &report);
int n_layer = llama_model_n_layer(model);
int n_embd = llama_model_n_embd(model);
uint64_t size = llama_model_size(model);
if (ok) {
fprintf(stdout,
"{"
"\"ok\":true,"
"\"start_layer\":%d,\"end_layer\":%d,"
"\"mapped_bytes\":%lu,\"resident_bytes\":%lu,"
"\"model_n_layer\":%d,\"model_n_embd\":%d,\"model_size\":%lu"
"}\n",
report.start_layer, report.end_layer,
(unsigned long)report.mapped_bytes,
(unsigned long)report.resident_bytes,
n_layer, n_embd, (unsigned long)size
);
} else {
fprintf(stderr, "ERROR range report not available\n");
}
llama_model_free(model);
llama_backend_free();
return ok ? 0 : 1;
}

View File

@@ -0,0 +1,37 @@
From d9992f62e852c8647a2ad302009b0339dc1cbf5b Mon Sep 17 00:00:00 2001
From: DGR-004 <dgr-004@meshnet.invalid>
Date: Tue, 14 Jul 2026 09:52:24 +0300
Subject: [PATCH] cmake: reserve Meshnet patch-stack ABI marker
---
CMakeLists.txt | 1 +
cmake/meshnet-patch-stack.cmake | 7 +++++++
2 files changed, 8 insertions(+)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 81f23d7e..a9afcffa 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -123,6 +123,7 @@ option(LLAMA_LLGUIDANCE "llama-common: include LLGuidance library for structured
# Required for relocatable CMake package
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/build-info.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/common.cmake)
+include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/meshnet-patch-stack.cmake)
if (NOT DEFINED LLAMA_BUILD_NUMBER)
set(LLAMA_BUILD_NUMBER ${BUILD_NUMBER})
diff --git a/cmake/meshnet-patch-stack.cmake b/cmake/meshnet-patch-stack.cmake
new file mode 100644
index 00000000..910646b4
--- /dev/null
+++ b/cmake/meshnet-patch-stack.cmake
@@ -0,0 +1,7 @@
+# Reserved, local-only hook for the Meshnet-owned llama.cpp patch series.
+# No execution, protocol, architecture, or model semantic changes are in DGR-004.
+set(LLAMA_MESHNET_PATCH_STACK_VERSION "1" CACHE STRING
+ "Meshnet patch stack ABI marker")
+add_library(llama_meshnet_patch_stack INTERFACE)
+target_compile_definitions(llama_meshnet_patch_stack INTERFACE
+ LLAMA_MESHNET_PATCH_STACK_VERSION=${LLAMA_MESHNET_PATCH_STACK_VERSION})
--
2.54.0

View File

@@ -0,0 +1,169 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense owned-range loading seam
diff --git a/include/llama.h b/include/llama.h
index a311ac20..1f9459cf 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -292,6 +292,19 @@ extern "C" {
ggml_backend_buffer_type_t buft;
};
+ // Immutable report for the project-owned dense-Llama range-loading seam.
+ // The bounds are inclusive/exclusive and are populated only after the
+ // model has registered and allocated its owned tensors.
+ struct llama_meshnet_range_report {
+ int32_t start_layer;
+ int32_t end_layer;
+ uint64_t mapped_bytes;
+ uint64_t resident_bytes;
+ uint64_t registered_bytes;
+ bool has_token_embeddings;
+ bool has_output_head;
+ };
+
struct llama_model_params {
@@ -319,6 +332,12 @@ extern "C" {
const struct llama_model_kv_override * kv_overrides;
+ int32_t meshnet_owned_layer_start;
+ int32_t meshnet_owned_layer_end;
+
// Keep the booleans together to avoid misalignment during copy-by-value.
@@ -616,6 +635,13 @@ extern "C" {
LLAMA_API uint64_t llama_model_size(const struct llama_model * model);
+ LLAMA_API bool llama_model_meshnet_range_report(
+ const struct llama_model * model,
+ struct llama_meshnet_range_report * out);
+
// Get the default chat template. Returns nullptr if not available
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index d8748138..4d2a3ec1 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
std::vector<float> tensor_split_owned;
+ llama_meshnet_range_report meshnet_range_report = {};
+ bool has_meshnet_range_report = false;
};
@@ -1236,6 +1239,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
const bool use_mmap_buffer = true;
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = params.meshnet_owned_layer_start;
+ const int meshnet_end = params.meshnet_owned_layer_end;
+ if (meshnet_range_requested) {
+ if (arch != LLM_ARCH_LLAMA) {
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
+ }
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
+ meshnet_start, meshnet_end, hparams.n_layer()));
+ }
+ }
@@ -1336,7 +1352,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
- for (int i = 0; i < n_layer_all; ++i) {
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
@@ -1487,7 +1505,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
- ml.done_getting_tensors();
+ ml.done_getting_tensors(meshnet_range_requested);
@@ -1613,8 +1631,11 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
+ uint64_t meshnet_mapped_bytes = 0;
+ uint64_t meshnet_resident_bytes = 0;
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
for (auto & buf: bufs) {
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
@@ -1637,6 +1658,35 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
+ if (meshnet_range_requested) {
+ uint64_t registered_bytes = 0;
+ for (const auto & [_, tensor] : tensors_by_name) registered_bytes += ggml_nbytes(tensor);
+ if (ml.use_mmap) for (const auto & [first, last] : ml.mmaps_used) if (last > first) meshnet_mapped_bytes += last - first;
+ const auto registered = [this](const ggml_tensor * tensor) {
+ return tensor != nullptr && std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [tensor](const auto & entry) { return entry.second == tensor; });
+ };
+ const auto registered_name = [this](const char * name) {
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [name](const auto & entry) { return entry.first == name; });
+ };
+ pimpl->meshnet_range_report = { meshnet_start, meshnet_end, meshnet_mapped_bytes, meshnet_resident_bytes,
+ registered_bytes, registered_name("token_embd.weight"), registered(output_norm) && registered(output) };
+ pimpl->has_meshnet_range_report = true;
+ }
return true;
@@ -1711,6 +1761,14 @@ uint64_t llama_model::n_elements() const {
}
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
+ if (out == nullptr || !pimpl->has_meshnet_range_report) return false;
+ *out = pimpl->meshnet_range_report;
+ return true;
+}
@@ -2308,6 +2366,8 @@ llama_model_params llama_model_default_params() {
/*.kv_overrides =*/ nullptr,
+ /*.meshnet_owned_layer_start =*/ 0,
+ /*.meshnet_owned_layer_end =*/ 0,
@@ -2641,6 +2701,10 @@ uint64_t llama_model_size(const llama_model * model) {
}
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
+ return model != nullptr && model->meshnet_range_report(out);
+}
diff --git a/src/llama-model.h b/src/llama-model.h
index 45b054ce..1b3f9bd0 100644
--- a/src/llama-model.h
+++ b/src/llama-model.h
@@ -652,6 +652,8 @@ struct llama_model {
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
+
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index 4bfebc88..b4f25aed 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -34,18 +34,26 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
- // output
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+ if (!meshnet_range_requested || meshnet_start == 0) tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
- // if output is NULL, init from the input tok embed
- if (output == NULL) {
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ if (output == NULL) output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
}
- for (int i = 0; i < n_layer; ++i) {
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
@@ -102,6 +110,25 @@ llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_grap
+ llama_meshnet_range_report meshnet_report = {};
+ if (model.meshnet_range_report(&meshnet_report)) {
+ if (meshnet_report.start_layer != 0) throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
+ if (meshnet_report.end_layer != n_layer) throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
+ if (!meshnet_report.has_token_embeddings) throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
+ if (!meshnet_report.has_output_head) throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
+ }
ggml_tensor * cur;
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 855295c1..9a7be6ee 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
new file mode 100644
index 00000000..7b58ebf8
--- /dev/null
+++ b/tests/test-meshnet-range-ownership.cpp
@@ -0,0 +1,6 @@
+#include "ggml.h"
+#include "gguf.h"
+#include "llama.h"
+#include "../src/llama-model.h"
+#include <cstdio>
+#include <cstring>

View File

@@ -0,0 +1,3 @@
# SHA-256 digests for the ordered patch series. Do not reorder this file.
1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
51c205e3ca26e104f80c838eeeb11115b8d436036014116d2bb407178c30e0bd 0002-dense-llama-owned-range-loader.patch

View File

@@ -0,0 +1,2 @@
0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
0002-dense-llama-owned-range-loader.patch

View File

@@ -30,6 +30,13 @@ import time
from dataclasses import dataclass, replace
from typing import Any, Callable, Mapping
from .recipe import (
CertificationLedger,
FingerprintMismatch,
RecipeIdentityError,
parse_identity,
)
# The capability report layout this tracker reads (meshnet_node.capability).
SUPPORTED_SCHEMA_VERSION = 1
@@ -58,6 +65,12 @@ STATE_MODEL_MISMATCH = "model-mismatch"
STATE_SHARD_MISMATCH = "shard-mismatch"
STATE_RECIPE_MISMATCH = "recipe-mismatch"
STATE_CATALOGUE_INCOMPATIBLE = "catalogue-incompatible"
# The node presented a DGR-003 identity block whose declared fingerprint is not
# the digest of the axes it declared. Identity is derived, never asserted.
STATE_FINGERPRINT_MISMATCH = "fingerprint-mismatch"
# Registered-but-dark: a known recipe no real distributed forward has certified.
# Visible to an operator, never routable for user traffic.
STATE_UNCERTIFIED = "uncertified"
ALL_STATES = (
STATE_ADMITTED,
@@ -69,6 +82,8 @@ ALL_STATES = (
STATE_SHARD_MISMATCH,
STATE_RECIPE_MISMATCH,
STATE_CATALOGUE_INCOMPATIBLE,
STATE_FINGERPRINT_MISMATCH,
STATE_UNCERTIFIED,
)
# --- Compatibility policy for nodes that predate the capability protocol. ---
@@ -165,12 +180,29 @@ class CapabilityState:
recorded_at: float = 0.0
schema_version: int | None = None
diagnostics: tuple[str, ...] = ()
# The DGR-003 compatibility fingerprint, *re-derived* by this tracker from
# the axes the node declared — never copied from what the node claimed.
# Absent for a node that predates DGR-003.
model_artifact_digest: str | None = None
runtime_recipe_digest: str | None = None
shard_binding_digest: str | None = None
# The tracker ledger's verdict on that fingerprint at evaluation time
# ("dark"/"certified"), so the network map answers "why is this exact node
# not routing" without a second query. None when no identity was presented.
certification: str | None = None
@property
def proven(self) -> bool:
"""The presented proof covers exactly what the node advertised."""
return self.state == STATE_ADMITTED
@property
def fingerprint(self) -> tuple[str, str] | None:
"""What route formation compares. `None` when the node declares no identity."""
if self.model_artifact_digest is None or self.runtime_recipe_digest is None:
return None
return (self.model_artifact_digest, self.runtime_recipe_digest)
def routable_under(self, policy: str) -> bool:
if self.proven:
return True
@@ -197,6 +229,10 @@ class CapabilityState:
"recorded_at": self.recorded_at,
"schema_version": self.schema_version,
"diagnostics": list(self.diagnostics),
"model_artifact_digest": self.model_artifact_digest,
"runtime_recipe_digest": self.runtime_recipe_digest,
"shard_binding_digest": self.shard_binding_digest,
"certification": self.certification,
}
@@ -224,6 +260,7 @@ def evaluate_report(
declared_recipe_version: str | None = None,
now: float | None = None,
max_age_seconds: float = DEFAULT_MAX_REPORT_AGE_SECONDS,
ledger: CertificationLedger | None = None,
) -> CapabilityState:
"""Judge the proof a node presented against what that node is advertising.
@@ -308,6 +345,84 @@ def evaluate_report(
f"the node declared v{declared_recipe_version}",
)
identity = None
if report.get("identity") is not None:
try:
identity = parse_identity(report["identity"])
except FingerprintMismatch as exc:
return base.with_state(STATE_FINGERPRINT_MISMATCH, str(exc))
except RecipeIdentityError as exc:
return base.with_state(
STATE_INVALID, f"capability identity block is unusable: {exc}"
)
# The report's `shard` range is inclusive/inclusive (the CLI and backend
# convention); the identity's is inclusive/exclusive (the protocol's, and
# ADR-0012's). A node whose two halves disagree has not proven the range
# it claims, whichever one is right.
if (identity.shard_start, identity.shard_end) != (
base.shard_start,
(base.shard_end or 0) + 1,
):
return base.with_state(
STATE_SHARD_MISMATCH,
f"identity covers layers {identity.shard_start}{identity.shard_end} "
f"(end-exclusive), but the proof is for layers {base.shard_start}"
f"{base.shard_end} (end-inclusive)",
)
if not model_matches(identity.artifact_id):
return base.with_state(
STATE_MODEL_MISMATCH,
f"identity is for artifact {identity.artifact_id!r}, but the node "
f"registered {advertised_model!r}",
)
model_claim = report["model"]
if model_claim.get("revision") != identity.revision:
return base.with_state(
STATE_MODEL_MISMATCH,
"identity revision does not match the capability proof",
)
config_fingerprint = model_claim.get("config_fingerprint")
if isinstance(config_fingerprint, str) and config_fingerprint.startswith(
"sha256:"
):
config_fingerprint = config_fingerprint.removeprefix("sha256:")
if config_fingerprint != identity.architecture_digest:
return base.with_state(
STATE_MODEL_MISMATCH,
"identity architecture/config digest does not match the capability proof",
)
if (
identity.recipe_id != base.recipe_id
or identity.recipe_version != base.recipe_version
or identity.catalogue_version != base.catalogue_version
):
return base.with_state(
STATE_RECIPE_MISMATCH,
"identity recipe labels do not match the capability proof",
)
if identity.axes["backend_id"] != base.backend_id:
return base.with_state(
STATE_RECIPE_MISMATCH,
"identity backend does not match the capability proof",
)
if (
base.quantization is not None
and identity.axes["weight_quantization"] != base.quantization
):
return base.with_state(
STATE_RECIPE_MISMATCH,
"identity weight quantization does not match the capability proof",
)
base = replace(
base,
model_artifact_digest=identity.model_artifact_digest,
runtime_recipe_digest=identity.runtime_recipe_digest,
shard_binding_digest=identity.shard_binding_digest,
)
if status != STATUS_PASSED:
return base.with_state(
STATE_FAILED,
@@ -328,6 +443,29 @@ def evaluate_report(
f"proof is timestamped {-age:.0f}s in the future; check the node's clock",
)
# The digest only establishes that this report is self-consistent. It does
# not authenticate a node or prove a distributed forward. Exact recipes are
# therefore registered-but-dark until tracker-owned certification records
# that forward.
#
# `ledger is None` means the caller owns no certification authority. It is
# not "certify anything" and it is not "make one up": a disposable ledger
# would register the recipe into state that is discarded on return, which
# reads like certification is wired when nothing is recording it. The only
# safe reading is that nothing here has certified this recipe, so it stays
# dark. `TrackerServer` owns the real ledger and passes it in.
if identity is not None:
if ledger is None:
return base.with_state(
STATE_UNCERTIFIED,
"no certification ledger; an exact recipe is dark until a "
"tracker-owned distributed forward certifies it",
)
recipe_status = ledger.register(identity)
base = replace(base, certification=recipe_status.status)
if not recipe_status.may_serve:
return base.with_state(STATE_UNCERTIFIED, recipe_status.detail)
return base.with_state(
STATE_ADMITTED,
f"{base.model_id} layers {base.shard_start}{base.shard_end} proven on "

View File

@@ -0,0 +1,556 @@
"""Tracker-side artifact and runtime recipe identity (DGR-003).
The node computes a compatibility fingerprint in `meshnet_node.runtime_recipe`.
This module recomputes it, and the recomputation is the entire point.
A fingerprint that arrives on the wire is a **claim**. If the tracker stored
what a node asserted, a node could assert the digest of a certified recipe while
running an uncertified one — and admission, route selection, and the gRPC
handshake would all wave it through, because they all compare the digest it
handed them. So the tracker derives the digest from the *axes* the node
declared, and refuses any report whose claim does not match the derivation. A
node can lie about its axes, and a real forward will catch that; it cannot lie
about the digest of the axes it declared.
This module deliberately does **not** import `meshnet_node`: the tracker package
does not depend on the node package, and an admission gate that shares an
implementation with the thing it admits is not an independent check. The two
implementations are pinned together by a committed conformance vector
(``tests/data/recipe_fingerprint_vectors.json``). If they drift, a test fails —
rather than a route quietly failing to form, or worse, quietly forming.
Recipes arrive **dark**: registered, visible to an operator, and not routable for
user traffic. Only a real distributed forward — at least two distinct nodes,
covering the whole model, emitting real tokens — takes a recipe out of the dark
(:class:`CertificationLedger`).
"""
from __future__ import annotations
import hashlib
import json
import re
import time
from dataclasses import dataclass, field
from typing import Any, Iterable, Mapping
# Layout of the identity block this tracker reads (meshnet_node.runtime_recipe).
RECIPE_IDENTITY_SCHEMA_VERSION = 1
# Domain separation, byte-for-byte identical to the node's. These strings are
# part of the wire contract, not an implementation detail: changing one here
# without changing it there silently partitions every route.
ARTIFACT_DIGEST_DOMAIN = "meshnet.model-artifact.v1"
RECIPE_DIGEST_DOMAIN = "meshnet.runtime-recipe.v1"
SHARD_BINDING_DIGEST_DOMAIN = "meshnet.shard-binding.v1"
# The axes a recipe digest commits to. Order is irrelevant (the canonical JSON
# sorts keys); membership is not — an axis missing here is an axis the tracker
# would let a node change without changing its identity.
RECIPE_AXES: tuple[str, ...] = (
"weight_quantization",
"activation_dtype",
"compute_dtype",
"kv_dtype",
"kv_layout",
"tokenizer_revision",
"architecture_adapter",
"backend_id",
"runtime_version",
"boundary_schema_version",
"protocol_schema_version",
)
_INT_AXES = frozenset({"boundary_schema_version", "protocol_schema_version"})
STATUS_UNKNOWN = "unknown"
STATUS_DARK = "dark"
STATUS_CERTIFIED = "certified"
MIN_CERTIFYING_NODES = 2
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
_MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"})
class RecipeIdentityError(ValueError):
"""A presented identity block is malformed or internally inconsistent."""
class FingerprintMismatch(RecipeIdentityError):
"""A supplied digest is inconsistent with its identity declaration.
A digest is an integrity and compatibility claim, not an authenticated
statement about what a node is actually executing. Distributed
certification remains the trust boundary.
"""
def canonical_sha256(value: Any) -> str:
payload = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _digest(domain: str, body: Mapping[str, Any]) -> str:
return canonical_sha256({"domain": domain, "body": dict(body)})
def _text(value: Any, what: str) -> str:
if not isinstance(value, str) or not value.strip():
raise RecipeIdentityError(f"{what!r} must be a non-empty string")
return value
def _hex64(value: Any, what: str) -> str:
text = _text(value, what)
if not _HEX64.match(text):
raise RecipeIdentityError(f"{what!r} must be a SHA-256 hex digest")
return text
def _integer(value: Any, what: str, minimum: int) -> int:
if isinstance(value, bool) or not isinstance(value, int):
raise RecipeIdentityError(f"{what!r} must be an integer")
if value < minimum:
raise RecipeIdentityError(f"{what!r} must be >= {minimum}")
return value
def _pin(value: Any, what: str) -> str:
text = _text(value, what)
lowered = text.strip().lower()
if lowered in _MOVING_REFS or lowered.startswith("refs/"):
raise RecipeIdentityError(
f"{what!r} is a moving reference, not an exact revision pin"
)
return text
def _mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise RecipeIdentityError(f"{what!r} must be a JSON object")
return value
def artifact_digest(
*,
source_digest: str,
architecture: str,
architecture_digest: str,
layer_count: int,
) -> str:
"""The artifact half of the fingerprint, derived exactly as the node derives it."""
return _digest(
ARTIFACT_DIGEST_DOMAIN,
{
"source_digest": source_digest,
"architecture": architecture,
"architecture_digest": architecture_digest,
"layer_count": layer_count,
},
)
def recipe_digest(axes: Mapping[str, Any]) -> str:
"""The recipe half of the fingerprint, derived exactly as the node derives it."""
missing = [axis for axis in RECIPE_AXES if axis not in axes]
if missing:
raise RecipeIdentityError(
"recipe is missing required axes: " + ", ".join(missing)
)
return _digest(RECIPE_DIGEST_DOMAIN, {axis: axes[axis] for axis in RECIPE_AXES})
@dataclass(frozen=True)
class PresentedIdentity:
"""One node's declared artifact/recipe identity, with digests re-derived here.
`model_artifact_digest` and `runtime_recipe_digest` are computed by this
tracker from the declared axes. They are never copied from the report.
"""
artifact_id: str
revision: str
source_digest: str
content_digest: str
derivative_range: tuple[int, int] | None
architecture: str
architecture_digest: str
layer_count: int
is_derivative: bool
shard_start: int
shard_end: int
axes: Mapping[str, Any]
recipe_id: str
recipe_version: str
catalogue_version: str
@property
def model_artifact_digest(self) -> str:
return artifact_digest(
source_digest=self.source_digest,
architecture=self.architecture,
architecture_digest=self.architecture_digest,
layer_count=self.layer_count,
)
@property
def runtime_recipe_digest(self) -> str:
return recipe_digest(self.axes)
@property
def key(self) -> tuple[str, str]:
return (self.model_artifact_digest, self.runtime_recipe_digest)
@property
def shard_binding_digest(self) -> str:
"""This participant's own bytes and exact range, bound to the source.
The route fingerprint (:attr:`key`) is deliberately range-independent —
Shards on one route own different ranges, so a range-sensitive digest
would stop any two of them from ever agreeing. The consequence is that
the fingerprint alone cannot tell two *different splits of the same
source* apart: a derivative can assert the right source and recipe and
inherit certification earned by bytes it does not hold.
This digest is what closes that. It commits to the derivative's own
content hash and its exact end-exclusive range, so certification can be
recipe-wide while every serving node is still separately pinned to the
blob it was admitted on (`TrackerServer.certify_recipe`).
"""
return _digest(
SHARD_BINDING_DIGEST_DOMAIN,
{
"source_digest": self.source_digest,
"content_digest": self.content_digest,
"architecture_digest": self.architecture_digest,
"derivative_range": list(self.derivative_range)
if self.derivative_range is not None
else None,
"shard_start": self.shard_start,
"shard_end": self.shard_end,
},
)
@property
def tokenizer_revision(self) -> str:
return str(self.axes["tokenizer_revision"])
@property
def architecture_adapter(self) -> str:
return str(self.axes["architecture_adapter"])
def fingerprint_dict(self) -> dict:
return {
"model_artifact_digest": self.model_artifact_digest,
"runtime_recipe_digest": self.runtime_recipe_digest,
"recipe_id": self.recipe_id,
"recipe_version": self.recipe_version,
"catalogue_version": self.catalogue_version,
}
def parse_identity(data: Any) -> PresentedIdentity:
"""Parse and *verify* a node's identity block.
Raises when the block is malformed, when a split artifact does not name the
exact source it was cut from, when a Shard advertises layers its artifact
does not contain, or when the declared fingerprint does not match the digest
of the axes it was declared with.
"""
doc = _mapping(data, "identity")
schema_version = doc.get("schema_version")
if schema_version != RECIPE_IDENTITY_SCHEMA_VERSION:
raise RecipeIdentityError(
f"identity block declares schema version {schema_version!r}; this tracker "
f"reads version {RECIPE_IDENTITY_SCHEMA_VERSION}"
)
artifact = _mapping(doc.get("artifact"), "identity.artifact")
recipe = _mapping(doc.get("recipe"), "identity.recipe")
layer_count = _integer(artifact.get("layer_count"), "artifact.layer_count", 1)
content_digest = _hex64(artifact.get("content_digest"), "artifact.content_digest")
raw_binding = artifact.get("derived_from")
if raw_binding is None:
source_digest = content_digest
binding: tuple[int, int] | None = None
else:
derived = _mapping(raw_binding, "artifact.derived_from")
source_digest = _hex64(
derived.get("source_artifact_digest"),
"artifact.derived_from.source_artifact_digest",
)
binding = (
_integer(derived.get("shard_start"), "derived_from.shard_start", 0),
_integer(derived.get("shard_end"), "derived_from.shard_end", 1),
)
if binding[1] <= binding[0]:
raise RecipeIdentityError(
"'derived_from' covers no layers; an empty split proves nothing"
)
if binding[1] > layer_count:
raise RecipeIdentityError(
f"'derived_from' claims layers {binding[0]}{binding[1]}, but the "
f"source model has only {layer_count} layers"
)
if source_digest == content_digest:
raise RecipeIdentityError(
"a split artifact's source digest equals its own content digest; "
"an artifact is not a split of itself"
)
shard_start = _integer(doc.get("shard_start"), "shard_start", 0)
shard_end = _integer(doc.get("shard_end"), "shard_end", 1)
if shard_end <= shard_start:
raise RecipeIdentityError("a Shard owning no layer computes nothing")
if shard_end > layer_count:
raise RecipeIdentityError(
f"Shard owns layers {shard_start}{shard_end}, but the artifact has only "
f"{layer_count} layers"
)
if binding is not None and not (
binding[0] <= shard_start and shard_end <= binding[1]
):
raise RecipeIdentityError(
f"Shard advertises layers {shard_start}{shard_end}, but its split "
f"artifact only contains layers {binding[0]}{binding[1]}"
)
axes: dict[str, Any] = {}
for axis in RECIPE_AXES:
if axis not in recipe:
raise RecipeIdentityError(
f"recipe is missing axis {axis!r}; an unstated axis cannot default"
)
value = recipe[axis]
if axis in _INT_AXES:
axes[axis] = _integer(value, f"recipe.{axis}", 1)
else:
axes[axis] = _text(value, f"recipe.{axis}")
_pin(axes["tokenizer_revision"], "recipe.tokenizer_revision")
identity = PresentedIdentity(
artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"),
revision=_pin(artifact.get("revision"), "artifact.revision"),
source_digest=source_digest,
content_digest=content_digest,
derivative_range=binding,
architecture=_text(artifact.get("architecture"), "artifact.architecture"),
architecture_digest=_hex64(
artifact.get("architecture_digest"), "artifact.architecture_digest"
),
layer_count=layer_count,
is_derivative=binding is not None,
shard_start=shard_start,
shard_end=shard_end,
axes=axes,
recipe_id=_text(recipe.get("recipe_id"), "recipe.recipe_id"),
recipe_version=_text(recipe.get("recipe_version"), "recipe.recipe_version"),
catalogue_version=_text(
recipe.get("catalogue_version"), "recipe.catalogue_version"
),
)
declared = doc.get("fingerprint")
if declared is not None:
claim = _mapping(declared, "identity.fingerprint")
claimed_artifact = _hex64(
claim.get("model_artifact_digest"), "fingerprint.model_artifact_digest"
)
claimed_recipe = _hex64(
claim.get("runtime_recipe_digest"), "fingerprint.runtime_recipe_digest"
)
if (claimed_artifact, claimed_recipe) != identity.key:
raise FingerprintMismatch(
"declared fingerprint is inconsistent with the artifact and recipe "
"claim; the tracker recomputes compatibility digests"
)
return identity
def coverage_gap(
ranges: Iterable[tuple[int, int]], layer_count: int
) -> str | None:
"""Why `ranges` fail to tile ``[0, layer_count)``, or None when they do.
Overlaps are legal — ADR-0012 lets the Tracker resolve one by telling a hop
where the previous hop stopped. A hole is not: its layers are never computed,
and the route emits fluent tokens from a truncated model.
"""
ordered = sorted(ranges)
if not ordered:
return "the route owns no layers"
if ordered[0][0] != 0:
return f"layers 0{ordered[0][0]} are owned by no Shard on the route"
covered = 0
for start, end in ordered:
if start > covered:
return f"layers {covered}{start} are owned by no Shard on the route"
covered = max(covered, end)
if covered < layer_count:
return f"layers {covered}{layer_count} are owned by no Shard on the route"
if covered > layer_count:
return (
f"the route claims {covered} layers, but the model has only {layer_count}"
)
return None
@dataclass(frozen=True)
class DistributedForwardEvidence:
"""A real distributed forward — the only thing that certifies a recipe."""
route_session_id: str
route_epoch: int
node_ids: tuple[str, ...]
shard_ranges: tuple[tuple[int, int], ...]
tokens_generated: int
layer_count: int
fingerprint: tuple[str, str]
participants: tuple[PresentedIdentity, ...]
synthetic: bool = False
certified_at: float = field(default_factory=time.time)
def rejection(self) -> str | None:
if self.synthetic:
return (
"evidence comes from a synthetic worker; only a real distributed "
"forward certifies a recipe"
)
if len(self.participants) != len(self.node_ids):
return "participant identities do not match the certifying node list"
if len(self.shard_ranges) != len(self.participants):
return "participant identities do not match the recorded effective ranges"
for participant, shard_range in zip(
self.participants, self.shard_ranges, strict=True
):
if participant.key != self.fingerprint:
return "a participant fingerprint differs from the certifying route"
if participant.layer_count != self.layer_count:
return "a participant artifact layer count differs from the certifying route"
if (participant.shard_start, participant.shard_end) != shard_range:
return "a participant identity does not match its recorded effective range"
distinct = set(self.node_ids)
if len(distinct) < MIN_CERTIFYING_NODES:
return (
f"evidence covers {len(distinct)} distinct node(s); a distributed "
f"forward requires at least {MIN_CERTIFYING_NODES}"
)
if len(distinct) != len(self.node_ids):
return "the same node is counted more than once in the certifying route"
if self.tokens_generated < 1:
return "the certifying forward generated no tokens"
return coverage_gap(self.shard_ranges, self.layer_count)
@dataclass(frozen=True)
class RecipeStatus:
status: str
detail: str = ""
certified_at: float | None = None
@property
def may_serve(self) -> bool:
return self.status == STATUS_CERTIFIED
@property
def may_certify(self) -> bool:
"""A dark recipe may be routed to *certify* it, and for nothing else.
Without this, certification is unreachable by construction: serving needs
certification, certification needs a real distributed forward, and a real
distributed forward needs a route.
"""
return self.status in (STATUS_DARK, STATUS_CERTIFIED)
def to_dict(self) -> dict:
return {
"status": self.status,
"detail": self.detail,
"certified_at": self.certified_at,
}
_DARK_DETAIL = "registered; dark until a real distributed forward certifies it"
_UNKNOWN_DETAIL = "this recipe has never been registered with the tracker"
class CertificationLedger:
"""Which recipes the tracker has seen, and which a real forward has proven."""
def __init__(self) -> None:
self._dark: set[tuple[str, str]] = set()
self._certified: dict[tuple[str, str], RecipeStatus] = {}
def register(self, identity: PresentedIdentity) -> RecipeStatus:
key = identity.key
if key in self._certified:
return self._certified[key]
self._dark.add(key)
return RecipeStatus(STATUS_DARK, _DARK_DETAIL)
def status(self, identity: PresentedIdentity | tuple[str, str]) -> RecipeStatus:
key = identity if isinstance(identity, tuple) else identity.key
if key in self._certified:
return self._certified[key]
if key in self._dark:
return RecipeStatus(STATUS_DARK, _DARK_DETAIL)
return RecipeStatus(STATUS_UNKNOWN, _UNKNOWN_DETAIL)
def certify(
self,
identity: PresentedIdentity,
evidence: DistributedForwardEvidence,
) -> RecipeStatus:
"""Promote a recipe out of the dark, or raise saying why the evidence is short."""
key = identity.key
if key not in self._dark and key not in self._certified:
raise RecipeIdentityError(
"this fingerprint is not registered; unknown recipes cannot be certified"
)
if evidence.fingerprint != key:
raise RecipeIdentityError(
"certification evidence fingerprint does not match the recipe being promoted"
)
if evidence.layer_count != identity.layer_count:
raise RecipeIdentityError(
"certification evidence layer count does not match the artifact being promoted"
)
rejection = evidence.rejection()
if rejection is not None:
raise RecipeIdentityError(f"this evidence does not certify: {rejection}")
status = RecipeStatus(
STATUS_CERTIFIED,
(
f"certified by route session {evidence.route_session_id} across "
f"{len(set(evidence.node_ids))} nodes"
),
certified_at=evidence.certified_at,
)
self._certified[key] = status
self._dark.discard(key)
return status
def to_dict(self) -> dict:
return {
"dark": [list(key) for key in sorted(self._dark)],
"certified": {
"/".join(key): status.to_dict()
for key, status in sorted(self._certified.items())
},
}
# Route formation itself lives in `server.py` (`_select_route`, `_enumerate_routes`,
# `_find_pinned_route`): candidates are partitioned by the exact fingerprint this
# module derives, so a route can only ever be assembled inside one identity. A
# second route gate here would be a second copy of that policy to keep in step —
# the same reason the node module holds no certification authority.

View File

@@ -45,7 +45,7 @@ import urllib.parse
import urllib.request
import uuid
from collections import deque
from dataclasses import dataclass, field
from dataclasses import dataclass, field, replace
from importlib.resources import files
from pathlib import Path
from typing import Any
@@ -60,6 +60,7 @@ from .capability import (
STATE_ADMITTED,
STATE_MODEL_MISMATCH,
STATE_SHARD_MISMATCH,
STATE_UNCERTIFIED,
CapabilityState,
absent_state,
evaluate_report,
@@ -84,6 +85,13 @@ from .routing_stats import (
)
from .model_files import files_for_layer_range, snapshot_dir_for_repo
from .raft import RaftNode
from .recipe import (
CertificationLedger,
DistributedForwardEvidence,
PresentedIdentity,
RecipeIdentityError,
RecipeStatus,
)
_CONSOLE_LIMIT = 300
@@ -792,6 +800,7 @@ def _capability_from_registration(
hf_repo: str | None,
shard_start: int | None,
shard_end: int | None,
recipe_certifications: CertificationLedger,
) -> CapabilityState:
"""The tracker's verdict on the proof carried by one registration payload.
@@ -811,6 +820,7 @@ def _capability_from_registration(
declared_recipe_version=(
recipe_version if isinstance(recipe_version, str) else None
),
ledger=recipe_certifications,
)
@@ -830,6 +840,11 @@ def _admitted_nodes(nodes: list["_NodeEntry"], policy: str | None) -> list["_Nod
return [node for node in nodes if _capability_routable(node, effective)]
def _route_identity_partition(node: "_NodeEntry") -> tuple[str, ...]:
fingerprint = _node_admission(node).fingerprint
return ("legacy",) if fingerprint is None else ("exact", *fingerprint)
def _select_route(
nodes: list[_NodeEntry],
required_start: int,
@@ -853,29 +868,51 @@ def _select_route(
],
key=lambda n: (n.shard_start, -n.shard_end), # type: ignore[operator]
)
route: list[_NodeEntry] = []
covered_up_to = required_start - 1
def _routing_score(node: "_NodeEntry") -> float:
return _effective_throughput(node, model) * _reputation_multiplier(node, contracts)
while covered_up_to < required_end:
best: _NodeEntry | None = None
for node in candidates:
if node.shard_start <= covered_up_to + 1 and node.shard_end > covered_up_to:
if best is None:
best = node
elif node.shard_end > best.shard_end:
best = node
elif node.shard_end == best.shard_end and _routing_score(node) > _routing_score(best):
best = node
if best is None:
missing = covered_up_to + 1
return [], f"no route available: no registered node covers layer {missing}"
route.append(best)
covered_up_to = best.shard_end
candidates = [n for n in candidates if n is not best]
partitions: dict[tuple[str, ...], list[_NodeEntry]] = {}
for node in candidates:
partitions.setdefault(_route_identity_partition(node), []).append(node)
complete: list[list[_NodeEntry]] = []
furthest = required_start - 1
for partition in partitions.values():
pool = list(partition)
route: list[_NodeEntry] = []
covered_up_to = required_start - 1
while covered_up_to < required_end:
best: _NodeEntry | None = None
for node in pool:
if node.shard_start <= covered_up_to + 1 and node.shard_end > covered_up_to:
if best is None:
best = node
elif node.shard_end > best.shard_end:
best = node
elif (
node.shard_end == best.shard_end
and _routing_score(node) > _routing_score(best)
):
best = node
if best is None:
break
route.append(best)
covered_up_to = best.shard_end
pool = [node for node in pool if node is not best]
furthest = max(furthest, covered_up_to)
if covered_up_to >= required_end:
complete.append(route)
if not complete:
return [], f"no route available: no registered node covers layer {furthest + 1}"
route = max(
complete,
key=lambda candidate: (
min(_routing_score(node) for node in candidate),
-len(candidate),
),
)
return route, ""
@@ -911,7 +948,12 @@ def _enumerate_routes(
for head in heads:
route = [head]
covered_up_to = head.shard_end
pool = [n for n in sharded if n is not head]
head_partition = _route_identity_partition(head)
pool = [
node
for node in sharded
if node is not head and _route_identity_partition(node) == head_partition
]
while covered_up_to < required_end:
best = None
for n in pool:
@@ -2056,14 +2098,24 @@ def _find_pinned_route(
hop_count: int,
) -> list[_NodeEntry] | None:
"""First combination of exactly ``hop_count`` distinct nodes covering the
layer range, where every node extends coverage (US-030 benchmark routes)."""
layer range, where every node extends coverage (US-030 benchmark routes).
Benchmark combos run real inference, so they obey the same DGR-003 rule as
every other route builder: one route, one exact identity. A combo that mixed
two fingerprints — or an exact Shard with a legacy one — would measure a
numerically incoherent route and record the garbage as a benchmark.
"""
for combo in itertools.permutations(nodes, hop_count):
covered = required_start - 1
valid = True
partition = _route_identity_partition(combo[0])
for candidate in combo:
if candidate.shard_start is None or candidate.shard_end is None:
valid = False
break
if _route_identity_partition(candidate) != partition:
valid = False
break
if candidate.shard_start > covered + 1 or candidate.shard_end <= covered:
valid = False
break
@@ -2847,9 +2899,11 @@ class _TrackerHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
relay_status: dict | None = None,
test_runner: "TestRunManager | None" = None,
capability_policy: str | None = None,
recipe_certifications: CertificationLedger | None = None,
) -> None:
super().__init__(addr, handler)
self.registry = registry
self.recipe_certifications = recipe_certifications or CertificationLedger()
self.capability_policy = normalize_policy(
capability_policy if capability_policy is not None else policy_from_env()
)
@@ -4555,6 +4609,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
hf_repo=hf_repo,
shard_start=shard_start,
shard_end=shard_end,
recipe_certifications=server.recipe_certifications,
)
node_id = _node_id_for_registration(
@@ -6531,6 +6586,7 @@ class TrackerServer:
self._embedded_relay: Any | None = None
self._embedded_relay_actual_port: int | None = None
self._registry: dict[str, _NodeEntry] = {}
self._recipe_certifications = CertificationLedger()
self._lock = threading.Lock()
self._server: _TrackerHTTPServer | None = None
self._thread: threading.Thread | None = None
@@ -6640,6 +6696,84 @@ class TrackerServer:
self._test_runner: TestRunManager | None = test_runner
self.port: int | None = None
def certify_recipe(
self,
identity: PresentedIdentity,
evidence: DistributedForwardEvidence,
) -> RecipeStatus:
"""Promote one exact recipe from tracker-recorded distributed evidence.
This is the tracker's only certification authority. The evidence names
the nodes that served the forward; this method refuses to take any of
them on the evidence's word. For each one it goes back to what *this
tracker* re-derived at admission and requires the evidence to describe
the same Shard: same route fingerprint, same registered range, and the
same shard binding digest — the derivative's own bytes and exact range.
Without the binding check, a route fingerprint is range-independent by
design (:mod:`meshnet_tracker.recipe`), so evidence could name the right
recipe while describing splits nobody on the route was actually serving,
and the certification would attach to blobs that never ran.
"""
with self._lock:
for node_id, participant in zip(
evidence.node_ids, evidence.participants, strict=True
):
node = self._registry.get(node_id)
if node is None:
raise RecipeIdentityError(
f"certification participant {node_id!r} is not registered"
)
admitted = node.capability
if admitted.fingerprint != identity.key:
raise RecipeIdentityError(
f"certification participant {node_id!r} is not admitted under "
"the promoted fingerprint"
)
if admitted.state not in (STATE_UNCERTIFIED, STATE_ADMITTED):
raise RecipeIdentityError(
f"certification participant {node_id!r} has capability state "
f"{admitted.state!r}"
)
registered_range = (
node.shard_start,
None if node.shard_end is None else node.shard_end + 1,
)
if registered_range != (
participant.shard_start,
participant.shard_end,
):
raise RecipeIdentityError(
f"certification participant {node_id!r} range differs from its "
"registered Shard range"
)
# The exact bytes and range this node was admitted on. A node that
# registered without an identity has no binding, and cannot be a
# participant in an exact certification at all.
if admitted.shard_binding_digest is None:
raise RecipeIdentityError(
f"certification participant {node_id!r} registered no exact "
"Shard binding; it cannot certify a recipe"
)
if admitted.shard_binding_digest != participant.shard_binding_digest:
raise RecipeIdentityError(
f"certification participant {node_id!r} presents a Shard "
"binding this tracker did not admit it on; the evidence "
"describes different derivative bytes or a different range"
)
status = self._recipe_certifications.certify(identity, evidence)
for node in self._registry.values():
if (
node.capability.state == STATE_UNCERTIFIED
and node.capability.fingerprint == identity.key
):
node.capability = replace(
node.capability.with_state(STATE_ADMITTED, status.detail),
certification=status.status,
)
return status
def _start_embedded_relay(self) -> dict:
"""Start the shared RelayServer class in-process for tracker+relay deployments."""
if not self._embedded_relay_enabled:
@@ -6725,6 +6859,7 @@ class TrackerServer:
relay_status=http_relay_status,
test_runner=self._test_runner,
capability_policy=self._capability_policy,
recipe_certifications=self._recipe_certifications,
)
self.port = self._server.server_address[1]
@@ -6983,6 +7118,7 @@ class TrackerServer:
hf_repo=payload.get("hf_repo"),
shard_start=shard_start,
shard_end=shard_end,
recipe_certifications=self._recipe_certifications,
),
)
with self._lock:

View File

@@ -0,0 +1,153 @@
#!/usr/bin/env python
"""Generate (or verify) the committed DGR-003 fingerprint conformance vectors.
The node and the tracker derive artifact, recipe and Shard-binding digests from
*separate* implementations on purpose: an admission gate that shares code with
the thing it admits is not an independent check. The cost of that independence
is drift — two canonicalizers that quietly stop agreeing would not fail, they
would silently stop forming routes, or worse, silently form wrong ones.
``tests/data/recipe_fingerprint_vectors.json`` is what makes drift loud. It is a
language-neutral artifact — canonical input blocks, expected digests, and the
serialized DGR-002 ``Fingerprint`` bytes — that the node tests, the tracker tests
and (later) the native C++ worker all check themselves against.
python scripts/gen_recipe_fingerprint_vectors.py --check # CI: no drift
python scripts/gen_recipe_fingerprint_vectors.py # rewrite vectors
Rewriting is a deliberate act: if this changes a digest, it changed the wire
contract, and every node and tracker in the fleet has to agree at the same time.
"""
from __future__ import annotations
import argparse
import json
import pathlib
import sys
_ROOT = pathlib.Path(__file__).resolve().parent.parent
sys.path[:0] = [str(_ROOT / "packages" / "node"), str(_ROOT / "packages" / "tracker")]
from meshnet_node.runtime_recipe import ( # noqa: E402
ArtifactIdentity,
DerivativeBinding,
RuntimeRecipe,
ShardIdentity,
)
from meshnet_tracker.recipe import parse_identity # noqa: E402
VECTORS = _ROOT / "tests" / "data" / "recipe_fingerprint_vectors.json"
SCHEMA_VERSION = 1
_RECIPE = RuntimeRecipe(
weight_quantization="Q4_K_M",
activation_dtype="bfloat16",
compute_dtype="float32",
kv_dtype="q8_0",
kv_layout="paged-v1",
tokenizer_revision="0123456789abcdef",
architecture_adapter="llama/range-v1",
backend_id="llama.cpp",
runtime_version="llama.cpp@deadbeef+meshnet.1",
recipe_id="example-gguf",
recipe_version="1",
catalogue_version="2026.07.1",
)
_SOURCE = "a" * 64
_SPLIT_BYTES = "c" * 64
_CONFIG = "b" * 64
def _cases() -> list[tuple[str, str, ShardIdentity]]:
whole = ShardIdentity(
ArtifactIdentity(
"example/model", "0123456789abcdef", _SOURCE, "dense-llama", _CONFIG, 8
),
_RECIPE,
0,
4,
)
# The same recipe on the same source, held as a split: identical route
# fingerprint, different Shard binding. Both halves of that are contract.
derivative = ShardIdentity(
ArtifactIdentity(
"example/model",
"0123456789abcdef",
_SPLIT_BYTES,
"dense-llama",
_CONFIG,
8,
DerivativeBinding(_SOURCE, 4, 8),
),
_RECIPE,
4,
8,
)
return [
("example-v1", "An undivided artifact: content digest is the source digest.", whole),
(
"example-v1-derivative",
"A split of the same source: same fingerprint, different Shard binding.",
derivative,
),
]
def build() -> dict:
vectors = []
for name, description, identity in _cases():
block = identity.to_dict()
presented = parse_identity(block)
# The two implementations must already agree before this is committed.
assert identity.fingerprint.to_dict() == presented.fingerprint_dict(), name
assert identity.shard_binding_digest == presented.shard_binding_digest, name
vectors.append(
{
"name": name,
"description": description,
"identity": block,
"fingerprint": identity.fingerprint.to_dict(),
"shard_binding_digest": identity.shard_binding_digest,
"fingerprint_proto_hex": identity.fingerprint.to_proto()
.SerializeToString(deterministic=True)
.hex(),
}
)
return {"schema_version": SCHEMA_VERSION, "vectors": vectors}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--check",
action="store_true",
help="fail if the committed vectors differ from what this code derives",
)
args = parser.parse_args()
built = json.dumps(build(), indent=2, sort_keys=True) + "\n"
if not args.check:
VECTORS.write_text(built, encoding="utf-8")
print(f"wrote {VECTORS.relative_to(_ROOT)}")
return 0
committed = VECTORS.read_text(encoding="utf-8")
if committed != built:
print(
f"{VECTORS.relative_to(_ROOT)} is stale: the identity implementation no "
"longer derives the committed digests.\nIf that change was intended, it "
"is a wire-contract change — rerun without --check and roll out node and "
"tracker together.",
file=sys.stderr,
)
return 1
print(f"{VECTORS.relative_to(_ROOT)} matches the identity implementation")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,268 @@
#!/usr/bin/env python3
"""Materialize, verify, build, and smoke-test DGR-004's llama.cpp pin.
This tool deliberately owns only a source dependency boundary. It never
downloads a model, invokes inference, or interprets generated text.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import pathlib
import shutil
import subprocess
import sys
from typing import Any
ROOT = pathlib.Path(__file__).resolve().parents[1]
LLAMA_DIR = ROOT / "packages/node/native/llama"
LOCK_PATH = LLAMA_DIR / "UPSTREAM_LOCK.json"
PATCH_DIR = LLAMA_DIR / "patches"
def _cmake() -> str:
"""Use an explicit override, PATH, or the active Python environment."""
configured = os.environ.get("CMAKE")
if configured:
return configured
on_path = shutil.which("cmake")
if on_path:
return on_path
sibling = pathlib.Path(sys.executable).parent / "cmake"
if sibling.is_file():
return str(sibling)
raise DependencyError("cmake is unavailable; set CMAKE or activate the project toolchain")
class DependencyError(RuntimeError):
"""A fail-closed reproducibility or source-integrity failure."""
def _run(*args: str, cwd: pathlib.Path | None = None) -> str:
try:
completed = subprocess.run(
args,
cwd=cwd,
check=True,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
except FileNotFoundError as error:
raise DependencyError(f"required executable is unavailable: {args[0]}") from error
except subprocess.CalledProcessError as error:
detail = (error.stderr or error.stdout).strip()
raise DependencyError(f"command failed: {' '.join(args)}\n{detail}") from error
return completed.stdout.strip()
def _load_lock() -> dict[str, Any]:
try:
lock = json.loads(LOCK_PATH.read_text())
except (OSError, json.JSONDecodeError) as error:
raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error
required = {
"upstream", "commit", "commit_tree", "patched_tree", "patch_series",
"required_upstream_blobs", "patched_paths", "build",
}
missing = sorted(required - lock.keys())
if missing:
raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}")
commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
if lock["commit"] != commit_file or len(commit_file) != 40:
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on a full commit")
return lock
def _patches(lock: dict[str, Any]) -> list[pathlib.Path]:
series = [line for line in (PATCH_DIR / "series").read_text().splitlines() if line]
if series != lock["patch_series"] or series != sorted(series) or not series:
raise DependencyError("patches/series is empty, unordered, or disagrees with UPSTREAM_LOCK.json")
sums: dict[str, str] = {}
for line in (PATCH_DIR / "SHA256SUMS").read_text().splitlines():
if line and not line.startswith("#"):
digest, name = line.split(maxsplit=1)
sums[name] = digest
patches: list[pathlib.Path] = []
for name in series:
patch = PATCH_DIR / name
if not patch.is_file():
raise DependencyError(f"patch listed in series is missing: {name}")
actual = hashlib.sha256(patch.read_bytes()).hexdigest()
if sums.get(name) != actual:
raise DependencyError(f"patch digest mismatch for {name}: expected {sums.get(name)}, got {actual}")
patches.append(patch)
return patches
def _git(source: pathlib.Path, *args: str) -> str:
return _run("git", "-C", str(source), *args)
def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None:
if not (source / ".git").exists():
raise DependencyError(f"not a materialized git checkout: {source}")
if _git(source, "rev-parse", "HEAD") != lock["commit"]:
raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit")
if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]:
raise DependencyError("upstream drift: checkout tree does not equal the locked tree")
if require_clean and _git(source, "status", "--porcelain"):
raise DependencyError("local edits detected in materialized llama.cpp checkout")
for relative, expected in lock["required_upstream_blobs"].items():
actual = _git(source, "rev-parse", f"HEAD:{relative}")
if actual != expected:
raise DependencyError(
f"upstream ABI/context drift for {relative}: expected {expected}, got {actual}"
)
if not (source / "LICENSE").is_file():
raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution")
def materialize(source: pathlib.Path, repository: str) -> None:
lock = _load_lock()
_patches(lock)
if source.exists():
raise DependencyError(f"destination already exists; refusing to reuse possibly edited source: {source}")
source.parent.mkdir(parents=True, exist_ok=True)
_run("git", "clone", "--no-checkout", repository, str(source))
_git(source, "checkout", "--detach", lock["commit"])
_verify_source(source, lock, require_clean=True)
def apply(source: pathlib.Path) -> None:
lock = _load_lock()
patches = _patches(lock)
_verify_source(source, lock, require_clean=True)
for patch in patches:
_git(source, "apply", "--check", str(patch))
_git(source, "apply", "--index", str(patch))
_verify_patched_source(source, lock)
def _verify_patched_source(source: pathlib.Path, lock: dict[str, Any]) -> None:
if _git(source, "diff", "--quiet"):
raise DependencyError("local unstaged edits detected after applying patch stack")
changed_paths = _git(source, "diff", "--cached", "--name-only").splitlines()
if changed_paths != lock["patched_paths"]:
raise DependencyError(f"patched paths drifted: expected {lock['patched_paths']}, got {changed_paths}")
if _git(source, "write-tree") != lock["patched_tree"]:
raise DependencyError("patched source tree differs from the locked patch stack")
if _git(source, "diff", "--name-only"):
raise DependencyError("local unstaged edits detected after applying patch stack")
untracked = _git(source, "ls-files", "--others", "--exclude-standard").splitlines()
if untracked:
raise DependencyError(f"untracked files detected after applying patch stack: {untracked}")
def build(source: pathlib.Path, build_dir: pathlib.Path) -> pathlib.Path:
lock = _load_lock()
_patches(lock)
_verify_source(source, lock, require_clean=False)
_verify_patched_source(source, lock)
expected_marker = source / "cmake/meshnet-patch-stack.cmake"
if not expected_marker.is_file():
raise DependencyError("patch stack is not applied: Meshnet CMake marker is absent")
if build_dir.exists():
raise DependencyError(f"build directory already exists; use reproduce for a clean rebuild: {build_dir}")
flags = lock["build"]["configure_flags"]
cmake = _cmake()
_run(cmake, "-G", lock["build"]["generator"], "-S", str(source), "-B", str(build_dir), *flags)
for target in lock["build"]["native_targets"]:
_run(cmake, "--build", str(build_dir), "--target", target, "-j2")
metadata = {
"commit": lock["commit"],
"commit_tree": lock["commit_tree"],
"patches": {patch.name: hashlib.sha256(patch.read_bytes()).hexdigest() for patch in _patches(lock)},
"configure_flags": flags,
"cmake": _run(cmake, "--version").splitlines()[0],
"cxx": _run("c++", "--version").splitlines()[0],
"model_downloads": False,
"semantic_certification": False,
}
(build_dir / "meshnet-build-metadata.json").write_text(json.dumps(metadata, indent=2, sort_keys=True) + "\n")
return build_dir / lock["build"]["smoke_binary"]
def smoke(binary: pathlib.Path) -> None:
if not binary.is_file():
raise DependencyError(f"native smoke binary is missing: {binary}")
smoke_config = _load_lock()["build"]
output = _run(str(binary), *smoke_config["smoke_args"])
expected = smoke_config["smoke_output_token"]
if expected not in output.lower():
raise DependencyError(f"native smoke output did not contain {expected!r}: {output}")
print(output)
def reproduce(work_dir: pathlib.Path, repository: str) -> None:
resolved = work_dir.resolve()
build_root = (ROOT / "build").resolve()
if build_root not in resolved.parents:
raise DependencyError(f"--work-dir must be below {build_root}: {resolved}")
if resolved.exists():
raise DependencyError(
f"work directory already exists; refusing to erase possible local edits: {resolved}"
)
source = resolved / "source"
build_dir = resolved / "build"
materialize(source, repository)
apply(source)
smoke(build(source, build_dir))
def inspect() -> None:
lock = _load_lock()
patches = _patches(lock)
print(json.dumps({
"commit": lock["commit"],
"patch_count": len(patches),
"patches": [patch.name for patch in patches],
"model_downloads": False,
"semantic_certification": False,
"glm_stock_limitations": lock["stock_glm_limitations"],
}, indent=2, sort_keys=True))
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
subcommands = parser.add_subparsers(dest="command", required=True)
subcommands.add_parser("inspect")
materialize_parser = subcommands.add_parser("materialize")
materialize_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
materialize_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
apply_parser = subcommands.add_parser("apply")
apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
build_parser = subcommands.add_parser("build")
build_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
build_parser.add_argument("--build-dir", type=pathlib.Path, required=True)
smoke_parser = subcommands.add_parser("smoke")
smoke_parser.add_argument("--binary", type=pathlib.Path, required=True)
reproduce_parser = subcommands.add_parser("reproduce")
reproduce_parser.add_argument("--work-dir", type=pathlib.Path, default=ROOT / "build/dgr-004-smoke")
reproduce_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
args = parser.parse_args()
try:
if args.command == "inspect":
inspect()
elif args.command == "materialize":
materialize(args.source_dir, args.source_repository)
elif args.command == "apply":
apply(args.source_dir)
elif args.command == "build":
build(args.source_dir, args.build_dir)
elif args.command == "smoke":
smoke(args.binary)
else:
reproduce(args.work_dir, args.source_repository)
except DependencyError as error:
print(f"DGR-004 dependency error: {error}", file=sys.stderr)
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,109 @@
{
"schema_version": 1,
"vectors": [
{
"description": "An undivided artifact: content digest is the source digest.",
"fingerprint": {
"catalogue_version": "2026.07.1",
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
},
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31",
"identity": {
"artifact": {
"architecture": "dense-llama",
"architecture_digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"artifact_id": "example/model",
"content_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"derived_from": null,
"layer_count": 8,
"revision": "0123456789abcdef"
},
"fingerprint": {
"catalogue_version": "2026.07.1",
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
},
"recipe": {
"activation_dtype": "bfloat16",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"boundary_schema_version": 1,
"catalogue_version": "2026.07.1",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"protocol_schema_version": 1,
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
"tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M"
},
"schema_version": 1,
"shard_end": 4,
"shard_start": 0
},
"name": "example-v1",
"shard_binding_digest": "f62d1f76cc18b548782c02850e05c2634da55c0e686c43b9f687bdee7bdefe19"
},
{
"description": "A split of the same source: same fingerprint, different Shard binding.",
"fingerprint": {
"catalogue_version": "2026.07.1",
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
},
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31",
"identity": {
"artifact": {
"architecture": "dense-llama",
"architecture_digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"artifact_id": "example/model",
"content_digest": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
"derived_from": {
"shard_end": 8,
"shard_start": 4,
"source_artifact_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
},
"layer_count": 8,
"revision": "0123456789abcdef"
},
"fingerprint": {
"catalogue_version": "2026.07.1",
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
},
"recipe": {
"activation_dtype": "bfloat16",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"boundary_schema_version": 1,
"catalogue_version": "2026.07.1",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"protocol_schema_version": 1,
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
"tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M"
},
"schema_version": 1,
"shard_end": 8,
"shard_start": 4
},
"name": "example-v1-derivative",
"shard_binding_digest": "55271611eb63cb81088b8109133f1dff845352298994fc8e9c5824a6a01c83e0"
}
]
}

View File

@@ -0,0 +1,233 @@
"""Distributed GGUF shard load integration test.
Downloads a small dense-Llama GGUF (TinyLlama 1.1B Q4_K_M ~670 MB),
loads it in shard ranges via the meshnet-range-loader C wrapper, registers
each shard with a live TrackerServer, and verifies routing, range reporting,
and memory scaling.
Set MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 to run.
Evidence class: real-model integration. Downloads ~670 MB on first run.
"""
from __future__ import annotations
import json
import os
import pathlib
import subprocess
import sys
import urllib.error
import urllib.request
import pytest
ROOT = pathlib.Path(__file__).resolve().parent.parent
sys.path[:0] = [
str(ROOT / "packages" / "tracker"),
str(ROOT / "packages" / "node"),
str(ROOT / "packages" / "contracts"),
]
from meshnet_tracker.server import TrackerServer # noqa: E402 — sys.path prepended above
# Only run when explicitly enabled
pytestmark = pytest.mark.skipif(
"MESHNET_ENABLE_REAL_INFERENCE_TESTS" not in os.environ,
reason="set MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 to download and load a real GGUF",
)
# ---------------------------------------------------------------------------
# Model configuration
# ---------------------------------------------------------------------------
HF_REPO = "TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF"
GGUF_FILE = "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf"
MODEL_URL = f"https://huggingface.co/{HF_REPO}/resolve/main/{GGUF_FILE}"
EXPECTED_LAYERS = 22
GGUF_FLAVOR = GGUF_FILE.replace(".gguf", "")
CACHE_DIR = ROOT / ".cache" / "gguf-models"
LOADER = ROOT / "build" / "dgr-004-final" / "build" / "bin" / "meshnet-range-loader"
LLAMA_LIB_DIR = LOADER.parent
# ---------------------------------------------------------------------------
# Loading helper
# ---------------------------------------------------------------------------
def load_shard(gguf_path: str, start: int, end: int) -> dict:
"""Load a shard range of the GGUF via the C wrapper and return JSON report."""
env = os.environ.copy()
env["LD_LIBRARY_PATH"] = str(LLAMA_LIB_DIR)
result = subprocess.run(
[str(LOADER), gguf_path, str(start), str(end)],
capture_output=True, text=True, timeout=120, env=env,
)
if result.returncode != 0:
# Parse JSON from stdout even on error if it exists
if result.stdout.strip():
try:
return json.loads(result.stdout)
except json.JSONDecodeError:
pass
raise RuntimeError(
f"meshnet-range-loader [{start}, {end}) failed (exit {result.returncode}):\n"
f"stderr: {result.stderr[:500]}"
)
return json.loads(result.stdout)
# ---------------------------------------------------------------------------
# Download and cache
# ---------------------------------------------------------------------------
def _ensure_model() -> pathlib.Path:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
model_path = CACHE_DIR / GGUF_FILE
if model_path.exists() and model_path.stat().st_size > 600 * 1024 * 1024:
return model_path
print(f"Downloading {MODEL_URL} (~670 MB)...", file=sys.stderr)
urllib.request.urlretrieve(MODEL_URL, model_path)
actual_mb = model_path.stat().st_size / (1024 * 1024)
print(f"Downloaded {GGUF_FLAVOR}: {actual_mb:.0f} MB", file=sys.stderr)
return model_path
# ---------------------------------------------------------------------------
# Tracker helpers
# ---------------------------------------------------------------------------
def _post_json(url: str, data: dict) -> dict:
req = urllib.request.Request(
url,
data=json.dumps(data).encode(),
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=10) as r:
return json.loads(r.read())
def _get_json(url: str) -> dict:
with urllib.request.urlopen(url, timeout=10) as r:
return json.loads(r.read())
# ---------------------------------------------------------------------------
# Tests
# ---------------------------------------------------------------------------
def test_whole_model_load_and_report():
"""Load the full TinyLlama GGUF and verify metadata."""
gguf = _ensure_model()
report = load_shard(str(gguf), 0, EXPECTED_LAYERS)
assert report["ok"] is True
assert report["start_layer"] == 0
assert report["end_layer"] == EXPECTED_LAYERS
assert report["mapped_bytes"] > 0
assert report["resident_bytes"] >= report["mapped_bytes"]
def test_head_shard_is_less_than_full_model():
"""A head-only shard maps fewer bytes than the full model."""
gguf = _ensure_model()
head = load_shard(str(gguf), 0, 8)
full = load_shard(str(gguf), 0, EXPECTED_LAYERS)
assert head["mapped_bytes"] <= full["mapped_bytes"]
def test_tail_shard_maps_fewer_bytes_than_middle():
"""Fewer layers = fewer bytes (tail has 6 layers, middle has 8)."""
gguf = _ensure_model()
middle = load_shard(str(gguf), 8, 16)
tail = load_shard(str(gguf), 16, EXPECTED_LAYERS)
# Middle (8 layers) must map more than tail (6 layers)
assert middle["mapped_bytes"] > tail["mapped_bytes"]
def test_memory_scales_with_owned_range():
"""More layers = more resident bytes."""
gguf = _ensure_model()
small = load_shard(str(gguf), 0, 4)
large = load_shard(str(gguf), 0, 12)
assert large["mapped_bytes"] > small["mapped_bytes"]
assert large["resident_bytes"] > small["resident_bytes"]
def test_invalid_range_rejected():
"""Loading a range outside GGUF layer bounds fails closed."""
gguf = _ensure_model()
env = os.environ.copy()
env["LD_LIBRARY_PATH"] = str(LLAMA_LIB_DIR)
result = subprocess.run(
[str(LOADER), str(gguf), str(EXPECTED_LAYERS + 1), str(EXPECTED_LAYERS + 5)],
capture_output=True, text=True, timeout=30, env=env,
)
# Should fail with exit code 1 and an error message
assert result.returncode != 0
assert "error" in result.stderr.lower() or "ERROR" in result.stderr
def test_tracker_registers_shard_nodes():
"""Start a tracker, register multiple shard nodes, verify the console."""
gguf = _ensure_model()
tracker = TrackerServer(heartbeat_timeout=60.0)
tracker_port = tracker.start()
shards = [
("head", 0, 8),
("middle", 8, 16),
("tail", 16, EXPECTED_LAYERS),
]
registered_ids = []
try:
for name_tag, start, end in shards:
report = load_shard(str(gguf), start, end)
node_id = f"{GGUF_FLAVOR}-{name_tag}"
registered_ids.append(node_id)
_post_json(
f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
{
"node_id": node_id,
"endpoint": f"http://localhost:{10000 + start}",
"model": GGUF_FLAVOR,
"num_layers": report["end_layer"] - report["start_layer"],
"shard_start": report["start_layer"],
"shard_end": report["end_layer"],
"hardware_profile": {
"mapped_bytes": report["mapped_bytes"],
"resident_bytes": report["resident_bytes"],
},
"score": 1.0,
},
)
# Verify console shows all three registered nodes
console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
registered_eps = {
f"http://localhost:{10000 + start}" for _, start, _ in shards
}
found_eps = set()
for event in console.get("events", []):
if event.get("message") == "node registered":
ep = event.get("fields", {}).get("endpoint", "")
if ep in registered_eps:
found_eps.add(ep)
for _, start, _ in shards:
expected_ep = f"http://localhost:{10000 + start}"
assert expected_ep in found_eps, \
f"endpoint {expected_ep} not found in registration events"
finally:
tracker.stop()

View File

@@ -80,7 +80,7 @@ def snapshot_doc(snapshot):
@pytest.fixture
def contract_doc(contract):
return copy.deepcopy(dict(contract.raw))
return contract.to_dict()
# --------------------------------------------------------------------------
@@ -746,7 +746,7 @@ def test_the_contract_locks_the_roadmap_thresholds(contract):
assert contract.threshold("performance", "quality_pass_with_speed_fail_verdict") == "stop"
assert contract.threshold("reliability", "synthetic_workers_satisfy_alpha") is False
assert contract.threshold("storage", "forbidden_path_prefixes") == ["/home"]
assert contract.threshold("storage", "forbidden_path_prefixes") == ("/home",)
def test_the_contract_offers_only_alpha_or_stop(contract):
@@ -820,17 +820,26 @@ def test_a_dropped_acceptance_section_is_rejected(contract_doc):
parse_alpha_contract(contract_doc)
def test_resealing_a_mutated_contract_changes_its_digest(contract, contract_doc):
"""Tamper-evidence, not tamper-proofing: a rewrite is legal but never silent."""
def test_resealing_a_mutated_v1_contract_is_rejected(contract, contract_doc):
contract_doc["performance"]["min_median_decode_tokens_per_second"] = 0.05
resealed = seal_contract(contract_doc)
# It parses — nothing in-repo can stop a determined rewrite of file plus digest.
reparsed = parse_alpha_contract(resealed)
with pytest.raises(AlphaContractError, match="trusted pre-execution digest"):
parse_alpha_contract(resealed)
assert resealed["contract_sha256"] != contract.digest
# But the digest moved, so the change is a visible diff on a file whose entire
# purpose is to not change, and the contract_id still claims to be v1.
assert reparsed.digest != contract.digest
def test_parsed_contract_nested_state_is_immutable(contract):
with pytest.raises(TypeError):
contract.raw["performance"]["min_median_decode_tokens_per_second"] = 0.05
with pytest.raises(TypeError):
contract.target["reasoning_effort"] = "high"
def test_contract_to_dict_returns_an_isolated_mutable_copy(contract):
copied = contract.to_dict()
copied["performance"]["min_median_decode_tokens_per_second"] = 0.05
assert contract.threshold("performance", "min_median_decode_tokens_per_second") == 0.5
def test_an_unknown_threshold_cannot_be_invented_at_read_time(contract):

View File

@@ -0,0 +1,68 @@
"""Offline guards for DGR-004's pinned llama.cpp dependency boundary."""
from __future__ import annotations
import hashlib
import json
import pathlib
import subprocess
import sys
ROOT = pathlib.Path(__file__).resolve().parents[1]
LLAMA_DIR = ROOT / "packages/node/native/llama"
SCRIPT = ROOT / "scripts/llama_cpp_dependency.py"
def _sha256(path: pathlib.Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
lock = json.loads((LLAMA_DIR / "UPSTREAM_LOCK.json").read_text())
commit = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
patches = (LLAMA_DIR / "patches/series").read_text().splitlines()
sums = {
name: digest
for digest, name in (
line.split(maxsplit=1)
for line in (LLAMA_DIR / "patches/SHA256SUMS").read_text().splitlines()
if line and not line.startswith("#")
)
}
assert commit == lock["commit"]
assert len(commit) == 40
assert patches == lock["patch_series"]
assert patches == sorted(patches)
assert patches
for patch_name in patches:
patch = LLAMA_DIR / "patches" / patch_name
assert sums[patch_name] == _sha256(patch)
assert "Subject: [PATCH" in patch.read_text()
def test_dependency_script_reports_the_locked_boundary_without_network() -> None:
completed = subprocess.run(
[sys.executable, str(SCRIPT), "inspect"],
cwd=ROOT,
check=True,
capture_output=True,
text=True,
)
report = json.loads(completed.stdout)
assert report["commit"] == (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
assert report["patch_count"] == 2
assert report["model_downloads"] is False
assert report["semantic_certification"] is False
assert "dense" in report["glm_stock_limitations"].lower()
def test_patch_stack_does_not_contain_meshnet_control_plane_code() -> None:
forbidden = ("tracker", "route session", "grpc", "http", "billing", "wallet")
patch_text = "\n".join(
(LLAMA_DIR / "patches" / name).read_text().lower()
for name in (LLAMA_DIR / "patches/series").read_text().splitlines()
)
assert not any(term in patch_text for term in forbidden)

View File

@@ -0,0 +1,157 @@
"""DGR-003 production-native identity emission boundary tests."""
from __future__ import annotations
import pytest
from meshnet_node.doctor import DoctorSelection, validate_loaded_backend
from meshnet_node.native_backend import (
ImmutableArtifactPin,
NativeIdentityInputs,
NativeLoadedArtifactReport,
NativeNumericalRecipe,
NativeSessionRejected,
NativeWorkerBackendAdapter,
shard_identity_from_native_report,
)
from meshnet_node.native_protocol import SCHEMA_VERSION, pb
from meshnet_node.recipe_manifest import parse_recipe_manifest
from meshnet_tracker.capability import STATE_UNCERTIFIED, evaluate_report
def _digest(letter: str) -> str:
return letter * 64
def _inputs(**changes: object) -> NativeIdentityInputs:
report = NativeLoadedArtifactReport(
owned_start_layer=2,
owned_end_layer=6,
mapped_bytes=1024,
resident_bytes=768,
registered_bytes=640,
architecture="llama",
architecture_digest=_digest("a"),
layer_count=8,
)
recipe = NativeNumericalRecipe(
weight_quantization="Q4_K_M",
activation_dtype="bfloat16",
compute_dtype="float32",
kv_dtype="q8_0",
kv_layout="llama-kv-v1",
architecture_adapter="dense-llama-v1",
backend_id="llama-cpp",
runtime_version="llama.cpp:e920c523",
recipe_id="native",
recipe_version="1",
catalogue_version="2026.07.1",
)
values: dict[str, object] = {
"loaded_artifact": report,
"artifact_pin": ImmutableArtifactPin(
artifact_id="acme/llama.gguf",
revision="0123456789abcdef",
content_digest=_digest("b"),
),
"tokenizer_revision": "abcdef0123456789",
"numerical_recipe": recipe,
}
values.update(changes)
return NativeIdentityInputs(**values) # type: ignore[arg-type]
def _open(adapter: NativeWorkerBackendAdapter, **changes: object) -> pb.SessionOpen:
identity = adapter.identity
fields: dict[str, object] = {
"schema_version": SCHEMA_VERSION,
"route_session_id": "tracker-session",
"route_epoch": 4,
"fingerprint": identity.fingerprint.to_proto(),
"shard_range": pb.ShardRange(
start_layer=identity.shard_start,
end_layer=identity.shard_end,
effective_start_layer=identity.shard_start,
),
}
fields.update(changes)
return pb.SessionOpen(**fields) # type: ignore[arg-type]
def test_native_identity_uses_loaded_report_not_a_caller_range():
identity = shard_identity_from_native_report(_inputs())
assert (identity.shard_start, identity.shard_end) == (2, 6)
assert identity.artifact.architecture == "llama"
assert identity.artifact.layer_count == 8
def test_native_identity_requires_an_immutable_pin_and_gguf_range():
with pytest.raises(Exception, match="moving reference"):
shard_identity_from_native_report(
_inputs(artifact_pin=ImmutableArtifactPin("a", "main", _digest("b")))
)
with pytest.raises(Exception, match="outside GGUF"):
NativeLoadedArtifactReport(0, 9, 1, 1, 1, "llama", _digest("a"), 8)
def test_native_worker_rejects_bad_session_open_before_session_acceptance():
adapter = NativeWorkerBackendAdapter(_inputs())
accepted = adapter.on_session_open(
_open(adapter), expected_route_session_id="tracker-session", expected_route_epoch=4
)
assert accepted.fingerprint.SerializeToString() == adapter.identity.fingerprint.to_proto().SerializeToString()
with pytest.raises(NativeSessionRejected) as rejected:
adapter.on_session_open(
_open(adapter, route_epoch=5),
expected_route_session_id="tracker-session",
expected_route_epoch=4,
)
assert rejected.value.error.code == pb.ERROR_CODE_EPOCH_STALE
def test_doctor_emits_native_identity_but_keeps_legacy_backend_dark():
manifest = parse_recipe_manifest(
{"schema_version": 1, "catalogue_version": "2026.07.1", "recipes": [
{"id": "native", "version": "1", "backend_id": "llama-cpp"}
]}
)
selection = DoctorSelection("acme/llama.gguf", 2, 5)
native = NativeWorkerBackendAdapter(_inputs())
# The probe needs only the normal backend shape; identity is supplied by the adapter.
native.hidden_size = 8
native.is_head = False
native.is_tail = False
native.device = "cpu"
native.forward_bytes = lambda *args, **kwargs: type("Payload", (), {"body": b"x", "shape": [1]})()
result = validate_loaded_backend(native, selection, manifest.recipes[0], manifest)
assert result.report.identity == native.identity
assert result.report.model.revision == native.identity.artifact.revision
assert result.report.model.config_fingerprint == "sha256:" + _digest("a")
assert result.report.backend.quantization == "Q4_K_M"
assert (result.report.shard.start, result.report.shard.end) == (2, 5)
state = evaluate_report(
result.report.to_dict(),
model_matches=lambda value: value == "acme/llama.gguf",
advertised_model="acme/llama.gguf",
shard_start=2,
shard_end=5,
declared_recipe_id="native",
declared_recipe_version="1",
now=result.report.validated_at,
)
assert state.state == STATE_UNCERTIFIED
class Legacy:
hidden_size = 8
is_head = False
is_tail = False
device = "cpu"
@staticmethod
def forward_bytes(*args, **kwargs):
return type("Payload", (), {"body": b"x", "shape": [1]})()
legacy = validate_loaded_backend(Legacy(), selection, manifest.recipes[0], manifest)
assert legacy.report.identity is None

View File

@@ -0,0 +1,786 @@
"""Deterministic DGR-003 identity and tracker-admission conformance tests."""
from __future__ import annotations
from dataclasses import replace
import json
from pathlib import Path
import time
import pytest
from meshnet_node.native_protocol import SCHEMA_VERSION, pb
from meshnet_node.runtime_recipe import (
ArtifactIdentity,
CompatibilityFingerprint,
DerivativeBinding,
RecipeIdentityError,
RuntimeRecipe,
ShardIdentity,
check_handshake,
check_session_open,
check_route,
handshake_error,
)
from meshnet_tracker.capability import (
POLICY_COMPAT,
POLICY_ENFORCE,
CapabilityState,
STATE_ADMITTED,
STATE_FINGERPRINT_MISMATCH,
STATE_MODEL_MISMATCH,
STATE_RECIPE_MISMATCH,
STATE_UNCERTIFIED,
absent_state,
evaluate_report,
)
from meshnet_tracker.server import (
TrackerServer,
_capability_from_registration,
_find_pinned_route,
_NodeEntry,
_select_route,
)
from meshnet_tracker.recipe import (
CertificationLedger,
DistributedForwardEvidence,
RecipeIdentityError as TrackerRecipeIdentityError,
parse_identity,
)
VECTORS = Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json"
def _digest(char: str) -> str:
return char * 64
def _recipe(**changes: object) -> RuntimeRecipe:
fields: dict[str, object] = {
"weight_quantization": "Q4_K_M",
"activation_dtype": "bfloat16",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"tokenizer_revision": "0123456789abcdef",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
"recipe_id": "example-gguf",
"recipe_version": "1",
"catalogue_version": "2026.07.1",
}
fields.update(changes)
return RuntimeRecipe(**fields) # type: ignore[arg-type]
def _identity(start: int = 0, end: int = 4, **recipe_changes: object) -> ShardIdentity:
"""A Shard of the whole-model artifact: every node holds the same file."""
return ShardIdentity(
ArtifactIdentity(
"example/model",
"0123456789abcdef",
_digest("a"),
"dense-llama",
_digest("b"),
8,
),
_recipe(**recipe_changes),
start,
end,
)
def _split(start: int, end: int, content: str, **recipe_changes: object) -> ShardIdentity:
"""A derivative: its own bytes, bound to the exact source it was cut from."""
return ShardIdentity(
ArtifactIdentity(
"example/model",
"0123456789abcdef",
_digest(content),
"dense-llama",
_digest("b"),
8,
DerivativeBinding(_digest("a"), start, end),
),
_recipe(**recipe_changes),
start,
end,
)
def _report(identity: ShardIdentity) -> dict:
return {
"schema_version": 1,
"model": {
"model_id": "example/model",
"revision": identity.artifact.revision,
"config_fingerprint": "sha256:" + identity.artifact.architecture_digest,
},
"shard": {"start": identity.shard_start, "end": identity.shard_end - 1},
"recipe": identity.recipe.to_dict() | {
"recipe_id": identity.recipe.recipe_id,
"recipe_version": identity.recipe.recipe_version,
"catalogue_version": identity.recipe.catalogue_version,
},
"backend": {"backend_id": "llama.cpp", "device": "test"},
"status": "passed",
"validated_at": 100.0,
"duration_ms": 1,
"diagnostics": [],
"identity": identity.to_dict(),
}
def _evaluate(report: dict, **kwargs: object):
kwargs.setdefault("shard_start", 0)
kwargs.setdefault("shard_end", 3)
return evaluate_report(
report,
model_matches=lambda model: model == "example/model",
advertised_model="example/model",
now=100.0,
**kwargs, # type: ignore[arg-type]
)
def _register(tracker: TrackerServer, node_id: str, identity: ShardIdentity) -> _NodeEntry:
"""Register one node exactly as the HTTP path does: its own report, the tracker's ledger."""
report = _report(identity)
report["validated_at"] = time.time()
# The registry range is end-inclusive; the identity's is end-exclusive.
start, end = identity.shard_start, identity.shard_end - 1
capability = _capability_from_registration(
{"capability_report": report},
model="example/model",
hf_repo=None,
shard_start=start,
shard_end=end,
recipe_certifications=tracker._recipe_certifications,
)
entry = _NodeEntry(
node_id=node_id,
endpoint=f"http://{node_id}",
shard_start=start,
shard_end=end,
model="example/model",
shard_checksum=None,
hardware_profile={},
wallet_address=None,
score=1.0,
capability=capability,
)
tracker._registry[node_id] = entry
return entry
def _evidence(
*shards: ShardIdentity,
node_ids: tuple[str, ...] = ("physical-a", "physical-b"),
layer_count: int = 8,
fingerprint: tuple[str, str] | None = None,
**changes: object,
) -> DistributedForwardEvidence:
presented = tuple(parse_identity(s.to_dict()) for s in shards)
fields: dict[str, object] = {
"route_session_id": "session",
"route_epoch": 1,
"node_ids": node_ids,
"shard_ranges": tuple((s.shard_start, s.shard_end) for s in shards),
"tokens_generated": 1,
"layer_count": layer_count,
"fingerprint": fingerprint or presented[0].key,
"participants": presented,
}
fields.update(changes)
return DistributedForwardEvidence(**fields) # type: ignore[arg-type]
# --- Identity: the axes, the digests, and what they do and do not commit to ---
def test_node_and_tracker_share_the_committed_canonical_fingerprint_vector():
for vector in json.loads(VECTORS.read_text(encoding="utf-8"))["vectors"]:
expected = vector["fingerprint"]
identity = ShardIdentity.from_dict(vector["identity"])
presented = parse_identity(vector["identity"])
assert identity.fingerprint.to_dict() == expected, vector["name"]
assert presented.fingerprint_dict() == expected, vector["name"]
assert (
CompatibilityFingerprint.from_proto(
identity.fingerprint.to_proto()
).to_dict()
== expected
), vector["name"]
# The binding digest is derived independently on both sides too, so a
# node and a tracker cannot disagree about which bytes a Shard holds.
assert identity.shard_binding_digest == vector["shard_binding_digest"], vector["name"]
assert presented.shard_binding_digest == vector["shard_binding_digest"], vector["name"]
# The DGR-002 wire encoding is contract as well, so the native worker can
# be held to these bytes without reimplementing the JSON canonicalizer.
wire = identity.fingerprint.to_proto().SerializeToString(deterministic=True)
assert wire.hex() == vector["fingerprint_proto_hex"], vector["name"]
def test_committed_vectors_cover_a_whole_model_and_a_derivative_shard():
names = {v["name"] for v in json.loads(VECTORS.read_text(encoding="utf-8"))["vectors"]}
assert {"example-v1", "example-v1-derivative"} <= names
@pytest.mark.parametrize(
"axis,value",
[
("weight_quantization", "Q5_K_M"),
("activation_dtype", "float16"),
("compute_dtype", "float16"),
("kv_dtype", "float16"),
("kv_layout", "contiguous-v2"),
("tokenizer_revision", "fedcba9876543210"),
("architecture_adapter", "llama/range-v2"),
("backend_id", "other-backend"),
("runtime_version", "llama.cpp@other+meshnet.1"),
("boundary_schema_version", 2),
("protocol_schema_version", 2),
],
)
def test_every_recipe_axis_changes_the_fingerprint_and_blocks_route(axis, value):
left = _identity()
right = _identity(4, 8, **{axis: value})
assert left.fingerprint.key != right.fingerprint.key
assert check_route([left, right])
def test_split_artifact_is_bound_to_exact_source_and_owned_range():
source = _identity()
split = _split(4, 8, "c")
assert source.fingerprint.matches(split.fingerprint)
with pytest.raises(RecipeIdentityError):
ShardIdentity(split.artifact, split.recipe, 3, 8)
def test_derivative_bytes_and_range_have_a_separate_shard_binding_digest():
left = _split(0, 4, "c")
changed_bytes = _split(0, 4, "d")
changed_range = _split(4, 8, "c")
# Same route fingerprint — all three are the same recipe on the same source,
# which is exactly what route formation should conclude.
assert left.fingerprint.key == changed_bytes.fingerprint.key
assert left.fingerprint.key == changed_range.fingerprint.key
# Different bytes and different ranges are still separately pinned.
assert left.shard_binding_digest != changed_bytes.shard_binding_digest
assert left.shard_binding_digest != changed_range.shard_binding_digest
assert (
parse_identity(left.to_dict()).shard_binding_digest
== left.shard_binding_digest
)
def test_declared_digest_mismatch_is_recomputed_and_rejected_not_authenticated():
report = _report(_identity())
report["identity"]["fingerprint"]["runtime_recipe_digest"] = _digest("f")
assert _evaluate(report).state == STATE_FINGERPRINT_MISMATCH
# --- Capability admission: the identity block must match the proof it rides with ---
def test_capability_identity_must_match_the_proof_labels():
identity = _identity()
wrong_model = _report(identity)
wrong_model["identity"]["artifact"]["artifact_id"] = "other/model"
wrong_model["identity"]["fingerprint"] = None
assert _evaluate(wrong_model).state == STATE_MODEL_MISMATCH
wrong_recipe = _report(identity)
wrong_recipe["recipe"]["recipe_id"] = "other-recipe"
assert _evaluate(wrong_recipe).state == STATE_RECIPE_MISMATCH
wrong_backend = _report(identity)
wrong_backend["backend"]["backend_id"] = "other-backend"
assert _evaluate(wrong_backend).state == STATE_RECIPE_MISMATCH
wrong_quantization = _report(identity)
wrong_quantization["backend"]["quantization"] = "Q5_K_M"
assert _evaluate(wrong_quantization).state == STATE_RECIPE_MISMATCH
def test_capability_identity_must_match_the_proven_revision_and_config():
identity = _identity()
wrong_revision = _report(identity)
wrong_revision["model"]["revision"] = "fedcba9876543210"
assert _evaluate(wrong_revision).state == STATE_MODEL_MISMATCH
wrong_config = _report(identity)
wrong_config["model"]["config_fingerprint"] = "sha256:" + _digest("c")
assert _evaluate(wrong_config).state == STATE_MODEL_MISMATCH
missing_config = _report(identity)
del missing_config["model"]["config_fingerprint"]
assert _evaluate(missing_config).state == STATE_MODEL_MISMATCH
def test_an_exact_identity_without_a_ledger_is_dark_not_admitted():
"""No certification authority is not permission to serve — it is no proof."""
state = _evaluate(_report(_identity()), ledger=None)
assert state.state == STATE_UNCERTIFIED
assert not state.routable_under(POLICY_COMPAT)
assert not state.routable_under(POLICY_ENFORCE)
def test_admission_records_the_rederived_binding_not_the_declared_one():
split = _split(0, 4, "c")
state = _evaluate(_report(split), ledger=CertificationLedger())
assert state.shard_binding_digest == split.shard_binding_digest
assert state.fingerprint == split.fingerprint.key
# --- Certification: only the tracker, only on evidence it re-derived itself ---
def test_certification_requires_prior_dark_registration():
ledger = CertificationLedger()
identity = parse_identity(_identity().to_dict())
with pytest.raises(TrackerRecipeIdentityError, match="not registered"):
ledger.certify(identity, _evidence(_identity(0, 4), _identity(4, 8)))
def test_synthetic_or_mismatched_evidence_never_certifies():
identity = _identity()
ledger = CertificationLedger()
assert _evaluate(_report(identity), ledger=ledger).state == STATE_UNCERTIFIED
presented = parse_identity(identity.to_dict())
# A unit fixture is not a distributed forward. This is the trust boundary.
with pytest.raises(TrackerRecipeIdentityError, match="synthetic"):
ledger.certify(
presented, _evidence(_identity(0, 4), _identity(4, 8), synthetic=True)
)
with pytest.raises(TrackerRecipeIdentityError, match="fingerprint does not match"):
ledger.certify(
presented,
_evidence(
_identity(0, 4),
_identity(4, 8),
fingerprint=(_digest("e"), _digest("f")),
),
)
# A single node, however real, is not a distributed forward.
with pytest.raises(TrackerRecipeIdentityError, match="distributed forward requires"):
ledger.certify(
presented, _evidence(_identity(0, 8), node_ids=("physical-a",))
)
# A hole in the coverage means those layers were never computed.
with pytest.raises(TrackerRecipeIdentityError, match="owned by no Shard"):
ledger.certify(
presented, _evidence(_identity(0, 3), _identity(4, 8))
)
def test_certification_evidence_layer_count_cannot_be_substituted():
"""An 8-layer recipe cannot be promoted by evidence for a 2-layer route."""
identity = _identity()
ledger = CertificationLedger()
assert _evaluate(_report(identity), ledger=ledger).state == STATE_UNCERTIFIED
with pytest.raises(TrackerRecipeIdentityError, match="layer count"):
ledger.certify(
parse_identity(identity.to_dict()),
_evidence(_identity(0, 4), _identity(4, 8), layer_count=2),
)
def test_evidence_participants_must_be_the_recipe_being_promoted():
identity = _identity()
ledger = CertificationLedger()
assert _evaluate(_report(identity), ledger=ledger).state == STATE_UNCERTIFIED
# Participants running a different recipe cannot vouch for this one, even
# when the evidence header names the right fingerprint.
with pytest.raises(TrackerRecipeIdentityError, match="participant fingerprint"):
ledger.certify(
parse_identity(identity.to_dict()),
_evidence(
_identity(0, 4),
_identity(4, 8, kv_dtype="float16"),
fingerprint=identity.fingerprint.key,
),
)
# Nor can a participant whose identity is not the range it is recorded under.
with pytest.raises(TrackerRecipeIdentityError, match="recorded effective range"):
ledger.certify(
parse_identity(identity.to_dict()),
_evidence(
_identity(0, 4),
_identity(4, 8),
shard_ranges=((0, 4), (0, 4)),
),
)
def test_tracker_owns_the_only_promotion_path_and_re_admits_dark_nodes():
tracker = TrackerServer()
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
node_a = _register(tracker, "physical-a", head)
node_b = _register(tracker, "physical-b", tail)
# A node on a *different* recipe, which must not be swept up by the promotion.
other = _register(tracker, "physical-c", _identity(0, 8, kv_dtype="float16"))
assert node_a.capability.state == STATE_UNCERTIFIED
assert node_b.capability.state == STATE_UNCERTIFIED
status = tracker.certify_recipe(
parse_identity(head.to_dict()), _evidence(head, tail)
)
assert status.may_serve
assert node_a.capability.state == STATE_ADMITTED
assert node_b.capability.state == STATE_ADMITTED
assert other.capability.state == STATE_UNCERTIFIED
# A node registering after the fact lands admitted, from the same ledger.
assert _register(tracker, "physical-d", head).capability.state == STATE_ADMITTED
def test_the_network_map_certification_field_tracks_the_ledger():
"""The map must say what the ledger knows — "dark" before, "certified" after."""
tracker = TrackerServer()
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
node_a = _register(tracker, "physical-a", head)
node_b = _register(tracker, "physical-b", tail)
assert node_a.capability.certification == "dark"
assert node_a.capability.to_dict()["certification"] == "dark"
tracker.certify_recipe(parse_identity(head.to_dict()), _evidence(head, tail))
assert node_a.capability.certification == "certified"
assert node_b.capability.certification == "certified"
assert _register(tracker, "physical-d", head).capability.certification == "certified"
# A node that presented no identity has nothing for the ledger to say.
assert absent_state().certification is None
def test_certification_rejects_participants_the_tracker_did_not_admit():
tracker = TrackerServer()
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
_register(tracker, "physical-a", head)
_register(tracker, "physical-b", tail)
promoted = parse_identity(head.to_dict())
# A node nobody registered cannot have served the forward.
with pytest.raises(TrackerRecipeIdentityError, match="not registered"):
tracker.certify_recipe(
promoted, _evidence(head, tail, node_ids=("physical-a", "ghost"))
)
# Same recipe, same ranges, but derivative bytes neither node was admitted
# on: certification must not attach to blobs that never ran.
with pytest.raises(TrackerRecipeIdentityError, match="binding this tracker did not admit"):
tracker.certify_recipe(
promoted, _evidence(_split(0, 4, "e"), _split(4, 8, "f"))
)
# Right bytes, but swapped between the nodes that served them.
with pytest.raises(TrackerRecipeIdentityError, match="range differs|binding this tracker"):
tracker.certify_recipe(
promoted, _evidence(tail, head)
)
assert tracker._registry["physical-a"].capability.state == STATE_UNCERTIFIED
def test_a_legacy_node_cannot_be_an_exact_certification_participant():
tracker = TrackerServer()
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
_register(tracker, "physical-a", head)
legacy = _NodeEntry(
node_id="physical-b",
endpoint="http://physical-b",
shard_start=4,
shard_end=7,
model="example/model",
shard_checksum=None,
hardware_profile={},
wallet_address=None,
score=1.0,
capability=CapabilityState(state=STATE_ADMITTED, model_id="example/model"),
)
tracker._registry["physical-b"] = legacy
with pytest.raises(TrackerRecipeIdentityError, match="not admitted under"):
tracker.certify_recipe(parse_identity(head.to_dict()), _evidence(head, tail))
def test_certification_does_not_survive_a_tracker_restart():
"""The ledger is in-memory. A restart loses it, and that must fail *closed*."""
tracker = TrackerServer()
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
_register(tracker, "physical-a", head)
_register(tracker, "physical-b", tail)
tracker.certify_recipe(parse_identity(head.to_dict()), _evidence(head, tail))
restarted = TrackerServer()
assert _register(restarted, "physical-a", head).capability.state == STATE_UNCERTIFIED
# --- Route formation: one route, one exact fingerprint ---
def _route_node(
node_id: str,
start: int,
end: int,
fingerprint: tuple[str, str] | None,
*,
speed: float = 1.0,
state: str = STATE_ADMITTED,
) -> _NodeEntry:
capability = CapabilityState(
state=state,
model_id="example/model",
shard_start=start,
shard_end=end,
model_artifact_digest=None if fingerprint is None else fingerprint[0],
runtime_recipe_digest=None if fingerprint is None else fingerprint[1],
)
return _NodeEntry(
node_id=node_id,
endpoint=f"http://{node_id}",
shard_start=start,
shard_end=end,
model="example/model",
shard_checksum=None,
hardware_profile={},
wallet_address=None,
score=1.0,
benchmark_tokens_per_sec=speed,
capability=capability,
)
def test_route_selection_never_mixes_exact_fingerprints():
key_a = (_digest("a"), _digest("b"))
key_b = (_digest("c"), _digest("d"))
route, error = _select_route(
[_route_node("a", 0, 3, key_a), _route_node("b", 4, 7, key_b)],
0,
7,
policy=POLICY_ENFORCE,
)
assert route == []
assert "no route available" in error
def test_route_selection_never_mixes_an_exact_shard_with_a_legacy_one():
"""A node with no canonical digests cannot complete an exact route."""
key_a = (_digest("a"), _digest("b"))
route, error = _select_route(
[_route_node("exact-head", 0, 3, key_a), _route_node("legacy-tail", 4, 7, None)],
0,
7,
policy=POLICY_COMPAT,
)
assert route == []
assert "no route available" in error
def test_route_selection_falls_back_to_a_complete_fingerprint_group():
key_a = (_digest("a"), _digest("b"))
key_b = (_digest("c"), _digest("d"))
route, error = _select_route(
[
_route_node("fast-incomplete", 0, 3, key_b, speed=100.0),
_route_node("a-head", 0, 3, key_a),
_route_node("a-tail", 4, 7, key_a),
],
0,
7,
policy=POLICY_ENFORCE,
)
assert error == ""
assert [node.node_id for node in route] == ["a-head", "a-tail"]
def test_a_homogeneous_legacy_fleet_routes_exactly_as_before():
"""DGR-003 partitions routes; it must not change a fleet that has no identity."""
route, error = _select_route(
[
_route_node("slow", 0, 3, None, speed=1.0),
_route_node("fast", 0, 3, None, speed=50.0),
_route_node("tail", 4, 7, None),
],
0,
7,
policy=POLICY_COMPAT,
)
assert error == ""
assert [node.node_id for node in route] == ["fast", "tail"]
def test_an_uncertified_node_is_not_routable_under_either_policy():
key = (_digest("a"), _digest("b"))
nodes = [
_route_node("head", 0, 3, key, state=STATE_UNCERTIFIED),
_route_node("tail", 4, 7, key, state=STATE_UNCERTIFIED),
]
for policy in (POLICY_COMPAT, POLICY_ENFORCE):
route, error = _select_route(nodes, 0, 7, policy=policy)
assert route == [], policy
assert "no route available" in error
def test_pinned_benchmark_routes_obey_the_same_partition_rule():
"""US-030 benchmark combos run real inference; they may not mix identities."""
key_a = (_digest("a"), _digest("b"))
key_b = (_digest("c"), _digest("d"))
# An exact head with only a legacy tail, or a tail on another fingerprint,
# has no two-hop combo at all.
assert _find_pinned_route(
[_route_node("exact", 0, 3, key_a), _route_node("legacy", 4, 7, None)], 0, 7, 2
) is None
assert _find_pinned_route(
[_route_node("a", 0, 3, key_a), _route_node("b", 4, 7, key_b)], 0, 7, 2
) is None
# Homogeneous fleets — exact or legacy — still form pinned combos.
exact = _find_pinned_route(
[_route_node("a", 0, 3, key_a), _route_node("b", 4, 7, key_a)], 0, 7, 2
)
assert exact is not None and [node.node_id for node in exact] == ["a", "b"]
legacy = _find_pinned_route(
[_route_node("a", 0, 3, None), _route_node("b", 4, 7, None)], 0, 7, 2
)
assert legacy is not None and [node.node_id for node in legacy] == ["a", "b"]
# --- gRPC handshake (DGR-002 SessionOpen) ---
def _session_open(identity: ShardIdentity, **changes: object) -> "pb.SessionOpen":
fields: dict[str, object] = {
"schema_version": SCHEMA_VERSION,
"route_session_id": "session",
"route_epoch": 1,
"fingerprint": identity.fingerprint.to_proto(),
"shard_range": pb.ShardRange(
start_layer=identity.shard_start,
end_layer=identity.shard_end,
effective_start_layer=identity.shard_start,
),
}
fields.update(changes)
return pb.SessionOpen(**fields) # type: ignore[arg-type]
def test_session_open_accepts_the_exact_local_shard():
local = _identity()
assert (
check_session_open(
local,
_session_open(local),
expected_route_session_id="session",
expected_route_epoch=1,
)
== ()
)
assert handshake_error(()) is None
def test_session_open_rejects_a_matching_fingerprint_with_the_wrong_range():
local = _identity()
opened = _session_open(
local,
shard_range=pb.ShardRange(start_layer=0, end_layer=5, effective_start_layer=0),
)
mismatches = check_session_open(local, opened)
assert mismatches
assert handshake_error(mismatches).code == pb.ERROR_CODE_SHARD_RANGE_MISMATCH
def test_session_open_rejects_an_effective_start_outside_the_shard():
local = _identity(4, 8)
opened = _session_open(
local,
shard_range=pb.ShardRange(start_layer=4, end_layer=8, effective_start_layer=8),
)
mismatches = check_session_open(local, opened)
assert mismatches
assert handshake_error(mismatches).code == pb.ERROR_CODE_SHARD_RANGE_MISMATCH
def test_session_open_rejects_a_foreign_protocol_schema():
local = _identity()
mismatches = check_session_open(
local, _session_open(local, schema_version=SCHEMA_VERSION + 1)
)
assert mismatches
# A schema disagreement is its own protocol outcome — not a range problem,
# and not a fingerprint problem either: the peer may hold the right recipe
# and simply speak a schema this node cannot.
assert handshake_error(mismatches).code == pb.ERROR_CODE_SCHEMA_UNSUPPORTED
@pytest.mark.parametrize(
("changes", "expected_session", "expected_epoch"),
[
({"route_session_id": "other-session"}, "session", 1),
({"route_epoch": 2}, "session", 1),
({"route_session_id": ""}, None, None),
({"route_epoch": 0}, None, None),
],
)
def test_session_open_rejects_missing_or_stale_tracker_route_assignment(
changes, expected_session, expected_epoch
):
local = _identity()
mismatches = check_session_open(
local,
_session_open(local, **changes),
expected_route_session_id=expected_session,
expected_route_epoch=expected_epoch,
)
assert mismatches
assert handshake_error(mismatches).code == pb.ERROR_CODE_EPOCH_STALE
def test_a_digest_disagreement_dominates_the_handshake_error_code():
"""Wrong fingerprint plus wrong range is a wrong peer, not a re-routable one."""
local = _identity()
opened = _session_open(
_identity(kv_dtype="float16"),
shard_range=pb.ShardRange(start_layer=0, end_layer=5, effective_start_layer=0),
)
mismatches = check_session_open(local, opened)
assert mismatches
assert handshake_error(mismatches).code == pb.ERROR_CODE_FINGERPRINT_MISMATCH
def test_grpc_handshake_uses_the_dgr_002_fingerprint_and_fails_closed():
local = _identity()
remote = replace(local.fingerprint, runtime_recipe_digest=_digest("f")).to_proto()
mismatches = check_handshake(local, remote)
assert mismatches
assert handshake_error(mismatches).code == pb.ERROR_CODE_FINGERPRINT_MISMATCH