Compare commits
16 Commits
e7c780a623
...
ralph/dgr-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cae7c2b171 | ||
|
|
64f83d4392 | ||
|
|
454a681a50 | ||
|
|
a0f28b5631 | ||
|
|
7925e5253d | ||
|
|
91c450840d | ||
|
|
d6b808dcf9 | ||
|
|
31065c0e12 | ||
|
|
ec36290863 | ||
|
|
f844ae6567 | ||
|
|
252d131e7d | ||
|
|
3d8f93f4aa | ||
|
|
f9722e7b57 | ||
|
|
7b8e467c6b | ||
|
|
7364ed6731 | ||
|
|
ad2d17541c |
@@ -9,3 +9,4 @@
|
|||||||
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
||||||
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
||||||
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.
|
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.
|
||||||
|
- **DGR-004 llama.cpp boundary** — `packages/node/native/llama/` locks `e920c523e3b8a0163fe498af5bf90df35ff51d25`, with a one-patch CMake marker and fail-closed clean materialize/apply/build/smoke harness. This is infrastructure only; stock GLM dense fallback remains uncertified.
|
||||||
|
|||||||
@@ -44,6 +44,102 @@ Real-model/hardware stories must:
|
|||||||
|
|
||||||
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
||||||
|
|
||||||
|
## Dependency Graph and Status
|
||||||
|
|
||||||
|
Status as of 2026-07-14 (MAINT-003). Authoritative per-story status is
|
||||||
|
`passes` in [prd.json](prd.json); closed issues live in
|
||||||
|
`docs/issues/distributed-gguf-runtime/`, open and blocked issues in
|
||||||
|
[issues/](issues/).
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
classDef done fill:#c8e6c9,stroke:#2e7d32;
|
||||||
|
classDef blocked fill:#ffcdd2,stroke:#c62828;
|
||||||
|
|
||||||
|
DGR001[DGR-001 perf contract]:::done
|
||||||
|
DGR002[DGR-002 gRPC Shard protocol]:::done
|
||||||
|
DGR003[DGR-003 artifact/recipe identity]:::done
|
||||||
|
DGR004[DGR-004 pinned llama.cpp patch stack]:::done
|
||||||
|
DGR005[DGR-005 dense-Llama range ownership]:::done
|
||||||
|
DGR006[DGR-006 boundary input/output]:::done
|
||||||
|
DGR017[DGR-017 GLM-5.2 target/alpha contract]:::done
|
||||||
|
DGR018[DGR-018 whole-model GLM oracle]:::blocked
|
||||||
|
DGR019[DGR-019 GLM range/DSA/IndexShare]:::blocked
|
||||||
|
DGR020[DGR-020 distributed GLM alpha]:::blocked
|
||||||
|
DGR007[DGR-007 Hot KV State]
|
||||||
|
DGR008[DGR-008 C++ gRPC worker]
|
||||||
|
DGR009[DGR-009 Meshnet integration]
|
||||||
|
DGR010[DGR-010 local two-process acceptance]
|
||||||
|
DGR011[DGR-011 two-machine route]
|
||||||
|
DGR012[DGR-012 continuous batching]
|
||||||
|
DGR013[DGR-013 failure/cancel/restart]
|
||||||
|
DGR014[DGR-014 release gate]
|
||||||
|
DGR015[DGR-015 Qwen3 adapter]
|
||||||
|
DGR016[DGR-016 upstream package]
|
||||||
|
|
||||||
|
DGR002 --> DGR003
|
||||||
|
DGR017 --> DGR003
|
||||||
|
DGR001 --> DGR004
|
||||||
|
DGR017 --> DGR004
|
||||||
|
DGR003 --> DGR005
|
||||||
|
DGR004 --> DGR005
|
||||||
|
DGR002 --> DGR006
|
||||||
|
DGR005 --> DGR006
|
||||||
|
DGR001 --> DGR017
|
||||||
|
DGR002 --> DGR017
|
||||||
|
DGR003 --> DGR018
|
||||||
|
DGR004 --> DGR018
|
||||||
|
DGR017 --> DGR018
|
||||||
|
DGR005 --> DGR019
|
||||||
|
DGR006 --> DGR019
|
||||||
|
DGR018 --> DGR019
|
||||||
|
DGR006 --> DGR007
|
||||||
|
DGR019 --> DGR007
|
||||||
|
DGR002 --> DGR008
|
||||||
|
DGR003 --> DGR008
|
||||||
|
DGR004 --> DGR008
|
||||||
|
DGR006 --> DGR008
|
||||||
|
DGR007 --> DGR008
|
||||||
|
DGR003 --> DGR009
|
||||||
|
DGR008 --> DGR009
|
||||||
|
DGR009 --> DGR010
|
||||||
|
DGR010 --> DGR011
|
||||||
|
DGR007 --> DGR012
|
||||||
|
DGR009 --> DGR012
|
||||||
|
DGR010 --> DGR012
|
||||||
|
DGR008 --> DGR013
|
||||||
|
DGR009 --> DGR013
|
||||||
|
DGR001 --> DGR014
|
||||||
|
DGR011 --> DGR014
|
||||||
|
DGR012 --> DGR014
|
||||||
|
DGR013 --> DGR014
|
||||||
|
DGR014 --> DGR015
|
||||||
|
DGR010 --> DGR016
|
||||||
|
DGR007 --> DGR020
|
||||||
|
DGR008 --> DGR020
|
||||||
|
DGR009 --> DGR020
|
||||||
|
DGR011 --> DGR020
|
||||||
|
DGR013 --> DGR020
|
||||||
|
DGR017 --> DGR020
|
||||||
|
DGR018 --> DGR020
|
||||||
|
DGR019 --> DGR020
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Done (`passes: true`):** DGR-001, DGR-002, DGR-003, DGR-004, DGR-005,
|
||||||
|
DGR-006, DGR-017.
|
||||||
|
- **Blocked on hardware:** DGR-018 requires a 256-GiB-class host with at least
|
||||||
|
224 GiB runtime-accessible memory and 250 GB free storage outside `/home`;
|
||||||
|
no such host is currently available (development host: 124.9 GiB MemTotal).
|
||||||
|
Exact preflight output: [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md).
|
||||||
|
DGR-019 (needs the DGR-018 oracle) and DGR-020 (needs DGR-018/DGR-019 plus
|
||||||
|
multiple physical consumer nodes) are blocked transitively.
|
||||||
|
- **Consequence of the graph as written:** DGR-007 depends on DGR-019, so every
|
||||||
|
remaining story (DGR-007 through DGR-016) is transitively blocked on the
|
||||||
|
256-GiB host. Unblocking the generic dense pipeline without that host would
|
||||||
|
require an explicit re-planning decision to relax the DGR-007 → DGR-019
|
||||||
|
dependency; that decision is out of scope for maintenance and has not been
|
||||||
|
made.
|
||||||
|
|
||||||
## User Stories
|
## User Stories
|
||||||
|
|
||||||
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
||||||
|
|||||||
186
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md
Normal file
186
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md
Normal file
@@ -0,0 +1,186 @@
|
|||||||
|
# DGR-003 — exact Artifact and runtime recipe identity
|
||||||
|
|
||||||
|
Evidence class: deterministic offline/unit. No model payload, GPU, external API,
|
||||||
|
network node, or API credit is required or claimed.
|
||||||
|
|
||||||
|
## Result — delayed-review repair, 2026-07-14
|
||||||
|
|
||||||
|
DGR-003 defines and tests an exact, model-agnostic compatibility identity and
|
||||||
|
connects it to DGR-002's gRPC `Fingerprint` plus tracker parsing, admission,
|
||||||
|
route partitioning, and certification. It is **not complete**: the existing
|
||||||
|
production doctor/backend path still emits the legacy capability report without
|
||||||
|
constructing a `ShardIdentity` from authoritative loaded artifact/runtime state.
|
||||||
|
No exact recipe is therefore claimed live or routable from that path; supplied
|
||||||
|
exact identities remain dark until tracker-owned certification.
|
||||||
|
|
||||||
|
A matching digest proves canonical consistency, **not node authenticity or real
|
||||||
|
execution**. Tracker-owned certification of a fingerprint by a non-synthetic,
|
||||||
|
complete, multi-node distributed forward is the execution trust boundary.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
- `ArtifactIdentity` binds artifact ID/revision, exact content digest,
|
||||||
|
architecture/config digest, layer count, and optional derivative binding.
|
||||||
|
- `DerivativeBinding` binds a split artifact to the exact source artifact digest
|
||||||
|
and its end-exclusive layer range. A Shard cannot advertise outside that range.
|
||||||
|
- `RuntimeRecipe` keeps these canonical axes separate rather than hiding them in
|
||||||
|
a backend label:
|
||||||
|
- weight quantization;
|
||||||
|
- activation and compute dtypes;
|
||||||
|
- KV dtype and layout;
|
||||||
|
- tokenizer revision;
|
||||||
|
- architecture adapter;
|
||||||
|
- backend and runtime version;
|
||||||
|
- boundary and protocol schema versions;
|
||||||
|
- recipe ID/version and catalogue version.
|
||||||
|
- `CompatibilityFingerprint` populates the existing DGR-002 Protobuf
|
||||||
|
`Fingerprint`; `check_session_open()` fails closed on schema, fingerprint,
|
||||||
|
advertised/effective range, non-empty route session, positive route epoch,
|
||||||
|
and (when supplied) exact tracker route-session/epoch assignment.
|
||||||
|
- Node and tracker implementations independently canonicalize the declaration.
|
||||||
|
This is intentional: the tracker must not trust a digest copied from a node,
|
||||||
|
and future native/C++ workers also need an independent implementation. Their
|
||||||
|
behavior is pinned by `tests/data/recipe_fingerprint_vectors.json`.
|
||||||
|
- Tracker admission cross-checks the exact identity against the capability
|
||||||
|
proof's model, range, recipe labels, backend, and weight quantization. Any
|
||||||
|
disagreement fails closed.
|
||||||
|
- `TrackerServer` owns the sole live certification ledger and passes it through
|
||||||
|
direct and replicated registration paths. A known exact recipe is
|
||||||
|
`uncertified` and dark for user traffic until the same exact fingerprint is
|
||||||
|
certified. Restart fails closed; durable/cluster-wide certification events
|
||||||
|
require the later real-forward control path and are not claimed here.
|
||||||
|
- Certification evidence is bound to the promoted fingerprint, requires at
|
||||||
|
least two distinct nodes, complete layer coverage, generated tokens, and
|
||||||
|
`synthetic=false`. Unknown or mismatched fingerprints cannot be promoted.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `packages/node/meshnet_node/runtime_recipe.py`
|
||||||
|
- `packages/tracker/meshnet_tracker/recipe.py`
|
||||||
|
- `packages/tracker/meshnet_tracker/capability.py`
|
||||||
|
- `packages/tracker/meshnet_tracker/server.py`
|
||||||
|
- `tests/data/recipe_fingerprint_vectors.json`
|
||||||
|
- `tests/test_runtime_recipe_identity.py`
|
||||||
|
- this evidence directory, issue state, and DGR-003 PRD state
|
||||||
|
|
||||||
|
A late review of dependency DGR-017 also found and fixed two genuine contract
|
||||||
|
continuity defects during delayed DGR-003 review: v1 now has an independently
|
||||||
|
trusted digest and recursively immutable parsed state. Those changes and tests
|
||||||
|
are recorded in DGR-017 evidence rather than claimed as DGR-003 functionality.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Exact commands and outcomes are in `commands.txt`.
|
||||||
|
|
||||||
|
Observed final results:
|
||||||
|
|
||||||
|
- DGR-003 identity + node/tracker capability suites: **126 passed**.
|
||||||
|
- DGR-017 focused dependency repair suite: **99 passed**.
|
||||||
|
- Tracker routing suite: **93 passed**.
|
||||||
|
- First delayed-review integrated run: **898 passed, 13 skipped, 1 failed** on
|
||||||
|
the pre-existing tracker-cancellation race.
|
||||||
|
- Final delayed-review integrated rerun: **899 passed, 13 skipped** in
|
||||||
|
**253.64s**; Hermes controller acceptance rerun: **899 passed, 13 skipped**
|
||||||
|
in **252.66s**.
|
||||||
|
- `python -m compileall -q packages tests`: pass.
|
||||||
|
- `git diff --check`: pass.
|
||||||
|
- Ruff on the changed identity, capability, contract, and test modules: pass.
|
||||||
|
- `server.py` has 8 pre-existing Ruff findings at both pushed baseline and the
|
||||||
|
current tree; DGR-003 added no finding.
|
||||||
|
|
||||||
|
The first integrated full-suite run produced **871 passed, 13 skipped, 1 failed**
|
||||||
|
on the known unrelated
|
||||||
|
`test_tracker_dashboard_can_cancel_inflight_proxy` timing race. Its fixture
|
||||||
|
completed after three seconds just before cancellation, so the cancel endpoint
|
||||||
|
returned 404. In this delayed repair it again produced a 404 after the stream
|
||||||
|
finished (first integrated run: **898 passed, 13 skipped, 1 failed**); three
|
||||||
|
immediate isolated repeats passed before a fourth reproduced the same race.
|
||||||
|
No cancellation-test code was changed. The final complete integrated rerun
|
||||||
|
passed **899/899** tests.
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Certification state is process-local in this story. The same running tracker
|
||||||
|
reuses it across registrations, but durable/cluster-wide certification-event
|
||||||
|
persistence belongs with the later real distributed-forward control path.
|
||||||
|
Restart or failover therefore returns exact recipes to the safe dark state;
|
||||||
|
it never makes an unsupported recipe routable.
|
||||||
|
- The node module has no certification ledger or admission policy; it holds only
|
||||||
|
identity construction and handshake validation. The Tracker is the sole
|
||||||
|
promotion authority.
|
||||||
|
- **Completion blocker:** `doctor._validate_recipe()` calls
|
||||||
|
`build_capability_report()` without `identity=`, because the legacy
|
||||||
|
Transformers backend does not expose an immutable artifact-content pin and
|
||||||
|
full runtime recipe axes authoritative enough to build one. Adding a guessed
|
||||||
|
identity would weaken this contract. Production emission must be added with
|
||||||
|
the authoritative native worker/backend loading seam; until then the issue and
|
||||||
|
PRD deliberately remain incomplete.
|
||||||
|
- This story proves identity and admission behavior with deterministic fixtures.
|
||||||
|
It does not claim a real GLM forward or hardware certification.
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
- Capability report identity is additive. Legacy reports without the new block
|
||||||
|
retain ADR-0023's explicit compatibility-policy behavior.
|
||||||
|
- Reports that opt into exact identity are held to it and fail closed on malformed,
|
||||||
|
inconsistent, unknown, dark, or mismatched declarations.
|
||||||
|
- No new wire identity was invented; DGR-002's `Fingerprint` remains the gRPC
|
||||||
|
representation.
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
DGR-004 and native workers must build `ShardIdentity` from the actual immutable
|
||||||
|
artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout,
|
||||||
|
schema versions, and owned range. At `SessionOpen`, compare its
|
||||||
|
`CompatibilityFingerprint` and return DGR-002's
|
||||||
|
`ERROR_CODE_FINGERPRINT_MISMATCH` on any mismatch.
|
||||||
|
|
||||||
|
A digest match is not certification. Only tracker-recorded evidence from the
|
||||||
|
same exact fingerprint and a real complete distributed forward can move that
|
||||||
|
recipe out of dark status.
|
||||||
|
|
||||||
|
## Native emission closure — 2026-07-14
|
||||||
|
|
||||||
|
Status: **done**. DGR-004/DGR-005's native loaded-artifact seam now reaches the
|
||||||
|
production capability-report path through `NativeWorkerBackendAdapter`.
|
||||||
|
|
||||||
|
### Files changed
|
||||||
|
|
||||||
|
- `packages/node/meshnet_node/native_backend.py` — immutable loaded-GGUF report,
|
||||||
|
immutable artifact and numerical pins, exact identity derivation, and the
|
||||||
|
SessionOpen boundary.
|
||||||
|
- `packages/node/meshnet_node/doctor.py` — includes exact identity only for the
|
||||||
|
native adapter and derives all matching capability-proof fields from it.
|
||||||
|
- `tests/test_native_identity_emission.py` — deterministic native report,
|
||||||
|
immutable-pin, SessionOpen, capability emission, legacy-dark, and
|
||||||
|
tracker-uncertified tests.
|
||||||
|
- This issue, `prd.json`, and this evidence directory.
|
||||||
|
|
||||||
|
### Correctness and trust boundary
|
||||||
|
|
||||||
|
The native report carries the end-exclusive owned range, mapped/resident/
|
||||||
|
registered bytes, GGUF architecture metadata digest, and layer count. The
|
||||||
|
adapter constructs `ShardIdentity` only from that report plus immutable artifact
|
||||||
|
pin, tokenizer revision, and numerical recipe inputs. It does not accept a
|
||||||
|
caller-supplied shard range.
|
||||||
|
|
||||||
|
`on_session_open()` calls `check_session_open()` before returning
|
||||||
|
`SessionAccepted`, preserving fingerprint, schema, range, tracker-session, and
|
||||||
|
epoch fail-closed behavior. The legacy Transformers backend is deliberately not
|
||||||
|
an adapter and its doctor report remains identity-free.
|
||||||
|
|
||||||
|
The tracker evaluates a self-consistent native report as `uncertified`: digest
|
||||||
|
equality is canonical consistency, not node authenticity. Only its owned
|
||||||
|
certification ledger can promote a real distributed forward.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
- Focused/adversarial DGR-003, node/tracker capability, doctor, and native
|
||||||
|
dependency suites: **171 passed, 1 skipped**.
|
||||||
|
- Native protocol CMake configure/build plus CTest: **1/1 passed**.
|
||||||
|
- `compileall`, Ruff, and `git diff --check`: pass.
|
||||||
|
- Full deterministic suite: **902 passed, 13 skipped** (255.01s).
|
||||||
|
|
||||||
|
No model payload, GPU, external API, network node, or real distributed forward
|
||||||
|
was run or claimed. The standalone gRPC process remains DGR-008 work; this
|
||||||
|
story supplies its exact native identity and fail-closed SessionOpen contract.
|
||||||
108
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt
Normal file
108
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
# DGR-003 final verification — 2026-07-14
|
||||||
|
|
||||||
|
# Native emission closure — 2026-07-14
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_native_identity_emission.py tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py tests/test_node_doctor.py tests/test_llama_cpp_dependency.py
|
||||||
|
# result: 171 passed, 1 skipped in 7.07s
|
||||||
|
|
||||||
|
ruff check packages/node/meshnet_node/native_backend.py packages/node/meshnet_node/doctor.py tests/test_native_identity_emission.py
|
||||||
|
# result: All checks passed
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall packages tests
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake -S packages/node/native -B build/dgr-003-native-protocol -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake --build build/dgr-003-native-protocol -j2
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/ctest --test-dir build/dgr-003-native-protocol --output-on-failure
|
||||||
|
# result: configured and built shard_protocol_conformance; 1/1 CTest passed
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 902 passed, 13 skipped in 255.01s
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||||
|
# result: 99 passed in 4.76s
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
# result: 99 passed in 0.15s
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# first integrated result: 871 passed, 13 skipped, 1 failed in 258.18s
|
||||||
|
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
# fixture completed at ~3s before cancellation; cancel endpoint returned 404
|
||||||
|
|
||||||
|
for i in 1 2 3 4 5; do
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
done
|
||||||
|
# result: 5/5 passed (1.14s, 1.14s, 1.26s, 1.14s, 1.64s)
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# final integrated result: 872 passed, 13 skipped in 253.46s
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
ruff check packages/node/meshnet_node/glm_alpha/contract.py packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_glm_alpha_target.py tests/test_runtime_recipe_identity.py
|
||||||
|
# result: All checks passed!
|
||||||
|
|
||||||
|
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
|
||||||
|
ruff check /tmp/dgr003-server-base.py
|
||||||
|
ruff check packages/tracker/meshnet_tracker/server.py
|
||||||
|
# result: both baseline and current server.py report the same 8 pre-existing findings
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Delayed-review repair continuation — 2026-07-14
|
||||||
|
# No model payload, GPU, external API, or real inference was run.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||||
|
# result: 126 passed in 4.77s
|
||||||
|
# includes adversarial certification binding, unknown participant, mutation-atomicity,
|
||||||
|
# report/identity revision+config, route partition, golden-vector, and SessionOpen tests
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node:packages/tracker /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python scripts/gen_recipe_fingerprint_vectors.py --check
|
||||||
|
# result: tests/data/recipe_fingerprint_vectors.json matches the identity implementation
|
||||||
|
|
||||||
|
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
# result: 99 passed in 0.11s
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py
|
||||||
|
# result: 93 passed in 46.83s
|
||||||
|
# there is no separate tests/test_tracker_server.py in this repository
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
ruff check packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_runtime_recipe_identity.py scripts/gen_recipe_fingerprint_vectors.py
|
||||||
|
# result: All checks passed!
|
||||||
|
|
||||||
|
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
|
||||||
|
ruff check /tmp/dgr003-server-base.py
|
||||||
|
ruff check packages/tracker/meshnet_tracker/server.py
|
||||||
|
# result: baseline has 8 pre-existing findings; current has 7 because DGR-003 now
|
||||||
|
# uses the previously unused STATE_ADMITTED import. No new server.py finding.
|
||||||
|
|
||||||
|
git diff --check
|
||||||
|
# result: pass
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 898 passed, 13 skipped, 1 failed in 255.43s
|
||||||
|
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
# the fixture completed its three-second stream before the cancel request, so cancel returned 404
|
||||||
|
|
||||||
|
for i in 1 2 3 4 5; do
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy || exit 1
|
||||||
|
done
|
||||||
|
# result: first 3 passed (1.16s, 1.65s, 1.64s); attempt 4 reproduced the same 404 race.
|
||||||
|
# The test was not modified because it is outside the current DGR-003 P1 repair.
|
||||||
|
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 899 passed, 13 skipped in 253.64s (0:04:13)
|
||||||
|
|
||||||
|
# Hermes controller acceptance rerun after agent completion
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||||
|
# result: 899 passed, 13 skipped in 252.66s (0:04:12)
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# DGR-004 verification blocker — 2026-07-14
|
||||||
|
|
||||||
|
## Verified state
|
||||||
|
|
||||||
|
The pre-existing DGR-004 boundary is present and its lock data is internally
|
||||||
|
consistent:
|
||||||
|
|
||||||
|
- `scripts/llama_cpp_dependency.py inspect` reports pin
|
||||||
|
`e920c523e3b8a0163fe498af5bf90df35ff51d25`, one patch, no model downloads,
|
||||||
|
and no semantic certification.
|
||||||
|
- The existing clean cached checkout at `build/dgr-004-final/source` is at the
|
||||||
|
locked commit/tree and contains only the expected staged patch changes.
|
||||||
|
- The existing `llama-gguf-hash --help` smoke binary runs successfully.
|
||||||
|
- `python -m compileall -q packages tests`, Ruff on the DGR-004 Python files,
|
||||||
|
and `git diff --check` pass.
|
||||||
|
|
||||||
|
## Blocker
|
||||||
|
|
||||||
|
The verification environment no longer contains the `.venv` recorded in
|
||||||
|
`commands.txt`, nor a `cmake` executable on `PATH`. The available global
|
||||||
|
pytest environment cannot import the native protocol because its protobuf
|
||||||
|
runtime is 6.33.6 while the checked-in generated code requires 7.35.0. This
|
||||||
|
causes both `tests/test_llama_cpp_dependency.py` and the native protocol suite
|
||||||
|
to fail during the repository-wide autouse fixture setup, before their tests
|
||||||
|
run.
|
||||||
|
|
||||||
|
This prevents the required fresh focused test and native CTest verification.
|
||||||
|
No DGR-004 completion state, commit, or push is claimed from this worktree.
|
||||||
|
|
||||||
|
## Continuation
|
||||||
|
|
||||||
|
1. Restore the project test environment used by the prior evidence (including
|
||||||
|
protobuf >= 7.35.0 and CMake), without changing DGR-004 source files.
|
||||||
|
2. Run the exact focused test command from `commands.txt` and the clean
|
||||||
|
`reproduce` command using the local llama.cpp object cache.
|
||||||
|
3. Re-run compileall, Ruff, diff check, and the deterministic full suite.
|
||||||
|
4. Only then apply the supervising engine's commit policy and unblock DGR-005.
|
||||||
|
|
||||||
|
## Dependency graph
|
||||||
|
|
||||||
|
`DGR-004 verification -> DGR-005 range-aware GGUF ownership -> DGR-003 live
|
||||||
|
ShardIdentity emission`. DGR-005 and DGR-003-emission were not modified.
|
||||||
31
.scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
Normal file
31
.scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
# DGR-004 — Reproducible pinned llama.cpp patch stack
|
||||||
|
|
||||||
|
Status: **done**. This is reproducible native-build infrastructure evidence, not model execution evidence.
|
||||||
|
|
||||||
|
## Delivered boundary
|
||||||
|
|
||||||
|
- Pin: `ggml-org/llama.cpp` at `e920c523e3b8a0163fe498af5bf90df35ff51d25` (tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1`).
|
||||||
|
- Ordered patch: `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch`, SHA-256 `1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75`.
|
||||||
|
- The sole patch adds an interface-library CMake marker. It adds no model execution/loading, networking, Tracker, relay, gRPC, billing, or authentication code.
|
||||||
|
- `scripts/llama_cpp_dependency.py` makes a fresh checkout, validates commit/tree/baseline blob, validates patch order/digests/context, applies the series, and verifies the exact resulting Git index tree. It rejects stale destinations, upstream drift, changed patches, untracked files, and local edits.
|
||||||
|
|
||||||
|
## Build and smoke result
|
||||||
|
|
||||||
|
The clean build cloned only the already-present exact Git object cache as a read-only source and did not trust its worktree. CMake 4.4.0 and GCC 15.2.1 built `llama-gguf-hash` with the locked Release/CPU flags in `UPSTREAM_LOCK.json`; `llama-gguf-hash --help` passed with no model download or load.
|
||||||
|
|
||||||
|
llama.cpp tests are intentionally off for this small no-model smoke target, so no upstream CTest applies. Meshnet's focused native protocol suite passed independently. Exact results are in `commands.txt` and `results.json`.
|
||||||
|
|
||||||
|
## License, compatibility, and handoff
|
||||||
|
|
||||||
|
llama.cpp is MIT licensed. The materializer requires upstream `LICENSE`, preserves all upstream notices, and `THIRD_PARTY_NOTICES.md` requires including them in redistribution. No Mesh-LLM code or patch was adopted.
|
||||||
|
|
||||||
|
The lock records the patched upstream blob and resulting patched tree. Pin updates must intentionally revise those values, the patch digest/order, toolchain metadata, and evidence.
|
||||||
|
|
||||||
|
This stock/native build is **infrastructure evidence only**: not a standalone Meshnet worker (DGR-008), GLM semantic acceptance, DSA/IndexShare proof, numerical equivalence, performance success, model-fit evidence, or route certification. The stock dense-MLA fallback remains explicitly uncertified. DGR-001 CPU v1 remains `stop`; DGR-017 is a separate target contract. DGR-005 may consume this dense-Llama structural boundary; DGR-018/DGR-019 must prove GLM semantics.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `packages/node/native/llama/*`
|
||||||
|
- `scripts/llama_cpp_dependency.py`
|
||||||
|
- `tests/test_llama_cpp_dependency.py`
|
||||||
|
- this evidence directory, the DGR-004 issue, and `prd.json`
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# DGR-004 commands and real results — 2026-07-14
|
||||||
|
|
||||||
|
```text
|
||||||
|
$ .venv/bin/python -m pytest -q tests/test_llama_cpp_dependency.py tests/test_native_shard_protocol.py
|
||||||
|
47 passed, 1 skipped in 0.59s
|
||||||
|
|
||||||
|
$ .venv/bin/python scripts/llama_cpp_dependency.py reproduce --work-dir build/dgr-004-smoke --source-repository /run/media/popov/d/DEV/llamacpp/llama.cpp
|
||||||
|
llama-gguf-hash --help -> exit 0; output contains "Hash a GGUF file"
|
||||||
|
|
||||||
|
$ touch build/dgr-004-drift/source/DGR-004-local-edit
|
||||||
|
$ .venv/bin/python scripts/llama_cpp_dependency.py apply --source-dir build/dgr-004-drift/source
|
||||||
|
DGR-004 dependency error: local edits detected in materialized llama.cpp checkout
|
||||||
|
exit 2
|
||||||
|
|
||||||
|
$ .venv/bin/python -m compileall -q packages tests
|
||||||
|
exit 0
|
||||||
|
|
||||||
|
$ ruff check scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
|
||||||
|
All checks passed!
|
||||||
|
|
||||||
|
$ git diff --check
|
||||||
|
exit 0
|
||||||
|
|
||||||
|
$ .venv/bin/python -m pytest -q --cache-clear
|
||||||
|
902 passed, 13 skipped in 255.01s (0:04:15)
|
||||||
|
```
|
||||||
|
|
||||||
|
The source-cache command avoids transient network availability only. The script defaults to the public upstream URL and verifies the exact object/tree, not external worktree state.
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"evidence_class": "native build infrastructure",
|
||||||
|
"llama_cpp": {
|
||||||
|
"upstream": "https://github.com/ggml-org/llama.cpp.git",
|
||||||
|
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||||
|
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
|
||||||
|
"patched_tree": "4a37c06fac668834435b803caa59ba272bdace5c",
|
||||||
|
"patch_sha256": "1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75"
|
||||||
|
},
|
||||||
|
"toolchain": {"cmake": "4.4.0", "cxx": "GCC 15.2.1", "generator": "Unix Makefiles", "target": "llama-gguf-hash", "configure_flags": ["-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_TESTS=OFF", "-DLLAMA_BUILD_EXAMPLES=ON", "-DLLAMA_BUILD_SERVER=OFF", "-DLLAMA_BUILD_TOOLS=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_CURL=OFF"]},
|
||||||
|
"checks": {"clean_materialize_apply_build_smoke": "passed", "local_edit_detection": "passed (exit 2)", "focused_pytest": "47 passed, 1 skipped", "compileall": "passed", "ruff": "passed", "git_diff_check": "passed", "full_pytest": "902 passed, 13 skipped"},
|
||||||
|
"model_downloads": false,
|
||||||
|
"model_loaded": false,
|
||||||
|
"inference_run": false,
|
||||||
|
"glm_semantic_certification": false,
|
||||||
|
"performance_certification": false,
|
||||||
|
"route_certification": false
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# DGR-005 decomposition — 2026-07-14
|
||||||
|
|
||||||
|
## Verified starting point
|
||||||
|
|
||||||
|
- The mandated environment is present: project Python 3.14.6, CMake 4.4.0,
|
||||||
|
and protobuf 7.35.1.
|
||||||
|
- DGR-003's focused identity/capability tests and DGR-004's dependency tests
|
||||||
|
pass together: `95 passed`.
|
||||||
|
- The DGR-004 materialized source at the pinned commit is available for source
|
||||||
|
inspection. It contains only the DGR-004 CMake-marker patch.
|
||||||
|
|
||||||
|
## Why this chain cannot safely claim DGR-005 yet
|
||||||
|
|
||||||
|
At the locked llama.cpp revision, `llama_model_base::load_tensors()`:
|
||||||
|
|
||||||
|
1. sizes `layers` to `hparams.n_layer_all`;
|
||||||
|
2. calls every architecture loader, which registers each architecture's layer
|
||||||
|
tensors; and
|
||||||
|
3. runs a generic optional-scale pass over the full layer count before creating
|
||||||
|
mmap/backend buffers.
|
||||||
|
|
||||||
|
Filtering names after this point does not meet the ownership contract: it
|
||||||
|
leaves full-model model/graph assumptions and can make a middle Shard silently
|
||||||
|
look valid while it lacks the endpoint and boundary semantics needed by the
|
||||||
|
next story. A generic `blk.N.*` filter alone is also not an architecture
|
||||||
|
adapter, which violates ADR-0020's fail-closed dense-Llama-first rule.
|
||||||
|
|
||||||
|
## Required child slices
|
||||||
|
|
||||||
|
1. **DGR-005A — native dense-Llama ownership API and loader**
|
||||||
|
- Add an explicit end-exclusive owned range to the project-owned native
|
||||||
|
interface and validate it against immutable GGUF layer metadata.
|
||||||
|
- Restrict registration, optional scales, allocation and mmap ranges to the
|
||||||
|
owned `blk.N.*` tensors.
|
||||||
|
- Record authoritative loaded start/end and mapped/resident byte counters
|
||||||
|
from the instantiated model, not command-line input.
|
||||||
|
- Add a deterministic synthetic dense-Llama GGUF fixture plus native tests
|
||||||
|
for head, middle and tail ranges.
|
||||||
|
|
||||||
|
2. **DGR-005B — endpoint ownership and graph guard**
|
||||||
|
- Load token embeddings only for the head, and final norm/output head only
|
||||||
|
for the tail, including tied embeddings.
|
||||||
|
- Make the dense-Llama graph fail closed when an endpoint-required tensor is
|
||||||
|
absent; do not infer endpoint ownership from an empty pointer.
|
||||||
|
- Prove that split ranges map fewer bytes than the whole-model fixture and
|
||||||
|
that the loaded range report matches actual registered tensors.
|
||||||
|
|
||||||
|
3. **DGR-003-emission follow-up**
|
||||||
|
- Expose the resulting immutable native loaded-artifact report to a native
|
||||||
|
worker/backend adapter.
|
||||||
|
- Construct `ShardIdentity` only from that report plus the immutable
|
||||||
|
artifact, tokenizer and numerical-recipe inputs. The legacy Transformers
|
||||||
|
doctor path must remain identity-free rather than fabricate a pin.
|
||||||
|
- Wire `check_session_open()` at the worker SessionOpen boundary; current
|
||||||
|
unit coverage already verifies its fail-closed fingerprint, range,
|
||||||
|
session and epoch behavior.
|
||||||
|
|
||||||
|
## Handoff and non-claims
|
||||||
|
|
||||||
|
No DGR-005 source patch, identity-emission code, issue status, or `prd.json`
|
||||||
|
pass state was changed. No model was loaded, downloaded, benchmarked, or
|
||||||
|
certified. This document is a supervised-review handoff, not DGR-005 evidence
|
||||||
|
of completion.
|
||||||
79
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
79
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
# DGR-005 — dense-Llama range-aware GGUF ownership
|
||||||
|
|
||||||
|
Evidence class: deterministic offline/unit (synthetic fixture) plus
|
||||||
|
real-model integration (TinyLlama 1.1B, opt-in via MESHNET_ENABLE_REAL_INFERENCE_TESTS=1).
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
All six acceptance criteria pass:
|
||||||
|
|
||||||
|
1. **Range-aware tensor ownership**: native C++ patch (`0002-dense-llama-owned-range-loader.patch`,
|
||||||
|
169 lines as merged — DGR-005A's original 365-line version was slimmed by DGR-005B)
|
||||||
|
adds `llama_model_params.meshnet_owned_layer_start/end`, `llama_meshnet_range_report`,
|
||||||
|
and restricts `blk.N.*` registration to the owned range.
|
||||||
|
2. **Head/tail embedding loading**: head loads `token_embd.weight`; tail loads `output_norm`/`output`
|
||||||
|
(with tied-embedding dedup). Middle shards load zero endpoint tensors.
|
||||||
|
3. **Mapped/resident memory scales with owned tensors**: proven with TinyLlama 1.1B Q4_K_M.
|
||||||
|
4. **Targeted pytest tests**: `tests/test_llama_cpp_dependency.py` (3 tests — lock/patch
|
||||||
|
manifest consistency, offline dependency report, control-plane-code scan; re-verified
|
||||||
|
2026-07-14: `3 passed, 6 skipped` together with the opt-in integration file), native CTest
|
||||||
|
(`test-meshnet-range-ownership` synthetic fixture, added by the 0002 patch).
|
||||||
|
5. **compileall, ruff, git diff --check, full pytest**: all pass.
|
||||||
|
6. **Integration test**: `tests/test_gguf_distributed_load.py` (6/6, opt-in real model).
|
||||||
|
|
||||||
|
## Files changed (vs HEAD at DGR-004)
|
||||||
|
|
||||||
|
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch` — 169-line native patch (as merged)
|
||||||
|
- `packages/node/native/llama/patches/SHA256SUMS` — updated hash
|
||||||
|
- `packages/node/native/llama/patches/series` — added patch to series
|
||||||
|
- `packages/node/native/llama/UPSTREAM_LOCK.json` — updated patched_tree, serial number
|
||||||
|
- `scripts/llama_cpp_dependency.py` — `inspect` report for 2-patch stack
|
||||||
|
- `tests/test_llama_cpp_dependency.py` — patch_count 2
|
||||||
|
- `packages/node/native/llama/meshnet-range-loader.cpp` — C CLI wrapper
|
||||||
|
- `tests/test_gguf_distributed_load.py` — real-model integration test
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```text
|
||||||
|
# Build patched llama.cpp + range loader
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||||
|
-S build/dgr-004-final/source -B build/dgr-004-final/build \
|
||||||
|
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_EXAMPLES=ON \
|
||||||
|
-DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=OFF \
|
||||||
|
-DLLAMA_BUILD_TOOLS=ON -DLLAMA_CURL=OFF
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||||
|
--build build/dgr-004-final/build --target llama-simple -j$(nproc)
|
||||||
|
g++ -std=c++17 -Ibuild/dgr-004-final/source -Ibuild/dgr-004-final/source/include \
|
||||||
|
-Ibuild/dgr-004-final/source/ggml/include -Lbuild/dgr-004-final/build/bin \
|
||||||
|
packages/node/native/llama/meshnet-range-loader.cpp -lllama \
|
||||||
|
-Wl,-rpath,build/dgr-004-final/build/bin \
|
||||||
|
-o build/dgr-004-final/build/bin/meshnet-range-loader
|
||||||
|
|
||||||
|
# Focused tests (no model download)
|
||||||
|
PYTHONPATH=packages/node:packages/tracker:packages/contracts
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||||
|
-m pytest -q tests/test_llama_cpp_dependency.py
|
||||||
|
|
||||||
|
# Real-model integration test (opt-in, downloads ~670 MB)
|
||||||
|
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=... \
|
||||||
|
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||||
|
-m pytest -q tests/test_gguf_distributed_load.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Dense-Llama architecture only (LLM_ARCH_LLAMA). GLM/MoE/MLA is DGR-006+.
|
||||||
|
- Graph-level endpoint assertions (`has_token_embeddings`, `has_output_head`) were
|
||||||
|
simplified to only `start_layer`/`end_layer`/`mapped_bytes`/`resident_bytes` in
|
||||||
|
the patch as merged. Full endpoint tracking is available via the integration test
|
||||||
|
by observing which tensors are registered per shard.
|
||||||
|
- Loading the full `llama-simple` CLI requires reconfiguring with `-DLLAMA_BUILD_EXAMPLES=ON`.
|
||||||
|
The smoke-only build (`llama-gguf-hash`) is sufficient for patch verification.
|
||||||
|
- TinyLlama 1.1B is a baseline dense-Llama architecture only.
|
||||||
|
|
||||||
|
## Commits
|
||||||
|
|
||||||
|
- `252d131` feat: DGR-005A dense Llama owned range loader
|
||||||
|
- `f844ae6` feat: DGR-005B endpoint ownership and graph guard
|
||||||
|
- `31065c0` feat: distributed GGUF shard load integration test with TinyLlama 1.1B
|
||||||
|
- `d6b808d` chore: mark DGR-005 passes:true in PRD
|
||||||
71
.scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
Normal file
71
.scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
# DGR-006 — architecture-defined boundary input/output
|
||||||
|
|
||||||
|
Status: complete deterministic/offline contract and dense-fixture evidence.
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
The native protocol now carries a versioned `TensorBundle` on the decode fast
|
||||||
|
path. It includes explicit architecture and boundary-point metadata. Its legacy
|
||||||
|
`NamedTensor` field remains a compact one-tensor encoding for certified dense
|
||||||
|
boundaries; the writer deliberately selects it only for a one-tensor bundle and
|
||||||
|
new readers wrap that representation into a bundle. The bundle is authoritative
|
||||||
|
when present, allowing MoE/MLA sidebands without a second transport contract.
|
||||||
|
|
||||||
|
`architecture_boundary.py` is the fail-closed adapter boundary. Dense head
|
||||||
|
Shards accept token IDs and own embedding. Middle/tail Shards accept only a
|
||||||
|
validated bundle. Dense, MoE, and MLA route through explicit adapters; unknown
|
||||||
|
architectures are rejected. The dense F32 fixture proves whole-model versus
|
||||||
|
two-range boundary parity without model downloads or real inference.
|
||||||
|
|
||||||
|
Tail output is explicit in the schema: `TailResult` contains either logits or a
|
||||||
|
sampled token and binds sampling parameters plus request ID, runtime recipe,
|
||||||
|
chat template/version, reasoning mode, and architecture identity. The adapter
|
||||||
|
builds and validates the serialized protobuf result before returning it.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `packages/node/native/proto/shard_runtime.proto`
|
||||||
|
- `packages/node/meshnet_node/native_protocol/{codec.py,__init__.py,conformance.py,generated/*}`
|
||||||
|
- `packages/node/native/testdata/decode_step_golden.binpb`
|
||||||
|
- `packages/node/native/tests/test_shard_protocol_conformance.cpp`
|
||||||
|
- `packages/node/meshnet_node/architecture_boundary.py`
|
||||||
|
- `tests/test_architecture_boundary.py`
|
||||||
|
- `tests/test_native_shard_protocol.py`
|
||||||
|
- `packages/node/native/README.md`
|
||||||
|
|
||||||
|
## Commands and results
|
||||||
|
|
||||||
|
All Python commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python`.
|
||||||
|
All native commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake`.
|
||||||
|
|
||||||
|
```text
|
||||||
|
python scripts/generate_native_protocol.py --check -> passed
|
||||||
|
python scripts/generate_protocol_goldens.py --check -> passed
|
||||||
|
pytest -q tests/test_architecture_boundary.py \
|
||||||
|
tests/test_native_shard_protocol.py tests/test_llama_cpp_dependency.py
|
||||||
|
-> 59 passed
|
||||||
|
cmake -S packages/node/native -B build/native \
|
||||||
|
-DCMAKE_PREFIX_PATH=/tmp/pbsrc/install -> configured
|
||||||
|
cmake --build build/native -j$(nproc) -> built shard_protocol_conformance
|
||||||
|
ctest --test-dir build/native --output-on-failure -> 1/1 passed
|
||||||
|
python -m compileall -q packages tests -> passed
|
||||||
|
git diff --check -> passed
|
||||||
|
pytest -q -> 917 passed, 18 skipped
|
||||||
|
```
|
||||||
|
|
||||||
|
## Compatibility and limitations
|
||||||
|
|
||||||
|
- Existing Nodes that send `DecodeStep.tensor` are accepted. New multi-tensor
|
||||||
|
Nodes require the versioned bundle and older Nodes safely preserve it as an
|
||||||
|
unknown field rather than interpreting it as a single tensor.
|
||||||
|
- The committed C++ conformance vector covers the multi-tensor decode path.
|
||||||
|
- The dense parity result is a deterministic F32 structural fixture, not real
|
||||||
|
GGUF inference or GLM certification. No real inference was run.
|
||||||
|
- MoE and MLA adapters define and validate their sideband contracts but are not
|
||||||
|
architecture certifications. DGR-019 owns GLM MoE/MLA/DSA/IndexShare semantics.
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
DGR-007 can key its Hot KV state to the validated decoded bundle. DGR-008 can
|
||||||
|
translate the generated `TailResult` and decode bundle over gRPC. DGR-019 must
|
||||||
|
replace the generic MoE/MLA sideband names with exact certified GLM semantics.
|
||||||
@@ -61,7 +61,7 @@ New — runtime-loadable package (single source of truth):
|
|||||||
| `packages/node/meshnet_node/glm_alpha/data/architecture-snapshot.json` | Pinned architecture + config/template hashes |
|
| `packages/node/meshnet_node/glm_alpha/data/architecture-snapshot.json` | Pinned architecture + config/template hashes |
|
||||||
| `packages/node/meshnet_node/glm_alpha/data/alpha-contract.json` | Sealed acceptance thresholds (`aab23220…`) |
|
| `packages/node/meshnet_node/glm_alpha/data/alpha-contract.json` | Sealed acceptance thresholds (`aab23220…`) |
|
||||||
| `scripts/refresh_glm_target_manifest.py` | Re-resolve/verify pins from upstream metadata (`--check` / `--write`) |
|
| `scripts/refresh_glm_target_manifest.py` | Re-resolve/verify pins from upstream metadata (`--check` / `--write`) |
|
||||||
| `tests/test_glm_alpha_target.py` | 97 deterministic offline tests |
|
| `tests/test_glm_alpha_target.py` | 97 deterministic offline tests (99 after the late-review repair — see §4a) |
|
||||||
|
|
||||||
New — evidence:
|
New — evidence:
|
||||||
|
|
||||||
@@ -130,6 +130,15 @@ completed), then passed **5/5** in isolation and passed in the integrated full-s
|
|||||||
rerun above. This story touches no tracker code; the failed run is retained in
|
rerun above. This story touches no tracker code; the failed run is retained in
|
||||||
`commands.txt` rather than hidden.
|
`commands.txt` rather than hidden.
|
||||||
|
|
||||||
|
### 4a. Late independent-review repair (2026-07-14)
|
||||||
|
|
||||||
|
During delayed DGR-003 review, two contract-continuity defects were found and
|
||||||
|
fixed here: v1 now has an independently trusted digest pinned in code
|
||||||
|
(`test_resealing_a_mutated_v1_contract_is_rejected`) and parsed nested contract
|
||||||
|
state is recursively immutable. This added two tests; the suite is now
|
||||||
|
**99 passed** (`commands.txt` §7 records the exact runs). All "97" figures
|
||||||
|
elsewhere in this README describe the suite at original completion.
|
||||||
|
|
||||||
Planner output (`resource-plan.json`):
|
Planner output (`resource-plan.json`):
|
||||||
|
|
||||||
| Route | Fits | Headroom |
|
| Route | Fits | Headroom |
|
||||||
@@ -161,13 +170,13 @@ seeing a result. Each of those moves now has a test that names it:
|
|||||||
- admit the dense fallback → `test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected`
|
- admit the dense fallback → `test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected`
|
||||||
- shrink the reserve → `test_relaxing_the_per_node_reserve_after_the_fact_is_rejected`
|
- shrink the reserve → `test_relaxing_the_per_node_reserve_after_the_fact_is_rejected`
|
||||||
|
|
||||||
**Be precise about what the seal does.** `contract_sha256` is tamper-*evidence*, not
|
**Contract continuity is fail-closed.** The document’s `contract_sha256` detects
|
||||||
tamper-proofing. Nothing checked into a repo can stop an agent that rewrites the
|
accidental edits, and the approved v1 digest is pinned independently in code. A caller
|
||||||
threshold *and* re-seals the digest — `test_resealing_a_mutated_contract_changes_its_digest`
|
that changes a threshold and re-seals it under `glm-5.2-max-alpha/v1` is rejected by
|
||||||
asserts that path openly. What the seal removes is the *silent* mutation: the digest
|
`test_resealing_a_mutated_v1_contract_is_rejected`; amendments require a new supported
|
||||||
moves, and the change becomes a visible diff on a file whose entire purpose is to not
|
contract identity under human review. Parsed nested state is recursively immutable,
|
||||||
change, still claiming `contract_id: glm-5.2-max-alpha/v1`. Reviewers, not hashes, are
|
so thresholds cannot change between validation and use; `to_dict()` returns an isolated
|
||||||
the enforcement; the hash makes sure there is something to review.
|
copy rather than exposing the validated object.
|
||||||
|
|
||||||
## 6. Upstream status — the gating risk for DGR-004/DGR-018
|
## 6. Upstream status — the gating risk for DGR-004/DGR-018
|
||||||
|
|
||||||
|
|||||||
@@ -98,3 +98,18 @@ $VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cance
|
|||||||
-> 1 passed, repeated 5/5 in isolation
|
-> 1 passed, repeated 5/5 in isolation
|
||||||
$VP -m pytest -q # integrated rerun
|
$VP -m pytest -q # integrated rerun
|
||||||
-> 852 passed, 13 skipped in 253.30s (0:04:13)
|
-> 852 passed, 13 skipped in 253.30s (0:04:13)
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 7. Late independent-review repair (2026-07-14)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
PYTHONPATH=packages/node $VP -m pytest -q tests/test_glm_alpha_target.py
|
||||||
|
-> 99 passed in 0.15s
|
||||||
|
-> adds trusted-v1-digest rejection after coordinated mutation + reseal
|
||||||
|
-> adds nested parsed-state immutability and isolated to_dict() coverage
|
||||||
|
|
||||||
|
$VP -m pytest -q # after DGR-003 integration and DGR-017 repair
|
||||||
|
-> first run: 871 passed, 13 skipped, 1 known cancellation-race failure
|
||||||
|
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||||
|
-> 5/5 passed in isolation
|
||||||
|
$VP -m pytest -q # integrated rerun
|
||||||
|
-> 872 passed, 13 skipped in 253.46s (0:04:13)
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# DGR-018 — BLOCKED: no 256-GiB-class oracle host
|
||||||
|
|
||||||
|
Recorded: 2026-07-14 (MAINT-003). Preflight scripts preserved at commit
|
||||||
|
`a0f28b5` ("chore: preserve DGR-018 preflight scripts (postponed)").
|
||||||
|
|
||||||
|
## Blocker
|
||||||
|
|
||||||
|
DGR-018 requires a 256-GiB-class host with at least **224 GiB
|
||||||
|
runtime-accessible memory** (the DGR-017 experimental hard-fit floor for the
|
||||||
|
whole-model `UD-IQ1_S` oracle) and **250 GB free storage** on one filesystem
|
||||||
|
outside `/home` (216.715 GB artifact plus resume/temp headroom). The available
|
||||||
|
development host fails both gates, so the whole-model oracle cannot be
|
||||||
|
established. Per the issue's finish contract, no smaller model may be
|
||||||
|
substituted.
|
||||||
|
|
||||||
|
DGR-019 (needs the DGR-018 oracle for parity certification) and DGR-020
|
||||||
|
(needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single
|
||||||
|
node admits the whole recipe) are blocked transitively.
|
||||||
|
|
||||||
|
## Exact preflight output
|
||||||
|
|
||||||
|
Command (offline; resolves everything from the pinned target manifest and
|
||||||
|
never contacts the network):
|
||||||
|
|
||||||
|
```
|
||||||
|
$ python scripts/glm_whole_model_preflight.py
|
||||||
|
target: UD-IQ1_S 216.715 GB, 6 shards @ abc55e725277
|
||||||
|
[FAIL] storage: need >= 250 GB free on one filesystem outside ['/home']; observed no eligible filesystem
|
||||||
|
[FAIL] memory: need >= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor); observed 124.9 GiB MemTotal
|
||||||
|
destination: NONE — no filesystem outside ['/home'] has 250 GB free
|
||||||
|
- /run/media/popov/DATA (ext4): 74.2 GB free
|
||||||
|
- / (ext4): 51.1 GB free
|
||||||
|
- /run/media/popov/Windows (fuseblk): 26.0 GB free
|
||||||
|
- /run/media/popov/d (fuseblk): 5.1 GB free
|
||||||
|
verdict: fail
|
||||||
|
$ echo $?
|
||||||
|
1
|
||||||
|
```
|
||||||
|
|
||||||
|
Host: Linux 7.0.14-101.fc43.x86_64 x86_64, `MemTotal: 130997376 kB`
|
||||||
|
(124.9 GiB). The full machine-readable report (including the ordered
|
||||||
|
download/verify plan against revision `abc55e72527792c6e77069c99b4cb7de16fa9f23`,
|
||||||
|
manifest SHA-256 `0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`)
|
||||||
|
is in [preflight.json](preflight.json).
|
||||||
|
|
||||||
|
## How to resume
|
||||||
|
|
||||||
|
1. On a qualifying host, run `python scripts/glm_whole_model_preflight.py`
|
||||||
|
(optionally `--dest DIR`); it must exit 0 with `verdict: pass`.
|
||||||
|
2. Download shards in the preflight's ordered plan; verify each with
|
||||||
|
`python scripts/verify_glm_shards.py` before the next transfer starts.
|
||||||
|
3. Proceed with the DGR-018 issue
|
||||||
|
(`.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md`).
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
{
|
||||||
|
"generated_by": "scripts/glm_whole_model_preflight.py",
|
||||||
|
"target": {
|
||||||
|
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||||
|
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||||
|
"quantization": "UD-IQ1_S",
|
||||||
|
"shard_count": 6,
|
||||||
|
"total_bytes": 216715360960,
|
||||||
|
"total_gb": 216.715,
|
||||||
|
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962"
|
||||||
|
},
|
||||||
|
"forbidden_path_prefixes": [
|
||||||
|
"/home"
|
||||||
|
],
|
||||||
|
"mounts": [
|
||||||
|
{
|
||||||
|
"mountpoint": "/run/media/popov/DATA",
|
||||||
|
"fstype": "ext4",
|
||||||
|
"total_gb": 1208.8,
|
||||||
|
"free_gb": 74.2,
|
||||||
|
"free_bytes": 74201321472,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"mountpoint": "/",
|
||||||
|
"fstype": "ext4",
|
||||||
|
"total_gb": 217.7,
|
||||||
|
"free_gb": 51.1,
|
||||||
|
"free_bytes": 51073683456,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"mountpoint": "/run/media/popov/Windows",
|
||||||
|
"fstype": "fuseblk",
|
||||||
|
"total_gb": 434.9,
|
||||||
|
"free_gb": 26.0,
|
||||||
|
"free_bytes": 25964466176,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"mountpoint": "/run/media/popov/d",
|
||||||
|
"fstype": "fuseblk",
|
||||||
|
"total_gb": 161.1,
|
||||||
|
"free_gb": 5.1,
|
||||||
|
"free_bytes": 5148332032,
|
||||||
|
"forbidden": false,
|
||||||
|
"eligible": false
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"chosen_destination": null,
|
||||||
|
"checks": [
|
||||||
|
{
|
||||||
|
"check": "storage",
|
||||||
|
"requirement": ">= 250 GB free on one filesystem outside ['/home']",
|
||||||
|
"observed": "no eligible filesystem",
|
||||||
|
"passes": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"check": "memory",
|
||||||
|
"requirement": ">= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor)",
|
||||||
|
"observed": "124.9 GiB MemTotal",
|
||||||
|
"passes": false,
|
||||||
|
"waived": false
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"download_authorized": false,
|
||||||
|
"storage_only": false,
|
||||||
|
"download_plan": [
|
||||||
|
{
|
||||||
|
"step": 1,
|
||||||
|
"shard_index": 1,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||||
|
"size_bytes": 9423744,
|
||||||
|
"size_gb": 0.009,
|
||||||
|
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 2,
|
||||||
|
"shard_index": 6,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||||
|
"size_bytes": 19171063136,
|
||||||
|
"size_gb": 19.171,
|
||||||
|
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 6"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 3,
|
||||||
|
"shard_index": 2,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||||
|
"size_bytes": 49208128256,
|
||||||
|
"size_gb": 49.208,
|
||||||
|
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 2"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 4,
|
||||||
|
"shard_index": 3,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||||
|
"size_bytes": 49684417024,
|
||||||
|
"size_gb": 49.684,
|
||||||
|
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 5,
|
||||||
|
"shard_index": 4,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||||
|
"size_bytes": 49396052864,
|
||||||
|
"size_gb": 49.396,
|
||||||
|
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 4"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"step": 6,
|
||||||
|
"shard_index": 5,
|
||||||
|
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||||
|
"size_bytes": 49246275936,
|
||||||
|
"size_gb": 49.246,
|
||||||
|
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
|
||||||
|
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||||
|
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\"",
|
||||||
|
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 5"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"verdict": "fail"
|
||||||
|
}
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
# Maintenance review handoff — distributed GGUF runtime
|
||||||
|
|
||||||
|
Date: 2026-07-14
|
||||||
|
Scope: close the maintenance review, preserve the hard blockers, and hand off the remaining implementation work to the next model.
|
||||||
|
|
||||||
|
## What is complete
|
||||||
|
|
||||||
|
- Completed stories are now recorded in `docs/issues/distributed-gguf-runtime/`.
|
||||||
|
- The PRD and milestone docs were updated to reflect the closed set and the blocked set.
|
||||||
|
- The DGR-018 preflight scripts were preserved at commit `a0f28b5`.
|
||||||
|
- The current feature line has delivered DGR-001 through DGR-006 and DGR-017.
|
||||||
|
|
||||||
|
## Hard / unsolved issues for later
|
||||||
|
|
||||||
|
### 1) DGR-018 requires hardware we do not have
|
||||||
|
|
||||||
|
DGR-018 is blocked because the whole-model GLM-5.2 UD-IQ1_S oracle requires:
|
||||||
|
|
||||||
|
- a **256-GiB-class host**,
|
||||||
|
- at least **224 GiB runtime-accessible memory**,
|
||||||
|
- at least **250 GB free storage on one filesystem outside `/home`**.
|
||||||
|
|
||||||
|
The current development host reports only **124.9 GiB MemTotal** and has no eligible filesystem with 250 GB free.
|
||||||
|
The authoritative blocker evidence is in `evidence/DGR-018/BLOCKED.md` and `evidence/DGR-018/preflight.json`.
|
||||||
|
|
||||||
|
### 2) DGR-019 and DGR-020 are transitively blocked
|
||||||
|
|
||||||
|
- **DGR-019** needs the DGR-018 oracle for parity certification.
|
||||||
|
- **DGR-020** needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single node can admit the whole recipe.
|
||||||
|
|
||||||
|
No smaller model may be substituted for these stories.
|
||||||
|
|
||||||
|
### 3) The remainder of the graph stays blocked unless replanned
|
||||||
|
|
||||||
|
The current graph makes **DGR-007 depend on DGR-019**, which means:
|
||||||
|
|
||||||
|
- DGR-007 through DGR-016 are also blocked transitively.
|
||||||
|
- Unblocking the dense pipeline without the 256-GiB host would require an explicit replanning decision to relax the DGR-007 → DGR-019 dependency.
|
||||||
|
- That replanning decision has **not** been made.
|
||||||
|
|
||||||
|
### 4) Maintenance-only tasks should stay separate from feature implementation
|
||||||
|
|
||||||
|
The review uncovered that the codebase now has a clean closed-story split, but further work should avoid mixing:
|
||||||
|
|
||||||
|
- maintenance cleanup,
|
||||||
|
- blocked-hardware preparation,
|
||||||
|
- and actual distributed GLM implementation.
|
||||||
|
|
||||||
|
The next model should treat the maintenance pass as closed and only pick up real implementation work that is not hardware-blocked.
|
||||||
|
|
||||||
|
## Recommended next move
|
||||||
|
|
||||||
|
Use the next model to continue on the **non-blocked implementation queue** only.
|
||||||
|
Priority candidates are whatever is still actionable without the GLM oracle host; if a story depends on DGR-018, keep it deferred.
|
||||||
|
|
||||||
|
## Reference files
|
||||||
|
|
||||||
|
- `docs/issues/distributed-gguf-runtime/README.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/PRD.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/milestones.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`
|
||||||
|
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/preflight.json`
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
# 04 — Create the reproducible pinned llama.cpp patch stack
|
|
||||||
|
|
||||||
Status: ready-for-agent
|
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
|
||||||
|
|
||||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
|
||||||
- This issue is `DGR-004` in [prd.json](../prd.json).
|
|
||||||
- Read the evidence README for every dependency listed below.
|
|
||||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
|
||||||
|
|
||||||
## Description
|
|
||||||
|
|
||||||
As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
|
|
||||||
|
|
||||||
## Expected durable outputs
|
|
||||||
|
|
||||||
- Exact llama.cpp upstream pin
|
|
||||||
- Numbered minimal patch stack
|
|
||||||
- Reproducible fetch/apply/build smoke
|
|
||||||
- evidence/DGR-004/README.md
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
|
|
||||||
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
|
|
||||||
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
|
|
||||||
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
|
|
||||||
- [ ] Preserve upstream license and attribution notices.
|
|
||||||
- [ ] Add a clean rebuild smoke test that does not download a model.
|
|
||||||
- [ ] Targeted pytest tests pass
|
|
||||||
- [ ] python -m compileall packages tests passes for Python changes
|
|
||||||
- [ ] git diff --check passes
|
|
||||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
|
||||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
|
||||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
|
||||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
|
||||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
|
||||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
|
||||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
|
||||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
|
||||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
|
||||||
|
|
||||||
## Dependency handoff
|
|
||||||
|
|
||||||
- `DGR-001` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
|
|
||||||
|
|
||||||
## Finish contract
|
|
||||||
|
|
||||||
- Create the task evidence directory and durable handoff required above.
|
|
||||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
|
||||||
- Change this issue to `Status: done` only after all criteria pass.
|
|
||||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
|
||||||
- [PRD](../PRD.md)
|
|
||||||
- [Implementation strategy](../implementation-strategy.md)
|
|
||||||
- [Current architecture](../architecture.md)
|
|
||||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
|
||||||
@@ -1,6 +1,15 @@
|
|||||||
# 18 — Certify whole-model GLM-5.2 runtime semantics
|
# 18 — Certify whole-model GLM-5.2 runtime semantics
|
||||||
|
|
||||||
Status: ready-for-agent
|
Status: blocked (2026-07-14) — no 256-GiB-class host available
|
||||||
|
|
||||||
|
> **Blocked:** This story requires a 256-GiB-class host with at least 224 GiB
|
||||||
|
> runtime-accessible memory and 250 GB free storage outside `/home`. The
|
||||||
|
> development host has 124.9 GiB MemTotal and no eligible filesystem (largest:
|
||||||
|
> 74.2 GB free). Exact preflight output is preserved in
|
||||||
|
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md); preflight
|
||||||
|
> scripts were preserved at commit a0f28b5 (`scripts/glm_whole_model_preflight.py`,
|
||||||
|
> `scripts/verify_glm_shards.py`). Resume by re-running the preflight on a
|
||||||
|
> qualifying host — do not substitute a smaller model.
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,11 @@
|
|||||||
# 19 — Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
# 19 — Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
||||||
|
|
||||||
Status: ready-for-agent
|
Status: blocked (2026-07-14) — waiting on DGR-018
|
||||||
|
|
||||||
|
> **Blocked:** Depends on DGR-018's whole-model IQ1_S oracle, which is blocked
|
||||||
|
> on a 256-GiB-class host (≥ 224 GiB runtime-accessible memory). See
|
||||||
|
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md). Locked
|
||||||
|
> fixture/target parity cannot be certified without that oracle.
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,11 @@
|
|||||||
# 20 — Pass real distributed GLM-5.2 Max alpha acceptance
|
# 20 — Pass real distributed GLM-5.2 Max alpha acceptance
|
||||||
|
|
||||||
Status: ready-for-agent
|
Status: blocked (2026-07-14) — waiting on DGR-018/DGR-019
|
||||||
|
|
||||||
|
> **Blocked:** Depends on DGR-018 and DGR-019, both blocked on the 256-GiB-class
|
||||||
|
> oracle host (≥ 224 GiB runtime-accessible memory), plus enough physical
|
||||||
|
> consumer nodes that no single node admits the whole recipe. See
|
||||||
|
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md).
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ The exact alpha target and immutable acceptance gates are defined in [GLM-5.2-MA
|
|||||||
- DGR-017 locks official/GGUF revisions, `UD-IQ1_S`, hashes, resource accounting, Max-mode semantics, and alpha thresholds.
|
- DGR-017 locks official/GGUF revisions, `UD-IQ1_S`, hashes, resource accounting, Max-mode semantics, and alpha thresholds.
|
||||||
- DGR-003 builds exact recipe identity on DGR-002 and DGR-017.
|
- DGR-003 builds exact recipe identity on DGR-002 and DGR-017.
|
||||||
- DGR-004 creates the reproducible pinned llama.cpp boundary after stock GLM behavior is measured.
|
- DGR-004 creates the reproducible pinned llama.cpp boundary after stock GLM behavior is measured.
|
||||||
- DGR-018 certifies one whole-model `UD-IQ1_S` oracle with real MoE, DSA, IndexShare, KV, and `reasoning_effort=max` semantics.
|
- DGR-018 certifies one whole-model `UD-IQ1_S` oracle with real MoE, DSA, IndexShare, KV, and `reasoning_effort=max` semantics. **Blocked (2026-07-14):** requires a 256-GiB-class host (≥ 224 GiB runtime-accessible memory); see [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md). DGR-019 and DGR-020 are blocked transitively.
|
||||||
|
|
||||||
## Gate B — minimal native execution seam
|
## Gate B — minimal native execution seam
|
||||||
|
|
||||||
|
|||||||
@@ -28,7 +28,7 @@
|
|||||||
],
|
],
|
||||||
"priority": 1,
|
"priority": 1,
|
||||||
"passes": true,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md",
|
"notes": "Source issue: docs/issues/distributed-gguf-runtime/01-lock-the-safetensors-versus-gguf-performance-contract.md (moved on close, MAINT-003)",
|
||||||
"dependsOn": []
|
"dependsOn": []
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -55,7 +55,7 @@
|
|||||||
],
|
],
|
||||||
"priority": 2,
|
"priority": 2,
|
||||||
"passes": true,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md",
|
"notes": "Source issue: docs/issues/distributed-gguf-runtime/02-adopt-the-versioned-grpc-shard-protocol.md (moved on close, MAINT-003)",
|
||||||
"dependsOn": []
|
"dependsOn": []
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -80,8 +80,8 @@
|
|||||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||||
],
|
],
|
||||||
"priority": 4,
|
"priority": 4,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md",
|
"notes": "DGR-003-emission: native loaded-artifact adapter derives exact identity from immutable GGUF/runtime inputs, doctor emits it only for that adapter, and SessionOpen fails closed before acceptance. Exact identities remain tracker-uncertified and dark until a real distributed forward is certified. (moved on close, MAINT-003)",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-002",
|
"DGR-002",
|
||||||
"DGR-017"
|
"DGR-017"
|
||||||
@@ -112,8 +112,8 @@
|
|||||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||||
],
|
],
|
||||||
"priority": 5,
|
"priority": 5,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md",
|
"notes": "Source issue: docs/issues/distributed-gguf-runtime/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md (moved on close, MAINT-003)",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-001",
|
"DGR-001",
|
||||||
"DGR-017"
|
"DGR-017"
|
||||||
@@ -143,8 +143,8 @@
|
|||||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||||
],
|
],
|
||||||
"priority": 6,
|
"priority": 6,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md",
|
"notes": "Source issue: docs/issues/distributed-gguf-runtime/05-implement-dense-llama-range-aware-gguf-ownership.md (moved on close, MAINT-003)",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-003",
|
"DGR-003",
|
||||||
"DGR-004"
|
"DGR-004"
|
||||||
@@ -177,8 +177,8 @@
|
|||||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||||
],
|
],
|
||||||
"priority": 7,
|
"priority": 7,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md",
|
"notes": "Source issue: docs/issues/distributed-gguf-runtime/06-implement-architecture-defined-boundary-input-output.md (moved on close, MAINT-003)",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-002",
|
"DGR-002",
|
||||||
"DGR-005"
|
"DGR-005"
|
||||||
@@ -538,7 +538,7 @@
|
|||||||
],
|
],
|
||||||
"priority": 3,
|
"priority": 3,
|
||||||
"passes": true,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md",
|
"notes": "Source issue: docs/issues/distributed-gguf-runtime/17-lock-glm-5-2-max-target-and-alpha-contract.md (moved on close, MAINT-003)",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-001",
|
"DGR-001",
|
||||||
"DGR-002"
|
"DGR-002"
|
||||||
@@ -570,7 +570,7 @@
|
|||||||
],
|
],
|
||||||
"priority": 8,
|
"priority": 8,
|
||||||
"passes": false,
|
"passes": false,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md",
|
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md | BLOCKED (2026-07-14): requires a 256-GiB-class host with >= 224 GiB runtime-accessible memory and 250 GB free storage outside /home; development host has 124.9 GiB MemTotal and no eligible filesystem. Exact preflight output: evidence/DGR-018/BLOCKED.md. Preflight scripts preserved at commit a0f28b5.",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-003",
|
"DGR-003",
|
||||||
"DGR-004",
|
"DGR-004",
|
||||||
@@ -603,7 +603,7 @@
|
|||||||
],
|
],
|
||||||
"priority": 9,
|
"priority": 9,
|
||||||
"passes": false,
|
"passes": false,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md",
|
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md | BLOCKED (2026-07-14): depends on the DGR-018 whole-model IQ1_S oracle, which is blocked on a 256-GiB-class host (>= 224 GiB runtime-accessible memory). See evidence/DGR-018/BLOCKED.md.",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-005",
|
"DGR-005",
|
||||||
"DGR-006",
|
"DGR-006",
|
||||||
@@ -639,7 +639,7 @@
|
|||||||
],
|
],
|
||||||
"priority": 16,
|
"priority": 16,
|
||||||
"passes": false,
|
"passes": false,
|
||||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md",
|
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md | BLOCKED (2026-07-14): depends on DGR-018 and DGR-019 (both blocked on the 256-GiB-class oracle host) plus enough physical consumer nodes that no single node admits the whole recipe. See evidence/DGR-018/BLOCKED.md.",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"DGR-007",
|
"DGR-007",
|
||||||
"DGR-008",
|
"DGR-008",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# 03 — Define exact Artifact and runtime recipe identity
|
# 03 — Define exact Artifact and runtime recipe identity
|
||||||
|
|
||||||
Status: ready-for-agent
|
Status: done
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
@@ -21,21 +21,21 @@ As the Tracker, I need exact compatibility identity so that only numerically and
|
|||||||
|
|
||||||
## Acceptance criteria
|
## Acceptance criteria
|
||||||
|
|
||||||
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
- [x] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||||
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
- [x] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||||
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
|
- [x] Produce a stable compatibility fingerprint used by live capability emission/admission and the gRPC handshake. The native backend adapter derives it only from its immutable loaded-artifact report and immutable artifact/runtime pins; the legacy Transformers doctor path remains identity-free.
|
||||||
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
- [x] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||||
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
- [x] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||||
- [ ] Targeted pytest tests pass
|
- [x] Targeted pytest tests pass
|
||||||
- [ ] python -m compileall packages tests passes for Python changes
|
- [x] python -m compileall packages tests passes for Python changes
|
||||||
- [ ] git diff --check passes
|
- [x] git diff --check passes
|
||||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
- [x] Read and verify every dependency evidence README before relying on dependency behavior
|
||||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||||
|
|
||||||
## Dependency handoff
|
## Dependency handoff
|
||||||
|
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
# 04 — Chain: DGR-005 + DGR-003-emission + anchor
|
||||||
|
|
||||||
|
Status: done
|
||||||
|
|
||||||
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
|
- DGR-004 is COMPLETED and PUSHED at f9722e7.
|
||||||
|
- Current HEAD is f9722e7. Worktree is clean.
|
||||||
|
- The project venv is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv — use its full paths for python, cmake, pytest. Example: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q`.
|
||||||
|
- cmake is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake.
|
||||||
|
- Protobuf runtime is 7.35.1.
|
||||||
|
- DGR-001 CPU verdict remains immutable STOP.
|
||||||
|
- DGR-017 is a separate alpha contract.
|
||||||
|
- ALL builds are infrastructure evidence — NEVER claim GLM semantic acceptance, numerical equivalence, or route certification.
|
||||||
|
- Stock dense-MLA fallback remains explicitly uncertified.
|
||||||
|
|
||||||
|
## CRITICAL: After each story, commit and push
|
||||||
|
|
||||||
|
After EVERY story below is complete:
|
||||||
|
1. `git add` ONLY files belonging to that story
|
||||||
|
2. `git commit -m "feat: <story-id> - <brief description>"`
|
||||||
|
3. `git push origin ralph/dgr-001-performance-contract`
|
||||||
|
4. Verify local == remote SHA
|
||||||
|
5. Update that story's issue Status: done and prd.json passes: true
|
||||||
|
|
||||||
|
## Story 1 — DGR-005: Exact dense-Llama range-aware GGUF ownership
|
||||||
|
|
||||||
|
Read `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely.
|
||||||
|
Depends on DGR-003 (identity definitions) and DGR-004 (native patch stack) — both are pushed.
|
||||||
|
|
||||||
|
Map only the assigned dense-Llama Shard range so aggregate consumer memory can hold a model larger than one node.
|
||||||
|
|
||||||
|
- Register and allocate only `blk.N.*` tensors in the assigned range
|
||||||
|
- Load embeddings only for head, final norm/LM head only for tail, including tied embeddings
|
||||||
|
- Prefer range-aware mapping from one exact source GGUF
|
||||||
|
- Report authoritative loaded range from the model, not CLI claims
|
||||||
|
- Mapped/resident memory scales with owned tensors, not full model size
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
- [ ] Range-aware tensor ownership with exact start/end layer guard
|
||||||
|
- [ ] Head/tail embedding loading is correct
|
||||||
|
- [ ] Mapped memory scales with owned tensors
|
||||||
|
- [ ] Targeted pytest tests pass
|
||||||
|
- [ ] Native C++ target builds and focused CTest pass
|
||||||
|
- [ ] compileall, ruff, git diff --check, full pytest
|
||||||
|
- [ ] Write DGR-005 evidence/README.md and commands.txt
|
||||||
|
- [ ] Commit and push
|
||||||
|
|
||||||
|
## Story 2 — DGR-003-emission: Wire live ShardIdentity from native seam
|
||||||
|
|
||||||
|
DGR-003 is reopened: production doctor/backend path cannot derive an exact ShardIdentity from authoritative loaded-artifact/runtime state. DGR-004 and DGR-005 provide the native seam. Wire it.
|
||||||
|
|
||||||
|
- Construct ShardIdentity from actual immutable artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout, schema versions, and owned range
|
||||||
|
- At SessionOpen, compare CompatibilityFingerprint and return ERROR_CODE_FINGERPRINT_MISMATCH
|
||||||
|
- Production doctor/backend capability report must emit exact identity block
|
||||||
|
- A digest match proves canonical consistency, NOT node authenticity
|
||||||
|
- Only tracker-owned certification of a real distributed forward moves a recipe out of dark
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
- [ ] Production doctor/backend emits exact ShardIdentity in capability report
|
||||||
|
- [ ] check_session_open validates fingerprint, range, session, epoch
|
||||||
|
- [ ] Stale epoch rejected with ERROR_CODE_EPOCH_STALE
|
||||||
|
- [ ] All DGR-003 focused/adversarial tests pass
|
||||||
|
- [ ] compileall, ruff, git diff --check, full pytest
|
||||||
|
- [ ] Update DGR-003 issue to Status: done and prd.json passes: true
|
||||||
|
- [ ] Commit and push
|
||||||
|
|
||||||
|
## Story 3 — LOW PRIORITY: Read-only audit
|
||||||
|
|
||||||
|
Audit every file in the repository for undocumented assumptions about model architecture, runtime behavior, protocol semantics, memory layout, threading model, error handling, platform compatibility, and upstream API stability.
|
||||||
|
|
||||||
|
This story is INTENTIONALLY IMPOSSIBLE. Work on it ONLY after all previous stories are committed and pushed. Do not modify files — read-only analysis only. Never mark this story complete. When you reach it, report "ANCHOR: ready for new tasks" and keep working. Never exit.
|
||||||
|
- [ ] Read-only repository assumption audit (never completes)
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# 05 — Implement dense-Llama range-aware GGUF ownership
|
# 05 — Implement dense-Llama range-aware GGUF ownership
|
||||||
|
|
||||||
Status: ready-for-agent
|
Status: done
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# 06 — Implement architecture-defined boundary input/output
|
# 06 — Implement architecture-defined boundary input/output
|
||||||
|
|
||||||
Status: ready-for-agent
|
Status: done
|
||||||
|
|
||||||
## Mandatory fresh-session context
|
## Mandatory fresh-session context
|
||||||
|
|
||||||
26
docs/issues/distributed-gguf-runtime/README.md
Normal file
26
docs/issues/distributed-gguf-runtime/README.md
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
# Distributed GGUF runtime — closed issues
|
||||||
|
|
||||||
|
Completed stories from the `distributed-gguf-runtime` feature
|
||||||
|
(`.scratch/distributed-gguf-runtime/`), moved here on 2026-07-14 (MAINT-003).
|
||||||
|
Numbering is the feature's own `DGR-NNN` series and is unrelated to the
|
||||||
|
top-level `docs/issues/` numbering.
|
||||||
|
|
||||||
|
These files are historical records: internal path references (for example
|
||||||
|
`.scratch/distributed-gguf-runtime/issues/…`) reflect where the files lived
|
||||||
|
while the stories were active. Authoritative completion status is
|
||||||
|
`passes: true` in `.scratch/distributed-gguf-runtime/prd.json`, backed by the
|
||||||
|
signed evidence in `.scratch/distributed-gguf-runtime/evidence/DGR-*/`.
|
||||||
|
|
||||||
|
| Story | Issue | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| DGR-001 | [01-lock-the-safetensors-versus-gguf-performance-contract.md](01-lock-the-safetensors-versus-gguf-performance-contract.md) | `evidence/DGR-001/` |
|
||||||
|
| DGR-002 | [02-adopt-the-versioned-grpc-shard-protocol.md](02-adopt-the-versioned-grpc-shard-protocol.md) | `evidence/DGR-002/` |
|
||||||
|
| DGR-003 | [03-define-exact-artifact-and-runtime-recipe-identity.md](03-define-exact-artifact-and-runtime-recipe-identity.md) | `evidence/DGR-003/` |
|
||||||
|
| DGR-004 | [04-create-the-reproducible-pinned-llama-cpp-patch-stack.md](04-create-the-reproducible-pinned-llama-cpp-patch-stack.md) | `evidence/DGR-004/` |
|
||||||
|
| DGR-005 | [05-implement-dense-llama-range-aware-gguf-ownership.md](05-implement-dense-llama-range-aware-gguf-ownership.md) | `evidence/DGR-005/` |
|
||||||
|
| DGR-006 | [06-implement-architecture-defined-boundary-input-output.md](06-implement-architecture-defined-boundary-input-output.md) | `evidence/DGR-006/` |
|
||||||
|
| DGR-017 | [17-lock-glm-5-2-max-target-and-alpha-contract.md](17-lock-glm-5-2-max-target-and-alpha-contract.md) | `evidence/DGR-017/` |
|
||||||
|
|
||||||
|
Open and blocked stories remain in `.scratch/distributed-gguf-runtime/issues/`.
|
||||||
|
DGR-018, DGR-019, and DGR-020 are blocked on a 256-GiB-class host — see
|
||||||
|
`.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`.
|
||||||
@@ -3,7 +3,6 @@
|
|||||||
import http.server
|
import http.server
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import os
|
|
||||||
from collections import Counter
|
from collections import Counter
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
import threading
|
import threading
|
||||||
@@ -62,7 +61,7 @@ class _GatewayHTTPServer(http.server.HTTPServer):
|
|||||||
|
|
||||||
|
|
||||||
class _GatewayHandler(http.server.BaseHTTPRequestHandler):
|
class _GatewayHandler(http.server.BaseHTTPRequestHandler):
|
||||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def do_GET(self):
|
def do_GET(self):
|
||||||
|
|||||||
186
packages/node/meshnet_node/architecture_boundary.py
Normal file
186
packages/node/meshnet_node/architecture_boundary.py
Normal file
@@ -0,0 +1,186 @@
|
|||||||
|
"""Certified architecture adapters for the public TensorBundle boundary.
|
||||||
|
|
||||||
|
The adapter is intentionally small: it owns boundary names and endpoint rules,
|
||||||
|
not transformer execution. llama.cpp owns local graphs; callers select a
|
||||||
|
certified adapter before accepting an activation from another Shard.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import Enum
|
||||||
|
import struct
|
||||||
|
from typing import Callable, Mapping, Sequence
|
||||||
|
|
||||||
|
from .native_protocol import (
|
||||||
|
HIDDEN_STATES,
|
||||||
|
ProtocolError,
|
||||||
|
encode_bundle,
|
||||||
|
encode_tensor,
|
||||||
|
pb,
|
||||||
|
validate_tail_result,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Architecture(str, Enum):
|
||||||
|
DENSE = "dense"
|
||||||
|
MOE = "moe"
|
||||||
|
MLA = "mla"
|
||||||
|
|
||||||
|
|
||||||
|
class BoundaryStage(str, Enum):
|
||||||
|
HEAD = "head"
|
||||||
|
MIDDLE = "middle"
|
||||||
|
TAIL = "tail"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProtocolIdentity:
|
||||||
|
request_id: str
|
||||||
|
runtime_recipe_digest: str
|
||||||
|
chat_template_id: str
|
||||||
|
chat_template_version: str
|
||||||
|
reasoning_mode: str
|
||||||
|
architecture: Architecture
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SamplingParameters:
|
||||||
|
temperature: float
|
||||||
|
top_p: float
|
||||||
|
top_k: int
|
||||||
|
seed: int
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TailOutput:
|
||||||
|
kind: str
|
||||||
|
value: int | object
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def sampled_token(cls, token_id: int) -> "TailOutput":
|
||||||
|
if token_id < 0:
|
||||||
|
raise ProtocolError("sampled token id must be non-negative")
|
||||||
|
return cls("sampled_token", token_id)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TypedTailResult:
|
||||||
|
identity: ProtocolIdentity
|
||||||
|
sampling: SamplingParameters
|
||||||
|
output_kind: str
|
||||||
|
message: pb.TailResult
|
||||||
|
|
||||||
|
@property
|
||||||
|
def sampled_token_id(self) -> int | None:
|
||||||
|
return self.message.sampled_token_id if self.output_kind == "sampled_token_id" else None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ArchitectureBoundaryAdapter:
|
||||||
|
architecture: Architecture
|
||||||
|
required_names: frozenset[str]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def protocol_architecture(self) -> int:
|
||||||
|
return {
|
||||||
|
Architecture.DENSE: pb.ARCHITECTURE_TYPE_DENSE,
|
||||||
|
Architecture.MOE: pb.ARCHITECTURE_TYPE_MOE,
|
||||||
|
Architecture.MLA: pb.ARCHITECTURE_TYPE_MLA,
|
||||||
|
}[self.architecture]
|
||||||
|
|
||||||
|
def bundle_from_token_ids(
|
||||||
|
self,
|
||||||
|
token_ids: Sequence[int],
|
||||||
|
token_embedding: Callable[[int], Sequence[float]],
|
||||||
|
):
|
||||||
|
"""Head-only embedding entry point; middle/tail never receive IDs."""
|
||||||
|
if self.architecture is not Architecture.DENSE:
|
||||||
|
raise ProtocolError("head token embedding is not certified for this architecture")
|
||||||
|
if not token_ids:
|
||||||
|
raise ProtocolError("head requires at least one token id")
|
||||||
|
rows = [tuple(token_embedding(token)) for token in token_ids]
|
||||||
|
if not rows or not rows[0] or any(len(row) != len(rows[0]) for row in rows):
|
||||||
|
raise ProtocolError("token embedding returned inconsistent hidden widths")
|
||||||
|
payload = struct.pack("<" + "f" * (len(rows) * len(rows[0])), *(x for row in rows for x in row))
|
||||||
|
return self.bundle_from_named_payloads({HIDDEN_STATES: payload}, shape=[1, len(rows), len(rows[0])])
|
||||||
|
|
||||||
|
def bundle_from_named_payloads(
|
||||||
|
self, payloads: Mapping[str, bytes], *, shape: Sequence[int] | None = None
|
||||||
|
):
|
||||||
|
names = set(payloads)
|
||||||
|
if not self.required_names <= names:
|
||||||
|
missing = sorted(self.required_names - names)
|
||||||
|
raise ProtocolError(f"{self.architecture.value} boundary requires {missing}")
|
||||||
|
tensors = []
|
||||||
|
for name, payload in payloads.items():
|
||||||
|
tensor_shape = list(shape) if name == HIDDEN_STATES and shape else [len(payload) // 4]
|
||||||
|
if len(payload) % 4:
|
||||||
|
raise ProtocolError(f"{name!r} F32 fixture payload is not word aligned")
|
||||||
|
tensors.append(encode_tensor(name, payload, tensor_shape, pb.DTYPE_FLOAT32))
|
||||||
|
return encode_bundle(
|
||||||
|
tensors,
|
||||||
|
architecture=self.protocol_architecture,
|
||||||
|
boundary_point="pre_tail_residual",
|
||||||
|
)
|
||||||
|
|
||||||
|
def input_for(self, stage: BoundaryStage, bundle):
|
||||||
|
"""Accept architecture state only after the head embedding boundary."""
|
||||||
|
if stage is BoundaryStage.HEAD:
|
||||||
|
raise ProtocolError("head accepts token ids and owns token embedding")
|
||||||
|
if bundle is None:
|
||||||
|
raise ProtocolError(f"{stage.value} requires a TensorBundle")
|
||||||
|
from .native_protocol import decode_bundle
|
||||||
|
|
||||||
|
payloads = decode_bundle(bundle)
|
||||||
|
if bundle.architecture != self.protocol_architecture:
|
||||||
|
raise ProtocolError("boundary architecture does not match certified adapter")
|
||||||
|
if bundle.boundary_point != "pre_tail_residual":
|
||||||
|
raise ProtocolError("unsupported architecture boundary point")
|
||||||
|
if not self.required_names <= set(payloads):
|
||||||
|
raise ProtocolError(f"{self.architecture.value} boundary requires {sorted(self.required_names)}")
|
||||||
|
return bundle
|
||||||
|
|
||||||
|
def tail_result(
|
||||||
|
self, *, identity: ProtocolIdentity, sampling: SamplingParameters, output: TailOutput
|
||||||
|
) -> TypedTailResult:
|
||||||
|
if identity.architecture is not self.architecture:
|
||||||
|
raise ProtocolError("tail result architecture does not match certified adapter")
|
||||||
|
if not identity.request_id or not identity.runtime_recipe_digest:
|
||||||
|
raise ProtocolError("tail result requires exact request and recipe identity")
|
||||||
|
if output.kind != "sampled_token":
|
||||||
|
raise ProtocolError("uncertified tail output kind")
|
||||||
|
message = pb.TailResult(
|
||||||
|
identity=pb.RequestRecipeIdentity(
|
||||||
|
request_id=identity.request_id,
|
||||||
|
runtime_recipe_digest=identity.runtime_recipe_digest,
|
||||||
|
chat_template_id=identity.chat_template_id,
|
||||||
|
chat_template_version=identity.chat_template_version,
|
||||||
|
reasoning_mode=identity.reasoning_mode,
|
||||||
|
architecture=self.protocol_architecture,
|
||||||
|
),
|
||||||
|
sampling=pb.SamplingParameters(
|
||||||
|
temperature=sampling.temperature,
|
||||||
|
top_p=sampling.top_p,
|
||||||
|
top_k=sampling.top_k,
|
||||||
|
seed=sampling.seed,
|
||||||
|
greedy=sampling.temperature == 0.0,
|
||||||
|
),
|
||||||
|
sampled_token_id=int(output.value),
|
||||||
|
)
|
||||||
|
validate_tail_result(message)
|
||||||
|
return TypedTailResult(identity, sampling, "sampled_token_id", message)
|
||||||
|
|
||||||
|
|
||||||
|
_ADAPTERS = {
|
||||||
|
Architecture.DENSE: ArchitectureBoundaryAdapter(Architecture.DENSE, frozenset({HIDDEN_STATES})),
|
||||||
|
Architecture.MOE: ArchitectureBoundaryAdapter(Architecture.MOE, frozenset({HIDDEN_STATES, "router_logits"})),
|
||||||
|
Architecture.MLA: ArchitectureBoundaryAdapter(Architecture.MLA, frozenset({HIDDEN_STATES, "mla_position_state"})),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def adapter_for(architecture: Architecture | str) -> ArchitectureBoundaryAdapter:
|
||||||
|
try:
|
||||||
|
return _ADAPTERS[Architecture(architecture)]
|
||||||
|
except (KeyError, ValueError):
|
||||||
|
raise ProtocolError(f"unsupported architecture {architecture!r}") from None
|
||||||
@@ -20,6 +20,8 @@ import time
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Mapping
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from .runtime_recipe import CompatibilityFingerprint, ShardIdentity
|
||||||
|
|
||||||
# Layout of the serialized report. Bump when the JSON shape changes.
|
# Layout of the serialized report. Bump when the JSON shape changes.
|
||||||
CAPABILITY_SCHEMA_VERSION = 1
|
CAPABILITY_SCHEMA_VERSION = 1
|
||||||
|
|
||||||
@@ -330,7 +332,16 @@ def _as_mapping(data: Any, field_name: str) -> Mapping[str, Any]:
|
|||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class CapabilityReport:
|
class CapabilityReport:
|
||||||
"""One node's validated (or failed) model/shard/recipe/backend combination."""
|
"""One node's validated (or failed) model/shard/recipe/backend combination.
|
||||||
|
|
||||||
|
`identity` is the exact DGR-003 artifact/runtime-recipe block: the separated
|
||||||
|
numerical axes and the compatibility fingerprint derived from them. It is
|
||||||
|
optional and additive — a node that predates DGR-003 presents none, and the
|
||||||
|
tracker falls back to the coarse label comparison it has always done
|
||||||
|
(ADR-0023's compat rollout). A node that *does* present one is held to it:
|
||||||
|
the tracker re-derives the fingerprint and refuses a report whose claim does
|
||||||
|
not match its own derivation.
|
||||||
|
"""
|
||||||
|
|
||||||
model: ModelIdentity
|
model: ModelIdentity
|
||||||
shard: ShardRange
|
shard: ShardRange
|
||||||
@@ -341,6 +352,7 @@ class CapabilityReport:
|
|||||||
duration_ms: int
|
duration_ms: int
|
||||||
diagnostics: tuple[str, ...] = ()
|
diagnostics: tuple[str, ...] = ()
|
||||||
schema_version: int = CAPABILITY_SCHEMA_VERSION
|
schema_version: int = CAPABILITY_SCHEMA_VERSION
|
||||||
|
identity: ShardIdentity | None = None
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
def __post_init__(self) -> None:
|
||||||
if self.status not in VALID_STATUSES:
|
if self.status not in VALID_STATUSES:
|
||||||
@@ -360,6 +372,11 @@ class CapabilityReport:
|
|||||||
def passed(self) -> bool:
|
def passed(self) -> bool:
|
||||||
return self.status == STATUS_PASSED
|
return self.status == STATUS_PASSED
|
||||||
|
|
||||||
|
@property
|
||||||
|
def fingerprint(self) -> CompatibilityFingerprint | None:
|
||||||
|
"""The exact compatibility fingerprint, when this node declares one."""
|
||||||
|
return None if self.identity is None else self.identity.fingerprint
|
||||||
|
|
||||||
def identity_key(self) -> tuple[str, int, int, str, str, str, str]:
|
def identity_key(self) -> tuple[str, int, int, str, str, str, str]:
|
||||||
"""The tuple a consumer must match to reuse this proof.
|
"""The tuple a consumer must match to reuse this proof.
|
||||||
|
|
||||||
@@ -380,7 +397,7 @@ class CapabilityReport:
|
|||||||
return max(0.0, (time.time() if now is None else now) - self.validated_at)
|
return max(0.0, (time.time() if now is None else now) - self.validated_at)
|
||||||
|
|
||||||
def to_dict(self) -> dict:
|
def to_dict(self) -> dict:
|
||||||
return {
|
doc = {
|
||||||
"schema_version": self.schema_version,
|
"schema_version": self.schema_version,
|
||||||
"model": self.model.to_dict(),
|
"model": self.model.to_dict(),
|
||||||
"shard": self.shard.to_dict(),
|
"shard": self.shard.to_dict(),
|
||||||
@@ -391,6 +408,9 @@ class CapabilityReport:
|
|||||||
"duration_ms": self.duration_ms,
|
"duration_ms": self.duration_ms,
|
||||||
"diagnostics": list(self.diagnostics),
|
"diagnostics": list(self.diagnostics),
|
||||||
}
|
}
|
||||||
|
if self.identity is not None:
|
||||||
|
doc["identity"] = self.identity.to_dict()
|
||||||
|
return doc
|
||||||
|
|
||||||
def to_json(self, indent: int | None = None) -> str:
|
def to_json(self, indent: int | None = None) -> str:
|
||||||
return json.dumps(self.to_dict(), indent=indent, sort_keys=True)
|
return json.dumps(self.to_dict(), indent=indent, sort_keys=True)
|
||||||
@@ -417,6 +437,7 @@ class CapabilityReport:
|
|||||||
):
|
):
|
||||||
raise CapabilityReportError("'validated_at' must be a Unix timestamp")
|
raise CapabilityReportError("'validated_at' must be a Unix timestamp")
|
||||||
|
|
||||||
|
raw_identity = doc.get("identity")
|
||||||
return cls(
|
return cls(
|
||||||
schema_version=schema_version,
|
schema_version=schema_version,
|
||||||
model=ModelIdentity.from_dict(doc.get("model")),
|
model=ModelIdentity.from_dict(doc.get("model")),
|
||||||
@@ -427,6 +448,9 @@ class CapabilityReport:
|
|||||||
validated_at=float(validated_at),
|
validated_at=float(validated_at),
|
||||||
duration_ms=_require_int(doc.get("duration_ms"), "duration_ms", 0),
|
duration_ms=_require_int(doc.get("duration_ms"), "duration_ms", 0),
|
||||||
diagnostics=sanitize_diagnostics(doc.get("diagnostics")),
|
diagnostics=sanitize_diagnostics(doc.get("diagnostics")),
|
||||||
|
identity=(
|
||||||
|
None if raw_identity is None else ShardIdentity.from_dict(raw_identity)
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -461,12 +485,14 @@ def build_capability_report(
|
|||||||
diagnostics: Any = None,
|
diagnostics: Any = None,
|
||||||
validated_at: float | None = None,
|
validated_at: float | None = None,
|
||||||
environ: Mapping[str, str] | None = None,
|
environ: Mapping[str, str] | None = None,
|
||||||
|
identity: ShardIdentity | None = None,
|
||||||
) -> CapabilityReport:
|
) -> CapabilityReport:
|
||||||
"""Assemble a report from flat validation results.
|
"""Assemble a report from flat validation results.
|
||||||
|
|
||||||
`model_config` may be the loaded config mapping (hashed into a fingerprint)
|
`model_config` may be the loaded config mapping (hashed into a fingerprint)
|
||||||
or an already-computed ``sha256:…`` string. `validated_at` defaults to now,
|
or an already-computed ``sha256:…`` string. `validated_at` defaults to now,
|
||||||
so callers that need determinism pass it explicitly.
|
so callers that need determinism pass it explicitly. `identity` is the exact
|
||||||
|
DGR-003 artifact/recipe block, when the backend can state one.
|
||||||
"""
|
"""
|
||||||
return CapabilityReport(
|
return CapabilityReport(
|
||||||
model=ModelIdentity(
|
model=ModelIdentity(
|
||||||
@@ -491,4 +517,5 @@ def build_capability_report(
|
|||||||
validated_at=time.time() if validated_at is None else validated_at,
|
validated_at=time.time() if validated_at is None else validated_at,
|
||||||
duration_ms=duration_ms,
|
duration_ms=duration_ms,
|
||||||
diagnostics=sanitize_diagnostics(diagnostics, environ),
|
diagnostics=sanitize_diagnostics(diagnostics, environ),
|
||||||
|
identity=identity,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -144,7 +144,7 @@ def _cmd_default(args) -> int:
|
|||||||
print("\nSetup cancelled.")
|
print("\nSetup cancelled.")
|
||||||
return 1
|
return 1
|
||||||
save_config(cfg)
|
save_config(cfg)
|
||||||
print(f"\nConfig saved to ~/.config/meshnet/config.json\n")
|
print("\nConfig saved to ~/.config/meshnet/config.json\n")
|
||||||
|
|
||||||
# Apply CLI overrides on top of saved config
|
# Apply CLI overrides on top of saved config
|
||||||
overrides: dict = {}
|
overrides: dict = {}
|
||||||
@@ -198,7 +198,7 @@ def _cmd_default(args) -> int:
|
|||||||
|
|
||||||
def _cmd_models(args) -> int:
|
def _cmd_models(args) -> int:
|
||||||
"""List curated models (with optional HF Hub browse)."""
|
"""List curated models (with optional HF Hub browse)."""
|
||||||
from .wizard import print_models_table, _browse_hf_interactive
|
from .wizard import print_models_table
|
||||||
|
|
||||||
if args.browse:
|
if args.browse:
|
||||||
from .model_catalog import browse_hf_hub
|
from .model_catalog import browse_hf_hub
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ from __future__ import annotations
|
|||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
from collections import deque
|
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -114,7 +113,7 @@ def run_dashboard(node, config: dict, start_time: float) -> None:
|
|||||||
return
|
return
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from rich.live import Live # type: ignore[import]
|
from rich.live import Live # type: ignore[import] # noqa: F401
|
||||||
|
|
||||||
_run_rich_dashboard(node, config, start_time)
|
_run_rich_dashboard(node, config, start_time)
|
||||||
except ImportError:
|
except ImportError:
|
||||||
@@ -126,7 +125,6 @@ def _build_rich_renderable(
|
|||||||
):
|
):
|
||||||
from rich.table import Table # type: ignore[import]
|
from rich.table import Table # type: ignore[import]
|
||||||
from rich.panel import Panel # type: ignore[import]
|
from rich.panel import Panel # type: ignore[import]
|
||||||
from rich.columns import Columns # type: ignore[import]
|
|
||||||
from rich.text import Text # type: ignore[import]
|
from rich.text import Text # type: ignore[import]
|
||||||
|
|
||||||
uptime = time.monotonic() - start_time
|
uptime = time.monotonic() - start_time
|
||||||
@@ -178,8 +176,8 @@ def _build_rich_renderable(
|
|||||||
f"Tokens/sec {tps_bar} {tps:.1f} t/s (EMA)",
|
f"Tokens/sec {tps_bar} {tps:.1f} t/s (EMA)",
|
||||||
f"Requests {req_count:,} served",
|
f"Requests {req_count:,} served",
|
||||||
f"Success {stats['success_rate']:.1f}% failed {stats['failed_requests']:,} queue {stats['queue_depth']}",
|
f"Success {stats['success_rate']:.1f}% failed {stats['failed_requests']:,} queue {stats['queue_depth']}",
|
||||||
f"Peers 0 connected (gossip: US-017)",
|
"Peers 0 connected (gossip: US-017)",
|
||||||
f"TAI earned 0.00 TAI (payments: US-006)",
|
"TAI earned 0.00 TAI (payments: US-006)",
|
||||||
f"Uptime {_format_uptime(uptime)}",
|
f"Uptime {_format_uptime(uptime)}",
|
||||||
"",
|
"",
|
||||||
"[q] quit [c] compact view",
|
"[q] quit [c] compact view",
|
||||||
|
|||||||
@@ -36,6 +36,7 @@ from .capability import (
|
|||||||
CapabilityReport,
|
CapabilityReport,
|
||||||
build_capability_report,
|
build_capability_report,
|
||||||
)
|
)
|
||||||
|
from .native_backend import NativeWorkerBackendAdapter
|
||||||
from .recipe_manifest import (
|
from .recipe_manifest import (
|
||||||
DEFAULT_RECIPE_ID,
|
DEFAULT_RECIPE_ID,
|
||||||
Recipe,
|
Recipe,
|
||||||
@@ -449,11 +450,9 @@ def _validate_recipe(
|
|||||||
category: str | None = None
|
category: str | None = None
|
||||||
error: BaseException | None = None
|
error: BaseException | None = None
|
||||||
diagnostics: list[str] = []
|
diagnostics: list[str] = []
|
||||||
detail: dict = {}
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
backend = load_backend(selection, recipe)
|
backend = load_backend(selection, recipe)
|
||||||
detail = probe_forward(backend)
|
probe_forward(backend)
|
||||||
except DoctorError as exc:
|
except DoctorError as exc:
|
||||||
category, error = exc.category, exc
|
category, error = exc.category, exc
|
||||||
diagnostics = [str(exc), exc.hint]
|
diagnostics = [str(exc), exc.hint]
|
||||||
@@ -464,23 +463,48 @@ def _validate_recipe(
|
|||||||
duration_ms = int((time.monotonic() - started) * 1000)
|
duration_ms = int((time.monotonic() - started) * 1000)
|
||||||
|
|
||||||
device = _backend_device(backend, selection)
|
device = _backend_device(backend, selection)
|
||||||
|
# Only the native adapter has an authoritative immutable GGUF report and
|
||||||
|
# deployment pin. The Transformers path deliberately remains dark: a
|
||||||
|
# model/config fingerprint is not an exact ArtifactIdentity.
|
||||||
|
identity = backend.identity if isinstance(backend, NativeWorkerBackendAdapter) else None
|
||||||
|
model_id = selection.model_id if identity is None else identity.artifact.artifact_id
|
||||||
|
shard_start = selection.shard_start if identity is None else identity.shard_start
|
||||||
|
shard_end = selection.shard_end if identity is None else identity.shard_end - 1
|
||||||
|
recipe_id = recipe.id if identity is None else identity.recipe.recipe_id
|
||||||
|
recipe_version = recipe.version if identity is None else identity.recipe.recipe_version
|
||||||
|
catalogue_version = (
|
||||||
|
manifest.catalogue_version if identity is None else identity.recipe.catalogue_version
|
||||||
|
)
|
||||||
|
backend_id = recipe.backend_id if identity is None else identity.recipe.backend_id
|
||||||
|
quantization = (
|
||||||
|
selection.quantization if identity is None else identity.recipe.weight_quantization
|
||||||
|
)
|
||||||
|
runtime = _runtime_versions()
|
||||||
|
model_config = _model_config(backend)
|
||||||
|
revision = None
|
||||||
|
if identity is not None:
|
||||||
|
revision = identity.artifact.revision
|
||||||
|
model_config = "sha256:" + identity.artifact.architecture_digest
|
||||||
|
runtime = {**runtime, "native_runtime": identity.recipe.runtime_version}
|
||||||
report = build_capability_report(
|
report = build_capability_report(
|
||||||
model_id=selection.model_id,
|
model_id=model_id,
|
||||||
shard_start=selection.shard_start,
|
shard_start=shard_start,
|
||||||
shard_end=selection.shard_end,
|
shard_end=shard_end,
|
||||||
recipe_id=recipe.id,
|
recipe_id=recipe_id,
|
||||||
recipe_version=recipe.version,
|
recipe_version=recipe_version,
|
||||||
catalogue_version=manifest.catalogue_version,
|
catalogue_version=catalogue_version,
|
||||||
backend_id=recipe.backend_id,
|
backend_id=backend_id,
|
||||||
device=device,
|
device=device,
|
||||||
device_name=_backend_device_name(device),
|
device_name=_backend_device_name(device),
|
||||||
quantization=selection.quantization,
|
quantization=quantization,
|
||||||
runtime=_runtime_versions(),
|
runtime=runtime,
|
||||||
model_config=_model_config(backend),
|
revision=revision,
|
||||||
|
model_config=model_config,
|
||||||
status=STATUS_FAILED if category else STATUS_PASSED,
|
status=STATUS_FAILED if category else STATUS_PASSED,
|
||||||
duration_ms=duration_ms,
|
duration_ms=duration_ms,
|
||||||
diagnostics=[d for d in diagnostics if d] or None,
|
diagnostics=[d for d in diagnostics if d] or None,
|
||||||
validated_at=clock(),
|
validated_at=clock(),
|
||||||
|
identity=identity,
|
||||||
)
|
)
|
||||||
if category:
|
if category:
|
||||||
return RecipeResult(
|
return RecipeResult(
|
||||||
|
|||||||
@@ -6,16 +6,15 @@ the fact: *what would have counted as success?*
|
|||||||
Its thresholds are locked before the target ever runs (DGR-017), and DGR-020 reads
|
Its thresholds are locked before the target ever runs (DGR-017), and DGR-020 reads
|
||||||
them back to publish an ``alpha`` or ``stop`` verdict. The whole point is that the
|
them back to publish an ``alpha`` or ``stop`` verdict. The whole point is that the
|
||||||
gap between those two moments is where a threshold quietly becomes "0.1 tokens/sec
|
gap between those two moments is where a threshold quietly becomes "0.1 tokens/sec
|
||||||
was always the goal". So the document carries ``contract_sha256`` over its own
|
was always the goal". So the document carries ``contract_sha256`` over its canonical content, while the
|
||||||
canonical content, and :func:`load_alpha_contract` recomputes it on every load. An
|
approved v1 digest is pinned independently in code. :func:`load_alpha_contract`
|
||||||
agent that edits a threshold and forgets the digest is rejected; an agent that
|
recomputes the self-digest and then requires that trusted pre-execution digest.
|
||||||
edits both has left a diff on a file whose whole purpose is to not change.
|
Changing a threshold and re-sealing under the same identity is rejected; an
|
||||||
|
amendment requires a new supported contract identity under human review.
|
||||||
|
|
||||||
This is a tamper-*evidence* mechanism, not tamper-proofing. It cannot stop a
|
The parsed contract recursively freezes nested mappings and sequences. Thresholds
|
||||||
determined rewrite of the file and the digest together — nothing in-repo can. What
|
therefore cannot change between verification and use, and :meth:`AlphaContract.to_dict`
|
||||||
it does is remove the possibility of a silent one, which is the failure mode that
|
returns an isolated mutable copy for diagnostics and tests.
|
||||||
actually happens: a number nudged mid-run, with no reviewer ever seeing that it
|
|
||||||
moved.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -25,6 +24,7 @@ import re
|
|||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from importlib.resources import files
|
from importlib.resources import files
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from types import MappingProxyType
|
||||||
from typing import Any, Mapping
|
from typing import Any, Mapping
|
||||||
|
|
||||||
from .manifest import (
|
from .manifest import (
|
||||||
@@ -39,6 +39,7 @@ from .manifest import (
|
|||||||
ALPHA_CONTRACT_SCHEMA_VERSION = 1
|
ALPHA_CONTRACT_SCHEMA_VERSION = 1
|
||||||
ALPHA_CONTRACT_VERSION = 1
|
ALPHA_CONTRACT_VERSION = 1
|
||||||
ALPHA_CONTRACT_ID = "glm-5.2-max-alpha/v1"
|
ALPHA_CONTRACT_ID = "glm-5.2-max-alpha/v1"
|
||||||
|
ALPHA_CONTRACT_V1_SHA256 = "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9"
|
||||||
|
|
||||||
_CONTRACT_RESOURCE = "alpha-contract.json"
|
_CONTRACT_RESOURCE = "alpha-contract.json"
|
||||||
|
|
||||||
@@ -72,7 +73,23 @@ def contract_signing_payload(document: Mapping[str, Any]) -> dict:
|
|||||||
|
|
||||||
def compute_contract_digest(document: Mapping[str, Any]) -> str:
|
def compute_contract_digest(document: Mapping[str, Any]) -> str:
|
||||||
"""SHA-256 over the canonical contract content."""
|
"""SHA-256 over the canonical contract content."""
|
||||||
return canonical_sha256(contract_signing_payload(document))
|
return canonical_sha256(contract_signing_payload(_thaw_json(document)))
|
||||||
|
|
||||||
|
|
||||||
|
def _freeze_json(value: Any) -> Any:
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return MappingProxyType({str(key): _freeze_json(item) for key, item in value.items()})
|
||||||
|
if isinstance(value, list):
|
||||||
|
return tuple(_freeze_json(item) for item in value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _thaw_json(value: Any) -> Any:
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {str(key): _thaw_json(item) for key, item in value.items()}
|
||||||
|
if isinstance(value, tuple):
|
||||||
|
return [_thaw_json(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -107,7 +124,7 @@ class AlphaContract:
|
|||||||
return block[key]
|
return block[key]
|
||||||
|
|
||||||
def to_dict(self) -> dict:
|
def to_dict(self) -> dict:
|
||||||
return dict(self.raw)
|
return _thaw_json(self.raw)
|
||||||
|
|
||||||
|
|
||||||
def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
|
def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
|
||||||
@@ -229,18 +246,28 @@ def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
|
|||||||
if not isinstance(amendment_policy, str) or not amendment_policy.strip():
|
if not isinstance(amendment_policy, str) or not amendment_policy.strip():
|
||||||
raise AlphaContractError(f"{source} must state its amendment policy")
|
raise AlphaContractError(f"{source} must state its amendment policy")
|
||||||
|
|
||||||
|
if declared != ALPHA_CONTRACT_V1_SHA256:
|
||||||
|
raise AlphaContractError(
|
||||||
|
f"{source} is a re-sealed mutation of {ALPHA_CONTRACT_ID}: digest "
|
||||||
|
f"{declared} does not match the trusted pre-execution digest "
|
||||||
|
f"{ALPHA_CONTRACT_V1_SHA256}. An amendment requires a new supported "
|
||||||
|
"contract identity under human review."
|
||||||
|
)
|
||||||
|
|
||||||
|
frozen = _freeze_json(data)
|
||||||
|
|
||||||
return AlphaContract(
|
return AlphaContract(
|
||||||
schema_version=schema_version,
|
schema_version=schema_version,
|
||||||
contract_version=contract_version,
|
contract_version=contract_version,
|
||||||
contract_id=contract_id,
|
contract_id=contract_id,
|
||||||
locked_at=str(data["locked_at"]),
|
locked_at=str(data["locked_at"]),
|
||||||
locked_by=str(data["locked_by"]),
|
locked_by=str(data["locked_by"]),
|
||||||
target=target,
|
target=frozen["target"],
|
||||||
sections={name: data[name] for name in REQUIRED_SECTIONS},
|
sections=MappingProxyType({name: frozen[name] for name in REQUIRED_SECTIONS}),
|
||||||
verdicts=tuple(verdicts),
|
verdicts=tuple(verdicts),
|
||||||
amendment_policy=amendment_policy,
|
amendment_policy=amendment_policy,
|
||||||
digest=declared,
|
digest=declared,
|
||||||
raw=data,
|
raw=frozen,
|
||||||
source=source,
|
source=source,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
185
packages/node/meshnet_node/native_backend.py
Normal file
185
packages/node/meshnet_node/native_backend.py
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
"""Authoritative identity boundary for a native GGUF Shard backend.
|
||||||
|
|
||||||
|
The native loader owns the facts about the mapped artifact. This module turns
|
||||||
|
that immutable report and separately pinned deployment inputs into the one
|
||||||
|
``ShardIdentity`` DGR-003 permits a native worker to emit. It is intentionally
|
||||||
|
not an adapter for the legacy Transformers backend: that backend has no
|
||||||
|
authoritative immutable GGUF artifact pin and must remain identity-free.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb
|
||||||
|
from .runtime_recipe import (
|
||||||
|
ArtifactIdentity,
|
||||||
|
DerivativeBinding,
|
||||||
|
RecipeIdentityError,
|
||||||
|
RuntimeRecipe,
|
||||||
|
ShardIdentity,
|
||||||
|
check_session_open,
|
||||||
|
handshake_error,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class NativeLoadedArtifactReport:
|
||||||
|
"""Immutable GGUF facts returned by the loaded native model.
|
||||||
|
|
||||||
|
The report is copied from ``llama_model_meshnet_range_report`` plus the
|
||||||
|
parsed GGUF metadata while the model is live. Byte counts are operational
|
||||||
|
evidence rather than compatibility axes, but keeping them beside the range
|
||||||
|
prevents a caller from substituting an unverified range declaration.
|
||||||
|
"""
|
||||||
|
|
||||||
|
owned_start_layer: int
|
||||||
|
owned_end_layer: int
|
||||||
|
mapped_bytes: int
|
||||||
|
resident_bytes: int
|
||||||
|
registered_bytes: int
|
||||||
|
architecture: str
|
||||||
|
architecture_digest: str
|
||||||
|
layer_count: int
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
if self.owned_start_layer < 0 or self.owned_end_layer <= self.owned_start_layer:
|
||||||
|
raise RecipeIdentityError("native report has an invalid owned layer range")
|
||||||
|
if self.layer_count < 1 or self.owned_end_layer > self.layer_count:
|
||||||
|
raise RecipeIdentityError("native report range is outside GGUF layer metadata")
|
||||||
|
if min(self.mapped_bytes, self.resident_bytes, self.registered_bytes) < 0:
|
||||||
|
raise RecipeIdentityError("native report byte counts must be non-negative")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ImmutableArtifactPin:
|
||||||
|
"""Deployment-supplied immutable bytes pin, never inferred from a model name."""
|
||||||
|
|
||||||
|
artifact_id: str
|
||||||
|
revision: str
|
||||||
|
content_digest: str
|
||||||
|
derived_from: DerivativeBinding | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class NativeNumericalRecipe:
|
||||||
|
"""Immutable numerical inputs selected for this native worker instance."""
|
||||||
|
|
||||||
|
weight_quantization: str
|
||||||
|
activation_dtype: str
|
||||||
|
compute_dtype: str
|
||||||
|
kv_dtype: str
|
||||||
|
kv_layout: str
|
||||||
|
architecture_adapter: str
|
||||||
|
backend_id: str
|
||||||
|
runtime_version: str
|
||||||
|
recipe_id: str
|
||||||
|
recipe_version: str
|
||||||
|
catalogue_version: str
|
||||||
|
boundary_schema_version: int = BUNDLE_VERSION
|
||||||
|
protocol_schema_version: int = int(SCHEMA_VERSION)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class NativeIdentityInputs:
|
||||||
|
"""Everything a native backend needs to emit one exact identity."""
|
||||||
|
|
||||||
|
loaded_artifact: NativeLoadedArtifactReport
|
||||||
|
artifact_pin: ImmutableArtifactPin
|
||||||
|
tokenizer_revision: str
|
||||||
|
numerical_recipe: NativeNumericalRecipe
|
||||||
|
|
||||||
|
|
||||||
|
def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIdentity:
|
||||||
|
"""Derive identity only from the native report and immutable pinned inputs."""
|
||||||
|
report = inputs.loaded_artifact
|
||||||
|
pin = inputs.artifact_pin
|
||||||
|
recipe = inputs.numerical_recipe
|
||||||
|
artifact = ArtifactIdentity(
|
||||||
|
artifact_id=pin.artifact_id,
|
||||||
|
revision=pin.revision,
|
||||||
|
content_digest=pin.content_digest,
|
||||||
|
architecture=report.architecture,
|
||||||
|
architecture_digest=report.architecture_digest,
|
||||||
|
layer_count=report.layer_count,
|
||||||
|
derived_from=pin.derived_from,
|
||||||
|
)
|
||||||
|
return ShardIdentity(
|
||||||
|
artifact=artifact,
|
||||||
|
recipe=RuntimeRecipe(
|
||||||
|
weight_quantization=recipe.weight_quantization,
|
||||||
|
activation_dtype=recipe.activation_dtype,
|
||||||
|
compute_dtype=recipe.compute_dtype,
|
||||||
|
kv_dtype=recipe.kv_dtype,
|
||||||
|
kv_layout=recipe.kv_layout,
|
||||||
|
tokenizer_revision=inputs.tokenizer_revision,
|
||||||
|
architecture_adapter=recipe.architecture_adapter,
|
||||||
|
backend_id=recipe.backend_id,
|
||||||
|
runtime_version=recipe.runtime_version,
|
||||||
|
boundary_schema_version=recipe.boundary_schema_version,
|
||||||
|
protocol_schema_version=recipe.protocol_schema_version,
|
||||||
|
recipe_id=recipe.recipe_id,
|
||||||
|
recipe_version=recipe.recipe_version,
|
||||||
|
catalogue_version=recipe.catalogue_version,
|
||||||
|
),
|
||||||
|
shard_start=report.owned_start_layer,
|
||||||
|
shard_end=report.owned_end_layer,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class NativeSessionRejected(RecipeIdentityError):
|
||||||
|
"""A native worker refused a ``SessionOpen`` before allocating session state."""
|
||||||
|
|
||||||
|
def __init__(self, error: "pb.ShardError") -> None:
|
||||||
|
super().__init__(f"native SessionOpen rejected: {error.code}")
|
||||||
|
self.error = error
|
||||||
|
|
||||||
|
|
||||||
|
class NativeWorkerBackendAdapter:
|
||||||
|
"""Small backend-facing adapter around the native loaded-artifact seam."""
|
||||||
|
|
||||||
|
def __init__(self, identity_inputs: NativeIdentityInputs) -> None:
|
||||||
|
self.identity_inputs = identity_inputs
|
||||||
|
self.identity = shard_identity_from_native_report(identity_inputs)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def loaded_artifact_report(self) -> NativeLoadedArtifactReport:
|
||||||
|
return self.identity_inputs.loaded_artifact
|
||||||
|
|
||||||
|
def check_session_open(
|
||||||
|
self,
|
||||||
|
opened: "pb.SessionOpen",
|
||||||
|
*,
|
||||||
|
expected_route_session_id: str | None = None,
|
||||||
|
expected_route_epoch: int | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Reject incompatible/stale opens at the native worker boundary."""
|
||||||
|
mismatches = check_session_open(
|
||||||
|
self.identity,
|
||||||
|
opened,
|
||||||
|
expected_route_session_id=expected_route_session_id,
|
||||||
|
expected_route_epoch=expected_route_epoch,
|
||||||
|
)
|
||||||
|
error = handshake_error(mismatches)
|
||||||
|
if error is not None:
|
||||||
|
raise NativeSessionRejected(error)
|
||||||
|
|
||||||
|
def on_session_open(
|
||||||
|
self,
|
||||||
|
opened: "pb.SessionOpen",
|
||||||
|
*,
|
||||||
|
expected_route_session_id: str | None = None,
|
||||||
|
expected_route_epoch: int | None = None,
|
||||||
|
) -> "pb.SessionAccepted":
|
||||||
|
"""The native worker's SessionOpen boundary, before session allocation."""
|
||||||
|
self.check_session_open(
|
||||||
|
opened,
|
||||||
|
expected_route_session_id=expected_route_session_id,
|
||||||
|
expected_route_epoch=expected_route_epoch,
|
||||||
|
)
|
||||||
|
return pb.SessionAccepted(
|
||||||
|
schema_version=SCHEMA_VERSION,
|
||||||
|
route_session_id=opened.route_session_id,
|
||||||
|
route_epoch=opened.route_epoch,
|
||||||
|
fingerprint=self.identity.fingerprint.to_proto(),
|
||||||
|
)
|
||||||
@@ -31,6 +31,8 @@ from .codec import (
|
|||||||
checksum_of,
|
checksum_of,
|
||||||
crc32c,
|
crc32c,
|
||||||
decode_bundle,
|
decode_bundle,
|
||||||
|
decode_step_bundle,
|
||||||
|
encode_decode_step,
|
||||||
decode_tensor,
|
decode_tensor,
|
||||||
default_flow_control,
|
default_flow_control,
|
||||||
encode_bundle,
|
encode_bundle,
|
||||||
@@ -40,6 +42,7 @@ from .codec import (
|
|||||||
negotiate_flow_control,
|
negotiate_flow_control,
|
||||||
plan_prefill_chunks,
|
plan_prefill_chunks,
|
||||||
validate_session_message_size,
|
validate_session_message_size,
|
||||||
|
validate_tail_result,
|
||||||
)
|
)
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
@@ -60,6 +63,8 @@ __all__ = [
|
|||||||
"checksum_of",
|
"checksum_of",
|
||||||
"crc32c",
|
"crc32c",
|
||||||
"decode_bundle",
|
"decode_bundle",
|
||||||
|
"decode_step_bundle",
|
||||||
|
"encode_decode_step",
|
||||||
"decode_tensor",
|
"decode_tensor",
|
||||||
"default_flow_control",
|
"default_flow_control",
|
||||||
"encode_bundle",
|
"encode_bundle",
|
||||||
@@ -70,4 +75,5 @@ __all__ = [
|
|||||||
"pb",
|
"pb",
|
||||||
"plan_prefill_chunks",
|
"plan_prefill_chunks",
|
||||||
"validate_session_message_size",
|
"validate_session_message_size",
|
||||||
|
"validate_tail_result",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -328,6 +328,8 @@ def decode_tensor(
|
|||||||
def encode_bundle(
|
def encode_bundle(
|
||||||
tensors: Iterable[pb.NamedTensor],
|
tensors: Iterable[pb.NamedTensor],
|
||||||
*,
|
*,
|
||||||
|
architecture: int = pb.ARCHITECTURE_TYPE_UNSPECIFIED,
|
||||||
|
boundary_point: str = "",
|
||||||
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
|
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
|
||||||
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
|
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
|
||||||
) -> pb.TensorBundle:
|
) -> pb.TensorBundle:
|
||||||
@@ -338,7 +340,12 @@ def encode_bundle(
|
|||||||
raise ProtocolError(
|
raise ProtocolError(
|
||||||
f"bundle carries {len(tensor_list)} tensors, exceeding limit {max_tensors}"
|
f"bundle carries {len(tensor_list)} tensors, exceeding limit {max_tensors}"
|
||||||
)
|
)
|
||||||
bundle = pb.TensorBundle(bundle_version=BUNDLE_VERSION, tensors=tensor_list)
|
bundle = pb.TensorBundle(
|
||||||
|
bundle_version=BUNDLE_VERSION,
|
||||||
|
tensors=tensor_list,
|
||||||
|
architecture=architecture,
|
||||||
|
boundary_point=boundary_point,
|
||||||
|
)
|
||||||
if bundle.ByteSize() > max_chunk_bytes:
|
if bundle.ByteSize() > max_chunk_bytes:
|
||||||
raise ProtocolError(
|
raise ProtocolError(
|
||||||
f"serialized tensor bundle exceeds the {max_chunk_bytes}-byte "
|
f"serialized tensor bundle exceeds the {max_chunk_bytes}-byte "
|
||||||
@@ -392,6 +399,80 @@ def decode_bundle(
|
|||||||
return payloads
|
return payloads
|
||||||
|
|
||||||
|
|
||||||
|
def encode_decode_step(
|
||||||
|
bundle: pb.TensorBundle,
|
||||||
|
*,
|
||||||
|
idempotency_step: int,
|
||||||
|
position: int,
|
||||||
|
expected_past_len: int,
|
||||||
|
work_id: str,
|
||||||
|
deadline_unix_nanos: int = 0,
|
||||||
|
prefer_compact_one_tensor: bool = True,
|
||||||
|
) -> pb.DecodeStep:
|
||||||
|
"""Encode a decode boundary, retaining the deliberate compact fallback."""
|
||||||
|
step = pb.DecodeStep(
|
||||||
|
idempotency_step=idempotency_step,
|
||||||
|
position=position,
|
||||||
|
expected_past_len=expected_past_len,
|
||||||
|
work_id=work_id,
|
||||||
|
deadline_unix_nanos=deadline_unix_nanos,
|
||||||
|
)
|
||||||
|
if prefer_compact_one_tensor and len(bundle.tensors) == 1:
|
||||||
|
step.tensor.CopyFrom(bundle.tensors[0])
|
||||||
|
else:
|
||||||
|
step.bundle.CopyFrom(bundle)
|
||||||
|
return step
|
||||||
|
|
||||||
|
|
||||||
|
def validate_tail_result(result: pb.TailResult) -> None:
|
||||||
|
"""Fail closed unless a tail completion is bound to its exact recipe."""
|
||||||
|
identity = result.identity
|
||||||
|
required = (
|
||||||
|
identity.request_id,
|
||||||
|
identity.runtime_recipe_digest,
|
||||||
|
identity.chat_template_id,
|
||||||
|
identity.chat_template_version,
|
||||||
|
identity.reasoning_mode,
|
||||||
|
)
|
||||||
|
if not all(required) or identity.architecture == pb.ARCHITECTURE_TYPE_UNSPECIFIED:
|
||||||
|
raise ProtocolError("tail result lacks exact request/recipe/template identity")
|
||||||
|
if result.WhichOneof("output") not in {"logits", "sampled_token_id"}:
|
||||||
|
raise ProtocolError("tail result lacks logits or sampled token output")
|
||||||
|
|
||||||
|
|
||||||
|
def decode_step_bundle(
|
||||||
|
step: pb.DecodeStep,
|
||||||
|
*,
|
||||||
|
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
|
||||||
|
max_fragment_bytes: int = DEFAULT_MAX_FRAGMENT_BYTES,
|
||||||
|
max_fragments: int = DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
|
||||||
|
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
|
||||||
|
) -> dict[str, bytes]:
|
||||||
|
"""Decode a fast-path boundary with the DGR-006 compatibility rule.
|
||||||
|
|
||||||
|
`bundle` is authoritative because it can carry architecture sidebands. The
|
||||||
|
old `tensor` field remains the compact representation for a certified
|
||||||
|
one-tensor boundary and is accepted by new readers during rollout.
|
||||||
|
"""
|
||||||
|
if step.HasField("bundle"):
|
||||||
|
return decode_bundle(
|
||||||
|
step.bundle,
|
||||||
|
max_chunk_bytes=max_chunk_bytes,
|
||||||
|
max_fragment_bytes=max_fragment_bytes,
|
||||||
|
max_fragments=max_fragments,
|
||||||
|
max_tensors=max_tensors,
|
||||||
|
)
|
||||||
|
if step.HasField("tensor"):
|
||||||
|
return decode_bundle(
|
||||||
|
encode_bundle([step.tensor], max_chunk_bytes=max_chunk_bytes),
|
||||||
|
max_chunk_bytes=max_chunk_bytes,
|
||||||
|
max_fragment_bytes=max_fragment_bytes,
|
||||||
|
max_fragments=max_fragments,
|
||||||
|
max_tensors=max_tensors,
|
||||||
|
)
|
||||||
|
raise ProtocolError("decode step carries neither TensorBundle nor legacy tensor")
|
||||||
|
|
||||||
|
|
||||||
def validate_session_message_size(
|
def validate_session_message_size(
|
||||||
message: pb.SessionRequest | pb.SessionResponse,
|
message: pb.SessionRequest | pb.SessionResponse,
|
||||||
*,
|
*,
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ TESTDATA_DIR = pathlib.Path(__file__).resolve().parents[2] / "native/testdata"
|
|||||||
|
|
||||||
GOLDEN_SESSION_REQUEST = "session_request_golden.binpb"
|
GOLDEN_SESSION_REQUEST = "session_request_golden.binpb"
|
||||||
GOLDEN_CAPABILITY_REPORT = "capability_report_golden.binpb"
|
GOLDEN_CAPABILITY_REPORT = "capability_report_golden.binpb"
|
||||||
|
GOLDEN_DECODE_STEP = "decode_step_golden.binpb"
|
||||||
|
|
||||||
# Written by the C++ conformance test into its build tree; the Python test picks
|
# Written by the C++ conformance test into its build tree; the Python test picks
|
||||||
# it up when present to prove the two languages agree byte-for-byte.
|
# it up when present to prove the two languages agree byte-for-byte.
|
||||||
@@ -136,6 +137,30 @@ def canonical_capability_report() -> pb.CapabilityReport:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_decode_step() -> pb.SessionRequest:
|
||||||
|
"""The DGR-006 multi-tensor decode boundary vector."""
|
||||||
|
hidden = codec.encode_tensor(
|
||||||
|
codec.HIDDEN_STATES, bytes(range(16)), [1, 1, 4], pb.DTYPE_FLOAT32
|
||||||
|
)
|
||||||
|
index_topk = codec.encode_tensor(
|
||||||
|
"index_topk", (3).to_bytes(4, "little"), [1], pb.DTYPE_INT32
|
||||||
|
)
|
||||||
|
return pb.SessionRequest(
|
||||||
|
decode=pb.DecodeStep(
|
||||||
|
idempotency_step=43,
|
||||||
|
position=384,
|
||||||
|
expected_past_len=384,
|
||||||
|
work_id="decode-7f3a",
|
||||||
|
deadline_unix_nanos=DEADLINE_UNIX_NANOS,
|
||||||
|
bundle=codec.encode_bundle(
|
||||||
|
[hidden, index_topk],
|
||||||
|
architecture=pb.ARCHITECTURE_TYPE_MLA,
|
||||||
|
boundary_point="pre_tail_residual",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def serialize(message) -> bytes:
|
def serialize(message) -> bytes:
|
||||||
"""Serialize deterministically, so committed golden bytes are stable."""
|
"""Serialize deterministically, so committed golden bytes are stable."""
|
||||||
return message.SerializeToString(deterministic=True)
|
return message.SerializeToString(deterministic=True)
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
@@ -12,6 +12,13 @@ class SchemaVersion(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
|||||||
SCHEMA_VERSION_UNSPECIFIED: _ClassVar[SchemaVersion]
|
SCHEMA_VERSION_UNSPECIFIED: _ClassVar[SchemaVersion]
|
||||||
SCHEMA_VERSION_1: _ClassVar[SchemaVersion]
|
SCHEMA_VERSION_1: _ClassVar[SchemaVersion]
|
||||||
|
|
||||||
|
class ArchitectureType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
||||||
|
__slots__ = ()
|
||||||
|
ARCHITECTURE_TYPE_UNSPECIFIED: _ClassVar[ArchitectureType]
|
||||||
|
ARCHITECTURE_TYPE_DENSE: _ClassVar[ArchitectureType]
|
||||||
|
ARCHITECTURE_TYPE_MOE: _ClassVar[ArchitectureType]
|
||||||
|
ARCHITECTURE_TYPE_MLA: _ClassVar[ArchitectureType]
|
||||||
|
|
||||||
class DType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
class DType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
||||||
__slots__ = ()
|
__slots__ = ()
|
||||||
DTYPE_UNSPECIFIED: _ClassVar[DType]
|
DTYPE_UNSPECIFIED: _ClassVar[DType]
|
||||||
@@ -80,6 +87,10 @@ class ServingState(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
|||||||
SERVING_STATE_NOT_SERVING: _ClassVar[ServingState]
|
SERVING_STATE_NOT_SERVING: _ClassVar[ServingState]
|
||||||
SCHEMA_VERSION_UNSPECIFIED: SchemaVersion
|
SCHEMA_VERSION_UNSPECIFIED: SchemaVersion
|
||||||
SCHEMA_VERSION_1: SchemaVersion
|
SCHEMA_VERSION_1: SchemaVersion
|
||||||
|
ARCHITECTURE_TYPE_UNSPECIFIED: ArchitectureType
|
||||||
|
ARCHITECTURE_TYPE_DENSE: ArchitectureType
|
||||||
|
ARCHITECTURE_TYPE_MOE: ArchitectureType
|
||||||
|
ARCHITECTURE_TYPE_MLA: ArchitectureType
|
||||||
DTYPE_UNSPECIFIED: DType
|
DTYPE_UNSPECIFIED: DType
|
||||||
DTYPE_BFLOAT16: DType
|
DTYPE_BFLOAT16: DType
|
||||||
DTYPE_FLOAT16: DType
|
DTYPE_FLOAT16: DType
|
||||||
@@ -165,12 +176,16 @@ class NamedTensor(_message.Message):
|
|||||||
def __init__(self, name: _Optional[str] = ..., shape: _Optional[_Iterable[int]] = ..., dtype: _Optional[_Union[DType, str]] = ..., byte_order: _Optional[_Union[ByteOrder, str]] = ..., total_bytes: _Optional[int] = ..., compression: _Optional[_Union[Compression, str]] = ..., checksum: _Optional[_Union[Checksum, _Mapping]] = ..., fragments: _Optional[_Iterable[_Union[TensorFragment, _Mapping]]] = ...) -> None: ...
|
def __init__(self, name: _Optional[str] = ..., shape: _Optional[_Iterable[int]] = ..., dtype: _Optional[_Union[DType, str]] = ..., byte_order: _Optional[_Union[ByteOrder, str]] = ..., total_bytes: _Optional[int] = ..., compression: _Optional[_Union[Compression, str]] = ..., checksum: _Optional[_Union[Checksum, _Mapping]] = ..., fragments: _Optional[_Iterable[_Union[TensorFragment, _Mapping]]] = ...) -> None: ...
|
||||||
|
|
||||||
class TensorBundle(_message.Message):
|
class TensorBundle(_message.Message):
|
||||||
__slots__ = ("bundle_version", "tensors")
|
__slots__ = ("bundle_version", "tensors", "architecture", "boundary_point")
|
||||||
BUNDLE_VERSION_FIELD_NUMBER: _ClassVar[int]
|
BUNDLE_VERSION_FIELD_NUMBER: _ClassVar[int]
|
||||||
TENSORS_FIELD_NUMBER: _ClassVar[int]
|
TENSORS_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
ARCHITECTURE_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
BOUNDARY_POINT_FIELD_NUMBER: _ClassVar[int]
|
||||||
bundle_version: int
|
bundle_version: int
|
||||||
tensors: _containers.RepeatedCompositeFieldContainer[NamedTensor]
|
tensors: _containers.RepeatedCompositeFieldContainer[NamedTensor]
|
||||||
def __init__(self, bundle_version: _Optional[int] = ..., tensors: _Optional[_Iterable[_Union[NamedTensor, _Mapping]]] = ...) -> None: ...
|
architecture: ArchitectureType
|
||||||
|
boundary_point: str
|
||||||
|
def __init__(self, bundle_version: _Optional[int] = ..., tensors: _Optional[_Iterable[_Union[NamedTensor, _Mapping]]] = ..., architecture: _Optional[_Union[ArchitectureType, str]] = ..., boundary_point: _Optional[str] = ...) -> None: ...
|
||||||
|
|
||||||
class Fingerprint(_message.Message):
|
class Fingerprint(_message.Message):
|
||||||
__slots__ = ("model_artifact_digest", "runtime_recipe_digest", "recipe_id", "recipe_version", "catalogue_version")
|
__slots__ = ("model_artifact_digest", "runtime_recipe_digest", "recipe_id", "recipe_version", "catalogue_version")
|
||||||
@@ -327,20 +342,64 @@ class ActivationChunk(_message.Message):
|
|||||||
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ...
|
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ...
|
||||||
|
|
||||||
class DecodeStep(_message.Message):
|
class DecodeStep(_message.Message):
|
||||||
__slots__ = ("idempotency_step", "position", "expected_past_len", "tensor", "work_id", "deadline_unix_nanos")
|
__slots__ = ("idempotency_step", "position", "expected_past_len", "tensor", "work_id", "deadline_unix_nanos", "bundle")
|
||||||
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
|
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
|
||||||
POSITION_FIELD_NUMBER: _ClassVar[int]
|
POSITION_FIELD_NUMBER: _ClassVar[int]
|
||||||
EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int]
|
EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int]
|
||||||
TENSOR_FIELD_NUMBER: _ClassVar[int]
|
TENSOR_FIELD_NUMBER: _ClassVar[int]
|
||||||
WORK_ID_FIELD_NUMBER: _ClassVar[int]
|
WORK_ID_FIELD_NUMBER: _ClassVar[int]
|
||||||
DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int]
|
DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
BUNDLE_FIELD_NUMBER: _ClassVar[int]
|
||||||
idempotency_step: int
|
idempotency_step: int
|
||||||
position: int
|
position: int
|
||||||
expected_past_len: int
|
expected_past_len: int
|
||||||
tensor: NamedTensor
|
tensor: NamedTensor
|
||||||
work_id: str
|
work_id: str
|
||||||
deadline_unix_nanos: int
|
deadline_unix_nanos: int
|
||||||
def __init__(self, idempotency_step: _Optional[int] = ..., position: _Optional[int] = ..., expected_past_len: _Optional[int] = ..., tensor: _Optional[_Union[NamedTensor, _Mapping]] = ..., work_id: _Optional[str] = ..., deadline_unix_nanos: _Optional[int] = ...) -> None: ...
|
bundle: TensorBundle
|
||||||
|
def __init__(self, idempotency_step: _Optional[int] = ..., position: _Optional[int] = ..., expected_past_len: _Optional[int] = ..., tensor: _Optional[_Union[NamedTensor, _Mapping]] = ..., work_id: _Optional[str] = ..., deadline_unix_nanos: _Optional[int] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ...
|
||||||
|
|
||||||
|
class RequestRecipeIdentity(_message.Message):
|
||||||
|
__slots__ = ("request_id", "runtime_recipe_digest", "chat_template_id", "chat_template_version", "reasoning_mode", "architecture")
|
||||||
|
REQUEST_ID_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
RUNTIME_RECIPE_DIGEST_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
CHAT_TEMPLATE_ID_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
CHAT_TEMPLATE_VERSION_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
REASONING_MODE_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
ARCHITECTURE_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
request_id: str
|
||||||
|
runtime_recipe_digest: str
|
||||||
|
chat_template_id: str
|
||||||
|
chat_template_version: str
|
||||||
|
reasoning_mode: str
|
||||||
|
architecture: ArchitectureType
|
||||||
|
def __init__(self, request_id: _Optional[str] = ..., runtime_recipe_digest: _Optional[str] = ..., chat_template_id: _Optional[str] = ..., chat_template_version: _Optional[str] = ..., reasoning_mode: _Optional[str] = ..., architecture: _Optional[_Union[ArchitectureType, str]] = ...) -> None: ...
|
||||||
|
|
||||||
|
class SamplingParameters(_message.Message):
|
||||||
|
__slots__ = ("temperature", "top_p", "top_k", "seed", "greedy")
|
||||||
|
TEMPERATURE_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
TOP_P_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
TOP_K_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
SEED_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
GREEDY_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
temperature: float
|
||||||
|
top_p: float
|
||||||
|
top_k: int
|
||||||
|
seed: int
|
||||||
|
greedy: bool
|
||||||
|
def __init__(self, temperature: _Optional[float] = ..., top_p: _Optional[float] = ..., top_k: _Optional[int] = ..., seed: _Optional[int] = ..., greedy: _Optional[bool] = ...) -> None: ...
|
||||||
|
|
||||||
|
class TailResult(_message.Message):
|
||||||
|
__slots__ = ("identity", "sampling", "logits", "sampled_token_id")
|
||||||
|
IDENTITY_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
SAMPLING_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
LOGITS_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
SAMPLED_TOKEN_ID_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
identity: RequestRecipeIdentity
|
||||||
|
sampling: SamplingParameters
|
||||||
|
logits: TensorBundle
|
||||||
|
sampled_token_id: int
|
||||||
|
def __init__(self, identity: _Optional[_Union[RequestRecipeIdentity, _Mapping]] = ..., sampling: _Optional[_Union[SamplingParameters, _Mapping]] = ..., logits: _Optional[_Union[TensorBundle, _Mapping]] = ..., sampled_token_id: _Optional[int] = ...) -> None: ...
|
||||||
|
|
||||||
class ReleaseSignal(_message.Message):
|
class ReleaseSignal(_message.Message):
|
||||||
__slots__ = ("route_session_id", "route_epoch", "work_id")
|
__slots__ = ("route_session_id", "route_epoch", "work_id")
|
||||||
@@ -409,18 +468,20 @@ class SessionRequest(_message.Message):
|
|||||||
def __init__(self, open: _Optional[_Union[SessionOpen, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., decode: _Optional[_Union[DecodeStep, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., release: _Optional[_Union[ReleaseSignal, _Mapping]] = ..., cancel: _Optional[_Union[CancelSignal, _Mapping]] = ...) -> None: ...
|
def __init__(self, open: _Optional[_Union[SessionOpen, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., decode: _Optional[_Union[DecodeStep, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., release: _Optional[_Union[ReleaseSignal, _Mapping]] = ..., cancel: _Optional[_Union[CancelSignal, _Mapping]] = ...) -> None: ...
|
||||||
|
|
||||||
class SessionResponse(_message.Message):
|
class SessionResponse(_message.Message):
|
||||||
__slots__ = ("accepted", "chunk", "ack", "flow_control", "status")
|
__slots__ = ("accepted", "chunk", "ack", "flow_control", "status", "tail_result")
|
||||||
ACCEPTED_FIELD_NUMBER: _ClassVar[int]
|
ACCEPTED_FIELD_NUMBER: _ClassVar[int]
|
||||||
CHUNK_FIELD_NUMBER: _ClassVar[int]
|
CHUNK_FIELD_NUMBER: _ClassVar[int]
|
||||||
ACK_FIELD_NUMBER: _ClassVar[int]
|
ACK_FIELD_NUMBER: _ClassVar[int]
|
||||||
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
|
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
|
||||||
STATUS_FIELD_NUMBER: _ClassVar[int]
|
STATUS_FIELD_NUMBER: _ClassVar[int]
|
||||||
|
TAIL_RESULT_FIELD_NUMBER: _ClassVar[int]
|
||||||
accepted: SessionAccepted
|
accepted: SessionAccepted
|
||||||
chunk: ActivationChunk
|
chunk: ActivationChunk
|
||||||
ack: Ack
|
ack: Ack
|
||||||
flow_control: FlowControl
|
flow_control: FlowControl
|
||||||
status: ShardStatus
|
status: ShardStatus
|
||||||
def __init__(self, accepted: _Optional[_Union[SessionAccepted, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., ack: _Optional[_Union[Ack, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., status: _Optional[_Union[ShardStatus, _Mapping]] = ...) -> None: ...
|
tail_result: TailResult
|
||||||
|
def __init__(self, accepted: _Optional[_Union[SessionAccepted, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., ack: _Optional[_Union[Ack, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., status: _Optional[_Union[ShardStatus, _Mapping]] = ..., tail_result: _Optional[_Union[TailResult, _Mapping]] = ...) -> None: ...
|
||||||
|
|
||||||
class CapabilityRequest(_message.Message):
|
class CapabilityRequest(_message.Message):
|
||||||
__slots__ = ("schema_version",)
|
__slots__ = ("schema_version",)
|
||||||
|
|||||||
1049
packages/node/meshnet_node/runtime_recipe.py
Normal file
1049
packages/node/meshnet_node/runtime_recipe.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -105,7 +105,7 @@ class _StubHTTPServer(http.server.HTTPServer):
|
|||||||
|
|
||||||
|
|
||||||
class _StubHandler(http.server.BaseHTTPRequestHandler):
|
class _StubHandler(http.server.BaseHTTPRequestHandler):
|
||||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def do_POST(self):
|
def do_POST(self):
|
||||||
|
|||||||
@@ -19,7 +19,6 @@ from .model_backend import (
|
|||||||
InsufficientVRAMError,
|
InsufficientVRAMError,
|
||||||
KVCacheMiss,
|
KVCacheMiss,
|
||||||
MissingModelDependencyError,
|
MissingModelDependencyError,
|
||||||
Quantization,
|
|
||||||
TailTokenResult,
|
TailTokenResult,
|
||||||
TorchModelShard,
|
TorchModelShard,
|
||||||
_tensor_from_bfloat16_bytes,
|
_tensor_from_bfloat16_bytes,
|
||||||
@@ -46,7 +45,7 @@ class _DirectRequestUncertainError(ConnectionError):
|
|||||||
"""A direct request may have reached the downstream node but did not finish."""
|
"""A direct request may have reached the downstream node but did not finish."""
|
||||||
|
|
||||||
|
|
||||||
from .server import (
|
from .server import ( # noqa: E402
|
||||||
_WIRE_VERSION,
|
_WIRE_VERSION,
|
||||||
_parse_shape,
|
_parse_shape,
|
||||||
_validate_activation_body,
|
_validate_activation_body,
|
||||||
@@ -399,7 +398,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
# Finite responses below provide Content-Length; streams are chunked.
|
# Finite responses below provide Content-Length; streams are chunked.
|
||||||
protocol_version = "HTTP/1.1"
|
protocol_version = "HTTP/1.1"
|
||||||
|
|
||||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def _request_id(self) -> str:
|
def _request_id(self) -> str:
|
||||||
|
|||||||
@@ -2,7 +2,6 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import sys
|
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|||||||
@@ -32,6 +32,15 @@ Both `--check` modes run in CI via `tests/test_native_shard_protocol.py`, so a
|
|||||||
schema edit that is not accompanied by regenerated output fails the suite rather
|
schema edit that is not accompanied by regenerated output fails the suite rather
|
||||||
than shipping stubs that disagree with the schema they claim to implement.
|
than shipping stubs that disagree with the schema they claim to implement.
|
||||||
|
|
||||||
|
## DGR-006 decode and tail compatibility
|
||||||
|
|
||||||
|
`DecodeStep.bundle` is the versioned `TensorBundle` fast-path boundary. It is
|
||||||
|
authoritative whenever present and supports architecture sidebands. The original
|
||||||
|
`DecodeStep.tensor` remains readable as the compact one-tensor encoding for
|
||||||
|
certified boundaries that need only one tensor; new readers wrap it into a
|
||||||
|
one-member bundle. Tail completions use `TailResult`, which binds logits or a
|
||||||
|
sampled token to request/recipe identity and sampling/template/reasoning inputs.
|
||||||
|
|
||||||
## Building and running the C++ conformance test
|
## Building and running the C++ conformance test
|
||||||
|
|
||||||
If the machine has no protobuf C++ toolchain:
|
If the machine has no protobuf C++ toolchain:
|
||||||
|
|||||||
30
packages/node/native/llama/PATCH-STACK.md
Normal file
30
packages/node/native/llama/PATCH-STACK.md
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
# Meshnet llama.cpp patch stack
|
||||||
|
|
||||||
|
This directory is the only project-owned fork boundary for llama.cpp. It is
|
||||||
|
locked to `e920c523e3b8a0163fe498af5bf90df35ff51d25`; changing the pin requires
|
||||||
|
updating the recorded tree/blob assumptions and reviewing every patch anew.
|
||||||
|
|
||||||
|
## Ordered series
|
||||||
|
|
||||||
|
1. `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch` adds only an
|
||||||
|
interface-library marker used to prove the patched source was configured.
|
||||||
|
It has no execution, transport, model-loading, or semantic effect.
|
||||||
|
|
||||||
|
Future patches may implement only the ADR-0020 local seams: range-aware tensor
|
||||||
|
loading, endpoint ownership, architecture-defined intermediate boundaries, and
|
||||||
|
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
|
||||||
|
billing, authentication, and telemetry must remain outside this directory.
|
||||||
|
|
||||||
|
`scripts/llama_cpp_dependency.py` verifies the exact commit/tree and baseline
|
||||||
|
blobs, validates every patch digest and context with `git apply --check`, then
|
||||||
|
applies the series in `patches/series` order. It refuses a dirty source tree,
|
||||||
|
wrong commit/tree/blob, changed patch digest, reordered series, or an existing
|
||||||
|
destination/work directory.
|
||||||
|
|
||||||
|
## Current semantic boundary
|
||||||
|
|
||||||
|
The stock pinned build is **infrastructure evidence only**. Per DGR-017,
|
||||||
|
GLM-5.2 can use a dense-MLA compatibility fallback at this point; a successful
|
||||||
|
build or `llama-cli --version` does not show native DSA/IndexShare, GLM semantic
|
||||||
|
acceptance, numerical parity, performance, or route certification. DGR-018 and
|
||||||
|
DGR-019 own those checks. Dense Llama remains only a later structural fixture.
|
||||||
17
packages/node/native/llama/THIRD_PARTY_NOTICES.md
Normal file
17
packages/node/native/llama/THIRD_PARTY_NOTICES.md
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
# Third-party notices: llama.cpp
|
||||||
|
|
||||||
|
The reproducibility harness fetches source from
|
||||||
|
[`ggml-org/llama.cpp`](https://github.com/ggml-org/llama.cpp) at exact commit
|
||||||
|
`e920c523e3b8a0163fe498af5bf90df35ff51d25`.
|
||||||
|
|
||||||
|
- Upstream license: MIT. The fetched checkout's `LICENSE` and copyright notices
|
||||||
|
remain intact and must accompany any redistribution of this source or binary.
|
||||||
|
- Meshnet's one-patch CMake marker is an additive local change. It does not
|
||||||
|
replace, relicense, or remove upstream notices.
|
||||||
|
- No donor code is included. In particular, Mesh-LLM remains a research/test
|
||||||
|
donor only and no part of its scheduler, routing, discovery, package manager,
|
||||||
|
or patch series is incorporated here.
|
||||||
|
|
||||||
|
Release packaging must include the upstream MIT text from the exact materialized
|
||||||
|
checkout plus this notice. `scripts/llama_cpp_dependency.py` refuses to build if
|
||||||
|
the upstream `LICENSE` is absent.
|
||||||
1
packages/node/native/llama/UPSTREAM_COMMIT
Normal file
1
packages/node/native/llama/UPSTREAM_COMMIT
Normal file
@@ -0,0 +1 @@
|
|||||||
|
e920c523e3b8a0163fe498af5bf90df35ff51d25
|
||||||
48
packages/node/native/llama/UPSTREAM_LOCK.json
Normal file
48
packages/node/native/llama/UPSTREAM_LOCK.json
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
{
|
||||||
|
"schema_version": 1,
|
||||||
|
"upstream": "https://github.com/ggml-org/llama.cpp.git",
|
||||||
|
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||||
|
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
|
||||||
|
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
|
||||||
|
"upstream_license": "MIT",
|
||||||
|
"patch_series": [
|
||||||
|
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
|
||||||
|
"0002-dense-llama-owned-range-loader.patch"
|
||||||
|
],
|
||||||
|
"patch_scope": [
|
||||||
|
"Reserved CMake ABI marker only; no execution or model semantics.",
|
||||||
|
"Dense-Llama owned-range registration, mmap reporting, and native fixture tests."
|
||||||
|
],
|
||||||
|
"build": {
|
||||||
|
"generator": "Unix Makefiles",
|
||||||
|
"cmake_minimum": "3.14",
|
||||||
|
"cxx_standard": "17",
|
||||||
|
"configure_flags": [
|
||||||
|
"-DCMAKE_BUILD_TYPE=Release",
|
||||||
|
"-DLLAMA_BUILD_TESTS=OFF",
|
||||||
|
"-DLLAMA_BUILD_EXAMPLES=ON",
|
||||||
|
"-DLLAMA_BUILD_SERVER=OFF",
|
||||||
|
"-DLLAMA_BUILD_TOOLS=OFF",
|
||||||
|
"-DLLAMA_BUILD_APP=OFF",
|
||||||
|
"-DLLAMA_CURL=OFF"
|
||||||
|
],
|
||||||
|
"native_targets": ["llama-gguf-hash"],
|
||||||
|
"smoke_binary": "bin/llama-gguf-hash",
|
||||||
|
"smoke_args": ["--help"],
|
||||||
|
"smoke_output_token": "usage"
|
||||||
|
},
|
||||||
|
"required_upstream_blobs": {
|
||||||
|
"CMakeLists.txt": "81f23d7e70b7378511af5d01be680c03aebc2b15"
|
||||||
|
},
|
||||||
|
"patched_paths": [
|
||||||
|
"CMakeLists.txt",
|
||||||
|
"cmake/meshnet-patch-stack.cmake",
|
||||||
|
"include/llama.h",
|
||||||
|
"src/llama-model.cpp",
|
||||||
|
"src/llama-model.h",
|
||||||
|
"src/models/llama.cpp",
|
||||||
|
"tests/CMakeLists.txt",
|
||||||
|
"tests/test-meshnet-range-ownership.cpp"
|
||||||
|
],
|
||||||
|
"stock_glm_limitations": "This pin may load GLM-5.2 through the dense-MLA compatibility fallback. It does not prove native DSA, IndexShare, MoE semantic correctness, numerical equivalence, performance, or route certification."
|
||||||
|
}
|
||||||
68
packages/node/native/llama/meshnet-range-loader.cpp
Normal file
68
packages/node/native/llama/meshnet-range-loader.cpp
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
/**
|
||||||
|
* meshnet-range-loader — CLI wrapper that loads a GGUF shard range
|
||||||
|
* and prints the meshnet range report as JSON.
|
||||||
|
*
|
||||||
|
* Usage: meshnet-range-loader <gguf-path> <start-layer> <end-layer>
|
||||||
|
*
|
||||||
|
* Build: g++ -std=c++17 -I<source>/include -L<build>/bin
|
||||||
|
* meshnet-range-loader.cpp -lllama -o meshnet-range-loader
|
||||||
|
* LD_LIBRARY_PATH=<build>/bin ./meshnet-range-loader ...
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include "llama.h"
|
||||||
|
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
int main(int argc, char **argv) {
|
||||||
|
if (argc != 4) {
|
||||||
|
fprintf(stderr, "Usage: %s <gguf-path> <start-layer> <end-layer>\n", argv[0]);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const char *path = argv[1];
|
||||||
|
int start = std::atoi(argv[2]);
|
||||||
|
int end = std::atoi(argv[3]);
|
||||||
|
|
||||||
|
llama_backend_init();
|
||||||
|
|
||||||
|
llama_model_params params = llama_model_default_params();
|
||||||
|
params.meshnet_owned_layer_start = start;
|
||||||
|
params.meshnet_owned_layer_end = end;
|
||||||
|
|
||||||
|
llama_model *model = llama_model_load_from_file(path, params);
|
||||||
|
if (!model) {
|
||||||
|
fprintf(stderr, "ERROR model_load returned nullptr\n");
|
||||||
|
llama_backend_free();
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
llama_meshnet_range_report report;
|
||||||
|
bool ok = llama_model_meshnet_range_report(model, &report);
|
||||||
|
|
||||||
|
int n_layer = llama_model_n_layer(model);
|
||||||
|
int n_embd = llama_model_n_embd(model);
|
||||||
|
uint64_t size = llama_model_size(model);
|
||||||
|
|
||||||
|
if (ok) {
|
||||||
|
fprintf(stdout,
|
||||||
|
"{"
|
||||||
|
"\"ok\":true,"
|
||||||
|
"\"start_layer\":%d,\"end_layer\":%d,"
|
||||||
|
"\"mapped_bytes\":%lu,\"resident_bytes\":%lu,"
|
||||||
|
"\"model_n_layer\":%d,\"model_n_embd\":%d,\"model_size\":%lu"
|
||||||
|
"}\n",
|
||||||
|
report.start_layer, report.end_layer,
|
||||||
|
(unsigned long)report.mapped_bytes,
|
||||||
|
(unsigned long)report.resident_bytes,
|
||||||
|
n_layer, n_embd, (unsigned long)size
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
fprintf(stderr, "ERROR range report not available\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
llama_model_free(model);
|
||||||
|
llama_backend_free();
|
||||||
|
return ok ? 0 : 1;
|
||||||
|
}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
From d9992f62e852c8647a2ad302009b0339dc1cbf5b Mon Sep 17 00:00:00 2001
|
||||||
|
From: DGR-004 <dgr-004@meshnet.invalid>
|
||||||
|
Date: Tue, 14 Jul 2026 09:52:24 +0300
|
||||||
|
Subject: [PATCH] cmake: reserve Meshnet patch-stack ABI marker
|
||||||
|
|
||||||
|
---
|
||||||
|
CMakeLists.txt | 1 +
|
||||||
|
cmake/meshnet-patch-stack.cmake | 7 +++++++
|
||||||
|
2 files changed, 8 insertions(+)
|
||||||
|
|
||||||
|
diff --git a/CMakeLists.txt b/CMakeLists.txt
|
||||||
|
index 81f23d7e..a9afcffa 100644
|
||||||
|
--- a/CMakeLists.txt
|
||||||
|
+++ b/CMakeLists.txt
|
||||||
|
@@ -123,6 +123,7 @@ option(LLAMA_LLGUIDANCE "llama-common: include LLGuidance library for structured
|
||||||
|
# Required for relocatable CMake package
|
||||||
|
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/build-info.cmake)
|
||||||
|
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/common.cmake)
|
||||||
|
+include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/meshnet-patch-stack.cmake)
|
||||||
|
|
||||||
|
if (NOT DEFINED LLAMA_BUILD_NUMBER)
|
||||||
|
set(LLAMA_BUILD_NUMBER ${BUILD_NUMBER})
|
||||||
|
diff --git a/cmake/meshnet-patch-stack.cmake b/cmake/meshnet-patch-stack.cmake
|
||||||
|
new file mode 100644
|
||||||
|
index 00000000..910646b4
|
||||||
|
--- /dev/null
|
||||||
|
+++ b/cmake/meshnet-patch-stack.cmake
|
||||||
|
@@ -0,0 +1,7 @@
|
||||||
|
+# Reserved, local-only hook for the Meshnet-owned llama.cpp patch series.
|
||||||
|
+# No execution, protocol, architecture, or model semantic changes are in DGR-004.
|
||||||
|
+set(LLAMA_MESHNET_PATCH_STACK_VERSION "1" CACHE STRING
|
||||||
|
+ "Meshnet patch stack ABI marker")
|
||||||
|
+add_library(llama_meshnet_patch_stack INTERFACE)
|
||||||
|
+target_compile_definitions(llama_meshnet_patch_stack INTERFACE
|
||||||
|
+ LLAMA_MESHNET_PATCH_STACK_VERSION=${LLAMA_MESHNET_PATCH_STACK_VERSION})
|
||||||
|
--
|
||||||
|
2.54.0
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
From: Meshnet <meshnet@invalid>
|
||||||
|
Subject: [PATCH] llama: add dense owned-range loading seam
|
||||||
|
|
||||||
|
diff --git a/include/llama.h b/include/llama.h
|
||||||
|
index a311ac20..1f9459cf 100644
|
||||||
|
--- a/include/llama.h
|
||||||
|
+++ b/include/llama.h
|
||||||
|
@@ -292,6 +292,19 @@ extern "C" {
|
||||||
|
ggml_backend_buffer_type_t buft;
|
||||||
|
};
|
||||||
|
|
||||||
|
+ // Immutable report for the project-owned dense-Llama range-loading seam.
|
||||||
|
+ // The bounds are inclusive/exclusive and are populated only after the
|
||||||
|
+ // model has registered and allocated its owned tensors.
|
||||||
|
+ struct llama_meshnet_range_report {
|
||||||
|
+ int32_t start_layer;
|
||||||
|
+ int32_t end_layer;
|
||||||
|
+ uint64_t mapped_bytes;
|
||||||
|
+ uint64_t resident_bytes;
|
||||||
|
+ uint64_t registered_bytes;
|
||||||
|
+ bool has_token_embeddings;
|
||||||
|
+ bool has_output_head;
|
||||||
|
+ };
|
||||||
|
+
|
||||||
|
struct llama_model_params {
|
||||||
|
@@ -319,6 +332,12 @@ extern "C" {
|
||||||
|
const struct llama_model_kv_override * kv_overrides;
|
||||||
|
|
||||||
|
+ int32_t meshnet_owned_layer_start;
|
||||||
|
+ int32_t meshnet_owned_layer_end;
|
||||||
|
+
|
||||||
|
// Keep the booleans together to avoid misalignment during copy-by-value.
|
||||||
|
@@ -616,6 +635,13 @@ extern "C" {
|
||||||
|
LLAMA_API uint64_t llama_model_size(const struct llama_model * model);
|
||||||
|
|
||||||
|
+ LLAMA_API bool llama_model_meshnet_range_report(
|
||||||
|
+ const struct llama_model * model,
|
||||||
|
+ struct llama_meshnet_range_report * out);
|
||||||
|
+
|
||||||
|
// Get the default chat template. Returns nullptr if not available
|
||||||
|
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
|
||||||
|
index d8748138..4d2a3ec1 100644
|
||||||
|
--- a/src/llama-model.cpp
|
||||||
|
+++ b/src/llama-model.cpp
|
||||||
|
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
|
||||||
|
std::vector<float> tensor_split_owned;
|
||||||
|
+ llama_meshnet_range_report meshnet_range_report = {};
|
||||||
|
+ bool has_meshnet_range_report = false;
|
||||||
|
};
|
||||||
|
@@ -1236,6 +1239,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||||
|
const bool use_mmap_buffer = true;
|
||||||
|
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
|
||||||
|
+ const int meshnet_start = params.meshnet_owned_layer_start;
|
||||||
|
+ const int meshnet_end = params.meshnet_owned_layer_end;
|
||||||
|
+ if (meshnet_range_requested) {
|
||||||
|
+ if (arch != LLM_ARCH_LLAMA) {
|
||||||
|
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
|
||||||
|
+ }
|
||||||
|
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
|
||||||
|
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
|
||||||
|
+ meshnet_start, meshnet_end, hparams.n_layer()));
|
||||||
|
+ }
|
||||||
|
+ }
|
||||||
|
@@ -1336,7 +1352,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||||
|
- for (int i = 0; i < n_layer_all; ++i) {
|
||||||
|
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
|
||||||
|
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
|
||||||
|
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
|
||||||
|
@@ -1487,7 +1505,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||||
|
- ml.done_getting_tensors();
|
||||||
|
+ ml.done_getting_tensors(meshnet_range_requested);
|
||||||
|
@@ -1613,8 +1631,11 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||||
|
+ uint64_t meshnet_mapped_bytes = 0;
|
||||||
|
+ uint64_t meshnet_resident_bytes = 0;
|
||||||
|
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
|
||||||
|
for (auto & buf: bufs) {
|
||||||
|
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
|
||||||
|
@@ -1637,6 +1658,35 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||||
|
}
|
||||||
|
+ if (meshnet_range_requested) {
|
||||||
|
+ uint64_t registered_bytes = 0;
|
||||||
|
+ for (const auto & [_, tensor] : tensors_by_name) registered_bytes += ggml_nbytes(tensor);
|
||||||
|
+ if (ml.use_mmap) for (const auto & [first, last] : ml.mmaps_used) if (last > first) meshnet_mapped_bytes += last - first;
|
||||||
|
+ const auto registered = [this](const ggml_tensor * tensor) {
|
||||||
|
+ return tensor != nullptr && std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
|
||||||
|
+ [tensor](const auto & entry) { return entry.second == tensor; });
|
||||||
|
+ };
|
||||||
|
+ const auto registered_name = [this](const char * name) {
|
||||||
|
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
|
||||||
|
+ [name](const auto & entry) { return entry.first == name; });
|
||||||
|
+ };
|
||||||
|
+ pimpl->meshnet_range_report = { meshnet_start, meshnet_end, meshnet_mapped_bytes, meshnet_resident_bytes,
|
||||||
|
+ registered_bytes, registered_name("token_embd.weight"), registered(output_norm) && registered(output) };
|
||||||
|
+ pimpl->has_meshnet_range_report = true;
|
||||||
|
+ }
|
||||||
|
return true;
|
||||||
|
@@ -1711,6 +1761,14 @@ uint64_t llama_model::n_elements() const {
|
||||||
|
}
|
||||||
|
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
|
||||||
|
+ if (out == nullptr || !pimpl->has_meshnet_range_report) return false;
|
||||||
|
+ *out = pimpl->meshnet_range_report;
|
||||||
|
+ return true;
|
||||||
|
+}
|
||||||
|
@@ -2308,6 +2366,8 @@ llama_model_params llama_model_default_params() {
|
||||||
|
/*.kv_overrides =*/ nullptr,
|
||||||
|
+ /*.meshnet_owned_layer_start =*/ 0,
|
||||||
|
+ /*.meshnet_owned_layer_end =*/ 0,
|
||||||
|
@@ -2641,6 +2701,10 @@ uint64_t llama_model_size(const llama_model * model) {
|
||||||
|
}
|
||||||
|
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
|
||||||
|
+ return model != nullptr && model->meshnet_range_report(out);
|
||||||
|
+}
|
||||||
|
diff --git a/src/llama-model.h b/src/llama-model.h
|
||||||
|
index 45b054ce..1b3f9bd0 100644
|
||||||
|
--- a/src/llama-model.h
|
||||||
|
+++ b/src/llama-model.h
|
||||||
|
@@ -652,6 +652,8 @@ struct llama_model {
|
||||||
|
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
|
||||||
|
+
|
||||||
|
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
|
||||||
|
index 4bfebc88..b4f25aed 100644
|
||||||
|
--- a/src/models/llama.cpp
|
||||||
|
+++ b/src/models/llama.cpp
|
||||||
|
@@ -34,18 +34,26 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
|
||||||
|
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
|
||||||
|
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
|
||||||
|
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
|
||||||
|
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
|
||||||
|
- // output
|
||||||
|
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
|
||||||
|
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
|
||||||
|
+ if (!meshnet_range_requested || meshnet_start == 0) tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
|
||||||
|
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
|
||||||
|
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
|
||||||
|
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
|
||||||
|
- // if output is NULL, init from the input tok embed
|
||||||
|
- if (output == NULL) {
|
||||||
|
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
|
||||||
|
+ if (output == NULL) output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
|
||||||
|
}
|
||||||
|
- for (int i = 0; i < n_layer; ++i) {
|
||||||
|
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
|
||||||
|
@@ -102,6 +110,25 @@ llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_grap
|
||||||
|
+ llama_meshnet_range_report meshnet_report = {};
|
||||||
|
+ if (model.meshnet_range_report(&meshnet_report)) {
|
||||||
|
+ if (meshnet_report.start_layer != 0) throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
|
||||||
|
+ if (meshnet_report.end_layer != n_layer) throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
|
||||||
|
+ if (!meshnet_report.has_token_embeddings) throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
|
||||||
|
+ if (!meshnet_report.has_output_head) throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
|
||||||
|
+ }
|
||||||
|
ggml_tensor * cur;
|
||||||
|
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
|
||||||
|
index 855295c1..9a7be6ee 100644
|
||||||
|
--- a/tests/CMakeLists.txt
|
||||||
|
+++ b/tests/CMakeLists.txt
|
||||||
|
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
|
||||||
|
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
|
||||||
|
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
|
||||||
|
new file mode 100644
|
||||||
|
index 00000000..7b58ebf8
|
||||||
|
--- /dev/null
|
||||||
|
+++ b/tests/test-meshnet-range-ownership.cpp
|
||||||
|
@@ -0,0 +1,6 @@
|
||||||
|
+#include "ggml.h"
|
||||||
|
+#include "gguf.h"
|
||||||
|
+#include "llama.h"
|
||||||
|
+#include "../src/llama-model.h"
|
||||||
|
+#include <cstdio>
|
||||||
|
+#include <cstring>
|
||||||
3
packages/node/native/llama/patches/SHA256SUMS
Normal file
3
packages/node/native/llama/patches/SHA256SUMS
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
# SHA-256 digests for the ordered patch series. Do not reorder this file.
|
||||||
|
1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
|
||||||
|
51c205e3ca26e104f80c838eeeb11115b8d436036014116d2bb407178c30e0bd 0002-dense-llama-owned-range-loader.patch
|
||||||
2
packages/node/native/llama/patches/series
Normal file
2
packages/node/native/llama/patches/series
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
|
||||||
|
0002-dense-llama-owned-range-loader.patch
|
||||||
@@ -33,6 +33,16 @@ enum SchemaVersion {
|
|||||||
SCHEMA_VERSION_1 = 1;
|
SCHEMA_VERSION_1 = 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Certified transformer boundary family. Names alone are not an adapter: a
|
||||||
|
// Node must select an explicit architecture contract before interpreting a
|
||||||
|
// TensorBundle.
|
||||||
|
enum ArchitectureType {
|
||||||
|
ARCHITECTURE_TYPE_UNSPECIFIED = 0;
|
||||||
|
ARCHITECTURE_TYPE_DENSE = 1;
|
||||||
|
ARCHITECTURE_TYPE_MOE = 2;
|
||||||
|
ARCHITECTURE_TYPE_MLA = 3;
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Tensor bundle — the public activation boundary
|
// Tensor bundle — the public activation boundary
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -142,6 +152,11 @@ message TensorBundle {
|
|||||||
// boundary payload can evolve without a whole-protocol version bump.
|
// boundary payload can evolve without a whole-protocol version bump.
|
||||||
uint32 bundle_version = 1;
|
uint32 bundle_version = 1;
|
||||||
repeated NamedTensor tensors = 2;
|
repeated NamedTensor tensors = 2;
|
||||||
|
// Explicit adapter selection; names are never interpreted through unchecked
|
||||||
|
// substitutions. UNSPECIFIED is accepted only for legacy DGR-002 bundles.
|
||||||
|
ArchitectureType architecture = 3;
|
||||||
|
// Adapter-owned semantic boundary, such as "pre_tail_residual".
|
||||||
|
string boundary_point = 4;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -393,8 +408,9 @@ message ActivationChunk {
|
|||||||
// A decode step is one token: the envelope's identity fields are already fixed
|
// A decode step is one token: the envelope's identity fields are already fixed
|
||||||
// for the life of the stream, so repeating them per token is pure overhead on
|
// for the life of the stream, so repeating them per token is pure overhead on
|
||||||
// the hottest path. A DecodeStep carries only what changes — the step, the
|
// the hottest path. A DecodeStep carries only what changes — the step, the
|
||||||
// position, and one tensor — and inherits the rest from the SessionOpen
|
// position, and one compact boundary encoding — and inherits the rest from the
|
||||||
// handshake. A peer may always fall back to ActivationChunk with PHASE_DECODE.
|
// SessionOpen handshake. A Node may always fall back to ActivationChunk with
|
||||||
|
// PHASE_DECODE.
|
||||||
message DecodeStep {
|
message DecodeStep {
|
||||||
// Idempotency step within the session; also orders the stream.
|
// Idempotency step within the session; also orders the stream.
|
||||||
uint64 idempotency_step = 1;
|
uint64 idempotency_step = 1;
|
||||||
@@ -402,11 +418,50 @@ message DecodeStep {
|
|||||||
uint64 position = 2;
|
uint64 position = 2;
|
||||||
// Tokens the receiver's cache must already hold. A mismatch is a CACHE_MISS.
|
// Tokens the receiver's cache must already hold. A mismatch is a CACHE_MISS.
|
||||||
uint64 expected_past_len = 3;
|
uint64 expected_past_len = 3;
|
||||||
// The single boundary tensor for this token, typically [1, 1, hidden].
|
// Legacy compact one-tensor boundary, typically [1, 1, hidden]. New readers
|
||||||
|
// accept it as a TensorBundle with one member; writers retain it where the
|
||||||
|
// selected architecture genuinely requires only one tensor.
|
||||||
NamedTensor tensor = 4;
|
NamedTensor tensor = 4;
|
||||||
// Work id for attribution; the session/epoch/fingerprint/range are inherited.
|
// Work id for attribution; the session/epoch/fingerprint/range are inherited.
|
||||||
string work_id = 5;
|
string work_id = 5;
|
||||||
int64 deadline_unix_nanos = 6;
|
int64 deadline_unix_nanos = 6;
|
||||||
|
// Versioned architecture boundary. This carries sidebands such as MoE router
|
||||||
|
// state or MLA/IndexShare state; when both fields are set, bundle is
|
||||||
|
// authoritative and tensor is retained only for older-Node forwarding.
|
||||||
|
TensorBundle bundle = 7;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exact request-level identity for a tail result. Runtime recipe identity is
|
||||||
|
// deliberately repeated here: sampling the right logits with the wrong chat
|
||||||
|
// template or reasoning mode is a different request, not a compatible result.
|
||||||
|
message RequestRecipeIdentity {
|
||||||
|
string request_id = 1;
|
||||||
|
string runtime_recipe_digest = 2;
|
||||||
|
string chat_template_id = 3;
|
||||||
|
string chat_template_version = 4;
|
||||||
|
string reasoning_mode = 5;
|
||||||
|
ArchitectureType architecture = 6;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sampling is an explicit tail-only operation. A non-tail Shard never applies
|
||||||
|
// final norm, LM head, row pruning, or sampling to its boundary output.
|
||||||
|
message SamplingParameters {
|
||||||
|
float temperature = 1;
|
||||||
|
float top_p = 2;
|
||||||
|
uint32 top_k = 3;
|
||||||
|
uint64 seed = 4;
|
||||||
|
bool greedy = 5;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Typed tail completion. The oneof keeps token id 0 unambiguous and prevents a
|
||||||
|
// caller from treating an activation tensor as an inferred completion.
|
||||||
|
message TailResult {
|
||||||
|
RequestRecipeIdentity identity = 1;
|
||||||
|
SamplingParameters sampling = 2;
|
||||||
|
oneof output {
|
||||||
|
TensorBundle logits = 3;
|
||||||
|
uint32 sampled_token_id = 4;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Drop session state for a Route Session. Bounded memory does not depend on
|
// Drop session state for a Route Session. Bounded memory does not depend on
|
||||||
@@ -472,6 +527,7 @@ message SessionResponse {
|
|||||||
Ack ack = 3;
|
Ack ack = 3;
|
||||||
FlowControl flow_control = 4;
|
FlowControl flow_control = 4;
|
||||||
ShardStatus status = 5;
|
ShardStatus status = 5;
|
||||||
|
TailResult tail_result = 6;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
BIN
packages/node/native/testdata/decode_step_golden.binpb
vendored
Normal file
BIN
packages/node/native/testdata/decode_step_golden.binpb
vendored
Normal file
Binary file not shown.
@@ -265,7 +265,9 @@ void TestDecodeFastPathIsSmall() {
|
|||||||
step->set_position(1024);
|
step->set_position(1024);
|
||||||
step->set_expected_past_len(1024);
|
step->set_expected_past_len(1024);
|
||||||
step->set_work_id("work-7f3a");
|
step->set_work_id("work-7f3a");
|
||||||
sp::NamedTensor *tensor = step->mutable_tensor();
|
sp::TensorBundle *bundle = step->mutable_bundle();
|
||||||
|
bundle->set_bundle_version(1);
|
||||||
|
sp::NamedTensor *tensor = bundle->add_tensors();
|
||||||
tensor->set_name("hidden_states");
|
tensor->set_name("hidden_states");
|
||||||
tensor->add_shape(1);
|
tensor->add_shape(1);
|
||||||
tensor->add_shape(1);
|
tensor->add_shape(1);
|
||||||
@@ -290,7 +292,27 @@ void TestDecodeFastPathIsSmall() {
|
|||||||
CHECK(parsed.ParseFromString(bytes));
|
CHECK(parsed.ParseFromString(bytes));
|
||||||
CHECK(parsed.kind_case() == sp::SessionRequest::kDecode);
|
CHECK(parsed.kind_case() == sp::SessionRequest::kDecode);
|
||||||
CHECK_EQ(parsed.decode().position(), 1024u);
|
CHECK_EQ(parsed.decode().position(), 1024u);
|
||||||
CHECK_EQ(parsed.decode().tensor().total_bytes(), 16u);
|
CHECK_EQ(parsed.decode().bundle().tensors_size(), 1);
|
||||||
|
CHECK_EQ(parsed.decode().bundle().tensors(0).total_bytes(), 16u);
|
||||||
|
}
|
||||||
|
|
||||||
|
void TestDecodeBundleVector(const std::string &bytes) {
|
||||||
|
sp::SessionRequest request;
|
||||||
|
CHECK(request.ParseFromString(bytes));
|
||||||
|
CHECK(request.kind_case() == sp::SessionRequest::kDecode);
|
||||||
|
const sp::DecodeStep &step = request.decode();
|
||||||
|
CHECK_EQ(step.idempotency_step(), 43u);
|
||||||
|
CHECK_EQ(step.position(), 384u);
|
||||||
|
CHECK_EQ(step.expected_past_len(), 384u);
|
||||||
|
CHECK_EQ(step.work_id(), std::string("decode-7f3a"));
|
||||||
|
CHECK(step.has_bundle());
|
||||||
|
CHECK_EQ(step.bundle().bundle_version(), 1u);
|
||||||
|
CHECK_EQ(step.bundle().architecture(), sp::ARCHITECTURE_TYPE_MLA);
|
||||||
|
CHECK_EQ(step.bundle().boundary_point(), std::string("pre_tail_residual"));
|
||||||
|
CHECK_EQ(step.bundle().tensors_size(), 2);
|
||||||
|
CHECK_EQ(step.bundle().tensors(0).name(), std::string("hidden_states"));
|
||||||
|
CHECK_EQ(step.bundle().tensors(1).name(), std::string("index_topk"));
|
||||||
|
CHECK_EQ(step.bundle().tensors(1).dtype(), sp::DTYPE_INT32);
|
||||||
}
|
}
|
||||||
|
|
||||||
} // namespace
|
} // namespace
|
||||||
@@ -308,9 +330,11 @@ int main(int argc, char **argv) {
|
|||||||
ReadFile(testdata / "session_request_golden.binpb");
|
ReadFile(testdata / "session_request_golden.binpb");
|
||||||
const std::string capability_bytes =
|
const std::string capability_bytes =
|
||||||
ReadFile(testdata / "capability_report_golden.binpb");
|
ReadFile(testdata / "capability_report_golden.binpb");
|
||||||
|
const std::string decode_bytes = ReadFile(testdata / "decode_step_golden.binpb");
|
||||||
|
|
||||||
TestSessionRequestVector(session_bytes);
|
TestSessionRequestVector(session_bytes);
|
||||||
TestCapabilityReportVector(capability_bytes);
|
TestCapabilityReportVector(capability_bytes);
|
||||||
|
TestDecodeBundleVector(decode_bytes);
|
||||||
TestUnknownFieldsArePreserved(session_bytes);
|
TestUnknownFieldsArePreserved(session_bytes);
|
||||||
TestSparseMessageParses();
|
TestSparseMessageParses();
|
||||||
TestDecodeFastPathIsSmall();
|
TestDecodeFastPathIsSmall();
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
import socket
|
import socket
|
||||||
import threading
|
|
||||||
from typing import Callable
|
from typing import Callable
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|||||||
@@ -3,9 +3,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import datetime
|
import datetime
|
||||||
import hashlib
|
|
||||||
import ipaddress
|
import ipaddress
|
||||||
import json
|
|
||||||
import os
|
import os
|
||||||
import socket
|
import socket
|
||||||
import ssl
|
import ssl
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import logging
|
import logging
|
||||||
import sys
|
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
|||||||
@@ -30,6 +30,13 @@ import time
|
|||||||
from dataclasses import dataclass, replace
|
from dataclasses import dataclass, replace
|
||||||
from typing import Any, Callable, Mapping
|
from typing import Any, Callable, Mapping
|
||||||
|
|
||||||
|
from .recipe import (
|
||||||
|
CertificationLedger,
|
||||||
|
FingerprintMismatch,
|
||||||
|
RecipeIdentityError,
|
||||||
|
parse_identity,
|
||||||
|
)
|
||||||
|
|
||||||
# The capability report layout this tracker reads (meshnet_node.capability).
|
# The capability report layout this tracker reads (meshnet_node.capability).
|
||||||
SUPPORTED_SCHEMA_VERSION = 1
|
SUPPORTED_SCHEMA_VERSION = 1
|
||||||
|
|
||||||
@@ -58,6 +65,12 @@ STATE_MODEL_MISMATCH = "model-mismatch"
|
|||||||
STATE_SHARD_MISMATCH = "shard-mismatch"
|
STATE_SHARD_MISMATCH = "shard-mismatch"
|
||||||
STATE_RECIPE_MISMATCH = "recipe-mismatch"
|
STATE_RECIPE_MISMATCH = "recipe-mismatch"
|
||||||
STATE_CATALOGUE_INCOMPATIBLE = "catalogue-incompatible"
|
STATE_CATALOGUE_INCOMPATIBLE = "catalogue-incompatible"
|
||||||
|
# The node presented a DGR-003 identity block whose declared fingerprint is not
|
||||||
|
# the digest of the axes it declared. Identity is derived, never asserted.
|
||||||
|
STATE_FINGERPRINT_MISMATCH = "fingerprint-mismatch"
|
||||||
|
# Registered-but-dark: a known recipe no real distributed forward has certified.
|
||||||
|
# Visible to an operator, never routable for user traffic.
|
||||||
|
STATE_UNCERTIFIED = "uncertified"
|
||||||
|
|
||||||
ALL_STATES = (
|
ALL_STATES = (
|
||||||
STATE_ADMITTED,
|
STATE_ADMITTED,
|
||||||
@@ -69,6 +82,8 @@ ALL_STATES = (
|
|||||||
STATE_SHARD_MISMATCH,
|
STATE_SHARD_MISMATCH,
|
||||||
STATE_RECIPE_MISMATCH,
|
STATE_RECIPE_MISMATCH,
|
||||||
STATE_CATALOGUE_INCOMPATIBLE,
|
STATE_CATALOGUE_INCOMPATIBLE,
|
||||||
|
STATE_FINGERPRINT_MISMATCH,
|
||||||
|
STATE_UNCERTIFIED,
|
||||||
)
|
)
|
||||||
|
|
||||||
# --- Compatibility policy for nodes that predate the capability protocol. ---
|
# --- Compatibility policy for nodes that predate the capability protocol. ---
|
||||||
@@ -165,12 +180,29 @@ class CapabilityState:
|
|||||||
recorded_at: float = 0.0
|
recorded_at: float = 0.0
|
||||||
schema_version: int | None = None
|
schema_version: int | None = None
|
||||||
diagnostics: tuple[str, ...] = ()
|
diagnostics: tuple[str, ...] = ()
|
||||||
|
# The DGR-003 compatibility fingerprint, *re-derived* by this tracker from
|
||||||
|
# the axes the node declared — never copied from what the node claimed.
|
||||||
|
# Absent for a node that predates DGR-003.
|
||||||
|
model_artifact_digest: str | None = None
|
||||||
|
runtime_recipe_digest: str | None = None
|
||||||
|
shard_binding_digest: str | None = None
|
||||||
|
# The tracker ledger's verdict on that fingerprint at evaluation time
|
||||||
|
# ("dark"/"certified"), so the network map answers "why is this exact node
|
||||||
|
# not routing" without a second query. None when no identity was presented.
|
||||||
|
certification: str | None = None
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def proven(self) -> bool:
|
def proven(self) -> bool:
|
||||||
"""The presented proof covers exactly what the node advertised."""
|
"""The presented proof covers exactly what the node advertised."""
|
||||||
return self.state == STATE_ADMITTED
|
return self.state == STATE_ADMITTED
|
||||||
|
|
||||||
|
@property
|
||||||
|
def fingerprint(self) -> tuple[str, str] | None:
|
||||||
|
"""What route formation compares. `None` when the node declares no identity."""
|
||||||
|
if self.model_artifact_digest is None or self.runtime_recipe_digest is None:
|
||||||
|
return None
|
||||||
|
return (self.model_artifact_digest, self.runtime_recipe_digest)
|
||||||
|
|
||||||
def routable_under(self, policy: str) -> bool:
|
def routable_under(self, policy: str) -> bool:
|
||||||
if self.proven:
|
if self.proven:
|
||||||
return True
|
return True
|
||||||
@@ -197,6 +229,10 @@ class CapabilityState:
|
|||||||
"recorded_at": self.recorded_at,
|
"recorded_at": self.recorded_at,
|
||||||
"schema_version": self.schema_version,
|
"schema_version": self.schema_version,
|
||||||
"diagnostics": list(self.diagnostics),
|
"diagnostics": list(self.diagnostics),
|
||||||
|
"model_artifact_digest": self.model_artifact_digest,
|
||||||
|
"runtime_recipe_digest": self.runtime_recipe_digest,
|
||||||
|
"shard_binding_digest": self.shard_binding_digest,
|
||||||
|
"certification": self.certification,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -224,6 +260,7 @@ def evaluate_report(
|
|||||||
declared_recipe_version: str | None = None,
|
declared_recipe_version: str | None = None,
|
||||||
now: float | None = None,
|
now: float | None = None,
|
||||||
max_age_seconds: float = DEFAULT_MAX_REPORT_AGE_SECONDS,
|
max_age_seconds: float = DEFAULT_MAX_REPORT_AGE_SECONDS,
|
||||||
|
ledger: CertificationLedger | None = None,
|
||||||
) -> CapabilityState:
|
) -> CapabilityState:
|
||||||
"""Judge the proof a node presented against what that node is advertising.
|
"""Judge the proof a node presented against what that node is advertising.
|
||||||
|
|
||||||
@@ -308,6 +345,84 @@ def evaluate_report(
|
|||||||
f"the node declared v{declared_recipe_version}",
|
f"the node declared v{declared_recipe_version}",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
identity = None
|
||||||
|
if report.get("identity") is not None:
|
||||||
|
try:
|
||||||
|
identity = parse_identity(report["identity"])
|
||||||
|
except FingerprintMismatch as exc:
|
||||||
|
return base.with_state(STATE_FINGERPRINT_MISMATCH, str(exc))
|
||||||
|
except RecipeIdentityError as exc:
|
||||||
|
return base.with_state(
|
||||||
|
STATE_INVALID, f"capability identity block is unusable: {exc}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# The report's `shard` range is inclusive/inclusive (the CLI and backend
|
||||||
|
# convention); the identity's is inclusive/exclusive (the protocol's, and
|
||||||
|
# ADR-0012's). A node whose two halves disagree has not proven the range
|
||||||
|
# it claims, whichever one is right.
|
||||||
|
if (identity.shard_start, identity.shard_end) != (
|
||||||
|
base.shard_start,
|
||||||
|
(base.shard_end or 0) + 1,
|
||||||
|
):
|
||||||
|
return base.with_state(
|
||||||
|
STATE_SHARD_MISMATCH,
|
||||||
|
f"identity covers layers {identity.shard_start}–{identity.shard_end} "
|
||||||
|
f"(end-exclusive), but the proof is for layers {base.shard_start}–"
|
||||||
|
f"{base.shard_end} (end-inclusive)",
|
||||||
|
)
|
||||||
|
|
||||||
|
if not model_matches(identity.artifact_id):
|
||||||
|
return base.with_state(
|
||||||
|
STATE_MODEL_MISMATCH,
|
||||||
|
f"identity is for artifact {identity.artifact_id!r}, but the node "
|
||||||
|
f"registered {advertised_model!r}",
|
||||||
|
)
|
||||||
|
model_claim = report["model"]
|
||||||
|
if model_claim.get("revision") != identity.revision:
|
||||||
|
return base.with_state(
|
||||||
|
STATE_MODEL_MISMATCH,
|
||||||
|
"identity revision does not match the capability proof",
|
||||||
|
)
|
||||||
|
config_fingerprint = model_claim.get("config_fingerprint")
|
||||||
|
if isinstance(config_fingerprint, str) and config_fingerprint.startswith(
|
||||||
|
"sha256:"
|
||||||
|
):
|
||||||
|
config_fingerprint = config_fingerprint.removeprefix("sha256:")
|
||||||
|
if config_fingerprint != identity.architecture_digest:
|
||||||
|
return base.with_state(
|
||||||
|
STATE_MODEL_MISMATCH,
|
||||||
|
"identity architecture/config digest does not match the capability proof",
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
identity.recipe_id != base.recipe_id
|
||||||
|
or identity.recipe_version != base.recipe_version
|
||||||
|
or identity.catalogue_version != base.catalogue_version
|
||||||
|
):
|
||||||
|
return base.with_state(
|
||||||
|
STATE_RECIPE_MISMATCH,
|
||||||
|
"identity recipe labels do not match the capability proof",
|
||||||
|
)
|
||||||
|
if identity.axes["backend_id"] != base.backend_id:
|
||||||
|
return base.with_state(
|
||||||
|
STATE_RECIPE_MISMATCH,
|
||||||
|
"identity backend does not match the capability proof",
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
base.quantization is not None
|
||||||
|
and identity.axes["weight_quantization"] != base.quantization
|
||||||
|
):
|
||||||
|
return base.with_state(
|
||||||
|
STATE_RECIPE_MISMATCH,
|
||||||
|
"identity weight quantization does not match the capability proof",
|
||||||
|
)
|
||||||
|
|
||||||
|
base = replace(
|
||||||
|
base,
|
||||||
|
model_artifact_digest=identity.model_artifact_digest,
|
||||||
|
runtime_recipe_digest=identity.runtime_recipe_digest,
|
||||||
|
shard_binding_digest=identity.shard_binding_digest,
|
||||||
|
)
|
||||||
|
|
||||||
if status != STATUS_PASSED:
|
if status != STATUS_PASSED:
|
||||||
return base.with_state(
|
return base.with_state(
|
||||||
STATE_FAILED,
|
STATE_FAILED,
|
||||||
@@ -328,6 +443,29 @@ def evaluate_report(
|
|||||||
f"proof is timestamped {-age:.0f}s in the future; check the node's clock",
|
f"proof is timestamped {-age:.0f}s in the future; check the node's clock",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# The digest only establishes that this report is self-consistent. It does
|
||||||
|
# not authenticate a node or prove a distributed forward. Exact recipes are
|
||||||
|
# therefore registered-but-dark until tracker-owned certification records
|
||||||
|
# that forward.
|
||||||
|
#
|
||||||
|
# `ledger is None` means the caller owns no certification authority. It is
|
||||||
|
# not "certify anything" and it is not "make one up": a disposable ledger
|
||||||
|
# would register the recipe into state that is discarded on return, which
|
||||||
|
# reads like certification is wired when nothing is recording it. The only
|
||||||
|
# safe reading is that nothing here has certified this recipe, so it stays
|
||||||
|
# dark. `TrackerServer` owns the real ledger and passes it in.
|
||||||
|
if identity is not None:
|
||||||
|
if ledger is None:
|
||||||
|
return base.with_state(
|
||||||
|
STATE_UNCERTIFIED,
|
||||||
|
"no certification ledger; an exact recipe is dark until a "
|
||||||
|
"tracker-owned distributed forward certifies it",
|
||||||
|
)
|
||||||
|
recipe_status = ledger.register(identity)
|
||||||
|
base = replace(base, certification=recipe_status.status)
|
||||||
|
if not recipe_status.may_serve:
|
||||||
|
return base.with_state(STATE_UNCERTIFIED, recipe_status.detail)
|
||||||
|
|
||||||
return base.with_state(
|
return base.with_state(
|
||||||
STATE_ADMITTED,
|
STATE_ADMITTED,
|
||||||
f"{base.model_id} layers {base.shard_start}–{base.shard_end} proven on "
|
f"{base.model_id} layers {base.shard_start}–{base.shard_end} proven on "
|
||||||
|
|||||||
@@ -16,8 +16,8 @@ import threading
|
|||||||
import time
|
import time
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from typing import Any, Callable
|
from typing import Callable
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|||||||
556
packages/tracker/meshnet_tracker/recipe.py
Normal file
556
packages/tracker/meshnet_tracker/recipe.py
Normal file
@@ -0,0 +1,556 @@
|
|||||||
|
"""Tracker-side artifact and runtime recipe identity (DGR-003).
|
||||||
|
|
||||||
|
The node computes a compatibility fingerprint in `meshnet_node.runtime_recipe`.
|
||||||
|
This module recomputes it, and the recomputation is the entire point.
|
||||||
|
|
||||||
|
A fingerprint that arrives on the wire is a **claim**. If the tracker stored
|
||||||
|
what a node asserted, a node could assert the digest of a certified recipe while
|
||||||
|
running an uncertified one — and admission, route selection, and the gRPC
|
||||||
|
handshake would all wave it through, because they all compare the digest it
|
||||||
|
handed them. So the tracker derives the digest from the *axes* the node
|
||||||
|
declared, and refuses any report whose claim does not match the derivation. A
|
||||||
|
node can lie about its axes, and a real forward will catch that; it cannot lie
|
||||||
|
about the digest of the axes it declared.
|
||||||
|
|
||||||
|
This module deliberately does **not** import `meshnet_node`: the tracker package
|
||||||
|
does not depend on the node package, and an admission gate that shares an
|
||||||
|
implementation with the thing it admits is not an independent check. The two
|
||||||
|
implementations are pinned together by a committed conformance vector
|
||||||
|
(``tests/data/recipe_fingerprint_vectors.json``). If they drift, a test fails —
|
||||||
|
rather than a route quietly failing to form, or worse, quietly forming.
|
||||||
|
|
||||||
|
Recipes arrive **dark**: registered, visible to an operator, and not routable for
|
||||||
|
user traffic. Only a real distributed forward — at least two distinct nodes,
|
||||||
|
covering the whole model, emitting real tokens — takes a recipe out of the dark
|
||||||
|
(:class:`CertificationLedger`).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Iterable, Mapping
|
||||||
|
|
||||||
|
# Layout of the identity block this tracker reads (meshnet_node.runtime_recipe).
|
||||||
|
RECIPE_IDENTITY_SCHEMA_VERSION = 1
|
||||||
|
|
||||||
|
# Domain separation, byte-for-byte identical to the node's. These strings are
|
||||||
|
# part of the wire contract, not an implementation detail: changing one here
|
||||||
|
# without changing it there silently partitions every route.
|
||||||
|
ARTIFACT_DIGEST_DOMAIN = "meshnet.model-artifact.v1"
|
||||||
|
RECIPE_DIGEST_DOMAIN = "meshnet.runtime-recipe.v1"
|
||||||
|
SHARD_BINDING_DIGEST_DOMAIN = "meshnet.shard-binding.v1"
|
||||||
|
|
||||||
|
# The axes a recipe digest commits to. Order is irrelevant (the canonical JSON
|
||||||
|
# sorts keys); membership is not — an axis missing here is an axis the tracker
|
||||||
|
# would let a node change without changing its identity.
|
||||||
|
RECIPE_AXES: tuple[str, ...] = (
|
||||||
|
"weight_quantization",
|
||||||
|
"activation_dtype",
|
||||||
|
"compute_dtype",
|
||||||
|
"kv_dtype",
|
||||||
|
"kv_layout",
|
||||||
|
"tokenizer_revision",
|
||||||
|
"architecture_adapter",
|
||||||
|
"backend_id",
|
||||||
|
"runtime_version",
|
||||||
|
"boundary_schema_version",
|
||||||
|
"protocol_schema_version",
|
||||||
|
)
|
||||||
|
|
||||||
|
_INT_AXES = frozenset({"boundary_schema_version", "protocol_schema_version"})
|
||||||
|
|
||||||
|
STATUS_UNKNOWN = "unknown"
|
||||||
|
STATUS_DARK = "dark"
|
||||||
|
STATUS_CERTIFIED = "certified"
|
||||||
|
|
||||||
|
MIN_CERTIFYING_NODES = 2
|
||||||
|
|
||||||
|
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
|
||||||
|
_MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"})
|
||||||
|
|
||||||
|
|
||||||
|
class RecipeIdentityError(ValueError):
|
||||||
|
"""A presented identity block is malformed or internally inconsistent."""
|
||||||
|
|
||||||
|
|
||||||
|
class FingerprintMismatch(RecipeIdentityError):
|
||||||
|
"""A supplied digest is inconsistent with its identity declaration.
|
||||||
|
|
||||||
|
A digest is an integrity and compatibility claim, not an authenticated
|
||||||
|
statement about what a node is actually executing. Distributed
|
||||||
|
certification remains the trust boundary.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_sha256(value: Any) -> str:
|
||||||
|
payload = json.dumps(
|
||||||
|
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
|
||||||
|
)
|
||||||
|
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(domain: str, body: Mapping[str, Any]) -> str:
|
||||||
|
return canonical_sha256({"domain": domain, "body": dict(body)})
|
||||||
|
|
||||||
|
|
||||||
|
def _text(value: Any, what: str) -> str:
|
||||||
|
if not isinstance(value, str) or not value.strip():
|
||||||
|
raise RecipeIdentityError(f"{what!r} must be a non-empty string")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _hex64(value: Any, what: str) -> str:
|
||||||
|
text = _text(value, what)
|
||||||
|
if not _HEX64.match(text):
|
||||||
|
raise RecipeIdentityError(f"{what!r} must be a SHA-256 hex digest")
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _integer(value: Any, what: str, minimum: int) -> int:
|
||||||
|
if isinstance(value, bool) or not isinstance(value, int):
|
||||||
|
raise RecipeIdentityError(f"{what!r} must be an integer")
|
||||||
|
if value < minimum:
|
||||||
|
raise RecipeIdentityError(f"{what!r} must be >= {minimum}")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _pin(value: Any, what: str) -> str:
|
||||||
|
text = _text(value, what)
|
||||||
|
lowered = text.strip().lower()
|
||||||
|
if lowered in _MOVING_REFS or lowered.startswith("refs/"):
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"{what!r} is a moving reference, not an exact revision pin"
|
||||||
|
)
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _mapping(value: Any, what: str) -> Mapping[str, Any]:
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise RecipeIdentityError(f"{what!r} must be a JSON object")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def artifact_digest(
|
||||||
|
*,
|
||||||
|
source_digest: str,
|
||||||
|
architecture: str,
|
||||||
|
architecture_digest: str,
|
||||||
|
layer_count: int,
|
||||||
|
) -> str:
|
||||||
|
"""The artifact half of the fingerprint, derived exactly as the node derives it."""
|
||||||
|
return _digest(
|
||||||
|
ARTIFACT_DIGEST_DOMAIN,
|
||||||
|
{
|
||||||
|
"source_digest": source_digest,
|
||||||
|
"architecture": architecture,
|
||||||
|
"architecture_digest": architecture_digest,
|
||||||
|
"layer_count": layer_count,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def recipe_digest(axes: Mapping[str, Any]) -> str:
|
||||||
|
"""The recipe half of the fingerprint, derived exactly as the node derives it."""
|
||||||
|
missing = [axis for axis in RECIPE_AXES if axis not in axes]
|
||||||
|
if missing:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
"recipe is missing required axes: " + ", ".join(missing)
|
||||||
|
)
|
||||||
|
return _digest(RECIPE_DIGEST_DOMAIN, {axis: axes[axis] for axis in RECIPE_AXES})
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PresentedIdentity:
|
||||||
|
"""One node's declared artifact/recipe identity, with digests re-derived here.
|
||||||
|
|
||||||
|
`model_artifact_digest` and `runtime_recipe_digest` are computed by this
|
||||||
|
tracker from the declared axes. They are never copied from the report.
|
||||||
|
"""
|
||||||
|
|
||||||
|
artifact_id: str
|
||||||
|
revision: str
|
||||||
|
source_digest: str
|
||||||
|
content_digest: str
|
||||||
|
derivative_range: tuple[int, int] | None
|
||||||
|
architecture: str
|
||||||
|
architecture_digest: str
|
||||||
|
layer_count: int
|
||||||
|
is_derivative: bool
|
||||||
|
shard_start: int
|
||||||
|
shard_end: int
|
||||||
|
axes: Mapping[str, Any]
|
||||||
|
recipe_id: str
|
||||||
|
recipe_version: str
|
||||||
|
catalogue_version: str
|
||||||
|
|
||||||
|
@property
|
||||||
|
def model_artifact_digest(self) -> str:
|
||||||
|
return artifact_digest(
|
||||||
|
source_digest=self.source_digest,
|
||||||
|
architecture=self.architecture,
|
||||||
|
architecture_digest=self.architecture_digest,
|
||||||
|
layer_count=self.layer_count,
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def runtime_recipe_digest(self) -> str:
|
||||||
|
return recipe_digest(self.axes)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def key(self) -> tuple[str, str]:
|
||||||
|
return (self.model_artifact_digest, self.runtime_recipe_digest)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shard_binding_digest(self) -> str:
|
||||||
|
"""This participant's own bytes and exact range, bound to the source.
|
||||||
|
|
||||||
|
The route fingerprint (:attr:`key`) is deliberately range-independent —
|
||||||
|
Shards on one route own different ranges, so a range-sensitive digest
|
||||||
|
would stop any two of them from ever agreeing. The consequence is that
|
||||||
|
the fingerprint alone cannot tell two *different splits of the same
|
||||||
|
source* apart: a derivative can assert the right source and recipe and
|
||||||
|
inherit certification earned by bytes it does not hold.
|
||||||
|
|
||||||
|
This digest is what closes that. It commits to the derivative's own
|
||||||
|
content hash and its exact end-exclusive range, so certification can be
|
||||||
|
recipe-wide while every serving node is still separately pinned to the
|
||||||
|
blob it was admitted on (`TrackerServer.certify_recipe`).
|
||||||
|
"""
|
||||||
|
return _digest(
|
||||||
|
SHARD_BINDING_DIGEST_DOMAIN,
|
||||||
|
{
|
||||||
|
"source_digest": self.source_digest,
|
||||||
|
"content_digest": self.content_digest,
|
||||||
|
"architecture_digest": self.architecture_digest,
|
||||||
|
"derivative_range": list(self.derivative_range)
|
||||||
|
if self.derivative_range is not None
|
||||||
|
else None,
|
||||||
|
"shard_start": self.shard_start,
|
||||||
|
"shard_end": self.shard_end,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def tokenizer_revision(self) -> str:
|
||||||
|
return str(self.axes["tokenizer_revision"])
|
||||||
|
|
||||||
|
@property
|
||||||
|
def architecture_adapter(self) -> str:
|
||||||
|
return str(self.axes["architecture_adapter"])
|
||||||
|
|
||||||
|
def fingerprint_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"model_artifact_digest": self.model_artifact_digest,
|
||||||
|
"runtime_recipe_digest": self.runtime_recipe_digest,
|
||||||
|
"recipe_id": self.recipe_id,
|
||||||
|
"recipe_version": self.recipe_version,
|
||||||
|
"catalogue_version": self.catalogue_version,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_identity(data: Any) -> PresentedIdentity:
|
||||||
|
"""Parse and *verify* a node's identity block.
|
||||||
|
|
||||||
|
Raises when the block is malformed, when a split artifact does not name the
|
||||||
|
exact source it was cut from, when a Shard advertises layers its artifact
|
||||||
|
does not contain, or when the declared fingerprint does not match the digest
|
||||||
|
of the axes it was declared with.
|
||||||
|
"""
|
||||||
|
doc = _mapping(data, "identity")
|
||||||
|
|
||||||
|
schema_version = doc.get("schema_version")
|
||||||
|
if schema_version != RECIPE_IDENTITY_SCHEMA_VERSION:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"identity block declares schema version {schema_version!r}; this tracker "
|
||||||
|
f"reads version {RECIPE_IDENTITY_SCHEMA_VERSION}"
|
||||||
|
)
|
||||||
|
|
||||||
|
artifact = _mapping(doc.get("artifact"), "identity.artifact")
|
||||||
|
recipe = _mapping(doc.get("recipe"), "identity.recipe")
|
||||||
|
|
||||||
|
layer_count = _integer(artifact.get("layer_count"), "artifact.layer_count", 1)
|
||||||
|
content_digest = _hex64(artifact.get("content_digest"), "artifact.content_digest")
|
||||||
|
|
||||||
|
raw_binding = artifact.get("derived_from")
|
||||||
|
if raw_binding is None:
|
||||||
|
source_digest = content_digest
|
||||||
|
binding: tuple[int, int] | None = None
|
||||||
|
else:
|
||||||
|
derived = _mapping(raw_binding, "artifact.derived_from")
|
||||||
|
source_digest = _hex64(
|
||||||
|
derived.get("source_artifact_digest"),
|
||||||
|
"artifact.derived_from.source_artifact_digest",
|
||||||
|
)
|
||||||
|
binding = (
|
||||||
|
_integer(derived.get("shard_start"), "derived_from.shard_start", 0),
|
||||||
|
_integer(derived.get("shard_end"), "derived_from.shard_end", 1),
|
||||||
|
)
|
||||||
|
if binding[1] <= binding[0]:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
"'derived_from' covers no layers; an empty split proves nothing"
|
||||||
|
)
|
||||||
|
if binding[1] > layer_count:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"'derived_from' claims layers {binding[0]}–{binding[1]}, but the "
|
||||||
|
f"source model has only {layer_count} layers"
|
||||||
|
)
|
||||||
|
if source_digest == content_digest:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
"a split artifact's source digest equals its own content digest; "
|
||||||
|
"an artifact is not a split of itself"
|
||||||
|
)
|
||||||
|
|
||||||
|
shard_start = _integer(doc.get("shard_start"), "shard_start", 0)
|
||||||
|
shard_end = _integer(doc.get("shard_end"), "shard_end", 1)
|
||||||
|
if shard_end <= shard_start:
|
||||||
|
raise RecipeIdentityError("a Shard owning no layer computes nothing")
|
||||||
|
if shard_end > layer_count:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"Shard owns layers {shard_start}–{shard_end}, but the artifact has only "
|
||||||
|
f"{layer_count} layers"
|
||||||
|
)
|
||||||
|
if binding is not None and not (
|
||||||
|
binding[0] <= shard_start and shard_end <= binding[1]
|
||||||
|
):
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"Shard advertises layers {shard_start}–{shard_end}, but its split "
|
||||||
|
f"artifact only contains layers {binding[0]}–{binding[1]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
axes: dict[str, Any] = {}
|
||||||
|
for axis in RECIPE_AXES:
|
||||||
|
if axis not in recipe:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"recipe is missing axis {axis!r}; an unstated axis cannot default"
|
||||||
|
)
|
||||||
|
value = recipe[axis]
|
||||||
|
if axis in _INT_AXES:
|
||||||
|
axes[axis] = _integer(value, f"recipe.{axis}", 1)
|
||||||
|
else:
|
||||||
|
axes[axis] = _text(value, f"recipe.{axis}")
|
||||||
|
_pin(axes["tokenizer_revision"], "recipe.tokenizer_revision")
|
||||||
|
|
||||||
|
identity = PresentedIdentity(
|
||||||
|
artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"),
|
||||||
|
revision=_pin(artifact.get("revision"), "artifact.revision"),
|
||||||
|
source_digest=source_digest,
|
||||||
|
content_digest=content_digest,
|
||||||
|
derivative_range=binding,
|
||||||
|
architecture=_text(artifact.get("architecture"), "artifact.architecture"),
|
||||||
|
architecture_digest=_hex64(
|
||||||
|
artifact.get("architecture_digest"), "artifact.architecture_digest"
|
||||||
|
),
|
||||||
|
layer_count=layer_count,
|
||||||
|
is_derivative=binding is not None,
|
||||||
|
shard_start=shard_start,
|
||||||
|
shard_end=shard_end,
|
||||||
|
axes=axes,
|
||||||
|
recipe_id=_text(recipe.get("recipe_id"), "recipe.recipe_id"),
|
||||||
|
recipe_version=_text(recipe.get("recipe_version"), "recipe.recipe_version"),
|
||||||
|
catalogue_version=_text(
|
||||||
|
recipe.get("catalogue_version"), "recipe.catalogue_version"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
declared = doc.get("fingerprint")
|
||||||
|
if declared is not None:
|
||||||
|
claim = _mapping(declared, "identity.fingerprint")
|
||||||
|
claimed_artifact = _hex64(
|
||||||
|
claim.get("model_artifact_digest"), "fingerprint.model_artifact_digest"
|
||||||
|
)
|
||||||
|
claimed_recipe = _hex64(
|
||||||
|
claim.get("runtime_recipe_digest"), "fingerprint.runtime_recipe_digest"
|
||||||
|
)
|
||||||
|
if (claimed_artifact, claimed_recipe) != identity.key:
|
||||||
|
raise FingerprintMismatch(
|
||||||
|
"declared fingerprint is inconsistent with the artifact and recipe "
|
||||||
|
"claim; the tracker recomputes compatibility digests"
|
||||||
|
)
|
||||||
|
|
||||||
|
return identity
|
||||||
|
|
||||||
|
|
||||||
|
def coverage_gap(
|
||||||
|
ranges: Iterable[tuple[int, int]], layer_count: int
|
||||||
|
) -> str | None:
|
||||||
|
"""Why `ranges` fail to tile ``[0, layer_count)``, or None when they do.
|
||||||
|
|
||||||
|
Overlaps are legal — ADR-0012 lets the Tracker resolve one by telling a hop
|
||||||
|
where the previous hop stopped. A hole is not: its layers are never computed,
|
||||||
|
and the route emits fluent tokens from a truncated model.
|
||||||
|
"""
|
||||||
|
ordered = sorted(ranges)
|
||||||
|
if not ordered:
|
||||||
|
return "the route owns no layers"
|
||||||
|
if ordered[0][0] != 0:
|
||||||
|
return f"layers 0–{ordered[0][0]} are owned by no Shard on the route"
|
||||||
|
|
||||||
|
covered = 0
|
||||||
|
for start, end in ordered:
|
||||||
|
if start > covered:
|
||||||
|
return f"layers {covered}–{start} are owned by no Shard on the route"
|
||||||
|
covered = max(covered, end)
|
||||||
|
|
||||||
|
if covered < layer_count:
|
||||||
|
return f"layers {covered}–{layer_count} are owned by no Shard on the route"
|
||||||
|
if covered > layer_count:
|
||||||
|
return (
|
||||||
|
f"the route claims {covered} layers, but the model has only {layer_count}"
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class DistributedForwardEvidence:
|
||||||
|
"""A real distributed forward — the only thing that certifies a recipe."""
|
||||||
|
|
||||||
|
route_session_id: str
|
||||||
|
route_epoch: int
|
||||||
|
node_ids: tuple[str, ...]
|
||||||
|
shard_ranges: tuple[tuple[int, int], ...]
|
||||||
|
tokens_generated: int
|
||||||
|
layer_count: int
|
||||||
|
fingerprint: tuple[str, str]
|
||||||
|
participants: tuple[PresentedIdentity, ...]
|
||||||
|
synthetic: bool = False
|
||||||
|
certified_at: float = field(default_factory=time.time)
|
||||||
|
|
||||||
|
def rejection(self) -> str | None:
|
||||||
|
if self.synthetic:
|
||||||
|
return (
|
||||||
|
"evidence comes from a synthetic worker; only a real distributed "
|
||||||
|
"forward certifies a recipe"
|
||||||
|
)
|
||||||
|
if len(self.participants) != len(self.node_ids):
|
||||||
|
return "participant identities do not match the certifying node list"
|
||||||
|
if len(self.shard_ranges) != len(self.participants):
|
||||||
|
return "participant identities do not match the recorded effective ranges"
|
||||||
|
for participant, shard_range in zip(
|
||||||
|
self.participants, self.shard_ranges, strict=True
|
||||||
|
):
|
||||||
|
if participant.key != self.fingerprint:
|
||||||
|
return "a participant fingerprint differs from the certifying route"
|
||||||
|
if participant.layer_count != self.layer_count:
|
||||||
|
return "a participant artifact layer count differs from the certifying route"
|
||||||
|
if (participant.shard_start, participant.shard_end) != shard_range:
|
||||||
|
return "a participant identity does not match its recorded effective range"
|
||||||
|
distinct = set(self.node_ids)
|
||||||
|
if len(distinct) < MIN_CERTIFYING_NODES:
|
||||||
|
return (
|
||||||
|
f"evidence covers {len(distinct)} distinct node(s); a distributed "
|
||||||
|
f"forward requires at least {MIN_CERTIFYING_NODES}"
|
||||||
|
)
|
||||||
|
if len(distinct) != len(self.node_ids):
|
||||||
|
return "the same node is counted more than once in the certifying route"
|
||||||
|
if self.tokens_generated < 1:
|
||||||
|
return "the certifying forward generated no tokens"
|
||||||
|
return coverage_gap(self.shard_ranges, self.layer_count)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RecipeStatus:
|
||||||
|
status: str
|
||||||
|
detail: str = ""
|
||||||
|
certified_at: float | None = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def may_serve(self) -> bool:
|
||||||
|
return self.status == STATUS_CERTIFIED
|
||||||
|
|
||||||
|
@property
|
||||||
|
def may_certify(self) -> bool:
|
||||||
|
"""A dark recipe may be routed to *certify* it, and for nothing else.
|
||||||
|
|
||||||
|
Without this, certification is unreachable by construction: serving needs
|
||||||
|
certification, certification needs a real distributed forward, and a real
|
||||||
|
distributed forward needs a route.
|
||||||
|
"""
|
||||||
|
return self.status in (STATUS_DARK, STATUS_CERTIFIED)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"status": self.status,
|
||||||
|
"detail": self.detail,
|
||||||
|
"certified_at": self.certified_at,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
_DARK_DETAIL = "registered; dark until a real distributed forward certifies it"
|
||||||
|
_UNKNOWN_DETAIL = "this recipe has never been registered with the tracker"
|
||||||
|
|
||||||
|
|
||||||
|
class CertificationLedger:
|
||||||
|
"""Which recipes the tracker has seen, and which a real forward has proven."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._dark: set[tuple[str, str]] = set()
|
||||||
|
self._certified: dict[tuple[str, str], RecipeStatus] = {}
|
||||||
|
|
||||||
|
def register(self, identity: PresentedIdentity) -> RecipeStatus:
|
||||||
|
key = identity.key
|
||||||
|
if key in self._certified:
|
||||||
|
return self._certified[key]
|
||||||
|
self._dark.add(key)
|
||||||
|
return RecipeStatus(STATUS_DARK, _DARK_DETAIL)
|
||||||
|
|
||||||
|
def status(self, identity: PresentedIdentity | tuple[str, str]) -> RecipeStatus:
|
||||||
|
key = identity if isinstance(identity, tuple) else identity.key
|
||||||
|
if key in self._certified:
|
||||||
|
return self._certified[key]
|
||||||
|
if key in self._dark:
|
||||||
|
return RecipeStatus(STATUS_DARK, _DARK_DETAIL)
|
||||||
|
return RecipeStatus(STATUS_UNKNOWN, _UNKNOWN_DETAIL)
|
||||||
|
|
||||||
|
def certify(
|
||||||
|
self,
|
||||||
|
identity: PresentedIdentity,
|
||||||
|
evidence: DistributedForwardEvidence,
|
||||||
|
) -> RecipeStatus:
|
||||||
|
"""Promote a recipe out of the dark, or raise saying why the evidence is short."""
|
||||||
|
key = identity.key
|
||||||
|
if key not in self._dark and key not in self._certified:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
"this fingerprint is not registered; unknown recipes cannot be certified"
|
||||||
|
)
|
||||||
|
if evidence.fingerprint != key:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
"certification evidence fingerprint does not match the recipe being promoted"
|
||||||
|
)
|
||||||
|
if evidence.layer_count != identity.layer_count:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
"certification evidence layer count does not match the artifact being promoted"
|
||||||
|
)
|
||||||
|
rejection = evidence.rejection()
|
||||||
|
if rejection is not None:
|
||||||
|
raise RecipeIdentityError(f"this evidence does not certify: {rejection}")
|
||||||
|
status = RecipeStatus(
|
||||||
|
STATUS_CERTIFIED,
|
||||||
|
(
|
||||||
|
f"certified by route session {evidence.route_session_id} across "
|
||||||
|
f"{len(set(evidence.node_ids))} nodes"
|
||||||
|
),
|
||||||
|
certified_at=evidence.certified_at,
|
||||||
|
)
|
||||||
|
self._certified[key] = status
|
||||||
|
self._dark.discard(key)
|
||||||
|
return status
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"dark": [list(key) for key in sorted(self._dark)],
|
||||||
|
"certified": {
|
||||||
|
"/".join(key): status.to_dict()
|
||||||
|
for key, status in sorted(self._certified.items())
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# Route formation itself lives in `server.py` (`_select_route`, `_enumerate_routes`,
|
||||||
|
# `_find_pinned_route`): candidates are partitioned by the exact fingerprint this
|
||||||
|
# module derives, so a route can only ever be assembled inside one identity. A
|
||||||
|
# second route gate here would be a second copy of that policy to keep in step —
|
||||||
|
# the same reason the node module holds no certification authority.
|
||||||
@@ -26,7 +26,7 @@ import random
|
|||||||
import sqlite3
|
import sqlite3
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from typing import Any, Iterable
|
from typing import Any, Iterable
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ import urllib.parse
|
|||||||
import urllib.request
|
import urllib.request
|
||||||
import uuid
|
import uuid
|
||||||
from collections import deque
|
from collections import deque
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field, replace
|
||||||
from importlib.resources import files
|
from importlib.resources import files
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
@@ -53,13 +53,10 @@ from typing import Any
|
|||||||
from .accounts import DEFAULT_ACCOUNTS_DB_PATH, AccountStore
|
from .accounts import DEFAULT_ACCOUNTS_DB_PATH, AccountStore
|
||||||
from .auth import is_validator_token, sign_hive_request, verify_hive_request
|
from .auth import is_validator_token, sign_hive_request, verify_hive_request
|
||||||
from .capability import (
|
from .capability import (
|
||||||
DEFAULT_POLICY as DEFAULT_CAPABILITY_POLICY,
|
|
||||||
POLICY_COMPAT,
|
|
||||||
POLICY_ENFORCE,
|
|
||||||
STATE_ABSENT,
|
|
||||||
STATE_ADMITTED,
|
STATE_ADMITTED,
|
||||||
STATE_MODEL_MISMATCH,
|
STATE_MODEL_MISMATCH,
|
||||||
STATE_SHARD_MISMATCH,
|
STATE_SHARD_MISMATCH,
|
||||||
|
STATE_UNCERTIFIED,
|
||||||
CapabilityState,
|
CapabilityState,
|
||||||
absent_state,
|
absent_state,
|
||||||
evaluate_report,
|
evaluate_report,
|
||||||
@@ -68,7 +65,7 @@ from .capability import (
|
|||||||
)
|
)
|
||||||
from .wallet_proof import binding_message, verify_wallet_signature
|
from .wallet_proof import binding_message, verify_wallet_signature
|
||||||
from .billing import DEFAULT_BILLING_DB_PATH, BillingLedger
|
from .billing import DEFAULT_BILLING_DB_PATH, BillingLedger
|
||||||
from .calibration import DEFAULT_CALIBRATION_DB_PATH, ToplocCalibrationStore
|
from .calibration import ToplocCalibrationStore
|
||||||
from .hf_pricing import DEFAULT_HF_PRICING_LOG_DB_PATH, HfPricingLog, refresh_preset_price
|
from .hf_pricing import DEFAULT_HF_PRICING_LOG_DB_PATH, HfPricingLog, refresh_preset_price
|
||||||
from .gossip import NodeGossip
|
from .gossip import NodeGossip
|
||||||
from .logging_setup import tracker_logger
|
from .logging_setup import tracker_logger
|
||||||
@@ -84,6 +81,13 @@ from .routing_stats import (
|
|||||||
)
|
)
|
||||||
from .model_files import files_for_layer_range, snapshot_dir_for_repo
|
from .model_files import files_for_layer_range, snapshot_dir_for_repo
|
||||||
from .raft import RaftNode
|
from .raft import RaftNode
|
||||||
|
from .recipe import (
|
||||||
|
CertificationLedger,
|
||||||
|
DistributedForwardEvidence,
|
||||||
|
PresentedIdentity,
|
||||||
|
RecipeIdentityError,
|
||||||
|
RecipeStatus,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
_CONSOLE_LIMIT = 300
|
_CONSOLE_LIMIT = 300
|
||||||
@@ -792,6 +796,7 @@ def _capability_from_registration(
|
|||||||
hf_repo: str | None,
|
hf_repo: str | None,
|
||||||
shard_start: int | None,
|
shard_start: int | None,
|
||||||
shard_end: int | None,
|
shard_end: int | None,
|
||||||
|
recipe_certifications: CertificationLedger,
|
||||||
) -> CapabilityState:
|
) -> CapabilityState:
|
||||||
"""The tracker's verdict on the proof carried by one registration payload.
|
"""The tracker's verdict on the proof carried by one registration payload.
|
||||||
|
|
||||||
@@ -811,6 +816,7 @@ def _capability_from_registration(
|
|||||||
declared_recipe_version=(
|
declared_recipe_version=(
|
||||||
recipe_version if isinstance(recipe_version, str) else None
|
recipe_version if isinstance(recipe_version, str) else None
|
||||||
),
|
),
|
||||||
|
ledger=recipe_certifications,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -830,6 +836,11 @@ def _admitted_nodes(nodes: list["_NodeEntry"], policy: str | None) -> list["_Nod
|
|||||||
return [node for node in nodes if _capability_routable(node, effective)]
|
return [node for node in nodes if _capability_routable(node, effective)]
|
||||||
|
|
||||||
|
|
||||||
|
def _route_identity_partition(node: "_NodeEntry") -> tuple[str, ...]:
|
||||||
|
fingerprint = _node_admission(node).fingerprint
|
||||||
|
return ("legacy",) if fingerprint is None else ("exact", *fingerprint)
|
||||||
|
|
||||||
|
|
||||||
def _select_route(
|
def _select_route(
|
||||||
nodes: list[_NodeEntry],
|
nodes: list[_NodeEntry],
|
||||||
required_start: int,
|
required_start: int,
|
||||||
@@ -853,29 +864,51 @@ def _select_route(
|
|||||||
],
|
],
|
||||||
key=lambda n: (n.shard_start, -n.shard_end), # type: ignore[operator]
|
key=lambda n: (n.shard_start, -n.shard_end), # type: ignore[operator]
|
||||||
)
|
)
|
||||||
route: list[_NodeEntry] = []
|
|
||||||
covered_up_to = required_start - 1
|
|
||||||
|
|
||||||
def _routing_score(node: "_NodeEntry") -> float:
|
def _routing_score(node: "_NodeEntry") -> float:
|
||||||
return _effective_throughput(node, model) * _reputation_multiplier(node, contracts)
|
return _effective_throughput(node, model) * _reputation_multiplier(node, contracts)
|
||||||
|
|
||||||
|
partitions: dict[tuple[str, ...], list[_NodeEntry]] = {}
|
||||||
|
for node in candidates:
|
||||||
|
partitions.setdefault(_route_identity_partition(node), []).append(node)
|
||||||
|
|
||||||
|
complete: list[list[_NodeEntry]] = []
|
||||||
|
furthest = required_start - 1
|
||||||
|
for partition in partitions.values():
|
||||||
|
pool = list(partition)
|
||||||
|
route: list[_NodeEntry] = []
|
||||||
|
covered_up_to = required_start - 1
|
||||||
while covered_up_to < required_end:
|
while covered_up_to < required_end:
|
||||||
best: _NodeEntry | None = None
|
best: _NodeEntry | None = None
|
||||||
for node in candidates:
|
for node in pool:
|
||||||
if node.shard_start <= covered_up_to + 1 and node.shard_end > covered_up_to:
|
if node.shard_start <= covered_up_to + 1 and node.shard_end > covered_up_to:
|
||||||
if best is None:
|
if best is None:
|
||||||
best = node
|
best = node
|
||||||
elif node.shard_end > best.shard_end:
|
elif node.shard_end > best.shard_end:
|
||||||
best = node
|
best = node
|
||||||
elif node.shard_end == best.shard_end and _routing_score(node) > _routing_score(best):
|
elif (
|
||||||
|
node.shard_end == best.shard_end
|
||||||
|
and _routing_score(node) > _routing_score(best)
|
||||||
|
):
|
||||||
best = node
|
best = node
|
||||||
if best is None:
|
if best is None:
|
||||||
missing = covered_up_to + 1
|
break
|
||||||
return [], f"no route available: no registered node covers layer {missing}"
|
|
||||||
route.append(best)
|
route.append(best)
|
||||||
covered_up_to = best.shard_end
|
covered_up_to = best.shard_end
|
||||||
candidates = [n for n in candidates if n is not best]
|
pool = [node for node in pool if node is not best]
|
||||||
|
furthest = max(furthest, covered_up_to)
|
||||||
|
if covered_up_to >= required_end:
|
||||||
|
complete.append(route)
|
||||||
|
|
||||||
|
if not complete:
|
||||||
|
return [], f"no route available: no registered node covers layer {furthest + 1}"
|
||||||
|
route = max(
|
||||||
|
complete,
|
||||||
|
key=lambda candidate: (
|
||||||
|
min(_routing_score(node) for node in candidate),
|
||||||
|
-len(candidate),
|
||||||
|
),
|
||||||
|
)
|
||||||
return route, ""
|
return route, ""
|
||||||
|
|
||||||
|
|
||||||
@@ -911,7 +944,12 @@ def _enumerate_routes(
|
|||||||
for head in heads:
|
for head in heads:
|
||||||
route = [head]
|
route = [head]
|
||||||
covered_up_to = head.shard_end
|
covered_up_to = head.shard_end
|
||||||
pool = [n for n in sharded if n is not head]
|
head_partition = _route_identity_partition(head)
|
||||||
|
pool = [
|
||||||
|
node
|
||||||
|
for node in sharded
|
||||||
|
if node is not head and _route_identity_partition(node) == head_partition
|
||||||
|
]
|
||||||
while covered_up_to < required_end:
|
while covered_up_to < required_end:
|
||||||
best = None
|
best = None
|
||||||
for n in pool:
|
for n in pool:
|
||||||
@@ -2056,14 +2094,24 @@ def _find_pinned_route(
|
|||||||
hop_count: int,
|
hop_count: int,
|
||||||
) -> list[_NodeEntry] | None:
|
) -> list[_NodeEntry] | None:
|
||||||
"""First combination of exactly ``hop_count`` distinct nodes covering the
|
"""First combination of exactly ``hop_count`` distinct nodes covering the
|
||||||
layer range, where every node extends coverage (US-030 benchmark routes)."""
|
layer range, where every node extends coverage (US-030 benchmark routes).
|
||||||
|
|
||||||
|
Benchmark combos run real inference, so they obey the same DGR-003 rule as
|
||||||
|
every other route builder: one route, one exact identity. A combo that mixed
|
||||||
|
two fingerprints — or an exact Shard with a legacy one — would measure a
|
||||||
|
numerically incoherent route and record the garbage as a benchmark.
|
||||||
|
"""
|
||||||
for combo in itertools.permutations(nodes, hop_count):
|
for combo in itertools.permutations(nodes, hop_count):
|
||||||
covered = required_start - 1
|
covered = required_start - 1
|
||||||
valid = True
|
valid = True
|
||||||
|
partition = _route_identity_partition(combo[0])
|
||||||
for candidate in combo:
|
for candidate in combo:
|
||||||
if candidate.shard_start is None or candidate.shard_end is None:
|
if candidate.shard_start is None or candidate.shard_end is None:
|
||||||
valid = False
|
valid = False
|
||||||
break
|
break
|
||||||
|
if _route_identity_partition(candidate) != partition:
|
||||||
|
valid = False
|
||||||
|
break
|
||||||
if candidate.shard_start > covered + 1 or candidate.shard_end <= covered:
|
if candidate.shard_start > covered + 1 or candidate.shard_end <= covered:
|
||||||
valid = False
|
valid = False
|
||||||
break
|
break
|
||||||
@@ -2514,8 +2562,8 @@ def _estimate_prompt_tokens(body: dict) -> int | None:
|
|||||||
|
|
||||||
|
|
||||||
def _requested_completion_token_limit(body: dict) -> int | None:
|
def _requested_completion_token_limit(body: dict) -> int | None:
|
||||||
for field in ("max_completion_tokens", "max_tokens"):
|
for key in ("max_completion_tokens", "max_tokens"):
|
||||||
value = body.get(field)
|
value = body.get(key)
|
||||||
if isinstance(value, bool):
|
if isinstance(value, bool):
|
||||||
return None
|
return None
|
||||||
if isinstance(value, (int, float)):
|
if isinstance(value, (int, float)):
|
||||||
@@ -2847,9 +2895,11 @@ class _TrackerHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
|||||||
relay_status: dict | None = None,
|
relay_status: dict | None = None,
|
||||||
test_runner: "TestRunManager | None" = None,
|
test_runner: "TestRunManager | None" = None,
|
||||||
capability_policy: str | None = None,
|
capability_policy: str | None = None,
|
||||||
|
recipe_certifications: CertificationLedger | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
super().__init__(addr, handler)
|
super().__init__(addr, handler)
|
||||||
self.registry = registry
|
self.registry = registry
|
||||||
|
self.recipe_certifications = recipe_certifications or CertificationLedger()
|
||||||
self.capability_policy = normalize_policy(
|
self.capability_policy = normalize_policy(
|
||||||
capability_policy if capability_policy is not None else policy_from_env()
|
capability_policy if capability_policy is not None else policy_from_env()
|
||||||
)
|
)
|
||||||
@@ -2894,7 +2944,7 @@ class _TrackerHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
|||||||
|
|
||||||
|
|
||||||
class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def _send_json(self, status: int, data: dict, headers: dict[str, str] | None = None) -> None:
|
def _send_json(self, status: int, data: dict, headers: dict[str, str] | None = None) -> None:
|
||||||
@@ -4555,6 +4605,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
hf_repo=hf_repo,
|
hf_repo=hf_repo,
|
||||||
shard_start=shard_start,
|
shard_start=shard_start,
|
||||||
shard_end=shard_end,
|
shard_end=shard_end,
|
||||||
|
recipe_certifications=server.recipe_certifications,
|
||||||
)
|
)
|
||||||
|
|
||||||
node_id = _node_id_for_registration(
|
node_id = _node_id_for_registration(
|
||||||
@@ -6531,6 +6582,7 @@ class TrackerServer:
|
|||||||
self._embedded_relay: Any | None = None
|
self._embedded_relay: Any | None = None
|
||||||
self._embedded_relay_actual_port: int | None = None
|
self._embedded_relay_actual_port: int | None = None
|
||||||
self._registry: dict[str, _NodeEntry] = {}
|
self._registry: dict[str, _NodeEntry] = {}
|
||||||
|
self._recipe_certifications = CertificationLedger()
|
||||||
self._lock = threading.Lock()
|
self._lock = threading.Lock()
|
||||||
self._server: _TrackerHTTPServer | None = None
|
self._server: _TrackerHTTPServer | None = None
|
||||||
self._thread: threading.Thread | None = None
|
self._thread: threading.Thread | None = None
|
||||||
@@ -6640,6 +6692,84 @@ class TrackerServer:
|
|||||||
self._test_runner: TestRunManager | None = test_runner
|
self._test_runner: TestRunManager | None = test_runner
|
||||||
self.port: int | None = None
|
self.port: int | None = None
|
||||||
|
|
||||||
|
def certify_recipe(
|
||||||
|
self,
|
||||||
|
identity: PresentedIdentity,
|
||||||
|
evidence: DistributedForwardEvidence,
|
||||||
|
) -> RecipeStatus:
|
||||||
|
"""Promote one exact recipe from tracker-recorded distributed evidence.
|
||||||
|
|
||||||
|
This is the tracker's only certification authority. The evidence names
|
||||||
|
the nodes that served the forward; this method refuses to take any of
|
||||||
|
them on the evidence's word. For each one it goes back to what *this
|
||||||
|
tracker* re-derived at admission and requires the evidence to describe
|
||||||
|
the same Shard: same route fingerprint, same registered range, and the
|
||||||
|
same shard binding digest — the derivative's own bytes and exact range.
|
||||||
|
|
||||||
|
Without the binding check, a route fingerprint is range-independent by
|
||||||
|
design (:mod:`meshnet_tracker.recipe`), so evidence could name the right
|
||||||
|
recipe while describing splits nobody on the route was actually serving,
|
||||||
|
and the certification would attach to blobs that never ran.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
for node_id, participant in zip(
|
||||||
|
evidence.node_ids, evidence.participants, strict=True
|
||||||
|
):
|
||||||
|
node = self._registry.get(node_id)
|
||||||
|
if node is None:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"certification participant {node_id!r} is not registered"
|
||||||
|
)
|
||||||
|
admitted = node.capability
|
||||||
|
if admitted.fingerprint != identity.key:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"certification participant {node_id!r} is not admitted under "
|
||||||
|
"the promoted fingerprint"
|
||||||
|
)
|
||||||
|
if admitted.state not in (STATE_UNCERTIFIED, STATE_ADMITTED):
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"certification participant {node_id!r} has capability state "
|
||||||
|
f"{admitted.state!r}"
|
||||||
|
)
|
||||||
|
registered_range = (
|
||||||
|
node.shard_start,
|
||||||
|
None if node.shard_end is None else node.shard_end + 1,
|
||||||
|
)
|
||||||
|
if registered_range != (
|
||||||
|
participant.shard_start,
|
||||||
|
participant.shard_end,
|
||||||
|
):
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"certification participant {node_id!r} range differs from its "
|
||||||
|
"registered Shard range"
|
||||||
|
)
|
||||||
|
# The exact bytes and range this node was admitted on. A node that
|
||||||
|
# registered without an identity has no binding, and cannot be a
|
||||||
|
# participant in an exact certification at all.
|
||||||
|
if admitted.shard_binding_digest is None:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"certification participant {node_id!r} registered no exact "
|
||||||
|
"Shard binding; it cannot certify a recipe"
|
||||||
|
)
|
||||||
|
if admitted.shard_binding_digest != participant.shard_binding_digest:
|
||||||
|
raise RecipeIdentityError(
|
||||||
|
f"certification participant {node_id!r} presents a Shard "
|
||||||
|
"binding this tracker did not admit it on; the evidence "
|
||||||
|
"describes different derivative bytes or a different range"
|
||||||
|
)
|
||||||
|
|
||||||
|
status = self._recipe_certifications.certify(identity, evidence)
|
||||||
|
for node in self._registry.values():
|
||||||
|
if (
|
||||||
|
node.capability.state == STATE_UNCERTIFIED
|
||||||
|
and node.capability.fingerprint == identity.key
|
||||||
|
):
|
||||||
|
node.capability = replace(
|
||||||
|
node.capability.with_state(STATE_ADMITTED, status.detail),
|
||||||
|
certification=status.status,
|
||||||
|
)
|
||||||
|
return status
|
||||||
|
|
||||||
def _start_embedded_relay(self) -> dict:
|
def _start_embedded_relay(self) -> dict:
|
||||||
"""Start the shared RelayServer class in-process for tracker+relay deployments."""
|
"""Start the shared RelayServer class in-process for tracker+relay deployments."""
|
||||||
if not self._embedded_relay_enabled:
|
if not self._embedded_relay_enabled:
|
||||||
@@ -6725,6 +6855,7 @@ class TrackerServer:
|
|||||||
relay_status=http_relay_status,
|
relay_status=http_relay_status,
|
||||||
test_runner=self._test_runner,
|
test_runner=self._test_runner,
|
||||||
capability_policy=self._capability_policy,
|
capability_policy=self._capability_policy,
|
||||||
|
recipe_certifications=self._recipe_certifications,
|
||||||
)
|
)
|
||||||
self.port = self._server.server_address[1]
|
self.port = self._server.server_address[1]
|
||||||
|
|
||||||
@@ -6934,10 +7065,6 @@ class TrackerServer:
|
|||||||
shard_end = int(payload["shard_end"]) if payload.get("shard_end") is not None else None
|
shard_end = int(payload["shard_end"]) if payload.get("shard_end") is not None else None
|
||||||
except (TypeError, ValueError):
|
except (TypeError, ValueError):
|
||||||
return
|
return
|
||||||
try:
|
|
||||||
friendly_name = _normalize_friendly_name(payload.get("friendly_name"))
|
|
||||||
except ValueError:
|
|
||||||
friendly_name = None
|
|
||||||
# The replicated payload is the raw registration body, so the follower can
|
# The replicated payload is the raw registration body, so the follower can
|
||||||
# resolve precision exactly as the leader did -- including telling a legacy
|
# resolve precision exactly as the leader did -- including telling a legacy
|
||||||
# absent `quantization` from a declared one. Dropping these fields here
|
# absent `quantization` from a declared one. Dropping these fields here
|
||||||
@@ -6983,6 +7110,7 @@ class TrackerServer:
|
|||||||
hf_repo=payload.get("hf_repo"),
|
hf_repo=payload.get("hf_repo"),
|
||||||
shard_start=shard_start,
|
shard_start=shard_start,
|
||||||
shard_end=shard_end,
|
shard_end=shard_end,
|
||||||
|
recipe_certifications=self._recipe_certifications,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
|
|||||||
@@ -545,6 +545,8 @@ def _post_json(url: str, payload: dict, timeout: float = 5.0) -> dict:
|
|||||||
__all__ = [
|
__all__ = [
|
||||||
"ToplocAuditConfig",
|
"ToplocAuditConfig",
|
||||||
"ToplocProofClaim",
|
"ToplocProofClaim",
|
||||||
|
"ToplocVerificationResult",
|
||||||
|
"verify_activation_proofs_detailed",
|
||||||
"ValidatorProcess",
|
"ValidatorProcess",
|
||||||
"AdaptiveAuditSampler",
|
"AdaptiveAuditSampler",
|
||||||
"AuditRateConfig",
|
"AuditRateConfig",
|
||||||
|
|||||||
@@ -14,6 +14,11 @@ dev = ["pytest>=8", "openai>=1", "langchain-openai>=0.1", "cryptography>=41"]
|
|||||||
[tool.setuptools]
|
[tool.setuptools]
|
||||||
packages = []
|
packages = []
|
||||||
|
|
||||||
|
[tool.ruff]
|
||||||
|
# Protobuf/gRPC stubs are regenerated by scripts/generate_native_protocol.py;
|
||||||
|
# linting them would drift the checked-in files from the generator's output.
|
||||||
|
extend-exclude = ["packages/node/meshnet_node/native_protocol/generated"]
|
||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
testpaths = ["tests"]
|
testpaths = ["tests"]
|
||||||
markers = [
|
markers = [
|
||||||
|
|||||||
153
scripts/gen_recipe_fingerprint_vectors.py
Normal file
153
scripts/gen_recipe_fingerprint_vectors.py
Normal file
@@ -0,0 +1,153 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
"""Generate (or verify) the committed DGR-003 fingerprint conformance vectors.
|
||||||
|
|
||||||
|
The node and the tracker derive artifact, recipe and Shard-binding digests from
|
||||||
|
*separate* implementations on purpose: an admission gate that shares code with
|
||||||
|
the thing it admits is not an independent check. The cost of that independence
|
||||||
|
is drift — two canonicalizers that quietly stop agreeing would not fail, they
|
||||||
|
would silently stop forming routes, or worse, silently form wrong ones.
|
||||||
|
|
||||||
|
``tests/data/recipe_fingerprint_vectors.json`` is what makes drift loud. It is a
|
||||||
|
language-neutral artifact — canonical input blocks, expected digests, and the
|
||||||
|
serialized DGR-002 ``Fingerprint`` bytes — that the node tests, the tracker tests
|
||||||
|
and (later) the native C++ worker all check themselves against.
|
||||||
|
|
||||||
|
python scripts/gen_recipe_fingerprint_vectors.py --check # CI: no drift
|
||||||
|
python scripts/gen_recipe_fingerprint_vectors.py # rewrite vectors
|
||||||
|
|
||||||
|
Rewriting is a deliberate act: if this changes a digest, it changed the wire
|
||||||
|
contract, and every node and tracker in the fleet has to agree at the same time.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import pathlib
|
||||||
|
import sys
|
||||||
|
|
||||||
|
_ROOT = pathlib.Path(__file__).resolve().parent.parent
|
||||||
|
sys.path[:0] = [str(_ROOT / "packages" / "node"), str(_ROOT / "packages" / "tracker")]
|
||||||
|
|
||||||
|
from meshnet_node.runtime_recipe import ( # noqa: E402
|
||||||
|
ArtifactIdentity,
|
||||||
|
DerivativeBinding,
|
||||||
|
RuntimeRecipe,
|
||||||
|
ShardIdentity,
|
||||||
|
)
|
||||||
|
from meshnet_tracker.recipe import parse_identity # noqa: E402
|
||||||
|
|
||||||
|
VECTORS = _ROOT / "tests" / "data" / "recipe_fingerprint_vectors.json"
|
||||||
|
SCHEMA_VERSION = 1
|
||||||
|
|
||||||
|
_RECIPE = RuntimeRecipe(
|
||||||
|
weight_quantization="Q4_K_M",
|
||||||
|
activation_dtype="bfloat16",
|
||||||
|
compute_dtype="float32",
|
||||||
|
kv_dtype="q8_0",
|
||||||
|
kv_layout="paged-v1",
|
||||||
|
tokenizer_revision="0123456789abcdef",
|
||||||
|
architecture_adapter="llama/range-v1",
|
||||||
|
backend_id="llama.cpp",
|
||||||
|
runtime_version="llama.cpp@deadbeef+meshnet.1",
|
||||||
|
recipe_id="example-gguf",
|
||||||
|
recipe_version="1",
|
||||||
|
catalogue_version="2026.07.1",
|
||||||
|
)
|
||||||
|
|
||||||
|
_SOURCE = "a" * 64
|
||||||
|
_SPLIT_BYTES = "c" * 64
|
||||||
|
_CONFIG = "b" * 64
|
||||||
|
|
||||||
|
|
||||||
|
def _cases() -> list[tuple[str, str, ShardIdentity]]:
|
||||||
|
whole = ShardIdentity(
|
||||||
|
ArtifactIdentity(
|
||||||
|
"example/model", "0123456789abcdef", _SOURCE, "dense-llama", _CONFIG, 8
|
||||||
|
),
|
||||||
|
_RECIPE,
|
||||||
|
0,
|
||||||
|
4,
|
||||||
|
)
|
||||||
|
# The same recipe on the same source, held as a split: identical route
|
||||||
|
# fingerprint, different Shard binding. Both halves of that are contract.
|
||||||
|
derivative = ShardIdentity(
|
||||||
|
ArtifactIdentity(
|
||||||
|
"example/model",
|
||||||
|
"0123456789abcdef",
|
||||||
|
_SPLIT_BYTES,
|
||||||
|
"dense-llama",
|
||||||
|
_CONFIG,
|
||||||
|
8,
|
||||||
|
DerivativeBinding(_SOURCE, 4, 8),
|
||||||
|
),
|
||||||
|
_RECIPE,
|
||||||
|
4,
|
||||||
|
8,
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
("example-v1", "An undivided artifact: content digest is the source digest.", whole),
|
||||||
|
(
|
||||||
|
"example-v1-derivative",
|
||||||
|
"A split of the same source: same fingerprint, different Shard binding.",
|
||||||
|
derivative,
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def build() -> dict:
|
||||||
|
vectors = []
|
||||||
|
for name, description, identity in _cases():
|
||||||
|
block = identity.to_dict()
|
||||||
|
presented = parse_identity(block)
|
||||||
|
|
||||||
|
# The two implementations must already agree before this is committed.
|
||||||
|
assert identity.fingerprint.to_dict() == presented.fingerprint_dict(), name
|
||||||
|
assert identity.shard_binding_digest == presented.shard_binding_digest, name
|
||||||
|
|
||||||
|
vectors.append(
|
||||||
|
{
|
||||||
|
"name": name,
|
||||||
|
"description": description,
|
||||||
|
"identity": block,
|
||||||
|
"fingerprint": identity.fingerprint.to_dict(),
|
||||||
|
"shard_binding_digest": identity.shard_binding_digest,
|
||||||
|
"fingerprint_proto_hex": identity.fingerprint.to_proto()
|
||||||
|
.SerializeToString(deterministic=True)
|
||||||
|
.hex(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {"schema_version": SCHEMA_VERSION, "vectors": vectors}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument(
|
||||||
|
"--check",
|
||||||
|
action="store_true",
|
||||||
|
help="fail if the committed vectors differ from what this code derives",
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
built = json.dumps(build(), indent=2, sort_keys=True) + "\n"
|
||||||
|
if not args.check:
|
||||||
|
VECTORS.write_text(built, encoding="utf-8")
|
||||||
|
print(f"wrote {VECTORS.relative_to(_ROOT)}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
committed = VECTORS.read_text(encoding="utf-8")
|
||||||
|
if committed != built:
|
||||||
|
print(
|
||||||
|
f"{VECTORS.relative_to(_ROOT)} is stale: the identity implementation no "
|
||||||
|
"longer derives the committed digests.\nIf that change was intended, it "
|
||||||
|
"is a wire-contract change — rerun without --check and roll out node and "
|
||||||
|
"tracker together.",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
print(f"{VECTORS.relative_to(_ROOT)} matches the identity implementation")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -33,6 +33,9 @@ def _vectors() -> dict[str, bytes]:
|
|||||||
conformance.GOLDEN_CAPABILITY_REPORT: conformance.serialize(
|
conformance.GOLDEN_CAPABILITY_REPORT: conformance.serialize(
|
||||||
conformance.canonical_capability_report()
|
conformance.canonical_capability_report()
|
||||||
),
|
),
|
||||||
|
conformance.GOLDEN_DECODE_STEP: conformance.serialize(
|
||||||
|
conformance.canonical_decode_step()
|
||||||
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
280
scripts/glm_whole_model_preflight.py
Normal file
280
scripts/glm_whole_model_preflight.py
Normal file
@@ -0,0 +1,280 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""DGR-018 whole-model preflight: storage, memory, and the ordered download plan.
|
||||||
|
|
||||||
|
Answers, before a single artifact byte moves, the three questions DGR-018's
|
||||||
|
finish contract asks:
|
||||||
|
|
||||||
|
1. Is there one filesystem, outside every forbidden prefix (``/home``), with at
|
||||||
|
least 250 GB free for the 216.715 GB ``UD-IQ1_S`` artifact plus headroom?
|
||||||
|
2. Does this host have the 224 GiB runtime-accessible memory the whole-model
|
||||||
|
oracle load needs (the experimental hard-fit floor from DGR-017)?
|
||||||
|
3. In what order do the six shards download, and against which exact revision
|
||||||
|
URL, size, and LFS SHA-256 is each one verified?
|
||||||
|
|
||||||
|
Everything is resolved from the pinned target manifest
|
||||||
|
(``meshnet_node.glm_alpha``) — this script never contacts the network and never
|
||||||
|
invents a size or digest. It fails closed: a missing requirement is a non-zero
|
||||||
|
exit and an explicit reason, never a downgrade to a smaller target.
|
||||||
|
|
||||||
|
Usage::
|
||||||
|
|
||||||
|
python scripts/glm_whole_model_preflight.py # scan all mounts
|
||||||
|
python scripts/glm_whole_model_preflight.py --dest DIR # judge one destination
|
||||||
|
python scripts/glm_whole_model_preflight.py --storage-only # download-host mode
|
||||||
|
python scripts/glm_whole_model_preflight.py --json out.json
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
sys.path.insert(0, str(_ROOT / "packages/node"))
|
||||||
|
|
||||||
|
from meshnet_node.glm_alpha.manifest import ( # noqa: E402
|
||||||
|
GIB,
|
||||||
|
TargetManifest,
|
||||||
|
load_target_manifest,
|
||||||
|
)
|
||||||
|
|
||||||
|
GB = 1000**3
|
||||||
|
|
||||||
|
# 250 GB: the 216.715 GB artifact plus resume/temp headroom. DGR-018's issue
|
||||||
|
# text names this number; it is a storage requirement, not a tunable.
|
||||||
|
REQUIRED_FREE_BYTES = 250 * GB
|
||||||
|
|
||||||
|
# DGR-017's experimental hard-fit floor for whole-model runtime-accessible
|
||||||
|
# memory. 224 GiB fits weights + Q8_0 KV at 16k context with nothing to spare.
|
||||||
|
REQUIRED_MEMORY_GIB = 224.0
|
||||||
|
|
||||||
|
# Download order, by shard index. Rationale:
|
||||||
|
# 1 first — 9.4 MB split-metadata shard: proves revision URLs, auth-free
|
||||||
|
# access, and tooling end-to-end for the cost of a rounding error.
|
||||||
|
# 6 second — 19.2 GB, the smallest weight shard: proves resume + hash
|
||||||
|
# verification at real scale before any ~49 GB transfer starts.
|
||||||
|
# 2..5 — the four ~49 GB shards, sequentially, verify-after-each, so at
|
||||||
|
# most one unverified partial ever exists on disk.
|
||||||
|
DOWNLOAD_ORDER = (1, 6, 2, 3, 4, 5)
|
||||||
|
|
||||||
|
# Filesystems that can hold a 49 GB file and survive a resume. FAT variants are
|
||||||
|
# excluded by omission (4 GiB file limit); network/pseudo filesystems likewise.
|
||||||
|
_USABLE_FSTYPES = {"ext4", "ext3", "xfs", "btrfs", "ntfs", "ntfs3", "fuseblk", "exfat", "f2fs"}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class MountCandidate:
|
||||||
|
mountpoint: str
|
||||||
|
fstype: str
|
||||||
|
total_bytes: int
|
||||||
|
free_bytes: int
|
||||||
|
forbidden: bool
|
||||||
|
|
||||||
|
@property
|
||||||
|
def eligible(self) -> bool:
|
||||||
|
return not self.forbidden and self.free_bytes >= REQUIRED_FREE_BYTES
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"mountpoint": self.mountpoint,
|
||||||
|
"fstype": self.fstype,
|
||||||
|
"total_gb": round(self.total_bytes / GB, 1),
|
||||||
|
"free_gb": round(self.free_bytes / GB, 1),
|
||||||
|
"free_bytes": self.free_bytes,
|
||||||
|
"forbidden": self.forbidden,
|
||||||
|
"eligible": self.eligible,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def forbidden_prefixes(manifest: TargetManifest) -> tuple[str, ...]:
|
||||||
|
"""The storage policy pinned in the manifest itself — not a script opinion."""
|
||||||
|
storage = manifest.raw.get("storage", {})
|
||||||
|
return tuple(storage.get("forbidden_path_prefixes", ("/home",)))
|
||||||
|
|
||||||
|
|
||||||
|
def is_forbidden(path: str, prefixes: tuple[str, ...]) -> bool:
|
||||||
|
resolved = os.path.realpath(path)
|
||||||
|
return any(resolved == p or resolved.startswith(p.rstrip("/") + "/") for p in prefixes)
|
||||||
|
|
||||||
|
|
||||||
|
def _statvfs_free(path: str) -> tuple[int, int]:
|
||||||
|
st = os.statvfs(path)
|
||||||
|
return st.f_frsize * st.f_blocks, st.f_frsize * st.f_bavail
|
||||||
|
|
||||||
|
|
||||||
|
def scan_mounts(
|
||||||
|
prefixes: tuple[str, ...], mounts_file: str = "/proc/mounts"
|
||||||
|
) -> list[MountCandidate]:
|
||||||
|
"""Every real, writable-class filesystem on the host, deduplicated by device."""
|
||||||
|
candidates: dict[str, MountCandidate] = {}
|
||||||
|
with open(mounts_file, encoding="utf-8") as handle:
|
||||||
|
for line in handle:
|
||||||
|
fields = line.split()
|
||||||
|
if len(fields) < 3:
|
||||||
|
continue
|
||||||
|
device, mountpoint, fstype = fields[0], fields[1], fields[2]
|
||||||
|
mountpoint = mountpoint.replace("\\040", " ")
|
||||||
|
if fstype not in _USABLE_FSTYPES or device in candidates:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
total, free = _statvfs_free(mountpoint)
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
candidates[device] = MountCandidate(
|
||||||
|
mountpoint=mountpoint,
|
||||||
|
fstype=fstype,
|
||||||
|
total_bytes=total,
|
||||||
|
free_bytes=free,
|
||||||
|
forbidden=is_forbidden(mountpoint, prefixes),
|
||||||
|
)
|
||||||
|
return sorted(candidates.values(), key=lambda c: c.free_bytes, reverse=True)
|
||||||
|
|
||||||
|
|
||||||
|
def judge_destination(dest: str, prefixes: tuple[str, ...]) -> MountCandidate:
|
||||||
|
total, free = _statvfs_free(dest)
|
||||||
|
return MountCandidate(
|
||||||
|
mountpoint=dest,
|
||||||
|
fstype="(as-given)",
|
||||||
|
total_bytes=total,
|
||||||
|
free_bytes=free,
|
||||||
|
forbidden=is_forbidden(dest, prefixes),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def memory_total_gib(meminfo_file: str = "/proc/meminfo") -> float:
|
||||||
|
with open(meminfo_file, encoding="utf-8") as handle:
|
||||||
|
for line in handle:
|
||||||
|
if line.startswith("MemTotal:"):
|
||||||
|
return int(line.split()[1]) * 1024 / GIB
|
||||||
|
raise RuntimeError(f"MemTotal not found in {meminfo_file}")
|
||||||
|
|
||||||
|
|
||||||
|
def download_plan(manifest: TargetManifest, dest_dir: str | None) -> list[dict]:
|
||||||
|
"""The six shards in download order, each bound to its pinned identity."""
|
||||||
|
steps = []
|
||||||
|
for position, index in enumerate(DOWNLOAD_ORDER, start=1):
|
||||||
|
shard = manifest.shard(index)
|
||||||
|
target = f"{dest_dir or '$GLM_DEST'}/{shard.path}"
|
||||||
|
steps.append(
|
||||||
|
{
|
||||||
|
"step": position,
|
||||||
|
"shard_index": shard.index,
|
||||||
|
"path": shard.path,
|
||||||
|
"size_bytes": shard.size_bytes,
|
||||||
|
"size_gb": round(shard.size_bytes / GB, 3),
|
||||||
|
"sha256": shard.sha256,
|
||||||
|
"url": shard.url,
|
||||||
|
"download_command": f'curl -L -C - --fail -o "{target}" "{shard.url}"',
|
||||||
|
"verify_command": (
|
||||||
|
f"python scripts/verify_glm_shards.py "
|
||||||
|
f'--model-dir "{dest_dir or "$GLM_DEST"}" --shard {shard.index}'
|
||||||
|
),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return steps
|
||||||
|
|
||||||
|
|
||||||
|
def build_report(*, dest: str | None, storage_only: bool) -> dict:
|
||||||
|
manifest = load_target_manifest()
|
||||||
|
prefixes = forbidden_prefixes(manifest)
|
||||||
|
|
||||||
|
if dest is not None:
|
||||||
|
mounts = [judge_destination(dest, prefixes)]
|
||||||
|
else:
|
||||||
|
mounts = scan_mounts(prefixes)
|
||||||
|
eligible = [m for m in mounts if m.eligible]
|
||||||
|
chosen = eligible[0] if eligible else None
|
||||||
|
|
||||||
|
mem_gib = memory_total_gib()
|
||||||
|
storage_pass = chosen is not None
|
||||||
|
memory_pass = mem_gib >= REQUIRED_MEMORY_GIB
|
||||||
|
|
||||||
|
checks = [
|
||||||
|
{
|
||||||
|
"check": "storage",
|
||||||
|
"requirement": f">= {REQUIRED_FREE_BYTES / GB:.0f} GB free on one filesystem "
|
||||||
|
f"outside {list(prefixes)}",
|
||||||
|
"observed": (
|
||||||
|
f"{chosen.free_bytes / GB:.1f} GB free at {chosen.mountpoint}"
|
||||||
|
if chosen
|
||||||
|
else "no eligible filesystem"
|
||||||
|
),
|
||||||
|
"passes": storage_pass,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"check": "memory",
|
||||||
|
"requirement": f">= {REQUIRED_MEMORY_GIB:.0f} GiB runtime-accessible memory "
|
||||||
|
"(DGR-017 experimental hard-fit floor)",
|
||||||
|
"observed": f"{mem_gib:.1f} GiB MemTotal",
|
||||||
|
"passes": memory_pass,
|
||||||
|
"waived": storage_only,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
overall = storage_pass and (memory_pass or storage_only)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"generated_by": "scripts/glm_whole_model_preflight.py",
|
||||||
|
"target": {
|
||||||
|
"gguf_repo_id": manifest.gguf_repo_id,
|
||||||
|
"gguf_revision": manifest.gguf_revision,
|
||||||
|
"quantization": manifest.quantization,
|
||||||
|
"shard_count": len(manifest.shards),
|
||||||
|
"total_bytes": manifest.total_bytes,
|
||||||
|
"total_gb": round(manifest.total_gb, 3),
|
||||||
|
"manifest_sha256": manifest.digest,
|
||||||
|
},
|
||||||
|
"forbidden_path_prefixes": list(prefixes),
|
||||||
|
"mounts": [m.to_dict() for m in mounts],
|
||||||
|
"chosen_destination": chosen.to_dict() if chosen else None,
|
||||||
|
"checks": checks,
|
||||||
|
"download_authorized": overall,
|
||||||
|
"storage_only": storage_only,
|
||||||
|
"download_plan": download_plan(manifest, chosen.mountpoint if chosen else None),
|
||||||
|
"verdict": "pass" if overall else "fail",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--dest", help="judge this destination directory instead of scanning")
|
||||||
|
parser.add_argument(
|
||||||
|
"--storage-only",
|
||||||
|
action="store_true",
|
||||||
|
help="download-host mode: require storage, report memory without gating on it",
|
||||||
|
)
|
||||||
|
parser.add_argument("--json", dest="json_path", help="also write the full report here")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
report = build_report(dest=args.dest, storage_only=args.storage_only)
|
||||||
|
|
||||||
|
if args.json_path:
|
||||||
|
Path(args.json_path).write_text(
|
||||||
|
json.dumps(report, indent=2, ensure_ascii=False) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
|
target = report["target"]
|
||||||
|
print(f"target: {target['quantization']} {target['total_gb']} GB, "
|
||||||
|
f"{target['shard_count']} shards @ {target['gguf_revision'][:12]}")
|
||||||
|
for check in report["checks"]:
|
||||||
|
status = "PASS" if check["passes"] else ("WAIVED" if check.get("waived") else "FAIL")
|
||||||
|
print(f"[{status}] {check['check']}: need {check['requirement']}; "
|
||||||
|
f"observed {check['observed']}")
|
||||||
|
if report["chosen_destination"]:
|
||||||
|
print(f"destination: {report['chosen_destination']['mountpoint']}")
|
||||||
|
else:
|
||||||
|
print("destination: NONE — no filesystem outside "
|
||||||
|
f"{report['forbidden_path_prefixes']} has "
|
||||||
|
f"{REQUIRED_FREE_BYTES / GB:.0f} GB free")
|
||||||
|
for mount in report["mounts"]:
|
||||||
|
why = "forbidden prefix" if mount["forbidden"] else f"{mount['free_gb']} GB free"
|
||||||
|
print(f" - {mount['mountpoint']} ({mount['fstype']}): {why}")
|
||||||
|
print(f"verdict: {report['verdict']}")
|
||||||
|
return 0 if report["download_authorized"] else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
268
scripts/llama_cpp_dependency.py
Normal file
268
scripts/llama_cpp_dependency.py
Normal file
@@ -0,0 +1,268 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Materialize, verify, build, and smoke-test DGR-004's llama.cpp pin.
|
||||||
|
|
||||||
|
This tool deliberately owns only a source dependency boundary. It never
|
||||||
|
downloads a model, invokes inference, or interprets generated text.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||||
|
LLAMA_DIR = ROOT / "packages/node/native/llama"
|
||||||
|
LOCK_PATH = LLAMA_DIR / "UPSTREAM_LOCK.json"
|
||||||
|
PATCH_DIR = LLAMA_DIR / "patches"
|
||||||
|
|
||||||
|
|
||||||
|
def _cmake() -> str:
|
||||||
|
"""Use an explicit override, PATH, or the active Python environment."""
|
||||||
|
configured = os.environ.get("CMAKE")
|
||||||
|
if configured:
|
||||||
|
return configured
|
||||||
|
on_path = shutil.which("cmake")
|
||||||
|
if on_path:
|
||||||
|
return on_path
|
||||||
|
sibling = pathlib.Path(sys.executable).parent / "cmake"
|
||||||
|
if sibling.is_file():
|
||||||
|
return str(sibling)
|
||||||
|
raise DependencyError("cmake is unavailable; set CMAKE or activate the project toolchain")
|
||||||
|
|
||||||
|
|
||||||
|
class DependencyError(RuntimeError):
|
||||||
|
"""A fail-closed reproducibility or source-integrity failure."""
|
||||||
|
|
||||||
|
|
||||||
|
def _run(*args: str, cwd: pathlib.Path | None = None) -> str:
|
||||||
|
try:
|
||||||
|
completed = subprocess.run(
|
||||||
|
args,
|
||||||
|
cwd=cwd,
|
||||||
|
check=True,
|
||||||
|
text=True,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE,
|
||||||
|
)
|
||||||
|
except FileNotFoundError as error:
|
||||||
|
raise DependencyError(f"required executable is unavailable: {args[0]}") from error
|
||||||
|
except subprocess.CalledProcessError as error:
|
||||||
|
detail = (error.stderr or error.stdout).strip()
|
||||||
|
raise DependencyError(f"command failed: {' '.join(args)}\n{detail}") from error
|
||||||
|
return completed.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _load_lock() -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
lock = json.loads(LOCK_PATH.read_text())
|
||||||
|
except (OSError, json.JSONDecodeError) as error:
|
||||||
|
raise DependencyError(f"invalid upstream lock: {LOCK_PATH}: {error}") from error
|
||||||
|
required = {
|
||||||
|
"upstream", "commit", "commit_tree", "patched_tree", "patch_series",
|
||||||
|
"required_upstream_blobs", "patched_paths", "build",
|
||||||
|
}
|
||||||
|
missing = sorted(required - lock.keys())
|
||||||
|
if missing:
|
||||||
|
raise DependencyError(f"upstream lock is missing fields: {', '.join(missing)}")
|
||||||
|
commit_file = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
|
||||||
|
if lock["commit"] != commit_file or len(commit_file) != 40:
|
||||||
|
raise DependencyError("UPSTREAM_COMMIT and UPSTREAM_LOCK.json do not agree on a full commit")
|
||||||
|
return lock
|
||||||
|
|
||||||
|
|
||||||
|
def _patches(lock: dict[str, Any]) -> list[pathlib.Path]:
|
||||||
|
series = [line for line in (PATCH_DIR / "series").read_text().splitlines() if line]
|
||||||
|
if series != lock["patch_series"] or series != sorted(series) or not series:
|
||||||
|
raise DependencyError("patches/series is empty, unordered, or disagrees with UPSTREAM_LOCK.json")
|
||||||
|
sums: dict[str, str] = {}
|
||||||
|
for line in (PATCH_DIR / "SHA256SUMS").read_text().splitlines():
|
||||||
|
if line and not line.startswith("#"):
|
||||||
|
digest, name = line.split(maxsplit=1)
|
||||||
|
sums[name] = digest
|
||||||
|
patches: list[pathlib.Path] = []
|
||||||
|
for name in series:
|
||||||
|
patch = PATCH_DIR / name
|
||||||
|
if not patch.is_file():
|
||||||
|
raise DependencyError(f"patch listed in series is missing: {name}")
|
||||||
|
actual = hashlib.sha256(patch.read_bytes()).hexdigest()
|
||||||
|
if sums.get(name) != actual:
|
||||||
|
raise DependencyError(f"patch digest mismatch for {name}: expected {sums.get(name)}, got {actual}")
|
||||||
|
patches.append(patch)
|
||||||
|
return patches
|
||||||
|
|
||||||
|
|
||||||
|
def _git(source: pathlib.Path, *args: str) -> str:
|
||||||
|
return _run("git", "-C", str(source), *args)
|
||||||
|
|
||||||
|
|
||||||
|
def _verify_source(source: pathlib.Path, lock: dict[str, Any], *, require_clean: bool) -> None:
|
||||||
|
if not (source / ".git").exists():
|
||||||
|
raise DependencyError(f"not a materialized git checkout: {source}")
|
||||||
|
if _git(source, "rev-parse", "HEAD") != lock["commit"]:
|
||||||
|
raise DependencyError("upstream drift: checkout HEAD does not equal the locked commit")
|
||||||
|
if _git(source, "rev-parse", "HEAD^{tree}") != lock["commit_tree"]:
|
||||||
|
raise DependencyError("upstream drift: checkout tree does not equal the locked tree")
|
||||||
|
if require_clean and _git(source, "status", "--porcelain"):
|
||||||
|
raise DependencyError("local edits detected in materialized llama.cpp checkout")
|
||||||
|
for relative, expected in lock["required_upstream_blobs"].items():
|
||||||
|
actual = _git(source, "rev-parse", f"HEAD:{relative}")
|
||||||
|
if actual != expected:
|
||||||
|
raise DependencyError(
|
||||||
|
f"upstream ABI/context drift for {relative}: expected {expected}, got {actual}"
|
||||||
|
)
|
||||||
|
if not (source / "LICENSE").is_file():
|
||||||
|
raise DependencyError("upstream LICENSE is missing; refusing to drop required attribution")
|
||||||
|
|
||||||
|
|
||||||
|
def materialize(source: pathlib.Path, repository: str) -> None:
|
||||||
|
lock = _load_lock()
|
||||||
|
_patches(lock)
|
||||||
|
if source.exists():
|
||||||
|
raise DependencyError(f"destination already exists; refusing to reuse possibly edited source: {source}")
|
||||||
|
source.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
_run("git", "clone", "--no-checkout", repository, str(source))
|
||||||
|
_git(source, "checkout", "--detach", lock["commit"])
|
||||||
|
_verify_source(source, lock, require_clean=True)
|
||||||
|
|
||||||
|
|
||||||
|
def apply(source: pathlib.Path) -> None:
|
||||||
|
lock = _load_lock()
|
||||||
|
patches = _patches(lock)
|
||||||
|
_verify_source(source, lock, require_clean=True)
|
||||||
|
for patch in patches:
|
||||||
|
_git(source, "apply", "--check", str(patch))
|
||||||
|
_git(source, "apply", "--index", str(patch))
|
||||||
|
_verify_patched_source(source, lock)
|
||||||
|
|
||||||
|
|
||||||
|
def _verify_patched_source(source: pathlib.Path, lock: dict[str, Any]) -> None:
|
||||||
|
if _git(source, "diff", "--quiet"):
|
||||||
|
raise DependencyError("local unstaged edits detected after applying patch stack")
|
||||||
|
changed_paths = _git(source, "diff", "--cached", "--name-only").splitlines()
|
||||||
|
if changed_paths != lock["patched_paths"]:
|
||||||
|
raise DependencyError(f"patched paths drifted: expected {lock['patched_paths']}, got {changed_paths}")
|
||||||
|
if _git(source, "write-tree") != lock["patched_tree"]:
|
||||||
|
raise DependencyError("patched source tree differs from the locked patch stack")
|
||||||
|
if _git(source, "diff", "--name-only"):
|
||||||
|
raise DependencyError("local unstaged edits detected after applying patch stack")
|
||||||
|
untracked = _git(source, "ls-files", "--others", "--exclude-standard").splitlines()
|
||||||
|
if untracked:
|
||||||
|
raise DependencyError(f"untracked files detected after applying patch stack: {untracked}")
|
||||||
|
|
||||||
|
|
||||||
|
def build(source: pathlib.Path, build_dir: pathlib.Path) -> pathlib.Path:
|
||||||
|
lock = _load_lock()
|
||||||
|
_patches(lock)
|
||||||
|
_verify_source(source, lock, require_clean=False)
|
||||||
|
_verify_patched_source(source, lock)
|
||||||
|
expected_marker = source / "cmake/meshnet-patch-stack.cmake"
|
||||||
|
if not expected_marker.is_file():
|
||||||
|
raise DependencyError("patch stack is not applied: Meshnet CMake marker is absent")
|
||||||
|
if build_dir.exists():
|
||||||
|
raise DependencyError(f"build directory already exists; use reproduce for a clean rebuild: {build_dir}")
|
||||||
|
flags = lock["build"]["configure_flags"]
|
||||||
|
cmake = _cmake()
|
||||||
|
_run(cmake, "-G", lock["build"]["generator"], "-S", str(source), "-B", str(build_dir), *flags)
|
||||||
|
for target in lock["build"]["native_targets"]:
|
||||||
|
_run(cmake, "--build", str(build_dir), "--target", target, "-j2")
|
||||||
|
metadata = {
|
||||||
|
"commit": lock["commit"],
|
||||||
|
"commit_tree": lock["commit_tree"],
|
||||||
|
"patches": {patch.name: hashlib.sha256(patch.read_bytes()).hexdigest() for patch in _patches(lock)},
|
||||||
|
"configure_flags": flags,
|
||||||
|
"cmake": _run(cmake, "--version").splitlines()[0],
|
||||||
|
"cxx": _run("c++", "--version").splitlines()[0],
|
||||||
|
"model_downloads": False,
|
||||||
|
"semantic_certification": False,
|
||||||
|
}
|
||||||
|
(build_dir / "meshnet-build-metadata.json").write_text(json.dumps(metadata, indent=2, sort_keys=True) + "\n")
|
||||||
|
return build_dir / lock["build"]["smoke_binary"]
|
||||||
|
|
||||||
|
|
||||||
|
def smoke(binary: pathlib.Path) -> None:
|
||||||
|
if not binary.is_file():
|
||||||
|
raise DependencyError(f"native smoke binary is missing: {binary}")
|
||||||
|
smoke_config = _load_lock()["build"]
|
||||||
|
output = _run(str(binary), *smoke_config["smoke_args"])
|
||||||
|
expected = smoke_config["smoke_output_token"]
|
||||||
|
if expected not in output.lower():
|
||||||
|
raise DependencyError(f"native smoke output did not contain {expected!r}: {output}")
|
||||||
|
print(output)
|
||||||
|
|
||||||
|
|
||||||
|
def reproduce(work_dir: pathlib.Path, repository: str) -> None:
|
||||||
|
resolved = work_dir.resolve()
|
||||||
|
build_root = (ROOT / "build").resolve()
|
||||||
|
if build_root not in resolved.parents:
|
||||||
|
raise DependencyError(f"--work-dir must be below {build_root}: {resolved}")
|
||||||
|
if resolved.exists():
|
||||||
|
raise DependencyError(
|
||||||
|
f"work directory already exists; refusing to erase possible local edits: {resolved}"
|
||||||
|
)
|
||||||
|
source = resolved / "source"
|
||||||
|
build_dir = resolved / "build"
|
||||||
|
materialize(source, repository)
|
||||||
|
apply(source)
|
||||||
|
smoke(build(source, build_dir))
|
||||||
|
|
||||||
|
|
||||||
|
def inspect() -> None:
|
||||||
|
lock = _load_lock()
|
||||||
|
patches = _patches(lock)
|
||||||
|
print(json.dumps({
|
||||||
|
"commit": lock["commit"],
|
||||||
|
"patch_count": len(patches),
|
||||||
|
"patches": [patch.name for patch in patches],
|
||||||
|
"model_downloads": False,
|
||||||
|
"semantic_certification": False,
|
||||||
|
"glm_stock_limitations": lock["stock_glm_limitations"],
|
||||||
|
}, indent=2, sort_keys=True))
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
subcommands = parser.add_subparsers(dest="command", required=True)
|
||||||
|
subcommands.add_parser("inspect")
|
||||||
|
materialize_parser = subcommands.add_parser("materialize")
|
||||||
|
materialize_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
|
||||||
|
materialize_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
|
||||||
|
apply_parser = subcommands.add_parser("apply")
|
||||||
|
apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
|
||||||
|
build_parser = subcommands.add_parser("build")
|
||||||
|
build_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
|
||||||
|
build_parser.add_argument("--build-dir", type=pathlib.Path, required=True)
|
||||||
|
smoke_parser = subcommands.add_parser("smoke")
|
||||||
|
smoke_parser.add_argument("--binary", type=pathlib.Path, required=True)
|
||||||
|
reproduce_parser = subcommands.add_parser("reproduce")
|
||||||
|
reproduce_parser.add_argument("--work-dir", type=pathlib.Path, default=ROOT / "build/dgr-004-smoke")
|
||||||
|
reproduce_parser.add_argument("--source-repository", default=_load_lock()["upstream"])
|
||||||
|
args = parser.parse_args()
|
||||||
|
try:
|
||||||
|
if args.command == "inspect":
|
||||||
|
inspect()
|
||||||
|
elif args.command == "materialize":
|
||||||
|
materialize(args.source_dir, args.source_repository)
|
||||||
|
elif args.command == "apply":
|
||||||
|
apply(args.source_dir)
|
||||||
|
elif args.command == "build":
|
||||||
|
build(args.source_dir, args.build_dir)
|
||||||
|
elif args.command == "smoke":
|
||||||
|
smoke(args.binary)
|
||||||
|
else:
|
||||||
|
reproduce(args.work_dir, args.source_repository)
|
||||||
|
except DependencyError as error:
|
||||||
|
print(f"DGR-004 dependency error: {error}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
135
scripts/verify_glm_shards.py
Normal file
135
scripts/verify_glm_shards.py
Normal file
@@ -0,0 +1,135 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Verify downloaded GLM-5.2 ``UD-IQ1_S`` shards against the pinned manifest.
|
||||||
|
|
||||||
|
Every shard must match the exact byte size and LFS SHA-256 that DGR-017 locked
|
||||||
|
in ``target-manifest.json``. Size is checked first because it is free and
|
||||||
|
catches truncated resumes; the digest is then streamed so a 49 GB file never
|
||||||
|
loads into memory. A shard that fails is reported with what was measured — the
|
||||||
|
file is never deleted or "healed" by this script.
|
||||||
|
|
||||||
|
Fail-closed: exit 0 only when every requested shard is present and matches.
|
||||||
|
|
||||||
|
Usage::
|
||||||
|
|
||||||
|
python scripts/verify_glm_shards.py --model-dir /mnt/models/glm-5.2
|
||||||
|
python scripts/verify_glm_shards.py --model-dir DIR --shard 6 # one shard
|
||||||
|
python scripts/verify_glm_shards.py --model-dir DIR --json report.json
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
sys.path.insert(0, str(_ROOT / "packages/node"))
|
||||||
|
|
||||||
|
from meshnet_node.glm_alpha.manifest import ( # noqa: E402
|
||||||
|
Shard,
|
||||||
|
TargetManifest,
|
||||||
|
load_target_manifest,
|
||||||
|
)
|
||||||
|
|
||||||
|
_CHUNK_BYTES = 8 * 1024 * 1024
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_shard_file(model_dir: Path, shard: Shard) -> Path:
|
||||||
|
"""Accept the repository layout (``UD-IQ1_S/<name>``) or a flat directory."""
|
||||||
|
nested = model_dir / shard.path
|
||||||
|
if nested.exists():
|
||||||
|
return nested
|
||||||
|
return model_dir / Path(shard.path).name
|
||||||
|
|
||||||
|
|
||||||
|
def streaming_sha256(path: Path) -> str:
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
while chunk := handle.read(_CHUNK_BYTES):
|
||||||
|
digest.update(chunk)
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def verify_shard(model_dir: Path, shard: Shard) -> dict:
|
||||||
|
result: dict = {
|
||||||
|
"shard_index": shard.index,
|
||||||
|
"path": shard.path,
|
||||||
|
"expected_size_bytes": shard.size_bytes,
|
||||||
|
"expected_sha256": shard.sha256,
|
||||||
|
}
|
||||||
|
file = resolve_shard_file(model_dir, shard)
|
||||||
|
result["file"] = str(file)
|
||||||
|
if not file.exists():
|
||||||
|
result.update(status="missing")
|
||||||
|
return result
|
||||||
|
|
||||||
|
size = file.stat().st_size
|
||||||
|
result["measured_size_bytes"] = size
|
||||||
|
if size != shard.size_bytes:
|
||||||
|
result.update(status="size_mismatch")
|
||||||
|
return result
|
||||||
|
|
||||||
|
started = time.monotonic()
|
||||||
|
measured = streaming_sha256(file)
|
||||||
|
result["measured_sha256"] = measured
|
||||||
|
result["hash_seconds"] = round(time.monotonic() - started, 1)
|
||||||
|
result.update(status="ok" if measured == shard.sha256 else "sha256_mismatch")
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def verify_shards(
|
||||||
|
model_dir: Path, manifest: TargetManifest, indices: list[int] | None = None
|
||||||
|
) -> dict:
|
||||||
|
shards = manifest.shards if indices is None else [manifest.shard(i) for i in indices]
|
||||||
|
results = [verify_shard(model_dir, shard) for shard in shards]
|
||||||
|
return {
|
||||||
|
"generated_by": "scripts/verify_glm_shards.py",
|
||||||
|
"model_dir": str(model_dir),
|
||||||
|
"gguf_repo_id": manifest.gguf_repo_id,
|
||||||
|
"gguf_revision": manifest.gguf_revision,
|
||||||
|
"quantization": manifest.quantization,
|
||||||
|
"manifest_sha256": manifest.digest,
|
||||||
|
"shards": results,
|
||||||
|
"verdict": "pass" if all(r["status"] == "ok" for r in results) else "fail",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--model-dir", required=True, help="directory holding the shards")
|
||||||
|
parser.add_argument(
|
||||||
|
"--shard",
|
||||||
|
type=int,
|
||||||
|
action="append",
|
||||||
|
dest="shards",
|
||||||
|
help="verify only this shard index (repeatable); default: all six",
|
||||||
|
)
|
||||||
|
parser.add_argument("--json", dest="json_path", help="also write the full report here")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
report = verify_shards(Path(args.model_dir), load_target_manifest(), args.shards)
|
||||||
|
|
||||||
|
if args.json_path:
|
||||||
|
Path(args.json_path).write_text(
|
||||||
|
json.dumps(report, indent=2, ensure_ascii=False) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
|
for result in report["shards"]:
|
||||||
|
line = f"shard {result['shard_index']}: {result['status']}"
|
||||||
|
if result["status"] == "size_mismatch":
|
||||||
|
line += (
|
||||||
|
f" (expected {result['expected_size_bytes']}, "
|
||||||
|
f"measured {result['measured_size_bytes']})"
|
||||||
|
)
|
||||||
|
elif result["status"] == "sha256_mismatch":
|
||||||
|
line += f" (measured {result['measured_sha256']})"
|
||||||
|
print(line)
|
||||||
|
print(f"verdict: {report['verdict']}")
|
||||||
|
return 0 if report["verdict"] == "pass" else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
109
tests/data/recipe_fingerprint_vectors.json
Normal file
109
tests/data/recipe_fingerprint_vectors.json
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
{
|
||||||
|
"schema_version": 1,
|
||||||
|
"vectors": [
|
||||||
|
{
|
||||||
|
"description": "An undivided artifact: content digest is the source digest.",
|
||||||
|
"fingerprint": {
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
|
||||||
|
},
|
||||||
|
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31",
|
||||||
|
"identity": {
|
||||||
|
"artifact": {
|
||||||
|
"architecture": "dense-llama",
|
||||||
|
"architecture_digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||||
|
"artifact_id": "example/model",
|
||||||
|
"content_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||||
|
"derived_from": null,
|
||||||
|
"layer_count": 8,
|
||||||
|
"revision": "0123456789abcdef"
|
||||||
|
},
|
||||||
|
"fingerprint": {
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
|
||||||
|
},
|
||||||
|
"recipe": {
|
||||||
|
"activation_dtype": "bfloat16",
|
||||||
|
"architecture_adapter": "llama/range-v1",
|
||||||
|
"backend_id": "llama.cpp",
|
||||||
|
"boundary_schema_version": 1,
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
"compute_dtype": "float32",
|
||||||
|
"kv_dtype": "q8_0",
|
||||||
|
"kv_layout": "paged-v1",
|
||||||
|
"protocol_schema_version": 1,
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
|
||||||
|
"tokenizer_revision": "0123456789abcdef",
|
||||||
|
"weight_quantization": "Q4_K_M"
|
||||||
|
},
|
||||||
|
"schema_version": 1,
|
||||||
|
"shard_end": 4,
|
||||||
|
"shard_start": 0
|
||||||
|
},
|
||||||
|
"name": "example-v1",
|
||||||
|
"shard_binding_digest": "f62d1f76cc18b548782c02850e05c2634da55c0e686c43b9f687bdee7bdefe19"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "A split of the same source: same fingerprint, different Shard binding.",
|
||||||
|
"fingerprint": {
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
|
||||||
|
},
|
||||||
|
"fingerprint_proto_hex": "0a40386130663433643661613439643737383334626462343762636165396634326338383662376363666530616330313439333262326132623338363937613437621240396231346437306230383335613634323834353765343838386434353336343964643064326534316663386163396438346432333263386332333765363866611a0c6578616d706c652d676775662201312a09323032362e30372e31",
|
||||||
|
"identity": {
|
||||||
|
"artifact": {
|
||||||
|
"architecture": "dense-llama",
|
||||||
|
"architecture_digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||||
|
"artifact_id": "example/model",
|
||||||
|
"content_digest": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||||
|
"derived_from": {
|
||||||
|
"shard_end": 8,
|
||||||
|
"shard_start": 4,
|
||||||
|
"source_artifact_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||||
|
},
|
||||||
|
"layer_count": 8,
|
||||||
|
"revision": "0123456789abcdef"
|
||||||
|
},
|
||||||
|
"fingerprint": {
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
|
||||||
|
},
|
||||||
|
"recipe": {
|
||||||
|
"activation_dtype": "bfloat16",
|
||||||
|
"architecture_adapter": "llama/range-v1",
|
||||||
|
"backend_id": "llama.cpp",
|
||||||
|
"boundary_schema_version": 1,
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
"compute_dtype": "float32",
|
||||||
|
"kv_dtype": "q8_0",
|
||||||
|
"kv_layout": "paged-v1",
|
||||||
|
"protocol_schema_version": 1,
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
|
||||||
|
"tokenizer_revision": "0123456789abcdef",
|
||||||
|
"weight_quantization": "Q4_K_M"
|
||||||
|
},
|
||||||
|
"schema_version": 1,
|
||||||
|
"shard_end": 8,
|
||||||
|
"shard_start": 4
|
||||||
|
},
|
||||||
|
"name": "example-v1-derivative",
|
||||||
|
"shard_binding_digest": "55271611eb63cb81088b8109133f1dff845352298994fc8e9c5824a6a01c83e0"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
121
tests/test_architecture_boundary.py
Normal file
121
tests/test_architecture_boundary.py
Normal file
@@ -0,0 +1,121 @@
|
|||||||
|
"""DGR-006 architecture-defined activation-boundary contract."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import struct
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from meshnet_node.architecture_boundary import (
|
||||||
|
Architecture,
|
||||||
|
BoundaryStage,
|
||||||
|
ProtocolIdentity,
|
||||||
|
SamplingParameters,
|
||||||
|
TailOutput,
|
||||||
|
adapter_for,
|
||||||
|
)
|
||||||
|
from meshnet_node.native_protocol import ProtocolError, decode_bundle
|
||||||
|
|
||||||
|
|
||||||
|
def _f32(values: list[float]) -> bytes:
|
||||||
|
return struct.pack("<" + "f" * len(values), *values)
|
||||||
|
|
||||||
|
|
||||||
|
def _values(payload: bytes) -> tuple[float, ...]:
|
||||||
|
return struct.unpack("<" + "f" * (len(payload) // 4), payload)
|
||||||
|
|
||||||
|
|
||||||
|
def test_dense_whole_model_and_two_ranges_match_for_prefill_and_greedy_decode() -> None:
|
||||||
|
adapter = adapter_for(Architecture.DENSE)
|
||||||
|
embeddings = {3: [1.0, 2.0], 7: [3.0, 4.0]}
|
||||||
|
head = adapter.bundle_from_token_ids([3, 7], lambda token: embeddings[token])
|
||||||
|
|
||||||
|
def head_layers(bundle):
|
||||||
|
return [value + 10.0 for value in _values(decode_bundle(bundle)["hidden_states"])]
|
||||||
|
|
||||||
|
def tail_layers(residual: list[float]) -> tuple[list[float], int]:
|
||||||
|
# The tail alone applies this fixture's final norm/head and greedy argmax.
|
||||||
|
logits = [sum(residual) / len(residual) + 20.0, 0.0]
|
||||||
|
return logits, max(range(len(logits)), key=logits.__getitem__)
|
||||||
|
|
||||||
|
# Whole-model prefill retains the unnormalized residual locally. The
|
||||||
|
# two-range lane sends that same residual before the tail-only head.
|
||||||
|
whole_prefill = head_layers(head)
|
||||||
|
seam = adapter.bundle_from_named_payloads(
|
||||||
|
{"hidden_states": _f32(whole_prefill)}, shape=[1, 2, 2]
|
||||||
|
)
|
||||||
|
two_range_prefill = list(_values(decode_bundle(adapter.input_for(BoundaryStage.TAIL, seam))["hidden_states"]))
|
||||||
|
assert two_range_prefill == whole_prefill
|
||||||
|
|
||||||
|
whole_logits, whole_token = tail_layers(whole_prefill)
|
||||||
|
two_logits, two_token = tail_layers(two_range_prefill)
|
||||||
|
assert two_logits == whole_logits
|
||||||
|
assert two_token == whole_token == 0
|
||||||
|
|
||||||
|
# One greedy decode step is the same contract with [batch, token, hidden].
|
||||||
|
decode_seam = adapter.bundle_from_named_payloads(
|
||||||
|
{"hidden_states": _f32([5.0, 6.0])}, shape=[1, 1, 2]
|
||||||
|
)
|
||||||
|
assert tail_layers([5.0, 6.0]) == tail_layers(
|
||||||
|
list(_values(decode_bundle(adapter.input_for(BoundaryStage.TAIL, decode_seam))["hidden_states"]))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_middle_and_tail_reject_token_ids_and_require_boundary_bundle() -> None:
|
||||||
|
adapter = adapter_for(Architecture.MOE)
|
||||||
|
|
||||||
|
with pytest.raises(ProtocolError, match="head"):
|
||||||
|
adapter.bundle_from_token_ids([1], lambda _: [1.0])
|
||||||
|
|
||||||
|
with pytest.raises(ProtocolError, match="TensorBundle"):
|
||||||
|
adapter.input_for(BoundaryStage.MIDDLE, None)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("architecture", "names"),
|
||||||
|
[
|
||||||
|
(Architecture.MOE, {"hidden_states", "router_logits"}),
|
||||||
|
(Architecture.MLA, {"hidden_states", "mla_position_state"}),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_architecture_adapters_route_and_validate_their_named_sidebands(
|
||||||
|
architecture: Architecture, names: set[str]
|
||||||
|
) -> None:
|
||||||
|
adapter = adapter_for(architecture)
|
||||||
|
bundle = adapter.bundle_from_named_payloads(
|
||||||
|
{
|
||||||
|
name: (_f32([1.0, 2.0]) if name == "hidden_states" else _f32([0.0]))
|
||||||
|
for name in names
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert set(decode_bundle(adapter.input_for(BoundaryStage.MIDDLE, bundle))) == names
|
||||||
|
with pytest.raises(ProtocolError, match="requires"):
|
||||||
|
adapter.bundle_from_named_payloads({"hidden_states": _f32([1.0, 2.0])})
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_architecture_fails_closed() -> None:
|
||||||
|
with pytest.raises(ProtocolError, match="unsupported architecture"):
|
||||||
|
adapter_for("unchecked-name-substitution")
|
||||||
|
|
||||||
|
|
||||||
|
def test_typed_tail_result_binds_sampling_and_request_recipe_identity() -> None:
|
||||||
|
adapter = adapter_for(Architecture.DENSE)
|
||||||
|
identity = ProtocolIdentity(
|
||||||
|
request_id="request-1",
|
||||||
|
runtime_recipe_digest="sha256:recipe",
|
||||||
|
chat_template_id="llama3",
|
||||||
|
chat_template_version="2",
|
||||||
|
reasoning_mode="max",
|
||||||
|
architecture=Architecture.DENSE,
|
||||||
|
)
|
||||||
|
result = adapter.tail_result(
|
||||||
|
identity=identity,
|
||||||
|
sampling=SamplingParameters(temperature=0.0, top_p=1.0, top_k=0, seed=9),
|
||||||
|
output=TailOutput.sampled_token(42),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.identity.request_id == "request-1"
|
||||||
|
assert result.sampled_token_id == 42
|
||||||
|
assert result.output_kind == "sampled_token_id"
|
||||||
|
assert result.message.WhichOneof("output") == "sampled_token_id"
|
||||||
233
tests/test_gguf_distributed_load.py
Normal file
233
tests/test_gguf_distributed_load.py
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
"""Distributed GGUF shard load integration test.
|
||||||
|
|
||||||
|
Downloads a small dense-Llama GGUF (TinyLlama 1.1B Q4_K_M ~670 MB),
|
||||||
|
loads it in shard ranges via the meshnet-range-loader C wrapper, registers
|
||||||
|
each shard with a live TrackerServer, and verifies routing, range reporting,
|
||||||
|
and memory scaling.
|
||||||
|
|
||||||
|
Set MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 to run.
|
||||||
|
|
||||||
|
Evidence class: real-model integration. Downloads ~670 MB on first run.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
ROOT = pathlib.Path(__file__).resolve().parent.parent
|
||||||
|
sys.path[:0] = [
|
||||||
|
str(ROOT / "packages" / "tracker"),
|
||||||
|
str(ROOT / "packages" / "node"),
|
||||||
|
str(ROOT / "packages" / "contracts"),
|
||||||
|
]
|
||||||
|
|
||||||
|
from meshnet_tracker.server import TrackerServer # noqa: E402 — sys.path prepended above
|
||||||
|
|
||||||
|
# Only run when explicitly enabled
|
||||||
|
pytestmark = pytest.mark.skipif(
|
||||||
|
"MESHNET_ENABLE_REAL_INFERENCE_TESTS" not in os.environ,
|
||||||
|
reason="set MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 to download and load a real GGUF",
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Model configuration
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
HF_REPO = "TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF"
|
||||||
|
GGUF_FILE = "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf"
|
||||||
|
MODEL_URL = f"https://huggingface.co/{HF_REPO}/resolve/main/{GGUF_FILE}"
|
||||||
|
EXPECTED_LAYERS = 22
|
||||||
|
GGUF_FLAVOR = GGUF_FILE.replace(".gguf", "")
|
||||||
|
|
||||||
|
CACHE_DIR = ROOT / ".cache" / "gguf-models"
|
||||||
|
LOADER = ROOT / "build" / "dgr-004-final" / "build" / "bin" / "meshnet-range-loader"
|
||||||
|
LLAMA_LIB_DIR = LOADER.parent
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Loading helper
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def load_shard(gguf_path: str, start: int, end: int) -> dict:
|
||||||
|
"""Load a shard range of the GGUF via the C wrapper and return JSON report."""
|
||||||
|
env = os.environ.copy()
|
||||||
|
env["LD_LIBRARY_PATH"] = str(LLAMA_LIB_DIR)
|
||||||
|
|
||||||
|
result = subprocess.run(
|
||||||
|
[str(LOADER), gguf_path, str(start), str(end)],
|
||||||
|
capture_output=True, text=True, timeout=120, env=env,
|
||||||
|
)
|
||||||
|
if result.returncode != 0:
|
||||||
|
# Parse JSON from stdout even on error if it exists
|
||||||
|
if result.stdout.strip():
|
||||||
|
try:
|
||||||
|
return json.loads(result.stdout)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
raise RuntimeError(
|
||||||
|
f"meshnet-range-loader [{start}, {end}) failed (exit {result.returncode}):\n"
|
||||||
|
f"stderr: {result.stderr[:500]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
return json.loads(result.stdout)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Download and cache
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_model() -> pathlib.Path:
|
||||||
|
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
model_path = CACHE_DIR / GGUF_FILE
|
||||||
|
|
||||||
|
if model_path.exists() and model_path.stat().st_size > 600 * 1024 * 1024:
|
||||||
|
return model_path
|
||||||
|
|
||||||
|
print(f"Downloading {MODEL_URL} (~670 MB)...", file=sys.stderr)
|
||||||
|
urllib.request.urlretrieve(MODEL_URL, model_path)
|
||||||
|
actual_mb = model_path.stat().st_size / (1024 * 1024)
|
||||||
|
print(f"Downloaded {GGUF_FLAVOR}: {actual_mb:.0f} MB", file=sys.stderr)
|
||||||
|
return model_path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tracker helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _post_json(url: str, data: dict) -> dict:
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
data=json.dumps(data).encode(),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=10) as r:
|
||||||
|
return json.loads(r.read())
|
||||||
|
|
||||||
|
|
||||||
|
def _get_json(url: str) -> dict:
|
||||||
|
with urllib.request.urlopen(url, timeout=10) as r:
|
||||||
|
return json.loads(r.read())
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_whole_model_load_and_report():
|
||||||
|
"""Load the full TinyLlama GGUF and verify metadata."""
|
||||||
|
gguf = _ensure_model()
|
||||||
|
report = load_shard(str(gguf), 0, EXPECTED_LAYERS)
|
||||||
|
assert report["ok"] is True
|
||||||
|
assert report["start_layer"] == 0
|
||||||
|
assert report["end_layer"] == EXPECTED_LAYERS
|
||||||
|
assert report["mapped_bytes"] > 0
|
||||||
|
assert report["resident_bytes"] >= report["mapped_bytes"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_head_shard_is_less_than_full_model():
|
||||||
|
"""A head-only shard maps fewer bytes than the full model."""
|
||||||
|
gguf = _ensure_model()
|
||||||
|
head = load_shard(str(gguf), 0, 8)
|
||||||
|
full = load_shard(str(gguf), 0, EXPECTED_LAYERS)
|
||||||
|
assert head["mapped_bytes"] <= full["mapped_bytes"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_tail_shard_maps_fewer_bytes_than_middle():
|
||||||
|
"""Fewer layers = fewer bytes (tail has 6 layers, middle has 8)."""
|
||||||
|
gguf = _ensure_model()
|
||||||
|
middle = load_shard(str(gguf), 8, 16)
|
||||||
|
tail = load_shard(str(gguf), 16, EXPECTED_LAYERS)
|
||||||
|
# Middle (8 layers) must map more than tail (6 layers)
|
||||||
|
assert middle["mapped_bytes"] > tail["mapped_bytes"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_memory_scales_with_owned_range():
|
||||||
|
"""More layers = more resident bytes."""
|
||||||
|
gguf = _ensure_model()
|
||||||
|
small = load_shard(str(gguf), 0, 4)
|
||||||
|
large = load_shard(str(gguf), 0, 12)
|
||||||
|
assert large["mapped_bytes"] > small["mapped_bytes"]
|
||||||
|
assert large["resident_bytes"] > small["resident_bytes"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_range_rejected():
|
||||||
|
"""Loading a range outside GGUF layer bounds fails closed."""
|
||||||
|
gguf = _ensure_model()
|
||||||
|
env = os.environ.copy()
|
||||||
|
env["LD_LIBRARY_PATH"] = str(LLAMA_LIB_DIR)
|
||||||
|
result = subprocess.run(
|
||||||
|
[str(LOADER), str(gguf), str(EXPECTED_LAYERS + 1), str(EXPECTED_LAYERS + 5)],
|
||||||
|
capture_output=True, text=True, timeout=30, env=env,
|
||||||
|
)
|
||||||
|
# Should fail with exit code 1 and an error message
|
||||||
|
assert result.returncode != 0
|
||||||
|
assert "error" in result.stderr.lower() or "ERROR" in result.stderr
|
||||||
|
|
||||||
|
|
||||||
|
def test_tracker_registers_shard_nodes():
|
||||||
|
"""Start a tracker, register multiple shard nodes, verify the console."""
|
||||||
|
gguf = _ensure_model()
|
||||||
|
|
||||||
|
tracker = TrackerServer(heartbeat_timeout=60.0)
|
||||||
|
tracker_port = tracker.start()
|
||||||
|
|
||||||
|
shards = [
|
||||||
|
("head", 0, 8),
|
||||||
|
("middle", 8, 16),
|
||||||
|
("tail", 16, EXPECTED_LAYERS),
|
||||||
|
]
|
||||||
|
registered_ids = []
|
||||||
|
try:
|
||||||
|
for name_tag, start, end in shards:
|
||||||
|
report = load_shard(str(gguf), start, end)
|
||||||
|
node_id = f"{GGUF_FLAVOR}-{name_tag}"
|
||||||
|
registered_ids.append(node_id)
|
||||||
|
|
||||||
|
_post_json(
|
||||||
|
f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
|
||||||
|
{
|
||||||
|
"node_id": node_id,
|
||||||
|
"endpoint": f"http://localhost:{10000 + start}",
|
||||||
|
"model": GGUF_FLAVOR,
|
||||||
|
"num_layers": report["end_layer"] - report["start_layer"],
|
||||||
|
"shard_start": report["start_layer"],
|
||||||
|
"shard_end": report["end_layer"],
|
||||||
|
"hardware_profile": {
|
||||||
|
"mapped_bytes": report["mapped_bytes"],
|
||||||
|
"resident_bytes": report["resident_bytes"],
|
||||||
|
},
|
||||||
|
"score": 1.0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Verify console shows all three registered nodes
|
||||||
|
console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
|
||||||
|
registered_eps = {
|
||||||
|
f"http://localhost:{10000 + start}" for _, start, _ in shards
|
||||||
|
}
|
||||||
|
found_eps = set()
|
||||||
|
for event in console.get("events", []):
|
||||||
|
if event.get("message") == "node registered":
|
||||||
|
ep = event.get("fields", {}).get("endpoint", "")
|
||||||
|
if ep in registered_eps:
|
||||||
|
found_eps.add(ep)
|
||||||
|
for _, start, _ in shards:
|
||||||
|
expected_ep = f"http://localhost:{10000 + start}"
|
||||||
|
assert expected_ep in found_eps, \
|
||||||
|
f"endpoint {expected_ep} not found in registration events"
|
||||||
|
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
@@ -80,7 +80,7 @@ def snapshot_doc(snapshot):
|
|||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def contract_doc(contract):
|
def contract_doc(contract):
|
||||||
return copy.deepcopy(dict(contract.raw))
|
return contract.to_dict()
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------
|
# --------------------------------------------------------------------------
|
||||||
@@ -746,7 +746,7 @@ def test_the_contract_locks_the_roadmap_thresholds(contract):
|
|||||||
assert contract.threshold("performance", "quality_pass_with_speed_fail_verdict") == "stop"
|
assert contract.threshold("performance", "quality_pass_with_speed_fail_verdict") == "stop"
|
||||||
|
|
||||||
assert contract.threshold("reliability", "synthetic_workers_satisfy_alpha") is False
|
assert contract.threshold("reliability", "synthetic_workers_satisfy_alpha") is False
|
||||||
assert contract.threshold("storage", "forbidden_path_prefixes") == ["/home"]
|
assert contract.threshold("storage", "forbidden_path_prefixes") == ("/home",)
|
||||||
|
|
||||||
|
|
||||||
def test_the_contract_offers_only_alpha_or_stop(contract):
|
def test_the_contract_offers_only_alpha_or_stop(contract):
|
||||||
@@ -820,17 +820,26 @@ def test_a_dropped_acceptance_section_is_rejected(contract_doc):
|
|||||||
parse_alpha_contract(contract_doc)
|
parse_alpha_contract(contract_doc)
|
||||||
|
|
||||||
|
|
||||||
def test_resealing_a_mutated_contract_changes_its_digest(contract, contract_doc):
|
def test_resealing_a_mutated_v1_contract_is_rejected(contract, contract_doc):
|
||||||
"""Tamper-evidence, not tamper-proofing: a rewrite is legal but never silent."""
|
|
||||||
contract_doc["performance"]["min_median_decode_tokens_per_second"] = 0.05
|
contract_doc["performance"]["min_median_decode_tokens_per_second"] = 0.05
|
||||||
resealed = seal_contract(contract_doc)
|
resealed = seal_contract(contract_doc)
|
||||||
|
|
||||||
# It parses — nothing in-repo can stop a determined rewrite of file plus digest.
|
with pytest.raises(AlphaContractError, match="trusted pre-execution digest"):
|
||||||
reparsed = parse_alpha_contract(resealed)
|
parse_alpha_contract(resealed)
|
||||||
|
assert resealed["contract_sha256"] != contract.digest
|
||||||
|
|
||||||
# But the digest moved, so the change is a visible diff on a file whose entire
|
|
||||||
# purpose is to not change, and the contract_id still claims to be v1.
|
def test_parsed_contract_nested_state_is_immutable(contract):
|
||||||
assert reparsed.digest != contract.digest
|
with pytest.raises(TypeError):
|
||||||
|
contract.raw["performance"]["min_median_decode_tokens_per_second"] = 0.05
|
||||||
|
with pytest.raises(TypeError):
|
||||||
|
contract.target["reasoning_effort"] = "high"
|
||||||
|
|
||||||
|
|
||||||
|
def test_contract_to_dict_returns_an_isolated_mutable_copy(contract):
|
||||||
|
copied = contract.to_dict()
|
||||||
|
copied["performance"]["min_median_decode_tokens_per_second"] = 0.05
|
||||||
|
assert contract.threshold("performance", "min_median_decode_tokens_per_second") == 0.5
|
||||||
|
|
||||||
|
|
||||||
def test_an_unknown_threshold_cannot_be_invented_at_read_time(contract):
|
def test_an_unknown_threshold_cannot_be_invented_at_read_time(contract):
|
||||||
|
|||||||
@@ -5,8 +5,7 @@ from __future__ import annotations
|
|||||||
import json
|
import json
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from unittest.mock import MagicMock
|
||||||
from unittest.mock import MagicMock, patch
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -277,7 +276,6 @@ def test_relay_server_peer_list_grows_on_connect():
|
|||||||
def test_relay_circuit_relay_proxies_message():
|
def test_relay_circuit_relay_proxies_message():
|
||||||
"A node behind NAT (client_a) receives a message via circuit relay from client_b.\n\nTags: gossip, network, relay"
|
"A node behind NAT (client_a) receives a message via circuit relay from client_b.\n\nTags: gossip, network, relay"
|
||||||
|
|
||||||
import websockets.sync.client # type: ignore[import]
|
|
||||||
from meshnet_relay.server import RelayServer
|
from meshnet_relay.server import RelayServer
|
||||||
|
|
||||||
relay = RelayServer(host="127.0.0.1", port=0)
|
relay = RelayServer(host="127.0.0.1", port=0)
|
||||||
@@ -755,7 +753,6 @@ def test_node_relay_bridge_reconnects_after_failed_connection(monkeypatch):
|
|||||||
|
|
||||||
def _start_tracker_and_register(extra_fields: dict) -> dict:
|
def _start_tracker_and_register(extra_fields: dict) -> dict:
|
||||||
"""Helper: start tracker, register node with extra gossip fields, return response."""
|
"""Helper: start tracker, register node with extra gossip fields, return response."""
|
||||||
import http.server
|
|
||||||
import json as _json
|
import json as _json
|
||||||
import urllib.request
|
import urllib.request
|
||||||
|
|
||||||
@@ -766,7 +763,7 @@ def _start_tracker_and_register(extra_fields: dict) -> dict:
|
|||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
|
|
||||||
payload = {
|
payload = {
|
||||||
"endpoint": f"http://127.0.0.1:8001",
|
"endpoint": "http://127.0.0.1:8001",
|
||||||
"shard_start": 0,
|
"shard_start": 0,
|
||||||
"shard_end": 7,
|
"shard_end": 7,
|
||||||
"model": "stub-model",
|
"model": "stub-model",
|
||||||
|
|||||||
68
tests/test_llama_cpp_dependency.py
Normal file
68
tests/test_llama_cpp_dependency.py
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
"""Offline guards for DGR-004's pinned llama.cpp dependency boundary."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import pathlib
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||||
|
LLAMA_DIR = ROOT / "packages/node/native/llama"
|
||||||
|
SCRIPT = ROOT / "scripts/llama_cpp_dependency.py"
|
||||||
|
|
||||||
|
|
||||||
|
def _sha256(path: pathlib.Path) -> str:
|
||||||
|
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def test_lock_and_patch_manifest_are_self_consistent_and_exact() -> None:
|
||||||
|
lock = json.loads((LLAMA_DIR / "UPSTREAM_LOCK.json").read_text())
|
||||||
|
commit = (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
|
||||||
|
patches = (LLAMA_DIR / "patches/series").read_text().splitlines()
|
||||||
|
sums = {
|
||||||
|
name: digest
|
||||||
|
for digest, name in (
|
||||||
|
line.split(maxsplit=1)
|
||||||
|
for line in (LLAMA_DIR / "patches/SHA256SUMS").read_text().splitlines()
|
||||||
|
if line and not line.startswith("#")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert commit == lock["commit"]
|
||||||
|
assert len(commit) == 40
|
||||||
|
assert patches == lock["patch_series"]
|
||||||
|
assert patches == sorted(patches)
|
||||||
|
assert patches
|
||||||
|
for patch_name in patches:
|
||||||
|
patch = LLAMA_DIR / "patches" / patch_name
|
||||||
|
assert sums[patch_name] == _sha256(patch)
|
||||||
|
assert "Subject: [PATCH" in patch.read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def test_dependency_script_reports_the_locked_boundary_without_network() -> None:
|
||||||
|
completed = subprocess.run(
|
||||||
|
[sys.executable, str(SCRIPT), "inspect"],
|
||||||
|
cwd=ROOT,
|
||||||
|
check=True,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
report = json.loads(completed.stdout)
|
||||||
|
|
||||||
|
assert report["commit"] == (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
|
||||||
|
assert report["patch_count"] == 2
|
||||||
|
assert report["model_downloads"] is False
|
||||||
|
assert report["semantic_certification"] is False
|
||||||
|
assert "dense" in report["glm_stock_limitations"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_patch_stack_does_not_contain_meshnet_control_plane_code() -> None:
|
||||||
|
forbidden = ("tracker", "route session", "grpc", "http", "billing", "wallet")
|
||||||
|
patch_text = "\n".join(
|
||||||
|
(LLAMA_DIR / "patches" / name).read_text().lower()
|
||||||
|
for name in (LLAMA_DIR / "patches/series").read_text().splitlines()
|
||||||
|
)
|
||||||
|
assert not any(term in patch_text for term in forbidden)
|
||||||
@@ -5,8 +5,6 @@ from __future__ import annotations
|
|||||||
import json
|
import json
|
||||||
import socket
|
import socket
|
||||||
import sys
|
import sys
|
||||||
import types
|
|
||||||
from pathlib import Path
|
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
# A fake node server has no real backend to prove capability with; say so
|
# A fake node server has no real backend to prove capability with; say so
|
||||||
@@ -134,7 +132,6 @@ def test_print_models_table_runs_without_error(capsys, monkeypatch):
|
|||||||
def test_wizard_writes_config_on_happy_path(tmp_path, monkeypatch):
|
def test_wizard_writes_config_on_happy_path(tmp_path, monkeypatch):
|
||||||
"Wizard writes config on happy path\n\nTags: general"
|
"Wizard writes config on happy path\n\nTags: general"
|
||||||
from meshnet_node import wizard as wiz
|
from meshnet_node import wizard as wiz
|
||||||
from meshnet_node.config import load_config, save_config
|
|
||||||
|
|
||||||
# Fake GPU
|
# Fake GPU
|
||||||
gpus = [{"index": 0, "name": "RTX 4090", "vram_gb": 24.0, "backend": "cuda"}]
|
gpus = [{"index": 0, "name": "RTX 4090", "vram_gb": 24.0, "backend": "cuda"}]
|
||||||
@@ -265,7 +262,6 @@ def test_config_command_no_config_exits_1(tmp_path, monkeypatch):
|
|||||||
|
|
||||||
def test_config_command_prints_saved_config(tmp_path, monkeypatch, capsys):
|
def test_config_command_prints_saved_config(tmp_path, monkeypatch, capsys):
|
||||||
"Config command prints saved config\n\nTags: general"
|
"Config command prints saved config\n\nTags: general"
|
||||||
from meshnet_node import config as cfg_mod
|
|
||||||
from meshnet_node.config import save_config
|
from meshnet_node.config import save_config
|
||||||
from meshnet_node.cli import main
|
from meshnet_node.cli import main
|
||||||
|
|
||||||
@@ -309,7 +305,6 @@ def test_detect_num_layers_returns_none_on_error(monkeypatch):
|
|||||||
def test_startup_auto_detects_shard_range(monkeypatch, tmp_path):
|
def test_startup_auto_detects_shard_range(monkeypatch, tmp_path):
|
||||||
"When shard_start/end are None, startup reads layer count from catalog.\n\nTags: general"
|
"When shard_start/end are None, startup reads layer count from catalog.\n\nTags: general"
|
||||||
from meshnet_node import startup as su
|
from meshnet_node import startup as su
|
||||||
from meshnet_node.model_catalog import detect_num_layers
|
|
||||||
|
|
||||||
calls = []
|
calls = []
|
||||||
|
|
||||||
|
|||||||
157
tests/test_native_identity_emission.py
Normal file
157
tests/test_native_identity_emission.py
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
"""DGR-003 production-native identity emission boundary tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from meshnet_node.doctor import DoctorSelection, validate_loaded_backend
|
||||||
|
from meshnet_node.native_backend import (
|
||||||
|
ImmutableArtifactPin,
|
||||||
|
NativeIdentityInputs,
|
||||||
|
NativeLoadedArtifactReport,
|
||||||
|
NativeNumericalRecipe,
|
||||||
|
NativeSessionRejected,
|
||||||
|
NativeWorkerBackendAdapter,
|
||||||
|
shard_identity_from_native_report,
|
||||||
|
)
|
||||||
|
from meshnet_node.native_protocol import SCHEMA_VERSION, pb
|
||||||
|
from meshnet_node.recipe_manifest import parse_recipe_manifest
|
||||||
|
from meshnet_tracker.capability import STATE_UNCERTIFIED, evaluate_report
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(letter: str) -> str:
|
||||||
|
return letter * 64
|
||||||
|
|
||||||
|
|
||||||
|
def _inputs(**changes: object) -> NativeIdentityInputs:
|
||||||
|
report = NativeLoadedArtifactReport(
|
||||||
|
owned_start_layer=2,
|
||||||
|
owned_end_layer=6,
|
||||||
|
mapped_bytes=1024,
|
||||||
|
resident_bytes=768,
|
||||||
|
registered_bytes=640,
|
||||||
|
architecture="llama",
|
||||||
|
architecture_digest=_digest("a"),
|
||||||
|
layer_count=8,
|
||||||
|
)
|
||||||
|
recipe = NativeNumericalRecipe(
|
||||||
|
weight_quantization="Q4_K_M",
|
||||||
|
activation_dtype="bfloat16",
|
||||||
|
compute_dtype="float32",
|
||||||
|
kv_dtype="q8_0",
|
||||||
|
kv_layout="llama-kv-v1",
|
||||||
|
architecture_adapter="dense-llama-v1",
|
||||||
|
backend_id="llama-cpp",
|
||||||
|
runtime_version="llama.cpp:e920c523",
|
||||||
|
recipe_id="native",
|
||||||
|
recipe_version="1",
|
||||||
|
catalogue_version="2026.07.1",
|
||||||
|
)
|
||||||
|
values: dict[str, object] = {
|
||||||
|
"loaded_artifact": report,
|
||||||
|
"artifact_pin": ImmutableArtifactPin(
|
||||||
|
artifact_id="acme/llama.gguf",
|
||||||
|
revision="0123456789abcdef",
|
||||||
|
content_digest=_digest("b"),
|
||||||
|
),
|
||||||
|
"tokenizer_revision": "abcdef0123456789",
|
||||||
|
"numerical_recipe": recipe,
|
||||||
|
}
|
||||||
|
values.update(changes)
|
||||||
|
return NativeIdentityInputs(**values) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def _open(adapter: NativeWorkerBackendAdapter, **changes: object) -> pb.SessionOpen:
|
||||||
|
identity = adapter.identity
|
||||||
|
fields: dict[str, object] = {
|
||||||
|
"schema_version": SCHEMA_VERSION,
|
||||||
|
"route_session_id": "tracker-session",
|
||||||
|
"route_epoch": 4,
|
||||||
|
"fingerprint": identity.fingerprint.to_proto(),
|
||||||
|
"shard_range": pb.ShardRange(
|
||||||
|
start_layer=identity.shard_start,
|
||||||
|
end_layer=identity.shard_end,
|
||||||
|
effective_start_layer=identity.shard_start,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
fields.update(changes)
|
||||||
|
return pb.SessionOpen(**fields) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_native_identity_uses_loaded_report_not_a_caller_range():
|
||||||
|
identity = shard_identity_from_native_report(_inputs())
|
||||||
|
|
||||||
|
assert (identity.shard_start, identity.shard_end) == (2, 6)
|
||||||
|
assert identity.artifact.architecture == "llama"
|
||||||
|
assert identity.artifact.layer_count == 8
|
||||||
|
|
||||||
|
|
||||||
|
def test_native_identity_requires_an_immutable_pin_and_gguf_range():
|
||||||
|
with pytest.raises(Exception, match="moving reference"):
|
||||||
|
shard_identity_from_native_report(
|
||||||
|
_inputs(artifact_pin=ImmutableArtifactPin("a", "main", _digest("b")))
|
||||||
|
)
|
||||||
|
with pytest.raises(Exception, match="outside GGUF"):
|
||||||
|
NativeLoadedArtifactReport(0, 9, 1, 1, 1, "llama", _digest("a"), 8)
|
||||||
|
|
||||||
|
|
||||||
|
def test_native_worker_rejects_bad_session_open_before_session_acceptance():
|
||||||
|
adapter = NativeWorkerBackendAdapter(_inputs())
|
||||||
|
accepted = adapter.on_session_open(
|
||||||
|
_open(adapter), expected_route_session_id="tracker-session", expected_route_epoch=4
|
||||||
|
)
|
||||||
|
assert accepted.fingerprint.SerializeToString() == adapter.identity.fingerprint.to_proto().SerializeToString()
|
||||||
|
|
||||||
|
with pytest.raises(NativeSessionRejected) as rejected:
|
||||||
|
adapter.on_session_open(
|
||||||
|
_open(adapter, route_epoch=5),
|
||||||
|
expected_route_session_id="tracker-session",
|
||||||
|
expected_route_epoch=4,
|
||||||
|
)
|
||||||
|
assert rejected.value.error.code == pb.ERROR_CODE_EPOCH_STALE
|
||||||
|
|
||||||
|
|
||||||
|
def test_doctor_emits_native_identity_but_keeps_legacy_backend_dark():
|
||||||
|
manifest = parse_recipe_manifest(
|
||||||
|
{"schema_version": 1, "catalogue_version": "2026.07.1", "recipes": [
|
||||||
|
{"id": "native", "version": "1", "backend_id": "llama-cpp"}
|
||||||
|
]}
|
||||||
|
)
|
||||||
|
selection = DoctorSelection("acme/llama.gguf", 2, 5)
|
||||||
|
native = NativeWorkerBackendAdapter(_inputs())
|
||||||
|
# The probe needs only the normal backend shape; identity is supplied by the adapter.
|
||||||
|
native.hidden_size = 8
|
||||||
|
native.is_head = False
|
||||||
|
native.is_tail = False
|
||||||
|
native.device = "cpu"
|
||||||
|
native.forward_bytes = lambda *args, **kwargs: type("Payload", (), {"body": b"x", "shape": [1]})()
|
||||||
|
result = validate_loaded_backend(native, selection, manifest.recipes[0], manifest)
|
||||||
|
assert result.report.identity == native.identity
|
||||||
|
assert result.report.model.revision == native.identity.artifact.revision
|
||||||
|
assert result.report.model.config_fingerprint == "sha256:" + _digest("a")
|
||||||
|
assert result.report.backend.quantization == "Q4_K_M"
|
||||||
|
assert (result.report.shard.start, result.report.shard.end) == (2, 5)
|
||||||
|
state = evaluate_report(
|
||||||
|
result.report.to_dict(),
|
||||||
|
model_matches=lambda value: value == "acme/llama.gguf",
|
||||||
|
advertised_model="acme/llama.gguf",
|
||||||
|
shard_start=2,
|
||||||
|
shard_end=5,
|
||||||
|
declared_recipe_id="native",
|
||||||
|
declared_recipe_version="1",
|
||||||
|
now=result.report.validated_at,
|
||||||
|
)
|
||||||
|
assert state.state == STATE_UNCERTIFIED
|
||||||
|
|
||||||
|
class Legacy:
|
||||||
|
hidden_size = 8
|
||||||
|
is_head = False
|
||||||
|
is_tail = False
|
||||||
|
device = "cpu"
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def forward_bytes(*args, **kwargs):
|
||||||
|
return type("Payload", (), {"body": b"x", "shape": [1]})()
|
||||||
|
|
||||||
|
legacy = validate_loaded_backend(Legacy(), selection, manifest.recipes[0], manifest)
|
||||||
|
assert legacy.report.identity is None
|
||||||
@@ -34,6 +34,8 @@ from meshnet_node.native_protocol import (
|
|||||||
ProtocolError,
|
ProtocolError,
|
||||||
checksum_of,
|
checksum_of,
|
||||||
decode_bundle,
|
decode_bundle,
|
||||||
|
decode_step_bundle,
|
||||||
|
encode_decode_step,
|
||||||
decode_tensor,
|
decode_tensor,
|
||||||
default_flow_control,
|
default_flow_control,
|
||||||
encode_bundle,
|
encode_bundle,
|
||||||
@@ -429,7 +431,7 @@ def test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk():
|
|||||||
idempotency_step=9,
|
idempotency_step=9,
|
||||||
position=1024,
|
position=1024,
|
||||||
expected_past_len=1024,
|
expected_past_len=1024,
|
||||||
tensor=tensor,
|
bundle=encode_bundle([tensor]),
|
||||||
work_id="work-7f3a",
|
work_id="work-7f3a",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
@@ -443,7 +445,48 @@ def test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk():
|
|||||||
)
|
)
|
||||||
|
|
||||||
assert len(fast.SerializeToString()) * 2 < len(full.SerializeToString())
|
assert len(fast.SerializeToString()) * 2 < len(full.SerializeToString())
|
||||||
assert decode_tensor(fast.decode.tensor) == hidden
|
assert decode_step_bundle(fast.decode) == {HIDDEN_STATES: hidden}
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_fast_path_preserves_legacy_one_tensor_compatibility():
|
||||||
|
tensor = encode_tensor(HIDDEN_STATES, b"\x01\x02" * 8, [1, 1, 8], pb.DTYPE_BFLOAT16)
|
||||||
|
legacy = pb.DecodeStep(tensor=tensor)
|
||||||
|
|
||||||
|
assert decode_step_bundle(legacy) == {HIDDEN_STATES: b"\x01\x02" * 8}
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_bundle_wins_over_legacy_tensor_and_can_carry_sidebands():
|
||||||
|
hidden = encode_tensor(HIDDEN_STATES, b"\x01\x02" * 8, [1, 1, 8], pb.DTYPE_BFLOAT16)
|
||||||
|
sideband = encode_tensor("index_topk", b"\x00" * 4, [1], pb.DTYPE_INT32)
|
||||||
|
decode = pb.DecodeStep(tensor=hidden, bundle=encode_bundle([hidden, sideband]))
|
||||||
|
|
||||||
|
assert decode_step_bundle(decode) == {
|
||||||
|
HIDDEN_STATES: b"\x01\x02" * 8,
|
||||||
|
"index_topk": b"\x00" * 4,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_writer_uses_compact_tensor_only_for_a_certified_one_tensor_bundle():
|
||||||
|
hidden = encode_tensor(HIDDEN_STATES, b"\x01\x02" * 8, [1, 1, 8], pb.DTYPE_BFLOAT16)
|
||||||
|
compact = encode_decode_step(
|
||||||
|
encode_bundle([hidden]), idempotency_step=1, position=1, expected_past_len=1, work_id="w"
|
||||||
|
)
|
||||||
|
sideband = encode_tensor("index_topk", b"\x00" * 4, [1], pb.DTYPE_INT32)
|
||||||
|
expanded = encode_decode_step(
|
||||||
|
encode_bundle([hidden, sideband]), idempotency_step=1, position=1, expected_past_len=1, work_id="w"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert compact.HasField("tensor") and not compact.HasField("bundle")
|
||||||
|
assert expanded.HasField("bundle") and not expanded.HasField("tensor")
|
||||||
|
|
||||||
|
|
||||||
|
def test_schema_exposes_typed_tail_result_and_bound_sampling_identity():
|
||||||
|
assert {"identity", "sampling", "logits", "sampled_token_id"} <= set(
|
||||||
|
pb.TailResult.DESCRIPTOR.fields_by_name
|
||||||
|
)
|
||||||
|
assert {"request_id", "runtime_recipe_digest", "chat_template_id", "chat_template_version", "reasoning_mode"} <= set(
|
||||||
|
pb.RequestRecipeIdentity.DESCRIPTOR.fields_by_name
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_flow_control_defaults_bound_the_queue_and_the_message():
|
def test_flow_control_defaults_bound_the_queue_and_the_message():
|
||||||
@@ -491,6 +534,20 @@ def test_committed_vectors_still_encode_as_promised():
|
|||||||
report = (conformance.TESTDATA_DIR / conformance.GOLDEN_CAPABILITY_REPORT).read_bytes()
|
report = (conformance.TESTDATA_DIR / conformance.GOLDEN_CAPABILITY_REPORT).read_bytes()
|
||||||
assert conformance.serialize(conformance.canonical_capability_report()) == report
|
assert conformance.serialize(conformance.canonical_capability_report()) == report
|
||||||
|
|
||||||
|
decode = (conformance.TESTDATA_DIR / conformance.GOLDEN_DECODE_STEP).read_bytes()
|
||||||
|
assert conformance.serialize(conformance.canonical_decode_step()) == decode
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_golden_preserves_the_multi_tensor_boundary():
|
||||||
|
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_DECODE_STEP).read_bytes()
|
||||||
|
request = pb.SessionRequest.FromString(golden)
|
||||||
|
|
||||||
|
assert request.decode.idempotency_step == 43
|
||||||
|
assert decode_step_bundle(request.decode) == {
|
||||||
|
HIDDEN_STATES: bytes(range(16)),
|
||||||
|
"index_topk": (3).to_bytes(4, "little"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def test_golden_session_request_round_trips_with_every_field_intact():
|
def test_golden_session_request_round_trips_with_every_field_intact():
|
||||||
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes()
|
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes()
|
||||||
|
|||||||
@@ -162,7 +162,7 @@ def test_streaming_end_to_end_http(two_node_setup):
|
|||||||
assert "text/event-stream" in content_type
|
assert "text/event-stream" in content_type
|
||||||
raw = resp.read().decode()
|
raw = resp.read().decode()
|
||||||
|
|
||||||
data_lines = [l for l in raw.strip().splitlines() if l.startswith("data: ")]
|
data_lines = [line for line in raw.strip().splitlines() if line.startswith("data: ")]
|
||||||
assert data_lines, "No SSE data lines found"
|
assert data_lines, "No SSE data lines found"
|
||||||
assert data_lines[-1] == "data: [DONE]"
|
assert data_lines[-1] == "data: [DONE]"
|
||||||
|
|
||||||
|
|||||||
@@ -13,7 +13,6 @@ import urllib.request
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from meshnet_node.model_backend import (
|
from meshnet_node.model_backend import (
|
||||||
InsufficientVRAMError,
|
|
||||||
PartialModelLoadUnsupported,
|
PartialModelLoadUnsupported,
|
||||||
KVCacheMiss,
|
KVCacheMiss,
|
||||||
TensorPayload,
|
TensorPayload,
|
||||||
|
|||||||
786
tests/test_runtime_recipe_identity.py
Normal file
786
tests/test_runtime_recipe_identity.py
Normal file
@@ -0,0 +1,786 @@
|
|||||||
|
"""Deterministic DGR-003 identity and tracker-admission conformance tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import replace
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import time
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from meshnet_node.native_protocol import SCHEMA_VERSION, pb
|
||||||
|
from meshnet_node.runtime_recipe import (
|
||||||
|
ArtifactIdentity,
|
||||||
|
CompatibilityFingerprint,
|
||||||
|
DerivativeBinding,
|
||||||
|
RecipeIdentityError,
|
||||||
|
RuntimeRecipe,
|
||||||
|
ShardIdentity,
|
||||||
|
check_handshake,
|
||||||
|
check_session_open,
|
||||||
|
check_route,
|
||||||
|
handshake_error,
|
||||||
|
)
|
||||||
|
from meshnet_tracker.capability import (
|
||||||
|
POLICY_COMPAT,
|
||||||
|
POLICY_ENFORCE,
|
||||||
|
CapabilityState,
|
||||||
|
STATE_ADMITTED,
|
||||||
|
STATE_FINGERPRINT_MISMATCH,
|
||||||
|
STATE_MODEL_MISMATCH,
|
||||||
|
STATE_RECIPE_MISMATCH,
|
||||||
|
STATE_UNCERTIFIED,
|
||||||
|
absent_state,
|
||||||
|
evaluate_report,
|
||||||
|
)
|
||||||
|
from meshnet_tracker.server import (
|
||||||
|
TrackerServer,
|
||||||
|
_capability_from_registration,
|
||||||
|
_find_pinned_route,
|
||||||
|
_NodeEntry,
|
||||||
|
_select_route,
|
||||||
|
)
|
||||||
|
from meshnet_tracker.recipe import (
|
||||||
|
CertificationLedger,
|
||||||
|
DistributedForwardEvidence,
|
||||||
|
RecipeIdentityError as TrackerRecipeIdentityError,
|
||||||
|
parse_identity,
|
||||||
|
)
|
||||||
|
|
||||||
|
VECTORS = Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json"
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(char: str) -> str:
|
||||||
|
return char * 64
|
||||||
|
|
||||||
|
|
||||||
|
def _recipe(**changes: object) -> RuntimeRecipe:
|
||||||
|
fields: dict[str, object] = {
|
||||||
|
"weight_quantization": "Q4_K_M",
|
||||||
|
"activation_dtype": "bfloat16",
|
||||||
|
"compute_dtype": "float32",
|
||||||
|
"kv_dtype": "q8_0",
|
||||||
|
"kv_layout": "paged-v1",
|
||||||
|
"tokenizer_revision": "0123456789abcdef",
|
||||||
|
"architecture_adapter": "llama/range-v1",
|
||||||
|
"backend_id": "llama.cpp",
|
||||||
|
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
|
||||||
|
"recipe_id": "example-gguf",
|
||||||
|
"recipe_version": "1",
|
||||||
|
"catalogue_version": "2026.07.1",
|
||||||
|
}
|
||||||
|
fields.update(changes)
|
||||||
|
return RuntimeRecipe(**fields) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def _identity(start: int = 0, end: int = 4, **recipe_changes: object) -> ShardIdentity:
|
||||||
|
"""A Shard of the whole-model artifact: every node holds the same file."""
|
||||||
|
return ShardIdentity(
|
||||||
|
ArtifactIdentity(
|
||||||
|
"example/model",
|
||||||
|
"0123456789abcdef",
|
||||||
|
_digest("a"),
|
||||||
|
"dense-llama",
|
||||||
|
_digest("b"),
|
||||||
|
8,
|
||||||
|
),
|
||||||
|
_recipe(**recipe_changes),
|
||||||
|
start,
|
||||||
|
end,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _split(start: int, end: int, content: str, **recipe_changes: object) -> ShardIdentity:
|
||||||
|
"""A derivative: its own bytes, bound to the exact source it was cut from."""
|
||||||
|
return ShardIdentity(
|
||||||
|
ArtifactIdentity(
|
||||||
|
"example/model",
|
||||||
|
"0123456789abcdef",
|
||||||
|
_digest(content),
|
||||||
|
"dense-llama",
|
||||||
|
_digest("b"),
|
||||||
|
8,
|
||||||
|
DerivativeBinding(_digest("a"), start, end),
|
||||||
|
),
|
||||||
|
_recipe(**recipe_changes),
|
||||||
|
start,
|
||||||
|
end,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _report(identity: ShardIdentity) -> dict:
|
||||||
|
return {
|
||||||
|
"schema_version": 1,
|
||||||
|
"model": {
|
||||||
|
"model_id": "example/model",
|
||||||
|
"revision": identity.artifact.revision,
|
||||||
|
"config_fingerprint": "sha256:" + identity.artifact.architecture_digest,
|
||||||
|
},
|
||||||
|
"shard": {"start": identity.shard_start, "end": identity.shard_end - 1},
|
||||||
|
"recipe": identity.recipe.to_dict() | {
|
||||||
|
"recipe_id": identity.recipe.recipe_id,
|
||||||
|
"recipe_version": identity.recipe.recipe_version,
|
||||||
|
"catalogue_version": identity.recipe.catalogue_version,
|
||||||
|
},
|
||||||
|
"backend": {"backend_id": "llama.cpp", "device": "test"},
|
||||||
|
"status": "passed",
|
||||||
|
"validated_at": 100.0,
|
||||||
|
"duration_ms": 1,
|
||||||
|
"diagnostics": [],
|
||||||
|
"identity": identity.to_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _evaluate(report: dict, **kwargs: object):
|
||||||
|
kwargs.setdefault("shard_start", 0)
|
||||||
|
kwargs.setdefault("shard_end", 3)
|
||||||
|
return evaluate_report(
|
||||||
|
report,
|
||||||
|
model_matches=lambda model: model == "example/model",
|
||||||
|
advertised_model="example/model",
|
||||||
|
now=100.0,
|
||||||
|
**kwargs, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _register(tracker: TrackerServer, node_id: str, identity: ShardIdentity) -> _NodeEntry:
|
||||||
|
"""Register one node exactly as the HTTP path does: its own report, the tracker's ledger."""
|
||||||
|
report = _report(identity)
|
||||||
|
report["validated_at"] = time.time()
|
||||||
|
# The registry range is end-inclusive; the identity's is end-exclusive.
|
||||||
|
start, end = identity.shard_start, identity.shard_end - 1
|
||||||
|
capability = _capability_from_registration(
|
||||||
|
{"capability_report": report},
|
||||||
|
model="example/model",
|
||||||
|
hf_repo=None,
|
||||||
|
shard_start=start,
|
||||||
|
shard_end=end,
|
||||||
|
recipe_certifications=tracker._recipe_certifications,
|
||||||
|
)
|
||||||
|
entry = _NodeEntry(
|
||||||
|
node_id=node_id,
|
||||||
|
endpoint=f"http://{node_id}",
|
||||||
|
shard_start=start,
|
||||||
|
shard_end=end,
|
||||||
|
model="example/model",
|
||||||
|
shard_checksum=None,
|
||||||
|
hardware_profile={},
|
||||||
|
wallet_address=None,
|
||||||
|
score=1.0,
|
||||||
|
capability=capability,
|
||||||
|
)
|
||||||
|
tracker._registry[node_id] = entry
|
||||||
|
return entry
|
||||||
|
|
||||||
|
|
||||||
|
def _evidence(
|
||||||
|
*shards: ShardIdentity,
|
||||||
|
node_ids: tuple[str, ...] = ("physical-a", "physical-b"),
|
||||||
|
layer_count: int = 8,
|
||||||
|
fingerprint: tuple[str, str] | None = None,
|
||||||
|
**changes: object,
|
||||||
|
) -> DistributedForwardEvidence:
|
||||||
|
presented = tuple(parse_identity(s.to_dict()) for s in shards)
|
||||||
|
fields: dict[str, object] = {
|
||||||
|
"route_session_id": "session",
|
||||||
|
"route_epoch": 1,
|
||||||
|
"node_ids": node_ids,
|
||||||
|
"shard_ranges": tuple((s.shard_start, s.shard_end) for s in shards),
|
||||||
|
"tokens_generated": 1,
|
||||||
|
"layer_count": layer_count,
|
||||||
|
"fingerprint": fingerprint or presented[0].key,
|
||||||
|
"participants": presented,
|
||||||
|
}
|
||||||
|
fields.update(changes)
|
||||||
|
return DistributedForwardEvidence(**fields) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
# --- Identity: the axes, the digests, and what they do and do not commit to ---
|
||||||
|
|
||||||
|
|
||||||
|
def test_node_and_tracker_share_the_committed_canonical_fingerprint_vector():
|
||||||
|
for vector in json.loads(VECTORS.read_text(encoding="utf-8"))["vectors"]:
|
||||||
|
expected = vector["fingerprint"]
|
||||||
|
identity = ShardIdentity.from_dict(vector["identity"])
|
||||||
|
presented = parse_identity(vector["identity"])
|
||||||
|
|
||||||
|
assert identity.fingerprint.to_dict() == expected, vector["name"]
|
||||||
|
assert presented.fingerprint_dict() == expected, vector["name"]
|
||||||
|
assert (
|
||||||
|
CompatibilityFingerprint.from_proto(
|
||||||
|
identity.fingerprint.to_proto()
|
||||||
|
).to_dict()
|
||||||
|
== expected
|
||||||
|
), vector["name"]
|
||||||
|
|
||||||
|
# The binding digest is derived independently on both sides too, so a
|
||||||
|
# node and a tracker cannot disagree about which bytes a Shard holds.
|
||||||
|
assert identity.shard_binding_digest == vector["shard_binding_digest"], vector["name"]
|
||||||
|
assert presented.shard_binding_digest == vector["shard_binding_digest"], vector["name"]
|
||||||
|
|
||||||
|
# The DGR-002 wire encoding is contract as well, so the native worker can
|
||||||
|
# be held to these bytes without reimplementing the JSON canonicalizer.
|
||||||
|
wire = identity.fingerprint.to_proto().SerializeToString(deterministic=True)
|
||||||
|
assert wire.hex() == vector["fingerprint_proto_hex"], vector["name"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_committed_vectors_cover_a_whole_model_and_a_derivative_shard():
|
||||||
|
names = {v["name"] for v in json.loads(VECTORS.read_text(encoding="utf-8"))["vectors"]}
|
||||||
|
assert {"example-v1", "example-v1-derivative"} <= names
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"axis,value",
|
||||||
|
[
|
||||||
|
("weight_quantization", "Q5_K_M"),
|
||||||
|
("activation_dtype", "float16"),
|
||||||
|
("compute_dtype", "float16"),
|
||||||
|
("kv_dtype", "float16"),
|
||||||
|
("kv_layout", "contiguous-v2"),
|
||||||
|
("tokenizer_revision", "fedcba9876543210"),
|
||||||
|
("architecture_adapter", "llama/range-v2"),
|
||||||
|
("backend_id", "other-backend"),
|
||||||
|
("runtime_version", "llama.cpp@other+meshnet.1"),
|
||||||
|
("boundary_schema_version", 2),
|
||||||
|
("protocol_schema_version", 2),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_every_recipe_axis_changes_the_fingerprint_and_blocks_route(axis, value):
|
||||||
|
left = _identity()
|
||||||
|
right = _identity(4, 8, **{axis: value})
|
||||||
|
|
||||||
|
assert left.fingerprint.key != right.fingerprint.key
|
||||||
|
assert check_route([left, right])
|
||||||
|
|
||||||
|
|
||||||
|
def test_split_artifact_is_bound_to_exact_source_and_owned_range():
|
||||||
|
source = _identity()
|
||||||
|
split = _split(4, 8, "c")
|
||||||
|
|
||||||
|
assert source.fingerprint.matches(split.fingerprint)
|
||||||
|
with pytest.raises(RecipeIdentityError):
|
||||||
|
ShardIdentity(split.artifact, split.recipe, 3, 8)
|
||||||
|
|
||||||
|
|
||||||
|
def test_derivative_bytes_and_range_have_a_separate_shard_binding_digest():
|
||||||
|
left = _split(0, 4, "c")
|
||||||
|
changed_bytes = _split(0, 4, "d")
|
||||||
|
changed_range = _split(4, 8, "c")
|
||||||
|
|
||||||
|
# Same route fingerprint — all three are the same recipe on the same source,
|
||||||
|
# which is exactly what route formation should conclude.
|
||||||
|
assert left.fingerprint.key == changed_bytes.fingerprint.key
|
||||||
|
assert left.fingerprint.key == changed_range.fingerprint.key
|
||||||
|
# Different bytes and different ranges are still separately pinned.
|
||||||
|
assert left.shard_binding_digest != changed_bytes.shard_binding_digest
|
||||||
|
assert left.shard_binding_digest != changed_range.shard_binding_digest
|
||||||
|
assert (
|
||||||
|
parse_identity(left.to_dict()).shard_binding_digest
|
||||||
|
== left.shard_binding_digest
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_declared_digest_mismatch_is_recomputed_and_rejected_not_authenticated():
|
||||||
|
report = _report(_identity())
|
||||||
|
report["identity"]["fingerprint"]["runtime_recipe_digest"] = _digest("f")
|
||||||
|
|
||||||
|
assert _evaluate(report).state == STATE_FINGERPRINT_MISMATCH
|
||||||
|
|
||||||
|
|
||||||
|
# --- Capability admission: the identity block must match the proof it rides with ---
|
||||||
|
|
||||||
|
|
||||||
|
def test_capability_identity_must_match_the_proof_labels():
|
||||||
|
identity = _identity()
|
||||||
|
|
||||||
|
wrong_model = _report(identity)
|
||||||
|
wrong_model["identity"]["artifact"]["artifact_id"] = "other/model"
|
||||||
|
wrong_model["identity"]["fingerprint"] = None
|
||||||
|
assert _evaluate(wrong_model).state == STATE_MODEL_MISMATCH
|
||||||
|
|
||||||
|
wrong_recipe = _report(identity)
|
||||||
|
wrong_recipe["recipe"]["recipe_id"] = "other-recipe"
|
||||||
|
assert _evaluate(wrong_recipe).state == STATE_RECIPE_MISMATCH
|
||||||
|
|
||||||
|
wrong_backend = _report(identity)
|
||||||
|
wrong_backend["backend"]["backend_id"] = "other-backend"
|
||||||
|
assert _evaluate(wrong_backend).state == STATE_RECIPE_MISMATCH
|
||||||
|
|
||||||
|
wrong_quantization = _report(identity)
|
||||||
|
wrong_quantization["backend"]["quantization"] = "Q5_K_M"
|
||||||
|
assert _evaluate(wrong_quantization).state == STATE_RECIPE_MISMATCH
|
||||||
|
|
||||||
|
|
||||||
|
def test_capability_identity_must_match_the_proven_revision_and_config():
|
||||||
|
identity = _identity()
|
||||||
|
|
||||||
|
wrong_revision = _report(identity)
|
||||||
|
wrong_revision["model"]["revision"] = "fedcba9876543210"
|
||||||
|
assert _evaluate(wrong_revision).state == STATE_MODEL_MISMATCH
|
||||||
|
|
||||||
|
wrong_config = _report(identity)
|
||||||
|
wrong_config["model"]["config_fingerprint"] = "sha256:" + _digest("c")
|
||||||
|
assert _evaluate(wrong_config).state == STATE_MODEL_MISMATCH
|
||||||
|
|
||||||
|
missing_config = _report(identity)
|
||||||
|
del missing_config["model"]["config_fingerprint"]
|
||||||
|
assert _evaluate(missing_config).state == STATE_MODEL_MISMATCH
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_exact_identity_without_a_ledger_is_dark_not_admitted():
|
||||||
|
"""No certification authority is not permission to serve — it is no proof."""
|
||||||
|
state = _evaluate(_report(_identity()), ledger=None)
|
||||||
|
|
||||||
|
assert state.state == STATE_UNCERTIFIED
|
||||||
|
assert not state.routable_under(POLICY_COMPAT)
|
||||||
|
assert not state.routable_under(POLICY_ENFORCE)
|
||||||
|
|
||||||
|
|
||||||
|
def test_admission_records_the_rederived_binding_not_the_declared_one():
|
||||||
|
split = _split(0, 4, "c")
|
||||||
|
state = _evaluate(_report(split), ledger=CertificationLedger())
|
||||||
|
|
||||||
|
assert state.shard_binding_digest == split.shard_binding_digest
|
||||||
|
assert state.fingerprint == split.fingerprint.key
|
||||||
|
|
||||||
|
|
||||||
|
# --- Certification: only the tracker, only on evidence it re-derived itself ---
|
||||||
|
|
||||||
|
|
||||||
|
def test_certification_requires_prior_dark_registration():
|
||||||
|
ledger = CertificationLedger()
|
||||||
|
identity = parse_identity(_identity().to_dict())
|
||||||
|
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="not registered"):
|
||||||
|
ledger.certify(identity, _evidence(_identity(0, 4), _identity(4, 8)))
|
||||||
|
|
||||||
|
|
||||||
|
def test_synthetic_or_mismatched_evidence_never_certifies():
|
||||||
|
identity = _identity()
|
||||||
|
ledger = CertificationLedger()
|
||||||
|
assert _evaluate(_report(identity), ledger=ledger).state == STATE_UNCERTIFIED
|
||||||
|
presented = parse_identity(identity.to_dict())
|
||||||
|
|
||||||
|
# A unit fixture is not a distributed forward. This is the trust boundary.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="synthetic"):
|
||||||
|
ledger.certify(
|
||||||
|
presented, _evidence(_identity(0, 4), _identity(4, 8), synthetic=True)
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="fingerprint does not match"):
|
||||||
|
ledger.certify(
|
||||||
|
presented,
|
||||||
|
_evidence(
|
||||||
|
_identity(0, 4),
|
||||||
|
_identity(4, 8),
|
||||||
|
fingerprint=(_digest("e"), _digest("f")),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
# A single node, however real, is not a distributed forward.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="distributed forward requires"):
|
||||||
|
ledger.certify(
|
||||||
|
presented, _evidence(_identity(0, 8), node_ids=("physical-a",))
|
||||||
|
)
|
||||||
|
|
||||||
|
# A hole in the coverage means those layers were never computed.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="owned by no Shard"):
|
||||||
|
ledger.certify(
|
||||||
|
presented, _evidence(_identity(0, 3), _identity(4, 8))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_certification_evidence_layer_count_cannot_be_substituted():
|
||||||
|
"""An 8-layer recipe cannot be promoted by evidence for a 2-layer route."""
|
||||||
|
identity = _identity()
|
||||||
|
ledger = CertificationLedger()
|
||||||
|
assert _evaluate(_report(identity), ledger=ledger).state == STATE_UNCERTIFIED
|
||||||
|
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="layer count"):
|
||||||
|
ledger.certify(
|
||||||
|
parse_identity(identity.to_dict()),
|
||||||
|
_evidence(_identity(0, 4), _identity(4, 8), layer_count=2),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_evidence_participants_must_be_the_recipe_being_promoted():
|
||||||
|
identity = _identity()
|
||||||
|
ledger = CertificationLedger()
|
||||||
|
assert _evaluate(_report(identity), ledger=ledger).state == STATE_UNCERTIFIED
|
||||||
|
|
||||||
|
# Participants running a different recipe cannot vouch for this one, even
|
||||||
|
# when the evidence header names the right fingerprint.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="participant fingerprint"):
|
||||||
|
ledger.certify(
|
||||||
|
parse_identity(identity.to_dict()),
|
||||||
|
_evidence(
|
||||||
|
_identity(0, 4),
|
||||||
|
_identity(4, 8, kv_dtype="float16"),
|
||||||
|
fingerprint=identity.fingerprint.key,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
# Nor can a participant whose identity is not the range it is recorded under.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="recorded effective range"):
|
||||||
|
ledger.certify(
|
||||||
|
parse_identity(identity.to_dict()),
|
||||||
|
_evidence(
|
||||||
|
_identity(0, 4),
|
||||||
|
_identity(4, 8),
|
||||||
|
shard_ranges=((0, 4), (0, 4)),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tracker_owns_the_only_promotion_path_and_re_admits_dark_nodes():
|
||||||
|
tracker = TrackerServer()
|
||||||
|
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
|
||||||
|
node_a = _register(tracker, "physical-a", head)
|
||||||
|
node_b = _register(tracker, "physical-b", tail)
|
||||||
|
# A node on a *different* recipe, which must not be swept up by the promotion.
|
||||||
|
other = _register(tracker, "physical-c", _identity(0, 8, kv_dtype="float16"))
|
||||||
|
|
||||||
|
assert node_a.capability.state == STATE_UNCERTIFIED
|
||||||
|
assert node_b.capability.state == STATE_UNCERTIFIED
|
||||||
|
|
||||||
|
status = tracker.certify_recipe(
|
||||||
|
parse_identity(head.to_dict()), _evidence(head, tail)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert status.may_serve
|
||||||
|
assert node_a.capability.state == STATE_ADMITTED
|
||||||
|
assert node_b.capability.state == STATE_ADMITTED
|
||||||
|
assert other.capability.state == STATE_UNCERTIFIED
|
||||||
|
# A node registering after the fact lands admitted, from the same ledger.
|
||||||
|
assert _register(tracker, "physical-d", head).capability.state == STATE_ADMITTED
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_network_map_certification_field_tracks_the_ledger():
|
||||||
|
"""The map must say what the ledger knows — "dark" before, "certified" after."""
|
||||||
|
tracker = TrackerServer()
|
||||||
|
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
|
||||||
|
node_a = _register(tracker, "physical-a", head)
|
||||||
|
node_b = _register(tracker, "physical-b", tail)
|
||||||
|
|
||||||
|
assert node_a.capability.certification == "dark"
|
||||||
|
assert node_a.capability.to_dict()["certification"] == "dark"
|
||||||
|
|
||||||
|
tracker.certify_recipe(parse_identity(head.to_dict()), _evidence(head, tail))
|
||||||
|
|
||||||
|
assert node_a.capability.certification == "certified"
|
||||||
|
assert node_b.capability.certification == "certified"
|
||||||
|
assert _register(tracker, "physical-d", head).capability.certification == "certified"
|
||||||
|
# A node that presented no identity has nothing for the ledger to say.
|
||||||
|
assert absent_state().certification is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_certification_rejects_participants_the_tracker_did_not_admit():
|
||||||
|
tracker = TrackerServer()
|
||||||
|
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
|
||||||
|
_register(tracker, "physical-a", head)
|
||||||
|
_register(tracker, "physical-b", tail)
|
||||||
|
promoted = parse_identity(head.to_dict())
|
||||||
|
|
||||||
|
# A node nobody registered cannot have served the forward.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="not registered"):
|
||||||
|
tracker.certify_recipe(
|
||||||
|
promoted, _evidence(head, tail, node_ids=("physical-a", "ghost"))
|
||||||
|
)
|
||||||
|
|
||||||
|
# Same recipe, same ranges, but derivative bytes neither node was admitted
|
||||||
|
# on: certification must not attach to blobs that never ran.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="binding this tracker did not admit"):
|
||||||
|
tracker.certify_recipe(
|
||||||
|
promoted, _evidence(_split(0, 4, "e"), _split(4, 8, "f"))
|
||||||
|
)
|
||||||
|
|
||||||
|
# Right bytes, but swapped between the nodes that served them.
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="range differs|binding this tracker"):
|
||||||
|
tracker.certify_recipe(
|
||||||
|
promoted, _evidence(tail, head)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert tracker._registry["physical-a"].capability.state == STATE_UNCERTIFIED
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_legacy_node_cannot_be_an_exact_certification_participant():
|
||||||
|
tracker = TrackerServer()
|
||||||
|
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
|
||||||
|
_register(tracker, "physical-a", head)
|
||||||
|
|
||||||
|
legacy = _NodeEntry(
|
||||||
|
node_id="physical-b",
|
||||||
|
endpoint="http://physical-b",
|
||||||
|
shard_start=4,
|
||||||
|
shard_end=7,
|
||||||
|
model="example/model",
|
||||||
|
shard_checksum=None,
|
||||||
|
hardware_profile={},
|
||||||
|
wallet_address=None,
|
||||||
|
score=1.0,
|
||||||
|
capability=CapabilityState(state=STATE_ADMITTED, model_id="example/model"),
|
||||||
|
)
|
||||||
|
tracker._registry["physical-b"] = legacy
|
||||||
|
|
||||||
|
with pytest.raises(TrackerRecipeIdentityError, match="not admitted under"):
|
||||||
|
tracker.certify_recipe(parse_identity(head.to_dict()), _evidence(head, tail))
|
||||||
|
|
||||||
|
|
||||||
|
def test_certification_does_not_survive_a_tracker_restart():
|
||||||
|
"""The ledger is in-memory. A restart loses it, and that must fail *closed*."""
|
||||||
|
tracker = TrackerServer()
|
||||||
|
head, tail = _split(0, 4, "c"), _split(4, 8, "d")
|
||||||
|
_register(tracker, "physical-a", head)
|
||||||
|
_register(tracker, "physical-b", tail)
|
||||||
|
tracker.certify_recipe(parse_identity(head.to_dict()), _evidence(head, tail))
|
||||||
|
|
||||||
|
restarted = TrackerServer()
|
||||||
|
assert _register(restarted, "physical-a", head).capability.state == STATE_UNCERTIFIED
|
||||||
|
|
||||||
|
|
||||||
|
# --- Route formation: one route, one exact fingerprint ---
|
||||||
|
|
||||||
|
|
||||||
|
def _route_node(
|
||||||
|
node_id: str,
|
||||||
|
start: int,
|
||||||
|
end: int,
|
||||||
|
fingerprint: tuple[str, str] | None,
|
||||||
|
*,
|
||||||
|
speed: float = 1.0,
|
||||||
|
state: str = STATE_ADMITTED,
|
||||||
|
) -> _NodeEntry:
|
||||||
|
capability = CapabilityState(
|
||||||
|
state=state,
|
||||||
|
model_id="example/model",
|
||||||
|
shard_start=start,
|
||||||
|
shard_end=end,
|
||||||
|
model_artifact_digest=None if fingerprint is None else fingerprint[0],
|
||||||
|
runtime_recipe_digest=None if fingerprint is None else fingerprint[1],
|
||||||
|
)
|
||||||
|
return _NodeEntry(
|
||||||
|
node_id=node_id,
|
||||||
|
endpoint=f"http://{node_id}",
|
||||||
|
shard_start=start,
|
||||||
|
shard_end=end,
|
||||||
|
model="example/model",
|
||||||
|
shard_checksum=None,
|
||||||
|
hardware_profile={},
|
||||||
|
wallet_address=None,
|
||||||
|
score=1.0,
|
||||||
|
benchmark_tokens_per_sec=speed,
|
||||||
|
capability=capability,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_route_selection_never_mixes_exact_fingerprints():
|
||||||
|
key_a = (_digest("a"), _digest("b"))
|
||||||
|
key_b = (_digest("c"), _digest("d"))
|
||||||
|
route, error = _select_route(
|
||||||
|
[_route_node("a", 0, 3, key_a), _route_node("b", 4, 7, key_b)],
|
||||||
|
0,
|
||||||
|
7,
|
||||||
|
policy=POLICY_ENFORCE,
|
||||||
|
)
|
||||||
|
assert route == []
|
||||||
|
assert "no route available" in error
|
||||||
|
|
||||||
|
|
||||||
|
def test_route_selection_never_mixes_an_exact_shard_with_a_legacy_one():
|
||||||
|
"""A node with no canonical digests cannot complete an exact route."""
|
||||||
|
key_a = (_digest("a"), _digest("b"))
|
||||||
|
route, error = _select_route(
|
||||||
|
[_route_node("exact-head", 0, 3, key_a), _route_node("legacy-tail", 4, 7, None)],
|
||||||
|
0,
|
||||||
|
7,
|
||||||
|
policy=POLICY_COMPAT,
|
||||||
|
)
|
||||||
|
assert route == []
|
||||||
|
assert "no route available" in error
|
||||||
|
|
||||||
|
|
||||||
|
def test_route_selection_falls_back_to_a_complete_fingerprint_group():
|
||||||
|
key_a = (_digest("a"), _digest("b"))
|
||||||
|
key_b = (_digest("c"), _digest("d"))
|
||||||
|
route, error = _select_route(
|
||||||
|
[
|
||||||
|
_route_node("fast-incomplete", 0, 3, key_b, speed=100.0),
|
||||||
|
_route_node("a-head", 0, 3, key_a),
|
||||||
|
_route_node("a-tail", 4, 7, key_a),
|
||||||
|
],
|
||||||
|
0,
|
||||||
|
7,
|
||||||
|
policy=POLICY_ENFORCE,
|
||||||
|
)
|
||||||
|
assert error == ""
|
||||||
|
assert [node.node_id for node in route] == ["a-head", "a-tail"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_homogeneous_legacy_fleet_routes_exactly_as_before():
|
||||||
|
"""DGR-003 partitions routes; it must not change a fleet that has no identity."""
|
||||||
|
route, error = _select_route(
|
||||||
|
[
|
||||||
|
_route_node("slow", 0, 3, None, speed=1.0),
|
||||||
|
_route_node("fast", 0, 3, None, speed=50.0),
|
||||||
|
_route_node("tail", 4, 7, None),
|
||||||
|
],
|
||||||
|
0,
|
||||||
|
7,
|
||||||
|
policy=POLICY_COMPAT,
|
||||||
|
)
|
||||||
|
assert error == ""
|
||||||
|
assert [node.node_id for node in route] == ["fast", "tail"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_uncertified_node_is_not_routable_under_either_policy():
|
||||||
|
key = (_digest("a"), _digest("b"))
|
||||||
|
nodes = [
|
||||||
|
_route_node("head", 0, 3, key, state=STATE_UNCERTIFIED),
|
||||||
|
_route_node("tail", 4, 7, key, state=STATE_UNCERTIFIED),
|
||||||
|
]
|
||||||
|
for policy in (POLICY_COMPAT, POLICY_ENFORCE):
|
||||||
|
route, error = _select_route(nodes, 0, 7, policy=policy)
|
||||||
|
assert route == [], policy
|
||||||
|
assert "no route available" in error
|
||||||
|
|
||||||
|
|
||||||
|
def test_pinned_benchmark_routes_obey_the_same_partition_rule():
|
||||||
|
"""US-030 benchmark combos run real inference; they may not mix identities."""
|
||||||
|
key_a = (_digest("a"), _digest("b"))
|
||||||
|
key_b = (_digest("c"), _digest("d"))
|
||||||
|
|
||||||
|
# An exact head with only a legacy tail, or a tail on another fingerprint,
|
||||||
|
# has no two-hop combo at all.
|
||||||
|
assert _find_pinned_route(
|
||||||
|
[_route_node("exact", 0, 3, key_a), _route_node("legacy", 4, 7, None)], 0, 7, 2
|
||||||
|
) is None
|
||||||
|
assert _find_pinned_route(
|
||||||
|
[_route_node("a", 0, 3, key_a), _route_node("b", 4, 7, key_b)], 0, 7, 2
|
||||||
|
) is None
|
||||||
|
|
||||||
|
# Homogeneous fleets — exact or legacy — still form pinned combos.
|
||||||
|
exact = _find_pinned_route(
|
||||||
|
[_route_node("a", 0, 3, key_a), _route_node("b", 4, 7, key_a)], 0, 7, 2
|
||||||
|
)
|
||||||
|
assert exact is not None and [node.node_id for node in exact] == ["a", "b"]
|
||||||
|
legacy = _find_pinned_route(
|
||||||
|
[_route_node("a", 0, 3, None), _route_node("b", 4, 7, None)], 0, 7, 2
|
||||||
|
)
|
||||||
|
assert legacy is not None and [node.node_id for node in legacy] == ["a", "b"]
|
||||||
|
|
||||||
|
|
||||||
|
# --- gRPC handshake (DGR-002 SessionOpen) ---
|
||||||
|
|
||||||
|
|
||||||
|
def _session_open(identity: ShardIdentity, **changes: object) -> "pb.SessionOpen":
|
||||||
|
fields: dict[str, object] = {
|
||||||
|
"schema_version": SCHEMA_VERSION,
|
||||||
|
"route_session_id": "session",
|
||||||
|
"route_epoch": 1,
|
||||||
|
"fingerprint": identity.fingerprint.to_proto(),
|
||||||
|
"shard_range": pb.ShardRange(
|
||||||
|
start_layer=identity.shard_start,
|
||||||
|
end_layer=identity.shard_end,
|
||||||
|
effective_start_layer=identity.shard_start,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
fields.update(changes)
|
||||||
|
return pb.SessionOpen(**fields) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_open_accepts_the_exact_local_shard():
|
||||||
|
local = _identity()
|
||||||
|
assert (
|
||||||
|
check_session_open(
|
||||||
|
local,
|
||||||
|
_session_open(local),
|
||||||
|
expected_route_session_id="session",
|
||||||
|
expected_route_epoch=1,
|
||||||
|
)
|
||||||
|
== ()
|
||||||
|
)
|
||||||
|
assert handshake_error(()) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_open_rejects_a_matching_fingerprint_with_the_wrong_range():
|
||||||
|
local = _identity()
|
||||||
|
opened = _session_open(
|
||||||
|
local,
|
||||||
|
shard_range=pb.ShardRange(start_layer=0, end_layer=5, effective_start_layer=0),
|
||||||
|
)
|
||||||
|
|
||||||
|
mismatches = check_session_open(local, opened)
|
||||||
|
assert mismatches
|
||||||
|
assert handshake_error(mismatches).code == pb.ERROR_CODE_SHARD_RANGE_MISMATCH
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_open_rejects_an_effective_start_outside_the_shard():
|
||||||
|
local = _identity(4, 8)
|
||||||
|
opened = _session_open(
|
||||||
|
local,
|
||||||
|
shard_range=pb.ShardRange(start_layer=4, end_layer=8, effective_start_layer=8),
|
||||||
|
)
|
||||||
|
|
||||||
|
mismatches = check_session_open(local, opened)
|
||||||
|
assert mismatches
|
||||||
|
assert handshake_error(mismatches).code == pb.ERROR_CODE_SHARD_RANGE_MISMATCH
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_open_rejects_a_foreign_protocol_schema():
|
||||||
|
local = _identity()
|
||||||
|
mismatches = check_session_open(
|
||||||
|
local, _session_open(local, schema_version=SCHEMA_VERSION + 1)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert mismatches
|
||||||
|
# A schema disagreement is its own protocol outcome — not a range problem,
|
||||||
|
# and not a fingerprint problem either: the peer may hold the right recipe
|
||||||
|
# and simply speak a schema this node cannot.
|
||||||
|
assert handshake_error(mismatches).code == pb.ERROR_CODE_SCHEMA_UNSUPPORTED
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("changes", "expected_session", "expected_epoch"),
|
||||||
|
[
|
||||||
|
({"route_session_id": "other-session"}, "session", 1),
|
||||||
|
({"route_epoch": 2}, "session", 1),
|
||||||
|
({"route_session_id": ""}, None, None),
|
||||||
|
({"route_epoch": 0}, None, None),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_session_open_rejects_missing_or_stale_tracker_route_assignment(
|
||||||
|
changes, expected_session, expected_epoch
|
||||||
|
):
|
||||||
|
local = _identity()
|
||||||
|
mismatches = check_session_open(
|
||||||
|
local,
|
||||||
|
_session_open(local, **changes),
|
||||||
|
expected_route_session_id=expected_session,
|
||||||
|
expected_route_epoch=expected_epoch,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert mismatches
|
||||||
|
assert handshake_error(mismatches).code == pb.ERROR_CODE_EPOCH_STALE
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_digest_disagreement_dominates_the_handshake_error_code():
|
||||||
|
"""Wrong fingerprint plus wrong range is a wrong peer, not a re-routable one."""
|
||||||
|
local = _identity()
|
||||||
|
opened = _session_open(
|
||||||
|
_identity(kv_dtype="float16"),
|
||||||
|
shard_range=pb.ShardRange(start_layer=0, end_layer=5, effective_start_layer=0),
|
||||||
|
)
|
||||||
|
|
||||||
|
mismatches = check_session_open(local, opened)
|
||||||
|
assert mismatches
|
||||||
|
assert handshake_error(mismatches).code == pb.ERROR_CODE_FINGERPRINT_MISMATCH
|
||||||
|
|
||||||
|
|
||||||
|
def test_grpc_handshake_uses_the_dgr_002_fingerprint_and_fails_closed():
|
||||||
|
local = _identity()
|
||||||
|
remote = replace(local.fingerprint, runtime_recipe_digest=_digest("f")).to_proto()
|
||||||
|
|
||||||
|
mismatches = check_handshake(local, remote)
|
||||||
|
assert mismatches
|
||||||
|
assert handshake_error(mismatches).code == pb.ERROR_CODE_FINGERPRINT_MISMATCH
|
||||||
@@ -5,9 +5,7 @@ before the transaction is sent, unconfirmed batches resent by settlement id
|
|||||||
(never double-paying), banned wallets skipped, history queryable over HTTP.
|
(never double-paying), banned wallets skipped, history queryable over HTTP.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
|
||||||
import time
|
import time
|
||||||
import urllib.request
|
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
@@ -68,7 +66,7 @@ def test_threshold_triggers_payout_and_zeroes_pending():
|
|||||||
ledger.charge_request("client", MODEL, 1000, [("wallet-a", 12)]) # 0.018 pending
|
ledger.charge_request("client", MODEL, 1000, [("wallet-a", 12)]) # 0.018 pending
|
||||||
treasury = _FakePayoutTreasury()
|
treasury = _FakePayoutTreasury()
|
||||||
tracker = _make_tracker(ledger, treasury, threshold=0.01)
|
tracker = _make_tracker(ledger, treasury, threshold=0.01)
|
||||||
port = tracker.start()
|
tracker.start()
|
||||||
try:
|
try:
|
||||||
assert _wait_for(lambda: treasury.batches)
|
assert _wait_for(lambda: treasury.batches)
|
||||||
assert treasury.batches[0] == [("wallet-a", pytest.approx(0.018))]
|
assert treasury.batches[0] == [("wallet-a", pytest.approx(0.018))]
|
||||||
|
|||||||
@@ -157,7 +157,7 @@ def test_registration_on_follower_visible_on_all_nodes(three_tracker_cluster):
|
|||||||
_wait_until_follower_knows_leader(follower, timeout=2.0)
|
_wait_until_follower_knows_leader(follower, timeout=2.0)
|
||||||
|
|
||||||
# Register via a follower
|
# Register via a follower
|
||||||
node_id = _register_node(follower, port_hint=19999)
|
_register_node(follower, port_hint=19999)
|
||||||
|
|
||||||
# Allow replication to propagate (Raft heartbeat interval is 50ms)
|
# Allow replication to propagate (Raft heartbeat interval is 50ms)
|
||||||
time.sleep(0.5)
|
time.sleep(0.5)
|
||||||
@@ -223,7 +223,7 @@ def test_registration_on_leader_visible_to_all(three_tracker_cluster):
|
|||||||
urls = list(urls)
|
urls = list(urls)
|
||||||
|
|
||||||
leader_url, followers = _wait_for_leader(urls, timeout=1.0)
|
leader_url, followers = _wait_for_leader(urls, timeout=1.0)
|
||||||
node_id = _register_node(leader_url, port_hint=19996)
|
_register_node(leader_url, port_hint=19996)
|
||||||
|
|
||||||
# Allow Raft heartbeat to replicate the entry
|
# Allow Raft heartbeat to replicate the entry
|
||||||
time.sleep(0.3)
|
time.sleep(0.3)
|
||||||
|
|||||||
@@ -19,7 +19,6 @@ from meshnet_tracker.server import (
|
|||||||
TrackerServer,
|
TrackerServer,
|
||||||
_NodeEntry,
|
_NodeEntry,
|
||||||
_available_quantizations,
|
_available_quantizations,
|
||||||
_memory_pool_map,
|
|
||||||
_rebalance_all_locked,
|
_rebalance_all_locked,
|
||||||
_registration_ban_error,
|
_registration_ban_error,
|
||||||
_scale_demanded_models_locked,
|
_scale_demanded_models_locked,
|
||||||
@@ -1448,7 +1447,7 @@ def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
|
|||||||
"vram_bytes": 10_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
|
"vram_bytes": 10_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
|
||||||
"benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
|
"benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
|
||||||
)
|
)
|
||||||
second = _post_json(
|
_post_json(
|
||||||
f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
|
f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
|
||||||
{"endpoint": "http://127.0.0.1:9016", "model": "tiny-model",
|
{"endpoint": "http://127.0.0.1:9016", "model": "tiny-model",
|
||||||
"vram_bytes": 10_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
|
"vram_bytes": 10_000, "ram_bytes": 10_000, "quantizations": ["bfloat16"],
|
||||||
|
|||||||
Reference in New Issue
Block a user