popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: popov:ralph/distributed-performance
Branches Tags
popov:master popov:ralph/distributed-gguf-runtime popov:ralph/distributed-performance popov:ralph/deepseek-v4-flash-epic popov:ralph/dgr-001-performance-contract popov:cursor/fix-meshnet-node-param-parsing popov:worktree-feat+us-016
...
compare: popov:7364ed673144873ec4341169a3e1d2bd1308f64a
Branches Tags
popov:ralph/distributed-gguf-runtime popov:master popov:ralph/distributed-performance popov:ralph/deepseek-v4-flash-epic popov:ralph/dgr-001-performance-contract popov:cursor/fix-meshnet-node-param-parsing popov:worktree-feat+us-016

12 Commits
ralph/dist ... 7364ed6731

Author SHA1 Message Date
Dobromir Popov
7364ed6731 fix: harden DGR-017 contract continuity 2026-07-14 01:10:33 +03:00
Dobromir Popov
ad2d17541c feat: DGR-003 - Define exact Artifact and runtime recipe identity 2026-07-14 01:10:07 +03:00
Dobromir Popov
e7c780a623 feat: DGR-017 - Lock the GLM-5.2 Max target and alpha contract 2026-07-14 00:19:16 +03:00
Dobromir Popov
9580ed643e docs: harden GLM alpha resource and protocol gates 2026-07-13 22:48:26 +03:00
Dobromir Popov
5ebce15d7a docs: target GLM-5.2 Max for distributed alpha 2026-07-13 22:32:14 +03:00
Dobromir Popov
ef2a9e67e8 feat: add signed ROCm diagnostic lane 2026-07-13 21:24:43 +03:00
Dobromir Popov
b1c9deeb01 fix: cryptographically bind DGR-001 evidence 2026-07-13 19:38:14 +03:00
Dobromir Popov
9e67b829e3 fix: harden DGR-001 performance contract evidence 2026-07-13 19:10:24 +03:00
Dobromir Popov
e24db7854f feat: DGR-001 - Lock the safetensors-versus-GGUF performance contract 2026-07-13 17:55:55 +03:00
Dobromir Popov
59f2486bf2 feat: DGR-001 - Lock the safetensors-versus-GGUF performance contract 2026-07-13 17:49:09 +03:00
Dobromir Popov
d904c40f66 fix: harden DGR-002 protocol bounds 2026-07-13 17:30:54 +03:00
Dobromir Popov
30dcf953fe feat: DGR-002 - Adopt the versioned gRPC Shard protocol 2026-07-13 16:00:49 +03:00
86 changed files with 21538 additions and 126 deletions
Expand all files Collapse all files

1
.claude/memory/MEMORY.md
View File

@@ -8,3 +8,4 @@
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.

29
.claude/memory/dgr-rocm-setup.md Normal file
View File

@@ -0,0 +1,29 @@
# DGR ROCm and llama.cpp setup
As of 2026-07-13:
- Project ROCm runtime: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm`
- ROCm/TheRock build: `7.13.0a20260513`, target `gfx1151`
- `rocm-sdk-devel` is installed. Its expanded SDK lives under the venv at
`site-packages/_rocm_sdk_devel`.
- The wheel's redundant packaged payload was relocated to
`/home/popov/.local/share/rocm-sdk/7.13.0a20260513/rocm_sdk_devel` and symlinked
back into the venv because installing both packaged and expanded forms filled
the mounted drive. Do not reinstall it blindly; the wheel expands beyond
20 GB.
- HIP llama.cpp source: `/run/media/popov/d/DEV/llamacpp/llama.cpp`, commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25` (version 9991).
- HIP build: `/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip`
- HIP `llama-server` SHA-256:
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`
- Verified device: `ROCm0: Radeon 8060S Graphics`, `gfx1151`.
- Model artifacts remain under `/run/media/popov/DATA/llm`; none were put under
`/home`.
DGR-001's immutable contract remains CPU-only. GPU evidence uses the distinct
signed `gpu-diagnostic` profile because llama-server process VRAM is not yet
measurable by the benchmark driver. The profile must capture measured
llama-server startup evidence for `ROCm0` and the actual offloaded/total layer
count; configured `device` and `n_gpu_layers` values alone are not evidence.
The accepted signer fingerprint is anchored in
`.scratch/distributed-gguf-runtime/trusted-evidence-signers.json`.

384
.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md Normal file
View File

@@ -0,0 +1,384 @@
# GLM-5.2 Max distributed alpha roadmap
Status: proposed executable epic target
Last updated: 2026-07-13
## Executive decision
The alpha-release target is the exact open-weight model `zai-org/GLM-5.2` served with `reasoning_effort=max` from the smallest published Unsloth GGUF recipe, `UD-IQ1_S`, across multiple consumer machines.
“Max” is a reasoning mode selected by the chat template/API request. It is not a separate model checkpoint.
Alpha is earned only when the real target model:
1. cannot fit within any one participating node's admitted memory;
2. loads as contiguous layer Shards on at least two physical consumer machines;
3. performs real GLM-5.2 MoE + DSA + IndexShare computation on every selected node;
4. responds through the existing OpenAI-compatible Meshnet API with `reasoning_effort=max`;
5. passes locked parity, usefulness, performance, telemetry, cancellation, and cleanup checks; and
6. stores all model artifacts on configured mounted-drive storage, never under `/home`.
The shortest safe path is not “support every GGUF architecture.” Dense Llama remains a small structural fixture. GLM-5.2 moves onto the critical path immediately after exact recipe identity and the pinned llama.cpp boundary. Qwen expansion, 1M context certification, MTP/speculative decoding, broad concurrency, and automatic route repair are post-alpha.
## 1. Exact target contract
### 1.1 Source model
| Field | Locked target |
|---|---|
| Official repository | `zai-org/GLM-5.2` |
| Official revision observed 2026-07-13 | `b4734de4facf877f85769a911abafc5283eab3d9` |
| Model-weight license | MIT |
| Official code/documentation license | Apache-2.0 |
| Architecture | `glm_moe_dsa` / `GlmMoeDsaForCausalLM` |
| Official architecture label | 744B total / approximately 40B active per token |
| Exact stored checkpoint tensors | 753,329,940,480 parameters |
| Transformer layers | 78 backbone layers plus one shared NextN/MTP layer in the artifact |
| Layer types | first 3 dense; remaining 75 sparse MoE |
| Routed experts | 256 |
| Experts selected | 8 routed experts plus shared expert path |
| Hidden width | 6,144 |
| Attention | MLA under DSA, lightning indexer top-k 2,048 |
| IndexShare | indexer roles are encoded by `indexer_types`; consumers reuse prior Full-layer indices |
| Architectural maximum context | 1,048,576 tokens |
| Alpha reasoning mode | `reasoning_effort=max` |
The runtime must derive these values from the pinned artifact and fail closed on contradictory metadata. Marketing names are not compatibility identity.
### 1.2 Alpha GGUF artifact
| Field | Locked target |
|---|---|
| GGUF repository | `unsloth/GLM-5.2-GGUF` |
| GGUF revision observed 2026-07-13 | `abc55e72527792c6e77069c99b4cb7de16fa9f23` |
| Quantization | `UD-IQ1_S` |
| Files | six GGUF shards |
| Exact published bytes | 216,715,360,960 bytes |
| Binary GiB | 201.832 GiB |
| Published quality indication | about 76.2% top-1 agreement with the high-precision reference on Unsloth's quantization analysis |
| Mounted storage rule | configured mounted drive only; never `/home` |
Before downloading 216.7 GB, DGR-017 must generate a checked-in target manifest containing repository revisions, expected filenames, byte sizes, and resolved LFS SHA-256 values. Download is resumable and verified before route admission.
`UD-IQ1_M` (228,492,966,624 bytes / 212.801 GiB) is the first diagnostic fallback if `UD-IQ1_S` exposes a runtime or quality defect. It does not satisfy the explicit “lowest quantization” alpha target unless the target contract is changed by human review.
### 1.3 Runtime semantics required for alpha
Required:
- GGUF parsing and quantized kernels from one exact llama.cpp pin.
- Correct GLM-5.2 MoE routing, selected experts, and shared expert.
- Correct compressed MLA KV cache for locally owned layers.
- Native DSA lightning indexer and sparse attention.
- Correct IndexShare Full/Shared role execution from artifact metadata.
- Range-owned contiguous transformer layers; each owned layer keeps all of its experts local.
- Head-only embeddings and tail-only final norm/output head/sampling.
- Architecture-defined activation boundary and optional DSA index sideband.
- `reasoning_effort=max` chat-template behavior through the public API.
- F32 seam correctness lane and a separately certified production activation dtype.
Not required for alpha:
- MTP/speculative decoding. The trailing NextN tensors may be loaded or explicitly excluded according to a certified recipe, but cannot be silently misinterpreted.
- Full 1,048,576-token context.
- Continuous batching beyond one target session.
- Public-WAN tensor or expert parallel collectives.
- Automatic mid-generation repair or KV migration.
- Every CPU/GPU backend combination.
## 2. Minimum resource envelope
### 2.1 Weight and runtime memory
The smallest artifact occupies 201.832 GiB before KV, DSA indexer state, scratch buffers, backend workspaces, process memory, and the operating system. **224 GiB aggregate runtime-accessible memory is only the experimental hard-fit floor**, consistent with Unsloth's approximate 223 GB one-bit requirement. It is not a conservative operational envelope.
For admission, each node reserves:
```text
max(20% of physically usable memory, 8 GiB)
```
The remainder is the combined weight-plus-KV placement budget. Actual peak scratch is measured by backend/context and can force one extra node. Unified memory is counted once: integrated-GPU “VRAM” must not be added again to the same physical system RAM.
| Physical usable tier | Minimum reserve | Weight + KV placement budget | IQ1_S 16K arithmetic minimum | Operational position |
|---:|---:|---:|---:|---|
| 32 GiB | 8.0 GiB | 24.0 GiB | 9 nodes | use 10 if attempted; latency-heavy |
| 48 GiB | 9.6 GiB | 38.4 GiB | 6 nodes | possible; latency-heavy |
| 64 GiB | 12.8 GiB | 51.2 GiB | 4 nodes | hard minimum; **5 recommended** |
| 96 GiB | 19.2 GiB | 76.8 GiB | 3 nodes | recommended |
| 128 GiB unified/system | 25.6 GiB | 102.4 GiB | 2 nodes | arithmetic hard minimum; **3 recommended** |
The planner must use exact tensor byte ownership, not equal percentages. Embeddings, final head, dense versus MoE layers, shared experts, indexer tensors, quant block alignment, KV distribution, and backend workspace make equal layer counts unequal in memory.
Recommended first target route: **three 96/128-GiB-class physical machines** or **five 64-GiB-class machines**, on the same wired switch with mounted model storage. Four 64-GiB or two 128-GiB machines are fit probes only and qualify solely if exact placement and measured peak-memory evidence retain the required reserve with no swap/overcommit.
### 2.2 KV cache
GLM-5.2 MLA caches 576 latent/rope values per token per backbone layer. Correct DSA also caches 128-dimensional indexer keys: ideally only for the 21 Full indexer layers, while the current experimental implementation may allocate them across all 78 layers. Alpha locks **Q8_0 KV** for quality and budgets the conservative current-implementation layout.
| Context × concurrency | MLA-only Q8 | Optimized DSA Q8 | Conservative current-DSA Q8 | Conservative current-DSA F16 |
|---:|---:|---:|---:|---:|
| 16,384 × 1 | 0.73 GiB | 0.77 GiB | **0.89 GiB** | 1.68 GiB |
| 131,072 × 1 | 5.83 GiB | 6.18 GiB | **7.12 GiB** | 13.41 GiB |
| 1,048,576 × 1 | 46.62 GiB | 49.41 GiB | **56.98 GiB** | 107.25 GiB |
These are planning estimates, not admission truth. The runtime must report measured allocated/resident MLA and indexer cache by Shard. Alpha configures a 16,384-token window, Q8_0 KV, and one session. Longer contexts and lower-bit KV are separate quality/resource certification gates.
### 2.3 Activation seams and network
A BF16 hidden-state boundary is 6,144 elements = 12,288 bytes/token before framing.
- A 16,384-token prefill sends about 192 MiB per seam.
- One decode token sends about 12 KiB per seam.
- A 512-token decode sends about 6 MiB per seam.
- Four nodes imply three serial seams.
If a Shard boundary splits an IndexShare producer/consumer group, a 2,048-entry int32 top-k sideband can add up to 8 KiB/query before framing. The route planner should prefer boundaries that preserve complete IndexShare ownership groups. The protocol must still support and validate the named sideband because memory fit may force an internal group split.
Decode bandwidth is small, but every generated token crosses all seams serially, so node count and per-hop latency dominate. Alpha requires a same-switch wired route: **2.5 GbE minimum and 10 GbE recommended**, with measured one-way/RTT, serialization, and queue latency. A 1 GbE route may be retained as fit-only evidence but is not the recommended alpha topology. Alpha records per-seam bytes, p50/p95 transfer latency, retries, and checksum failures; no speed claim is inferred from link rate alone.
### 2.4 Storage
The shortest alpha path allows every node to hold the complete six-file source GGUF while mapping/allocating only owned tensors. This minimizes artifact-transformation risk but costs 216.7 GB disk per node.
Deterministic source-bound layer packages are a follow-up optimization. If needed before target fit, every package must retain:
- source repository/revision and source file hashes;
- exact owned tensor names, layer range, and endpoint role;
- tokenizer/config identity;
- deterministic package hash; and
- proof that composing all packages matches the source tensor inventory.
## 3. Current state and critical gaps
### Completed foundation
- DGR-001: immutable CPU contract plus separate signed ROCm diagnostic. CPU v1 remains `stop`; the GPU diagnostic establishes a viable fit/performance investigation lane but does not rewrite CPU evidence.
- DGR-002: versioned backend-neutral gRPC/Protobuf Shard protocol with bounded fragments and compatibility checks.
- Existing Meshnet: Tracker, contiguous Shards, Route Sessions/epochs, relay/direct transport, local Hot KV semantics in the reference backend, cancellation, telemetry, billing, and model-agnostic admission.
### Missing before target alpha
1. Exact GLM target/artifact manifest and memory-fit planner.
2. A current llama.cpp pin proven to load and generate with the exact `UD-IQ1_S` artifact.
3. A narrow decision on native GLM-5.2 DSA/IndexShare support. As observed 2026-07-13, merged llama.cpp PR #24770 loads GLM-5.2 through a dense-MLA compatibility path, while full IndexShare/DSA PR #25407 remains open and its generic sparse path can be slower than dense fallback. Generic CPU lightning-indexer support is merged; backend coverage remains uneven.
4. A decode protocol amendment. `ActivationChunk` carries `TensorBundle`, but the current `DecodeStep` fast path carries only one `NamedTensor`; it cannot transport a hidden state plus GLM top-k sideband. Tail token/logit and sampling behavior also needs an explicit typed result contract.
5. Correct range-owned GGUF loading and memory proof.
6. GLM-specific boundary/KV/IndexShare semantics.
7. Standalone native worker and Meshnet integration.
8. Real target hardware route with no node individually able to admit the whole model.
9. Locked target parity, usefulness, speed, failure, and cleanup evidence.
### Donor policy
`Mesh-LLM/mesh-llm` is a high-value test and patch donor. Its live GLM branch was observed with 261 llama.cpp patches, 167 named for GLM/DSA/MTP-related work. That is evidence of the problem's depth, not an acceptable maintained fork boundary.
Audit and selectively reproduce the smallest independently understood pieces for:
- GLM DSA graph semantics;
- lightning indexer and sparse-attention tests;
- IndexShare metadata/Full/Shared validation;
- top-k sideband shape and lifecycle;
- stage-local KV filtering; and
- target parity/performance fixtures.
Do not import Mesh-LLM routing, discovery, scheduler, public mesh, package manager, or full patch stack. Keep Meshnet as the sole control plane and collaborate narrowly upstream with llama.cpp/Mesh-LLM maintainers where practical.
## 4. Revised roadmap
## Phase 0 — lock the target before implementation
### DGR-017: lock GLM-5.2 Max target and alpha contract
Deliver:
- machine-readable target manifest for official and GGUF revisions;
- exact `UD-IQ1_S` file/size/hash inventory;
- architecture/config/chat-template snapshot;
- memory/KV/network planner with unified-memory de-duplication;
- immutable alpha acceptance thresholds from section 5; and
- current upstream/donor status report.
Exit: target identity and alpha requirements are reviewable without downloading the model.
## Phase 1 — establish a correct whole-model oracle
### DGR-003: exact runtime recipe identity
Extend the existing generic identity with GLM fields: DSA/IndexShare metadata, adapter version, reasoning template revision, activation bundle schema, KV dtype/layout, llama.cpp pin/patch hash, and target artifact manifest hash.
### DGR-004: reproducible llama.cpp pin and narrow patch boundary
Select a current exact upstream commit only after testing its stock GLM behavior. Add clean fetch/apply/build checks. Record every donor patch and whether it is adopted, rewritten, rejected, or waiting upstream.
### DGR-018: certify whole-model GLM-5.2 runtime semantics
On a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation, or a measured equivalent:
1. verify all six `UD-IQ1_S` shards;
2. load with a stock pinned runtime and capture tensor/metadata warnings;
3. prove whether DSA, IndexShare, shared expert, and Max template are actually active;
4. add the minimum correctness patches/tests required;
5. run deterministic prefill/decode and fixed Max-mode sentinel prompts; and
6. sign the oracle recipe, output, telemetry, and limitations.
Exit: one whole-model oracle exists for the same artifact/runtime semantics the distributed path will implement. “It emits text” is insufficient.
## Phase 2 — build the generic local seam using small fixtures
### DGR-005: range-owned GGUF tensors
Keep dense Llama as a cheap structural fixture. Implement authoritative owned-tensor registration/loading, head/tail ownership, and measured resident-memory scaling. Design tensor classification so GLM adds explicit rules rather than unchecked name substitution.
### DGR-006: architecture-defined boundary
Implement named boundary bundles, F32 correctness lane, bounded fragmentation, and optional sidebands. Amend the decode fast path so it carries a versioned `TensorBundle` rather than one `NamedTensor`, while preserving a small one-tensor encoding. Define an explicit typed tail result for logits/token output and bind sampling/chat-template parameters to the recipe/request. Regenerate Python/C++ schema code and compatibility goldens. Dense fixture parity proves the seam mechanism, not GLM certification.
Exit: two local processes can execute a small dense model with correct range ownership and boundary parity.
## Phase 3 — add GLM-5.2 as the product adapter
### DGR-019: implement and certify GLM-5.2 range/DSA/IndexShare semantics
Deliver explicit support for:
- 78 main layers and endpoint tensor ownership;
- 256-expert MoE routing/top-8 and shared expert;
- compressed MLA KV by owned layers;
- DSA lightning indexer and sparse attention;
- IndexShare metadata, Full producer, Shared consumer, and sideband behavior;
- NextN/MTP tensor policy with MTP disabled or enabled explicitly;
- shard-boundary planner aware of IndexShare ownership groups; and
- whole-model versus two-stage parity against DGR-018.
Exit: a same-host two-stage target run matches the locked oracle tolerance with real GLM computation in both stages. If the full target cannot fit on one host for this check, use a layer-reduced GLM architecture fixture for graph parity and defer full-artifact output parity to DGR-020; label the distinction explicitly.
## Phase 4 — worker, KV, and Meshnet route
Execute existing stories with GLM requirements included:
1. DGR-007 — isolated local Hot KV keyed by `(Route Session, epoch)`, including DSA/IndexShare state.
2. DGR-008 — standalone C++ gRPC worker.
3. DGR-009 — Meshnet backend, capability, relay/direct, cancellation, and telemetry integration.
4. DGR-010 — small-model local two-process acceptance.
5. DGR-011 — real two-physical-machine route and heterogeneous fail-closed behavior.
6. DGR-013 subset required by alpha — node loss, cancellation, stale epoch, restart, and memory/KV cleanup.
Continuous batching (DGR-012) is deliberately not an alpha dependency. The first target release supports one admitted GLM route session; concurrency follows after target correctness and fit.
## Phase 5 — target alpha gate
### DGR-020: pass real distributed GLM-5.2 Max alpha acceptance
Use at least two physical machines and enough aggregate usable memory to meet the locked target planner. No participating node may individually admit the complete target. All stages must report real compute and exact tensor ownership.
Run the complete acceptance matrix in section 5, preserve raw logs/metrics/output, sign the evidence, and publish an explicit `alpha` or `stop` verdict. Thresholds cannot be weakened after results are known.
## Phase 6 — post-alpha hardening
After DGR-020 passes:
1. DGR-012 — 1/2/4-session continuous batching and bounded admission.
2. DGR-014 — final distributed GGUF versus reference-route performance decision.
3. 32K, 128K, 200K, then 1M context certification with quantized KV.
4. MTP/speculative decoding.
5. Deterministic range packages to remove full-artifact replication.
6. Additional backend compatibility classes and route topologies.
7. DGR-016 — narrow upstream collaboration package, split by independently reviewable llama.cpp changes.
8. DGR-015 — Qwen3/Qwen3-MoE only as later architecture expansion, not as the GLM alpha target.
## 5. Locked alpha acceptance matrix
These thresholds are set before target execution.
### 5.1 Identity and fit
- Exact official and GGUF repository revisions match the target manifest.
- All six source GGUF sizes and LFS SHA-256 values verify.
- Every route node reports owned tensor names/bytes, layer range, endpoint role, backend, KV recipe, and patch fingerprint.
- Union of owned tensors equals the certified runtime-required tensor inventory; unintended overlap is zero.
- No node's weight-plus-KV placement budget can hold the complete recipe.
- Every node reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-KV placement; measured peak scratch must remain inside that reserve.
- Aggregate peak RSS/VRAM stays within physical budgets with no swap, overcommit, mmap-only, or double-counted unified-memory success claim.
- Arithmetic-minimum topologies require exact contiguous tensor placement evidence; recommended alpha topology is 5×64 GiB or 3×96/128 GiB.
- Unified RAM/VRAM is not double-counted.
### 5.2 Semantic correctness
- Logs and graph tests prove GLM MoE/shared-expert, DSA lightning indexer, sparse attention, and IndexShare Full/Shared paths are active; dense-attention compatibility fallback cannot satisfy alpha.
- `reasoning_effort=max` is observable in the rendered template/API recipe.
- F32 same-backend seam fixture: 32 greedy decode tokens exactly match the whole-model oracle and activation tolerance is locked by DGR-006.
- Production seam on the fixed prompt corpus: greedy token agreement is at least 0.90 and mean compared-state/logit cosine similarity is at least 0.999 versus DGR-018, with no malformed or non-finite tensors.
- Incompatible artifact, tokenizer, adapter, DSA metadata, boundary, activation, KV, backend class, or runtime patch fingerprints fail closed.
### 5.3 End-to-end target run
- Context configured to 16,384 tokens with Q8_0 MLA/indexer KV.
- Fixed 4,096-token prompt lane completes prefill.
- Route uses a same-switch wired network; 2.5 GbE is the alpha minimum and 10 GbE is recommended.
- One Max-mode request generates at least 512 output tokens or reaches a valid natural EOS after at least 128 tokens.
- Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable, relevant outputs; raw prompts and outputs are retained for review.
- OpenAI-compatible response includes stable model ID, finish reason, and token usage.
### 5.4 Minimum useful performance
On the declared minimum alpha topology after one warm-up:
- median decode throughput is at least 0.5 generated token/s for the fixed Max-mode lane;
- 4,096-token-prompt TTFT is at most 10 minutes;
- no unexplained stall exceeds 60 seconds without progress telemetry;
- per-stage compute, queue, KV, seam bytes/latency, RSS/VRAM, and backend timing are present; and
- results are labeled by hardware/topology and are not generalized to other consumer systems.
If output quality passes but the speed floor fails, verdict is `stop` for alpha and the evidence selects the next optimization target. It is not relabeled as success merely because the model loaded.
### 5.5 Reliability and security
- Two consecutive cold starts load, generate, release, and exit cleanly.
- Cancellation during prefill and decode releases every stage's queued buffers and KV lease.
- One worker loss aborts the route; alpha retries only from token zero on a new compatible route.
- Stale epochs and duplicate step IDs are rejected.
- Artifact paths stay outside `/home`; logs contain no secrets or unrestricted prompt payloads.
- Synthetic workers and layer-reduced fixtures are labeled unit/integration coverage and cannot satisfy target alpha.
## 6. First execution order
The next unattended work should run in this order:
1. DGR-017 — target contract, manifest, planner, and upstream status.
2. DGR-003 — exact recipe identity.
3. DGR-004 — current llama.cpp pin and minimal patch harness.
4. Run in parallel:
- DGR-018 — whole-model oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory.
- DGR-005 and DGR-006 — generic range/boundary seam on local small fixtures.
5. DGR-019 — GLM semantics and parity after both parallel lanes pass.
6. DGR-007 through DGR-011 — native worker and real transport route.
7. Required DGR-013 failure subset.
8. DGR-020 — real target alpha verdict.
The first external hardware blocker is DGR-018, but DGR-005/DGR-006 proceed locally while that host is sourced. Do not download the full model until DGR-017's exact manifest and storage preflight pass.
## 7. Sources checked on 2026-07-13
Authoritative or primary:
- Official model card and config: <https://huggingface.co/zai-org/GLM-5.2>
- Official release/architecture blog: <https://z.ai/blog/glm-5.2>
- Official code/documentation repository: <https://github.com/zai-org/GLM-5>
- Official source revision API: <https://huggingface.co/api/models/zai-org/GLM-5.2>
- Official GLM-5 technical report: <https://arxiv.org/abs/2602.15763>
- Unsloth GGUF repository: <https://huggingface.co/unsloth/GLM-5.2-GGUF>
- Unsloth local-run/quantization guide: <https://unsloth.ai/docs/models/glm-5.2>
- llama.cpp GLM-5.2 support issue: <https://github.com/ggml-org/llama.cpp/issues/24730>
- llama.cpp merged dense-MLA compatibility loader: <https://github.com/ggml-org/llama.cpp/pull/24770>
- llama.cpp open GLM-5.2 DSA/IndexShare implementation: <https://github.com/ggml-org/llama.cpp/pull/25407>
- llama.cpp merged generic CPU lightning indexer: <https://github.com/ggml-org/llama.cpp/pull/24231>
- llama.cpp 1M-context discussion: <https://github.com/ggml-org/llama.cpp/discussions/24622>
- IndexCache/IndexShare paper: <https://arxiv.org/abs/2603.12201>
Donor/current implementation evidence:
- Mesh-LLM repository: <https://github.com/Mesh-LLM/mesh-llm>
- Mesh-LLM GLM branch noted by llama.cpp collaborator in issue #24730: `feat/jianyang-glm-52`
Web/repository observations are pinned by date and must be refreshed in DGR-017 before implementation because upstream support is moving quickly.

38
.scratch/distributed-gguf-runtime/PRD.md
View File

@@ -6,6 +6,8 @@ Build one lean native GGUF execution path that lets an Inference Route combine c
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
The alpha target is now exact: `zai-org/GLM-5.2` at a pinned revision, the lowest published Unsloth `UD-IQ1_S` GGUF, and `reasoning_effort=max`, distributed across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a cheap structural fixture; Qwen is post-alpha expansion.
## Goals
- Execute one GGUF model across independently addressable contiguous Shards.
@@ -15,6 +17,7 @@ The program is benchmark-gated. GGUF is not assumed faster merely because it is
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
- Pass an immutable GLM-5.2 Max `alpha`/`stop` gate with native MoE, DSA, IndexShare, parity, usefulness, speed, failure, and cleanup evidence.
## Quality Gates
@@ -101,6 +104,8 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
**Acceptance Criteria:**
- [ ] Head accepts token IDs and owns token embedding.
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
- [ ] Amend the decode fast path from one `NamedTensor` to a versioned `TensorBundle`, preserving compact one-tensor compatibility and regenerating Python/C++ protocol goldens.
- [ ] Define a typed tail logits/token result with sampling and chat-template/reasoning identity.
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
- [ ] Tail emits logits or token output through an explicit sampling contract.
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
@@ -213,6 +218,35 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
### DGR-017: Lock the GLM-5.2 Max target and alpha contract
**Description:** Pin exact official/GGUF revisions, `UD-IQ1_S` files and hashes, Max-mode semantics, resource accounting, and immutable target thresholds before implementation results exist.
**Acceptance Criteria:**
- [ ] Produce machine-readable target, resource, upstream-status, and immutable acceptance contracts without downloading full weights.
- [ ] Distinguish the 224-GiB hard-fit floor from the recommended 5×64 or 3×96/128 topology, using Q8 KV, 20%/8-GiB node reserves, and a wired 2.5-GbE minimum.
- [ ] Count unified RAM/VRAM once and test revision/shard/byte/threshold mutation failures.
### DGR-018: Certify whole-model GLM-5.2 runtime semantics
**Description:** Establish the exact IQ1_S oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory; lock Q8_0 MLA/indexer KV and native target semantics before distributed parity work.
**Acceptance Criteria:**
- [ ] Verify the complete artifact and prove native MoE/shared expert, DSA, IndexShare, KV, NextN policy, and Max-template behavior.
- [ ] Dense/replicated compatibility fallback cannot become the oracle merely because it emits text.
### DGR-019: Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
**Description:** Add explicit target-model tensor, graph, boundary, sideband, and local-KV ownership after the generic dense seam.
**Acceptance Criteria:**
- [ ] Preserve MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared semantics across contiguous Shards.
- [ ] Pass locked fixture/target parity and measured per-Shard memory ownership; never claim full-target parity from a reduced fixture.
### DGR-020: Pass real distributed GLM-5.2 Max alpha acceptance
**Description:** Run the exact lowest-quant target through Meshnet on enough physical consumer nodes that no node can admit the whole recipe.
**Acceptance Criteria:**
- [ ] Pass the immutable identity, semantic, parity, Max-mode usefulness, 0.5 token/s, TTFT, reliability, and mounted-storage gates.
- [ ] Preserve signed raw evidence and emit `alpha` only if every target criterion passes; otherwise emit `stop`.
## Functional Requirements
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
@@ -234,7 +268,7 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
- QUIC, WebRTC, or a custom socket protocol.
- Automatic KV migration or mid-generation route repair in the first release.
- Speculative decoding or disaggregated prefill before the core release gate.
- Supporting every GGUF architecture before dense Llama and Qwen3-family certification.
- Supporting every GGUF architecture before the exact GLM-5.2 target; Qwen3-family certification is post-alpha.
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
## Success Metrics
@@ -247,6 +281,6 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
## Open Questions
- Exact benchmark model and quantization lanes are selected by DGR-001 from currently supported, legally redistributable artifacts.
- DGR-001 remains immutable. DGR-017 locks the exact GLM-5.2/`UD-IQ1_S` target and alpha thresholds without rewriting DGR-001 evidence.
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.

16
.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md
View File

@@ -9,7 +9,7 @@ Before changing code, every Ralph agent must:
1. Read this file completely.
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
3. Read `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
3. Read `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`, and the relevant part of `architecture.md`.
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
6. Read the evidence/handoff directories for every declared dependency.
@@ -35,6 +35,8 @@ If interrupted after code changes, record every changed file, command result and
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact served with `reasoning_effort=max` across physical consumer machines. Dense Llama is a structural fixture. Synthetic workers, dense-attention compatibility fallback, a smaller model, or a single host cannot satisfy target alpha. The immutable target contract and resource envelope are in `GLM-5.2-MAX-ALPHA-ROADMAP.md`.
A distributed demo is not success. The product must provide:
- Useful measured prefill and decode speed.
@@ -84,6 +86,10 @@ Do not introduce another scheduler/control plane from vLLM, Nakshatra, prima.cpp
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
16. Unified system RAM and integrated-GPU memory are one physical pool and must never be double-counted for admission.
17. Alpha requires native GLM-5.2 MoE, DSA, and IndexShare semantics; MTP/speculative decoding and 1M-context certification are post-alpha.
18. DGR-006 amends the decode fast path to carry a versioned `TensorBundle` and defines a typed tail logits/token result; the current single-`NamedTensor` fast path is insufficient for GLM sidebands.
19. Alpha reserves at least `max(20% of physical usable memory, 8 GiB)` per node outside weight-plus-Q8-KV placement and uses a same-switch wired 2.5 GbE minimum route.
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
@@ -154,7 +160,7 @@ The project patch stack is limited to:
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
Dense Llama-family is first. Qwen3/Qwen3-MoE is a separate adapter after the dense release gate. Do not generalize through unchecked tensor-name substitutions.
Dense Llama is first only as a cheap range/boundary fixture. GLM-5.2 is the explicit product adapter and alpha target immediately afterward. Qwen3/Qwen3-MoE is post-alpha. Do not generalize through unchecked tensor-name substitutions.
## Existing code seams to inspect first
@@ -211,7 +217,7 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
| DGR-003 | exact runtime-recipe/fingerprint implementation and admission tests; `evidence/DGR-003/` |
| DGR-004 | exact upstream pin, numbered patch series, reproducible fetch/apply/build smoke; `evidence/DGR-004/` |
| DGR-005 | dense-Llama range ownership loader and memory evidence; `evidence/DGR-005/` |
| DGR-006 | architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
| DGR-006 | decode `TensorBundle` protocol amendment, typed tail-result contract, architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
| DGR-007 | concurrent session/KV manager, isolation/cleanup tests; `evidence/DGR-007/` |
| DGR-008 | standalone C++ gRPC worker, fake-model integration tests, lifecycle evidence; `evidence/DGR-008/` |
| DGR-009 | Meshnet backend/registration/relay integration and tests; `evidence/DGR-009/` |
@@ -222,6 +228,10 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
| DGR-017 | exact GLM-5.2/GGUF target manifest, resource planner, immutable alpha contract and upstream status; `evidence/DGR-017/` |
| DGR-018 | verified whole-model `UD-IQ1_S` oracle with native GLM semantic evidence; `evidence/DGR-018/` |
| DGR-019 | explicit range-owned GLM MoE/MLA/DSA/IndexShare adapter, fixtures and parity; `evidence/DGR-019/` |
| DGR-020 | real multi-node GLM-5.2 Max target evidence and immutable `alpha`/`stop` verdict; `evidence/DGR-020/` |
## Dependency handoff rule

4
.scratch/distributed-gguf-runtime/README.md
View File

@@ -4,7 +4,9 @@ Status: active benchmark-gated implementation program.
## Objective
Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.
Serve the exact pinned GLM-5.2 `UD-IQ1_S` artifact in `reasoning_effort=max` mode across consumer machines with useful measured performance. Dense Llama is a structural fixture; the real multi-node GLM target is the alpha release gate.
See **[GLM-5.2 Max distributed alpha roadmap](GLM-5.2-MAX-ALPHA-ROADMAP.md)** for the target identity, minimum hardware, immutable acceptance matrix, and revised execution order. The 224-GiB figure is an experimental hard-fit floor; recommended topology is 5×64 GiB or 3×96/128 GiB after the required per-node reserve.
## Critical path

31
.scratch/distributed-gguf-runtime/architecture.md
View File

@@ -7,6 +7,8 @@ Last updated: 2026-07-13
The system exists to serve high-quality models that exceed one consumer node's memory while retaining useful interactive speed and aggregate concurrency. A feature that only produces a distributed demo but is slower, globally serialized, or impossible to operate on consumer hardware is not complete.
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact in `reasoning_effort=max` mode. Its target-specific architecture/resource/acceptance contract is [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a structural fixture, not the product target.
## Existing control plane
Meshnet remains the only public control plane:
@@ -94,7 +96,9 @@ named tensor bundle
compression/checksum
```
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small.
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small. DGR-006 amends the current v1 decode fast path—which carries only one `NamedTensor`—to carry a versioned `TensorBundle`, while preserving compact one-tensor encoding and explicit compatibility behavior.
Tail completion is not inferred from an activation tensor name. The protocol exposes a typed logits and/or sampled-token result, and exact sampling parameters plus chat-template/reasoning mode are bound to request/runtime identity.
Direct nodes use gRPC. Nodes requiring the existing relay carry the same protobuf frames as opaque binary through the relay session. This preserves one semantic protocol instead of maintaining separate direct and relay payload contracts.
@@ -113,6 +117,8 @@ compression/checksum
Dense Llama may use one residual tensor. Other adapters may require more. vLLM's Llama and Qwen3-MoE PP paths demonstrate a boundary with both `hidden_states` and `residual`; therefore the generic protocol must not assume one anonymous tensor.
GLM-5.2 normally exchanges a 6,144-element hidden state. If a memory-balanced Shard boundary splits an IndexShare Full producer from Shared consumers, the bundle also carries the typed top-k index sideband. The planner prefers boundaries that keep an IndexShare ownership group local, but the protocol validates the sideband rather than assuming it never crosses a seam.
Only the head owns token embedding. Only the tail owns final normalization, LM head and sampling. Middle Shards exchange the architecture-defined pre-tail boundary, not final normalized embeddings.
## Hot KV State and concurrency
@@ -243,17 +249,16 @@ The GGUF path ships only if it is faster at acceptable quality or enables a larg
## Implementation sequence
1. Lock benchmark/performance contract.
2. Define gRPC/protobuf and exact recipe identity.
3. Pin llama.cpp and create the minimal patch stack.
4. Implement dense-Llama range loading and boundary parity.
5. Implement concurrent local KV.
6. Build and integrate the standalone worker.
7. Pass local two-process real-model acceptance.
8. Pass real heterogeneous two-machine acceptance.
9. Add continuous batching and failure hardening.
10. Enforce the GGUF-versus-safetensors release gate.
11. Add Qwen3/Qwen3-MoE as a separately certified adapter.
12. Prepare narrow upstream collaboration patches/tests.
1. Preserve completed DGR-001 performance and DGR-002 protocol contracts.
2. DGR-017 locks exact GLM-5.2 Max artifact, resource, and alpha acceptance identity.
3. Define exact recipe identity and pin one reproducible llama.cpp boundary.
4. Run two lanes in parallel: DGR-018 establishes the whole-model `UD-IQ1_S` oracle on 224+ GiB usable memory, while DGR-005/DGR-006 implement range loading and named boundary parity with a cheap dense fixture.
5. DGR-019 adds explicit GLM-5.2 MoE/MLA/DSA/IndexShare semantics after both lanes pass.
6. Implement local KV; build and integrate the standalone worker.
7. Pass local two-process and real two-physical-machine execution.
8. Harden cancellation, node loss, restart, and cleanup required by alpha.
9. DGR-020 executes the exact multi-node target and emits immutable `alpha` or `stop`.
10. Post-alpha: continuous batching, final comparison, longer context, MTP, and package optimization.
11. Prepare narrow upstream patches/tests; add Qwen as later architecture expansion.
See [the Ralph backlog](prd.json) and [implementation strategy](implementation-strategy.md).

43
.scratch/distributed-gguf-runtime/evidence/DGR-001/BLOCKED.md Normal file
View File

@@ -0,0 +1,43 @@
# DGR-001 downstream stop-condition handoff
Status: **DGR-001 is complete; native-track promotion is blocked by the immutable v1 verdict.**
This is no longer an execution-prerequisite blocker. The required real benchmark
ran successfully, every recipe completed at concurrency 1 and 4, artifacts were
verified, and deterministic/full test gates passed.
## Locked result
`contract-evaluation.json` records:
```text
verdict: stop
quality_lane_pass: false
speed_benefit: true
fit_benefit: true
stop_condition_met: true
```
The exact-revision BF16 GGUF quality lane compared every prompt but achieved
`0.3333` exact match and `0.9471` mean similarity against the Transformers BF16
reference. V1 requires `0.90` and `0.97`. Quantized Q4_K_M had substantial speed
and fit benefits, but the contract explicitly forbids speed from redeeming a
failed near-lossless quality lane.
## Scope of this stop
The measured baseline is Qwen2.5-0.5B on CPU using a CPU-only llama.cpp build.
It is not a Radeon, large-model, distributed, or native-shard result. Therefore:
1. Do not silently mark v1 promoted or weaken its thresholds after observing the
data.
2. Do not let DGR-004 or later runtime stories treat DGR-001 completion as a
positive promotion signal.
3. A human may choose one of these explicit paths:
- stop the native GGUF track as v1 directs;
- diagnose and fix the BF16 runtime divergence, then rerun the exact v1 plan;
- authorize a separately versioned GPU/large-model contract whose scope and
workload are locked before its measurements.
All raw evidence, configuration, artifacts, hashes, and reproduction commands
are in this directory and `README.md`.

199
.scratch/distributed-gguf-runtime/evidence/DGR-001/README.md Normal file
View File

@@ -0,0 +1,199 @@
# DGR-001 — Safetensors versus GGUF performance contract
Status: **complete; immutable v1 verdict is `stop`.**
DGR-001 successfully produced a controlled local-real CPU baseline. Completion
means the experiment and decision contract are durable and verified; it does
**not** mean the native GGUF track is approved to continue. The locked quality
gate failed, so dependent runtime work requires a human decision or a new,
explicitly versioned experiment/contract rather than silently weakening v1.
## Controlled workload
- Model: `Qwen/Qwen2.5-0.5B-Instruct`
- Exact source revision: `7ae557604adf67be50417f59c2c2f167def9a775`
- Machine: `fedora`, Linux `7.0.14-101.fc43.x86_64`, 32 logical CPUs
- Device: CPU for every recipe; VRAM is therefore correctly reported as zero
- Runtime reference: Transformers `5.13.0`, PyTorch
`2.10.0+rocm7.13.0a20260513`, BF16 safetensors
- GGUF runtime: llama.cpp version 9991, commit
`e920c523e3b8a0163fe498af5bf90df35ff51d25`
- Workload: three fixed short/medium/long prompts, greedy sampling, 32 output
tokens, three repeats, two warmups, concurrency 1 and 4, 16 CPU threads
- Evidence class: `local-real`
All artifacts are beneath `/run/media/popov/DATA/llm/`; no model artifact was
created under `/home`.
## Recipes and exact artifacts
| Recipe | Artifact | SHA-256 |
|---|---|---|
| Transformers BF16 reference | complete mounted Hugging Face snapshot | `e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6` |
| llama.cpp BF16 quality lane | `Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf` | `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862` |
| llama.cpp Q4_K_M performance/fit lane | `Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf` | `a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5` |
The snapshot digest covers every sorted relative path, resolved size, and file
byte, so tokenizer/config drift is included. The BF16 GGUF was converted
directly from the exact snapshot while preserving BF16 weights. Q4_K_M was
quantized from an exact-revision F16 conversion with the pinned quantizer.
Runtime validation recomputes every declared digest before model loading.
## Real results
All recipes completed every request with zero failures.
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|---|---:|---:|---:|
| Decode tok/s, c=1 | 40.8 | 98.5 | 207.7 |
| Aggregate decode tok/s, c=4 | 46.5 | 222.8 | 195.7 |
| TTFT p50, c=1 | 40.0 ms | 15.1 ms | 21.6 ms |
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
| Failures | 0 | 0 | 0 |
Against the reference, the eligible Q4_K_M lane measured:
- single-request decode speedup: **5.10×**;
- concurrency-4 aggregate throughput speedup: **4.20×**;
- resident-memory ratio: **0.279×**;
- artifact-size ratio: **0.398×**.
The near-lossless BF16 quality lane compared all three prompts but measured:
- exact match: **0.3333** (v1 requires at least `0.90`);
- mean text similarity: **0.9471** (v1 requires at least `0.97`).
Tokenization and stopping were controlled: every runtime saw the same prompt
token counts and reported 31 post-TTFT decode tokens. The v1 mismatch is a
real greedy-output divergence on two prompts, not missing coverage or a
text-length artifact. Its root cause remains undetermined; no post-contract
logit-tie claim is acceptance evidence. Therefore `contract-evaluation.json`
records:
```text
verdict: stop
quality_lane_pass: false
speed_benefit: true
fit_benefit: true
stop_condition_met: true
```
Thresholds were not changed after observing these results.
## Post-contract parity and ROCm diagnostics
`summarize-quality-parity.py` verifies and separates two signed sources. The CPU
v1 row uses CPU kernels and a Transformers BF16 oracle; it remains at `0.3333`
exact match with an unexplained divergence. The ROCm row uses a different plan,
GPU kernels, and a Transformers float32 oracle. In that narrower diagnostic,
the same BF16 GGUF artifact matches all three 32-token sequences exactly (`1.0`
exact match and `1.0` similarity). No conversion corruption was observed in
that three-sequence ROCm sample; this does not prove global conversion
correctness or explain the CPU result.
A separate HIP build at commit `e920c523` was compiled for `gfx1151` and
measured `ROCm0: Radeon 8060S Graphics`; its `llama-server` SHA-256 is
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`.
A signed `gpu-diagnostic` profile measured zero failures:
| GPU metric | Transformers BF16 ROCm | llama.cpp Q4 ROCm | Q4 ratio |
|---|---:|---:|---:|
| Decode tok/s, c=1 | 81.12 | 251.25 | **3.10×** |
| Aggregate decode tok/s, c=4 | 91.24 | 511.33 | **5.60×** |
| TTFT p50, c=1 | 13.77 ms | 11.80 ms | **0.857×** |
The GPU report is signed under the distinct
`run_configured_gpu_diagnostic/v1` producer. The v1 evaluator rejects that
producer even when its signature is valid. llama-server process VRAM remains
unmeasured, so this diagnostic cannot replace or satisfy the immutable v1
contract. Its signed backend detail records the measured `ROCm0: Radeon 8060S
Graphics` device and `25/25` offloaded layers.
## Implementation
- `recipe_benchmark.py` provides the runtime-neutral measurement core, true
concurrency, continuous in-flight peak-memory sampling, percentile/throughput
aggregation, failures, and output drift.
- `recipe_drivers.py` provides opt-in Transformers and llama-server drivers,
mounted-drive confinement, exact artifact/runtime verification, equal
device/thread budgets, greedy-only validation, measured host provenance, a
CPU-only v1 guard until process VRAM can be measured honestly, and a distinct
signed GPU diagnostic profile that the v1 evaluator cannot accept.
- Peak RSS is runtime-scoped: Transformers reports growth above its pre-runtime
Python baseline, while llama.cpp reports its isolated server process tree.
Both are sampled continuously during in-flight requests.
- TTFT uses each runtime's prompt/first-token compute boundary; end-to-end HTTP,
scheduling, and queue overhead remains in latency and `queue_wait_ms`.
- The exact canonical plan SHA-256 locks prompts, model/revision, sampling,
output length, repeats, warmups, and concurrency. The evaluator also requires
equal prompt/decode token counts across recipes.
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
after it; the driver subtracts that token so decode throughput matches the
Transformers inter-token convention.
- `performance_contract.py` rejects wrong plans, unsigned or incorrectly signed
real evidence, wrong config/artifact/runtime/backend/host bindings, missing
recipes/concurrency, mixed model revisions, incomplete quality coverage, and
failed references.
- Every non-synthetic report is Ed25519-signed over the complete canonical JSON,
including raw outcomes and metrics. The contract pins the public key and exact
config SHA-256; the private key remains outside Git at mode `0600`.
- The signer fingerprint is independently anchored outside this evidence
directory in `../../trusted-evidence-signers.json` and checked by tests.
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
quality gate, and only performance-fit recipes can earn speed/fit benefits.
## Evidence files
- `performance-contract.json` — immutable v1 thresholds and stop condition
- `benchmark-config.json` — exact real-run plan, drivers, artifacts, and hashes
- `results.json` — raw machine-readable per-request and aggregate evidence
- `results.txt` — human-readable benchmark summary
- `baseline.json` — distilled measurements for later comparison
- `contract-evaluation.json` — fail-closed v1 verdict
- `quality-parity-diagnosis.json` / `.md` — run/device-scoped signed-evidence summary
- `summarize-quality-parity.py` — verifies both evidence chains and regenerates it
- `gpu-diagnostic-config.json` — exact ROCm diagnostic artifacts and runtimes
- `gpu-diagnostic-results.json` / `.txt` — signed GPU outcomes and summary
- `commands.txt` — reproducible conversion, benchmark, evaluation, and test commands
- `BLOCKED.md` — downstream stop-condition handoff
- `known-unrelated-failure.md` — clean-base reproduction of the tracker race
- `../../trusted-evidence-signers.json` — repository-reviewed signer fingerprint
## Verification
```text
Targeted: 28 passed (5/5 consecutive focused runs)
Latest full suite: 755 passed, 13 skipped
Earlier full suite: 751 passed, 13 skipped
Current cancellation retry matrix, DGR-001: 4/5 passed
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
compileall: passed
git diff --check: passed
Evidence JSON parse/integrity checks: passed
```
The intermittent tracker cancellation race reproduced at the same rate on the
clean base and is retained in `known-unrelated-failure.md`; the final full suite
completed green. DGR-001 changes no tracker/proxy files.
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
invalid: it used Hermes Agent's internal venv. Verification above used the
project `.venv`, which has the DGR-002-compatible runtime. Real inference used
`.venv-rocm` Python 3.12.
## Limitations and dependent-story handoff
- The immutable contract result is a **0.5B CPU baseline**. The separate Radeon
diagnostic is real local GPU evidence, but neither result covers a large
model, distributed execution, network transport, or a native shard worker.
- A separate `GGML_HIP=ON` llama.cpp build exists and produced GPU timings, but
llama-server process VRAM is not measurable by the current driver; GPU
memory/fit claims therefore remain ineligible for v1.
- Absolute timings are developer-machine measurements; locked ratios and raw
artifacts are provided for reproducibility.
- DGR-014 may consume v1 only with the exact plan/evidence requirements enforced
by `performance_contract.py`.
- DGR-004 and later native-runtime work must not treat DGR-001 completion as a
promotion. V1 says `stop`; proceeding requires a human decision backed by a
separately versioned GPU/large-model contract or a diagnosed quality fix.

169
.scratch/distributed-gguf-runtime/evidence/DGR-001/baseline.json Normal file
View File

@@ -0,0 +1,169 @@
{
"artifact_sha256": {
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6"
},
"backend_detail": {
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16"
},
"evidence_class": "local-real",
"host": {
"accelerator_name": "Radeon 8060S Graphics",
"accelerator_runtime": "7.13.26183",
"benchmark_lane": "cpu-controlled-baseline",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"cpu_count": 32,
"cuda_available": true,
"hostname": "fedora",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_identities": {
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
}
},
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"platform": "Linux-7.0.14-101.fc43.x86_64-x86_64-with-glibc2.42",
"python": "3.12.13",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0"
},
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"provenance": {
"completed_at": "2026-07-13T16:27:19.647692Z",
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"producer": "meshnet_node.recipe_drivers.run_configured_benchmark/v1",
"run_id": "e4eedadf-22f6-4907-8990-985456961099",
"schema_version": 1,
"signature": "owev+/ToswP20C923G6E+srOCUBV5vrjmndVatr9CbTXakiFGqlHrTiEo+aymA4BcSwmG6KJTxlxO6WpLnpcAg==",
"signature_algorithm": "ed25519",
"signer_public_key_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
"started_at": "2026-07-13T16:26:22.361501Z"
},
"recipe_runtime": {
"llama-cpp-near-lossless-quality": {
"device": "cpu",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16"
},
"llama-cpp-quantized-performance-fit": {
"device": "cpu",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M"
},
"transformers-safetensors-reference": {
"device": "cpu",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16"
}
},
"recipes": {
"llama-cpp-near-lossless-quality": {
"artifact_bytes": 994156448,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 86.7339,
"decode_tokens_per_sec": 98.5178,
"failures": 0,
"latency_p50_ms": 333.023,
"latency_p95_ms": 383.0597,
"peak_rss_bytes": 1110728704,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 1717.9451,
"ttft_p50_ms": 15.069,
"ttft_p95_ms": 63.766
},
"4": {
"aggregate_decode_tokens_per_sec": 222.788,
"decode_tokens_per_sec": 76.6297,
"failures": 0,
"latency_p50_ms": 490.8738,
"latency_p95_ms": 646.26,
"peak_rss_bytes": 1139466240,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 859.8985,
"ttft_p50_ms": 32.445,
"ttft_p95_ms": 218.387
}
},
"device": "cpu",
"lane": "quality"
},
"llama-cpp-quantized-performance-fit": {
"artifact_bytes": 397807520,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 139.2693,
"decode_tokens_per_sec": 207.712,
"failures": 0,
"latency_p50_ms": 168.3307,
"latency_p95_ms": 305.1338,
"peak_rss_bytes": 542081024,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 967.0195,
"ttft_p50_ms": 21.582,
"ttft_p95_ms": 147.859
},
"4": {
"aggregate_decode_tokens_per_sec": 195.6789,
"decode_tokens_per_sec": 76.9497,
"failures": 0,
"latency_p50_ms": 437.9196,
"latency_p95_ms": 885.5355,
"peak_rss_bytes": 573259776,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 572.4424,
"ttft_p50_ms": 48.127,
"ttft_p95_ms": 416.531
}
},
"device": "cpu",
"lane": "performance-fit"
},
"transformers-safetensors-reference": {
"artifact_bytes": 999586347,
"available": true,
"concurrency": {
"1": {
"aggregate_decode_tokens_per_sec": 35.4722,
"decode_tokens_per_sec": 40.7545,
"failures": 0,
"latency_p50_ms": 818.3864,
"latency_p95_ms": 1258.0673,
"peak_rss_bytes": 1941458944,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 625.6467,
"ttft_p50_ms": 40.0018,
"ttft_p95_ms": 195.2551
},
"4": {
"aggregate_decode_tokens_per_sec": 46.5375,
"decode_tokens_per_sec": 12.9506,
"failures": 0,
"latency_p50_ms": 2481.8662,
"latency_p95_ms": 3365.8395,
"peak_rss_bytes": 2104832000,
"peak_vram_bytes": 0,
"prefill_tokens_per_sec": 264.0101,
"ttft_p50_ms": 97.0403,
"ttft_p95_ms": 429.0665
}
},
"device": "cpu",
"lane": "quality"
}
},
"reference_recipe_id": "transformers-safetensors-reference"
}

118
.scratch/distributed-gguf-runtime/evidence/DGR-001/benchmark-config.json Normal file
View File

@@ -0,0 +1,118 @@
{
"artifact_storage_root": "/run/media/popov/DATA/llm",
"evidence_class": "local-real",
"host": {
"benchmark_lane": "cpu-controlled-baseline",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"transformers_version": "5.13.0"
},
"plan": {
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"prompts": [
{
"id": "short-fact",
"text": "The capital of France is",
"context_class": "short"
},
{
"id": "medium-code",
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
"context_class": "medium"
},
{
"id": "long-summary",
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
"context_class": "long"
}
],
"sampling": {
"temperature": 0.0,
"top_p": 1.0,
"top_k": 1,
"seed": 1234,
"max_output_tokens": 32
},
"concurrency_levels": [1, 4],
"repeats": 3,
"warmup_requests": 2
},
"recipes": [
{
"id": "transformers-safetensors-reference",
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": true,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cpu",
"dtype": "bfloat16",
"threads": 16
}
},
{
"id": "llama-cpp-near-lossless-quality",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
},
{
"id": "llama-cpp-quantized-performance-fit",
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"lane": "performance-fit",
"device": "cpu",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server",
"binary_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"device": "cpu",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 0
}
}
]
}

87
.scratch/distributed-gguf-runtime/evidence/DGR-001/commands.txt Normal file
View File

@@ -0,0 +1,87 @@
# Exact source snapshot (already present on mounted storage)
SOURCE=/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775
LLAMA=/run/media/popov/d/DEV/llamacpp/llama.cpp
ROCM_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm/bin/python
PROJECT_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
OUT=/run/media/popov/DATA/llm/dgr-001
SIGNING_KEY=/home/popov/.config/neuron-tai/keys/dgr-001-evidence-ed25519.pem
# Private signing key is outside Git and must remain owner-only
stat -c '%a %n' "$SIGNING_KEY" # expected: 600
# Converter support check (no writes)
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" --dry-run
# Exact-revision near-lossless and performance-fit artifacts
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf"
$LLAMA/build/bin/llama-quantize "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf" Q4_K_M
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype bf16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf"
# Runtime and artifact identity
git -C "$LLAMA" rev-parse HEAD
$LLAMA/build/bin/llama-server --version
sha256sum "$LLAMA/build/bin/llama-server" "$LLAMA/convert_hf_to_gguf.py" "$LLAMA/build/bin/llama-quantize"
sha256sum "$SOURCE/model.safetensors" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf" "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf"
# Deterministic complete-snapshot digest used by benchmark-config.json
PYTHONPATH=packages/node $ROCM_PY - <<'PY'
from pathlib import Path
from meshnet_node.recipe_drivers import _artifact_sha256
print(_artifact_sha256(Path('/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775')))
PY
# Canonical opt-in local-real benchmark
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/benchmark-config.json \
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.json \
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.txt
# Distil the baseline and evaluate immutable v1
PYTHONPATH=packages/node $PROJECT_PY - <<'PY'
from pathlib import Path
import json
from meshnet_node.performance_contract import baseline_from_report, evaluate_contract, load_contract
root = Path('.scratch/distributed-gguf-runtime/evidence/DGR-001')
report = json.loads((root / 'results.json').read_text())
contract = load_contract(root / 'performance-contract.json')
(root / 'baseline.json').write_text(json.dumps(baseline_from_report(report), indent=2, sort_keys=True) + '\n')
(root / 'contract-evaluation.json').write_text(json.dumps(evaluate_contract(contract, report).to_dict(), indent=2, sort_keys=True) + '\n')
PY
# Optional ROCm GPU diagnostic (not eligible for immutable v1)
# The version-matched rocm[devel] wheel expands beyond 20 GB; ensure sufficient
# space or relocate its packaged payload before installation.
uv pip install --python "$ROCM_PY" --prerelease=allow \
--index-url https://rocm.nightlies.amd.com/v2/gfx1151/ \
'rocm[devel]==7.13.0a20260513'
ROCM_VENV=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm
ROCM_SDK="$ROCM_VENV/bin/rocm-sdk"
ROCM_ROOT="$($ROCM_SDK path --root)"
ROCM_BIN="$($ROCM_SDK path --bin)"
export PATH="$ROCM_VENV/bin:$ROCM_BIN:$PATH"
export ROCM_PATH="$ROCM_ROOT" HIP_PATH="$ROCM_ROOT"
export CMAKE_PREFIX_PATH="$($ROCM_SDK path --cmake):$ROCM_ROOT"
export LD_LIBRARY_PATH="$ROCM_ROOT/lib:$ROCM_ROOT/lib64:${LD_LIBRARY_PATH:-}"
$ROCM_VENV/bin/cmake -S /run/media/popov/d/DEV/llamacpp/llama.cpp \
-B /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip -G Ninja \
-DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
-DCMAKE_HIP_COMPILER="$ROCM_VENV/bin/amdclang++" \
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_BUILD_SERVER=ON
$ROCM_VENV/bin/cmake --build /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip \
--target llama-server llama-cli llama-bench -j 16
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
--profile gpu-diagnostic \
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-config.json \
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.json \
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.txt
PYTHONPATH=packages/node $PROJECT_PY \
.scratch/distributed-gguf-runtime/evidence/DGR-001/summarize-quality-parity.py
# Deterministic verification
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q tests/test_recipe_benchmark.py
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q
PYTHONPATH=packages/node $PROJECT_PY -m compileall -q packages tests
git diff --check

71
.scratch/distributed-gguf-runtime/evidence/DGR-001/contract-evaluation.json Normal file
View File

@@ -0,0 +1,71 @@
{
"contract_version": 1,
"fit_benefit": true,
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"quality_lane_pass": false,
"rationale": [
"the near-lossless quality lane failed: the GGUF runtime disagrees with the safetensors reference beyond what near-lossless weights can explain",
"a meaningful speed benefit was measured",
"a meaningful fit benefit was measured"
],
"recipes": [
{
"comparable": true,
"failures": 0,
"fit_benefit": false,
"incomparable_reason": "",
"lane": "quality",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.7873,
"artifact_size_ratio": 0.9946,
"artifact_size_win": false,
"compared_prompts": 3,
"decode_speedup": 2.4173,
"exact_match_rate": 0.3333,
"expected_prompts": 3,
"failure_rate": 0.0,
"mean_similarity": 0.9471,
"resident_memory_ratio": 0.5721,
"ttft_ratio": 0.3767
},
"quality_pass": false,
"reasons": [
"single-request decode 2.42x reference (>= 1.25x) at TTFT ratio 0.38",
"aggregate throughput at concurrency 4 is 4.79x reference (>= 1.25x)",
"peak resident memory is 0.57x reference (<= 0.75x)",
"quality lane exact-match 0.33 / similarity 0.947 versus the reference (fail)"
],
"recipe_id": "llama-cpp-near-lossless-quality",
"speed_benefit": false
},
{
"comparable": true,
"failures": 0,
"fit_benefit": true,
"incomparable_reason": "",
"lane": "performance-fit",
"measurements": {
"aggregate_concurrency": 4,
"aggregate_throughput_speedup": 4.2048,
"artifact_size_ratio": 0.398,
"artifact_size_win": true,
"decode_speedup": 5.0967,
"failure_rate": 0.0,
"resident_memory_ratio": 0.2792,
"ttft_ratio": 0.5395
},
"quality_pass": null,
"reasons": [
"single-request decode 5.10x reference (>= 1.25x) at TTFT ratio 0.54",
"aggregate throughput at concurrency 4 is 4.20x reference (>= 1.25x)",
"peak resident memory is 0.28x reference (<= 0.75x)"
],
"recipe_id": "llama-cpp-quantized-performance-fit",
"speed_benefit": true
}
],
"speed_benefit": true,
"stop_condition_met": true,
"verdict": "stop"
}

143
.scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-config.json Normal file
View File

@@ -0,0 +1,143 @@
{
"artifact_storage_root": "/run/media/popov/DATA/llm",
"evidence_class": "local-real",
"host": {
"benchmark_lane": "rocm-gpu-diagnostic",
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"llama_cpp_version": "9991",
"llama_server_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
"transformers_version": "5.13.0",
"rocm_target": "gfx1151"
},
"plan": {
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"prompts": [
{
"id": "short-fact",
"text": "The capital of France is",
"context_class": "short"
},
{
"id": "medium-code",
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
"context_class": "medium"
},
{
"id": "long-summary",
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
"context_class": "long"
}
],
"sampling": {
"temperature": 0.0,
"top_p": 1.0,
"top_k": 1,
"seed": 1234,
"max_output_tokens": 32
},
"concurrency_levels": [
1,
4
],
"repeats": 3,
"warmup_requests": 2
},
"recipes": [
{
"id": "transformers-fp32-rocm-quality-oracle",
"runtime": "transformers-5.13.0-rocm-float32",
"weight_format": "safetensors",
"weight_quantization": "bfloat16-weights-float32-accumulation",
"lane": "quality",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": true,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cuda",
"dtype": "float32",
"threads": 16
}
},
{
"id": "llama-cpp-bf16-rocm-quality",
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"lane": "quality",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
"device": "cuda",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 99
}
},
{
"id": "transformers-bf16-rocm-throughput",
"runtime": "transformers-5.13.0-rocm-bfloat16",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"lane": "performance-fit",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
"driver": {
"type": "transformers",
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
"device": "cuda",
"dtype": "bfloat16",
"threads": 16
}
},
{
"id": "llama-cpp-q4-rocm-throughput",
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"lane": "performance-fit",
"device": "cuda",
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"is_reference": false,
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
"driver": {
"type": "llama-cpp-server",
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
"device": "cuda",
"threads": 16,
"n_parallel": 4,
"context_per_slot": 512,
"n_gpu_layers": 99
}
}
]
}

3308
.scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.json Normal file
View File

File diff suppressed because it is too large Load Diff

13
.scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.txt Normal file
View File

@@ -0,0 +1,13 @@
Recipe benchmark dgr-001-rocm-gpu-diagnostic-v1 (local-real)
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
transformers-fp32-rocm-quality-oracle [quality ] c= 1 ttft p50/p95 11.0/ 35.5 ms; prefill 5746.7 tok/s; decode 35.7 tok/s; aggregate 29.6 tok/s; rss 1.39 GB; vram 2.26 GB; artifact 1.00 GB; failures 0
transformers-fp32-rocm-quality-oracle [quality ] c= 4 ttft p50/p95 27.5/ 80.4 ms; prefill 1985.4 tok/s; decode 9.4 tok/s; aggregate 35.4 tok/s; rss 1.39 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
llama-cpp-bf16-rocm-quality [quality ] c= 1 ttft p50/p95 13.2/ 83.4 ms; prefill 4154.4 tok/s; decode 148.0 tok/s; aggregate 127.4 tok/s; rss 0.84 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-bf16-rocm-quality [quality ] c= 4 ttft p50/p95 25.1/ 52.1 ms; prefill 2205.4 tok/s; decode 115.1 tok/s; aggregate 337.1 tok/s; rss 0.86 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
transformers-bf16-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 13.8/ 22.2 ms; prefill 4787.3 tok/s; decode 81.1 tok/s; aggregate 73.5 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
transformers-bf16-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 29.7/ 58.5 ms; prefill 2666.5 tok/s; decode 24.4 tok/s; aggregate 91.2 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
llama-cpp-q4-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 11.8/ 37.1 ms; prefill 4219.3 tok/s; decode 251.2 tok/s; aggregate 200.1 tok/s; rss 0.69 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
llama-cpp-q4-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 21.4/ 101.0 ms; prefill 2126.9 tok/s; decode 189.7 tok/s; aggregate 511.3 tok/s; rss 0.72 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
drift llama-cpp-bf16-rocm-quality vs transformers-fp32-rocm-quality-oracle exact 1.00; similarity 1.000 (gated)
drift transformers-bf16-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.33; similarity 0.946 (advisory)
drift llama-cpp-q4-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.00; similarity 0.628 (advisory)

55
.scratch/distributed-gguf-runtime/evidence/DGR-001/known-unrelated-failure.md Normal file
View File

@@ -0,0 +1,55 @@
# Observed pre-existing intermittent tracker race
This file records an unrelated timing observation and its repeated reproduction;
it is **not** a DGR-001 benchmark/contract failure.
Test:
```text
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
```
One earlier full-suite run produced:
```text
1 failed, 745 passed, 13 skipped
```
A five-run isolated retry matrix reproduced the same rate repeatedly:
```text
current DGR-001 branch: 4/5 passed, 1/5 failed
clean d904c40: 4/5 passed, 1/5 failed
```
An earlier full-suite run on the signed-provenance DGR-001 state completed
green:
```text
751 passed, 13 skipped
```
Two full-suite runs after adding the isolated GPU diagnostic profile each hit
the same race and otherwise passed:
```text
1 failed, 750 passed, 13 skipped
```
The latest expanded hardening suite hit the same race and otherwise passed:
```text
1 failed, 754 passed, 13 skipped
```
The final hardened state subsequently completed a full green run:
```text
755 passed, 13 skipped
```
In each failure, the mock upstream's three-second release timeout completed the
stream before the cancel POST, so the request was already absent and the cancel
endpoint returned 404. No tracker/proxy file changed in DGR-001. The race is
therefore timing-sensitive, pre-existing, and unrelated to the benchmark,
provenance, or GPU-diagnostic code.

87
.scratch/distributed-gguf-runtime/evidence/DGR-001/performance-contract.json Normal file
View File

@@ -0,0 +1,87 @@
{
"schema_version": 1,
"contract_version": 1,
"locked_at": "2026-07-13T00:00:00Z",
"locked_by": "DGR-001",
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"thresholds": {
"min_decode_speedup": 1.25,
"max_ttft_ratio": 1.25,
"min_aggregate_throughput_speedup": 1.25,
"max_resident_memory_ratio": 0.75,
"max_artifact_size_ratio": 0.6,
"min_quality_exact_match_rate": 0.9,
"min_quality_mean_similarity": 0.97,
"max_failure_rate": 0.0
},
"baseline": {
"status": "pending-real-evidence",
"required_evidence_class": "local-real",
"required_recipes": [
"transformers-safetensors-reference",
"llama-cpp-near-lossless-quality",
"llama-cpp-quantized-performance-fit"
],
"required_concurrency_levels": [
1,
4
],
"required_controlled_variables": [
"model architecture",
"model revision",
"machine and device",
"formatted prompts and context lengths",
"output length and greedy sampling policy"
],
"required_plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"minimum_prompt_count": 3,
"minimum_repeats": 3,
"minimum_output_tokens": 32,
"required_device": "cpu",
"required_config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"required_signer_public_key": "zQ/qRMwF/ydazzaxEI24Xvnrl5bZxzw16JYpP0bfRuI=",
"required_artifact_sha256": {
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5"
},
"required_recipe_runtime": {
"transformers-safetensors-reference": {
"runtime": "transformers-5.13.0",
"weight_format": "safetensors",
"weight_quantization": "bfloat16",
"device": "cpu"
},
"llama-cpp-near-lossless-quality": {
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "bfloat16",
"device": "cpu"
},
"llama-cpp-quantized-performance-fit": {
"runtime": "llama.cpp-9991-e920c523",
"weight_format": "gguf",
"weight_quantization": "Q4_K_M",
"device": "cpu"
}
},
"required_backend_detail": {
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16",
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0"
},
"required_host_identity": {
"python": "3.12.13",
"torch_version": "2.10.0+rocm7.13.0a20260513",
"transformers_version": "5.13.0",
"llama_server_identities": {
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
}
}
}
},
"stop_condition": "Stop the native llama.cpp/GGUF track when, on the same machine and device as the Transformers/safetensors reference and under this plan, no performance-fit GGUF recipe delivers either a meaningful speed benefit (>=25% higher single-request decode tokens/sec without a >25% worse TTFT, or >=25% higher aggregate throughput under concurrency) or a meaningful fit benefit (>=25% lower peak resident memory), or when the near-lossless quality lane fails, which indicates a broken runtime rather than a quantization trade-off.",
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract. Non-synthetic evidence must be Ed25519-signed by the pinned key and match the exact locked config, artifacts, runtimes, backends, and host runtime identity."
}

47
.scratch/distributed-gguf-runtime/evidence/DGR-001/quality-parity-diagnosis.json Normal file
View File

@@ -0,0 +1,47 @@
{
"conclusion": {
"conversion_corruption_observed_in_rocm_sample": false,
"cpu_bf16_divergence_explained": false,
"recommended_v2_design": "Predeclare a float32 quality oracle separately from the BF16 performance reference, with a larger prompt corpus and immutable thresholds.",
"scope": "The ROCm diagnostic establishes only that the same BF16 GGUF artifact matched the float32 oracle for three GPU sequences; it does not explain the CPU BF16 divergence or prove global conversion correctness.",
"v1_verdict_changed": false
},
"cpu_v1": {
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
"contract_verdict": "stop",
"device": "cpu",
"exact_match_rate": 0.3333,
"mean_similarity": 0.9471,
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
"quality_oracle": "Transformers BF16 safetensors",
"report": "results.json",
"report_sha256": "5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5",
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
"run_id": "e4eedadf-22f6-4907-8990-985456961099"
},
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
"rocm_diagnostic": {
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
"config_sha256": "b0f0c846c818f1307d034cee1f81daa311efc20985c32a4cdbbbd8ffe4153892",
"device": "cuda (ROCm)",
"exact_match_rate": 1.0,
"failures": 0,
"mean_similarity": 1.0,
"measured_backend_detail": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139; threads 16; parallel slots 4; ctx/slot 512; requested gpu layers 99; measured accelerator ROCm0: Radeon 8060S Graphics; measured offload 25/25 layers",
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
"plan_sha256": "dae8e40963588f71f5d201fd163d39bd762e392544b5603d483e90d21abee2e8",
"producer": "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1",
"quality_oracle": "Transformers float32 safetensors",
"report": "gpu-diagnostic-results.json",
"report_sha256": "527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d",
"run_id": "31bf44e7-ccd4-4277-84ac-c775dee65411",
"signer_fingerprint": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
"v1_eligible": false
},
"schema_version": 2
}

31
.scratch/distributed-gguf-runtime/evidence/DGR-001/quality-parity-diagnosis.md Normal file
View File

@@ -0,0 +1,31 @@
# DGR-001 quality-parity evidence summary
This summary is generated by `summarize-quality-parity.py` from signed reports.
It contains no independent logit measurements or self-asserted verification flag.
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|---|---|---|---|---:|---:|---|
| CPU v1 (`e4eedadf-22f6-4907-8990-985456961099`) | CPU | Transformers BF16 | llama.cpp BF16 | 0.3333 | 0.9471 | immutable `stop` |
| ROCm diagnostic (`31bf44e7-ccd4-4277-84ac-c775dee65411`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | 1.0000 | 1.0000 | diagnostic only |
## Interpretation
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
matched the float32 oracle for all three GPU sequences with zero failures.
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
`25/25` layer offload.
No conversion corruption was observed in that three-sequence ROCm sample. This
does not prove global conversion correctness and does not retroactively change
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
separately from its BF16 performance reference and use a larger corpus.
## Reproduction and bindings
- CPU report SHA-256: `5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5`
- GPU report SHA-256: `527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d`
- BF16 GGUF SHA-256: `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862`
- Signer fingerprint: `8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de`
- Exact verification command: see `commands.txt`.

2491
.scratch/distributed-gguf-runtime/evidence/DGR-001/results.json Normal file
View File

File diff suppressed because it is too large Load Diff

10
.scratch/distributed-gguf-runtime/evidence/DGR-001/results.txt Normal file
View File

@@ -0,0 +1,10 @@
Recipe benchmark dgr-001-controlled-whole-model-baseline-v1 (local-real)
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 40.0/ 195.3 ms; prefill 625.6 tok/s; decode 40.8 tok/s; aggregate 35.5 tok/s; rss 1.94 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 97.0/ 429.1 ms; prefill 264.0 tok/s; decode 13.0 tok/s; aggregate 46.5 tok/s; rss 2.10 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 15.1/ 63.8 ms; prefill 1717.9 tok/s; decode 98.5 tok/s; aggregate 86.7 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 32.4/ 218.4 ms; prefill 859.9 tok/s; decode 76.6 tok/s; aggregate 222.8 tok/s; rss 1.14 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 21.6/ 147.9 ms; prefill 967.0 tok/s; decode 207.7 tok/s; aggregate 139.3 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 48.1/ 416.5 ms; prefill 572.4 tok/s; decode 76.9 tok/s; aggregate 195.7 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
drift llama-cpp-near-lossless-quality vs transformers-safetensors-reference exact 0.33; similarity 0.947 (gated)
drift llama-cpp-quantized-performance-fit vs transformers-safetensors-reference exact 0.00; similarity 0.456 (advisory)

261
.scratch/distributed-gguf-runtime/evidence/DGR-001/summarize-quality-parity.py Normal file
View File

@@ -0,0 +1,261 @@
#!/usr/bin/env python3
"""Build the DGR-001 parity summary from cryptographically verified reports."""
from __future__ import annotations
import base64
import hashlib
import json
from pathlib import Path
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from meshnet_node.performance_contract import (
_canonical_sha256,
evaluate_contract,
load_contract,
report_signing_payload,
)
ROOT = Path(__file__).resolve().parent
def _read(name: str) -> dict:
return json.loads((ROOT / name).read_text(encoding="utf-8"))
def _file_sha256(name: str) -> str:
return hashlib.sha256((ROOT / name).read_bytes()).hexdigest()
def _drift(report: dict, recipe_id: str) -> dict:
return next(item for item in report["drift"] if item["recipe_id"] == recipe_id)
def _recipe(report: dict, recipe_id: str) -> dict:
return next(item for item in report["recipes"] if item["recipe"]["id"] == recipe_id)
def main() -> None:
contract = load_contract(ROOT / "performance-contract.json")
cpu_report = _read("results.json")
gpu_config = _read("gpu-diagnostic-config.json")
gpu_report = _read("gpu-diagnostic-results.json")
cpu_evaluation = evaluate_contract(contract, cpu_report)
if cpu_evaluation.verdict != "stop":
raise RuntimeError("immutable CPU v1 evidence no longer evaluates to stop")
public_key_bytes = base64.b64decode(contract.baseline["required_signer_public_key"])
public_key = Ed25519PublicKey.from_public_bytes(public_key_bytes)
public_key.verify(
base64.b64decode(gpu_report["provenance"]["signature"]),
report_signing_payload(gpu_report),
)
signer_fingerprint = hashlib.sha256(public_key_bytes).hexdigest()
if gpu_report["provenance"]["signer_public_key_sha256"] != signer_fingerprint:
raise RuntimeError("GPU report signer fingerprint does not match the contract trust key")
if gpu_report["provenance"]["config_sha256"] != _canonical_sha256(gpu_config):
raise RuntimeError("GPU report is not bound to gpu-diagnostic-config.json")
if gpu_report.get("schema_version") != 1 or gpu_report.get("evidence_class") != "local-real":
raise RuntimeError("GPU report must be schema-v1 local-real evidence")
expected_producer = "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1"
if gpu_report["provenance"].get("producer") != expected_producer:
raise RuntimeError("GPU report was not emitted by the canonical diagnostic producer")
if gpu_report.get("reference_recipe_id") != "transformers-fp32-rocm-quality-oracle":
raise RuntimeError("GPU report uses the wrong quality reference")
if gpu_report.get("host", {}).get("benchmark_lane") != "rocm-gpu-diagnostic":
raise RuntimeError("GPU report lacks the diagnostic host marker")
trusted = json.loads(
(ROOT.parents[1] / "trusted-evidence-signers.json").read_text(encoding="utf-8")
)
if not any(
signer.get("algorithm") == "ed25519"
and signer.get("fingerprint_sha256") == signer_fingerprint
and signer.get("status") == "active"
for signer in trusted.get("signers", ())
):
raise RuntimeError("GPU signer is not active in the trusted-signers registry")
for field in ("model_id", "model_revision"):
if gpu_report["plan"].get(field) != cpu_report["plan"].get(field):
raise RuntimeError(f"CPU and GPU reports do not share {field}")
if gpu_config["plan"].get(field) != gpu_report["plan"].get(field):
raise RuntimeError(f"GPU config and report do not share {field}")
expected_recipes = {
"transformers-fp32-rocm-quality-oracle": ("quality", "cuda"),
"llama-cpp-bf16-rocm-quality": ("quality", "cuda"),
"transformers-bf16-rocm-throughput": ("performance-fit", "cuda"),
"llama-cpp-q4-rocm-throughput": ("performance-fit", "cuda"),
}
actual_recipes = {
entry["recipe"]["id"]: (entry["recipe"]["lane"], entry["recipe"]["device"])
for entry in gpu_report["recipes"]
}
if actual_recipes != expected_recipes:
raise RuntimeError("GPU report recipe identities, lanes, or devices changed")
gpu_prompt_ids = {prompt["id"] for prompt in gpu_report["plan"]["prompts"]}
levels = {int(level) for level in gpu_report["plan"]["concurrency_levels"]}
repeats = int(gpu_report["plan"]["repeats"])
expected_outcomes = len(gpu_prompt_ids) * repeats * sum(levels)
for entry in gpu_report["recipes"]:
recipe_id = entry["recipe"]["id"]
if not entry.get("available") or len(entry.get("outcomes", ())) != expected_outcomes:
raise RuntimeError(f"GPU recipe {recipe_id!r} lacks complete outcomes")
if any(
not outcome.get("ok")
or outcome.get("recipe_id") != recipe_id
or outcome.get("prompt_id") not in gpu_prompt_ids
or int(outcome.get("concurrency", 0)) not in levels
or not 0 <= int(outcome.get("repeat", -1)) < repeats
for outcome in entry["outcomes"]
):
raise RuntimeError(f"GPU recipe {recipe_id!r} contains failed or invalid outcomes")
if {int(level) for level in entry["concurrency"]} != levels:
raise RuntimeError(f"GPU recipe {recipe_id!r} has wrong concurrency cells")
for prompt_id in gpu_prompt_ids:
for level in levels:
for repeat in range(repeats):
count = sum(
outcome["prompt_id"] == prompt_id
and int(outcome["concurrency"]) == level
and int(outcome["repeat"]) == repeat
for outcome in entry["outcomes"]
)
if count != level:
raise RuntimeError(
f"GPU recipe {recipe_id!r} lacks complete request coverage"
)
if any(
int(cell.get("failures", -1)) != 0
or int(cell.get("requests", -1))
!= len(
[
outcome
for outcome in entry["outcomes"]
if int(outcome["concurrency"]) == int(level)
]
)
for level, cell in entry["concurrency"].items()
):
raise RuntimeError(f"GPU recipe {recipe_id!r} aggregates do not match outcomes")
cpu_quality = _drift(cpu_report, "llama-cpp-near-lossless-quality")
gpu_quality = _drift(gpu_report, "llama-cpp-bf16-rocm-quality")
cpu_recipe = _recipe(cpu_report, "llama-cpp-near-lossless-quality")
gpu_recipe = _recipe(gpu_report, "llama-cpp-bf16-rocm-quality")
gpu_backend = gpu_recipe["load"]["backend_detail"]
if "measured accelerator ROCm0: Radeon 8060S Graphics" not in gpu_backend:
raise RuntimeError("GPU report lacks measured ROCm device evidence")
if "measured offload 25/25 layers" not in gpu_backend:
raise RuntimeError("GPU report lacks measured layer-offload evidence")
if cpu_recipe["recipe"]["artifact_sha256"] != gpu_recipe["recipe"]["artifact_sha256"]:
raise RuntimeError("CPU and GPU diagnostics use different BF16 GGUF artifacts")
if gpu_quality.get("compared_prompts") != len(gpu_prompt_ids):
raise RuntimeError("GPU quality drift lacks complete prompt coverage")
if {item["prompt_id"] for item in gpu_quality.get("per_prompt", ())} != gpu_prompt_ids:
raise RuntimeError("GPU quality drift prompt identities do not match the plan")
summary = {
"schema_version": 2,
"model_id": cpu_report["plan"]["model_id"],
"model_revision": cpu_report["plan"]["model_revision"],
"cpu_v1": {
"report": "results.json",
"report_sha256": _file_sha256("results.json"),
"run_id": cpu_report["provenance"]["run_id"],
"plan_id": cpu_report["plan"]["plan_id"],
"plan_sha256": _canonical_sha256(cpu_report["plan"]),
"config_sha256": cpu_report["provenance"]["config_sha256"],
"device": "cpu",
"quality_oracle": "Transformers BF16 safetensors",
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": cpu_recipe["recipe"]["artifact_sha256"],
"exact_match_rate": cpu_quality["exact_match_rate"],
"mean_similarity": cpu_quality["mean_similarity"],
"contract_verdict": cpu_evaluation.verdict,
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
},
"rocm_diagnostic": {
"report": "gpu-diagnostic-results.json",
"report_sha256": _file_sha256("gpu-diagnostic-results.json"),
"run_id": gpu_report["provenance"]["run_id"],
"producer": gpu_report["provenance"]["producer"],
"signer_fingerprint": signer_fingerprint,
"plan_id": gpu_report["plan"]["plan_id"],
"plan_sha256": _canonical_sha256(gpu_report["plan"]),
"config_sha256": gpu_report["provenance"]["config_sha256"],
"device": "cuda (ROCm)",
"quality_oracle": "Transformers float32 safetensors",
"candidate": "llama.cpp BF16 GGUF",
"candidate_artifact_sha256": gpu_recipe["recipe"]["artifact_sha256"],
"measured_backend_detail": gpu_backend,
"exact_match_rate": gpu_quality["exact_match_rate"],
"mean_similarity": gpu_quality["mean_similarity"],
"failures": sum(
metrics["failures"]
for entry in gpu_report["recipes"]
for metrics in entry["concurrency"].values()
),
"v1_eligible": False,
},
"conclusion": {
"v1_verdict_changed": False,
"cpu_bf16_divergence_explained": False,
"conversion_corruption_observed_in_rocm_sample": False,
"scope": (
"The ROCm diagnostic establishes only that the same BF16 GGUF artifact "
"matched the float32 oracle for three GPU sequences; it does not explain "
"the CPU BF16 divergence or prove global conversion correctness."
),
"recommended_v2_design": (
"Predeclare a float32 quality oracle separately from the BF16 performance "
"reference, with a larger prompt corpus and immutable thresholds."
),
},
}
(ROOT / "quality-parity-diagnosis.json").write_text(
json.dumps(summary, indent=2, sort_keys=True) + "\n", encoding="utf-8"
)
md = f"""# DGR-001 quality-parity evidence summary
This summary is generated by `summarize-quality-parity.py` from signed reports.
It contains no independent logit measurements or self-asserted verification flag.
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|---|---|---|---|---:|---:|---|
| CPU v1 (`{summary['cpu_v1']['run_id']}`) | CPU | Transformers BF16 | llama.cpp BF16 | {summary['cpu_v1']['exact_match_rate']:.4f} | {summary['cpu_v1']['mean_similarity']:.4f} | immutable `stop` |
| ROCm diagnostic (`{summary['rocm_diagnostic']['run_id']}`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | {summary['rocm_diagnostic']['exact_match_rate']:.4f} | {summary['rocm_diagnostic']['mean_similarity']:.4f} | diagnostic only |
## Interpretation
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
matched the float32 oracle for all three GPU sequences with zero failures.
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
`25/25` layer offload.
No conversion corruption was observed in that three-sequence ROCm sample. This
does not prove global conversion correctness and does not retroactively change
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
separately from its BF16 performance reference and use a larger corpus.
## Reproduction and bindings
- CPU report SHA-256: `{summary['cpu_v1']['report_sha256']}`
- GPU report SHA-256: `{summary['rocm_diagnostic']['report_sha256']}`
- BF16 GGUF SHA-256: `{summary['rocm_diagnostic']['candidate_artifact_sha256']}`
- Signer fingerprint: `{signer_fingerprint}`
- Exact verification command: see `commands.txt`.
"""
(ROOT / "quality-parity-diagnosis.md").write_text(md, encoding="utf-8")
if __name__ == "__main__":
main()

242
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md Normal file
View File

@@ -0,0 +1,242 @@
# DGR-002 — Adopt the versioned gRPC Shard protocol
Status: **done**. Every acceptance criterion is met with real command output.
Evidence class: **synthetic/unit** — this story defines a schema and proves both
languages agree on it. No model, GPU, network peer or benchmark is involved, and
none is claimed.
## 1. Summary
`packages/node/native/proto/shard_runtime.proto` is now the semantic contract for
the native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). Python
and C++ both generate from it, and a shared committed conformance vector proves
they encode it identically — byte for byte.
Design decisions worth carrying forward:
- **Everything gRPC gives you is *also* in the schema.** Deadline, cancellation,
identity and flow control are carried as fields, not left to HTTP/2 metadata,
because the existing relay carries these frames as **opaque binary**. A relayed
frame has no HTTP/2 context to inherit a deadline or a channel identity from.
If it is not in the schema, it does not survive the relay.
- **Cancellation is both in-band and out-of-band.** `CancelSignal` rides the
stream; `Cancel` is also a unary RPC. A cancel that can only travel down a
stream that flow control has wedged is not a cancel.
- **Checksums cover the uncompressed payload.** Compression is a per-hop
transport decision (reusing the existing `activation_compression` policies), so
a checksum over the compressed frame would be invalidated by a hop that merely
chose differently.
- **Application-level flow-control credits, not just HTTP/2 windows.** HTTP/2
bounds *bytes in flight*; it does not bound how much *work* a worker has queued,
and a relayed frame gets no window at all. Credits bound queue occupancy and KV
pressure, and negotiation takes the strictest bound of either peer so a sender
cannot talk a worker into unbounded queues.
## 2. Files changed
New:
| Path | What |
|---|---|
| `packages/node/native/proto/shard_runtime.proto` | The schema (sha256 `9e211660…`, see `protocol.json`) |
| `packages/node/native/CMakeLists.txt` | C++ generation + build wiring + ctest |
| `packages/node/native/tests/test_shard_protocol_conformance.cpp` | C++ conformance test |
| `packages/node/native/testdata/*.binpb` | Committed cross-language vectors |
| `packages/node/native/README.md` | How to regenerate and build |
| `packages/node/meshnet_node/native_protocol/__init__.py` | Public Python surface |
| `packages/node/meshnet_node/native_protocol/codec.py` | Bundle encode/decode, fragmentation, CRC32C, chunking, FC negotiation |
| `packages/node/meshnet_node/native_protocol/conformance.py` | Canonical vectors shared by both languages |
| `packages/node/meshnet_node/native_protocol/generated/` | Generated Python stubs (committed) |
| `scripts/generate_native_protocol.py` | Python generation, with `--check` |
| `scripts/generate_protocol_goldens.py` | Vector generation, with `--check` |
| `scripts/bootstrap_native_toolchain.sh` | Builds protobuf C++ from source |
| `tests/test_native_shard_protocol.py` | 45 Python tests |
Modified:
- `packages/node/pyproject.toml` — added runtime floors `grpcio>=1.82.1` and
`protobuf>=7.35.0`, matching the committed generated-code requirements; new
`proto` extra pinning `grpcio-tools==1.82.1`.
- `packages/node/meshnet_node/activation_compression.py` — optional bounded zstd
output for untrusted protocol frames; existing callers remain compatible.
- `packages/node/meshnet_node/native_protocol/__init__.py` — exports negotiated
bound constants and whole-session-message validation.
The canonical PRD marks only DGR-002 passed. `git status` before this story was clean.
## 3. Commands and real results
See `commands.txt` for the exact ordered list. Results:
```
python scripts/generate_native_protocol.py --check -> generated stubs are up to date
python scripts/generate_protocol_goldens.py --check -> conformance vectors are up to date
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
-- gRPC C++ not found: building message types only (sufficient for the conformance test)
cmake --build build/native -j -> Built target shard_protocol_conformance
ctest --test-dir build/native --output-on-failure -> 1/1 Test #1: shard_protocol_conformance ... Passed
100% tests passed out of 1
cmp build/native/cpp_roundtrip.binpb \
packages/node/native/testdata/session_request_golden.binpb -> identical (exit 0)
pytest -q tests/test_native_shard_protocol.py -> 45 passed
pytest -q tests/test_native_shard_protocol.py \
tests/test_activation_compression.py -> 51 passed
pytest -q (final full suite) -> 728 passed, 12 skipped
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
(after an earlier flaky full-suite failure) -> 1 passed, 1 passed, 1 passed
clean minimum-runtime import + codec smoke test -> passed
grpcio==1.82.1, protobuf==7.35.0
compileall -q packages tests -> OK (exit 0)
git diff --check -> clean (exit 0)
```
The C++ lane was rebuilt from scratch by Ralph (`rm -rf build/native`) using only
the documented commands, and reproduced the same result. During controller
review the user explicitly chose not to repeat the destructive build-directory
cleanup, so the independent controller relied on the recorded CMake/CTest run
while reproducing every Python/generation/full-suite gate.
### Controller review corrections
Independent controller review found and fixed two classes of issue before
integration:
1. Generated stubs required gRPC 1.82.1 and Protobuf 7.35.0, while the initial
package metadata allowed much older runtimes that could fail at import time.
2. Flow-control bounds were described but not enforced by the reference decoder.
Tensor declarations, shape rank/dimensions, fragment/tensor counts, fragments,
wire bodies, whole bundles, complete session messages (including envelope
overhead), and zstd window/output expansion are now fail-closed against the
negotiated/default bounds. Unspecified bundle versions, compression and
checksums are rejected rather than interpreted as valid data.
3. Negotiated initial credits could exceed `max_inflight_chunks`; credits are now
capped by the settled in-flight limit.
Controller results: protocol tests `45 passed`; protocol plus shared compression
tests `51 passed`; final full suite `728 passed, 12 skipped`. A clean environment
at the declared minimum gRPC/Protobuf runtime versions imported both generated
stub modules and round-tripped the codec. Generation checks, `compileall`, static
secret scan, and `git diff --check` all passed.
### Full-suite note — a pre-existing flaky test
`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy`
is **flaky on a clean tree, independent of this story**. Reproduction, run
*before any DGR-002 file existed* (working tree clean, `git status` empty):
```
pytest -q -> 1 failed, 682 passed, 12 skipped
FAILED tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# same test, three consecutive isolated runs on the same clean tree:
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 1 passed in 1.76s
-> 1 failed in 4.39s
-> 1 passed in 1.10s
```
It is a timing race in proxy cancellation (a 3-second in-flight generation raced
against the cancel assertion), not a deterministic failure, and it touches no code
this story changes. One controller full-suite run reported exactly that one failure
(`1 failed, 719 passed, 12 skipped`); three immediate isolated retries all passed
in 1.11 seconds, and the final exact-code full suite was green (`728 passed,
12 skipped`). It is flagged for whoever owns the tracker cancel path and is **not**
fixed here, since silently touching another story's code is out of scope.
## 4. Acceptance criteria
| Criterion | Where it is proven |
|---|---|
| Schema for capability, health, session stream, release, cancellation | `shard_runtime.proto` `service ShardRuntime`; `test_service_exposes_capability_health_session_release_and_cancel` |
| One long-lived bidi stream per Activation Seam, with deadlines, cancellation, flow control, structured errors | `rpc Session (stream) returns (stream)`; `test_session_is_one_long_lived_bidirectional_stream`; `Envelope.deadline_unix_nanos`, `CancelSignal` + unary `Cancel`, `FlowControl`, `ShardError` |
| Bounded chunking for prefill; small decode fast path | `ChunkInfo` + `plan_prefill_chunks` (128-token bound, ADR-0008); `DecodeStep`; `test_prefill_is_split_into_bounded_token_aligned_chunks`, `test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk` |
| Envelope carries schema version, work id, session id, epoch, fingerprint, range/effective start, phase, position, idempotency step, cache expectation, compression, checksum | `Envelope` + `NamedTensor`; `test_envelope_carries_every_field_the_protocol_promises` asserts against the **descriptor**, so deleting a field from the `.proto` fails the test |
| Versioned named-tensor bundle: name, shape, dtype, byte order, fragments | `TensorBundle`/`NamedTensor`/`TensorFragment`; `test_named_tensor_bundle_is_versioned_and_fully_described`, `test_bundle_round_trips_multiple_named_tensors` |
| Round-trip + compatibility tests in Python and C++ | 45 Python tests; C++ `ctest` 1/1; cross-language byte equality |
| Targeted pytest passes | 45 passed |
| `compileall packages tests` | exit 0 |
| `git diff --check` | exit 0 |
| Default tests deterministic, download-free, credit-free, GPU-free | Pure in-memory protobuf; no model, no network, no GPU |
| Full deterministic pytest passes, or pre-existing failure recorded | Final exact-code run: 728 passed, 12 skipped; earlier sole flaky failure documented with clean-tree reproduction and 3/3 passing retries |
## 5. How the cross-language claim is actually earned
Two codecs that each round-trip their own output prove only that each is
self-consistent. Instead:
1. Python builds the canonical `SessionRequest` and commits its bytes.
2. The C++ test parses **those** bytes, asserts every field, recomputes the CRC32C
**from the polynomial in independent C++ code**, reassembles the multi-fragment
tensor, and re-serializes to `cpp_roundtrip.binpb`.
3. `test_cpp_and_python_agree_byte_for_byte` asserts that file equals the golden.
Compatibility is tested in both languages: an unknown field from a newer peer
survives a parse/serialize hop (a Shard forwards activations — silently stripping
fields would corrupt a route it is merely a waypoint on), and a sparse message
from an older peer parses to proto3 defaults.
## 6. Limitations and deferred work
- **gRPC C++ was not built or linked.** The C++ lane verifies the *schema* (message
types), not a running gRPC C++ server, because this machine has no gRPC C++ stack
and building it is a large dependency the conformance test does not need.
`CMakeLists.txt` already generates and exports `shard_runtime_grpc` when
`find_package(gRPC)` succeeds. **DGR-008 must install gRPC C++ and extend
`scripts/bootstrap_native_toolchain.sh`.**
- **No wire is exercised.** No client, server, or stream lifecycle exists yet — no
deadline actually fires, no credit is actually consumed. This story defines and
proves the contract; DGR-008/DGR-009 implement it.
- The protobuf C++ toolchain used here was installed to `/tmp/pbsrc/install` (ephemeral).
`scripts/bootstrap_native_toolchain.sh` reproduces it; prefer a durable prefix such
as `build/native-toolchain`.
- `crc32c` has a pure-Python fallback (used here) and picks up `google_crc32c` when
present. The fallback is byte-exact but slow; a worker on the hot path should install
the native package. Not a correctness limitation.
- Compression on the wire is zstd-or-none only, matching the existing seam.
## 7. Compatibility and migration notes
- **This does not change the existing HTTP activation wire.** `X-Meshnet-Wire` stays
at `2` and the legacy `/forward` path is untouched. The native protocol is a
*separate* contract with its own `SchemaVersion`, starting at 1. Nothing in this
story is on any live request path — it is additive.
- Semantics are deliberately preserved from the existing ADRs so the two transports
mean the same thing: `effective_start_layer` (ADR-0012), `CacheMode`/`expected_past_len`
and `ERROR_CODE_CACHE_MISS` mapping to today's HTTP 409 `cache_miss` (ADR-0022),
bfloat16 boundary dtype and 128-token prefill chunks (ADR-0008), fingerprint/recipe
identity mirroring the capability report (ADR-0023).
- `TensorFragment` field 5 (`uncompressed_size`) is **reserved**: it was removed
because `NamedTensor.total_bytes` is the single source of truth. Never recycle it —
a recycled field number is the one schema change peers cannot detect, because the
bytes still parse.
- Committed Python stubs are guarded by `--check` in the test suite, so they cannot
drift from the schema unnoticed.
## 8. Handoff to dependent stories
- **DGR-003 (runtime recipe/fingerprint):** populate `Fingerprint`
(`model_artifact_digest`, `runtime_recipe_digest`, `recipe_id`, `recipe_version`,
`catalogue_version`). The mismatch outcome is already specified:
`ERROR_CODE_FINGERPRINT_MISMATCH`. Do not invent a second identity struct.
- **DGR-005/006 (range loading, architecture boundary):** the boundary payload is a
**named bundle**, not a bare tensor — a boundary needing more than one tensor is
already representable. Execute `[effective_start_layer, end_layer)`, never from
`start_layer`.
- **DGR-007 (concurrent sessions/KV):** isolate on `(route_session_id, route_epoch)`.
`CacheExpectation`/`CacheResult` and `ERROR_CODE_CACHE_MISS` are the contract; a
decode step whose `expected_past_len` does not match **must** miss, never fall back
to a silent stateless forward. `idempotency_step` means a retried step is
acknowledged (`Ack.duplicate`), not re-applied — re-applying advances the KV cache
twice and desynchronises the route.
- **DGR-008 (C++ worker):** link `shard_runtime_grpc` from `CMakeLists.txt`; you must
first install gRPC C++ (see limitations). Honour `FlowControl` credits and the
`max_chunk_bytes` bound. Use `packages/node/meshnet_node/native_protocol/codec.py`
as the reference for fragment reassembly and checksum validation.
- **DGR-009 (Meshnet integration):** the relay may carry these serialized frames as
opaque binary — that is exactly why deadline/cancel/identity are in-band. Do not add
a second control plane.
- **Anyone editing the schema:** run both `--check` scripts; if a vector legitimately
changes, regenerate it and say so, because the C++ test asserts those exact bytes.

45
.scratch/distributed-gguf-runtime/evidence/DGR-002/commands.txt Normal file
View File

@@ -0,0 +1,45 @@
# DGR-002 — exact commands, in order. Run from the repository root.
# Interpreter: <repo>/.venv/bin/python (CPython 3.14.6). Deterministic, GPU-free,
# no model download, no API credits.
# --- toolchain (this machine had no protoc, no cmake, no protobuf C++ headers)
.venv/bin/python -m pip install grpcio-tools==1.82.1 grpcio==1.82.1 cmake==4.4.0
scripts/bootstrap_native_toolchain.sh /tmp/pbsrc/install # protobuf C++ 33.1 + abseil 20250814.1
# --- schema generation (Python stubs; committed)
.venv/bin/python scripts/generate_native_protocol.py
.venv/bin/python scripts/generate_native_protocol.py --check # -> "generated stubs are up to date"
# --- cross-language conformance vectors (committed)
.venv/bin/python scripts/generate_protocol_goldens.py
.venv/bin/python scripts/generate_protocol_goldens.py --check # -> "conformance vectors are up to date"
# --- C++ generation, build and conformance test
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
cmake --build build/native -j"$(nproc)"
ctest --test-dir build/native --output-on-failure # -> 1/1 Passed
cmp build/native/cpp_roundtrip.binpb packages/node/native/testdata/session_request_golden.binpb
# --- Python tests
.venv/bin/python -m pytest -q tests/test_native_shard_protocol.py # -> 29 passed
.venv/bin/python -m pytest -q # full suite
# --- repository gates
.venv/bin/python -m compileall -q packages tests
git diff --check
# --- independent controller review after Ralph
PYTHONPATH=packages/node .venv/bin/python -m pytest -q tests/test_native_shard_protocol.py
# -> 45 passed
PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
tests/test_native_shard_protocol.py tests/test_activation_compression.py
# -> 51 passed
PYTHONPATH=packages/node .venv/bin/python -m pytest -q
# -> final exact-code run: 728 passed, 12 skipped
for i in 1 2 3; do PYTHONPATH=packages/node .venv/bin/python -m pytest -q \
tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy; done
# -> 1 passed, 1 passed, 1 passed
# clean minimum-runtime venv: protobuf==7.35.0 grpcio==1.82.1
# generated pb2 + pb2_grpc imports and one-byte codec round trip -> passed
# The user chose to rely on Ralph's recorded successful C++ CMake/CTest run
# rather than repeat deletion of an isolated generated build directory.

95
.scratch/distributed-gguf-runtime/evidence/DGR-002/protocol.json Normal file
View File

@@ -0,0 +1,95 @@
{
"schema_version": "SCHEMA_VERSION_1",
"bundle_version": 1,
"proto_path": "packages/node/native/proto/shard_runtime.proto",
"proto_sha256": "9e211660b3fcefc88bcdf3851c3571088c00349aacb5adc5ef45083c83d0cce2",
"protoc": "grpc_tools 1.82.1 (python) / protobuf 33.1 (C++)",
"service": {
"GetCapability": {
"client_streaming": false,
"server_streaming": false
},
"Health": {
"client_streaming": false,
"server_streaming": false
},
"Session": {
"client_streaming": true,
"server_streaming": true
},
"Release": {
"client_streaming": false,
"server_streaming": false
},
"Cancel": {
"client_streaming": false,
"server_streaming": false
}
},
"envelope_fields": [
"cache_expectation",
"chunk",
"deadline_unix_nanos",
"fingerprint",
"idempotency_step",
"phase",
"position",
"route_epoch",
"route_session_id",
"schema_version",
"shard_range",
"work_id"
],
"named_tensor_fields": [
"byte_order",
"checksum",
"compression",
"dtype",
"fragments",
"name",
"shape",
"total_bytes"
],
"phases": [
"PHASE_UNSPECIFIED",
"PHASE_PREFILL",
"PHASE_DECODE",
"PHASE_RELEASE",
"PHASE_CANCEL"
],
"error_codes": [
"ERROR_CODE_UNSPECIFIED",
"ERROR_CODE_SCHEMA_UNSUPPORTED",
"ERROR_CODE_FINGERPRINT_MISMATCH",
"ERROR_CODE_EPOCH_STALE",
"ERROR_CODE_SHARD_RANGE_MISMATCH",
"ERROR_CODE_CACHE_MISS",
"ERROR_CODE_RESOURCE_EXHAUSTED",
"ERROR_CODE_PAYLOAD_CORRUPT",
"ERROR_CODE_CANCELLED",
"ERROR_CODE_DEADLINE_EXCEEDED",
"ERROR_CODE_FLOW_CONTROL_VIOLATION",
"ERROR_CODE_INTERNAL"
],
"bounds": {
"max_prefill_chunk_tokens": 128,
"max_chunk_bytes": 4194304,
"max_fragment_bytes": 1048576,
"max_inflight_chunks": 8,
"max_fragments_per_tensor": 64,
"max_tensors_per_bundle": 64,
"max_tensor_rank": 8,
"max_tensor_dimension": 2147483647,
"whole_session_message_enforced": true
},
"golden_vectors": {
"session_request_golden.binpb": "c2c3df8a717ddeae7bd99624d2c7f34c09a518988de990237fe313b75cff0817",
"capability_report_golden.binpb": "71ac5f150775f398515b43a63596a5cbe8d2ad607e7e4de56bd44fbe7987080c"
},
"verification": {
"python_protocol_tests": "45 passed",
"python_protocol_and_compression_tests": "51 passed",
"full_suite": "728 passed, 12 skipped",
"minimum_runtime": "grpcio 1.82.1 / protobuf 7.35.0 passed import and codec smoke"
}
}

119
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md Normal file
View File

@@ -0,0 +1,119 @@
# DGR-003 — exact Artifact and runtime recipe identity
Evidence class: deterministic offline/unit. No model payload, GPU, external API,
network node, or API credit is required or claimed.
## Result
DGR-003 defines an exact, model-agnostic compatibility identity and connects it
to both DGR-002's gRPC `Fingerprint` and tracker capability admission.
A matching digest proves canonical consistency, **not node authenticity or real
execution**. Tracker-owned certification of a fingerprint by a non-synthetic,
complete, multi-node distributed forward is the execution trust boundary.
## Implementation
- `ArtifactIdentity` binds artifact ID/revision, exact content digest,
architecture/config digest, layer count, and optional derivative binding.
- `DerivativeBinding` binds a split artifact to the exact source artifact digest
and its end-exclusive layer range. A Shard cannot advertise outside that range.
- `RuntimeRecipe` keeps these canonical axes separate rather than hiding them in
a backend label:
- weight quantization;
- activation and compute dtypes;
- KV dtype and layout;
- tokenizer revision;
- architecture adapter;
- backend and runtime version;
- boundary and protocol schema versions;
- recipe ID/version and catalogue version.
- `CompatibilityFingerprint` populates the existing DGR-002 Protobuf
`Fingerprint`; `check_handshake()` returns DGR-002's structured fingerprint
mismatch error.
- Node and tracker implementations independently canonicalize the declaration.
This is intentional: the tracker must not trust a digest copied from a node,
and future native/C++ workers also need an independent implementation. Their
behavior is pinned by `tests/data/recipe_fingerprint_vectors.json`.
- Tracker admission cross-checks the exact identity against the capability
proof's model, range, recipe labels, backend, and weight quantization. Any
disagreement fails closed.
- `TrackerServer` owns one certification ledger and passes it through direct and
replicated registration paths. A known exact recipe is `uncertified` and dark
for user traffic until the same exact fingerprint is certified.
- Certification evidence is bound to the promoted fingerprint, requires at
least two distinct nodes, complete layer coverage, generated tokens, and
`synthetic=false`. Unknown or mismatched fingerprints cannot be promoted.
## Files changed
- `packages/node/meshnet_node/runtime_recipe.py`
- `packages/node/meshnet_node/capability.py`
- `packages/tracker/meshnet_tracker/recipe.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/data/recipe_fingerprint_vectors.json`
- `tests/test_runtime_recipe_identity.py`
- this evidence directory, issue state, and DGR-003 PRD state
A late review of dependency DGR-017 also found and fixed two genuine contract
continuity defects before DGR-003 was accepted: v1 now has an independently
trusted digest and recursively immutable parsed state. Those changes and tests
are recorded in DGR-017 evidence rather than claimed as DGR-003 functionality.
## Verification
Exact commands and outcomes are in `commands.txt`.
Observed final results:
- DGR-003 identity + node/tracker capability suites: **99 passed**.
- DGR-017 focused dependency repair suite: **99 passed**.
- Full deterministic suite: **872 passed, 13 skipped**.
- `python -m compileall -q packages tests`: pass.
- `git diff --check`: pass.
- Ruff on the changed identity, capability, contract, and test modules: pass.
- `server.py` has 8 pre-existing Ruff findings at both pushed baseline and the
current tree; DGR-003 added no finding.
The first integrated full-suite run produced **871 passed, 13 skipped, 1 failed**
on the known unrelated
`test_tracker_dashboard_can_cancel_inflight_proxy` timing race. Its fixture
completed after three seconds just before cancellation, so the cancel endpoint
returned 404. The same test then passed **5/5 in isolation**, and the complete
integrated rerun passed **872/872** tests. No cancellation-test code was changed.
## Limitations
- Certification state is process-local in this story. The same running tracker
reuses it across registrations, but durable/cluster-wide certification-event
persistence belongs with the later real distributed-forward control path.
Restart or failover therefore returns exact recipes to the safe dark state;
it never makes an unsupported recipe routable.
- The node module still contains a local registry helper from the interrupted
partial implementation. It has no call sites and is not used by admission;
tracker remains the live certification authority. Removing that unpushed
helper is safe cleanup, not an acceptance dependency.
- This story proves identity and admission behavior with deterministic fixtures.
It does not claim a real GLM forward or hardware certification.
## Compatibility
- Capability report identity is additive. Legacy reports without the new block
retain ADR-0023's explicit compatibility-policy behavior.
- Reports that opt into exact identity are held to it and fail closed on malformed,
inconsistent, unknown, dark, or mismatched declarations.
- No new wire identity was invented; DGR-002's `Fingerprint` remains the gRPC
representation.
## Handoff
DGR-004 and native workers must build `ShardIdentity` from the actual immutable
artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout,
schema versions, and owned range. At `SessionOpen`, compare its
`CompatibilityFingerprint` and return DGR-002's
`ERROR_CODE_FINGERPRINT_MISMATCH` on any mismatch.
A digest match is not certification. Only tracker-recorded evidence from the
same exact fingerprint and a real complete distributed forward can move that
recipe out of dark status.

34
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt Normal file
View File

@@ -0,0 +1,34 @@
# DGR-003 final verification — 2026-07-14
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
# result: 99 passed in 4.76s
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
# result: 99 passed in 0.15s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# first integrated result: 871 passed, 13 skipped, 1 failed in 258.18s
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
# fixture completed at ~3s before cancellation; cancel endpoint returned 404
for i in 1 2 3 4 5; do
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
done
# result: 5/5 passed (1.14s, 1.14s, 1.26s, 1.14s, 1.64s)
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
# final integrated result: 872 passed, 13 skipped in 253.46s
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
# result: pass
git diff --check
# result: pass
ruff check packages/node/meshnet_node/glm_alpha/contract.py packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_glm_alpha_target.py tests/test_runtime_recipe_identity.py
# result: All checks passed!
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
ruff check /tmp/dgr003-server-base.py
ruff check packages/tracker/meshnet_tracker/server.py
# result: both baseline and current server.py report the same 8 pre-existing findings

266
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md Normal file
View File

@@ -0,0 +1,266 @@
# DGR-017 — Lock the GLM-5.2 Max target and alpha contract
Status: **done**. Every acceptance criterion is met with real command output.
Evidence class: **real upstream metadata + deterministic arithmetic**. No weight
payload was downloaded, no model was loaded, no GPU was used, and no benchmark was
run — and none is claimed. This story makes the target *reviewable before* the
216.7 GB download, which is exactly its job.
## 1. Summary
The alpha target is now pinned, planned, and sealed:
- **Identity.** `zai-org/GLM-5.2` @ `b4734de4facf877f85769a911abafc5283eab3d9` and
`unsloth/GLM-5.2-GGUF` @ `abc55e72527792c6e77069c99b4cb7de16fa9f23`, quantization
`UD-IQ1_S`, six shards, 216,715,360,960 bytes, every shard's LFS SHA-256 resolved.
- **Architecture.** The config/tokenizer/chat-template metadata the runtime cannot
shard without, hashed at the pinned revision.
- **Resources.** A deterministic planner that counts unified memory once, applies the
`max(20% , 8 GiB)` reserve, and reports the arithmetic minimum and the recommended
node count as two different numbers.
- **Contract.** The roadmap's section-5 acceptance matrix as a machine-readable,
digest-sealed document, locked before the target ever runs and cross-bound to the
exact manifest and architecture-snapshot digests.
- **Upstream.** A refreshed llama.cpp/donor status report.
Three findings are worth a reader's attention.
**Every number in the roadmap reproduced from primary sources.** The 216,715,360,960
byte total, the 201.832 GiB figure, the whole KV table (0.73 / 0.77 / 0.89 / 1.68 GiB
at 16K, through 46.62 / 49.41 / 56.98 / 107.25 GiB at 1M), and the whole tier table
(9 / 6 / 4 / 3 / 2 arithmetic minimum nodes) fall out of the exact config and the
exact shard bytes. The roadmap was not approximating. The planner is written as a
*reproduction* of those tables, so if the arithmetic ever stops matching, a test says
which numbers moved.
**The roadmap's "recommended" column is an imbalance factor of exactly 1.10.** Nodes
= `ceil(total x 1.10 / budget)` yields 10 / 6 / 5 / 3 / 3 for the 32 / 48 / 64 / 96 /
128 GiB tiers — precisely the roadmap's recommendations. That constant is now named
(`PLACEMENT_IMBALANCE_FACTOR`) and documented as a placeholder for measured
per-tensor placement, not a fudge factor to be tuned once results are in.
**224 GiB aggregate does not actually fit.** Two 112 GiB nodes hit the 224 GiB
"hard-fit floor" exactly, and still come up **23.5 GiB short** once each node honours
its reserve. That is what makes 224 GiB an *experimental floor* rather than an
envelope, and it is now a test, not a caveat in prose. Relatedly, the 2×128 and 4×64
"fit probe" topologies fit with only **2.08 GiB of headroom across the entire route** —
which is why they require measured placement evidence and are not the recommendation.
## 2. Files changed
New — runtime-loadable package (single source of truth):
| Path | What |
|---|---|
| `packages/node/meshnet_node/glm_alpha/__init__.py` | Public surface |
| `packages/node/meshnet_node/glm_alpha/manifest.py` | Target manifest + architecture snapshot; fail-closed identity |
| `packages/node/meshnet_node/glm_alpha/planner.py` | Memory / KV / seam planner; unified-memory de-duplication |
| `packages/node/meshnet_node/glm_alpha/contract.py` | Immutable, digest-sealed alpha acceptance contract |
| `packages/node/meshnet_node/glm_alpha/data/target-manifest.json` | The six pinned shards, sizes, SHA-256, URLs, licenses |
| `packages/node/meshnet_node/glm_alpha/data/architecture-snapshot.json` | Pinned architecture + config/template hashes |
| `packages/node/meshnet_node/glm_alpha/data/alpha-contract.json` | Sealed acceptance thresholds (`aab23220…`) |
| `scripts/refresh_glm_target_manifest.py` | Re-resolve/verify pins from upstream metadata (`--check` / `--write`) |
| `tests/test_glm_alpha_target.py` | 97 deterministic offline tests |
New — evidence:
- `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md` (this file)
- `.../commands.txt` — exact commands and real results
- `.../resource-plan.json` — generated tier/route/seam/KV plan
- `.../upstream-status.json` — refreshed llama.cpp and donor status
Modified:
- `.scratch/distributed-gguf-runtime/issues/17-...md` — `Status: done`
- `.scratch/distributed-gguf-runtime/prd.json` — DGR-017 `passes: true` (this story only)
- `.ralph-tui/progress.md` — learnings
The data files live **in the package**, not in evidence, because the runtime must load
them (DGR-018 verifies downloads against these digests; DGR-003 folds the manifest
digest into the recipe fingerprint). Duplicating them into evidence would create two
sources of truth that could drift.
## 3. Acceptance criteria
| Criterion | Where it is proven |
|---|---|
| Pin both repos by exact observed revision; `UD-IQ1_S` is the alpha quant | `target-manifest.json`; `test_manifest_pins_both_repositories_by_exact_revision` |
| Six filenames, exact bytes, LFS SHA-256, aggregate GB/GiB, license, URLs, no payload download | HF `paths-info` API (LFS pointer metadata); `test_manifest_resolves_all_six_shards…`, `test_manifest_aggregate_bytes_are_exact_and_self_consistent` |
| Snapshot + hash architecture-critical config/tokenizer/chat-template metadata | `architecture-snapshot.json`; `test_snapshot_captures_the_architecture_critical_metadata`, `test_snapshot_hashes_the_config_and_chat_template_bytes` |
| Deterministic minimum-node calc from exact bytes, Q8_0 KV @16K/c1, imbalance, reserve | `planner.plan_topology`; `test_topology_planner_reproduces_the_published_tier_table` |
| 224 GiB is a hard-fit floor, not an envelope; recommend 5×64 or 3×96/128 | `test_224_gib_aggregate_is_a_hard_fit_floor_not_an_operational_envelope`, `test_the_recommended_topologies_are_five_by_64_or_three_by_96_or_128` |
| Unified memory counted once; additive RAM+VRAM rejected | `NodeMemory.from_host`; `test_adding_integrated_gpu_memory_to_system_ram_is_rejected`, `test_unified_memory_is_counted_once` |
| 2.5 GbE minimum / 10 GbE recommended; serial latency modelled apart from bandwidth | `planner.plan_seams`; `test_2_5_gbe_is_the_alpha_minimum_and_10_gbe_is_recommended`, `test_serial_seam_latency_is_modelled_separately_from_bandwidth` |
| Identity/semantic/target-run/performance/reliability/storage criteria locked before execution | `alpha-contract.json` (sealed, `locked_before_target_execution: true`); `test_the_contract_locks_every_roadmap_acceptance_section` |
| Refresh upstream llama.cpp + donor status; no broad fork/scheduler | `upstream-status.json`; `adoption_state: none adopted` |
| Tests reject changed revisions, missing shards, coordinated digest/config substitutions, inconsistent bytes, duplicate unified memory, malformed telemetry, and post-result threshold mutation | 97 tests; see §5 |
| Targeted pytest passes | `97 passed` |
| Installed wheel includes and loads all locked JSON resources | Real wheel build/install plus `load_locked_target()` outside the source tree |
| `compileall packages tests` | exit 0 |
| `git diff --check` | exit 0 |
| Default tests deterministic, download-free, credit-free, GPU-free | Pure JSON + arithmetic; the only network code is an opt-in script excluded from the suite |
| Full deterministic `pytest -q` | **852 passed, 13 skipped** on final rerun |
## 4. Real results
```
scripts/refresh_glm_target_manifest.py --check -> match upstream (exit 0)
pytest -q tests/test_glm_alpha_target.py -> 97 passed
build + install wheel; load_locked_target() -> INSTALLED_WHEEL_PASS
compileall -q packages tests -> exit 0
git diff --check -> exit 0
pytest -q -> 852 passed, 13 skipped (253.30s)
```
Controller review found and fixed four gaps before the commit was accepted:
1. the initial wheel omitted `glm_alpha/data/*.json`, despite source-tree tests passing;
2. the self-sealed contract did not bind the manifest/snapshot digests, so coordinated
shard-hash or architecture substitutions could be accepted;
3. `--check` followed moving repository HEAD instead of validating immutable pins;
4. non-finite resource telemetry could bypass ordinary range checks.
Each failure now has either an adversarial regression test or a real installed-wheel /
live-metadata gate. None of the provisional agent results were accepted on trust.
The intermittent tracker cancellation race that DGR-001 and DGR-002 both recorded as
flaky on a clean tree failed once in the final validation (`404` after the request
completed), then passed **5/5** in isolation and passed in the integrated full-suite
rerun above. This story touches no tracker code; the failed run is retained in
`commands.txt` rather than hidden.
Planner output (`resource-plan.json`):
| Route | Fits | Headroom |
|---|---|---:|
| 5×64 GiB unified (recommended) | yes | +53.28 GiB |
| 3×96 GiB unified (recommended) | yes | +27.68 GiB |
| 3×128 GiB unified (recommended) | yes | +104.48 GiB |
| 4×64 GiB (fit probe) | yes | **+2.08 GiB** |
| 2×128 GiB (fit probe) | yes | **+2.08 GiB** |
| 2×112 GiB (= 224 GiB floor) | **no** | −23.52 GiB |
| 3×64 GiB | no | −49.12 GiB |
## 5. How the "no silent swap" claim is earned
The story's whole purpose is to stop a later agent from changing the target after
seeing a result. Each of those moves now has a test that names it:
- swap the artifact → `test_a_changed_gguf_revision_is_rejected`
- pin a branch instead of a commit → `test_a_branch_name_is_not_an_acceptable_revision_pin`
- drop a shard → `test_a_missing_shard_is_rejected`
- shrink a shard so it "fits" → `test_a_shard_size_edited_to_make_the_model_look_smaller_is_rejected`
- change a valid-looking shard SHA without changing bytes → `test_coordinated_shard_hash_substitution_is_rejected_by_contract`
- change internally consistent architecture metadata → `test_internally_consistent_architecture_substitution_is_rejected_by_contract`
- take the bigger quant quietly → `test_swapping_in_a_different_quantization_is_rejected`
- add iGPU "VRAM" to system RAM → `test_adding_integrated_gpu_memory_to_system_ram_is_rejected`
- count one machine twice → `test_the_same_machine_counted_twice_in_a_route_is_rejected`
- lower the speed floor → `test_lowering_the_speed_floor_after_seeing_a_result_is_rejected`
- call a slow pass an alpha → `test_relabelling_a_speed_failure_as_a_pass_is_rejected`
- admit the dense fallback → `test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected`
- shrink the reserve → `test_relaxing_the_per_node_reserve_after_the_fact_is_rejected`
**Contract continuity is fail-closed.** The document’s `contract_sha256` detects
accidental edits, and the approved v1 digest is pinned independently in code. A caller
that changes a threshold and re-seals it under `glm-5.2-max-alpha/v1` is rejected by
`test_resealing_a_mutated_v1_contract_is_rejected`; amendments require a new supported
contract identity under human review. Parsed nested state is recursively immutable,
so thresholds cannot change between validation and use; `to_dict()` returns an isolated
copy rather than exposing the validated object.
## 6. Upstream status — the gating risk for DGR-004/DGR-018
Refreshed against live GitHub on 2026-07-13. One item **changed** since the roadmap:
- **#24231 is now MERGED** (2026-07-11) — a generic `GGML_OP_LIGHTNING_INDEXER` exists.
- #24770 MERGED (2026-06-20) — GLM-5.2 loads via a **dense-MLA compatibility path**.
- **#25407 still OPEN** (updated today) — the real GLM DSA/IndexShare wiring.
- #24730 still OPEN — the umbrella GLM-5.2 support request.
**No released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare today.** A
stock pin taken now would load the artifact and emit text through the dense fallback —
which the alpha contract explicitly refuses (`dense_attention_fallback_satisfies_alpha:
false`). DGR-018 must therefore prove those paths are *active*, not that output appeared.
Donor policy holds, and the evidence now supports it more strongly than before: PR
#25407 is **12 files, +414/−7**. The semantics alpha needs are small enough to track
and reproduce upstream. That is the argument against adopting Mesh-LLM's 261-patch
fork — recorded as a donor (Apache-2.0, branch head `9bd18f15`, 2026-07-12), nothing
adopted here.
## 7. Limitations and deferred work
- **No artifact was downloaded or loaded.** Sizes and SHA-256 come from Hugging Face
LFS pointer metadata. DGR-018 must verify the digests against the real files on
mounted storage before route admission. A matching size with a wrong hash is exactly
the failure this manifest exists to catch, and only a local verify can catch it.
- **`PLACEMENT_IMBALANCE_FACTOR = 1.10` is a planning assumption, not a measurement.**
It reproduces the roadmap's recommendations, but the real per-node share depends on
exact tensor bytes (embeddings, output head, dense vs MoE layers, shared experts,
indexer tensors, quant block alignment). DGR-019 must replace it with measured
placement. Until then, arithmetic-minimum topologies (2×128, 4×64) stay fit probes.
- **KV numbers are planning estimates, not admission truth.** The planner deliberately
budgets the *conservative* indexer layout (keys across all 78 layers, not just the 21
Full ones) so a route admitted here cannot be surprised by the implementation it
actually gets. The runtime must still report measured allocated/resident MLA and
indexer cache per shard.
- **Peak scratch is unmodelled.** The reserve exists precisely because backend
workspaces and graph scratch are not predictable from the artifact; measured peak
must land inside the reserve, and the contract requires that evidence.
- **Upstream is moving fast.** #25407 was updated the same day it was observed. Refresh
`upstream-status.json` before DGR-004 picks a llama.cpp pin. The manifest script's
`--check` deliberately validates the immutable Hugging Face pins, not moving HEAD.
## 8. Compatibility and migration notes
- Purely additive. No existing module, wire format, or test changed. Nothing in this
story is on a live request path.
- `meshnet_node.glm_alpha` has **no heavy imports** — no torch, no transformers, no
network at import time — so a tracker or planner can read the target contract without
paying for a model runtime.
- Re-pinning is deliberately awkward: `--write` follows current HEAD and leaves the
existing contract binding invalid until a new contract is reviewed and sealed.
`--check` uses revision-specific APIs and exits non-zero rather than healing any
integrity drift in the already locked target.
## 9. Handoff to dependent stories
**DGR-003 (recipe identity):** fold `TargetManifest.digest`
(`0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`) and
`ArchitectureSnapshot.digest` (`253fbd94…`) into the runtime recipe fingerprint. The
GLM fields the roadmap asks you to add (DSA/IndexShare metadata, context max, expert
counts) are already resolved in `architecture-snapshot.json` — read them, do not
re-derive them by hand. Populate the DGR-002 `Fingerprint` message; do not invent a
second identity struct.
**DGR-004 (llama.cpp pin):** read `upstream-status.json` first. Any pin taken before
#25407 merges gives you the dense-MLA fallback, which cannot satisfy alpha. Track
#25407 (12 files) as a numbered patch; do not adopt the Mesh-LLM fork.
**DGR-018 (whole-model oracle):** the six digests in `target-manifest.json` are what you
verify the download against. Your host needs ≥224 GiB runtime-accessible memory — and
note that 224 GiB is the *floor*, not a comfortable target (see §1). Prove DSA,
IndexShare, shared expert, and the Max template are **active**; the contract's
`require_rendered_reasoning_effort_marker` is `<|system|>Reasoning Effort: Max`. Assert
the rendered marker, not the request field: the template's only non-max level is
`'high'`, so *every other value — including an absent one — renders Max*, and "the
request said max" proves nothing.
**DGR-019 (GLM semantics):** the IndexShare split is **21 Full producer layers and 57
Shared consumers**, in a `[full, full, full] + repeating [shared, shared, shared, full]`
pattern. Prefer shard boundaries that keep an ownership group whole; the 8 KiB
(2048 × int32) top-k sideband is the cost when you cannot. Replace
`PLACEMENT_IMBALANCE_FACTOR` with measured per-tensor placement.
**DGR-020 (alpha verdict):** load the target with `load_locked_target()`. It verifies
the contract seal and cross-binds the manifest and architecture snapshot before
returning them. Judge against
`contract.threshold(section, key)` — an unlocked threshold raises rather than
defaulting, so a criterion cannot be invented at read time. The verdict is `alpha` or
`stop`; there is no third outcome, and a quality pass with a speed failure is `stop`.
**Everyone:** unified system RAM and integrated-GPU memory are one pool. Build nodes
with `NodeMemory.from_host(..., unified=True)` and it is impossible to write the
double-count; pass a GPU size alongside `unified=True` and it raises rather than
silently ignoring the argument.

115
.scratch/distributed-gguf-runtime/evidence/DGR-017/commands.txt Normal file
View File

@@ -0,0 +1,115 @@
# DGR-017 — exact commands and real results (2026-07-13)
# Project venv is used explicitly. NOTE: bare `pytest` on this machine resolves to
# Hermes Agent's internal venv (/home/popov/.hermes/...), which DGR-001 already
# recorded as the cause of a bogus "suite is blocked" claim. Always use $VP.
VP=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python # Python 3.14.6
# ---------------------------------------------------------------------------
# 1. Resolve the target from upstream metadata ONLY. No weight payload downloaded.
# Sizes and SHA-256 come from the HF LFS pointer metadata (paths-info), not the blobs.
# ---------------------------------------------------------------------------
curl -sS "https://huggingface.co/api/models/zai-org/GLM-5.2"
-> sha b4734de4facf877f85769a911abafc5283eab3d9 (matches the roadmap pin)
-> license mit, lastModified 2026-07-02T08:08:14.000Z
curl -sS "https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF"
-> sha abc55e72527792c6e77069c99b4cb7de16fa9f23 (matches the roadmap pin)
-> license mit, lastModified 2026-06-23T15:18:23.000Z
-> six UD-IQ1_S shards present
curl -sS -X POST -d '{"paths": [<6 UD-IQ1_S shards>]}' \
"https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF/paths-info/abc55e7..."
-> all six shards resolved with exact size + LFS oid (sha256)
-> sum = 216,715,360,960 bytes = 201.832 GiB = 216.715 GB
-> matches the roadmap's published byte total EXACTLY
-> UD-IQ1_M fallback = 228,492,966,624 bytes = 212.801 GiB (also matches)
curl -sS ".../resolve/b4734de4.../{config.json,chat_template.jinja,
generation_config.json,tokenizer_config.json}"
-> config.json 3732 B sha256 185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a
-> chat_template.jinja 5076 B sha256 172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679
-> generation_config.json 194 B sha256 ac76b43d8683d3b930126870fc8be73d8679308fe752fa1f381096d8354f6a55
-> tokenizer_config.json 761 B sha256 98b1271574f41abf89427ae2dda030d94dc9478f0edc5a8bd240db213c6fd5fc
# ---------------------------------------------------------------------------
# 2. Verify the checked-in pins still match live upstream (reproducible, no weights)
# ---------------------------------------------------------------------------
$VP scripts/refresh_glm_target_manifest.py --check
-> "target manifest and architecture snapshot match upstream"
-> exit 0
# ---------------------------------------------------------------------------
# 3. Upstream llama.cpp / donor status refresh (GitHub REST API, read-only)
# ---------------------------------------------------------------------------
curl -sS "https://api.github.com/repos/ggml-org/llama.cpp/issues/{24730,24770,25407,24231}"
-> #24730 issue OPEN "Feature Request: Support for GLM 5.2"
-> #24770 PR MERGED 2026-06-20 dense-MLA compatibility loader (DSA tensors optional)
-> #24231 PR MERGED 2026-07-11 generic GGML_OP_LIGHTNING_INDEXER [CHANGED since roadmap]
-> #25407 PR OPEN updated 2026-07-13, non-draft, 12 files, +414/-7 GLM 5.2 Indexer support
curl -sS "https://api.github.com/repos/Mesh-LLM/mesh-llm{,/branches/feat%2Fjianyang-glm-52}"
-> Apache-2.0, 2048 stars, branch head 9bd18f1509dff7fac21578635084035b3ba90a38 (2026-07-12)
-> recorded as donor only; nothing forked, nothing adopted
# ---------------------------------------------------------------------------
# 4. Seal the alpha contract (digest over its own canonical content)
# ---------------------------------------------------------------------------
$VP -c "seal_contract(...)" -> contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
# ---------------------------------------------------------------------------
# 5. Generate the machine-readable resource plan from the pinned artifact
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node $VP <generate resource-plan.json>
-> manifest_sha256 0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962
-> architecture_sha256 253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1
-> alpha_contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
-> tier arithmetic minimum 32:9 48:6 64:4 96:3 128:2 (reproduces the roadmap table)
-> tier recommended 32:10 48:6 64:5 96:3 128:3 (reproduces the roadmap table)
-> 5x64 GiB unified fits, +53.28 GiB headroom
-> 3x96 GiB unified fits, +27.68 GiB headroom
-> 2x128 / 4x64 (fit probes) fit with only +2.08 GiB headroom across the WHOLE route
-> 2x112 GiB (= 224 GiB, the hard-fit floor) DOES NOT FIT: -23.52 GiB
-> 3x64 GiB does not fit: -49.12 GiB
# ---------------------------------------------------------------------------
# 6. Quality gates (project .venv, deterministic, offline, GPU-free)
# ---------------------------------------------------------------------------
$VP -m pytest -q tests/test_glm_alpha_target.py
-> 97 passed in 0.12s
-> includes coordinated shard/config substitution, malformed telemetry, and
contract-ID reseal rejection tests added during controller review
$VP -m pip wheel --no-deps packages/node -w /tmp/dgr017-wheel
$VP -m pip install --no-deps --target /tmp/dgr017-install /tmp/dgr017-wheel/*.whl
$VP -I -c "... from meshnet_node.glm_alpha import load_locked_target ..."
-> INSTALLED_WHEEL_PASS
-> packaged alpha-contract.json, target-manifest.json, and architecture-snapshot.json
load and cross-bind successfully outside the source tree
$VP -m compileall -q packages tests
-> exit 0
git diff --check
-> exit 0
$VP -m pytest -q # full deterministic suite
-> first final run: 1 failed, 851 passed, 13 skipped; only the tracker cancellation
race already documented by DGR-001/DGR-002 failed
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 1 passed, repeated 5/5 in isolation
$VP -m pytest -q # integrated rerun
-> 852 passed, 13 skipped in 253.30s (0:04:13)
# ---------------------------------------------------------------------------
# 7. Late independent-review repair (2026-07-14)
# ---------------------------------------------------------------------------
PYTHONPATH=packages/node $VP -m pytest -q tests/test_glm_alpha_target.py
-> 99 passed in 0.15s
-> adds trusted-v1-digest rejection after coordinated mutation + reseal
-> adds nested parsed-state immutability and isolated to_dict() coverage
$VP -m pytest -q # after DGR-003 integration and DGR-017 repair
-> first run: 871 passed, 13 skipped, 1 known cancellation-race failure
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
-> 5/5 passed in isolation
$VP -m pytest -q # integrated rerun
-> 872 passed, 13 skipped in 253.46s (0:04:13)

255
.scratch/distributed-gguf-runtime/evidence/DGR-017/resource-plan.json Normal file
View File

@@ -0,0 +1,255 @@
{
"generated_by": "DGR-017 meshnet_node.glm_alpha.planner",
"target": {
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
"quantization": "UD-IQ1_S",
"total_bytes": 216715360960,
"total_gib": 201.832,
"total_gb": 216.715
},
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962",
"architecture_snapshot_sha256": "253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1",
"alpha_contract_sha256": "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9",
"kv_assumptions": {
"dtype": "Q8_0",
"bytes_per_value": 1.0625,
"context_tokens": 16384,
"concurrency": 1,
"indexer_layout": "conservative",
"mla_values_per_token_per_layer": 576,
"backbone_layers": 78,
"indexer_full_layers": 21,
"note": "Alpha budgets indexer keys across all 78 layers (current experimental DSA layout), not only the 21 Full layers."
},
"kv_table_gib": {
"16384": {
"mla_only_q8_gib": 0.73,
"optimized_dsa_q8_gib": 0.77,
"conservative_dsa_q8_gib": 0.89,
"conservative_dsa_f16_gib": 1.68
},
"131072": {
"mla_only_q8_gib": 5.83,
"optimized_dsa_q8_gib": 6.18,
"conservative_dsa_q8_gib": 7.12,
"conservative_dsa_f16_gib": 13.41
},
"1048576": {
"mla_only_q8_gib": 46.62,
"optimized_dsa_q8_gib": 49.41,
"conservative_dsa_q8_gib": 56.98,
"conservative_dsa_f16_gib": 107.25
}
},
"aggregate_hard_fit_floor_gib": 224.0,
"aggregate_floor_class": "experimental_hard_fit_floor",
"placement_imbalance_factor": 1.1,
"tier_table": {
"32": {
"physical_usable_gib": 32.0,
"reserve_gib": 8.0,
"placement_budget_gib": 24.0,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 9,
"recommended_nodes": 10,
"imbalance_factor": 1.1
},
"48": {
"physical_usable_gib": 48.0,
"reserve_gib": 9.6,
"placement_budget_gib": 38.4,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 6,
"recommended_nodes": 6,
"imbalance_factor": 1.1
},
"64": {
"physical_usable_gib": 64.0,
"reserve_gib": 12.8,
"placement_budget_gib": 51.2,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 4,
"recommended_nodes": 5,
"imbalance_factor": 1.1
},
"96": {
"physical_usable_gib": 96.0,
"reserve_gib": 19.2,
"placement_budget_gib": 76.8,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 3,
"recommended_nodes": 3,
"imbalance_factor": 1.1
},
"128": {
"physical_usable_gib": 128.0,
"reserve_gib": 25.6,
"placement_budget_gib": 102.4,
"weight_gib": 201.832,
"kv_gib": 0.89,
"total_placement_gib": 202.722,
"arithmetic_minimum_nodes": 2,
"recommended_nodes": 3,
"imbalance_factor": 1.1
}
},
"routes": {
"recommended_5x64_unified": {
"node_count": 5,
"aggregate_usable_gib": 320.0,
"aggregate_placement_budget_gib": 256.0,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 53.278,
"reasons": []
},
"recommended_3x96_unified": {
"node_count": 3,
"aggregate_usable_gib": 288.0,
"aggregate_placement_budget_gib": 230.4,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 27.678,
"reasons": []
},
"recommended_3x128_unified": {
"node_count": 3,
"aggregate_usable_gib": 384.0,
"aggregate_placement_budget_gib": 307.2,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 104.478,
"reasons": []
},
"fit_probe_2x128_unified": {
"node_count": 2,
"aggregate_usable_gib": 256.0,
"aggregate_placement_budget_gib": 204.8,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 2.078,
"reasons": []
},
"fit_probe_4x64_unified": {
"node_count": 4,
"aggregate_usable_gib": 256.0,
"aggregate_placement_budget_gib": 204.8,
"required_placement_gib": 202.722,
"fits": true,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": 2.078,
"reasons": []
},
"hard_fit_floor_2x112_unified": {
"node_count": 2,
"aggregate_usable_gib": 224.0,
"aggregate_placement_budget_gib": 179.2,
"required_placement_gib": 202.722,
"fits": false,
"meets_hard_fit_floor": true,
"no_single_node_can_admit_target": true,
"headroom_gib": -23.522,
"reasons": [
"aggregate placement budget 179.2 GiB is below the 202.7 GiB the target needs after each node's reserve"
]
},
"insufficient_3x64_unified": {
"node_count": 3,
"aggregate_usable_gib": 192.0,
"aggregate_placement_budget_gib": 153.6,
"required_placement_gib": 202.722,
"fits": false,
"meets_hard_fit_floor": false,
"no_single_node_can_admit_target": true,
"headroom_gib": -49.122,
"reasons": [
"aggregate placement budget 153.6 GiB is below the 202.7 GiB the target needs after each node's reserve",
"aggregate usable memory 192.0 GiB is below the 224 GiB experimental hard-fit floor"
]
}
},
"seams": {
"3_nodes_2.5gbe": {
"node_count": 3,
"seam_count": 2,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 2.5,
"meets_alpha_minimum": true,
"is_recommended_link": false,
"decode_serialization_ms_per_token": 0.0786,
"decode_latency_ms_per_token": 1.0,
"decode_bandwidth_share_ms_per_token": 0.0786,
"prefill_serialization_ms": 1288.49
},
"3_nodes_10.0gbe": {
"node_count": 3,
"seam_count": 2,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 10.0,
"meets_alpha_minimum": true,
"is_recommended_link": true,
"decode_serialization_ms_per_token": 0.0197,
"decode_latency_ms_per_token": 1.0,
"decode_bandwidth_share_ms_per_token": 0.0197,
"prefill_serialization_ms": 322.123
},
"5_nodes_2.5gbe": {
"node_count": 5,
"seam_count": 4,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 2.5,
"meets_alpha_minimum": true,
"is_recommended_link": false,
"decode_serialization_ms_per_token": 0.1573,
"decode_latency_ms_per_token": 2.0,
"decode_bandwidth_share_ms_per_token": 0.1573,
"prefill_serialization_ms": 2576.98
},
"5_nodes_10.0gbe": {
"node_count": 5,
"seam_count": 4,
"hidden_size": 6144,
"bytes_per_token_per_seam": 12288,
"prefill_bytes_per_seam": 201326592,
"decode_bytes_per_seam_per_token": 12288,
"dsa_sideband_bytes_per_query": 8192,
"link_rate_gbps": 10.0,
"meets_alpha_minimum": true,
"is_recommended_link": true,
"decode_serialization_ms_per_token": 0.0393,
"decode_latency_ms_per_token": 2.0,
"decode_bandwidth_share_ms_per_token": 0.0393,
"prefill_serialization_ms": 644.245
}
}
}

89
.scratch/distributed-gguf-runtime/evidence/DGR-017/upstream-status.json Normal file
View File

@@ -0,0 +1,89 @@
{
"observed_at": "2026-07-13",
"observed_by": "DGR-017",
"method": "GitHub REST API (api.github.com), read-only; no fork, no clone, no patch adopted",
"refresh_note": "The roadmap's 2026-07-13 observations were re-verified against live upstream. One item changed: PR #24231 is now MERGED (2026-07-11), which the roadmap already anticipated as 'generic CPU lightning-indexer support is merged'.",
"llama_cpp": {
"repo": "ggml-org/llama.cpp",
"items": [
{
"ref": "issue #24730",
"url": "https://github.com/ggml-org/llama.cpp/issues/24730",
"title": "Feature Request: Support for GLM 5.2",
"type": "issue",
"state": "open",
"updated_at": "2026-07-03T22:02:15Z",
"meaning": "The umbrella GLM-5.2 support request is still open. GLM-5.2 is not fully supported upstream."
},
{
"ref": "PR #24770",
"url": "https://github.com/ggml-org/llama.cpp/pull/24770",
"title": "model : glm-dsa load DSA indexer tensors as optional",
"type": "pull_request",
"state": "closed",
"merged_at": "2026-06-20T10:48:24Z",
"meaning": "MERGED. GLM-5.2 loads, but through a dense-MLA compatibility path with DSA indexer tensors treated as optional. This is the fallback the alpha contract explicitly refuses: it can produce text without performing DSA/IndexShare computation."
},
{
"ref": "PR #24231",
"url": "https://github.com/ggml-org/llama.cpp/pull/24231",
"title": "New GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer",
"type": "pull_request",
"state": "closed",
"merged_at": "2026-07-11T09:39:07Z",
"meaning": "MERGED since the roadmap was written. A generic lightning-indexer op now exists in GGML. Backend coverage beyond CPU remains uneven and must be verified per backend by DGR-018, not assumed."
},
{
"ref": "PR #25407",
"url": "https://github.com/ggml-org/llama.cpp/pull/25407",
"title": "GLM 5.2 Indexer support",
"type": "pull_request",
"state": "open",
"draft": false,
"mergeable_state": "unstable",
"head_sha": "8dedd06415f36f10fc6091241a39b23c1bf0ee11",
"base": "master",
"commits": 6,
"changed_files": 12,
"additions": 414,
"deletions": 7,
"updated_at": "2026-07-13T15:28:51Z",
"meaning": "OPEN and actively moving (updated today). This is the real DSA/IndexShare implementation alpha needs. It is narrow — 12 files, +414/-7 — which is the single most important finding for donor policy: the semantics alpha requires are reviewable and trackable upstream, not a 261-patch fork."
}
]
},
"capability_status_for_alpha": {
"gguf_load_of_UD-IQ1_S": "expected via merged #24770, unverified by this project; DGR-018 must prove it against the exact pinned artifact",
"moe_routing_and_shared_expert": "expected supported; unverified here",
"compressed_mla_kv": "supported via the dense-MLA compatibility path",
"dsa_lightning_indexer": "generic GGML op merged (#24231); GLM-5.2 wiring still open (#25407)",
"indexshare_full_shared_roles": "NOT upstream; only in open PR #25407",
"mtp_nextn": "not required for alpha; NextN tensors must be explicitly loaded or excluded, never silently reinterpreted",
"conclusion": "As of 2026-07-13 no released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare. A stock pin today would satisfy 'it emits text' via the dense fallback and would FAIL the alpha semantic-correctness contract. This is the gating technical risk for DGR-004 and DGR-018."
},
"donor": {
"repo": "Mesh-LLM/mesh-llm",
"url": "https://github.com/Mesh-LLM/mesh-llm",
"license": "Apache-2.0",
"stars_observed": 2048,
"pushed_at": "2026-07-13T06:45:51Z",
"glm_branch": "feat/jianyang-glm-52",
"glm_branch_head": "9bd18f1509dff7fac21578635084035b3ba90a38",
"glm_branch_head_date": "2026-07-12T06:37:43Z",
"policy": "TEST AND PATCH DONOR ONLY. Do not adopt the fork, its scheduler, discovery, routing, public mesh, or package manager. Meshnet remains the sole control plane (RALPH-CONTEXT runtime decision, ADR-0020).",
"focused_candidates": [
"GLM DSA graph semantics",
"lightning indexer and sparse-attention tests",
"IndexShare metadata and Full/Shared role validation",
"top-k sideband shape and lifecycle",
"stage-local KV filtering",
"target parity and performance fixtures"
],
"adoption_state": "none adopted in DGR-017. This story reads and records upstream state; it takes no patch and forks nothing."
},
"recommendation_for_dgr_004_and_dgr_018": [
"Track upstream PR #25407 rather than forking Mesh-LLM. At 12 files and +414/-7 it is small enough to review, reproduce, and carry as a numbered patch in the project's own pinned stack.",
"Any pin chosen before #25407 merges will load GLM-5.2 through the dense-MLA compatibility path. DGR-018 must therefore prove DSA/IndexShare are ACTIVE, not merely that the model emits text — the alpha contract already forbids the fallback.",
"Verify lightning-indexer backend coverage (#24231) on the specific backend the route will use. CPU support being merged says nothing about ROCm/HIP."
]
}

14
.scratch/distributed-gguf-runtime/implementation-strategy.md
View File

@@ -7,6 +7,8 @@ Last updated: 2026-07-13
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
The alpha-release target is the exact `zai-org/GLM-5.2` model, pinned by revision and served with `reasoning_effort=max`, using the smallest published Unsloth `UD-IQ1_S` GGUF across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama remains a cheap structural fixture; Qwen expansion is post-alpha.
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
1. Models larger than one node's RAM/VRAM.
@@ -36,7 +38,7 @@ GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kerne
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
Before expensive native work, establish controlled lanes:
Before expensive native work, establish controlled lanes. DGR-001 remains immutable; DGR-017 adds a target-specific fit and semantics contract without rewriting DGR-001 evidence:
- Same model architecture and upstream revision.
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
@@ -134,6 +136,10 @@ Use as source and test donors only:
Do not adopt or fork their repositories wholesale.
### Mesh-LLM GLM branch: focused test/patch donor only
Use its GLM-5.2 branch to study DSA, IndexShare, stage-local KV, and sideband tests. Do not import its scheduler, discovery/control plane, package manager, or broad llama.cpp patch stack. Every adopted idea must be independently understood, minimized, attributed, and tested against our exact pin.
## Battle-proven transport decision
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
@@ -155,7 +161,7 @@ Scope boundary:
- Large prefill tensors are chunked into bounded frames; decode bundles stay small.
- No QUIC/WebRTC/custom transport in this milestone.
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`.
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`. DGR-006 updates the current single-`NamedTensor` decode fast path to carry the same bundle semantics and adds an explicit typed tail logits/token result with sampling/template identity.
Minimum identity:
@@ -219,9 +225,9 @@ Two physical machines execute one model that uses both Shards. Synthetic activat
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
### Gate F: architecture expansion
### Gate F: exact GLM-5.2 alpha target
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
After the generic dense fixture proves range and boundary mechanics, certify explicit GLM-5.2 MoE, MLA KV, DSA, IndexShare, and NextN policy. Alpha requires the exact `UD-IQ1_S` target across physical consumer nodes, native Max-mode semantics, locked parity/usefulness/performance thresholds, and bounded failure cleanup. Qwen3/Qwen3-MoE is later architecture expansion.
## Scope discipline

2
.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md
View File

@@ -1,6 +1,6 @@
# 01 — Lock the safetensors-versus-GGUF performance contract
Status: ready-for-agent
Status: done
## Mandatory fresh-session context

34
.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md
View File

@@ -1,6 +1,6 @@
# 02 — Adopt the versioned gRPC Shard protocol
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -22,22 +22,22 @@ As a node developer, I need a battle-proven streaming protocol so that Python an
## Acceptance criteria
- [ ] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
- [ ] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
- [ ] Define bounded chunking for prefill and a small decode fast path.
- [ ] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
- [ ] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
- [ ] Add generated-schema round-trip and compatibility tests in Python and C++.
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
- [x] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
- [x] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
- [x] Define bounded chunking for prefill and a small decode fast path.
- [x] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
- [x] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
- [x] Add generated-schema round-trip and compatibility tests in Python and C++.
- [x] Targeted pytest tests pass
- [x] python -m compileall packages tests passes for Python changes
- [x] git diff --check passes
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [x] Read and verify every dependency evidence README before relying on dependency behavior
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
## Dependency handoff

34
.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md
View File

@@ -1,6 +1,6 @@
# 03 — Define exact Artifact and runtime recipe identity
Status: ready-for-agent
Status: done
## Mandatory fresh-session context
@@ -21,25 +21,25 @@ As the Tracker, I need exact compatibility identity so that only numerically and
## Acceptance criteria
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
- [ ] Targeted pytest tests pass
- [ ] python -m compileall packages tests passes for Python changes
- [ ] git diff --check passes
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
- [x] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
- [x] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
- [x] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
- [x] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
- [x] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
- [x] Targeted pytest tests pass
- [x] python -m compileall packages tests passes for Python changes
- [x] git diff --check passes
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
- [x] Read and verify every dependency evidence README before relying on dependency behavior
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
## Dependency handoff
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
- `DGR-002` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
## Finish contract

2
.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md
View File

@@ -43,7 +43,7 @@ As a maintainer, I need a small auditable fork boundary so that upstream updates
## Dependency handoff
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
- `DGR-001` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
## Finish contract

2
.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md
View File

@@ -23,6 +23,8 @@ As a Shard, I need to consume and emit the correct transformer boundary state so
- [ ] Head accepts token IDs and owns token embedding.
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
- [ ] Amend the decode fast path so `DecodeStep` carries a versioned `TensorBundle`, not one `NamedTensor`; retain a compact one-tensor encoding and fallback compatibility, regenerate Python/C++ schema code, and update protocol goldens.
- [ ] Define an explicit typed tail result for logits and/or sampled token output, with sampling parameters and chat-template/reasoning mode bound to exact request/recipe identity.
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
- [ ] Tail emits logits or token output through an explicit sampling contract.
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.

2
.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md
View File

@@ -42,7 +42,7 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
## Dependency handoff
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
- `DGR-006` and `DGR-019` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
## Finish contract

2
.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md
View File

@@ -44,7 +44,7 @@ As a client, I need failures to be bounded and explicit so that distributed spee
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
- DGR-012 continuous batching is post-alpha; this story must harden single-session failure/cancellation first and later extend the same invariants to batched sessions.
## Finish contract

62
.scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md Normal file
View File

@@ -0,0 +1,62 @@
# 17 — Lock the GLM-5.2 Max target and alpha contract
Status: done
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-017` in [prd.json](../prd.json).
- Read the evidence README for every dependency listed below.
- Inspect current code, upstream sources, and `git status`; historical text and previous agent claims are not evidence.
## Description
As a release owner, I need one exact GLM-5.2 Max target and immutable alpha contract so that later agents cannot swap artifacts, runtime semantics, hardware accounting, or thresholds after seeing results.
## Expected durable outputs
- Machine-readable official/GGUF target manifest with revisions, filenames, sizes, and resolved hashes
- Architecture/config/chat-template snapshot and source links
- Deterministic memory/KV/network planner with unified-memory de-duplication
- Current llama.cpp and donor-support status report
- Immutable machine-readable alpha acceptance contract
- `evidence/DGR-017/README.md`
## Acceptance criteria
- [ ] Pin `zai-org/GLM-5.2` and `unsloth/GLM-5.2-GGUF` by exact observed repository revisions and identify `UD-IQ1_S` as the alpha quantization.
- [ ] Resolve all six `UD-IQ1_S` filenames, exact byte sizes, LFS SHA-256 values, aggregate decimal GB/GiB, license, and source URLs without downloading the weight payloads.
- [ ] Snapshot and hash architecture-critical config/tokenizer/chat-template metadata, including main/NextN layer counts, hidden width, experts/top-k, DSA top-k, IndexShare roles, context maximum, and `reasoning_effort=max` behavior.
- [ ] Generate deterministic minimum-node calculations from exact artifact bytes, Q8_0 MLA/indexer KV at 16K context/concurrency 1, endpoint/tensor imbalance, and a per-node reserve of at least `max(20% of physically usable memory, 8 GiB)`.
- [ ] Classify 224 GiB aggregate runtime-accessible memory as an experimental hard-fit floor, not a conservative envelope; recommend 5×64 GiB or 3×96/128 GiB unless exact measured placement proves an arithmetic-minimum topology.
- [ ] Count unified system RAM/integrated-GPU memory once and reject additive RAM+VRAM claims for one shared pool.
- [ ] Lock same-switch wired 2.5 GbE as the alpha minimum and 10 GbE as recommended; model serial seam latency separately from bandwidth.
- [ ] Lock the identity, semantic correctness, target-run, performance, reliability, and mounted-storage criteria from the roadmap in a machine-readable contract before full target execution.
- [ ] Refresh and record current upstream llama.cpp GLM-5.2/DSA/IndexShare/MTP support and focused donor candidates; do not adopt a broad donor fork or scheduler.
- [ ] Add tests that reject changed revisions, missing shards, inconsistent aggregate bytes, duplicate unified-memory accounting, and post-result threshold mutation.
- [ ] Targeted pytest tests pass.
- [ ] `python -m compileall packages tests` passes for Python changes.
- [ ] `git diff --check` passes.
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
- [ ] Full deterministic `pytest -q` passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction.
- [ ] Read and verify every dependency evidence README before relying on dependency behavior.
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
- [ ] Write `evidence/DGR-017/README.md` with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff.
- [ ] Update only this story issue to `Status: done` after every acceptance criterion and quality gate passes.
## Dependency handoff
- `DGR-001` and `DGR-002` must have `passes: true`; read their evidence READMEs and verify referenced files/commands.
## Finish contract
- Create the task evidence directory and durable handoff required above.
- Preserve real failures and blockers; never fabricate source, benchmark, model, test, or hardware output.
- Change this issue to `Status: done` only after all criteria pass.
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [Ralph execution context](../RALPH-CONTEXT.md)
- [Current architecture](../architecture.md)

58
.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md Normal file
View File

@@ -0,0 +1,58 @@
# 18 — Certify whole-model GLM-5.2 runtime semantics
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-018` in [prd.json](../prd.json).
- Read and verify every dependency evidence README.
- Inspect current upstream behavior and `git status`; community claims that a model “works” are not correctness evidence.
## Description
As a runtime maintainer, I need a certified whole-model oracle for the exact lowest-quant GLM-5.2 artifact so that distributed parity is measured against correct MoE, DSA, IndexShare, cache, and Max-template semantics.
## Expected durable outputs
- Verified target artifact on mounted storage
- Stock-pin baseline and warning/tensor inventory
- Focused GLM runtime correctness tests and minimum patch series, if required
- Signed whole-model oracle recipe, deterministic outputs, metrics, and limitations
- `evidence/DGR-018/README.md`
## Acceptance criteria
- [ ] Preflight a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation and approximately 250 GB free mounted storage; abort before download if storage resolves under `/home` or requirements are unmet.
- [ ] Configure the oracle's alpha lane for 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; record actual MLA, indexer-cache, scratch, and peak resident allocations separately.
- [ ] Download/resume and verify all six DGR-017 `UD-IQ1_S` files against exact sizes and LFS SHA-256 values.
- [ ] Load the target with the unmodified DGR-004 pin first and retain full logs, tensor warnings, peak memory, context/KV allocation, TTFT, and decode timing.
- [ ] Prove from graph/runtime evidence whether 256-expert MoE/top-8/shared expert, DSA lightning indexer/sparse attention, IndexShare Full/Shared reuse, and `reasoning_effort=max` are active.
- [ ] Dense-attention or replicated-indexer compatibility fallback is labeled incomplete and cannot become the oracle solely because it emits plausible text.
- [ ] Audit focused upstream/Mesh-LLM donor tests and patches; adopt or rewrite only independently understood minimum changes and record provenance/rejection rationale.
- [ ] Handle the trailing NextN/MTP tensors explicitly; MTP may remain disabled for alpha, but ignored tensors and layer-count behavior must be understood and tested.
- [ ] Run deterministic prefill/greedy decode and fixed Max-mode coding, structured-output/tool-call, and reasoning sentinels; retain raw prompts, outputs, token IDs, and compared state/logit evidence.
- [ ] Bind the oracle to exact artifact, tokenizer/template, adapter, backend class, compute/activation/KV dtypes, context/RoPE, llama.cpp commit, and patch-series hash.
- [ ] Produce a signed `pass` or `stop` semantic-runtime verdict; no performance threshold is weakened after execution.
- [ ] Targeted native/unit tests and clean pinned rebuild pass.
- [ ] `git diff --check` passes.
- [ ] Default tests remain model-download-free/GPU-free; full-target execution is opt-in and never runs in default CI.
- [ ] Model artifacts and caches remain on configured mounted-drive storage and never under `/home`.
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
- [ ] Write `evidence/DGR-018/README.md` with exact hardware, storage, source revisions, files, commands, raw-result paths, real results, limitations, and handoff.
- [ ] Update only this story issue to `Status: done` after every acceptance criterion passes.
## Dependency handoff
- `DGR-003`, `DGR-004`, and `DGR-017` must have `passes: true`; read and verify their evidence READMEs.
## Finish contract
- If a 256-GiB-class host with at least 224 GiB runtime-accessible memory is unavailable, write `evidence/DGR-018/BLOCKED.md` with exact preflight output; do not substitute a smaller model.
- Preserve real failures and blockers; never fabricate output.
- Emit `<promise>COMPLETE</promise>` only after the signed oracle evidence exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [llama.cpp GLM-5.2 issue](https://github.com/ggml-org/llama.cpp/issues/24730)

58
.scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md Normal file
View File

@@ -0,0 +1,58 @@
# 19 — Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-019` in [prd.json](../prd.json).
- Read and verify every dependency evidence README.
- Inspect current source and `git status`; do not generalize through tensor-name substitution.
## Description
As a target-model operator, I need explicit range-owned GLM-5.2 semantics so that contiguous consumer Shards preserve the whole-model MoE, DSA, IndexShare, and local-KV computation.
## Expected durable outputs
- Explicit GLM-5.2 architecture adapter and tensor-ownership rules
- IndexShare-aware Shard planner and named DSA sideband protocol mapping
- GLM architecture fixture and target parity evidence
- Memory/tensor/KV ownership report
- `evidence/DGR-019/README.md`
## Acceptance criteria
- [ ] Implement explicit ownership for 78 main layers, head embedding, tail norm/output head, routed/shared experts, DSA/indexer tensors, and the certified NextN/MTP policy.
- [ ] Each contiguous layer owner keeps all 256 experts and the shared-expert path for its layers local; no public cross-machine expert collectives are introduced.
- [ ] Define and fixture-test compressed MLA KV ownership for locally owned layers, keyed by Route Session/epoch, for DGR-007 to implement.
- [ ] Implement native DSA lightning indexer/top-2,048 and sparse-attention graph behavior matching DGR-018.
- [ ] Parse and validate artifact `indexer_types`; implement Full producer and Shared consumer behavior without fabricated/duplicated indexer tensors.
- [ ] Prefer Shard boundaries that preserve IndexShare ownership groups; when memory fit forces a split, carry a typed, bounded, validated top-k sideband in the DGR-006 named bundle.
- [ ] Reject missing, stale, wrong-width, wrong-position, or Shared-before-Full sideband/index state.
- [ ] Demonstrate mapped/resident weights and allocated KV scale with exact owned tensors/layers rather than the full model.
- [ ] Same-host two-stage F32 seam fixture produces 32 exact greedy tokens against the DGR-018 semantics; production seam meets the pre-locked token/similarity thresholds.
- [ ] If full-target same-host execution cannot fit, use a layer-reduced GLM architecture fixture only for graph parity and defer full-artifact parity to DGR-020; label it non-target evidence.
- [ ] Add deterministic fixture tests for MoE route/shared expert, DSA Full/Shared groups, internal group split, endpoint ownership, KV filtering, malformed metadata, and NextN policy.
- [ ] Targeted pytest/CTest/native tests pass; pinned patch stack applies and rebuilds cleanly.
- [ ] `python -m compileall packages tests` and `git diff --check` pass.
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
- [ ] Full deterministic `pytest -q` passes, or exact pre-existing unrelated failures are recorded.
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
- [ ] Write `evidence/DGR-019/README.md` with files, commands, real results, raw evidence, limitations, and dependent-story handoff.
- [ ] Update only this story issue to `Status: done` after every acceptance criterion passes.
## Dependency handoff
- `DGR-005`, `DGR-006`, and `DGR-018` must have `passes: true`; read and verify their evidence READMEs.
- DGR-007 integration may be completed in parallel only after the adapter's cache contract is explicit.
## Finish contract
- Preserve real failures and blockers; never claim target parity from the architecture fixture.
- Emit `<promise>COMPLETE</promise>` only after evidence exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [Current architecture](../architecture.md)

62
.scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md Normal file
View File

@@ -0,0 +1,62 @@
# 20 — Pass real distributed GLM-5.2 Max alpha acceptance
Status: ready-for-agent
## Mandatory fresh-session context
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
- This issue is `DGR-020` in [prd.json](../prd.json).
- Read and verify every dependency evidence README and the immutable DGR-017 contract.
- Inspect live source, hardware, network, storage, and `git status`; synthetic or layer-reduced evidence cannot satisfy this story.
## Description
As an alpha user, I need the exact lowest-quant GLM-5.2 model to run in Max reasoning mode across consumer machines through Meshnet so that the project has proven its end goal on real hardware.
## Expected durable outputs
- Exact multi-node hardware/network/storage/runtime manifest
- Signed target identity, ownership, parity, quality, performance, and reliability reports
- Raw worker/tracker/API logs, metrics, prompts, outputs, and cleanup evidence
- Explicit immutable `alpha` or `stop` verdict
- `evidence/DGR-020/README.md`
## Acceptance criteria
- [ ] Use the exact DGR-017 `UD-IQ1_S` artifact and certified DGR-018/DGR-019 runtime recipe with all source and patch hashes verified.
- [ ] Use at least two physical consumer machines; each reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-Q8-KV placement, no node can place the complete recipe, unified memory is counted once, and measured peak use stays within physical memory without swap/overcommit.
- [ ] Treat 5×64 GiB or 3×96/128 GiB as the recommended topology; arithmetic-minimum 4×64 or 2×128 qualifies only with exact contiguous placement and measured reserve evidence.
- [ ] Use a same-switch wired network with at least 2.5 GbE; record 10 GbE as recommended and measure hop RTT/serialization/queue latency.
- [ ] Tracker selects disjoint contiguous Shards whose exact required tensor inventory has complete union and no unintended overlap.
- [ ] Every stage reports real CPU/GPU compute, owned tensor bytes/layers, local KV, backend, queue, and seam telemetry; synthetic, passthrough, or zero-layer workers fail acceptance.
- [ ] Runtime evidence proves native GLM MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared paths are active; dense fallback fails acceptance.
- [ ] OpenAI-compatible API applies and records `reasoning_effort=max`, stable model ID, finish reason, and token usage.
- [ ] Configure 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; complete the fixed 4,096-token prefill lane and generate at least 512 output tokens or valid natural EOS after at least 128 tokens.
- [ ] Production seam meets at least 0.90 greedy token agreement and 0.999 mean compared-state/logit cosine similarity against DGR-018 on the fixed corpus, with no non-finite or malformed tensors.
- [ ] Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable relevant outputs retained for human review.
- [ ] After warm-up, fixed Max lane median decode is at least 0.5 token/s, 4,096-token-prompt TTFT is at most 10 minutes, and no unexplained stall exceeds 60 seconds without progress telemetry.
- [ ] Record per-stage/seam p50/p95 latency, bytes, compute time, peak RSS/VRAM, KV pressure, network properties, errors, and total energy when available.
- [ ] Two consecutive cold starts load, generate, release, and exit without leaked processes, mapped weights, queues, or KV leases.
- [ ] Cancellation in prefill and decode releases all stages; one worker loss aborts the route; retry starts from token zero on a newly compatible route; stale epochs and duplicate steps fail closed.
- [ ] Model artifacts/caches remain on configured mounted-drive storage and never under `/home`; secret scan passes.
- [ ] Run targeted/full deterministic tests and clean native rebuild in addition to target execution; `git diff --check` passes.
- [ ] Preserve all raw failures and do not weaken DGR-017 thresholds after results.
- [ ] Write a signed `alpha` verdict only if every criterion passes; otherwise write `stop` with the measured bottleneck and next narrow action.
- [ ] Write `evidence/DGR-020/README.md` with exact commands, manifests, raw-result paths, outputs, limitations, and post-alpha handoff.
- [ ] Update only this story issue to `Status: done` after the immutable verdict and all evidence exist.
## Dependency handoff
- `DGR-007`, `DGR-008`, `DGR-009`, `DGR-011`, `DGR-013`, `DGR-017`, `DGR-018`, and `DGR-019` must have `passes: true`; read and verify their evidence READMEs.
- DGR-012 continuous batching and DGR-014 final comparative release gate are post-alpha and do not block one target session.
## Finish contract
- If required physical nodes/storage are unavailable, write `evidence/DGR-020/BLOCKED.md`; do not substitute a smaller model, API, synthetic worker, or single host.
- Preserve real failures and blockers; never fabricate target output or telemetry.
- Emit `<promise>COMPLETE</promise>` only after the signed verdict exists.
## References
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
- [Ralph execution context](../RALPH-CONTEXT.md)

50
.scratch/distributed-gguf-runtime/milestones.md
View File

@@ -1,35 +1,47 @@
# Distributed GGUF runtime milestones
## Gate A — measured runtime value
The exact alpha target and immutable acceptance gates are defined in [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md).
- DGR-001 locks the safetensors-versus-GGUF performance/fit/quality contract.
- DGR-002 can proceed independently and defines the battle-proven backend-neutral wire protocol.
- DGR-003 builds exact recipe identity on DGR-002.
- Expensive native llama.cpp work remains gated by DGR-001.
## Completed foundation
- DGR-001 locks the safetensors-versus-GGUF performance/fit/quality contract. Its immutable CPU v1 verdict remains `stop`; separate ROCm evidence supports a fit-oriented investigation without rewriting CPU evidence.
- DGR-002 defines the backend-neutral gRPC/Protobuf Shard protocol.
## Gate A — exact GLM-5.2 target and oracle
- DGR-017 locks official/GGUF revisions, `UD-IQ1_S`, hashes, resource accounting, Max-mode semantics, and alpha thresholds.
- DGR-003 builds exact recipe identity on DGR-002 and DGR-017.
- DGR-004 creates the reproducible pinned llama.cpp boundary after stock GLM behavior is measured.
- DGR-018 certifies one whole-model `UD-IQ1_S` oracle with real MoE, DSA, IndexShare, KV, and `reasoning_effort=max` semantics.
## Gate B — minimal native execution seam
- DGR-004 creates the reproducible pinned fork boundary.
- DGR-005 implements dense-Llama range ownership.
- DGR-006 proves architecture-defined boundary parity.
- DGR-005 implements range-owned tensors using dense Llama as a cheap structural fixture.
- DGR-006 proves the generic named activation boundary and F32 correctness lane.
- DGR-019 implements explicit GLM-5.2 MoE/MLA/DSA/IndexShare/NextN range semantics and parity.
## Gate C — concurrent production worker
## Gate C — native Meshnet route
- DGR-007 isolates concurrent Hot KV State.
- DGR-007 isolates Shard-local Hot KV State.
- DGR-008 exposes the native worker over gRPC.
- DGR-009 integrates the worker without replacing Meshnet's control plane.
- DGR-010 passes local real-model two-process acceptance.
- DGR-011 passes real two-physical-machine execution.
- DGR-013 supplies the cancellation/node-loss/restart/cleanup subset required by alpha.
## Gate D — real consumer-hardware route
## Gate D — GLM-5.2 Max alpha verdict
- DGR-020 runs the exact `UD-IQ1_S` target across enough physical consumer nodes that no one node can admit the whole recipe.
- It produces an immutable `alpha` or `stop` verdict from target identity, native GLM semantics, parity, Max-mode usefulness, minimum speed, telemetry, failure, and cleanup evidence.
- Synthetic workers, layer-reduced fixtures, dense-attention fallbacks, and single-host execution cannot satisfy this gate.
## Gate E — post-alpha product hardening
- DGR-011 passes two-physical-machine execution.
- DGR-012 adds continuous batching and bounded admission.
- DGR-013 hardens failure and cancellation.
## Gate E — product release decision
- DGR-014 compares distributed GGUF against the current distributed safetensors route under locked thresholds.
- DGR-015 adds Qwen3/Qwen3-MoE only after the dense runtime passes.
- DGR-014 compares distributed GGUF against the current reference route under locked thresholds.
- Longer contexts progress through 32K/128K/200K before 1M certification.
- MTP/speculative decoding and source-bound layer packages remain measured optimizations.
- DGR-016 prepares narrow upstream llama.cpp collaboration material.
- DGR-015 adds Qwen3/Qwen3-MoE only as later architecture expansion.
No later gate may be claimed from synthetic workers or documentation-only evidence.
No later gate may be claimed from synthetic workers or documentation-only evidence. Model artifacts remain on mounted-drive storage and never under `/home`.

226
.scratch/distributed-gguf-runtime/prd.json
View File

@@ -1,12 +1,12 @@
{
"name": "Performant Concurrent Distributed GGUF Runtime",
"branchName": "ralph/performant-concurrent-distributed-gguf",
"description": "Benchmark-gated native llama.cpp/GGUF Shards with gRPC streaming, concurrent local KV, continuous batching, real heterogeneous acceptance, and a measured release gate against Transformers/safetensors.",
"description": "Build a performant concurrent distributed GGUF runtime on Meshnet. Alpha is gated by exact GLM-5.2 UD-IQ1_S execution in reasoning_effort=max mode across consumer machines; synthetic or dense-fallback evidence cannot satisfy the target.",
"userStories": [
{
"id": "DGR-001",
"title": "Lock the safetensors-versus-GGUF performance contract",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.",
"acceptanceCriteria": [
"Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.",
"Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.",
@@ -26,15 +26,15 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-001/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 2,
"passes": false,
"priority": 1,
"passes": true,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md",
"dependsOn": []
},
{
"id": "DGR-002",
"title": "Adopt the versioned gRPC Shard protocol",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.",
"acceptanceCriteria": [
"Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.",
"Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.",
@@ -53,15 +53,15 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 1,
"passes": false,
"priority": 2,
"passes": true,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md",
"dependsOn": []
},
{
"id": "DGR-003",
"title": "Define exact Artifact and runtime recipe identity",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.",
"acceptanceCriteria": [
"Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.",
"Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.",
@@ -79,17 +79,18 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 3,
"passes": false,
"priority": 4,
"passes": true,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md",
"dependsOn": [
"DGR-002"
"DGR-002",
"DGR-017"
]
},
{
"id": "DGR-004",
"title": "Create the reproducible pinned llama.cpp patch stack",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.",
"acceptanceCriteria": [
"Pin one exact llama.cpp commit through a reproducible source dependency mechanism.",
"Store a numbered minimal patch stack separately from Meshnet networking code.",
@@ -110,17 +111,18 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 4,
"priority": 5,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md",
"dependsOn": [
"DGR-001"
"DGR-001",
"DGR-017"
]
},
{
"id": "DGR-005",
"title": "Implement dense-Llama range-aware GGUF ownership",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.",
"acceptanceCriteria": [
"Register and allocate only `blk.N.*` tensors in the assigned range.",
"Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.",
@@ -140,7 +142,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-005/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 5,
"priority": 6,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md",
"dependsOn": [
@@ -151,10 +153,12 @@
{
"id": "DGR-006",
"title": "Implement architecture-defined boundary input/output",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.",
"acceptanceCriteria": [
"Head accepts token IDs and owns token embedding.",
"Middle/tail bypass token embedding and accept the named boundary bundle.",
"Amend the decode fast path so `DecodeStep` carries a versioned `TensorBundle`, not one `NamedTensor`; retain a compact one-tensor encoding and fallback compatibility, regenerate Python/C++ schema code, and update protocol goldens.",
"Define an explicit typed tail result for logits and/or sampled token output, with sampling parameters and chat-template/reasoning mode bound to exact request/recipe identity.",
"Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.",
"Tail emits logits or token output through an explicit sampling contract.",
"Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.",
@@ -172,7 +176,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-006/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 6,
"priority": 7,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md",
"dependsOn": [
@@ -183,7 +187,7 @@
{
"id": "DGR-007",
"title": "Add isolated concurrent local Hot KV State",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.",
"acceptanceCriteria": [
"Map `(Route Session ID, route epoch)` to an isolated llama sequence or bounded context.",
"Allocate KV only for owned layers.",
@@ -204,17 +208,18 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-007/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 7,
"priority": 10,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md",
"dependsOn": [
"DGR-006"
"DGR-006",
"DGR-019"
]
},
{
"id": "DGR-008",
"title": "Build the standalone C++ gRPC Shard worker",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.",
"acceptanceCriteria": [
"Worker exposes capability, health, session stream, release, cancellation, and metrics services from DGR-002.",
"Worker loads one exact Artifact/recipe/Shard identity and refuses mismatched requests.",
@@ -235,7 +240,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-008/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 8,
"priority": 11,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md",
"dependsOn": [
@@ -249,7 +254,7 @@
{
"id": "DGR-009",
"title": "Integrate the native worker with Meshnet",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.",
"acceptanceCriteria": [
"Implement the existing model-backend surface without changing Transformers behavior.",
"Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.",
@@ -270,7 +275,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 9,
"priority": 12,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md",
"dependsOn": [
@@ -281,7 +286,7 @@
{
"id": "DGR-010",
"title": "Pass local real-model two-process acceptance",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a release engineer, I need real local distributed parity before involving network variability.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a release engineer, I need real local distributed parity before involving network variability.",
"acceptanceCriteria": [
"Two local worker processes open disjoint dense-Llama ranges from the certified Artifact.",
"Prefill and at least 32 greedy decode tokens match whole-model llama.cpp within the certified tolerance.",
@@ -304,7 +309,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-010/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 10,
"priority": 13,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md",
"dependsOn": [
@@ -314,7 +319,7 @@
{
"id": "DGR-011",
"title": "Pass a real heterogeneous two-machine route",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.",
"acceptanceCriteria": [
"Tracker selects two physical nodes with disjoint Shards and one exact certified recipe/compatibility class.",
"Actual CPU/GPU execution occurs on both nodes; synthetic workers do not satisfy acceptance.",
@@ -337,7 +342,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-011/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 11,
"priority": 14,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md",
"dependsOn": [
@@ -347,7 +352,7 @@
{
"id": "DGR-012",
"title": "Implement continuous batching and bounded admission",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.",
"acceptanceCriteria": [
"Node scheduler admits sessions against weight, KV, scratch, and queue budgets.",
"Compatible decode steps from multiple sessions form llama.cpp batches while preserving per-session positions and outputs.",
@@ -368,7 +373,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-012/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 12,
"priority": 17,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md",
"dependsOn": [
@@ -380,7 +385,7 @@
{
"id": "DGR-013",
"title": "Harden failure, cancellation, and restart semantics",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.",
"acceptanceCriteria": [
"Deadlines and heartbeat/health loss terminate blocked stream operations.",
"Cancellation propagates across every Shard and releases local KV and queued buffers.",
@@ -401,19 +406,18 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-013/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 13,
"priority": 15,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md",
"dependsOn": [
"DGR-008",
"DGR-009",
"DGR-012"
"DGR-009"
]
},
{
"id": "DGR-014",
"title": "Enforce the GGUF-versus-safetensors release gate",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.",
"acceptanceCriteria": [
"Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.",
"Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.",
@@ -435,7 +439,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-014/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 14,
"priority": 18,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md",
"dependsOn": [
@@ -448,7 +452,7 @@
{
"id": "DGR-015",
"title": "Add and certify a Qwen3/Qwen3-MoE adapter",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.",
"acceptanceCriteria": [
"Implement explicit tensor ownership, router/top-k, expert/shared-expert, Q/K normalization, boundary bundle, and cache semantics for the selected Qwen3 family recipe.",
"Do not reuse the dense-Llama adapter through unchecked name substitutions.",
@@ -470,7 +474,7 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-015/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 15,
"priority": 20,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md",
"dependsOn": [
@@ -480,7 +484,7 @@
{
"id": "DGR-016",
"title": "Produce the upstream llama.cpp collaboration package",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker\u2014not a stitched collection of runtimes.\n\nAs a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md` and `.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md` completely before coding. Read the evidence handoff for every dependency. The global goal is performant concurrent inference for models larger than one consumer node, using Meshnet as the sole control plane, gRPC/Protobuf as the Shard protocol, and a small pinned llama.cpp worker—not a stitched collection of runtimes.\n\nAs a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.",
"acceptanceCriteria": [
"Separate generic llama.cpp hooks from Meshnet protocol/control-plane code.",
"Prepare minimal reproducible examples and tests for range-aware loading, boundary input/output, and layer-filtered KV.",
@@ -500,12 +504,152 @@
"Write .scratch/distributed-gguf-runtime/evidence/DGR-016/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
],
"priority": 16,
"priority": 19,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md",
"dependsOn": [
"DGR-010"
]
},
{
"id": "DGR-017",
"title": "Lock the GLM-5.2 Max target and alpha contract",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md` completely before coding. Read and verify every dependency evidence README.\n\nAs a release owner, I need one exact GLM-5.2 Max target and immutable alpha contract so that later agents cannot swap artifacts, runtime semantics, hardware accounting, or thresholds after seeing results.",
"acceptanceCriteria": [
"Pin `zai-org/GLM-5.2` and `unsloth/GLM-5.2-GGUF` by exact observed repository revisions and identify `UD-IQ1_S` as the alpha quantization.",
"Resolve all six `UD-IQ1_S` filenames, exact byte sizes, LFS SHA-256 values, aggregate decimal GB/GiB, license, and source URLs without downloading the weight payloads.",
"Snapshot and hash architecture-critical config/tokenizer/chat-template metadata, including main/NextN layer counts, hidden width, experts/top-k, DSA top-k, IndexShare roles, context maximum, and `reasoning_effort=max` behavior.",
"Generate deterministic minimum-node calculations from exact artifact bytes, Q8_0 MLA/indexer KV at 16K context/concurrency 1, endpoint/tensor imbalance, and a per-node reserve of at least `max(20% of physically usable memory, 8 GiB)`.",
"Classify 224 GiB aggregate runtime-accessible memory as an experimental hard-fit floor, not a conservative envelope; recommend 5×64 GiB or 3×96/128 GiB unless exact measured placement proves an arithmetic-minimum topology.",
"Count unified system RAM/integrated-GPU memory once and reject additive RAM+VRAM claims for one shared pool.",
"Lock same-switch wired 2.5 GbE as the alpha minimum and 10 GbE as recommended; model serial seam latency separately from bandwidth.",
"Lock the identity, semantic correctness, target-run, performance, reliability, and mounted-storage criteria from the roadmap in a machine-readable contract before full target execution.",
"Refresh and record current upstream llama.cpp GLM-5.2/DSA/IndexShare/MTP support and focused donor candidates; do not adopt a broad donor fork or scheduler.",
"Add tests that reject changed revisions, missing shards, inconsistent aggregate bytes, duplicate unified-memory accounting, and post-result threshold mutation.",
"Targeted pytest tests pass.",
"`python -m compileall packages tests` passes for Python changes.",
"`git diff --check` passes.",
"Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.",
"Full deterministic `pytest -q` passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction.",
"Read and verify every dependency evidence README before relying on dependency behavior.",
"Preserve all pre-existing working-tree changes and stage only files belonging to this story.",
"Write `evidence/DGR-017/README.md` with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff.",
"Update only this story issue to `Status: done` after every acceptance criterion and quality gate passes."
],
"priority": 3,
"passes": true,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md",
"dependsOn": [
"DGR-001",
"DGR-002"
]
},
{
"id": "DGR-018",
"title": "Certify whole-model GLM-5.2 runtime semantics",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md` completely before coding. Read and verify every dependency evidence README.\n\nAs a runtime maintainer, I need a certified whole-model oracle for the exact lowest-quant GLM-5.2 artifact so that distributed parity is measured against correct MoE, DSA, IndexShare, cache, and Max-template semantics.",
"acceptanceCriteria": [
"Preflight a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation and approximately 250 GB free mounted storage; abort before download if storage resolves under `/home` or requirements are unmet.",
"Configure the oracle's alpha lane for 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; record actual MLA, indexer-cache, scratch, and peak resident allocations separately.",
"Download/resume and verify all six DGR-017 `UD-IQ1_S` files against exact sizes and LFS SHA-256 values.",
"Load the target with the unmodified DGR-004 pin first and retain full logs, tensor warnings, peak memory, context/KV allocation, TTFT, and decode timing.",
"Prove from graph/runtime evidence whether 256-expert MoE/top-8/shared expert, DSA lightning indexer/sparse attention, IndexShare Full/Shared reuse, and `reasoning_effort=max` are active.",
"Dense-attention or replicated-indexer compatibility fallback is labeled incomplete and cannot become the oracle solely because it emits plausible text.",
"Audit focused upstream/Mesh-LLM donor tests and patches; adopt or rewrite only independently understood minimum changes and record provenance/rejection rationale.",
"Handle the trailing NextN/MTP tensors explicitly; MTP may remain disabled for alpha, but ignored tensors and layer-count behavior must be understood and tested.",
"Run deterministic prefill/greedy decode and fixed Max-mode coding, structured-output/tool-call, and reasoning sentinels; retain raw prompts, outputs, token IDs, and compared state/logit evidence.",
"Bind the oracle to exact artifact, tokenizer/template, adapter, backend class, compute/activation/KV dtypes, context/RoPE, llama.cpp commit, and patch-series hash.",
"Produce a signed `pass` or `stop` semantic-runtime verdict; no performance threshold is weakened after execution.",
"Targeted native/unit tests and clean pinned rebuild pass.",
"`git diff --check` passes.",
"Default tests remain model-download-free/GPU-free; full-target execution is opt-in and never runs in default CI.",
"Model artifacts and caches remain on configured mounted-drive storage and never under `/home`.",
"Preserve all pre-existing working-tree changes and stage only files belonging to this story.",
"Write `evidence/DGR-018/README.md` with exact hardware, storage, source revisions, files, commands, raw-result paths, real results, limitations, and handoff.",
"Update only this story issue to `Status: done` after every acceptance criterion passes."
],
"priority": 8,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md",
"dependsOn": [
"DGR-003",
"DGR-004",
"DGR-017"
]
},
{
"id": "DGR-019",
"title": "Implement and certify GLM-5.2 range, DSA, and IndexShare semantics",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md` completely before coding. Read and verify every dependency evidence README.\n\nAs a target-model operator, I need explicit range-owned GLM-5.2 semantics so that contiguous consumer Shards preserve the whole-model MoE, DSA, IndexShare, and local-KV computation.",
"acceptanceCriteria": [
"Implement explicit ownership for 78 main layers, head embedding, tail norm/output head, routed/shared experts, DSA/indexer tensors, and the certified NextN/MTP policy.",
"Each contiguous layer owner keeps all 256 experts and the shared-expert path for its layers local; no public cross-machine expert collectives are introduced.",
"Define and fixture-test compressed MLA KV ownership for locally owned layers, keyed by Route Session/epoch, for DGR-007 to implement.",
"Implement native DSA lightning indexer/top-2,048 and sparse-attention graph behavior matching DGR-018.",
"Parse and validate artifact `indexer_types`; implement Full producer and Shared consumer behavior without fabricated/duplicated indexer tensors.",
"Prefer Shard boundaries that preserve IndexShare ownership groups; when memory fit forces a split, carry a typed, bounded, validated top-k sideband in the DGR-006 named bundle.",
"Reject missing, stale, wrong-width, wrong-position, or Shared-before-Full sideband/index state.",
"Demonstrate mapped/resident weights and allocated KV scale with exact owned tensors/layers rather than the full model.",
"Same-host two-stage F32 seam fixture produces 32 exact greedy tokens against the DGR-018 semantics; production seam meets the pre-locked token/similarity thresholds.",
"If full-target same-host execution cannot fit, use a layer-reduced GLM architecture fixture only for graph parity and defer full-artifact parity to DGR-020; label it non-target evidence.",
"Add deterministic fixture tests for MoE route/shared expert, DSA Full/Shared groups, internal group split, endpoint ownership, KV filtering, malformed metadata, and NextN policy.",
"Targeted pytest/CTest/native tests pass; pinned patch stack applies and rebuilds cleanly.",
"`python -m compileall packages tests` and `git diff --check` pass.",
"Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.",
"Full deterministic `pytest -q` passes, or exact pre-existing unrelated failures are recorded.",
"Preserve all pre-existing working-tree changes and stage only files belonging to this story.",
"Write `evidence/DGR-019/README.md` with files, commands, real results, raw evidence, limitations, and dependent-story handoff.",
"Update only this story issue to `Status: done` after every acceptance criterion passes."
],
"priority": 9,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md",
"dependsOn": [
"DGR-005",
"DGR-006",
"DGR-018"
]
},
{
"id": "DGR-020",
"title": "Pass real distributed GLM-5.2 Max alpha acceptance",
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md` completely before coding. Read and verify every dependency evidence README.\n\nAs an alpha user, I need the exact lowest-quant GLM-5.2 model to run in Max reasoning mode across consumer machines through Meshnet so that the project has proven its end goal on real hardware.",
"acceptanceCriteria": [
"Use the exact DGR-017 `UD-IQ1_S` artifact and certified DGR-018/DGR-019 runtime recipe with all source and patch hashes verified.",
"Use at least two physical consumer machines; each reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-Q8-KV placement, no node can place the complete recipe, unified memory is counted once, and measured peak use stays within physical memory without swap/overcommit.",
"Treat 5×64 GiB or 3×96/128 GiB as the recommended topology; arithmetic-minimum 4×64 or 2×128 qualifies only with exact contiguous placement and measured reserve evidence.",
"Use a same-switch wired network with at least 2.5 GbE; record 10 GbE as recommended and measure hop RTT/serialization/queue latency.",
"Tracker selects disjoint contiguous Shards whose exact required tensor inventory has complete union and no unintended overlap.",
"Every stage reports real CPU/GPU compute, owned tensor bytes/layers, local KV, backend, queue, and seam telemetry; synthetic, passthrough, or zero-layer workers fail acceptance.",
"Runtime evidence proves native GLM MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared paths are active; dense fallback fails acceptance.",
"OpenAI-compatible API applies and records `reasoning_effort=max`, stable model ID, finish reason, and token usage.",
"Configure 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; complete the fixed 4,096-token prefill lane and generate at least 512 output tokens or valid natural EOS after at least 128 tokens.",
"Production seam meets at least 0.90 greedy token agreement and 0.999 mean compared-state/logit cosine similarity against DGR-018 on the fixed corpus, with no non-finite or malformed tensors.",
"Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable relevant outputs retained for human review.",
"After warm-up, fixed Max lane median decode is at least 0.5 token/s, 4,096-token-prompt TTFT is at most 10 minutes, and no unexplained stall exceeds 60 seconds without progress telemetry.",
"Record per-stage/seam p50/p95 latency, bytes, compute time, peak RSS/VRAM, KV pressure, network properties, errors, and total energy when available.",
"Two consecutive cold starts load, generate, release, and exit without leaked processes, mapped weights, queues, or KV leases.",
"Cancellation in prefill and decode releases all stages; one worker loss aborts the route; retry starts from token zero on a newly compatible route; stale epochs and duplicate steps fail closed.",
"Model artifacts/caches remain on configured mounted-drive storage and never under `/home`; secret scan passes.",
"Run targeted/full deterministic tests and clean native rebuild in addition to target execution; `git diff --check` passes.",
"Preserve all raw failures and do not weaken DGR-017 thresholds after results.",
"Write a signed `alpha` verdict only if every criterion passes; otherwise write `stop` with the measured bottleneck and next narrow action.",
"Write `evidence/DGR-020/README.md` with exact commands, manifests, raw-result paths, outputs, limitations, and post-alpha handoff.",
"Update only this story issue to `Status: done` after the immutable verdict and all evidence exist."
],
"priority": 16,
"passes": false,
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md",
"dependsOn": [
"DGR-007",
"DGR-008",
"DGR-009",
"DGR-011",
"DGR-013",
"DGR-017",
"DGR-018",
"DGR-019"
]
}
]
}

11
.scratch/distributed-gguf-runtime/trusted-evidence-signers.json Normal file
View File

@@ -0,0 +1,11 @@
{
"schema_version": 1,
"signers": [
{
"algorithm": "ed25519",
"fingerprint_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
"scope": "DGR-001 local-real and gpu-diagnostic benchmark evidence",
"status": "active"
}
]
}

24
packages/node/meshnet_node/activation_compression.py
View File

@@ -99,7 +99,12 @@ def compress_activation(body: bytes, policy: CompressionPolicy) -> CompressionRe
return CompressionResult(candidate, "zstd", len(body), len(candidate), time.monotonic() - started, "compressed")
def decompress_activation(body: bytes, encoding: str | None) -> CompressionResult:
def decompress_activation(
body: bytes,
encoding: str | None,
*,
max_output_bytes: int | None = None,
) -> CompressionResult:
"""Decode a modern zstd body or preserve a legacy raw body with metrics."""
started = time.monotonic()
if not encoding:
@@ -110,8 +115,23 @@ def decompress_activation(body: bytes, encoding: str | None) -> CompressionResul
import zstandard as zstd
except ImportError as exc:
raise ValueError("zstd support is unavailable") from exc
if max_output_bytes is not None and max_output_bytes < 0:
raise ValueError("max_output_bytes must be non-negative")
try:
raw = zstd.ZstdDecompressor().decompress(body)
if max_output_bytes is None:
raw = zstd.ZstdDecompressor().decompress(body)
else:
# Cap both decoder window allocation and bytes read. zstandard's
# max_window_size unit is KiB.
max_window_kib = max(1024, (max_output_bytes + 1023) // 1024)
decompressor = zstd.ZstdDecompressor(max_window_size=max_window_kib)
# `decompress(max_output_size=...)` may trust a frame's advertised
# content size. A bounded stream read enforces the limit regardless
# of frame metadata and detects trailing expansion with one byte.
with decompressor.stream_reader(body) as reader:
raw = reader.read(max_output_bytes + 1)
if len(raw) > max_output_bytes:
raise ValueError("zstd activation body exceeds its output limit")
except zstd.ZstdError as exc:
raise ValueError("invalid zstd activation body") from exc
return CompressionResult(raw, "zstd", len(body), len(raw), time.monotonic() - started, "decompressed")

33
packages/node/meshnet_node/capability.py
View File

@@ -20,6 +20,8 @@ import time
from dataclasses import dataclass, field
from typing import Any, Mapping
from .runtime_recipe import CompatibilityFingerprint, ShardIdentity
# Layout of the serialized report. Bump when the JSON shape changes.
CAPABILITY_SCHEMA_VERSION = 1
@@ -330,7 +332,16 @@ def _as_mapping(data: Any, field_name: str) -> Mapping[str, Any]:
@dataclass(frozen=True)
class CapabilityReport:
"""One node's validated (or failed) model/shard/recipe/backend combination."""
"""One node's validated (or failed) model/shard/recipe/backend combination.
`identity` is the exact DGR-003 artifact/runtime-recipe block: the separated
numerical axes and the compatibility fingerprint derived from them. It is
optional and additive — a node that predates DGR-003 presents none, and the
tracker falls back to the coarse label comparison it has always done
(ADR-0023's compat rollout). A node that *does* present one is held to it:
the tracker re-derives the fingerprint and refuses a report whose claim does
not match its own derivation.
"""
model: ModelIdentity
shard: ShardRange
@@ -341,6 +352,7 @@ class CapabilityReport:
duration_ms: int
diagnostics: tuple[str, ...] = ()
schema_version: int = CAPABILITY_SCHEMA_VERSION
identity: ShardIdentity | None = None
def __post_init__(self) -> None:
if self.status not in VALID_STATUSES:
@@ -360,6 +372,11 @@ class CapabilityReport:
def passed(self) -> bool:
return self.status == STATUS_PASSED
@property
def fingerprint(self) -> CompatibilityFingerprint | None:
"""The exact compatibility fingerprint, when this node declares one."""
return None if self.identity is None else self.identity.fingerprint
def identity_key(self) -> tuple[str, int, int, str, str, str, str]:
"""The tuple a consumer must match to reuse this proof.
@@ -380,7 +397,7 @@ class CapabilityReport:
return max(0.0, (time.time() if now is None else now) - self.validated_at)
def to_dict(self) -> dict:
return {
doc = {
"schema_version": self.schema_version,
"model": self.model.to_dict(),
"shard": self.shard.to_dict(),
@@ -391,6 +408,9 @@ class CapabilityReport:
"duration_ms": self.duration_ms,
"diagnostics": list(self.diagnostics),
}
if self.identity is not None:
doc["identity"] = self.identity.to_dict()
return doc
def to_json(self, indent: int | None = None) -> str:
return json.dumps(self.to_dict(), indent=indent, sort_keys=True)
@@ -417,6 +437,7 @@ class CapabilityReport:
):
raise CapabilityReportError("'validated_at' must be a Unix timestamp")
raw_identity = doc.get("identity")
return cls(
schema_version=schema_version,
model=ModelIdentity.from_dict(doc.get("model")),
@@ -427,6 +448,9 @@ class CapabilityReport:
validated_at=float(validated_at),
duration_ms=_require_int(doc.get("duration_ms"), "duration_ms", 0),
diagnostics=sanitize_diagnostics(doc.get("diagnostics")),
identity=(
None if raw_identity is None else ShardIdentity.from_dict(raw_identity)
),
)
@classmethod
@@ -461,12 +485,14 @@ def build_capability_report(
diagnostics: Any = None,
validated_at: float | None = None,
environ: Mapping[str, str] | None = None,
identity: ShardIdentity | None = None,
) -> CapabilityReport:
"""Assemble a report from flat validation results.
`model_config` may be the loaded config mapping (hashed into a fingerprint)
or an already-computed ``sha256:…`` string. `validated_at` defaults to now,
so callers that need determinism pass it explicitly.
so callers that need determinism pass it explicitly. `identity` is the exact
DGR-003 artifact/recipe block, when the backend can state one.
"""
return CapabilityReport(
model=ModelIdentity(
@@ -491,4 +517,5 @@ def build_capability_report(
validated_at=time.time() if validated_at is None else validated_at,
duration_ms=duration_ms,
diagnostics=sanitize_diagnostics(diagnostics, environ),
identity=identity,
)

123
packages/node/meshnet_node/glm_alpha/__init__.py Normal file
View File

@@ -0,0 +1,123 @@
"""The locked GLM-5.2 Max alpha target: identity, resource plan, and acceptance contract.
Three files, three jobs:
- :mod:`~meshnet_node.glm_alpha.manifest` — *is this the exact artifact?* Pinned
repository revisions, six shard digests, and the architecture-critical config
snapshot they must agree with.
- :mod:`~meshnet_node.glm_alpha.planner` — *can this route hold it?* Deterministic
memory, KV, and seam arithmetic over the exact artifact bytes, counting unified
memory once.
- :mod:`~meshnet_node.glm_alpha.contract` — *what would have counted as success?*
The acceptance thresholds, locked before the target runs and digest-bound so a
later change cannot be silent.
Nothing here downloads, loads, or executes a model. This package is the contract
DGR-018, DGR-019, and DGR-020 are judged against.
"""
from __future__ import annotations
from .contract import (
ALPHA_CONTRACT_ID,
ALPHA_CONTRACT_SCHEMA_VERSION,
VERDICT_ALPHA,
VERDICT_STOP,
AlphaContract,
AlphaContractError,
compute_contract_digest,
load_alpha_contract,
parse_alpha_contract,
require_contract_target,
seal_contract,
)
from .manifest import (
ALPHA_QUANTIZATION,
ALPHA_SHARD_COUNT,
ArchitectureSnapshot,
GlmTargetError,
Shard,
TargetManifest,
canonical_sha256,
load_architecture_snapshot,
load_target_manifest,
parse_architecture_snapshot,
parse_target_manifest,
require_pinned_target,
)
from .planner import (
AGGREGATE_HARD_FIT_FLOOR_GIB,
ALPHA_CONTEXT_TOKENS,
ALPHA_KV_DTYPE,
MIN_LINK_RATE_GBPS,
PLACEMENT_IMBALANCE_FACTOR,
RECOMMENDED_LINK_RATE_GBPS,
RESERVE_FLOOR_GIB,
RESERVE_FRACTION,
NodeMemory,
ResourcePlanError,
RouteFit,
SeamPlan,
TopologyPlan,
kv_bytes,
plan_all_tiers,
plan_route,
plan_seams,
plan_topology,
)
def load_locked_target() -> tuple[AlphaContract, TargetManifest, ArchitectureSnapshot]:
"""Load and cross-bind the packaged alpha contract, manifest, and snapshot."""
contract = load_alpha_contract()
manifest = load_target_manifest()
snapshot = load_architecture_snapshot()
require_contract_target(contract, manifest, snapshot)
return contract, manifest, snapshot
__all__ = [
"AGGREGATE_HARD_FIT_FLOOR_GIB",
"ALPHA_CONTEXT_TOKENS",
"ALPHA_CONTRACT_ID",
"ALPHA_CONTRACT_SCHEMA_VERSION",
"ALPHA_KV_DTYPE",
"ALPHA_QUANTIZATION",
"ALPHA_SHARD_COUNT",
"MIN_LINK_RATE_GBPS",
"PLACEMENT_IMBALANCE_FACTOR",
"RECOMMENDED_LINK_RATE_GBPS",
"RESERVE_FLOOR_GIB",
"RESERVE_FRACTION",
"VERDICT_ALPHA",
"VERDICT_STOP",
"AlphaContract",
"AlphaContractError",
"ArchitectureSnapshot",
"GlmTargetError",
"NodeMemory",
"ResourcePlanError",
"RouteFit",
"SeamPlan",
"Shard",
"TargetManifest",
"TopologyPlan",
"canonical_sha256",
"compute_contract_digest",
"kv_bytes",
"load_alpha_contract",
"load_architecture_snapshot",
"load_locked_target",
"load_target_manifest",
"parse_alpha_contract",
"parse_architecture_snapshot",
"parse_target_manifest",
"plan_all_tiers",
"plan_route",
"plan_seams",
"plan_topology",
"require_contract_target",
"require_pinned_target",
"seal_contract",
]

359
packages/node/meshnet_node/glm_alpha/contract.py Normal file
View File

@@ -0,0 +1,359 @@
"""The immutable GLM-5.2 Max alpha acceptance contract.
The contract exists to answer one question that cannot be answered honestly after
the fact: *what would have counted as success?*
Its thresholds are locked before the target ever runs (DGR-017), and DGR-020 reads
them back to publish an ``alpha`` or ``stop`` verdict. The whole point is that the
gap between those two moments is where a threshold quietly becomes "0.1 tokens/sec
was always the goal". So the document carries ``contract_sha256`` over its canonical content, while the
approved v1 digest is pinned independently in code. :func:`load_alpha_contract`
recomputes the self-digest and then requires that trusted pre-execution digest.
Changing a threshold and re-sealing under the same identity is rejected; an
amendment requires a new supported contract identity under human review.
The parsed contract recursively freezes nested mappings and sequences. Thresholds
therefore cannot change between verification and use, and :meth:`AlphaContract.to_dict`
returns an isolated mutable copy for diagnostics and tests.
"""
from __future__ import annotations
import json
import re
from dataclasses import dataclass
from importlib.resources import files
from pathlib import Path
from types import MappingProxyType
from typing import Any, Mapping
from .manifest import (
ALPHA_QUANTIZATION,
ALPHA_SHARD_COUNT,
ArchitectureSnapshot,
GlmTargetError,
TargetManifest,
canonical_sha256,
)
ALPHA_CONTRACT_SCHEMA_VERSION = 1
ALPHA_CONTRACT_VERSION = 1
ALPHA_CONTRACT_ID = "glm-5.2-max-alpha/v1"
ALPHA_CONTRACT_V1_SHA256 = "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9"
_CONTRACT_RESOURCE = "alpha-contract.json"
DIGEST_FIELD = "contract_sha256"
VERDICT_ALPHA = "alpha"
VERDICT_STOP = "stop"
# Every section the roadmap's acceptance matrix (section 5) locks. A contract that
# omits one is not a weaker contract, it is an unreviewable one.
REQUIRED_SECTIONS: tuple[str, ...] = (
"identity_and_fit",
"semantic_correctness",
"target_run",
"performance",
"reliability",
"storage",
)
class AlphaContractError(GlmTargetError):
"""Raised when the alpha contract is missing, malformed, or has been mutated."""
def contract_signing_payload(document: Mapping[str, Any]) -> dict:
"""The contract content the digest covers: everything except the digest itself."""
unsigned = dict(document)
unsigned.pop(DIGEST_FIELD, None)
return unsigned
def compute_contract_digest(document: Mapping[str, Any]) -> str:
"""SHA-256 over the canonical contract content."""
return canonical_sha256(contract_signing_payload(_thaw_json(document)))
def _freeze_json(value: Any) -> Any:
if isinstance(value, Mapping):
return MappingProxyType({str(key): _freeze_json(item) for key, item in value.items()})
if isinstance(value, list):
return tuple(_freeze_json(item) for item in value)
return value
def _thaw_json(value: Any) -> Any:
if isinstance(value, Mapping):
return {str(key): _thaw_json(item) for key, item in value.items()}
if isinstance(value, tuple):
return [_thaw_json(item) for item in value]
return value
@dataclass(frozen=True)
class AlphaContract:
"""A locked, digest-bound alpha acceptance contract."""
schema_version: int
contract_version: int
contract_id: str
locked_at: str
locked_by: str
target: Mapping[str, Any]
sections: Mapping[str, Mapping[str, Any]]
verdicts: tuple[str, ...]
amendment_policy: str
digest: str
raw: Mapping[str, Any]
source: str = "<memory>"
def section(self, name: str) -> Mapping[str, Any]:
if name not in self.sections:
raise AlphaContractError(f"contract section {name!r} is missing from {self.source}")
return self.sections[name]
def threshold(self, section: str, key: str) -> Any:
block = self.section(section)
if key not in block:
raise AlphaContractError(
f"threshold {section}.{key} is not locked in {self.source}; an unlocked "
"threshold cannot be used to judge a result"
)
return block[key]
def to_dict(self) -> dict:
return _thaw_json(self.raw)
def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
"""Validate a contract document and verify it has not been mutated since locking."""
if not isinstance(data, Mapping):
raise AlphaContractError(f"contract root in {source} must be a JSON object")
schema_version = data.get("schema_version")
if (
not isinstance(schema_version, int)
or isinstance(schema_version, bool)
or schema_version != ALPHA_CONTRACT_SCHEMA_VERSION
):
raise AlphaContractError(
f"{source} declares alpha-contract schema version {schema_version!r}, but this "
f"node reads version {ALPHA_CONTRACT_SCHEMA_VERSION}"
)
contract_version = data.get("contract_version")
if (
not isinstance(contract_version, int)
or isinstance(contract_version, bool)
or contract_version != ALPHA_CONTRACT_VERSION
):
raise AlphaContractError(
f"{source} declares contract version {contract_version!r}, but this node "
f"reads version {ALPHA_CONTRACT_VERSION}"
)
contract_id = data.get("contract_id")
if contract_id != ALPHA_CONTRACT_ID:
raise AlphaContractError(
f"{source} declares contract_id {contract_id!r}, but this node is locked "
f"to {ALPHA_CONTRACT_ID!r}"
)
for field in ("locked_at", "locked_by"):
value = data.get(field)
if not isinstance(value, str) or not value.strip():
raise AlphaContractError(f"{source} must carry a non-empty {field}")
declared = data.get(DIGEST_FIELD)
if not isinstance(declared, str) or not declared:
raise AlphaContractError(
f"{source} carries no {DIGEST_FIELD}; an unsealed contract cannot prove it "
"predates the results it judges"
)
computed = compute_contract_digest(data)
if computed != declared:
raise AlphaContractError(
f"{source} has been modified since it was locked: its content hashes to "
f"{computed}, but it declares {declared}. Alpha thresholds are locked before "
"target execution and may not be weakened afterwards. To change them, open a "
"new contract_id under human review; do not edit this one."
)
if not data.get("locked_before_target_execution"):
raise AlphaContractError(
f"{source} does not assert locked_before_target_execution; a contract written "
"after the results are known is not a contract"
)
missing = [name for name in REQUIRED_SECTIONS if not isinstance(data.get(name), Mapping)]
if missing:
raise AlphaContractError(
f"{source} is missing locked acceptance section(s) {missing}"
)
verdicts = data.get("verdicts")
if not isinstance(verdicts, list) or sorted(verdicts) != sorted([VERDICT_ALPHA, VERDICT_STOP]):
raise AlphaContractError(
f"{source} must offer exactly the verdicts {[VERDICT_ALPHA, VERDICT_STOP]}; a "
"third outcome is how 'it loaded' becomes a pass"
)
target = data.get("target")
if not isinstance(target, Mapping):
raise AlphaContractError(f"{source} is missing its 'target' block")
for field in (
"source_repo_id",
"source_revision",
"gguf_repo_id",
"gguf_revision",
"quantization",
"target_manifest_sha256",
"architecture_snapshot_sha256",
"reasoning_effort",
):
if not target.get(field):
raise AlphaContractError(f"{source} target block is missing {field!r}")
for field in ("source_revision", "gguf_revision"):
if not re.fullmatch(r"[0-9a-f]{40}", str(target[field])):
raise AlphaContractError(f"{source} target.{field} is not a full commit revision")
for field in ("target_manifest_sha256", "architecture_snapshot_sha256"):
if not re.fullmatch(r"[0-9a-f]{64}", str(target[field])):
raise AlphaContractError(f"{source} target.{field} is not a SHA-256 digest")
if target["quantization"] != ALPHA_QUANTIZATION:
raise AlphaContractError(
f"{source} targets {target['quantization']!r}, not locked alpha quantization "
f"{ALPHA_QUANTIZATION!r}"
)
if target["reasoning_effort"] != "max":
raise AlphaContractError(f"{source} must lock reasoning_effort='max'")
shard_count = target.get("shard_count")
if (
not isinstance(shard_count, int)
or isinstance(shard_count, bool)
or shard_count != ALPHA_SHARD_COUNT
):
raise AlphaContractError(
f"{source} target.shard_count must be exactly {ALPHA_SHARD_COUNT}"
)
total_bytes = target.get("total_bytes")
if not isinstance(total_bytes, int) or isinstance(total_bytes, bool) or total_bytes <= 0:
raise AlphaContractError(f"{source} target.total_bytes must be a positive integer")
amendment_policy = data.get("amendment_policy")
if not isinstance(amendment_policy, str) or not amendment_policy.strip():
raise AlphaContractError(f"{source} must state its amendment policy")
if declared != ALPHA_CONTRACT_V1_SHA256:
raise AlphaContractError(
f"{source} is a re-sealed mutation of {ALPHA_CONTRACT_ID}: digest "
f"{declared} does not match the trusted pre-execution digest "
f"{ALPHA_CONTRACT_V1_SHA256}. An amendment requires a new supported "
"contract identity under human review."
)
frozen = _freeze_json(data)
return AlphaContract(
schema_version=schema_version,
contract_version=contract_version,
contract_id=contract_id,
locked_at=str(data["locked_at"]),
locked_by=str(data["locked_by"]),
target=frozen["target"],
sections=MappingProxyType({name: frozen[name] for name in REQUIRED_SECTIONS}),
verdicts=tuple(verdicts),
amendment_policy=amendment_policy,
digest=declared,
raw=frozen,
source=source,
)
def require_contract_target(
contract: AlphaContract,
manifest: TargetManifest,
snapshot: ArchitectureSnapshot,
) -> None:
"""Bind the sealed contract to the exact manifest and architecture snapshot.
Repository revisions alone do not bind shard LFS objects or derived architecture
semantics. Call this before planning, admission, download, or execution.
"""
expected = contract.target
actual = {
"source_repo_id": manifest.source_repo_id,
"source_revision": manifest.source_revision,
"gguf_repo_id": manifest.gguf_repo_id,
"gguf_revision": manifest.gguf_revision,
"quantization": manifest.quantization,
"shard_count": len(manifest.shards),
"total_bytes": manifest.total_bytes,
"target_manifest_sha256": manifest.digest,
"architecture_snapshot_sha256": snapshot.digest,
}
if snapshot.source_repo_id != manifest.source_repo_id:
raise AlphaContractError(
"architecture snapshot repository does not match the target manifest repository"
)
if snapshot.source_revision != manifest.source_revision:
raise AlphaContractError(
"architecture snapshot revision does not match the target manifest revision"
)
mismatches = {
key: (expected.get(key), value)
for key, value in actual.items()
if expected.get(key) != value
}
if mismatches:
details = ", ".join(
f"{key}: locked={locked!r}, actual={value!r}"
for key, (locked, value) in sorted(mismatches.items())
)
raise AlphaContractError(f"target documents do not match the sealed contract: {details}")
def load_alpha_contract(path: Path | None = None) -> AlphaContract:
"""Load the packaged alpha contract, or one at ``path``."""
if path is not None:
source = str(path)
try:
raw = path.read_text(encoding="utf-8")
except OSError as exc:
raise AlphaContractError(f"cannot read {source}: {exc.strerror or exc}") from exc
else:
source = f"packaged {_CONTRACT_RESOURCE}"
try:
raw = (
files("meshnet_node.glm_alpha")
.joinpath("data", _CONTRACT_RESOURCE)
.read_text(encoding="utf-8")
)
except (OSError, FileNotFoundError, ModuleNotFoundError) as exc:
raise AlphaContractError(
f"{source} is missing from this node installation ({type(exc).__name__})"
) from exc
try:
data = json.loads(raw)
except json.JSONDecodeError as exc:
raise AlphaContractError(
f"{source} is not valid JSON: {exc.msg} at line {exc.lineno} column {exc.colno}"
) from exc
return parse_alpha_contract(data, source=source)
def seal_contract(document: Mapping[str, Any]) -> dict:
"""Return the document with a freshly computed digest.
This is the only supported way to produce a contract file. It is deliberately
*not* called at load time: sealing on load would turn every mutation into a
valid contract, which is precisely the property the digest exists to deny.
"""
sealed = dict(document)
sealed[DIGEST_FIELD] = compute_contract_digest(document)
return sealed

122
packages/node/meshnet_node/glm_alpha/data/alpha-contract.json Normal file
View File

@@ -0,0 +1,122 @@
{
"schema_version": 1,
"contract_version": 1,
"contract_id": "glm-5.2-max-alpha/v1",
"locked_at": "2026-07-13",
"locked_by": "DGR-017",
"locked_before_target_execution": true,
"target": {
"source_repo_id": "zai-org/GLM-5.2",
"source_revision": "b4734de4facf877f85769a911abafc5283eab3d9",
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
"quantization": "UD-IQ1_S",
"shard_count": 6,
"total_bytes": 216715360960,
"target_manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962",
"architecture_snapshot_sha256": "253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1",
"reasoning_effort": "max"
},
"identity_and_fit": {
"require_exact_revisions": true,
"require_all_shard_sha256": true,
"require_per_node_owned_tensor_report": true,
"require_owned_tensor_union_equals_inventory": true,
"max_unintended_tensor_overlap": 0,
"require_no_single_node_can_admit_complete_recipe": true,
"min_node_reserve_fraction": 0.2,
"min_node_reserve_gib": 8.0,
"require_measured_peak_scratch_inside_reserve": true,
"forbid_swap": true,
"forbid_overcommit": true,
"forbid_mmap_only_fit_claim": true,
"forbid_double_counted_unified_memory": true,
"aggregate_hard_fit_floor_gib": 224.0,
"aggregate_floor_class": "experimental_hard_fit_floor",
"recommended_topologies": [
"5x64GiB",
"3x96GiB",
"3x128GiB"
],
"arithmetic_minimum_requires_measured_placement_evidence": true
},
"semantic_correctness": {
"require_active_moe_routing": true,
"require_active_shared_expert": true,
"require_active_dsa_lightning_indexer": true,
"require_active_sparse_attention": true,
"require_active_indexshare_full_and_shared": true,
"dense_attention_fallback_satisfies_alpha": false,
"require_rendered_reasoning_effort_marker": "<|system|>Reasoning Effort: Max",
"f32_seam_fixture_exact_match_tokens": 32,
"f32_seam_fixture_requires_exact_match": true,
"min_greedy_token_agreement": 0.9,
"min_mean_state_cosine_similarity": 0.999,
"forbid_nonfinite_tensors": true,
"require_fail_closed_on_fingerprint_mismatch": true
},
"target_run": {
"context_tokens": 16384,
"kv_dtype": "Q8_0",
"concurrency": 1,
"prompt_lane_tokens": 4096,
"min_output_tokens": 512,
"min_output_tokens_with_natural_eos": 128,
"require_same_switch_wired_network": true,
"min_link_rate_gbps": 2.5,
"recommended_link_rate_gbps": 10.0,
"require_sentinels": [
"coding",
"structured_tool_call_json",
"multi_step_reasoning"
],
"require_openai_compatible_response_fields": [
"model",
"finish_reason",
"usage"
]
},
"performance": {
"min_median_decode_tokens_per_second": 0.5,
"max_ttft_seconds_at_4096_prompt": 600,
"max_unexplained_stall_seconds": 60,
"warmups": 1,
"require_per_stage_telemetry": [
"compute",
"queue",
"kv",
"seam_bytes",
"seam_latency",
"rss",
"vram",
"backend_timing"
],
"quality_pass_with_speed_fail_verdict": "stop",
"forbid_generalising_results_to_other_hardware": true
},
"reliability": {
"consecutive_clean_cold_starts": 2,
"require_cancellation_releases_buffers_and_kv": true,
"require_worker_loss_aborts_route": true,
"retry_policy": "from_token_zero_on_new_compatible_route",
"forbid_silent_kv_migration": true,
"require_reject_stale_epoch": true,
"require_reject_duplicate_step_id": true,
"synthetic_workers_satisfy_alpha": false,
"layer_reduced_fixtures_satisfy_alpha": false
},
"storage": {
"mounted_storage_only": true,
"forbidden_path_prefixes": [
"/home"
],
"forbid_secrets_in_logs": true,
"forbid_unrestricted_prompt_payloads_in_logs": true
},
"verdicts": [
"alpha",
"stop"
],
"amendment_policy": "Thresholds are locked before target execution. They may not be weakened, moved, or reinterpreted after results are known. A change requires a new contract_id and contract_version under human review, and the superseded contract is retained.",
"contract_sha256": "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9"
}

91
packages/node/meshnet_node/glm_alpha/data/architecture-snapshot.json Normal file
View File

@@ -0,0 +1,91 @@
{
"schema_version": 1,
"observed_at": "2026-07-13",
"source_repo_id": "zai-org/GLM-5.2",
"source_revision": "b4734de4facf877f85769a911abafc5283eab3d9",
"source_files": [
{
"path": "config.json",
"size_bytes": 3732,
"sha256": "185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a",
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/config.json"
},
{
"path": "chat_template.jinja",
"size_bytes": 5076,
"sha256": "172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679",
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/chat_template.jinja"
},
{
"path": "generation_config.json",
"size_bytes": 194,
"sha256": "ac76b43d8683d3b930126870fc8be73d8679308fe752fa1f381096d8354f6a55",
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/generation_config.json"
},
{
"path": "tokenizer_config.json",
"size_bytes": 761,
"sha256": "98b1271574f41abf89427ae2dda030d94dc9478f0edc5a8bd240db213c6fd5fc",
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/tokenizer_config.json"
}
],
"architecture": {
"architectures": [
"GlmMoeDsaForCausalLM"
],
"model_type": "glm_moe_dsa",
"num_hidden_layers": 78,
"num_nextn_predict_layers": 1,
"total_artifact_layers": 79,
"first_k_dense_replace": 3,
"dense_layers": 3,
"sparse_moe_layers": 75,
"hidden_size": 6144,
"intermediate_size": 12288,
"moe_intermediate_size": 2048,
"n_routed_experts": 256,
"num_experts_per_tok": 8,
"n_shared_experts": 1,
"scoring_func": "sigmoid",
"topk_method": "noaux_tc",
"norm_topk_prob": true,
"routed_scaling_factor": 2.5,
"num_attention_heads": 64,
"head_dim": 192,
"qk_nope_head_dim": 192,
"qk_rope_head_dim": 64,
"qk_head_dim": 256,
"v_head_dim": 256,
"kv_lora_rank": 512,
"q_lora_rank": 2048,
"mla_cached_values_per_token_per_layer": 576,
"index_topk": 2048,
"index_head_dim": 128,
"index_n_heads": 32,
"index_topk_freq": 4,
"index_skip_topk_offset": 3,
"index_share_for_mtp_iteration": true,
"indexer_full_layers": 21,
"indexer_shared_layers": 57,
"indexer_types_sha256": "ec3b4927af83cf02baf37fb10454c40176ec8bf501ae89334b27a9df5fa17025",
"max_position_embeddings": 1048576,
"vocab_size": 154880,
"rope_theta": 8000000,
"dtype": "bfloat16",
"tie_word_embeddings": false
},
"reasoning_effort": {
"alpha_mode": "max",
"rendered_marker": "<|system|>Reasoning Effort: Max",
"template_rule": "effective_reasoning_effort = 'high' if reasoning_effort == 'high' else 'max'",
"default_is_max": true,
"suppressed_when": "enable_thinking is defined and false",
"note": "The template recognises exactly one non-max level ('high'); every other value, including an absent reasoning_effort, renders Max. Alpha therefore asserts the rendered 'Reasoning Effort: Max' marker, not merely the presence of a request field."
},
"notes": [
"indexer_types has 78 entries: layers 0-2 are 'full', then a repeating [shared, shared, shared, full] pattern, giving 21 Full producer layers and 57 Shared consumer layers.",
"MLA caches kv_lora_rank (512) + qk_rope_head_dim (64) = 576 values per token per backbone layer.",
"num_nextn_predict_layers=1 is the NextN/MTP layer present in the artifact. Alpha does not run MTP; the tensors are loaded or explicitly excluded by a certified recipe and must never be silently reinterpreted as a 79th backbone layer.",
"Values are derived from the pinned config.json. The runtime must re-derive them from the artifact and fail closed on contradictory metadata; marketing names are not compatibility identity."
]
}

90
packages/node/meshnet_node/glm_alpha/data/target-manifest.json Normal file
View File

@@ -0,0 +1,90 @@
{
"schema_version": 1,
"manifest_version": 1,
"observed_at": "2026-07-13",
"observed_by": "DGR-017",
"alpha_quantization": "UD-IQ1_S",
"source_model": {
"repo_id": "zai-org/GLM-5.2",
"revision": "b4734de4facf877f85769a911abafc5283eab3d9",
"last_modified": "2026-07-02T08:08:14.000Z",
"weight_license": "mit",
"code_documentation_license": "apache-2.0",
"url": "https://huggingface.co/zai-org/GLM-5.2",
"revision_url": "https://huggingface.co/zai-org/GLM-5.2/tree/b4734de4facf877f85769a911abafc5283eab3d9",
"api_url": "https://huggingface.co/api/models/zai-org/GLM-5.2"
},
"gguf_artifact": {
"repo_id": "unsloth/GLM-5.2-GGUF",
"revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
"last_modified": "2026-06-23T15:18:23.000Z",
"license": "mit",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF",
"revision_url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/tree/abc55e72527792c6e77069c99b4cb7de16fa9f23",
"api_url": "https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF",
"quantization": "UD-IQ1_S",
"shard_count": 6,
"total_bytes": 216715360960,
"total_gib": 201.832,
"total_gb": 216.715,
"shards": [
{
"index": 1,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
"size_bytes": 9423744,
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf"
},
{
"index": 2,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
"size_bytes": 49208128256,
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf"
},
{
"index": 3,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
"size_bytes": 49684417024,
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf"
},
{
"index": 4,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
"size_bytes": 49396052864,
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf"
},
{
"index": 5,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
"size_bytes": 49246275936,
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf"
},
{
"index": 6,
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
"size_bytes": 19171063136,
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf"
}
]
},
"diagnostic_fallback": {
"quantization": "UD-IQ1_M",
"shard_count": 6,
"total_bytes": 228492966624,
"total_gib": 212.801,
"total_gb": 228.493,
"policy": "First diagnostic fallback only if UD-IQ1_S exposes a runtime or quality defect. It does not satisfy the alpha 'lowest published quantization' target unless human review changes the target contract."
},
"storage": {
"mounted_storage_only": true,
"forbidden_path_prefixes": [
"/home"
],
"note": "Model artifacts resolve through the machine-specific .env.<hostname> mounted-drive configuration. A path under /home fails admission closed."
}
}

490
packages/node/meshnet_node/glm_alpha/manifest.py Normal file
View File

@@ -0,0 +1,490 @@
"""The pinned GLM-5.2 Max target manifest and architecture snapshot.
This module is the *identity* half of the alpha target contract. It answers one
question: is the artifact on this disk the exact artifact alpha was locked
against?
Identity is pinned by repository revision **and** by every shard's LFS SHA-256.
A revision alone is not enough — a repository can be force-pushed, and a tag can
be moved — and a size alone is not enough, because two different quantizations of
the same model land within a rounding error of each other. The manifest therefore
carries both, plus the aggregate byte total, and cross-checks the aggregate
against the sum of the shards. A manifest whose declared total disagrees with its
own shards is rejected rather than trusted, because that is the exact shape a
hand-edited "it fits now" manifest takes.
Nothing here downloads a weight payload. Sizes and hashes come from the Hugging
Face metadata API (see ``scripts/refresh_glm_target_manifest.py``); verification
against a local file is DGR-018's job, using the digests locked here.
"""
from __future__ import annotations
import hashlib
import json
import re
from dataclasses import dataclass
from importlib.resources import files
from pathlib import Path
from typing import Any, Mapping
TARGET_MANIFEST_SCHEMA_VERSION = 1
ARCHITECTURE_SNAPSHOT_SCHEMA_VERSION = 1
ALPHA_QUANTIZATION = "UD-IQ1_S"
ALPHA_SHARD_COUNT = 6
_SHA256_RE = re.compile(r"\A[0-9a-f]{64}\Z")
_MANIFEST_RESOURCE = "target-manifest.json"
_ARCHITECTURE_RESOURCE = "architecture-snapshot.json"
GIB = 1024**3
GB = 1000**3
class GlmTargetError(ValueError):
"""Raised when the target manifest or architecture snapshot is not the pinned target."""
def canonical_sha256(value: Any) -> str:
"""SHA-256 over canonical JSON — the repository's digest convention."""
payload = json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _require_mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise GlmTargetError(f"{what} must be a JSON object, got {type(value).__name__}")
return value
def _require_text(value: Any, what: str) -> str:
if not isinstance(value, str) or not value.strip():
raise GlmTargetError(f"{what} must be a non-empty string")
return value
def _require_int(value: Any, what: str) -> int:
if not isinstance(value, int) or isinstance(value, bool):
raise GlmTargetError(f"{what} must be an integer, got {type(value).__name__}")
return value
def _require_sha256(value: Any, what: str) -> str:
text = _require_text(value, what)
if not _SHA256_RE.match(text):
raise GlmTargetError(
f"{what} must be a lowercase 64-character hex SHA-256, got {text!r}"
)
return text
def _require_revision(value: Any, what: str) -> str:
text = _require_text(value, what)
if not re.fullmatch(r"[0-9a-f]{40}", text):
raise GlmTargetError(
f"{what} must be a full 40-character commit revision, got {text!r}; "
"a branch name or short SHA is not an immutable pin"
)
return text
@dataclass(frozen=True)
class Shard:
"""One GGUF shard of the alpha artifact."""
index: int
path: str
size_bytes: int
sha256: str
url: str
def to_dict(self) -> dict:
return {
"index": self.index,
"path": self.path,
"size_bytes": self.size_bytes,
"sha256": self.sha256,
"url": self.url,
}
@dataclass(frozen=True)
class TargetManifest:
"""The pinned, self-consistent GLM-5.2 ``UD-IQ1_S`` target."""
schema_version: int
manifest_version: int
observed_at: str
quantization: str
source_repo_id: str
source_revision: str
source_license: str
gguf_repo_id: str
gguf_revision: str
gguf_license: str
total_bytes: int
shards: tuple[Shard, ...]
raw: Mapping[str, Any]
source: str = "<memory>"
@property
def total_gib(self) -> float:
return self.total_bytes / GIB
@property
def total_gb(self) -> float:
return self.total_bytes / GB
def shard(self, index: int) -> Shard:
for shard in self.shards:
if shard.index == index:
return shard
raise GlmTargetError(f"shard {index} is not in {self.source}")
@property
def digest(self) -> str:
"""Stable identity of this manifest, for the DGR-003 runtime recipe."""
return canonical_sha256(self.raw)
def to_dict(self) -> dict:
return dict(self.raw)
def _parse_shards(raw: Any, expected_count: int, expected_total: int) -> tuple[Shard, ...]:
if not isinstance(raw, list):
raise GlmTargetError("gguf_artifact.shards must be a JSON array")
if len(raw) != expected_count:
raise GlmTargetError(
f"the alpha artifact has exactly {expected_count} shards, "
f"but the manifest lists {len(raw)}"
)
shards: list[Shard] = []
seen_index: set[int] = set()
seen_sha: set[str] = set()
for position, entry in enumerate(raw):
item = _require_mapping(entry, f"shards[{position}]")
index = _require_int(item.get("index"), f"shards[{position}].index")
if index in seen_index:
raise GlmTargetError(f"duplicate shard index {index} in the manifest")
seen_index.add(index)
size_bytes = _require_int(item.get("size_bytes"), f"shards[{index}].size_bytes")
if size_bytes <= 0:
raise GlmTargetError(f"shards[{index}].size_bytes must be positive")
sha256 = _require_sha256(item.get("sha256"), f"shards[{index}].sha256")
if sha256 in seen_sha:
raise GlmTargetError(
f"shard {index} repeats SHA-256 {sha256}; two distinct shards cannot "
"have the same content digest"
)
seen_sha.add(sha256)
shards.append(
Shard(
index=index,
path=_require_text(item.get("path"), f"shards[{index}].path"),
size_bytes=size_bytes,
sha256=sha256,
url=_require_text(item.get("url"), f"shards[{index}].url"),
)
)
expected_indices = set(range(1, expected_count + 1))
if seen_index != expected_indices:
missing = sorted(expected_indices - seen_index)
raise GlmTargetError(
f"the manifest is missing shard(s) {missing}; all {expected_count} "
"shards of the alpha artifact must be pinned"
)
summed = sum(shard.size_bytes for shard in shards)
if summed != expected_total:
raise GlmTargetError(
f"declared total_bytes {expected_total} does not equal the sum of the "
f"shard sizes {summed}; the manifest is not self-consistent"
)
return tuple(sorted(shards, key=lambda shard: shard.index))
def parse_target_manifest(data: Any, source: str = "<memory>") -> TargetManifest:
"""Validate an already-decoded target manifest, failing closed."""
doc = _require_mapping(data, f"manifest root in {source}")
schema_version = _require_int(doc.get("schema_version"), f"'schema_version' in {source}")
if schema_version != TARGET_MANIFEST_SCHEMA_VERSION:
raise GlmTargetError(
f"{source} declares target-manifest schema version {schema_version}, "
f"but this node reads version {TARGET_MANIFEST_SCHEMA_VERSION}"
)
quantization = _require_text(doc.get("alpha_quantization"), f"'alpha_quantization' in {source}")
if quantization != ALPHA_QUANTIZATION:
raise GlmTargetError(
f"{source} pins quantization {quantization!r}, but the locked alpha "
f"quantization is {ALPHA_QUANTIZATION!r}; a different quantization is a "
"different target and requires a human contract change"
)
source_model = _require_mapping(doc.get("source_model"), f"'source_model' in {source}")
gguf = _require_mapping(doc.get("gguf_artifact"), f"'gguf_artifact' in {source}")
gguf_quant = _require_text(gguf.get("quantization"), f"gguf_artifact.quantization in {source}")
if gguf_quant != quantization:
raise GlmTargetError(
f"{source} declares alpha_quantization {quantization!r} but the GGUF "
f"artifact block says {gguf_quant!r}"
)
shard_count = _require_int(gguf.get("shard_count"), f"gguf_artifact.shard_count in {source}")
if shard_count != ALPHA_SHARD_COUNT:
raise GlmTargetError(
f"{source} declares {shard_count} shards; the pinned alpha artifact has "
f"exactly {ALPHA_SHARD_COUNT}"
)
total_bytes = _require_int(gguf.get("total_bytes"), f"gguf_artifact.total_bytes in {source}")
shards = _parse_shards(gguf.get("shards"), shard_count, total_bytes)
return TargetManifest(
schema_version=schema_version,
manifest_version=_require_int(doc.get("manifest_version"), f"'manifest_version' in {source}"),
observed_at=_require_text(doc.get("observed_at"), f"'observed_at' in {source}"),
quantization=quantization,
source_repo_id=_require_text(source_model.get("repo_id"), "source_model.repo_id"),
source_revision=_require_revision(source_model.get("revision"), "source_model.revision"),
source_license=_require_text(source_model.get("weight_license"), "source_model.weight_license"),
gguf_repo_id=_require_text(gguf.get("repo_id"), "gguf_artifact.repo_id"),
gguf_revision=_require_revision(gguf.get("revision"), "gguf_artifact.revision"),
gguf_license=_require_text(gguf.get("license"), "gguf_artifact.license"),
total_bytes=total_bytes,
shards=shards,
raw=doc,
source=source,
)
@dataclass(frozen=True)
class ArchitectureSnapshot:
"""Architecture-critical metadata derived from the pinned ``config.json``."""
schema_version: int
source_repo_id: str
source_revision: str
architecture: Mapping[str, Any]
reasoning_effort: Mapping[str, Any]
source_files: Mapping[str, Mapping[str, Any]]
raw: Mapping[str, Any]
source: str = "<memory>"
def __getitem__(self, key: str) -> Any:
if key not in self.architecture:
raise GlmTargetError(f"architecture field {key!r} is missing from {self.source}")
return self.architecture[key]
@property
def digest(self) -> str:
return canonical_sha256(self.raw)
def file_sha256(self, path: str) -> str:
entry = self.source_files.get(path)
if entry is None:
raise GlmTargetError(f"{path!r} is not pinned in {self.source}")
return str(entry["sha256"])
def to_dict(self) -> dict:
return dict(self.raw)
# Fields the distributed runtime cannot plan or shard without. Absent or
# contradictory values fail closed rather than defaulting.
REQUIRED_ARCHITECTURE_FIELDS: tuple[str, ...] = (
"model_type",
"num_hidden_layers",
"num_nextn_predict_layers",
"total_artifact_layers",
"first_k_dense_replace",
"dense_layers",
"sparse_moe_layers",
"hidden_size",
"n_routed_experts",
"num_experts_per_tok",
"n_shared_experts",
"kv_lora_rank",
"qk_rope_head_dim",
"mla_cached_values_per_token_per_layer",
"index_topk",
"index_head_dim",
"indexer_full_layers",
"indexer_shared_layers",
"max_position_embeddings",
"vocab_size",
)
def parse_architecture_snapshot(data: Any, source: str = "<memory>") -> ArchitectureSnapshot:
"""Validate an architecture snapshot and its internal arithmetic."""
doc = _require_mapping(data, f"snapshot root in {source}")
schema_version = _require_int(doc.get("schema_version"), f"'schema_version' in {source}")
if schema_version != ARCHITECTURE_SNAPSHOT_SCHEMA_VERSION:
raise GlmTargetError(
f"{source} declares architecture-snapshot schema version {schema_version}, "
f"but this node reads version {ARCHITECTURE_SNAPSHOT_SCHEMA_VERSION}"
)
arch = _require_mapping(doc.get("architecture"), f"'architecture' in {source}")
missing = [field for field in REQUIRED_ARCHITECTURE_FIELDS if field not in arch]
if missing:
raise GlmTargetError(
f"{source} is missing architecture-critical field(s) {missing}; the "
"runtime cannot shard or plan an architecture it cannot fully describe"
)
layers = _require_int(arch["num_hidden_layers"], "num_hidden_layers")
nextn = _require_int(arch["num_nextn_predict_layers"], "num_nextn_predict_layers")
total_layers = _require_int(arch["total_artifact_layers"], "total_artifact_layers")
if total_layers != layers + nextn:
raise GlmTargetError(
f"total_artifact_layers {total_layers} != num_hidden_layers {layers} + "
f"num_nextn_predict_layers {nextn}; the NextN layer must be counted "
"explicitly, never folded into the backbone"
)
dense = _require_int(arch["dense_layers"], "dense_layers")
sparse = _require_int(arch["sparse_moe_layers"], "sparse_moe_layers")
if dense != _require_int(arch["first_k_dense_replace"], "first_k_dense_replace"):
raise GlmTargetError("dense_layers must equal first_k_dense_replace")
if dense + sparse != layers:
raise GlmTargetError(
f"dense_layers {dense} + sparse_moe_layers {sparse} != num_hidden_layers {layers}"
)
full = _require_int(arch["indexer_full_layers"], "indexer_full_layers")
shared = _require_int(arch["indexer_shared_layers"], "indexer_shared_layers")
if full + shared != layers:
raise GlmTargetError(
f"indexer_full_layers {full} + indexer_shared_layers {shared} != "
f"num_hidden_layers {layers}; every layer holds exactly one IndexShare role"
)
if full <= 0:
raise GlmTargetError(
"indexer_full_layers must be positive; a route with no Full producer layer "
"has no index for its Shared consumers to reuse"
)
mla = _require_int(
arch["mla_cached_values_per_token_per_layer"], "mla_cached_values_per_token_per_layer"
)
expected_mla = _require_int(arch["kv_lora_rank"], "kv_lora_rank") + _require_int(
arch["qk_rope_head_dim"], "qk_rope_head_dim"
)
if mla != expected_mla:
raise GlmTargetError(
f"mla_cached_values_per_token_per_layer {mla} != kv_lora_rank + "
f"qk_rope_head_dim ({expected_mla})"
)
reasoning = _require_mapping(doc.get("reasoning_effort"), f"'reasoning_effort' in {source}")
if reasoning.get("alpha_mode") != "max":
raise GlmTargetError(
f"{source} does not lock reasoning_effort=max; alpha is defined as the "
"Max reasoning mode of this exact checkpoint"
)
_require_text(reasoning.get("rendered_marker"), "reasoning_effort.rendered_marker")
files_raw = doc.get("source_files")
if not isinstance(files_raw, list) or not files_raw:
raise GlmTargetError(f"'source_files' in {source} must be a non-empty JSON array")
source_files: dict[str, Mapping[str, Any]] = {}
for position, entry in enumerate(files_raw):
item = _require_mapping(entry, f"source_files[{position}]")
path = _require_text(item.get("path"), f"source_files[{position}].path")
_require_sha256(item.get("sha256"), f"source_files[{path}].sha256")
source_files[path] = item
for required in ("config.json", "chat_template.jinja"):
if required not in source_files:
raise GlmTargetError(
f"{source} does not pin {required!r}; config and chat-template drift "
"silently changes runtime semantics"
)
return ArchitectureSnapshot(
schema_version=schema_version,
source_repo_id=_require_text(doc.get("source_repo_id"), "source_repo_id"),
source_revision=_require_revision(doc.get("source_revision"), "source_revision"),
architecture=arch,
reasoning_effort=reasoning,
source_files=source_files,
raw=doc,
source=source,
)
def _read_resource(resource: str, path: Path | None) -> tuple[str, str]:
if path is not None:
try:
return str(path), path.read_text(encoding="utf-8")
except OSError as exc:
raise GlmTargetError(f"cannot read {path}: {exc.strerror or exc}") from exc
source = f"packaged {resource}"
try:
raw = files("meshnet_node.glm_alpha").joinpath("data", resource).read_text(encoding="utf-8")
except (OSError, FileNotFoundError, ModuleNotFoundError) as exc:
raise GlmTargetError(
f"{source} is missing from this node installation ({type(exc).__name__})"
) from exc
return source, raw
def _load_json(resource: str, path: Path | None) -> tuple[str, Any]:
source, raw = _read_resource(resource, path)
try:
return source, json.loads(raw)
except json.JSONDecodeError as exc:
raise GlmTargetError(
f"{source} is not valid JSON: {exc.msg} at line {exc.lineno} column {exc.colno}"
) from exc
def load_target_manifest(path: Path | None = None) -> TargetManifest:
"""Load the packaged target manifest, or one at ``path``."""
source, data = _load_json(_MANIFEST_RESOURCE, path)
return parse_target_manifest(data, source=source)
def load_architecture_snapshot(path: Path | None = None) -> ArchitectureSnapshot:
"""Load the packaged architecture snapshot, or one at ``path``."""
source, data = _load_json(_ARCHITECTURE_RESOURCE, path)
return parse_architecture_snapshot(data, source=source)
def require_pinned_target(
manifest: TargetManifest,
snapshot: ArchitectureSnapshot,
*,
expected_source_revision: str,
expected_gguf_revision: str,
) -> None:
"""Reject any target whose revisions are not the ones alpha was locked against.
Callers pass the revisions from the locked alpha contract, so a swapped
manifest cannot quietly re-point the target at a different upstream commit.
"""
if manifest.source_revision != expected_source_revision:
raise GlmTargetError(
f"source revision {manifest.source_revision} does not match the locked "
f"alpha revision {expected_source_revision}"
)
if manifest.gguf_revision != expected_gguf_revision:
raise GlmTargetError(
f"GGUF revision {manifest.gguf_revision} does not match the locked alpha "
f"revision {expected_gguf_revision}"
)
if snapshot.source_revision != manifest.source_revision:
raise GlmTargetError(
f"the architecture snapshot was taken at {snapshot.source_revision} but the "
f"manifest pins {manifest.source_revision}; config metadata and weights must "
"come from one revision"
)

522
packages/node/meshnet_node/glm_alpha/planner.py Normal file
View File

@@ -0,0 +1,522 @@
"""Deterministic memory, KV, and network planner for the GLM-5.2 Max alpha route.
Everything here is arithmetic over the exact pinned artifact bytes and the exact
pinned architecture. There is no measurement, no probing, and no heuristic tuned
to a result — the planner is written *before* the target runs so that a later
story cannot discover a topology that "works" and then rationalise it.
Three ideas do the real work.
**Unified memory is one pool.** On an integrated-GPU machine the "VRAM" the driver
reports is carved out of the same physical DRAM the OS is already counting. Adding
them produces a node that appears to hold twice what it holds, and the failure mode
is not a clean admission rejection — it is an OOM or a swap-thrash halfway through
a 200 GiB load. :class:`NodeMemory` therefore refuses to be constructed from an
additive claim about one shared pool.
**The reserve is not optional headroom.** Weights plus KV are not the whole
resident cost: backend workspaces, quantization scratch, the graph plan, the
process, and the OS all live outside them, and the largest of those scale with the
backend rather than with the shard. Alpha reserves ``max(20% of physically usable
memory, 8 GiB)`` per node, and the *remainder* is the placement budget.
**Equal layer counts are not equal bytes.** Embeddings and the output head are
endpoint-only; three layers are dense and 75 are MoE; shared experts, indexer
tensors, and quant block alignment all skew the per-node share. Until DGR-018/019
report measured per-tensor placement, the planner carries an explicit
:data:`PLACEMENT_IMBALANCE_FACTOR` and reports the arithmetic minimum and the
recommended count as two separate numbers. The arithmetic minimum is a fit probe;
it is admissible only with exact measured placement evidence behind it.
"""
from __future__ import annotations
import math
from dataclasses import dataclass
from typing import Literal
from .manifest import GIB, ArchitectureSnapshot, TargetManifest
# Q8_0 stores 32 int8 quants plus one fp16 scale per block: 34 bytes / 32 values.
Q8_0_BYTES_PER_VALUE = 34 / 32
F16_BYTES_PER_VALUE = 2.0
KV_DTYPES: dict[str, float] = {
"Q8_0": Q8_0_BYTES_PER_VALUE,
"F16": F16_BYTES_PER_VALUE,
}
# The alpha KV configuration, locked by the roadmap.
ALPHA_KV_DTYPE = "Q8_0"
ALPHA_CONTEXT_TOKENS = 16384
ALPHA_CONCURRENCY = 1
# The reserve every node holds outside its weight-plus-KV placement budget.
RESERVE_FRACTION = 0.20
RESERVE_FLOOR_GIB = 8.0
# The aggregate runtime-accessible memory at which the artifact *just* fits.
# This is an experimental hard-fit floor, not an operational envelope: it has no
# room for a backend that allocates more scratch than another, and none for the
# imbalance below.
AGGREGATE_HARD_FIT_FLOOR_GIB = 224.0
# How much more than an equal share the worst-placed node is expected to hold.
# 1.10 is the roadmap's recommended-topology column expressed as arithmetic: it
# reproduces 10 / 6 / 5 / 3 / 3 nodes for the 32 / 48 / 64 / 96 / 128 GiB tiers.
# DGR-019 must replace it with measured per-tensor placement.
PLACEMENT_IMBALANCE_FACTOR = 1.10
# Alpha network floor. A link rate is a bandwidth claim, never a speed claim.
MIN_LINK_RATE_GBPS = 2.5
RECOMMENDED_LINK_RATE_GBPS = 10.0
BF16_BYTES = 2
DSA_SIDEBAND_INT32_BYTES = 4
IndexerLayout = Literal["optimized", "conservative"]
class ResourcePlanError(ValueError):
"""Raised when a node or route cannot be accounted for honestly."""
@dataclass(frozen=True)
class NodeMemory:
"""One node's physically usable memory, counted once.
``physical_usable_gib`` is what the node can actually place bytes into after
firmware and fixed carve-outs — not the marketing capacity, and not a sum of
two views of the same DRAM.
"""
name: str
physical_usable_gib: float
unified: bool
def __post_init__(self) -> None:
if not isinstance(self.name, str) or not self.name.strip():
raise ResourcePlanError("node name must be a non-empty physical-host identity")
if (
isinstance(self.physical_usable_gib, bool)
or not isinstance(self.physical_usable_gib, (int, float))
or not math.isfinite(self.physical_usable_gib)
or self.physical_usable_gib <= 0
):
raise ResourcePlanError(
f"node {self.name!r} must declare finite positive usable memory"
)
@classmethod
def from_host(
cls,
name: str,
*,
system_ram_gib: float,
gpu_memory_gib: float = 0.0,
unified: bool,
) -> "NodeMemory":
"""Build a node from a host's reported RAM and GPU memory.
On a unified machine the GPU memory *is* system RAM, so it is counted once
and never added. Passing a non-zero ``gpu_memory_gib`` alongside
``unified=True`` is the double-count this project has already decided is a
bug (RALPH-CONTEXT runtime decision 16), so it is rejected rather than
silently discarded: a caller who believes an integrated GPU adds memory has
a wrong model of the machine, and quietly ignoring the argument would let
that belief survive.
"""
if not isinstance(unified, bool):
raise ResourcePlanError(f"node {name!r} unified flag must be boolean")
for value, label, allow_zero in (
(system_ram_gib, "system RAM", False),
(gpu_memory_gib, "GPU memory", True),
):
if (
isinstance(value, bool)
or not isinstance(value, (int, float))
or not math.isfinite(value)
or value < 0
or (not allow_zero and value == 0)
):
qualifier = "finite non-negative" if allow_zero else "finite positive"
raise ResourcePlanError(f"node {name!r} must declare {qualifier} {label}")
if unified:
if gpu_memory_gib:
raise ResourcePlanError(
f"node {name!r} declares unified memory and {gpu_memory_gib} GiB of "
"separate GPU memory. Integrated-GPU memory is carved out of the same "
"physical DRAM as system RAM; adding them double-counts one pool. "
"Pass unified=True with system_ram_gib only."
)
usable = system_ram_gib
else:
usable = system_ram_gib + gpu_memory_gib
return cls(name=name, physical_usable_gib=usable, unified=unified)
@property
def reserve_gib(self) -> float:
"""``max(20% of physically usable memory, 8 GiB)``."""
return max(RESERVE_FRACTION * self.physical_usable_gib, RESERVE_FLOOR_GIB)
@property
def placement_budget_gib(self) -> float:
"""What remains for weights plus KV after the reserve."""
return self.physical_usable_gib - self.reserve_gib
def kv_bytes(
snapshot: ArchitectureSnapshot,
*,
context_tokens: int = ALPHA_CONTEXT_TOKENS,
concurrency: int = ALPHA_CONCURRENCY,
dtype: str = ALPHA_KV_DTYPE,
indexer_layout: IndexerLayout = "conservative",
include_indexer: bool = True,
) -> int:
"""Bytes of MLA (and DSA indexer) KV cache for the whole model.
``indexer_layout`` is the honest part. Correct DSA only needs indexer keys for
the Full producer layers, but the current experimental implementation may
allocate them across every backbone layer. Alpha budgets ``conservative``
(all 78) so that a route admitted by this planner cannot be surprised by the
implementation it actually gets.
"""
if (
not isinstance(context_tokens, int)
or isinstance(context_tokens, bool)
or context_tokens <= 0
or not isinstance(concurrency, int)
or isinstance(concurrency, bool)
or concurrency <= 0
):
raise ResourcePlanError("context_tokens and concurrency must be positive integers")
if dtype not in KV_DTYPES:
raise ResourcePlanError(
f"unsupported KV dtype {dtype!r}; alpha locks {ALPHA_KV_DTYPE} "
f"(known: {', '.join(sorted(KV_DTYPES))})"
)
bytes_per_value = KV_DTYPES[dtype]
layers = int(snapshot["num_hidden_layers"])
mla_values = int(snapshot["mla_cached_values_per_token_per_layer"])
total_values = mla_values * layers
if include_indexer:
if indexer_layout == "optimized":
indexer_layers = int(snapshot["indexer_full_layers"])
elif indexer_layout == "conservative":
indexer_layers = layers
else: # pragma: no cover - Literal keeps this unreachable from typed callers
raise ResourcePlanError(f"unknown indexer_layout {indexer_layout!r}")
total_values += int(snapshot["index_head_dim"]) * indexer_layers
return int(total_values * context_tokens * concurrency * bytes_per_value)
@dataclass(frozen=True)
class TopologyPlan:
"""The node count a homogeneous tier needs, and how it was reached."""
physical_usable_gib: float
reserve_gib: float
placement_budget_gib: float
weight_gib: float
kv_gib: float
total_placement_gib: float
arithmetic_minimum_nodes: int
recommended_nodes: int
imbalance_factor: float
@property
def is_arithmetic_minimum_topology(self) -> bool:
"""True when the recommendation offers no imbalance headroom at all."""
return self.recommended_nodes == self.arithmetic_minimum_nodes
def to_dict(self) -> dict:
return {
"physical_usable_gib": round(self.physical_usable_gib, 3),
"reserve_gib": round(self.reserve_gib, 3),
"placement_budget_gib": round(self.placement_budget_gib, 3),
"weight_gib": round(self.weight_gib, 3),
"kv_gib": round(self.kv_gib, 3),
"total_placement_gib": round(self.total_placement_gib, 3),
"arithmetic_minimum_nodes": self.arithmetic_minimum_nodes,
"recommended_nodes": self.recommended_nodes,
"imbalance_factor": self.imbalance_factor,
}
def plan_topology(
manifest: TargetManifest,
snapshot: ArchitectureSnapshot,
*,
physical_usable_gib: float,
context_tokens: int = ALPHA_CONTEXT_TOKENS,
concurrency: int = ALPHA_CONCURRENCY,
kv_dtype: str = ALPHA_KV_DTYPE,
indexer_layout: IndexerLayout = "conservative",
imbalance_factor: float = PLACEMENT_IMBALANCE_FACTOR,
) -> TopologyPlan:
"""Minimum and recommended node count for a homogeneous tier of this size."""
if (
isinstance(imbalance_factor, bool)
or not isinstance(imbalance_factor, (int, float))
or not math.isfinite(imbalance_factor)
or imbalance_factor < 1.0
):
raise ResourcePlanError(
"imbalance_factor must be finite and at least 1.0; a lower value would "
"assume the worst-placed node holds less than an equal share"
)
node = NodeMemory(
name=f"{physical_usable_gib:g}GiB-tier",
physical_usable_gib=physical_usable_gib,
unified=False,
)
budget = node.placement_budget_gib
if budget <= 0:
raise ResourcePlanError(
f"a {physical_usable_gib:g} GiB node has no placement budget after its "
f"{node.reserve_gib:.1f} GiB reserve"
)
weight_gib = manifest.total_bytes / GIB
kv_gib = (
kv_bytes(
snapshot,
context_tokens=context_tokens,
concurrency=concurrency,
dtype=kv_dtype,
indexer_layout=indexer_layout,
)
/ GIB
)
total = weight_gib + kv_gib
return TopologyPlan(
physical_usable_gib=physical_usable_gib,
reserve_gib=node.reserve_gib,
placement_budget_gib=budget,
weight_gib=weight_gib,
kv_gib=kv_gib,
total_placement_gib=total,
arithmetic_minimum_nodes=math.ceil(total / budget),
recommended_nodes=math.ceil(total * imbalance_factor / budget),
imbalance_factor=imbalance_factor,
)
@dataclass(frozen=True)
class RouteFit:
"""Whether a concrete, possibly heterogeneous set of nodes can hold the target."""
node_count: int
aggregate_usable_gib: float
aggregate_placement_budget_gib: float
required_placement_gib: float
fits: bool
meets_hard_fit_floor: bool
no_single_node_can_admit_target: bool
headroom_gib: float
reasons: tuple[str, ...]
def to_dict(self) -> dict:
return {
"node_count": self.node_count,
"aggregate_usable_gib": round(self.aggregate_usable_gib, 3),
"aggregate_placement_budget_gib": round(self.aggregate_placement_budget_gib, 3),
"required_placement_gib": round(self.required_placement_gib, 3),
"fits": self.fits,
"meets_hard_fit_floor": self.meets_hard_fit_floor,
"no_single_node_can_admit_target": self.no_single_node_can_admit_target,
"headroom_gib": round(self.headroom_gib, 3),
"reasons": list(self.reasons),
}
def plan_route(
manifest: TargetManifest,
snapshot: ArchitectureSnapshot,
nodes: list[NodeMemory],
*,
context_tokens: int = ALPHA_CONTEXT_TOKENS,
concurrency: int = ALPHA_CONCURRENCY,
kv_dtype: str = ALPHA_KV_DTYPE,
indexer_layout: IndexerLayout = "conservative",
) -> RouteFit:
"""Evaluate a concrete route. Every node's memory is already counted once."""
if len(nodes) < 2:
raise ResourcePlanError(
"the alpha target is distributed by definition; a route needs at least two "
"physical nodes"
)
names = [node.name for node in nodes]
if len(set(names)) != len(names):
raise ResourcePlanError(
"duplicate node names in the route; one physical machine counted twice is "
"the same double-count as adding integrated-GPU memory to system RAM"
)
weight_gib = manifest.total_bytes / GIB
kv_gib = (
kv_bytes(
snapshot,
context_tokens=context_tokens,
concurrency=concurrency,
dtype=kv_dtype,
indexer_layout=indexer_layout,
)
/ GIB
)
required = weight_gib + kv_gib
aggregate_usable = sum(node.physical_usable_gib for node in nodes)
aggregate_budget = sum(node.placement_budget_gib for node in nodes)
fits = aggregate_budget >= required
largest_budget = max(node.placement_budget_gib for node in nodes)
no_single_node = largest_budget < required
reasons: list[str] = []
if not fits:
reasons.append(
f"aggregate placement budget {aggregate_budget:.1f} GiB is below the "
f"{required:.1f} GiB the target needs after each node's reserve"
)
if not no_single_node:
reasons.append(
"at least one node could admit the complete target alone; that is a "
"single-host run, not distributed alpha"
)
if aggregate_usable < AGGREGATE_HARD_FIT_FLOOR_GIB:
reasons.append(
f"aggregate usable memory {aggregate_usable:.1f} GiB is below the "
f"{AGGREGATE_HARD_FIT_FLOOR_GIB:g} GiB experimental hard-fit floor"
)
return RouteFit(
node_count=len(nodes),
aggregate_usable_gib=aggregate_usable,
aggregate_placement_budget_gib=aggregate_budget,
required_placement_gib=required,
fits=fits,
meets_hard_fit_floor=aggregate_usable >= AGGREGATE_HARD_FIT_FLOOR_GIB,
no_single_node_can_admit_target=no_single_node,
headroom_gib=aggregate_budget - required,
reasons=tuple(reasons),
)
@dataclass(frozen=True)
class SeamPlan:
"""Bytes and latency across the activation seams of a route.
Bandwidth and latency are reported separately on purpose. Decode moves almost
nothing — 12 KiB per token per seam — so a faster link barely helps it. What
decode pays is *serial*: every generated token crosses every seam in order, so
the cost that matters is ``seams x per-hop latency``. A route that claims to be
fast because it is on 10 GbE has confused the two.
"""
node_count: int
seam_count: int
hidden_size: int
bytes_per_token_per_seam: int
prefill_bytes_per_seam: int
decode_bytes_per_seam_per_token: int
dsa_sideband_bytes_per_query: int
link_rate_gbps: float
meets_alpha_minimum: bool
is_recommended_link: bool
decode_serialization_ms_per_token: float
decode_latency_ms_per_token: float
decode_bandwidth_share_ms_per_token: float
prefill_serialization_ms: float
def to_dict(self) -> dict:
return {
"node_count": self.node_count,
"seam_count": self.seam_count,
"hidden_size": self.hidden_size,
"bytes_per_token_per_seam": self.bytes_per_token_per_seam,
"prefill_bytes_per_seam": self.prefill_bytes_per_seam,
"decode_bytes_per_seam_per_token": self.decode_bytes_per_seam_per_token,
"dsa_sideband_bytes_per_query": self.dsa_sideband_bytes_per_query,
"link_rate_gbps": self.link_rate_gbps,
"meets_alpha_minimum": self.meets_alpha_minimum,
"is_recommended_link": self.is_recommended_link,
"decode_serialization_ms_per_token": round(self.decode_serialization_ms_per_token, 4),
"decode_latency_ms_per_token": round(self.decode_latency_ms_per_token, 4),
"decode_bandwidth_share_ms_per_token": round(
self.decode_bandwidth_share_ms_per_token, 4
),
"prefill_serialization_ms": round(self.prefill_serialization_ms, 3),
}
def plan_seams(
snapshot: ArchitectureSnapshot,
*,
node_count: int,
context_tokens: int = ALPHA_CONTEXT_TOKENS,
link_rate_gbps: float = MIN_LINK_RATE_GBPS,
per_hop_latency_ms: float = 0.5,
) -> SeamPlan:
"""Model seam bytes, wire serialization, and serial per-hop latency separately."""
if not isinstance(node_count, int) or isinstance(node_count, bool) or node_count < 2:
raise ResourcePlanError("a seam exists only between two nodes")
if not isinstance(context_tokens, int) or isinstance(context_tokens, bool) or context_tokens <= 0:
raise ResourcePlanError("context_tokens must be a positive integer")
if (
isinstance(link_rate_gbps, bool)
or not isinstance(link_rate_gbps, (int, float))
or not math.isfinite(link_rate_gbps)
or link_rate_gbps <= 0
):
raise ResourcePlanError("link_rate_gbps must be finite and positive")
if (
isinstance(per_hop_latency_ms, bool)
or not isinstance(per_hop_latency_ms, (int, float))
or not math.isfinite(per_hop_latency_ms)
or per_hop_latency_ms < 0
):
raise ResourcePlanError("per_hop_latency_ms must be finite and non-negative")
hidden = int(snapshot["hidden_size"])
bytes_per_token = hidden * BF16_BYTES
seams = node_count - 1
bits_per_ms = link_rate_gbps * 1e9 / 1e3
decode_serialization_ms = (bytes_per_token * 8) / bits_per_ms
prefill_serialization_ms = (bytes_per_token * context_tokens * 8) / bits_per_ms
return SeamPlan(
node_count=node_count,
seam_count=seams,
hidden_size=hidden,
bytes_per_token_per_seam=bytes_per_token,
prefill_bytes_per_seam=bytes_per_token * context_tokens,
decode_bytes_per_seam_per_token=bytes_per_token,
dsa_sideband_bytes_per_query=int(snapshot["index_topk"]) * DSA_SIDEBAND_INT32_BYTES,
link_rate_gbps=link_rate_gbps,
meets_alpha_minimum=link_rate_gbps >= MIN_LINK_RATE_GBPS,
is_recommended_link=link_rate_gbps >= RECOMMENDED_LINK_RATE_GBPS,
decode_serialization_ms_per_token=decode_serialization_ms * seams,
decode_latency_ms_per_token=per_hop_latency_ms * seams,
decode_bandwidth_share_ms_per_token=decode_serialization_ms * seams,
prefill_serialization_ms=prefill_serialization_ms * seams,
)
ALPHA_TIERS_GIB: tuple[float, ...] = (32.0, 48.0, 64.0, 96.0, 128.0)
def plan_all_tiers(
manifest: TargetManifest, snapshot: ArchitectureSnapshot
) -> dict[str, TopologyPlan]:
"""The alpha tier table, recomputed from the pinned artifact and architecture."""
return {
f"{tier:g}": plan_topology(manifest, snapshot, physical_usable_gib=tier)
for tier in ALPHA_TIERS_GIB
}

73
packages/node/meshnet_node/native_protocol/__init__.py Normal file
View File

@@ -0,0 +1,73 @@
"""The native Shard protocol: Protobuf over gRPC/HTTP2 (ADR-0020).
`packages/node/native/proto/shard_runtime.proto` is the contract. This package
is how Python speaks it: the generated stubs plus the validation, framing and
chunking rules that the stubs cannot express.
Import the message types from here rather than reaching into `.generated`, so
the location of build output stays an implementation detail::
from meshnet_node.native_protocol import pb, encode_tensor, decode_bundle
"""
from __future__ import annotations
from .generated import shard_runtime_pb2 as pb
from .codec import (
BUNDLE_VERSION,
DEFAULT_MAX_CHUNK_BYTES,
DEFAULT_MAX_FRAGMENT_BYTES,
DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
DEFAULT_MAX_INFLIGHT_CHUNKS,
DEFAULT_MAX_PREFILL_CHUNK_TOKENS,
DEFAULT_MAX_TENSORS_PER_BUNDLE,
DEFAULT_MAX_TENSOR_DIMENSION,
DEFAULT_MAX_TENSOR_RANK,
HIDDEN_STATES,
SCHEMA_VERSION,
PayloadCorrupt,
PrefillChunk,
ProtocolError,
checksum_of,
crc32c,
decode_bundle,
decode_tensor,
default_flow_control,
encode_bundle,
encode_tensor,
expected_bytes,
itemsize,
negotiate_flow_control,
plan_prefill_chunks,
validate_session_message_size,
)
__all__ = [
"BUNDLE_VERSION",
"DEFAULT_MAX_CHUNK_BYTES",
"DEFAULT_MAX_FRAGMENT_BYTES",
"DEFAULT_MAX_FRAGMENTS_PER_TENSOR",
"DEFAULT_MAX_INFLIGHT_CHUNKS",
"DEFAULT_MAX_PREFILL_CHUNK_TOKENS",
"DEFAULT_MAX_TENSORS_PER_BUNDLE",
"DEFAULT_MAX_TENSOR_DIMENSION",
"DEFAULT_MAX_TENSOR_RANK",
"HIDDEN_STATES",
"SCHEMA_VERSION",
"PayloadCorrupt",
"PrefillChunk",
"ProtocolError",
"checksum_of",
"crc32c",
"decode_bundle",
"decode_tensor",
"default_flow_control",
"encode_bundle",
"encode_tensor",
"expected_bytes",
"itemsize",
"negotiate_flow_control",
"pb",
"plan_prefill_chunks",
"validate_session_message_size",
]

527
packages/node/meshnet_node/native_protocol/codec.py Normal file
View File

@@ -0,0 +1,527 @@
"""Encode and decode the native Shard protocol's named-tensor bundles.
The generated stubs give us message *structure*; they cannot enforce the
invariants that keep a distributed forward correct. A bundle whose declared
shape disagrees with its byte count, whose fragments leave a hole, or whose
checksum does not match is not a slightly-wrong activation — it is silently
wrong tokens for the rest of the generation. So decoding is validating: every
path into a tensor's bytes goes through :func:`decode_tensor`, which refuses a
payload it cannot fully account for.
Compression is a transport optimisation and is decided by the same policy layer
the existing HTTP seam already uses (``activation_compression``), so a node's
tuned thresholds apply to both transports.
"""
from __future__ import annotations
from dataclasses import dataclass
import struct
from typing import Iterable, Sequence
from ..activation_compression import (
CompressionPolicy,
compress_activation,
decompress_activation,
)
from .generated import shard_runtime_pb2 as pb
# The schema generation this build speaks. A peer offering something else is
# rejected at the handshake rather than being half-understood.
SCHEMA_VERSION = pb.SCHEMA_VERSION_1
# Generation of the tensor-bundle layout, versioned independently of the
# protocol so a boundary payload can evolve without a protocol bump.
BUNDLE_VERSION = 1
# Token-aligned prefill chunk bound. 128 tokens is the size ADR-0008 already
# uses on the HTTP seam; keeping it identical means seam bytes stay comparable
# across transports.
DEFAULT_MAX_PREFILL_CHUNK_TOKENS = 128
# gRPC's default maximum receive size. Fragmenting below it keeps us inside the
# default limits of any conformant peer instead of requiring every client to
# raise its window.
DEFAULT_MAX_CHUNK_BYTES = 4 * 1024 * 1024
# Leave room for envelope and framing overhead inside one chunk message.
DEFAULT_MAX_FRAGMENT_BYTES = 1024 * 1024
DEFAULT_MAX_INFLIGHT_CHUNKS = 8
DEFAULT_MAX_FRAGMENTS_PER_TENSOR = 64
DEFAULT_MAX_TENSORS_PER_BUNDLE = 64
DEFAULT_MAX_TENSOR_RANK = 8
DEFAULT_MAX_TENSOR_DIMENSION = (1 << 31) - 1
# Canonical boundary tensor name for a dense transformer hidden state.
HIDDEN_STATES = "hidden_states"
_DTYPE_ITEMSIZE: dict[int, int] = {
pb.DTYPE_BFLOAT16: 2,
pb.DTYPE_FLOAT16: 2,
pb.DTYPE_FLOAT32: 4,
pb.DTYPE_INT32: 4,
pb.DTYPE_INT64: 8,
pb.DTYPE_UINT8: 1,
pb.DTYPE_INT8: 1,
pb.DTYPE_BOOL: 1,
}
class ProtocolError(Exception):
"""A peer sent something this build cannot safely interpret."""
class PayloadCorrupt(ProtocolError):
"""A tensor payload failed validation: size, coverage, or checksum."""
def itemsize(dtype: int) -> int:
try:
return _DTYPE_ITEMSIZE[dtype]
except KeyError:
raise ProtocolError(f"unsupported dtype {dtype}") from None
def expected_bytes(
shape: Sequence[int],
dtype: int,
*,
max_rank: int = DEFAULT_MAX_TENSOR_RANK,
max_dimension: int = DEFAULT_MAX_TENSOR_DIMENSION,
max_bytes: int | None = None,
) -> int:
"""Byte count a tensor of `shape` and `dtype` must occupy."""
if len(shape) > max_rank:
raise ProtocolError(f"tensor rank {len(shape)} exceeds limit {max_rank}")
if any(dim < 0 or dim > max_dimension for dim in shape):
raise ProtocolError(
f"dimension outside 0..{max_dimension} in shape {list(shape)}"
)
size = itemsize(dtype)
count = 1
for dim in shape:
count *= dim
if max_bytes is not None and count * size > max_bytes:
raise ProtocolError(f"tensor shape {list(shape)} exceeds byte limit {max_bytes}")
return count * size
# --- CRC32C ----------------------------------------------------------------
#
# CRC32C (Castagnoli), not zlib's CRC32: it is the checksum gRPC, and the
# storage systems these payloads pass through, already use, and hardware
# implements it. `google_crc32c` is used when present; the table fallback keeps
# the default test suite dependency-free.
_CRC32C_POLY = 0x82F63B78
_CRC32C_TABLE: list[int] = []
for _i in range(256):
_c = _i
for _ in range(8):
_c = (_c >> 1) ^ (_CRC32C_POLY if _c & 1 else 0)
_CRC32C_TABLE.append(_c)
try: # pragma: no cover - depends on an optional native package
from google_crc32c import value as _fast_crc32c
except ImportError: # pragma: no cover
_fast_crc32c = None
def crc32c(data: bytes) -> int:
if _fast_crc32c is not None: # pragma: no cover - optional fast path
return _fast_crc32c(data)
crc = 0xFFFFFFFF
for byte in data:
crc = (crc >> 8) ^ _CRC32C_TABLE[(crc ^ byte) & 0xFF]
return crc ^ 0xFFFFFFFF
def checksum_of(data: bytes) -> pb.Checksum:
return pb.Checksum(
algorithm=pb.CHECKSUM_ALGORITHM_CRC32C,
value=struct.pack(">I", crc32c(data)),
)
# --- Tensors ---------------------------------------------------------------
def encode_tensor(
name: str,
data: bytes,
shape: Sequence[int],
dtype: int = pb.DTYPE_BFLOAT16,
*,
policy: CompressionPolicy | None = None,
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
max_fragment_bytes: int = DEFAULT_MAX_FRAGMENT_BYTES,
max_fragments: int = DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
) -> pb.NamedTensor:
"""Build a NamedTensor, compressing and fragmenting as needed.
`data` is the uncompressed little-endian payload. The checksum is taken over
it *before* compression so it stays valid whichever framing a hop chooses.
"""
if max_chunk_bytes <= 0 or max_fragment_bytes <= 0 or max_fragments <= 0:
raise ProtocolError("tensor byte/count bounds must be positive")
declared = expected_bytes(shape, dtype, max_bytes=max_chunk_bytes)
if len(data) != declared:
raise ProtocolError(
f"tensor {name!r} declares shape {list(shape)} ({declared} bytes) "
f"but carries {len(data)} bytes"
)
body = data
compression = pb.COMPRESSION_NONE
if policy is not None:
result = compress_activation(data, policy)
if result.compressed:
body = result.body
compression = pb.COMPRESSION_ZSTD
tensor = pb.NamedTensor(
name=name,
shape=list(shape),
dtype=dtype,
byte_order=pb.BYTE_ORDER_LITTLE_ENDIAN,
total_bytes=len(data),
compression=compression,
checksum=checksum_of(data),
)
# Fragment the wire body (compressed if we compressed). Offsets walk the
# wire body so a receiver can verify coverage without assuming arrival
# order; a zstd frame is not decodable per fragment, so reassembly comes
# first and decompression happens once, in decode_tensor.
slices = [body[i : i + max_fragment_bytes] for i in range(0, len(body), max_fragment_bytes)]
if not slices:
# A zero-element tensor is legal (e.g. an empty mask) and still needs a
# fragment, so coverage checks have something to verify.
slices = [b""]
if len(slices) > max_fragments:
raise ProtocolError(
f"tensor {name!r} needs {len(slices)} fragments, exceeding limit {max_fragments}"
)
offset = 0
for index, piece in enumerate(slices):
tensor.fragments.append(
pb.TensorFragment(
fragment_index=index,
fragment_count=len(slices),
byte_offset=offset,
payload=piece,
)
)
offset += len(piece)
return tensor
def decode_tensor(
tensor: pb.NamedTensor,
*,
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
max_fragment_bytes: int = DEFAULT_MAX_FRAGMENT_BYTES,
max_fragments: int = DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
) -> bytes:
"""Reassemble, decompress and validate a NamedTensor's payload.
Raises PayloadCorrupt rather than returning a payload it cannot fully
account for: a hole in the fragments or a bad checksum means the activation
is not what the sender computed, and continuing would corrupt the route.
"""
if max_chunk_bytes <= 0 or max_fragment_bytes <= 0 or max_fragments <= 0:
raise ProtocolError("negotiated byte/count bounds must be positive")
if tensor.total_bytes > max_chunk_bytes:
raise ProtocolError(
f"tensor {tensor.name!r} declares {tensor.total_bytes} bytes, exceeding "
f"the {max_chunk_bytes}-byte negotiated chunk bound"
)
if tensor.byte_order == pb.BYTE_ORDER_BIG_ENDIAN:
raise ProtocolError(f"tensor {tensor.name!r} is big-endian; wire order is little-endian")
if tensor.byte_order != pb.BYTE_ORDER_LITTLE_ENDIAN:
raise ProtocolError(f"tensor {tensor.name!r} declares no byte order")
declared = expected_bytes(
tensor.shape, tensor.dtype, max_bytes=max_chunk_bytes
)
if declared != tensor.total_bytes:
raise PayloadCorrupt(
f"tensor {tensor.name!r} shape {list(tensor.shape)} implies {declared} bytes "
f"but declares {tensor.total_bytes}"
)
if not tensor.fragments:
raise PayloadCorrupt(f"tensor {tensor.name!r} carries no fragments")
if len(tensor.fragments) > max_fragments:
raise PayloadCorrupt(
f"tensor {tensor.name!r} carries {len(tensor.fragments)} fragments, "
f"exceeding limit {max_fragments}"
)
if any(len(fragment.payload) > max_fragment_bytes for fragment in tensor.fragments):
raise PayloadCorrupt(
f"tensor {tensor.name!r} carries a fragment larger than "
f"{max_fragment_bytes} bytes"
)
wire_bytes = sum(len(fragment.payload) for fragment in tensor.fragments)
if wire_bytes > max_chunk_bytes:
raise PayloadCorrupt(
f"tensor {tensor.name!r} wire body exceeds the "
f"{max_chunk_bytes}-byte negotiated chunk bound"
)
fragments = sorted(tensor.fragments, key=lambda f: f.byte_offset)
count = fragments[0].fragment_count
if any(f.fragment_count != count for f in fragments):
raise PayloadCorrupt(f"tensor {tensor.name!r} has inconsistent fragment_count")
if len(fragments) != count:
raise PayloadCorrupt(
f"tensor {tensor.name!r} expects {count} fragments but carries {len(fragments)}"
)
if {f.fragment_index for f in fragments} != set(range(count)):
raise PayloadCorrupt(f"tensor {tensor.name!r} has duplicate or missing fragment indices")
# Contiguity: offsets must tile the body exactly, with no hole and no overlap.
body = bytearray()
for fragment in fragments:
if fragment.byte_offset != len(body):
raise PayloadCorrupt(
f"tensor {tensor.name!r} fragment {fragment.fragment_index} starts at "
f"{fragment.byte_offset}, expected {len(body)}"
)
body.extend(fragment.payload)
if tensor.compression == pb.COMPRESSION_ZSTD:
try:
data = decompress_activation(
bytes(body), "zstd", max_output_bytes=tensor.total_bytes
).body
except ValueError as exc:
raise PayloadCorrupt(
f"tensor {tensor.name!r} has invalid bounded zstd payload"
) from exc
elif tensor.compression == pb.COMPRESSION_NONE:
data = bytes(body)
else:
raise ProtocolError(
f"tensor {tensor.name!r} uses unspecified or unsupported compression"
)
if len(data) != tensor.total_bytes:
raise PayloadCorrupt(
f"tensor {tensor.name!r} declares {tensor.total_bytes} bytes but "
f"reassembled {len(data)}"
)
algorithm = tensor.checksum.algorithm
if algorithm == pb.CHECKSUM_ALGORITHM_CRC32C:
if tensor.checksum.value != struct.pack(">I", crc32c(data)):
raise PayloadCorrupt(f"tensor {tensor.name!r} failed its CRC32C check")
elif algorithm != pb.CHECKSUM_ALGORITHM_NONE:
raise ProtocolError(
f"tensor {tensor.name!r} uses unspecified or unsupported checksum algorithm"
)
return data
def encode_bundle(
tensors: Iterable[pb.NamedTensor],
*,
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
) -> pb.TensorBundle:
if max_chunk_bytes <= 0 or max_tensors <= 0:
raise ProtocolError("bundle byte/count bounds must be positive")
tensor_list = list(tensors)
if len(tensor_list) > max_tensors:
raise ProtocolError(
f"bundle carries {len(tensor_list)} tensors, exceeding limit {max_tensors}"
)
bundle = pb.TensorBundle(bundle_version=BUNDLE_VERSION, tensors=tensor_list)
if bundle.ByteSize() > max_chunk_bytes:
raise ProtocolError(
f"serialized tensor bundle exceeds the {max_chunk_bytes}-byte "
"negotiated chunk bound"
)
return bundle
def decode_bundle(
bundle: pb.TensorBundle,
*,
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
max_fragment_bytes: int = DEFAULT_MAX_FRAGMENT_BYTES,
max_fragments: int = DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
) -> dict[str, bytes]:
"""Validate every tensor in a bundle and return name -> payload."""
if bundle.bundle_version != BUNDLE_VERSION:
raise ProtocolError(
f"bundle version {bundle.bundle_version} is not supported by this build "
f"({BUNDLE_VERSION})"
)
if (
max_chunk_bytes <= 0
or max_fragment_bytes <= 0
or max_fragments <= 0
or max_tensors <= 0
):
raise ProtocolError("negotiated byte/count bounds must be positive")
if len(bundle.tensors) > max_tensors:
raise ProtocolError(
f"bundle carries {len(bundle.tensors)} tensors, exceeding limit {max_tensors}"
)
if bundle.ByteSize() > max_chunk_bytes:
raise ProtocolError(
f"serialized tensor bundle exceeds the {max_chunk_bytes}-byte "
"negotiated chunk bound"
)
payloads: dict[str, bytes] = {}
for tensor in bundle.tensors:
if not tensor.name:
raise ProtocolError("bundle carries an unnamed tensor")
if tensor.name in payloads:
raise ProtocolError(f"bundle carries duplicate tensor {tensor.name!r}")
payloads[tensor.name] = decode_tensor(
tensor,
max_chunk_bytes=max_chunk_bytes,
max_fragment_bytes=max_fragment_bytes,
max_fragments=max_fragments,
)
return payloads
def validate_session_message_size(
message: pb.SessionRequest | pb.SessionResponse,
*,
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
) -> int:
"""Reject an oversized complete stream frame, including protobuf overhead.
Bundle validation alone is insufficient because the envelope and oneof
framing are part of the same gRPC message. Senders call this immediately
before writing; receivers configure gRPC's receive limit to the same value
and call it again before semantic decoding.
"""
if max_chunk_bytes <= 0:
raise ProtocolError("max_chunk_bytes must be positive")
if not isinstance(message, (pb.SessionRequest, pb.SessionResponse)):
raise ProtocolError("size validation requires a session request or response")
size = message.ByteSize()
if size > max_chunk_bytes:
raise ProtocolError(
f"serialized session message is {size} bytes, exceeding the "
f"{max_chunk_bytes}-byte negotiated chunk bound"
)
return size
# --- Bounded prefill chunking ----------------------------------------------
@dataclass(frozen=True)
class PrefillChunk:
"""One token-aligned slice of a prefill."""
chunk_index: int
chunk_count: int
first_position: int
token_count: int
@property
def final_chunk(self) -> bool:
return self.chunk_index == self.chunk_count - 1
def chunk_info(self) -> pb.ChunkInfo:
return pb.ChunkInfo(
chunk_index=self.chunk_index,
chunk_count=self.chunk_count,
final_chunk=self.final_chunk,
)
def position(self) -> pb.PositionSpan:
return pb.PositionSpan(
first_position=self.first_position, token_count=self.token_count
)
def plan_prefill_chunks(
total_tokens: int,
*,
first_position: int = 0,
max_tokens: int = DEFAULT_MAX_PREFILL_CHUNK_TOKENS,
) -> list[PrefillChunk]:
"""Split a prefill into bounded, token-aligned chunks.
Splits fall on token boundaries only (ADR-0008): a fragment of a token's
hidden state is not a thing a receiver can execute.
"""
if total_tokens <= 0:
raise ProtocolError("a prefill must carry at least one token")
if max_tokens <= 0:
raise ProtocolError("max_tokens must be positive")
count = (total_tokens + max_tokens - 1) // max_tokens
chunks = []
for index in range(count):
offset = index * max_tokens
chunks.append(
PrefillChunk(
chunk_index=index,
chunk_count=count,
first_position=first_position + offset,
token_count=min(max_tokens, total_tokens - offset),
)
)
return chunks
def default_flow_control() -> pb.FlowControl:
return pb.FlowControl(
credits_granted=DEFAULT_MAX_INFLIGHT_CHUNKS,
max_inflight_chunks=DEFAULT_MAX_INFLIGHT_CHUNKS,
max_chunk_bytes=DEFAULT_MAX_CHUNK_BYTES,
max_prefill_chunk_tokens=DEFAULT_MAX_PREFILL_CHUNK_TOKENS,
)
def negotiate_flow_control(
proposed: pb.FlowControl, limits: pb.FlowControl
) -> pb.FlowControl:
"""Settle a stream's limits: the strictest bound of either peer wins.
Taking the minimum means neither peer can raise the other's ceiling, so a
misconfigured — or hostile — sender cannot talk a worker into unbounded
queues by proposing a large window.
"""
def _min(a: int, b: int, fallback: int) -> int:
candidates = [v for v in (a, b) if v > 0]
return min(candidates) if candidates else fallback
max_inflight_chunks = _min(
proposed.max_inflight_chunks,
limits.max_inflight_chunks,
DEFAULT_MAX_INFLIGHT_CHUNKS,
)
credits_granted = min(
_min(
proposed.credits_granted,
limits.credits_granted,
DEFAULT_MAX_INFLIGHT_CHUNKS,
),
max_inflight_chunks,
)
return pb.FlowControl(
credits_granted=credits_granted,
max_inflight_chunks=max_inflight_chunks,
max_chunk_bytes=_min(
proposed.max_chunk_bytes, limits.max_chunk_bytes, DEFAULT_MAX_CHUNK_BYTES
),
max_prefill_chunk_tokens=_min(
proposed.max_prefill_chunk_tokens,
limits.max_prefill_chunk_tokens,
DEFAULT_MAX_PREFILL_CHUNK_TOKENS,
),
)

141
packages/node/meshnet_node/native_protocol/conformance.py Normal file
View File

@@ -0,0 +1,141 @@
"""Canonical conformance vectors for the native Shard protocol.
Two independently-written codecs that each round-trip their own output prove
nothing about each other. These vectors are the shared reference: Python builds
the canonical message, the bytes are committed under
`packages/node/native/testdata/`, and the C++ test parses those exact bytes and
asserts the same field values. A change that alters the wire meaning of a field
breaks the vector in both languages instead of drifting silently in one.
The vector deliberately exercises every field group the protocol promises to
carry — identity, epoch, fingerprint, range, phase, position, idempotency,
cache expectation, deadline, chunking, compression, checksum and a multi-
fragment named tensor — so it doubles as an executable inventory of the
contract.
"""
from __future__ import annotations
import pathlib
from . import codec
from .generated import shard_runtime_pb2 as pb
# Committed vectors live beside the schema, under `packages/node/native/`.
# parents[2] is `packages/node`: native_protocol -> meshnet_node -> node.
TESTDATA_DIR = pathlib.Path(__file__).resolve().parents[2] / "native/testdata"
GOLDEN_SESSION_REQUEST = "session_request_golden.binpb"
GOLDEN_CAPABILITY_REPORT = "capability_report_golden.binpb"
# Written by the C++ conformance test into its build tree; the Python test picks
# it up when present to prove the two languages agree byte-for-byte.
CPP_ROUNDTRIP = "cpp_roundtrip.binpb"
# Fixed, non-default values. Every one is chosen to be distinguishable from a
# proto3 default so an unset field can never masquerade as a correct one.
WORK_ID = "work-7f3a"
ROUTE_SESSION_ID = "rs-2b91"
ROUTE_EPOCH = 7
IDEMPOTENCY_STEP = 42
FIRST_POSITION = 256
TOKEN_COUNT = 128
EXPECTED_PAST_LEN = 256
DEADLINE_UNIX_NANOS = 1_800_000_000_000_000_000
MODEL_ARTIFACT_DIGEST = "sha256:1f0c9d2e"
RUNTIME_RECIPE_DIGEST = "sha256:ab77e410"
RECIPE_ID = "llama-gguf-q4km-rocm"
RECIPE_VERSION = "3"
CATALOGUE_VERSION = "2026.07.1"
START_LAYER = 12
END_LAYER = 24
EFFECTIVE_START_LAYER = 16
HIDDEN_SIZE = 8
# A payload big enough to force more than one fragment at the bound below, so
# the vector actually exercises reassembly rather than the one-fragment path.
FRAGMENT_BYTES = 64
TENSOR_SHAPE = [1, TOKEN_COUNT, HIDDEN_SIZE]
def canonical_payload() -> bytes:
"""Deterministic bfloat16-sized payload for the canonical tensor."""
total = codec.expected_bytes(TENSOR_SHAPE, pb.DTYPE_BFLOAT16)
return bytes((i * 7 + 11) % 256 for i in range(total))
def canonical_session_request() -> pb.SessionRequest:
"""The canonical prefill chunk carried on a session stream."""
tensor = codec.encode_tensor(
codec.HIDDEN_STATES,
canonical_payload(),
TENSOR_SHAPE,
pb.DTYPE_BFLOAT16,
max_fragment_bytes=FRAGMENT_BYTES,
)
envelope = pb.Envelope(
schema_version=pb.SCHEMA_VERSION_1,
work_id=WORK_ID,
route_session_id=ROUTE_SESSION_ID,
route_epoch=ROUTE_EPOCH,
fingerprint=pb.Fingerprint(
model_artifact_digest=MODEL_ARTIFACT_DIGEST,
runtime_recipe_digest=RUNTIME_RECIPE_DIGEST,
recipe_id=RECIPE_ID,
recipe_version=RECIPE_VERSION,
catalogue_version=CATALOGUE_VERSION,
),
shard_range=pb.ShardRange(
start_layer=START_LAYER,
end_layer=END_LAYER,
effective_start_layer=EFFECTIVE_START_LAYER,
),
phase=pb.PHASE_PREFILL,
position=pb.PositionSpan(
first_position=FIRST_POSITION, token_count=TOKEN_COUNT
),
idempotency_step=IDEMPOTENCY_STEP,
cache_expectation=pb.CacheExpectation(
mode=pb.CACHE_MODE_PREFILL, expected_past_len=EXPECTED_PAST_LEN
),
deadline_unix_nanos=DEADLINE_UNIX_NANOS,
chunk=pb.ChunkInfo(chunk_index=1, chunk_count=3, final_chunk=False),
)
return pb.SessionRequest(
chunk=pb.ActivationChunk(
envelope=envelope, bundle=codec.encode_bundle([tensor])
)
)
def canonical_capability_report() -> pb.CapabilityReport:
"""The canonical capability report a worker answers admission with."""
return pb.CapabilityReport(
schema_version=pb.SCHEMA_VERSION_1,
fingerprint=pb.Fingerprint(
model_artifact_digest=MODEL_ARTIFACT_DIGEST,
runtime_recipe_digest=RUNTIME_RECIPE_DIGEST,
recipe_id=RECIPE_ID,
recipe_version=RECIPE_VERSION,
catalogue_version=CATALOGUE_VERSION,
),
shard_range=pb.ShardRange(
start_layer=START_LAYER,
end_layer=END_LAYER,
effective_start_layer=EFFECTIVE_START_LAYER,
),
backend="rocm",
device="gfx1151",
validated=True,
max_concurrent_sessions=4,
max_context_tokens=8192,
flow_control=codec.default_flow_control(),
accepted_compression=[pb.COMPRESSION_NONE, pb.COMPRESSION_ZSTD],
supported_schema_versions=[pb.SCHEMA_VERSION_1],
validated_at_unix_nanos=DEADLINE_UNIX_NANOS,
)
def serialize(message) -> bytes:
"""Serialize deterministically, so committed golden bytes are stable."""
return message.SerializeToString(deterministic=True)

2
packages/node/meshnet_node/native_protocol/generated/__init__.py Normal file
View File

@@ -0,0 +1,2 @@
# Generated by scripts/generate_native_protocol.py. Do not edit.
"""Generated protobuf/gRPC stubs for the native Shard protocol."""

117
packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.py Normal file
View File

File diff suppressed because one or more lines are too long

525
packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.pyi Normal file
View File

@@ -0,0 +1,525 @@
from google.protobuf.internal import containers as _containers
from google.protobuf.internal import enum_type_wrapper as _enum_type_wrapper
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from collections.abc import Iterable as _Iterable, Mapping as _Mapping
from typing import ClassVar as _ClassVar, Optional as _Optional, Union as _Union
DESCRIPTOR: _descriptor.FileDescriptor
class SchemaVersion(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
SCHEMA_VERSION_UNSPECIFIED: _ClassVar[SchemaVersion]
SCHEMA_VERSION_1: _ClassVar[SchemaVersion]
class DType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
DTYPE_UNSPECIFIED: _ClassVar[DType]
DTYPE_BFLOAT16: _ClassVar[DType]
DTYPE_FLOAT16: _ClassVar[DType]
DTYPE_FLOAT32: _ClassVar[DType]
DTYPE_INT32: _ClassVar[DType]
DTYPE_INT64: _ClassVar[DType]
DTYPE_UINT8: _ClassVar[DType]
DTYPE_INT8: _ClassVar[DType]
DTYPE_BOOL: _ClassVar[DType]
class ByteOrder(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
BYTE_ORDER_UNSPECIFIED: _ClassVar[ByteOrder]
BYTE_ORDER_LITTLE_ENDIAN: _ClassVar[ByteOrder]
BYTE_ORDER_BIG_ENDIAN: _ClassVar[ByteOrder]
class Compression(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
COMPRESSION_UNSPECIFIED: _ClassVar[Compression]
COMPRESSION_NONE: _ClassVar[Compression]
COMPRESSION_ZSTD: _ClassVar[Compression]
class ChecksumAlgorithm(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
CHECKSUM_ALGORITHM_UNSPECIFIED: _ClassVar[ChecksumAlgorithm]
CHECKSUM_ALGORITHM_NONE: _ClassVar[ChecksumAlgorithm]
CHECKSUM_ALGORITHM_CRC32C: _ClassVar[ChecksumAlgorithm]
class Phase(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
PHASE_UNSPECIFIED: _ClassVar[Phase]
PHASE_PREFILL: _ClassVar[Phase]
PHASE_DECODE: _ClassVar[Phase]
PHASE_RELEASE: _ClassVar[Phase]
PHASE_CANCEL: _ClassVar[Phase]
class CacheMode(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
CACHE_MODE_UNSPECIFIED: _ClassVar[CacheMode]
CACHE_MODE_STATELESS: _ClassVar[CacheMode]
CACHE_MODE_PREFILL: _ClassVar[CacheMode]
CACHE_MODE_DECODE: _ClassVar[CacheMode]
class ErrorCode(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
ERROR_CODE_UNSPECIFIED: _ClassVar[ErrorCode]
ERROR_CODE_SCHEMA_UNSUPPORTED: _ClassVar[ErrorCode]
ERROR_CODE_FINGERPRINT_MISMATCH: _ClassVar[ErrorCode]
ERROR_CODE_EPOCH_STALE: _ClassVar[ErrorCode]
ERROR_CODE_SHARD_RANGE_MISMATCH: _ClassVar[ErrorCode]
ERROR_CODE_CACHE_MISS: _ClassVar[ErrorCode]
ERROR_CODE_RESOURCE_EXHAUSTED: _ClassVar[ErrorCode]
ERROR_CODE_PAYLOAD_CORRUPT: _ClassVar[ErrorCode]
ERROR_CODE_CANCELLED: _ClassVar[ErrorCode]
ERROR_CODE_DEADLINE_EXCEEDED: _ClassVar[ErrorCode]
ERROR_CODE_FLOW_CONTROL_VIOLATION: _ClassVar[ErrorCode]
ERROR_CODE_INTERNAL: _ClassVar[ErrorCode]
class ServingState(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
__slots__ = ()
SERVING_STATE_UNSPECIFIED: _ClassVar[ServingState]
SERVING_STATE_SERVING: _ClassVar[ServingState]
SERVING_STATE_DRAINING: _ClassVar[ServingState]
SERVING_STATE_NOT_SERVING: _ClassVar[ServingState]
SCHEMA_VERSION_UNSPECIFIED: SchemaVersion
SCHEMA_VERSION_1: SchemaVersion
DTYPE_UNSPECIFIED: DType
DTYPE_BFLOAT16: DType
DTYPE_FLOAT16: DType
DTYPE_FLOAT32: DType
DTYPE_INT32: DType
DTYPE_INT64: DType
DTYPE_UINT8: DType
DTYPE_INT8: DType
DTYPE_BOOL: DType
BYTE_ORDER_UNSPECIFIED: ByteOrder
BYTE_ORDER_LITTLE_ENDIAN: ByteOrder
BYTE_ORDER_BIG_ENDIAN: ByteOrder
COMPRESSION_UNSPECIFIED: Compression
COMPRESSION_NONE: Compression
COMPRESSION_ZSTD: Compression
CHECKSUM_ALGORITHM_UNSPECIFIED: ChecksumAlgorithm
CHECKSUM_ALGORITHM_NONE: ChecksumAlgorithm
CHECKSUM_ALGORITHM_CRC32C: ChecksumAlgorithm
PHASE_UNSPECIFIED: Phase
PHASE_PREFILL: Phase
PHASE_DECODE: Phase
PHASE_RELEASE: Phase
PHASE_CANCEL: Phase
CACHE_MODE_UNSPECIFIED: CacheMode
CACHE_MODE_STATELESS: CacheMode
CACHE_MODE_PREFILL: CacheMode
CACHE_MODE_DECODE: CacheMode
ERROR_CODE_UNSPECIFIED: ErrorCode
ERROR_CODE_SCHEMA_UNSUPPORTED: ErrorCode
ERROR_CODE_FINGERPRINT_MISMATCH: ErrorCode
ERROR_CODE_EPOCH_STALE: ErrorCode
ERROR_CODE_SHARD_RANGE_MISMATCH: ErrorCode
ERROR_CODE_CACHE_MISS: ErrorCode
ERROR_CODE_RESOURCE_EXHAUSTED: ErrorCode
ERROR_CODE_PAYLOAD_CORRUPT: ErrorCode
ERROR_CODE_CANCELLED: ErrorCode
ERROR_CODE_DEADLINE_EXCEEDED: ErrorCode
ERROR_CODE_FLOW_CONTROL_VIOLATION: ErrorCode
ERROR_CODE_INTERNAL: ErrorCode
SERVING_STATE_UNSPECIFIED: ServingState
SERVING_STATE_SERVING: ServingState
SERVING_STATE_DRAINING: ServingState
SERVING_STATE_NOT_SERVING: ServingState
class Checksum(_message.Message):
__slots__ = ("algorithm", "value")
ALGORITHM_FIELD_NUMBER: _ClassVar[int]
VALUE_FIELD_NUMBER: _ClassVar[int]
algorithm: ChecksumAlgorithm
value: bytes
def __init__(self, algorithm: _Optional[_Union[ChecksumAlgorithm, str]] = ..., value: _Optional[bytes] = ...) -> None: ...
class TensorFragment(_message.Message):
__slots__ = ("fragment_index", "fragment_count", "byte_offset", "payload")
FRAGMENT_INDEX_FIELD_NUMBER: _ClassVar[int]
FRAGMENT_COUNT_FIELD_NUMBER: _ClassVar[int]
BYTE_OFFSET_FIELD_NUMBER: _ClassVar[int]
PAYLOAD_FIELD_NUMBER: _ClassVar[int]
fragment_index: int
fragment_count: int
byte_offset: int
payload: bytes
def __init__(self, fragment_index: _Optional[int] = ..., fragment_count: _Optional[int] = ..., byte_offset: _Optional[int] = ..., payload: _Optional[bytes] = ...) -> None: ...
class NamedTensor(_message.Message):
__slots__ = ("name", "shape", "dtype", "byte_order", "total_bytes", "compression", "checksum", "fragments")
NAME_FIELD_NUMBER: _ClassVar[int]
SHAPE_FIELD_NUMBER: _ClassVar[int]
DTYPE_FIELD_NUMBER: _ClassVar[int]
BYTE_ORDER_FIELD_NUMBER: _ClassVar[int]
TOTAL_BYTES_FIELD_NUMBER: _ClassVar[int]
COMPRESSION_FIELD_NUMBER: _ClassVar[int]
CHECKSUM_FIELD_NUMBER: _ClassVar[int]
FRAGMENTS_FIELD_NUMBER: _ClassVar[int]
name: str
shape: _containers.RepeatedScalarFieldContainer[int]
dtype: DType
byte_order: ByteOrder
total_bytes: int
compression: Compression
checksum: Checksum
fragments: _containers.RepeatedCompositeFieldContainer[TensorFragment]
def __init__(self, name: _Optional[str] = ..., shape: _Optional[_Iterable[int]] = ..., dtype: _Optional[_Union[DType, str]] = ..., byte_order: _Optional[_Union[ByteOrder, str]] = ..., total_bytes: _Optional[int] = ..., compression: _Optional[_Union[Compression, str]] = ..., checksum: _Optional[_Union[Checksum, _Mapping]] = ..., fragments: _Optional[_Iterable[_Union[TensorFragment, _Mapping]]] = ...) -> None: ...
class TensorBundle(_message.Message):
__slots__ = ("bundle_version", "tensors")
BUNDLE_VERSION_FIELD_NUMBER: _ClassVar[int]
TENSORS_FIELD_NUMBER: _ClassVar[int]
bundle_version: int
tensors: _containers.RepeatedCompositeFieldContainer[NamedTensor]
def __init__(self, bundle_version: _Optional[int] = ..., tensors: _Optional[_Iterable[_Union[NamedTensor, _Mapping]]] = ...) -> None: ...
class Fingerprint(_message.Message):
__slots__ = ("model_artifact_digest", "runtime_recipe_digest", "recipe_id", "recipe_version", "catalogue_version")
MODEL_ARTIFACT_DIGEST_FIELD_NUMBER: _ClassVar[int]
RUNTIME_RECIPE_DIGEST_FIELD_NUMBER: _ClassVar[int]
RECIPE_ID_FIELD_NUMBER: _ClassVar[int]
RECIPE_VERSION_FIELD_NUMBER: _ClassVar[int]
CATALOGUE_VERSION_FIELD_NUMBER: _ClassVar[int]
model_artifact_digest: str
runtime_recipe_digest: str
recipe_id: str
recipe_version: str
catalogue_version: str
def __init__(self, model_artifact_digest: _Optional[str] = ..., runtime_recipe_digest: _Optional[str] = ..., recipe_id: _Optional[str] = ..., recipe_version: _Optional[str] = ..., catalogue_version: _Optional[str] = ...) -> None: ...
class ShardRange(_message.Message):
__slots__ = ("start_layer", "end_layer", "effective_start_layer")
START_LAYER_FIELD_NUMBER: _ClassVar[int]
END_LAYER_FIELD_NUMBER: _ClassVar[int]
EFFECTIVE_START_LAYER_FIELD_NUMBER: _ClassVar[int]
start_layer: int
end_layer: int
effective_start_layer: int
def __init__(self, start_layer: _Optional[int] = ..., end_layer: _Optional[int] = ..., effective_start_layer: _Optional[int] = ...) -> None: ...
class PositionSpan(_message.Message):
__slots__ = ("first_position", "token_count")
FIRST_POSITION_FIELD_NUMBER: _ClassVar[int]
TOKEN_COUNT_FIELD_NUMBER: _ClassVar[int]
first_position: int
token_count: int
def __init__(self, first_position: _Optional[int] = ..., token_count: _Optional[int] = ...) -> None: ...
class ChunkInfo(_message.Message):
__slots__ = ("chunk_index", "chunk_count", "final_chunk")
CHUNK_INDEX_FIELD_NUMBER: _ClassVar[int]
CHUNK_COUNT_FIELD_NUMBER: _ClassVar[int]
FINAL_CHUNK_FIELD_NUMBER: _ClassVar[int]
chunk_index: int
chunk_count: int
final_chunk: bool
def __init__(self, chunk_index: _Optional[int] = ..., chunk_count: _Optional[int] = ..., final_chunk: _Optional[bool] = ...) -> None: ...
class CacheExpectation(_message.Message):
__slots__ = ("mode", "expected_past_len")
MODE_FIELD_NUMBER: _ClassVar[int]
EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int]
mode: CacheMode
expected_past_len: int
def __init__(self, mode: _Optional[_Union[CacheMode, str]] = ..., expected_past_len: _Optional[int] = ...) -> None: ...
class CacheResult(_message.Message):
__slots__ = ("mode", "past_len", "cache_hit")
MODE_FIELD_NUMBER: _ClassVar[int]
PAST_LEN_FIELD_NUMBER: _ClassVar[int]
CACHE_HIT_FIELD_NUMBER: _ClassVar[int]
mode: CacheMode
past_len: int
cache_hit: bool
def __init__(self, mode: _Optional[_Union[CacheMode, str]] = ..., past_len: _Optional[int] = ..., cache_hit: _Optional[bool] = ...) -> None: ...
class Envelope(_message.Message):
__slots__ = ("schema_version", "work_id", "route_session_id", "route_epoch", "fingerprint", "shard_range", "phase", "position", "idempotency_step", "cache_expectation", "deadline_unix_nanos", "chunk")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
WORK_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
FINGERPRINT_FIELD_NUMBER: _ClassVar[int]
SHARD_RANGE_FIELD_NUMBER: _ClassVar[int]
PHASE_FIELD_NUMBER: _ClassVar[int]
POSITION_FIELD_NUMBER: _ClassVar[int]
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
CACHE_EXPECTATION_FIELD_NUMBER: _ClassVar[int]
DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int]
CHUNK_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
work_id: str
route_session_id: str
route_epoch: int
fingerprint: Fingerprint
shard_range: ShardRange
phase: Phase
position: PositionSpan
idempotency_step: int
cache_expectation: CacheExpectation
deadline_unix_nanos: int
chunk: ChunkInfo
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., work_id: _Optional[str] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ..., shard_range: _Optional[_Union[ShardRange, _Mapping]] = ..., phase: _Optional[_Union[Phase, str]] = ..., position: _Optional[_Union[PositionSpan, _Mapping]] = ..., idempotency_step: _Optional[int] = ..., cache_expectation: _Optional[_Union[CacheExpectation, _Mapping]] = ..., deadline_unix_nanos: _Optional[int] = ..., chunk: _Optional[_Union[ChunkInfo, _Mapping]] = ...) -> None: ...
class ShardError(_message.Message):
__slots__ = ("code", "detail", "retryable", "actual_past_len")
CODE_FIELD_NUMBER: _ClassVar[int]
DETAIL_FIELD_NUMBER: _ClassVar[int]
RETRYABLE_FIELD_NUMBER: _ClassVar[int]
ACTUAL_PAST_LEN_FIELD_NUMBER: _ClassVar[int]
code: ErrorCode
detail: str
retryable: bool
actual_past_len: int
def __init__(self, code: _Optional[_Union[ErrorCode, str]] = ..., detail: _Optional[str] = ..., retryable: _Optional[bool] = ..., actual_past_len: _Optional[int] = ...) -> None: ...
class FlowControl(_message.Message):
__slots__ = ("credits_granted", "max_inflight_chunks", "max_chunk_bytes", "max_prefill_chunk_tokens")
CREDITS_GRANTED_FIELD_NUMBER: _ClassVar[int]
MAX_INFLIGHT_CHUNKS_FIELD_NUMBER: _ClassVar[int]
MAX_CHUNK_BYTES_FIELD_NUMBER: _ClassVar[int]
MAX_PREFILL_CHUNK_TOKENS_FIELD_NUMBER: _ClassVar[int]
credits_granted: int
max_inflight_chunks: int
max_chunk_bytes: int
max_prefill_chunk_tokens: int
def __init__(self, credits_granted: _Optional[int] = ..., max_inflight_chunks: _Optional[int] = ..., max_chunk_bytes: _Optional[int] = ..., max_prefill_chunk_tokens: _Optional[int] = ...) -> None: ...
class SessionOpen(_message.Message):
__slots__ = ("schema_version", "route_session_id", "route_epoch", "fingerprint", "shard_range", "proposed_flow_control", "accepted_compression")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
FINGERPRINT_FIELD_NUMBER: _ClassVar[int]
SHARD_RANGE_FIELD_NUMBER: _ClassVar[int]
PROPOSED_FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
ACCEPTED_COMPRESSION_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
route_session_id: str
route_epoch: int
fingerprint: Fingerprint
shard_range: ShardRange
proposed_flow_control: FlowControl
accepted_compression: _containers.RepeatedScalarFieldContainer[Compression]
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ..., shard_range: _Optional[_Union[ShardRange, _Mapping]] = ..., proposed_flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., accepted_compression: _Optional[_Iterable[_Union[Compression, str]]] = ...) -> None: ...
class SessionAccepted(_message.Message):
__slots__ = ("schema_version", "route_session_id", "route_epoch", "flow_control", "accepted_compression", "fingerprint")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
ACCEPTED_COMPRESSION_FIELD_NUMBER: _ClassVar[int]
FINGERPRINT_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
route_session_id: str
route_epoch: int
flow_control: FlowControl
accepted_compression: _containers.RepeatedScalarFieldContainer[Compression]
fingerprint: Fingerprint
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., accepted_compression: _Optional[_Iterable[_Union[Compression, str]]] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ...) -> None: ...
class ActivationChunk(_message.Message):
__slots__ = ("envelope", "bundle")
ENVELOPE_FIELD_NUMBER: _ClassVar[int]
BUNDLE_FIELD_NUMBER: _ClassVar[int]
envelope: Envelope
bundle: TensorBundle
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ...
class DecodeStep(_message.Message):
__slots__ = ("idempotency_step", "position", "expected_past_len", "tensor", "work_id", "deadline_unix_nanos")
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
POSITION_FIELD_NUMBER: _ClassVar[int]
EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int]
TENSOR_FIELD_NUMBER: _ClassVar[int]
WORK_ID_FIELD_NUMBER: _ClassVar[int]
DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int]
idempotency_step: int
position: int
expected_past_len: int
tensor: NamedTensor
work_id: str
deadline_unix_nanos: int
def __init__(self, idempotency_step: _Optional[int] = ..., position: _Optional[int] = ..., expected_past_len: _Optional[int] = ..., tensor: _Optional[_Union[NamedTensor, _Mapping]] = ..., work_id: _Optional[str] = ..., deadline_unix_nanos: _Optional[int] = ...) -> None: ...
class ReleaseSignal(_message.Message):
__slots__ = ("route_session_id", "route_epoch", "work_id")
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
WORK_ID_FIELD_NUMBER: _ClassVar[int]
route_session_id: str
route_epoch: int
work_id: str
def __init__(self, route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., work_id: _Optional[str] = ...) -> None: ...
class CancelSignal(_message.Message):
__slots__ = ("route_session_id", "route_epoch", "work_id", "reason")
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
WORK_ID_FIELD_NUMBER: _ClassVar[int]
REASON_FIELD_NUMBER: _ClassVar[int]
route_session_id: str
route_epoch: int
work_id: str
reason: str
def __init__(self, route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., work_id: _Optional[str] = ..., reason: _Optional[str] = ...) -> None: ...
class Ack(_message.Message):
__slots__ = ("work_id", "idempotency_step", "cache_result", "duplicate", "execution_nanos")
WORK_ID_FIELD_NUMBER: _ClassVar[int]
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
CACHE_RESULT_FIELD_NUMBER: _ClassVar[int]
DUPLICATE_FIELD_NUMBER: _ClassVar[int]
EXECUTION_NANOS_FIELD_NUMBER: _ClassVar[int]
work_id: str
idempotency_step: int
cache_result: CacheResult
duplicate: bool
execution_nanos: int
def __init__(self, work_id: _Optional[str] = ..., idempotency_step: _Optional[int] = ..., cache_result: _Optional[_Union[CacheResult, _Mapping]] = ..., duplicate: _Optional[bool] = ..., execution_nanos: _Optional[int] = ...) -> None: ...
class ShardStatus(_message.Message):
__slots__ = ("work_id", "route_session_id", "idempotency_step", "error", "terminal")
WORK_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
ERROR_FIELD_NUMBER: _ClassVar[int]
TERMINAL_FIELD_NUMBER: _ClassVar[int]
work_id: str
route_session_id: str
idempotency_step: int
error: ShardError
terminal: bool
def __init__(self, work_id: _Optional[str] = ..., route_session_id: _Optional[str] = ..., idempotency_step: _Optional[int] = ..., error: _Optional[_Union[ShardError, _Mapping]] = ..., terminal: _Optional[bool] = ...) -> None: ...
class SessionRequest(_message.Message):
__slots__ = ("open", "chunk", "decode", "flow_control", "release", "cancel")
OPEN_FIELD_NUMBER: _ClassVar[int]
CHUNK_FIELD_NUMBER: _ClassVar[int]
DECODE_FIELD_NUMBER: _ClassVar[int]
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
RELEASE_FIELD_NUMBER: _ClassVar[int]
CANCEL_FIELD_NUMBER: _ClassVar[int]
open: SessionOpen
chunk: ActivationChunk
decode: DecodeStep
flow_control: FlowControl
release: ReleaseSignal
cancel: CancelSignal
def __init__(self, open: _Optional[_Union[SessionOpen, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., decode: _Optional[_Union[DecodeStep, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., release: _Optional[_Union[ReleaseSignal, _Mapping]] = ..., cancel: _Optional[_Union[CancelSignal, _Mapping]] = ...) -> None: ...
class SessionResponse(_message.Message):
__slots__ = ("accepted", "chunk", "ack", "flow_control", "status")
ACCEPTED_FIELD_NUMBER: _ClassVar[int]
CHUNK_FIELD_NUMBER: _ClassVar[int]
ACK_FIELD_NUMBER: _ClassVar[int]
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
STATUS_FIELD_NUMBER: _ClassVar[int]
accepted: SessionAccepted
chunk: ActivationChunk
ack: Ack
flow_control: FlowControl
status: ShardStatus
def __init__(self, accepted: _Optional[_Union[SessionAccepted, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., ack: _Optional[_Union[Ack, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., status: _Optional[_Union[ShardStatus, _Mapping]] = ...) -> None: ...
class CapabilityRequest(_message.Message):
__slots__ = ("schema_version",)
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ...) -> None: ...
class CapabilityReport(_message.Message):
__slots__ = ("schema_version", "fingerprint", "shard_range", "backend", "device", "validated", "detail", "max_concurrent_sessions", "max_context_tokens", "flow_control", "accepted_compression", "supported_schema_versions", "validated_at_unix_nanos")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
FINGERPRINT_FIELD_NUMBER: _ClassVar[int]
SHARD_RANGE_FIELD_NUMBER: _ClassVar[int]
BACKEND_FIELD_NUMBER: _ClassVar[int]
DEVICE_FIELD_NUMBER: _ClassVar[int]
VALIDATED_FIELD_NUMBER: _ClassVar[int]
DETAIL_FIELD_NUMBER: _ClassVar[int]
MAX_CONCURRENT_SESSIONS_FIELD_NUMBER: _ClassVar[int]
MAX_CONTEXT_TOKENS_FIELD_NUMBER: _ClassVar[int]
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
ACCEPTED_COMPRESSION_FIELD_NUMBER: _ClassVar[int]
SUPPORTED_SCHEMA_VERSIONS_FIELD_NUMBER: _ClassVar[int]
VALIDATED_AT_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
fingerprint: Fingerprint
shard_range: ShardRange
backend: str
device: str
validated: bool
detail: str
max_concurrent_sessions: int
max_context_tokens: int
flow_control: FlowControl
accepted_compression: _containers.RepeatedScalarFieldContainer[Compression]
supported_schema_versions: _containers.RepeatedScalarFieldContainer[SchemaVersion]
validated_at_unix_nanos: int
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ..., shard_range: _Optional[_Union[ShardRange, _Mapping]] = ..., backend: _Optional[str] = ..., device: _Optional[str] = ..., validated: _Optional[bool] = ..., detail: _Optional[str] = ..., max_concurrent_sessions: _Optional[int] = ..., max_context_tokens: _Optional[int] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., accepted_compression: _Optional[_Iterable[_Union[Compression, str]]] = ..., supported_schema_versions: _Optional[_Iterable[_Union[SchemaVersion, str]]] = ..., validated_at_unix_nanos: _Optional[int] = ...) -> None: ...
class HealthRequest(_message.Message):
__slots__ = ("schema_version",)
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ...) -> None: ...
class HealthReport(_message.Message):
__slots__ = ("schema_version", "state", "active_sessions", "queued_chunks", "batch_occupancy", "kv_pressure", "resident_bytes", "detail")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
STATE_FIELD_NUMBER: _ClassVar[int]
ACTIVE_SESSIONS_FIELD_NUMBER: _ClassVar[int]
QUEUED_CHUNKS_FIELD_NUMBER: _ClassVar[int]
BATCH_OCCUPANCY_FIELD_NUMBER: _ClassVar[int]
KV_PRESSURE_FIELD_NUMBER: _ClassVar[int]
RESIDENT_BYTES_FIELD_NUMBER: _ClassVar[int]
DETAIL_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
state: ServingState
active_sessions: int
queued_chunks: int
batch_occupancy: int
kv_pressure: float
resident_bytes: int
detail: str
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., state: _Optional[_Union[ServingState, str]] = ..., active_sessions: _Optional[int] = ..., queued_chunks: _Optional[int] = ..., batch_occupancy: _Optional[int] = ..., kv_pressure: _Optional[float] = ..., resident_bytes: _Optional[int] = ..., detail: _Optional[str] = ...) -> None: ...
class ReleaseRequest(_message.Message):
__slots__ = ("schema_version", "route_session_id", "route_epoch")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
route_session_id: str
route_epoch: int
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ...) -> None: ...
class ReleaseResponse(_message.Message):
__slots__ = ("released", "error")
RELEASED_FIELD_NUMBER: _ClassVar[int]
ERROR_FIELD_NUMBER: _ClassVar[int]
released: bool
error: ShardError
def __init__(self, released: _Optional[bool] = ..., error: _Optional[_Union[ShardError, _Mapping]] = ...) -> None: ...
class CancelRequest(_message.Message):
__slots__ = ("schema_version", "route_session_id", "route_epoch", "work_id", "reason")
SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int]
ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int]
ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int]
WORK_ID_FIELD_NUMBER: _ClassVar[int]
REASON_FIELD_NUMBER: _ClassVar[int]
schema_version: SchemaVersion
route_session_id: str
route_epoch: int
work_id: str
reason: str
def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., work_id: _Optional[str] = ..., reason: _Optional[str] = ...) -> None: ...
class CancelResponse(_message.Message):
__slots__ = ("cancelled_work_items", "error")
CANCELLED_WORK_ITEMS_FIELD_NUMBER: _ClassVar[int]
ERROR_FIELD_NUMBER: _ClassVar[int]
cancelled_work_items: int
error: ShardError
def __init__(self, cancelled_work_items: _Optional[int] = ..., error: _Optional[_Union[ShardError, _Mapping]] = ...) -> None: ...

295
packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2_grpc.py Normal file
View File

@@ -0,0 +1,295 @@
# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
"""Client and server classes corresponding to protobuf-defined services."""
import grpc
import warnings
from . import shard_runtime_pb2 as shard__runtime__pb2
GRPC_GENERATED_VERSION = '1.82.1'
GRPC_VERSION = grpc.__version__
_version_not_supported = False
try:
from grpc._utilities import first_version_is_lower
_version_not_supported = first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION)
except ImportError:
_version_not_supported = True
if _version_not_supported:
raise RuntimeError(
f'The grpc package installed is at version {GRPC_VERSION},'
+ ' but the generated code in shard_runtime_pb2_grpc.py depends on'
+ f' grpcio>={GRPC_GENERATED_VERSION}.'
+ f' Please upgrade your grpc module to grpcio>={GRPC_GENERATED_VERSION}'
+ f' or downgrade your generated code using grpcio-tools<={GRPC_VERSION}.'
)
class ShardRuntimeStub:
"""---------------------------------------------------------------------------
Service
---------------------------------------------------------------------------
"""
def __init__(self, channel):
"""Constructor.
Args:
channel: A grpc.Channel.
"""
self.GetCapability = channel.unary_unary(
'/meshnet.shard.v1.ShardRuntime/GetCapability',
request_serializer=shard__runtime__pb2.CapabilityRequest.SerializeToString,
response_deserializer=shard__runtime__pb2.CapabilityReport.FromString,
_registered_method=True)
self.Health = channel.unary_unary(
'/meshnet.shard.v1.ShardRuntime/Health',
request_serializer=shard__runtime__pb2.HealthRequest.SerializeToString,
response_deserializer=shard__runtime__pb2.HealthReport.FromString,
_registered_method=True)
self.Session = channel.stream_stream(
'/meshnet.shard.v1.ShardRuntime/Session',
request_serializer=shard__runtime__pb2.SessionRequest.SerializeToString,
response_deserializer=shard__runtime__pb2.SessionResponse.FromString,
_registered_method=True)
self.Release = channel.unary_unary(
'/meshnet.shard.v1.ShardRuntime/Release',
request_serializer=shard__runtime__pb2.ReleaseRequest.SerializeToString,
response_deserializer=shard__runtime__pb2.ReleaseResponse.FromString,
_registered_method=True)
self.Cancel = channel.unary_unary(
'/meshnet.shard.v1.ShardRuntime/Cancel',
request_serializer=shard__runtime__pb2.CancelRequest.SerializeToString,
response_deserializer=shard__runtime__pb2.CancelResponse.FromString,
_registered_method=True)
class ShardRuntimeServicer:
"""---------------------------------------------------------------------------
Service
---------------------------------------------------------------------------
"""
def GetCapability(self, request, context):
"""What this worker can execute. Read before a route is built.
"""
context.set_code(grpc.StatusCode.UNIMPLEMENTED)
context.set_details('Method not implemented!')
raise NotImplementedError('Method not implemented!')
def Health(self, request, context):
"""Live load and serving state.
"""
context.set_code(grpc.StatusCode.UNIMPLEMENTED)
context.set_details('Method not implemented!')
raise NotImplementedError('Method not implemented!')
def Session(self, request_iterator, context):
"""One long-lived bidirectional stream per Route Session Activation Seam.
The stream opens with SessionOpen/SessionAccepted, then carries bounded
prefill chunks and decode steps in both directions for the life of the
session. Per-token channel creation is a non-goal: the handshake cost is
paid once and the hot path carries only what changes.
"""
context.set_code(grpc.StatusCode.UNIMPLEMENTED)
context.set_details('Method not implemented!')
raise NotImplementedError('Method not implemented!')
def Release(self, request, context):
"""Drop session state out of band. Idempotent.
"""
context.set_code(grpc.StatusCode.UNIMPLEMENTED)
context.set_details('Method not implemented!')
raise NotImplementedError('Method not implemented!')
def Cancel(self, request, context):
"""Cancel out of band, on a fresh call.
In-band CancelSignal is preferred, but a sender that is blocked on flow
control cannot write one — a cancel that can only travel down a wedged
stream is not a cancel. This RPC always has a path to the worker.
"""
context.set_code(grpc.StatusCode.UNIMPLEMENTED)
context.set_details('Method not implemented!')
raise NotImplementedError('Method not implemented!')
def add_ShardRuntimeServicer_to_server(servicer, server):
rpc_method_handlers = {
'GetCapability': grpc.unary_unary_rpc_method_handler(
servicer.GetCapability,
request_deserializer=shard__runtime__pb2.CapabilityRequest.FromString,
response_serializer=shard__runtime__pb2.CapabilityReport.SerializeToString,
),
'Health': grpc.unary_unary_rpc_method_handler(
servicer.Health,
request_deserializer=shard__runtime__pb2.HealthRequest.FromString,
response_serializer=shard__runtime__pb2.HealthReport.SerializeToString,
),
'Session': grpc.stream_stream_rpc_method_handler(
servicer.Session,
request_deserializer=shard__runtime__pb2.SessionRequest.FromString,
response_serializer=shard__runtime__pb2.SessionResponse.SerializeToString,
),
'Release': grpc.unary_unary_rpc_method_handler(
servicer.Release,
request_deserializer=shard__runtime__pb2.ReleaseRequest.FromString,
response_serializer=shard__runtime__pb2.ReleaseResponse.SerializeToString,
),
'Cancel': grpc.unary_unary_rpc_method_handler(
servicer.Cancel,
request_deserializer=shard__runtime__pb2.CancelRequest.FromString,
response_serializer=shard__runtime__pb2.CancelResponse.SerializeToString,
),
}
generic_handler = grpc.method_handlers_generic_handler(
'meshnet.shard.v1.ShardRuntime', rpc_method_handlers)
server.add_generic_rpc_handlers((generic_handler,))
server.add_registered_method_handlers('meshnet.shard.v1.ShardRuntime', rpc_method_handlers)
# This class is part of an EXPERIMENTAL API.
class ShardRuntime:
"""---------------------------------------------------------------------------
Service
---------------------------------------------------------------------------
"""
@staticmethod
def GetCapability(request,
target,
options=(),
channel_credentials=None,
call_credentials=None,
insecure=False,
compression=None,
wait_for_ready=None,
timeout=None,
metadata=None):
return grpc.experimental.unary_unary(
request,
target,
'/meshnet.shard.v1.ShardRuntime/GetCapability',
shard__runtime__pb2.CapabilityRequest.SerializeToString,
shard__runtime__pb2.CapabilityReport.FromString,
options,
channel_credentials,
insecure,
call_credentials,
compression,
wait_for_ready,
timeout,
metadata,
_registered_method=True)
@staticmethod
def Health(request,
target,
options=(),
channel_credentials=None,
call_credentials=None,
insecure=False,
compression=None,
wait_for_ready=None,
timeout=None,
metadata=None):
return grpc.experimental.unary_unary(
request,
target,
'/meshnet.shard.v1.ShardRuntime/Health',
shard__runtime__pb2.HealthRequest.SerializeToString,
shard__runtime__pb2.HealthReport.FromString,
options,
channel_credentials,
insecure,
call_credentials,
compression,
wait_for_ready,
timeout,
metadata,
_registered_method=True)
@staticmethod
def Session(request_iterator,
target,
options=(),
channel_credentials=None,
call_credentials=None,
insecure=False,
compression=None,
wait_for_ready=None,
timeout=None,
metadata=None):
return grpc.experimental.stream_stream(
request_iterator,
target,
'/meshnet.shard.v1.ShardRuntime/Session',
shard__runtime__pb2.SessionRequest.SerializeToString,
shard__runtime__pb2.SessionResponse.FromString,
options,
channel_credentials,
insecure,
call_credentials,
compression,
wait_for_ready,
timeout,
metadata,
_registered_method=True)
@staticmethod
def Release(request,
target,
options=(),
channel_credentials=None,
call_credentials=None,
insecure=False,
compression=None,
wait_for_ready=None,
timeout=None,
metadata=None):
return grpc.experimental.unary_unary(
request,
target,
'/meshnet.shard.v1.ShardRuntime/Release',
shard__runtime__pb2.ReleaseRequest.SerializeToString,
shard__runtime__pb2.ReleaseResponse.FromString,
options,
channel_credentials,
insecure,
call_credentials,
compression,
wait_for_ready,
timeout,
metadata,
_registered_method=True)
@staticmethod
def Cancel(request,
target,
options=(),
channel_credentials=None,
call_credentials=None,
insecure=False,
compression=None,
wait_for_ready=None,
timeout=None,
metadata=None):
return grpc.experimental.unary_unary(
request,
target,
'/meshnet.shard.v1.ShardRuntime/Cancel',
shard__runtime__pb2.CancelRequest.SerializeToString,
shard__runtime__pb2.CancelResponse.FromString,
options,
channel_credentials,
insecure,
call_credentials,
compression,
wait_for_ready,
timeout,
metadata,
_registered_method=True)

902
packages/node/meshnet_node/performance_contract.py Normal file
View File

@@ -0,0 +1,902 @@
"""The versioned safetensors-versus-GGUF performance contract.
The contract is the decision rule the native GGUF track is judged by, written
down *before* the numbers arrive and consumed later by the release gate
(DGR-014). Its thresholds are ratios against the Transformers/safetensors
reference recipe rather than absolute tokens/sec, because the absolute figure is
a property of whichever machine ran the benchmark and would have to be re-argued
on every host; a ratio is a claim about the runtime.
Three rules give the contract its teeth:
* **Thresholds are locked.** ``CONTRACT_SCHEMA_VERSION`` and ``locked_at``
travel with the document. Moving a threshold after seeing results is a new
contract version and a human decision, not a tweak.
* **Only like-for-like comparisons count.** A recipe measured on a different
device than the reference is marked non-comparable and is granted no benefit,
so a GPU-versus-CPU mismatch can never be laundered into a speed win.
* **Quantized recipes never claim numerical equivalence.** Quality is gated on
the near-lossless quality lane; the performance-fit lane is judged on speed,
memory and fit alone.
The verdict is one of ``promote``, ``optimize`` or ``stop`` — the three outcomes
the release gate is allowed to reach.
"""
from __future__ import annotations
import base64
import binascii
import hashlib
import json
from collections import Counter
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Mapping
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from .recipe_benchmark import Lane, REPORT_SCHEMA_VERSION
# Layout of the contract document understood by this reader.
CONTRACT_SCHEMA_VERSION = 1
PROVENANCE_SCHEMA_VERSION = 1
REAL_REPORT_PRODUCER = "meshnet_node.recipe_drivers.run_configured_benchmark/v1"
VERDICT_PROMOTE = "promote"
VERDICT_OPTIMIZE = "optimize"
VERDICT_STOP = "stop"
class PerformanceContractError(ValueError):
"""Raised when a contract is missing, malformed, or of an unsupported version."""
@dataclass(frozen=True)
class ContractThresholds:
"""The locked decision thresholds.
Every value is a ratio of a GGUF recipe's metric to the reference recipe's
metric on the same machine, same device, same plan.
A *meaningful speed benefit* means the GGUF recipe decodes at least 25%
faster for a single request without making time-to-first-token materially
worse, or sustains at least 25% more aggregate throughput under concurrency.
Either route is a real win for the product: one helps a single user, the
other helps a loaded node.
A *meaningful fit benefit* means peak resident memory (RSS plus VRAM) drops
by at least 25%. Fit is the product thesis — models larger than one
consumer node — so it is measured in resident bytes, not in how small the
file on disk is. Artifact size has its own reported threshold because a
smaller download is a real but secondary good.
25% is chosen to sit well clear of ordinary run-to-run variance on a busy
developer machine while still being a benefit a user would notice. A 5%
edge would not justify owning a native runtime and a patch stack.
"""
min_decode_speedup: float = 1.25
max_ttft_ratio: float = 1.25
min_aggregate_throughput_speedup: float = 1.25
max_resident_memory_ratio: float = 0.75
max_artifact_size_ratio: float = 0.60
min_quality_exact_match_rate: float = 0.90
min_quality_mean_similarity: float = 0.97
max_failure_rate: float = 0.0
def to_dict(self) -> dict:
return asdict(self)
@dataclass(frozen=True)
class PerformanceContract:
"""A locked, versioned contract plus the baseline it was locked against."""
contract_version: int
locked_at: str
locked_by: str
plan_id: str
thresholds: ContractThresholds
baseline: Mapping[str, Any]
stop_condition: str
notes: str = ""
schema_version: int = CONTRACT_SCHEMA_VERSION
def to_dict(self) -> dict:
return {
"schema_version": self.schema_version,
"contract_version": self.contract_version,
"locked_at": self.locked_at,
"locked_by": self.locked_by,
"plan_id": self.plan_id,
"thresholds": self.thresholds.to_dict(),
"baseline": dict(self.baseline),
"stop_condition": self.stop_condition,
"notes": self.notes,
}
STOP_CONDITION = (
"Stop the native llama.cpp/GGUF track when, on the same machine and device "
"as the Transformers/safetensors reference and under this plan, no "
"performance-fit GGUF recipe delivers either a meaningful speed benefit "
"(>=25% higher single-request decode tokens/sec without a >25% worse TTFT, "
"or >=25% higher aggregate throughput under concurrency) or a meaningful fit "
"benefit (>=25% lower peak resident memory), or when the near-lossless "
"quality lane fails, which indicates a broken runtime rather than a "
"quantization trade-off."
)
def _recipe_entries(report: Mapping[str, Any]) -> dict[str, Mapping[str, Any]]:
return {entry["recipe"]["id"]: entry for entry in report["recipes"]}
def _cell(entry: Mapping[str, Any], concurrency: int) -> Mapping[str, Any] | None:
return entry["concurrency"].get(str(concurrency))
def _resident_bytes(cell: Mapping[str, Any]) -> int:
return int(cell["peak_rss_bytes"]) + int(cell["peak_vram_bytes"])
def _ratio(value: float, reference: float) -> float:
"""Ratio guarded against a zero reference, which means "not measured"."""
if reference <= 0:
return 0.0
return round(value / reference, 4)
def _canonical_sha256(value: Any) -> str:
payload = json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def report_signing_payload(report: Mapping[str, Any]) -> bytes:
"""Canonical report bytes covered by the Ed25519 signature."""
provenance = report.get("provenance")
if not isinstance(provenance, Mapping):
raise PerformanceContractError("real benchmark report lacks signed provenance")
unsigned = dict(report)
unsigned_provenance = dict(provenance)
unsigned_provenance.pop("signature", None)
unsigned["provenance"] = unsigned_provenance
return json.dumps(
unsigned, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
def _decode_base64(value: Any, label: str) -> bytes:
if not isinstance(value, str):
raise PerformanceContractError(f"report lacks {label}")
try:
return base64.b64decode(value, validate=True)
except (binascii.Error, ValueError) as exc:
raise PerformanceContractError(f"report carries invalid {label}") from exc
def _verify_real_provenance(
contract: PerformanceContract, report: Mapping[str, Any]
) -> None:
provenance = report.get("provenance")
if not isinstance(provenance, Mapping):
raise PerformanceContractError("real benchmark report lacks signed provenance")
if provenance.get("schema_version") != PROVENANCE_SCHEMA_VERSION:
raise PerformanceContractError("report provenance schema is unsupported")
if provenance.get("producer") != REAL_REPORT_PRODUCER:
raise PerformanceContractError("report was not emitted by the canonical real runner")
if provenance.get("signature_algorithm") != "ed25519":
raise PerformanceContractError("report provenance is not Ed25519 signed")
for field in ("run_id", "started_at", "completed_at"):
if not provenance.get(field):
raise PerformanceContractError(f"report provenance lacks {field}")
required_config = contract.baseline.get("required_config_sha256")
if not required_config or provenance.get("config_sha256") != required_config:
raise PerformanceContractError("report config digest does not match the locked config")
encoded_public_key = contract.baseline.get("required_signer_public_key")
public_key_bytes = _decode_base64(encoded_public_key, "locked signer public key")
if len(public_key_bytes) != 32:
raise PerformanceContractError("locked Ed25519 public key must be 32 bytes")
expected_fingerprint = hashlib.sha256(public_key_bytes).hexdigest()
if provenance.get("signer_public_key_sha256") != expected_fingerprint:
raise PerformanceContractError("report signer fingerprint does not match the contract")
signature = _decode_base64(provenance.get("signature"), "Ed25519 signature")
try:
Ed25519PublicKey.from_public_bytes(public_key_bytes).verify(
signature, report_signing_payload(report)
)
except (InvalidSignature, ValueError) as exc:
raise PerformanceContractError("report Ed25519 signature verification failed") from exc
def _validate_report(contract: PerformanceContract, report: Mapping[str, Any]) -> None:
"""Fail closed when a report is not the experiment the contract locked."""
try:
schema_version = report["schema_version"]
evidence_class = report["evidence_class"]
plan = report["plan"]
recipes = report["recipes"]
reference_id = report["reference_recipe_id"]
host = report["host"]
except (KeyError, TypeError) as exc:
raise PerformanceContractError("benchmark report is missing required structure") from exc
if schema_version != REPORT_SCHEMA_VERSION:
raise PerformanceContractError(
f"report schema {schema_version!r} is not supported schema {REPORT_SCHEMA_VERSION}"
)
if plan.get("plan_id") != contract.plan_id:
raise PerformanceContractError(
f"report plan {plan.get('plan_id')!r} does not match locked plan {contract.plan_id!r}"
)
required_plan_sha256 = contract.baseline.get("required_plan_sha256")
measured_plan_sha256 = _canonical_sha256(plan)
if required_plan_sha256 and measured_plan_sha256 != required_plan_sha256:
raise PerformanceContractError(
f"report plan digest {measured_plan_sha256} does not match locked digest "
f"{required_plan_sha256}"
)
minimum_repeats = int(contract.baseline.get("minimum_repeats", 0))
minimum_prompts = int(contract.baseline.get("minimum_prompt_count", 0))
if int(plan.get("repeats", 0)) < minimum_repeats:
raise PerformanceContractError("report has too few repeats for the locked contract")
if len(plan.get("prompts", ())) < minimum_prompts:
raise PerformanceContractError("report has too few prompts for the locked contract")
minimum_output_tokens = int(contract.baseline.get("minimum_output_tokens", 0))
if int(plan.get("sampling", {}).get("max_output_tokens", 0)) < minimum_output_tokens:
raise PerformanceContractError("report output length is below the locked contract")
required_evidence = contract.baseline.get("required_evidence_class")
if required_evidence and evidence_class != required_evidence:
raise PerformanceContractError(
f"report evidence class {evidence_class!r} does not satisfy {required_evidence!r}"
)
if evidence_class in {"local-real", "multi-machine-real"}:
_verify_real_provenance(contract, report)
if required_evidence and (
not isinstance(host, Mapping)
or any(key not in host for key in ("hostname", "platform", "python", "cpu_count"))
):
raise PerformanceContractError("report lacks measured host provenance")
if not isinstance(recipes, list) or not recipes:
raise PerformanceContractError("report contains no recipes")
recipe_ids = [entry.get("recipe", {}).get("id") for entry in recipes]
if len(set(recipe_ids)) != len(recipe_ids) or None in recipe_ids:
raise PerformanceContractError("report recipe IDs must be present and unique")
required_recipes = set(contract.baseline.get("required_recipes", ()))
missing_recipes = required_recipes - set(recipe_ids)
if missing_recipes:
raise PerformanceContractError(
f"report is missing required recipes {sorted(missing_recipes)}"
)
if reference_id not in recipe_ids:
raise PerformanceContractError("report reference recipe is absent")
levels = {int(level) for level in plan.get("concurrency_levels", ())}
required_levels = {
int(level) for level in contract.baseline.get("required_concurrency_levels", ())
}
if not required_levels.issubset(levels):
raise PerformanceContractError(
f"report concurrency {sorted(levels)} lacks required levels {sorted(required_levels)}"
)
if not plan.get("prompts"):
raise PerformanceContractError("report plan contains no prompts")
model_id = plan.get("model_id")
model_revision = plan.get("model_revision")
required_device = contract.baseline.get("required_device")
required_artifacts = dict(contract.baseline.get("required_artifact_sha256") or {})
required_runtimes = dict(contract.baseline.get("required_recipe_runtime") or {})
required_backends = dict(contract.baseline.get("required_backend_detail") or {})
required_host = dict(contract.baseline.get("required_host_identity") or {})
for field, expected in required_host.items():
if host.get(field) != expected:
raise PerformanceContractError(
f"report host/runtime field {field!r} does not match the locked identity"
)
for entry in recipes:
recipe = entry.get("recipe", {})
recipe_id = recipe.get("id")
if required_device and recipe.get("device") != required_device:
raise PerformanceContractError(
f"recipe {recipe.get('id')!r} did not run on locked device {required_device!r}"
)
if required_evidence:
if recipe.get("source_model_id") != model_id:
raise PerformanceContractError("report mixes source model IDs")
if recipe.get("source_model_revision") != model_revision:
raise PerformanceContractError("report mixes source model revisions")
digest = recipe.get("artifact_sha256", "")
if not isinstance(digest, str) or len(digest) != 64:
raise PerformanceContractError("report lacks an artifact SHA-256 digest")
expected_digest = required_artifacts.get(recipe_id)
if not expected_digest or digest != expected_digest:
raise PerformanceContractError(
f"recipe {recipe_id!r} artifact digest does not match the contract"
)
expected_runtime = required_runtimes.get(recipe_id)
if not isinstance(expected_runtime, Mapping):
raise PerformanceContractError(
f"contract lacks runtime identity for recipe {recipe_id!r}"
)
actual_runtime = {
field: recipe.get(field) for field in expected_runtime
}
if actual_runtime != dict(expected_runtime):
raise PerformanceContractError(
f"recipe {recipe_id!r} runtime identity does not match the contract"
)
if entry.get("available"):
expected_backend = required_backends.get(recipe_id)
actual_backend = entry.get("load", {}).get("backend_detail")
if not expected_backend or actual_backend != expected_backend:
raise PerformanceContractError(
f"recipe {recipe_id!r} backend identity does not match the contract"
)
if entry.get("available"):
cells = entry.get("concurrency", {})
missing_cells = required_levels - {int(level) for level in cells}
if missing_cells:
raise PerformanceContractError(
f"recipe {recipe.get('id')!r} lacks concurrency cells {sorted(missing_cells)}"
)
reference = next(entry for entry in recipes if entry["recipe"]["id"] == reference_id)
if not reference.get("available"):
raise PerformanceContractError("reference recipe is unavailable")
reference_failures = sum(
int(cell.get("failures", 0)) for cell in reference.get("concurrency", {}).values()
)
if reference_failures:
raise PerformanceContractError("reference recipe contains failed requests")
def token_counts(entry: Mapping[str, Any]) -> dict[tuple[str, int, int], list[tuple[int, int]]]:
counts: dict[tuple[str, int, int], list[tuple[int, int]]] = {}
for outcome in entry.get("outcomes", ()):
if not outcome.get("ok"):
continue
key = (
str(outcome.get("prompt_id")),
int(outcome.get("concurrency", 0)),
int(outcome.get("repeat", -1)),
)
counts.setdefault(key, []).append(
(int(outcome.get("prompt_tokens", 0)), int(outcome.get("decode_tokens", 0)))
)
return {key: sorted(values) for key, values in counts.items()}
def outcome_counts(entry: Mapping[str, Any]) -> Counter[tuple[str, int, int]]:
return Counter(
(
str(outcome.get("prompt_id")),
int(outcome.get("concurrency", 0)),
int(outcome.get("repeat", -1)),
)
for outcome in entry.get("outcomes", ())
)
prompt_ids = {str(prompt["id"]) for prompt in plan["prompts"]}
repeats = int(plan["repeats"])
for entry in recipes:
if not entry.get("available"):
continue
recipe_id = str(entry["recipe"]["id"])
cells = entry.get("concurrency", {})
actual_levels = {int(level) for level in cells}
if actual_levels != levels:
raise PerformanceContractError(
f"recipe {recipe_id!r} has unexpected concurrency cells"
)
for outcome in entry.get("outcomes", ()):
try:
outcome_recipe_id = str(outcome["recipe_id"])
prompt_id = str(outcome["prompt_id"])
level = int(outcome["concurrency"])
repeat = int(outcome["repeat"])
except (KeyError, TypeError, ValueError) as exc:
raise PerformanceContractError(
f"recipe {recipe_id!r} contains a malformed raw outcome"
) from exc
if outcome_recipe_id != recipe_id:
raise PerformanceContractError(
f"recipe {recipe_id!r} contains an outcome for {outcome_recipe_id!r}"
)
if prompt_id not in prompt_ids or level not in levels or not 0 <= repeat < repeats:
raise PerformanceContractError(
f"recipe {recipe_id!r} contains an out-of-domain raw outcome"
)
if not isinstance(outcome.get("ok"), bool):
raise PerformanceContractError(
f"recipe {recipe_id!r} contains an outcome without boolean ok status"
)
coverage = outcome_counts(entry)
for prompt_id in prompt_ids:
for level in levels:
for repeat in range(repeats):
if coverage[(prompt_id, level, repeat)] != level:
raise PerformanceContractError(
f"recipe {recipe_id!r} lacks complete request coverage"
)
for level in levels:
cell = cells.get(str(level), cells.get(level))
raw = [
outcome
for outcome in entry.get("outcomes", ())
if int(outcome["concurrency"]) == level
]
if int(cell.get("concurrency", -1)) != level:
raise PerformanceContractError(
f"recipe {recipe_id!r} cell identity does not match concurrency {level}"
)
if int(cell.get("requests", -1)) != len(raw):
raise PerformanceContractError(
f"recipe {recipe_id!r} aggregate requests do not match raw outcomes"
)
raw_failures = sum(not outcome["ok"] for outcome in raw)
if int(cell.get("failures", -1)) != raw_failures:
raise PerformanceContractError(
f"recipe {recipe_id!r} aggregate failures do not match raw outcomes"
)
reference_counts = token_counts(reference)
for prompt_id in prompt_ids:
for level in levels:
for repeat in range(repeats):
values = reference_counts.get((prompt_id, level, repeat), ())
if len(values) != level:
raise PerformanceContractError(
"reference recipe lacks complete prompt/repeat/concurrency coverage"
)
if contract.thresholds.max_failure_rate != 0:
raise PerformanceContractError(
"nonzero failure tolerance requires an explicit failed-request token policy"
)
for entry in recipes:
if not entry.get("available"):
continue
for key, values in token_counts(entry).items():
if Counter(values) - Counter(reference_counts.get(key, ())):
raise PerformanceContractError(
f"recipe {entry['recipe']['id']!r} used different prompt/decode token counts"
)
@dataclass(frozen=True)
class RecipeEvaluation:
"""How one GGUF recipe fared against the reference under the contract."""
recipe_id: str
lane: str
comparable: bool
incomparable_reason: str
speed_benefit: bool
fit_benefit: bool
quality_pass: bool | None
failures: int
measurements: Mapping[str, Any]
reasons: tuple[str, ...]
def to_dict(self) -> dict:
data = asdict(self)
data["measurements"] = dict(self.measurements)
data["reasons"] = list(self.reasons)
return data
@dataclass(frozen=True)
class ContractEvaluation:
"""The release-gate answer: a verdict plus every reason behind it."""
contract_version: int
plan_id: str
verdict: str
quality_lane_pass: bool
speed_benefit: bool
fit_benefit: bool
stop_condition_met: bool
recipes: tuple[RecipeEvaluation, ...]
rationale: tuple[str, ...]
def to_dict(self) -> dict:
return {
"contract_version": self.contract_version,
"plan_id": self.plan_id,
"verdict": self.verdict,
"quality_lane_pass": self.quality_lane_pass,
"speed_benefit": self.speed_benefit,
"fit_benefit": self.fit_benefit,
"stop_condition_met": self.stop_condition_met,
"recipes": [recipe.to_dict() for recipe in self.recipes],
"rationale": list(self.rationale),
}
def _evaluate_recipe(
entry: Mapping[str, Any],
reference: Mapping[str, Any],
drift_by_recipe: Mapping[str, Mapping[str, Any]],
thresholds: ContractThresholds,
concurrency_levels: list[int],
expected_prompt_count: int,
) -> RecipeEvaluation:
recipe = entry["recipe"]
lane = recipe["lane"]
reasons: list[str] = []
if not entry["available"]:
return RecipeEvaluation(
recipe_id=recipe["id"], lane=lane, comparable=False,
incomparable_reason=entry["unavailable_reason"] or "recipe was not measured",
speed_benefit=False, fit_benefit=False, quality_pass=None, failures=0,
measurements={}, reasons=("recipe unavailable; no benefit granted",),
)
if recipe["device"] != reference["recipe"]["device"]:
return RecipeEvaluation(
recipe_id=recipe["id"], lane=lane, comparable=False,
incomparable_reason=(
f"recipe ran on device {recipe['device']!r} but the reference ran on "
f"{reference['recipe']['device']!r}; a cross-device ratio is not a runtime result"
),
speed_benefit=False, fit_benefit=False, quality_pass=None, failures=0,
measurements={}, reasons=("cross-device comparison; no benefit granted",),
)
single = _cell(entry, 1)
reference_single = _cell(reference, 1)
measurements: dict[str, Any] = {}
speed_benefit = False
if single and reference_single:
decode_speedup = _ratio(
single["decode_tokens_per_sec"], reference_single["decode_tokens_per_sec"]
)
ttft_ratio = _ratio(single["ttft_p50_ms"], reference_single["ttft_p50_ms"])
measurements["decode_speedup"] = decode_speedup
measurements["ttft_ratio"] = ttft_ratio
single_request_win = (
decode_speedup >= thresholds.min_decode_speedup
and 0 < ttft_ratio <= thresholds.max_ttft_ratio
)
if single_request_win:
speed_benefit = lane == Lane.PERFORMANCE_FIT.value
reasons.append(
f"single-request decode {decode_speedup:.2f}x reference "
f"(>= {thresholds.min_decode_speedup:.2f}x) at TTFT ratio {ttft_ratio:.2f}"
)
else:
reasons.append(
f"no single-request speed win: decode {decode_speedup:.2f}x, TTFT {ttft_ratio:.2f}x"
)
concurrent = [level for level in concurrency_levels if level > 1]
if concurrent:
top = max(concurrent)
cell, reference_cell = _cell(entry, top), _cell(reference, top)
if cell and reference_cell:
aggregate_speedup = _ratio(
cell["aggregate_decode_tokens_per_sec"],
reference_cell["aggregate_decode_tokens_per_sec"],
)
measurements["aggregate_throughput_speedup"] = aggregate_speedup
measurements["aggregate_concurrency"] = top
if aggregate_speedup >= thresholds.min_aggregate_throughput_speedup:
speed_benefit = lane == Lane.PERFORMANCE_FIT.value
reasons.append(
f"aggregate throughput at concurrency {top} is {aggregate_speedup:.2f}x reference "
f"(>= {thresholds.min_aggregate_throughput_speedup:.2f}x)"
)
else:
reasons.append(
f"no concurrency speed win: aggregate throughput at {top} is "
f"{aggregate_speedup:.2f}x reference"
)
fit_benefit = False
if single and reference_single:
resident_ratio = _ratio(_resident_bytes(single), _resident_bytes(reference_single))
artifact_ratio = _ratio(
entry["load"]["artifact_bytes"], reference["load"]["artifact_bytes"]
)
measurements["resident_memory_ratio"] = resident_ratio
measurements["artifact_size_ratio"] = artifact_ratio
if 0 < resident_ratio <= thresholds.max_resident_memory_ratio:
fit_benefit = lane == Lane.PERFORMANCE_FIT.value
reasons.append(
f"peak resident memory is {resident_ratio:.2f}x reference "
f"(<= {thresholds.max_resident_memory_ratio:.2f}x)"
)
else:
reasons.append(f"no fit win: peak resident memory is {resident_ratio:.2f}x reference")
measurements["artifact_size_win"] = (
0 < artifact_ratio <= thresholds.max_artifact_size_ratio
)
failures = sum(cell["failures"] for cell in entry["concurrency"].values())
requests = sum(cell["requests"] for cell in entry["concurrency"].values())
failure_rate = _ratio(failures, requests) if requests else 0.0
measurements["failure_rate"] = failure_rate
failure_limit_exceeded = requests == 0 or (
failures > 0
if thresholds.max_failure_rate == 0
else failures / requests > thresholds.max_failure_rate
)
if failure_limit_exceeded:
reasons.append(f"failure rate {failure_rate:.2%} exceeds the contract limit")
speed_benefit = False
fit_benefit = False
# Quality is a claim only the near-lossless lane is allowed to make. A
# quantized recipe's drift is recorded elsewhere and deliberately not read
# here: Q4 disagreeing with bf16 is the trade-off, not a failure.
quality_pass: bool | None = None
if lane == Lane.QUALITY.value:
drift = drift_by_recipe.get(recipe["id"])
if drift is None:
quality_pass = False
reasons.append("quality-lane recipe has no drift measurement against the reference")
else:
complete_coverage = drift.get("compared_prompts") == expected_prompt_count
quality_pass = (
complete_coverage
and failures == 0
and drift["exact_match_rate"] >= thresholds.min_quality_exact_match_rate
and drift["mean_similarity"] >= thresholds.min_quality_mean_similarity
)
measurements["compared_prompts"] = drift.get("compared_prompts", 0)
measurements["expected_prompts"] = expected_prompt_count
measurements["exact_match_rate"] = drift["exact_match_rate"]
measurements["mean_similarity"] = drift["mean_similarity"]
if not complete_coverage:
reasons.append(
f"quality lane compared {drift.get('compared_prompts', 0)} of "
f"{expected_prompt_count} required prompts"
)
if failures:
reasons.append("quality lane contains failed requests")
reasons.append(
f"quality lane exact-match {drift['exact_match_rate']:.2f} / similarity "
f"{drift['mean_similarity']:.3f} versus the reference "
f"({'pass' if quality_pass else 'fail'})"
)
return RecipeEvaluation(
recipe_id=recipe["id"], lane=lane, comparable=True, incomparable_reason="",
speed_benefit=speed_benefit, fit_benefit=fit_benefit, quality_pass=quality_pass,
failures=failures, measurements=measurements, reasons=tuple(reasons),
)
def evaluate_contract(
contract: PerformanceContract,
report: Mapping[str, Any],
) -> ContractEvaluation:
"""Judge a benchmark report against the locked contract.
Only performance-fit recipes can earn a speed or fit benefit; the quality
lane decides only whether the GGUF runtime is numerically sane. A GGUF
runtime that fails the quality lane is broken, and no amount of speed
redeems it, so the verdict is ``stop`` regardless of the other numbers.
"""
_validate_report(contract, report)
entries = _recipe_entries(report)
reference_id = report["reference_recipe_id"]
reference = entries.get(reference_id)
if reference is None:
raise PerformanceContractError(
f"report names reference recipe {reference_id!r}, which it does not contain"
)
drift_by_recipe = {entry["recipe_id"]: entry for entry in report["drift"]}
concurrency_levels = list(report["plan"]["concurrency_levels"])
evaluations = tuple(
_evaluate_recipe(
entry,
reference,
drift_by_recipe,
contract.thresholds,
concurrency_levels,
len(report["plan"]["prompts"]),
)
for recipe_id, entry in entries.items()
if recipe_id != reference_id
)
quality_lane = [
evaluation for evaluation in evaluations
if evaluation.lane == Lane.QUALITY.value and evaluation.comparable
]
quality_lane_pass = bool(quality_lane) and all(
evaluation.quality_pass for evaluation in quality_lane
)
performance_lane = [
evaluation for evaluation in evaluations
if evaluation.lane == Lane.PERFORMANCE_FIT.value
]
speed_benefit = any(evaluation.speed_benefit for evaluation in performance_lane)
fit_benefit = any(evaluation.fit_benefit for evaluation in performance_lane)
rationale: list[str] = []
if not quality_lane:
rationale.append(
"no comparable near-lossless GGUF recipe was measured, so the runtime's "
"numerical correctness is unproven"
)
elif not quality_lane_pass:
rationale.append(
"the near-lossless quality lane failed: the GGUF runtime disagrees with the "
"safetensors reference beyond what near-lossless weights can explain"
)
else:
rationale.append("the near-lossless quality lane passed against the safetensors reference")
rationale.append(
"a meaningful speed benefit was measured" if speed_benefit
else "no performance-fit recipe delivered a meaningful speed benefit"
)
rationale.append(
"a meaningful fit benefit was measured" if fit_benefit
else "no performance-fit recipe delivered a meaningful fit benefit"
)
stop_condition_met = not quality_lane_pass or not (speed_benefit or fit_benefit)
if stop_condition_met:
verdict = VERDICT_STOP
elif speed_benefit and fit_benefit:
verdict = VERDICT_PROMOTE
else:
verdict = VERDICT_OPTIMIZE
rationale.append(
"exactly one of speed or fit cleared the contract: the benefit is real but partial, "
"so the measured bottleneck needs a bounded optimization task before promotion"
)
return ContractEvaluation(
contract_version=contract.contract_version,
plan_id=contract.plan_id,
verdict=verdict,
quality_lane_pass=quality_lane_pass,
speed_benefit=speed_benefit,
fit_benefit=fit_benefit,
stop_condition_met=stop_condition_met,
recipes=evaluations,
rationale=tuple(rationale),
)
def parse_contract(data: Any, source: str = "<memory>") -> PerformanceContract:
"""Validate an already-decoded contract document."""
if not isinstance(data, Mapping):
raise PerformanceContractError(f"contract root in {source} must be a JSON object")
schema_version = data.get("schema_version")
if not isinstance(schema_version, int) or isinstance(schema_version, bool):
raise PerformanceContractError(f"'schema_version' in {source} must be an integer")
if schema_version != CONTRACT_SCHEMA_VERSION:
raise PerformanceContractError(
f"{source} declares contract schema version {schema_version}, but this node reads "
f"version {CONTRACT_SCHEMA_VERSION}; upgrade the node or use a supported contract"
)
for required in ("contract_version", "locked_at", "locked_by", "plan_id", "stop_condition"):
if not data.get(required):
raise PerformanceContractError(f"{source} is missing {required!r}")
raw_thresholds = data.get("thresholds")
if not isinstance(raw_thresholds, Mapping):
raise PerformanceContractError(f"'thresholds' in {source} must be a JSON object")
known = set(ContractThresholds().to_dict())
unknown = set(raw_thresholds) - known
missing = known - set(raw_thresholds)
if unknown or missing:
raise PerformanceContractError(
f"{source} threshold keys differ from v1; unknown={sorted(unknown)}, "
f"missing={sorted(missing)}"
)
thresholds = ContractThresholds(**{
key: float(value) for key, value in raw_thresholds.items()
})
contract_version = int(data["contract_version"])
if contract_version != 1 or thresholds != ContractThresholds():
raise PerformanceContractError(
f"{source} changes immutable v1 thresholds without a supported contract version"
)
if str(data["stop_condition"]) != STOP_CONDITION:
raise PerformanceContractError(
f"{source} stop condition differs from executable v1 semantics"
)
return PerformanceContract(
contract_version=contract_version,
locked_at=str(data["locked_at"]),
locked_by=str(data["locked_by"]),
plan_id=str(data["plan_id"]),
thresholds=thresholds,
baseline=dict(data.get("baseline") or {}),
stop_condition=str(data["stop_condition"]),
notes=str(data.get("notes", "")),
schema_version=schema_version,
)
def load_contract(path: Path) -> PerformanceContract:
"""Load and validate the contract at ``path``."""
try:
raw = path.read_text(encoding="utf-8")
except OSError as exc:
raise PerformanceContractError(
f"cannot read performance contract {path}: {exc.strerror or exc}"
) from exc
try:
data = json.loads(raw)
except json.JSONDecodeError as exc:
raise PerformanceContractError(
f"{path} is not valid JSON: {exc.msg} at line {exc.lineno} column {exc.colno}"
) from exc
return parse_contract(data, source=str(path))
def baseline_from_report(report: Mapping[str, Any]) -> dict[str, Any]:
"""Distil the reference numbers a later gate needs to compare against."""
entries = _recipe_entries(report)
baseline: dict[str, Any] = {
"evidence_class": report["evidence_class"],
"model_id": report["plan"]["model_id"],
"model_revision": report["plan"]["model_revision"],
"plan_sha256": _canonical_sha256(report["plan"]),
"reference_recipe_id": report["reference_recipe_id"],
"host": report["host"],
"provenance": dict(report.get("provenance") or {}),
"artifact_sha256": {
recipe_id: entry["recipe"]["artifact_sha256"]
for recipe_id, entry in entries.items()
},
"recipe_runtime": {
recipe_id: {
field: entry["recipe"].get(field)
for field in ("runtime", "weight_format", "weight_quantization", "device")
}
for recipe_id, entry in entries.items()
},
"backend_detail": {
recipe_id: entry.get("load", {}).get("backend_detail")
for recipe_id, entry in entries.items()
if entry.get("available")
},
"recipes": {},
}
for recipe_id, entry in entries.items():
if not entry["available"]:
baseline["recipes"][recipe_id] = {"available": False,
"reason": entry["unavailable_reason"]}
continue
baseline["recipes"][recipe_id] = {
"available": True,
"lane": entry["recipe"]["lane"],
"device": entry["recipe"]["device"],
"artifact_bytes": entry["load"]["artifact_bytes"],
"concurrency": {
level: {
"ttft_p50_ms": cell["ttft_p50_ms"],
"ttft_p95_ms": cell["ttft_p95_ms"],
"latency_p50_ms": cell["latency_p50_ms"],
"latency_p95_ms": cell["latency_p95_ms"],
"prefill_tokens_per_sec": cell["prefill_tokens_per_sec"],
"decode_tokens_per_sec": cell["decode_tokens_per_sec"],
"aggregate_decode_tokens_per_sec": cell["aggregate_decode_tokens_per_sec"],
"peak_rss_bytes": cell["peak_rss_bytes"],
"peak_vram_bytes": cell["peak_vram_bytes"],
"failures": cell["failures"],
}
for level, cell in entry["concurrency"].items()
},
}
return baseline

694
packages/node/meshnet_node/recipe_benchmark.py Normal file
View File

@@ -0,0 +1,694 @@
"""Controlled safetensors-versus-GGUF recipe benchmark.
This is a *model recipe* benchmark, unlike
:mod:`meshnet_node.route_session_benchmark`, which is a transport harness. It
answers one question: on one machine, with one model revision and one fixed
workload, what do the Transformers/safetensors recipe and the whole-model
llama.cpp/GGUF recipes actually cost in speed, memory, fit, and output drift?
Two ideas keep the answer honest.
**Lanes.** A recipe belongs to exactly one :class:`Lane`. The quality lane
holds near-lossless recipes (bf16/f16 weights) whose outputs may legitimately be
compared for numerical agreement. The performance-fit lane holds quantized
recipes (Q8_0, Q4_K_M, ...). Quantized recipes are judged on speed, memory, and
artifact size only; their drift is *reported* but never read as evidence that
Q4 and bf16 are numerically equivalent, because they are not. The lane is a
property of the recipe, so nothing downstream can quietly cross the boundary.
**Drivers.** The measurement core here is pure and runtime-free: it drives a
:class:`RecipeDriver` and computes metrics. Real runtimes live in
:mod:`meshnet_node.recipe_drivers` and are imported only on demand, which keeps
the default test suite deterministic, GPU-free and model-download-free while the
same code path produces the real evidence.
"""
from __future__ import annotations
import argparse
import json
import statistics
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from dataclasses import asdict, dataclass, field
from difflib import SequenceMatcher
from enum import Enum
from pathlib import Path
from typing import Any, Mapping, Protocol, Sequence
# Layout of the report document produced by :func:`build_report`.
REPORT_SCHEMA_VERSION = 1
class Lane(str, Enum):
"""Why a recipe is being measured at all.
``QUALITY`` recipes carry near-lossless weights, so comparing their output
with the reference recipe is a meaningful correctness signal.
``PERFORMANCE_FIT`` recipes carry quantized weights: they exist to be faster
or to fit, and their drift is descriptive, never a pass/fail equivalence
claim.
"""
QUALITY = "quality"
PERFORMANCE_FIT = "performance-fit"
class BenchmarkError(RuntimeError):
"""Raised when a benchmark cannot be run as specified."""
@dataclass(frozen=True)
class SamplingPolicy:
"""The sampling policy every recipe must be given, identically.
Greedy by default: sampling noise would otherwise be indistinguishable from
quantization drift, and the whole point of the quality lane is to tell those
two apart.
"""
temperature: float = 0.0
top_p: float = 1.0
top_k: int = 1
seed: int = 1234
max_output_tokens: int = 64
def to_dict(self) -> dict:
return asdict(self)
@dataclass(frozen=True)
class PromptSpec:
"""One fixed prompt, tagged with the context length it is meant to exercise."""
id: str
text: str
context_class: str = "short"
def to_dict(self) -> dict:
return asdict(self)
@dataclass(frozen=True)
class BenchmarkPlan:
"""The controlled variables: identical for every recipe in a report.
A plan is the experiment. If two recipes were measured under different
plans, their numbers are not comparable and the report must not pretend they
are, so the plan is recorded once at the top of the document rather than
per-recipe.
"""
plan_id: str
model_id: str
model_revision: str
prompts: tuple[PromptSpec, ...]
sampling: SamplingPolicy = SamplingPolicy()
concurrency_levels: tuple[int, ...] = (1, 4)
repeats: int = 1
warmup_requests: int = 1
def __post_init__(self) -> None:
if not self.prompts:
raise BenchmarkError("a benchmark plan needs at least one prompt")
if not self.concurrency_levels or any(level < 1 for level in self.concurrency_levels):
raise BenchmarkError("concurrency levels must all be >= 1")
if self.repeats < 1:
raise BenchmarkError("repeats must be >= 1")
if 1 not in self.concurrency_levels or 4 not in self.concurrency_levels:
raise BenchmarkError("a controlled baseline must include concurrency levels 1 and 4")
def to_dict(self) -> dict:
return {
"plan_id": self.plan_id,
"model_id": self.model_id,
"model_revision": self.model_revision,
"prompts": [prompt.to_dict() for prompt in self.prompts],
"sampling": self.sampling.to_dict(),
"concurrency_levels": list(self.concurrency_levels),
"repeats": self.repeats,
"warmup_requests": self.warmup_requests,
}
@dataclass(frozen=True)
class RecipeSpec:
"""One runtime recipe under test.
``is_reference`` marks the single recipe every other recipe's output drift is
measured against — the current Transformers/safetensors route, which
decision Gate 8 keeps as the correctness backend.
"""
id: str
runtime: str
weight_format: str
weight_quantization: str
lane: Lane
device: str
artifact_path: str = ""
source_model_id: str = ""
source_model_revision: str = ""
artifact_sha256: str = ""
is_reference: bool = False
notes: str = ""
def to_dict(self) -> dict:
data = asdict(self)
data["lane"] = self.lane.value
return data
@dataclass(frozen=True)
class LoadStats:
"""What loading the recipe cost, before any token is generated."""
artifact_bytes: int
load_ms: float
rss_bytes: int = 0
vram_bytes: int = 0
backend_detail: str = ""
def to_dict(self) -> dict:
return asdict(self)
@dataclass(frozen=True)
class GenerationSample:
"""One completed generation as reported by a driver.
``prefill_ms``/``decode_ms`` are the runtime's own split where it exposes one
(llama.cpp does); drivers that cannot split honestly report ``prefill_ms`` as
the time to the first token. ``queue_wait_ms`` separates time spent waiting
for a runtime slot from time spent computing, so a concurrency-4 TTFT is not
silently read as a slower prefill.
"""
text: str
prompt_tokens: int
decode_tokens: int
ttft_ms: float
prefill_ms: float
decode_ms: float
total_ms: float
queue_wait_ms: float = 0.0
class RecipeDriver(Protocol):
"""The seam every runtime implements; the measurement core knows nothing else."""
def load(self) -> LoadStats:
"""Load the artifact and return its cost."""
def generate(self, prompt: str, sampling: SamplingPolicy) -> GenerationSample:
"""Run one complete generation under the given sampling policy."""
def memory_probe(self) -> tuple[int, int]:
"""Return ``(rss_bytes, vram_bytes)`` observed right now."""
def close(self) -> None:
"""Release the runtime."""
@dataclass(frozen=True)
class RequestOutcome:
"""One request attempt, successful or not.
A failure is a first-class result, not an exception that aborts the run: a
recipe that cannot sustain concurrency 4 has told us something, and the
report must carry it rather than lose it.
"""
recipe_id: str
concurrency: int
prompt_id: str
repeat: int
ok: bool
latency_ms: float
ttft_ms: float = 0.0
prefill_ms: float = 0.0
decode_ms: float = 0.0
queue_wait_ms: float = 0.0
prompt_tokens: int = 0
decode_tokens: int = 0
text: str = ""
error: str = ""
def to_dict(self) -> dict:
return asdict(self)
def _percentile(values: Sequence[float], percentile: float) -> float:
"""Nearest-rank percentile; 0.0 for an empty sample."""
ordered = sorted(values)
if not ordered:
return 0.0
rank = max(1, -(-len(ordered) * percentile // 100))
return round(ordered[int(rank) - 1], 4)
def _mean(values: Sequence[float]) -> float:
return round(statistics.fmean(values), 4) if values else 0.0
@dataclass(frozen=True)
class ConcurrencyMetrics:
"""Aggregate metrics for one recipe at one concurrency level."""
concurrency: int
requests: int
failures: int
wall_ms: float
ttft_p50_ms: float
ttft_p95_ms: float
latency_p50_ms: float
latency_p95_ms: float
prefill_tokens_per_sec: float
decode_tokens_per_sec: float
aggregate_decode_tokens_per_sec: float
peak_rss_bytes: int
peak_vram_bytes: int
failure_reasons: tuple[str, ...] = ()
def to_dict(self) -> dict:
data = asdict(self)
data["failure_reasons"] = list(self.failure_reasons)
return data
def summarize_concurrency(
outcomes: Sequence[RequestOutcome],
*,
concurrency: int,
wall_ms: float,
peak_rss_bytes: int,
peak_vram_bytes: int,
) -> ConcurrencyMetrics:
"""Aggregate one recipe/concurrency cell.
Per-request rates are averaged over successful requests; aggregate
throughput is total decoded tokens over the wall clock of the whole cell,
which is the only figure that credits a runtime for overlapping work.
"""
ok = [outcome for outcome in outcomes if outcome.ok]
failures = [outcome for outcome in outcomes if not outcome.ok]
decode_tokens = sum(outcome.decode_tokens for outcome in ok)
prefill_rates = [
outcome.prompt_tokens / (outcome.prefill_ms / 1000)
for outcome in ok
if outcome.prefill_ms > 0 and outcome.prompt_tokens
]
decode_rates = [
outcome.decode_tokens / (outcome.decode_ms / 1000)
for outcome in ok
if outcome.decode_ms > 0 and outcome.decode_tokens
]
return ConcurrencyMetrics(
concurrency=concurrency,
requests=len(outcomes),
failures=len(failures),
wall_ms=round(wall_ms, 4),
ttft_p50_ms=_percentile([outcome.ttft_ms for outcome in ok], 50),
ttft_p95_ms=_percentile([outcome.ttft_ms for outcome in ok], 95),
latency_p50_ms=_percentile([outcome.latency_ms for outcome in ok], 50),
latency_p95_ms=_percentile([outcome.latency_ms for outcome in ok], 95),
prefill_tokens_per_sec=_mean(prefill_rates),
decode_tokens_per_sec=_mean(decode_rates),
aggregate_decode_tokens_per_sec=round(decode_tokens / max(1e-6, wall_ms / 1000), 4),
peak_rss_bytes=peak_rss_bytes,
peak_vram_bytes=peak_vram_bytes,
failure_reasons=tuple(sorted({outcome.error for outcome in failures if outcome.error})),
)
@dataclass
class RecipeMeasurement:
"""Everything measured for one recipe across every concurrency level."""
recipe: RecipeSpec
load: LoadStats
metrics: dict[int, ConcurrencyMetrics] = field(default_factory=dict)
outcomes: list[RequestOutcome] = field(default_factory=list)
unavailable_reason: str = ""
@property
def available(self) -> bool:
return not self.unavailable_reason
def outputs_by_prompt(self) -> dict[str, str]:
"""First successful output per prompt, at the lowest concurrency measured.
Drift is a property of the recipe, not of load: concurrency must not
change greedy output, so the least-contended sample is the fair one.
"""
best: dict[str, tuple[int, str]] = {}
for outcome in self.outcomes:
if not outcome.ok:
continue
seen = best.get(outcome.prompt_id)
if seen is None or outcome.concurrency < seen[0]:
best[outcome.prompt_id] = (outcome.concurrency, outcome.text)
return {prompt_id: text for prompt_id, (_, text) in best.items()}
def to_dict(self) -> dict:
return {
"recipe": self.recipe.to_dict(),
"available": self.available,
"unavailable_reason": self.unavailable_reason,
"load": self.load.to_dict(),
"concurrency": {
str(level): metrics.to_dict() for level, metrics in sorted(self.metrics.items())
},
"outcomes": [outcome.to_dict() for outcome in self.outcomes],
}
@dataclass(frozen=True)
class DriftReport:
"""Output drift of one recipe against the reference recipe.
``advisory`` is true for every performance-fit recipe: the number is
published, but a Q4 recipe disagreeing with bf16 is expected behaviour, not a
defect, and no gate may read it as one.
"""
recipe_id: str
lane: Lane
reference_id: str
compared_prompts: int
exact_match_rate: float
mean_similarity: float
advisory: bool
per_prompt: tuple[dict[str, Any], ...] = ()
def to_dict(self) -> dict:
return {
"recipe_id": self.recipe_id,
"lane": self.lane.value,
"reference_id": self.reference_id,
"compared_prompts": self.compared_prompts,
"exact_match_rate": self.exact_match_rate,
"mean_similarity": self.mean_similarity,
"advisory": self.advisory,
"per_prompt": list(self.per_prompt),
}
def _first_divergence(left: str, right: str) -> int:
"""Index of the first differing character, or -1 when the strings agree."""
for index, (a, b) in enumerate(zip(left, right)):
if a != b:
return index
return -1 if len(left) == len(right) else min(len(left), len(right))
def compute_drift(
measurement: RecipeMeasurement,
reference: RecipeMeasurement,
) -> DriftReport:
"""Compare one recipe's greedy outputs with the reference recipe's."""
reference_outputs = reference.outputs_by_prompt()
outputs = measurement.outputs_by_prompt()
shared = sorted(set(outputs) & set(reference_outputs))
per_prompt: list[dict[str, Any]] = []
exact = 0
similarities: list[float] = []
for prompt_id in shared:
got, want = outputs[prompt_id], reference_outputs[prompt_id]
matches = got == want
exact += matches
similarity = round(SequenceMatcher(None, want, got).ratio(), 4)
similarities.append(similarity)
per_prompt.append({
"prompt_id": prompt_id,
"exact_match": matches,
"similarity": similarity,
"first_divergence_char": _first_divergence(want, got),
"reference_text": want,
"recipe_text": got,
})
return DriftReport(
recipe_id=measurement.recipe.id,
lane=measurement.recipe.lane,
reference_id=reference.recipe.id,
compared_prompts=len(shared),
exact_match_rate=round(exact / len(shared), 4) if shared else 0.0,
mean_similarity=_mean(similarities),
advisory=measurement.recipe.lane is Lane.PERFORMANCE_FIT,
per_prompt=tuple(per_prompt),
)
class _PeakMemory:
"""Continuously sample a driver's memory while requests are in flight."""
def __init__(self, driver: RecipeDriver) -> None:
self._driver = driver
self.peak_rss = 0
self.peak_vram = 0
def sample(self) -> None:
try:
rss, vram = self._driver.memory_probe()
except Exception: # a probe must never fail a benchmark run
return
self.peak_rss = max(self.peak_rss, rss)
self.peak_vram = max(self.peak_vram, vram)
def sample_until(self, stop: threading.Event, interval_s: float = 0.01) -> None:
"""Sample until ``stop`` is set, including one final post-request probe."""
while not stop.wait(interval_s):
self.sample()
self.sample()
def _run_request(
driver: RecipeDriver,
recipe: RecipeSpec,
prompt: PromptSpec,
sampling: SamplingPolicy,
concurrency: int,
repeat: int,
memory: _PeakMemory,
) -> RequestOutcome:
started = time.monotonic()
try:
sample = driver.generate(prompt.text, sampling)
except Exception as exc: # a failed request is data, not a crashed benchmark
return RequestOutcome(
recipe_id=recipe.id,
concurrency=concurrency,
prompt_id=prompt.id,
repeat=repeat,
ok=False,
latency_ms=round((time.monotonic() - started) * 1000, 4),
error=f"{type(exc).__name__}: {exc}",
)
finally:
memory.sample()
return RequestOutcome(
recipe_id=recipe.id,
concurrency=concurrency,
prompt_id=prompt.id,
repeat=repeat,
ok=True,
latency_ms=round(sample.total_ms, 4),
ttft_ms=round(sample.ttft_ms, 4),
prefill_ms=round(sample.prefill_ms, 4),
decode_ms=round(sample.decode_ms, 4),
queue_wait_ms=round(sample.queue_wait_ms, 4),
prompt_tokens=sample.prompt_tokens,
decode_tokens=sample.decode_tokens,
text=sample.text,
)
def measure_recipe(
driver: RecipeDriver,
recipe: RecipeSpec,
plan: BenchmarkPlan,
) -> RecipeMeasurement:
"""Load one recipe and run the whole plan against it.
The driver is closed exactly once, whatever happens, so a recipe that dies at
concurrency 4 still releases its weights before the next recipe loads.
"""
load = driver.load()
measurement = RecipeMeasurement(recipe=recipe, load=load)
try:
for _ in range(plan.warmup_requests):
try:
driver.generate(plan.prompts[0].text, plan.sampling)
except Exception: # a failing warmup is reported by the real requests
break
for concurrency in plan.concurrency_levels:
# Measure each cell independently and sample while requests are in
# flight; post-request probes alone miss transient KV/workspace peaks.
memory = _PeakMemory(driver)
memory.sample()
stop_sampling = threading.Event()
sampler = threading.Thread(
target=memory.sample_until,
args=(stop_sampling,),
name=f"recipe-memory-{recipe.id}-c{concurrency}",
daemon=True,
)
requests = [
(prompt, repeat)
for repeat in range(plan.repeats)
for prompt in plan.prompts
for _ in range(concurrency)
]
started = time.monotonic()
sampler.start()
try:
with ThreadPoolExecutor(max_workers=concurrency) as pool:
outcomes = list(pool.map(
lambda item: _run_request(
driver, recipe, item[0], plan.sampling, concurrency, item[1], memory
),
requests,
))
finally:
stop_sampling.set()
sampler.join()
wall_ms = (time.monotonic() - started) * 1000
measurement.outcomes.extend(outcomes)
measurement.metrics[concurrency] = summarize_concurrency(
outcomes,
concurrency=concurrency,
wall_ms=wall_ms,
peak_rss_bytes=memory.peak_rss,
peak_vram_bytes=memory.peak_vram,
)
finally:
driver.close()
return measurement
def build_report(
plan: BenchmarkPlan,
measurements: Sequence[RecipeMeasurement],
*,
host: dict[str, Any],
evidence_class: str,
provenance: Mapping[str, Any] | None = None,
) -> dict:
"""Assemble the machine-readable benchmark document.
``evidence_class`` is one of ``synthetic``, ``local-real`` or
``multi-machine-real`` and is never inferred: a report that cannot say how it
was produced cannot be trusted by a release gate.
"""
if evidence_class not in {"synthetic", "local-real", "multi-machine-real"}:
raise BenchmarkError(f"unknown evidence class {evidence_class!r}")
if evidence_class != "synthetic" and not isinstance(provenance, Mapping):
raise BenchmarkError("non-synthetic reports require canonical signed provenance")
references = [m for m in measurements if m.recipe.is_reference]
if len(references) != 1:
raise BenchmarkError(
f"exactly one reference recipe is required, got {len(references)}"
)
reference = references[0]
if reference.recipe.lane is not Lane.QUALITY:
raise BenchmarkError("the reference recipe must sit in the quality lane")
drift = [
compute_drift(measurement, reference).to_dict()
for measurement in measurements
if measurement is not reference and measurement.available
]
report = {
"schema_version": REPORT_SCHEMA_VERSION,
"evidence_class": evidence_class,
"plan": plan.to_dict(),
"host": host,
"reference_recipe_id": reference.recipe.id,
"recipes": [measurement.to_dict() for measurement in measurements],
"drift": drift,
}
if provenance is not None:
report["provenance"] = dict(provenance)
return report
def format_summary(report: dict) -> str:
"""Render the human-readable companion to the JSON artifact."""
plan = report["plan"]
lines = [
f"Recipe benchmark {plan['plan_id']} ({report['evidence_class']})",
f"model {plan['model_id']}@{plan['model_revision']}",
]
for entry in report["recipes"]:
recipe = entry["recipe"]
if not entry["available"]:
lines.append(f"{recipe['id']:38} UNAVAILABLE: {entry['unavailable_reason']}")
continue
artifact_gb = entry["load"]["artifact_bytes"] / 1e9
for level, metrics in entry["concurrency"].items():
lines.append(
f"{recipe['id']:38} [{recipe['lane']:16}] c={level:>2} "
f"ttft p50/p95 {metrics['ttft_p50_ms']:8.1f}/{metrics['ttft_p95_ms']:8.1f} ms; "
f"prefill {metrics['prefill_tokens_per_sec']:7.1f} tok/s; "
f"decode {metrics['decode_tokens_per_sec']:6.1f} tok/s; "
f"aggregate {metrics['aggregate_decode_tokens_per_sec']:7.1f} tok/s; "
f"rss {metrics['peak_rss_bytes'] / 1e9:5.2f} GB; "
f"vram {metrics['peak_vram_bytes'] / 1e9:5.2f} GB; "
f"artifact {artifact_gb:5.2f} GB; failures {metrics['failures']}"
)
for entry in report["drift"]:
tag = "advisory" if entry["advisory"] else "gated"
lines.append(
f"drift {entry['recipe_id']:32} vs {entry['reference_id']:28} "
f"exact {entry['exact_match_rate']:.2f}; similarity {entry['mean_similarity']:.3f} ({tag})"
)
return "\n".join(lines)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Run the controlled safetensors-versus-GGUF recipe benchmark"
)
parser.add_argument("--config", type=Path, required=True, help="benchmark configuration JSON")
parser.add_argument(
"--profile",
choices=("contract-v1", "gpu-diagnostic"),
default="contract-v1",
help="validation and provenance profile (GPU diagnostics are not v1-eligible)",
)
parser.add_argument("--json-out", type=Path, help="write the JSON report to this path")
parser.add_argument("--summary-out", type=Path, help="write the text summary to this path")
args = parser.parse_args(argv)
from .recipe_drivers import ( # heavy runtimes: import on demand
run_configured_benchmark,
run_configured_gpu_diagnostic,
)
runner = (
run_configured_gpu_diagnostic
if args.profile == "gpu-diagnostic"
else run_configured_benchmark
)
report = runner(json.loads(args.config.read_text(encoding="utf-8")))
summary = format_summary(report)
if args.json_out:
args.json_out.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n", encoding="utf-8")
if args.summary_out:
args.summary_out.write_text(summary + "\n", encoding="utf-8")
print(summary)
return 0
if __name__ == "__main__": # pragma: no cover - CLI entry point
raise SystemExit(main())

855
packages/node/meshnet_node/recipe_drivers.py Normal file
View File

@@ -0,0 +1,855 @@
"""Real runtime drivers for the recipe benchmark.
This module is the only place that imports torch, transformers, or spawns a
llama.cpp server, and :mod:`meshnet_node.recipe_benchmark` imports it lazily.
That keeps the default test suite deterministic, GPU-free and download-free
while the real evidence runs through exactly the same measurement core.
Fairness is the whole point of a baseline, so both drivers are held to the same
rules:
* They are handed a **pre-formatted prompt string**. Neither applies a chat
template, because a template applied twice — or differently — by two runtimes
would show up as a speed and drift difference that has nothing to do with the
runtime.
* They are given the **same CPU thread budget**, so the comparison measures
kernels rather than how many cores each runtime felt entitled to take.
* They report the runtime's **own prefill/decode split** where it has one, and
say so honestly where it does not.
"""
from __future__ import annotations
import base64
import hashlib
import hmac
import json
import os
import platform
import re
import socket
import stat
import subprocess
import sys
import tempfile
import time
import urllib.error
import urllib.request
import uuid
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from .performance_contract import (
PROVENANCE_SCHEMA_VERSION,
REAL_REPORT_PRODUCER,
_canonical_sha256,
report_signing_payload,
)
from .recipe_benchmark import (
BenchmarkError,
BenchmarkPlan,
GenerationSample,
Lane,
LoadStats,
PromptSpec,
RecipeSpec,
SamplingPolicy,
build_report,
measure_recipe,
)
REAL_INFERENCE_ENV = "MESHNET_ENABLE_REAL_INFERENCE_TESTS"
EVIDENCE_SIGNING_KEY_ENV = "MESHNET_EVIDENCE_SIGNING_KEY"
CONTRACT_V1_PROFILE = "contract-v1"
GPU_DIAGNOSTIC_PROFILE = "gpu-diagnostic"
GPU_DIAGNOSTIC_REPORT_PRODUCER = "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1"
def real_inference_enabled() -> bool:
"""Real runtimes stay off unless the operator opts in explicitly."""
return os.environ.get(REAL_INFERENCE_ENV) == "1"
def require_real_inference() -> None:
if not real_inference_enabled():
raise BenchmarkError(
f"real model execution is opt-in: set {REAL_INFERENCE_ENV}=1 to run this benchmark"
)
def _load_evidence_signing_key() -> Ed25519PrivateKey:
raw_path = os.environ.get(EVIDENCE_SIGNING_KEY_ENV)
if not raw_path:
raise BenchmarkError(
f"real evidence requires {EVIDENCE_SIGNING_KEY_ENV} to name an Ed25519 private key"
)
path = Path(raw_path).expanduser().resolve(strict=True)
if os.name != "nt" and stat.S_IMODE(path.stat().st_mode) != 0o600:
raise BenchmarkError("evidence signing key must have mode 0600")
key = serialization.load_pem_private_key(path.read_bytes(), password=None)
if not isinstance(key, Ed25519PrivateKey):
raise BenchmarkError("evidence signing key must be Ed25519")
return key
def _utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _sign_report(report: dict[str, Any], key: Ed25519PrivateKey) -> None:
public_key = key.public_key().public_bytes(
serialization.Encoding.Raw, serialization.PublicFormat.Raw
)
report["provenance"]["signer_public_key_sha256"] = hashlib.sha256(
public_key
).hexdigest()
report["provenance"]["signature"] = base64.b64encode(
key.sign(report_signing_payload(report))
).decode("ascii")
def _process_rss(pid: int | None = None) -> int:
"""Resident bytes for a process and its children, or 0 when unobservable."""
try:
import psutil
except ImportError:
return 0
try:
process = psutil.Process(pid) if pid else psutil.Process()
total = process.memory_info().rss
for child in process.children(recursive=True):
try:
total += child.memory_info().rss
except psutil.Error:
continue
return int(total)
except Exception:
return 0
def _directory_bytes(path: Path) -> int:
if path.is_file():
return path.stat().st_size
return sum(entry.stat().st_size for entry in path.rglob("*") if entry.is_file())
def _artifact_sha256(path: Path) -> str:
"""Hash an artifact file or a deterministic directory content manifest.
A file uses the ordinary SHA-256 digest. A directory hashes each sorted
relative path, resolved file size, and file bytes, so tokenizer/config drift
cannot hide behind a weight-only digest.
"""
digest = hashlib.sha256()
if path.is_file():
entries = [(None, path)]
else:
entries = [
(entry.relative_to(path).as_posix(), entry)
for entry in sorted(path.rglob("*"))
if entry.is_file()
]
if not entries:
raise BenchmarkError(f"artifact directory is empty: {path}")
for relative, entry in entries:
if relative is not None:
encoded = relative.encode("utf-8")
digest.update(len(encoded).to_bytes(8, "big"))
digest.update(encoded)
digest.update(entry.stat().st_size.to_bytes(8, "big"))
with entry.open("rb") as stream:
while chunk := stream.read(8 * 1024 * 1024):
digest.update(chunk)
return digest.hexdigest()
def _host_manifest(config: Mapping[str, Any] | None = None) -> dict[str, Any]:
"""Capture non-secret host facts with the report rather than trusting prose."""
manifest: dict[str, Any] = {
"hostname": socket.gethostname(),
"platform": platform.platform(),
"python": sys.version.split()[0],
"cpu_count": os.cpu_count(),
}
try:
import torch
import transformers
manifest["torch_version"] = torch.__version__
manifest["transformers_version"] = transformers.__version__
manifest["cuda_available"] = bool(torch.cuda.is_available())
if torch.cuda.is_available():
manifest["accelerator_name"] = torch.cuda.get_device_name(0)
manifest["accelerator_runtime"] = getattr(torch.version, "cuda", None) or getattr(
torch.version, "hip", None
)
except ImportError:
manifest["torch_version"] = None
llama_identities: dict[str, dict[str, str]] = {}
for spec in (config or {}).get("recipes", ()):
driver = spec.get("driver", {})
if driver.get("type") != "llama-cpp-server":
continue
binary = Path(driver["binary"]).resolve(strict=True)
key = str(binary)
if key in llama_identities:
continue
version_result = subprocess.run(
[str(binary), "--version"],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
timeout=10,
)
llama_identities[key] = {
"sha256": _artifact_sha256(binary),
"version": " | ".join(version_result.stdout.strip().splitlines()),
}
if llama_identities:
manifest["llama_server_identities"] = llama_identities
return manifest
def _validate_config(
config: Mapping[str, Any], *, profile: str = CONTRACT_V1_PROFILE
) -> None:
"""Reject comparisons that mix models, artifacts, devices, or budgets."""
if profile not in {CONTRACT_V1_PROFILE, GPU_DIAGNOSTIC_PROFILE}:
raise BenchmarkError(f"unknown benchmark validation profile {profile!r}")
try:
plan = config["plan"]
root = Path(config["artifact_storage_root"]).resolve(strict=True)
recipes = config["recipes"]
except (KeyError, TypeError, OSError) as exc:
raise BenchmarkError(
"benchmark config needs an existing artifact_storage_root, plan, and recipes"
) from exc
if not root.is_absolute() or root == Path("/home") or Path("/home") in root.parents:
raise BenchmarkError("model artifacts must use configured mounted-drive storage, never /home")
if not isinstance(recipes, list) or not recipes:
raise BenchmarkError("benchmark config needs at least one recipe")
sampling = plan.get("sampling", {})
if (
float(sampling.get("temperature", 0.0)) != 0.0
or int(sampling.get("top_k", 1)) != 1
or float(sampling.get("top_p", 1.0)) != 1.0
):
raise BenchmarkError("the quality comparison requires greedy sampling")
if len(plan.get("prompts", ())) < 3 or int(plan.get("repeats", 0)) < 3:
raise BenchmarkError("contract-grade evidence requires at least 3 prompts and 3 repeats")
if int(plan.get("warmup_requests", 0)) < 1:
raise BenchmarkError("contract-grade evidence requires at least one warmup")
if int(sampling.get("max_output_tokens", 0)) < 32:
raise BenchmarkError("contract-grade evidence requires at least 32 output tokens")
thread_budgets: set[int] = set()
max_concurrency = max(int(level) for level in plan.get("concurrency_levels", (1, 4)))
for spec in recipes:
if spec.get("source_model_id") != plan.get("model_id"):
raise BenchmarkError("every recipe must declare the plan's exact source_model_id")
if spec.get("source_model_revision") != plan.get("model_revision"):
raise BenchmarkError("every recipe must declare the plan's exact source_model_revision")
digest = spec.get("artifact_sha256", "")
if not isinstance(digest, str) or re.fullmatch(r"[0-9a-f]{64}", digest) is None:
raise BenchmarkError("every recipe must declare a lowercase SHA-256 artifact digest")
artifact = Path(spec.get("artifact_path", "")).resolve(strict=True)
if artifact != root and root not in artifact.parents:
raise BenchmarkError("every model artifact must be beneath artifact_storage_root")
actual_digest = _artifact_sha256(artifact)
if not hmac.compare_digest(digest, actual_digest):
raise BenchmarkError(
f"artifact digest mismatch for {spec.get('id', '<unknown>')}: "
f"declared {digest}, measured {actual_digest}"
)
driver = spec.get("driver")
if not isinstance(driver, Mapping):
raise BenchmarkError("every recipe must declare a driver block")
if "artifact_sha256" in driver:
raise BenchmarkError(
"driver artifact_sha256 is forbidden; the validated recipe digest is authoritative"
)
kind = driver.get("type")
if kind == "transformers":
driver_artifact = Path(driver.get("model_path", "")).resolve(strict=True)
elif kind == "llama-cpp-server":
driver_artifact = Path(driver.get("gguf_path", "")).resolve(strict=True)
binary = Path(driver.get("binary", "")).resolve(strict=True)
binary_digest = driver.get("binary_sha256", "")
if (
not isinstance(binary_digest, str)
or re.fullmatch(r"[0-9a-f]{64}", binary_digest) is None
or not hmac.compare_digest(binary_digest, _artifact_sha256(binary))
):
raise BenchmarkError("llama.cpp binary SHA-256 mismatch")
if int(driver.get("n_parallel", max_concurrency)) < max_concurrency:
raise BenchmarkError("llama.cpp parallel slots must cover maximum concurrency")
driver_device = driver.get("device", "cpu")
gpu_layers = int(driver.get("n_gpu_layers", 0))
if profile == CONTRACT_V1_PROFILE and (
driver_device != "cpu" or gpu_layers != 0
):
raise BenchmarkError(
"v1 benchmark supports CPU-only llama.cpp until process VRAM is measurable"
)
if profile == GPU_DIAGNOSTIC_PROFILE and (
driver_device != "cuda" or gpu_layers <= 0
):
raise BenchmarkError(
"GPU diagnostic requires CUDA/ROCm llama.cpp with positive n_gpu_layers"
)
else:
raise BenchmarkError(f"unknown driver type {kind!r}")
if driver_artifact != artifact:
raise BenchmarkError("driver artifact path must match the hashed recipe artifact")
if driver.get("device", "cpu") != spec.get("device"):
raise BenchmarkError("recipe and driver must declare the same device")
if profile == CONTRACT_V1_PROFILE and spec.get("device") != "cpu":
raise BenchmarkError("contract-v1 requires every recipe to run on CPU")
if profile == GPU_DIAGNOSTIC_PROFILE and spec.get("device") != "cuda":
raise BenchmarkError("GPU diagnostic requires every recipe to declare device 'cuda'")
thread_budgets.add(int(driver.get("threads", 8)))
host = config.get("host", {})
if not isinstance(host, Mapping):
raise BenchmarkError("benchmark host metadata must be an object")
if profile == GPU_DIAGNOSTIC_PROFILE and host.get(
"benchmark_lane"
) != "rocm-gpu-diagnostic":
raise BenchmarkError("GPU diagnostic requires the rocm-gpu-diagnostic host marker")
if len(thread_budgets) != 1:
raise BenchmarkError("every recipe must use the same CPU thread budget")
class TransformersDriver:
"""The current Transformers/safetensors recipe: the correctness reference.
Generation is a hand-written prefill-then-decode loop rather than
``model.generate`` because the benchmark needs the two phases separated: one
forward over the prompt gives an exact prefill time and TTFT, and the cached
single-token steps that follow give an exact decode rate. ``generate`` would
hand back one blended number.
"""
def __init__(
self,
model_path: str,
*,
artifact_sha256: str | None = None,
device: str = "cpu",
dtype: str = "bfloat16",
threads: int = 8,
) -> None:
self.model_path = Path(model_path)
self.artifact_sha256 = artifact_sha256
self.device = device
self.dtype = dtype
self.threads = threads
self._model: Any = None
self._tokenizer: Any = None
self._torch: Any = None
self._rss_baseline = 0
def load(self) -> LoadStats:
if self.artifact_sha256 is not None:
measured_artifact_sha256 = _artifact_sha256(self.model_path)
if not hmac.compare_digest(self.artifact_sha256, measured_artifact_sha256):
raise BenchmarkError("Transformers artifact changed after config validation")
self._rss_baseline = _process_rss()
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
self._torch = torch
torch.set_num_threads(self.threads)
torch.manual_seed(0)
started = time.monotonic()
self._tokenizer = AutoTokenizer.from_pretrained(
str(self.model_path), local_files_only=True
)
self._model = AutoModelForCausalLM.from_pretrained(
str(self.model_path),
dtype=getattr(torch, self.dtype),
local_files_only=True,
)
self._model.to(self.device)
self._model.eval()
load_ms = (time.monotonic() - started) * 1000
return LoadStats(
artifact_bytes=_directory_bytes(self.model_path),
load_ms=round(load_ms, 4),
rss_bytes=max(0, _process_rss() - self._rss_baseline),
vram_bytes=self._vram_bytes(),
backend_detail=(
f"torch {torch.__version__}; dtype {self.dtype}; "
f"device {self.device}; intra-op threads {self.threads}"
),
)
def _vram_bytes(self) -> int:
torch = self._torch
if torch is None or self.device == "cpu":
return 0
try:
if torch.cuda.is_available():
return int(torch.cuda.max_memory_allocated())
except Exception:
return 0
return 0
def generate(self, prompt: str, sampling: SamplingPolicy) -> GenerationSample:
if self._model is None:
raise BenchmarkError("TransformersDriver.generate called before load()")
torch = self._torch
# add_special_tokens=False: the plan owns the prompt format, and the
# llama.cpp recipe is given the identical string.
encoded = self._tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
input_ids = encoded["input_ids"].to(self.device)
prompt_tokens = int(input_ids.shape[-1])
eos_ids = {self._tokenizer.eos_token_id} | set(
getattr(self._model.generation_config, "eos_token_id", None) or []
if isinstance(getattr(self._model.generation_config, "eos_token_id", None), list)
else []
)
eos_ids.discard(None)
started = time.monotonic()
with torch.inference_mode():
outputs = self._model(input_ids=input_ids, use_cache=True)
past = outputs.past_key_values
next_id = self._select(outputs.logits[:, -1, :], sampling)
ttft_ms = (time.monotonic() - started) * 1000
token_ids = [int(next_id.item())]
decode_started = time.monotonic()
while len(token_ids) < sampling.max_output_tokens and token_ids[-1] not in eos_ids:
outputs = self._model(
input_ids=next_id.view(1, 1), past_key_values=past, use_cache=True
)
past = outputs.past_key_values
next_id = self._select(outputs.logits[:, -1, :], sampling)
token_ids.append(int(next_id.item()))
decode_ms = (time.monotonic() - decode_started) * 1000
total_ms = (time.monotonic() - started) * 1000
emitted = [token for token in token_ids if token not in eos_ids]
return GenerationSample(
text=self._tokenizer.decode(emitted, skip_special_tokens=True),
prompt_tokens=prompt_tokens,
# The first token is produced by the prefill forward, so the decode
# rate must not be credited with it.
decode_tokens=max(0, len(token_ids) - 1),
ttft_ms=ttft_ms,
prefill_ms=ttft_ms,
decode_ms=decode_ms,
total_ms=total_ms,
)
def _select(self, logits: Any, sampling: SamplingPolicy) -> Any:
if sampling.temperature > 0:
raise BenchmarkError(
"this benchmark is greedy-only: sampling noise is indistinguishable from "
"quantization drift, which is precisely what the quality lane must isolate"
)
return logits.argmax(dim=-1)
def memory_probe(self) -> tuple[int, int]:
return max(0, _process_rss() - self._rss_baseline), self._vram_bytes()
def close(self) -> None:
self._model = None
self._tokenizer = None
if self._torch is not None:
import gc
gc.collect()
def _free_port() -> int:
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
return int(probe.getsockname()[1])
def _gpu_offload_evidence(log_text: str, requested_layers: int) -> str:
"""Extract measured ROCm device and layer placement from llama-server logs."""
device_matches = re.findall(
r"-\s+(ROCm\d+)\s+:\s+(.+?)\s+\(\d+\s+MiB", log_text
)
offload_matches = re.findall(
r"offloaded\s+(\d+)/(\d+)\s+layers\s+to\s+GPU", log_text
)
if not device_matches:
raise BenchmarkError("GPU diagnostic found no measured ROCm device in llama-server logs")
if not offload_matches:
raise BenchmarkError("GPU diagnostic found no measured layer offload in llama-server logs")
backend, device_name = device_matches[-1]
offloaded, total = (int(value) for value in offload_matches[-1])
required = min(requested_layers, total)
if requested_layers <= 0 or offloaded < required:
raise BenchmarkError(
f"GPU diagnostic requested {requested_layers} layers but measured "
f"only {offloaded}/{total} offloaded"
)
return (
f"measured accelerator {backend}: {device_name}; "
f"measured offload {offloaded}/{total} layers"
)
def _gpu_layer_config_detail(device: str, layers: int) -> str:
# Immutable v1 pins the historical CPU wording exactly. GPU diagnostics use
# the clearer requested/measured distinction without changing v1 identity.
return f"gpu layers {layers}" if device == "cpu" else f"requested gpu layers {layers}"
class LlamaCppServerDriver:
"""The whole-model llama.cpp/GGUF recipe, driven through ``llama-server``.
``llama-server`` is used rather than an in-process binding because it is the
shape llama.cpp is actually deployed in and the only one that offers
continuous batching across parallel slots — which is the runtime property
this project cares about most. It also reports its own prefill/decode
timings per request, so the decode rate is the runtime's own number and not
an inference drawn from a client-side stopwatch.
"""
def __init__(
self,
binary: str,
gguf_path: str,
*,
binary_sha256: str,
artifact_sha256: str | None = None,
device: str = "cpu",
threads: int = 8,
n_parallel: int = 4,
context_per_slot: int = 1024,
n_gpu_layers: int = 0,
startup_timeout_s: float = 120.0,
) -> None:
self.binary = Path(binary)
self.binary_sha256 = binary_sha256
self.gguf_path = Path(gguf_path)
self.artifact_sha256 = artifact_sha256
self.device = device
self.threads = threads
self.n_parallel = n_parallel
self.context_per_slot = context_per_slot
self.n_gpu_layers = n_gpu_layers
self.startup_timeout_s = startup_timeout_s
self._process: subprocess.Popen | None = None
self._port = 0
self._log: Any = None
@property
def _url(self) -> str:
return f"http://127.0.0.1:{self._port}"
def _log_text(self) -> str:
if self._log is None:
return ""
try:
size = min(os.fstat(self._log.fileno()).st_size, 8 * 1024 * 1024)
return os.pread(self._log.fileno(), size, 0).decode("utf-8", errors="replace")
except Exception:
return ""
def _log_excerpt(self) -> str:
return self._log_text()[-4096:].strip()
def load(self) -> LoadStats:
if not self.binary.exists():
raise BenchmarkError(f"llama-server binary not found at {self.binary}")
if not self.gguf_path.exists():
raise BenchmarkError(f"GGUF artifact not found at {self.gguf_path}")
if self.artifact_sha256 is not None:
measured_artifact_sha256 = _artifact_sha256(self.gguf_path)
if not hmac.compare_digest(self.artifact_sha256, measured_artifact_sha256):
raise BenchmarkError("GGUF artifact changed after config validation")
measured_binary_sha256 = _artifact_sha256(self.binary)
if not hmac.compare_digest(self.binary_sha256, measured_binary_sha256):
raise BenchmarkError("llama-server binary changed after config validation")
version = " | ".join(
subprocess.run(
[str(self.binary), "--version"],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
timeout=10,
).stdout.strip().splitlines()
)
self._port = _free_port()
command = [str(self.binary)]
if self.device == "cuda":
# Debug verbosity is required to capture measured device placement
# and the offloaded/total layer count in signed diagnostics.
command.extend(["-lv", "5"])
command.extend([
"--model", str(self.gguf_path),
"--host", "127.0.0.1",
"--port", str(self._port),
"--threads", str(self.threads),
"--parallel", str(self.n_parallel),
# Every slot must hold a whole request, so the pool is sized for the
# worst case rather than letting llama.cpp silently truncate context.
"--ctx-size", str(self.context_per_slot * self.n_parallel),
"--n-gpu-layers", str(self.n_gpu_layers),
"--no-webui",
])
started = time.monotonic()
self._log = tempfile.TemporaryFile(mode="w+b")
self._process = subprocess.Popen(
command, stdout=self._log, stderr=subprocess.STDOUT
)
self._await_health(started)
load_ms = (time.monotonic() - started) * 1000
gpu_evidence = ""
if self.device == "cuda":
gpu_evidence = _gpu_offload_evidence(self._log_text(), self.n_gpu_layers)
return LoadStats(
artifact_bytes=self.gguf_path.stat().st_size,
load_ms=round(load_ms, 4),
rss_bytes=_process_rss(self._process.pid),
vram_bytes=0,
backend_detail=(
f"{version}; binary sha256 {measured_binary_sha256}; "
f"threads {self.threads}; parallel slots {self.n_parallel}; "
f"ctx/slot {self.context_per_slot}; "
f"{_gpu_layer_config_detail(self.device, self.n_gpu_layers)}"
+ (f"; {gpu_evidence}" if gpu_evidence else "")
),
)
def _await_health(self, started: float) -> None:
while time.monotonic() - started < self.startup_timeout_s:
if self._process is not None and self._process.poll() is not None:
raise BenchmarkError(
f"llama-server exited with code {self._process.returncode} during startup; "
f"log tail: {self._log_excerpt()}"
)
try:
with urllib.request.urlopen(f"{self._url}/health", timeout=2) as response:
if response.status == 200:
return
except (urllib.error.URLError, OSError):
time.sleep(0.25)
raise BenchmarkError(
f"llama-server did not become healthy within {self.startup_timeout_s:.0f}s; "
f"log tail: {self._log_excerpt()}"
)
def generate(self, prompt: str, sampling: SamplingPolicy) -> GenerationSample:
if self._process is None:
raise BenchmarkError("LlamaCppServerDriver.generate called before load()")
if sampling.temperature > 0:
raise BenchmarkError("this benchmark is greedy-only; see TransformersDriver._select")
body = json.dumps({
"prompt": prompt,
"n_predict": sampling.max_output_tokens,
"temperature": 0.0,
"top_k": 1,
"top_p": 1.0,
"seed": sampling.seed,
# Prompt cache reuse across repeats would measure the cache, not the
# prefill, and the safetensors recipe has no equivalent.
"cache_prompt": False,
"stream": True,
}).encode()
request = urllib.request.Request(
f"{self._url}/completion", data=body,
headers={"Content-Type": "application/json"}, method="POST",
)
started = time.monotonic()
chunks: list[str] = []
timings: Mapping[str, Any] = {}
with urllib.request.urlopen(request, timeout=600) as response:
for raw in response:
line = raw.decode("utf-8").strip()
if not line.startswith("data:"):
continue
payload = json.loads(line[len("data:"):].strip())
content = payload.get("content", "")
chunks.append(content)
if payload.get("stop"):
timings = payload.get("timings") or {}
total_ms = (time.monotonic() - started) * 1000
if not timings:
raise BenchmarkError("llama-server returned no timings; cannot report an honest split")
prefill_ms = float(timings.get("prompt_ms", 0.0))
decode_ms = float(timings.get("predicted_ms", 0.0))
return GenerationSample(
text="".join(chunks),
prompt_tokens=int(timings.get("prompt_n", 0)),
# llama.cpp starts predicted_ms after sampling the first token while
# predicted_n includes it. Exclude that token to match the
# Transformers inter-token decode metric.
decode_tokens=max(0, int(timings.get("predicted_n", 0)) - 1),
# Use the runtime's prompt/first-token timing, matching the
# in-process Transformers boundary. HTTP/SSE and slot delay remain
# represented by total latency and queue_wait_ms.
ttft_ms=prefill_ms,
prefill_ms=prefill_ms,
decode_ms=decode_ms,
total_ms=total_ms,
# Whatever the wall clock saw but the runtime did not attribute to
# compute is time this request spent waiting for a slot.
queue_wait_ms=max(0.0, total_ms - prefill_ms - decode_ms),
)
def memory_probe(self) -> tuple[int, int]:
if self._process is None:
return 0, 0
return _process_rss(self._process.pid), 0
def close(self) -> None:
if self._process is not None:
if self._process.poll() is None:
self._process.terminate()
try:
self._process.wait(timeout=20)
except subprocess.TimeoutExpired:
self._process.kill()
self._process.wait(timeout=10)
self._process = None
if self._log is not None:
self._log.close()
self._log = None
def build_driver(spec: Mapping[str, Any], plan: BenchmarkPlan) -> RecipeDriverBundle:
"""Construct the driver named by a recipe's ``driver`` block."""
driver_spec = dict(spec["driver"])
kind = driver_spec.pop("type")
driver_spec["artifact_sha256"] = spec.get("artifact_sha256")
if kind == "transformers":
return TransformersDriver(**driver_spec)
if kind == "llama-cpp-server":
driver_spec.setdefault("n_parallel", max(plan.concurrency_levels))
return LlamaCppServerDriver(**driver_spec)
raise BenchmarkError(f"unknown driver type {kind!r}")
RecipeDriverBundle = Any # a RecipeDriver; named for readability at the call site
def _plan_from_config(config: Mapping[str, Any]) -> BenchmarkPlan:
raw = config["plan"]
return BenchmarkPlan(
plan_id=raw["plan_id"],
model_id=raw["model_id"],
model_revision=raw["model_revision"],
prompts=tuple(PromptSpec(**prompt) for prompt in raw["prompts"]),
sampling=SamplingPolicy(**raw.get("sampling", {})),
concurrency_levels=tuple(raw.get("concurrency_levels", (1, 4))),
repeats=int(raw.get("repeats", 1)),
warmup_requests=int(raw.get("warmup_requests", 1)),
)
def _recipe_from_config(spec: Mapping[str, Any]) -> RecipeSpec:
return RecipeSpec(
id=spec["id"],
runtime=spec["runtime"],
weight_format=spec["weight_format"],
weight_quantization=spec["weight_quantization"],
lane=Lane(spec["lane"]),
device=spec["device"],
artifact_path=spec.get("artifact_path", ""),
source_model_id=spec.get("source_model_id", ""),
source_model_revision=spec.get("source_model_revision", ""),
artifact_sha256=spec.get("artifact_sha256", ""),
is_reference=bool(spec.get("is_reference", False)),
notes=spec.get("notes", ""),
)
def run_configured_benchmark(config: Mapping[str, Any]) -> dict:
"""Run contract-v1 evidence through the CPU-only, fail-closed profile."""
return _run_profiled_benchmark(config, profile=CONTRACT_V1_PROFILE)
def run_configured_gpu_diagnostic(config: Mapping[str, Any]) -> dict:
"""Run signed ROCm diagnostics that are intentionally ineligible for v1."""
return _run_profiled_benchmark(config, profile=GPU_DIAGNOSTIC_PROFILE)
def _producer_for_profile(profile: str) -> str:
if profile == CONTRACT_V1_PROFILE:
return REAL_REPORT_PRODUCER
if profile == GPU_DIAGNOSTIC_PROFILE:
return GPU_DIAGNOSTIC_REPORT_PRODUCER
raise BenchmarkError(f"unknown benchmark validation profile {profile!r}")
def _run_profiled_benchmark(config: Mapping[str, Any], *, profile: str) -> dict:
"""Validate a closed profile and derive its producer before measurement."""
require_real_inference()
_validate_config(config, profile=profile)
producer = _producer_for_profile(profile)
evidence_class = config.get("evidence_class", "local-real")
if evidence_class not in {"local-real", "multi-machine-real"}:
raise BenchmarkError("canonical real runner cannot emit synthetic evidence")
signing_key = _load_evidence_signing_key()
started_at = _utc_now()
run_id = str(uuid.uuid4())
config_sha256 = _canonical_sha256(config)
plan = _plan_from_config(config)
from .recipe_benchmark import RecipeMeasurement # local import keeps the seam obvious
measurements = []
for spec in config["recipes"]:
recipe = _recipe_from_config(spec)
driver = None
try:
driver = build_driver(spec, plan)
measurements.append(measure_recipe(driver, recipe, plan))
except Exception as exc:
measurements.append(RecipeMeasurement(
recipe=recipe,
load=LoadStats(artifact_bytes=0, load_ms=0.0),
unavailable_reason=f"{type(exc).__name__}: {exc}",
))
finally:
if driver is not None:
driver.close()
report = build_report(
plan,
measurements,
host={**dict(config.get("host", {})), **_host_manifest(config)},
evidence_class=evidence_class,
provenance={
"schema_version": PROVENANCE_SCHEMA_VERSION,
"producer": producer,
"run_id": run_id,
"started_at": started_at,
"completed_at": _utc_now(),
"config_sha256": config_sha256,
"signature_algorithm": "ed25519",
},
)
_sign_report(report, signing_key)
return report

1147
packages/node/meshnet_node/runtime_recipe.py Normal file
View File

File diff suppressed because it is too large Load Diff

74
packages/node/native/CMakeLists.txt Normal file
View File

@@ -0,0 +1,74 @@
# Native Shard protocol: C++ schema generation, build wiring and conformance test.
#
# The C++ stubs are generated into the build tree at configure time — they are
# never committed. A C++ consumer already needs a toolchain, so committing
# generated C++ would only create a second copy of the schema that can rot.
#
# gRPC C++ is optional here on purpose. The conformance test only needs message
# types, so the schema can be verified on a machine that has protobuf but not
# the gRPC C++ stack. When gRPC *is* found, the service stubs are generated too
# and exported as `shard_runtime_grpc` for the worker (DGR-008) to link.
#
# Build:
# cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=<protobuf-install>
# cmake --build build/native -j
# ctest --test-dir build/native --output-on-failure
#
# `scripts/bootstrap_native_toolchain.sh` builds a protobuf install from source
# when the system has none.
cmake_minimum_required(VERSION 3.24)
project(meshnet_shard_protocol CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(protobuf CONFIG REQUIRED)
find_package(gRPC CONFIG QUIET)
set(SHARD_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/proto/shard_runtime.proto")
# Message types: always available.
add_library(shard_runtime_proto STATIC "${SHARD_PROTO}")
target_link_libraries(shard_runtime_proto PUBLIC protobuf::libprotobuf)
target_include_directories(shard_runtime_proto PUBLIC "${CMAKE_CURRENT_BINARY_DIR}")
protobuf_generate(
TARGET shard_runtime_proto
LANGUAGE cpp
IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto"
PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}"
)
# Service stubs: only when the gRPC C++ stack is present.
if(gRPC_FOUND)
add_library(shard_runtime_grpc STATIC "${SHARD_PROTO}")
target_link_libraries(shard_runtime_grpc PUBLIC shard_runtime_proto gRPC::grpc++)
target_include_directories(shard_runtime_grpc PUBLIC "${CMAKE_CURRENT_BINARY_DIR}")
protobuf_generate(
TARGET shard_runtime_grpc
LANGUAGE grpc
GENERATE_EXTENSIONS .grpc.pb.h .grpc.pb.cc
PLUGIN "protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>"
IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto"
PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}"
)
message(STATUS "gRPC C++ found: building ShardRuntime service stubs")
else()
message(STATUS "gRPC C++ not found: building message types only "
"(sufficient for the conformance test)")
endif()
enable_testing()
add_executable(shard_protocol_conformance tests/test_shard_protocol_conformance.cpp)
target_link_libraries(shard_protocol_conformance PRIVATE shard_runtime_proto)
# The test asserts against the same committed vectors the Python test uses, and
# writes its own re-serialization back out so Python can prove the two languages
# agree byte-for-byte.
add_test(
NAME shard_protocol_conformance
COMMAND shard_protocol_conformance
"${CMAKE_CURRENT_SOURCE_DIR}/testdata"
"${CMAKE_CURRENT_BINARY_DIR}"
)

69
packages/node/native/README.md Normal file
View File

@@ -0,0 +1,69 @@
# Native Shard protocol
`proto/shard_runtime.proto` is the semantic contract between a Meshnet node and
a Shard worker: Protocol Buffers over gRPC/HTTP2 (ADR-0020). It is the source of
truth. The Python and C++ types are generated from it; neither is the contract.
## What lives here
| Path | Purpose |
|---|---|
| `proto/shard_runtime.proto` | The schema: capability, health, session stream, release, cancel |
| `testdata/*.binpb` | Committed conformance vectors both languages assert against |
| `tests/test_shard_protocol_conformance.cpp` | C++ conformance test |
| `CMakeLists.txt` | C++ generation, build wiring, and `ctest` registration |
The Python stubs are generated into
`packages/node/meshnet_node/native_protocol/generated/` and are committed, so
installing a node needs no protoc. The C++ stubs are generated into the build
tree and are never committed — a C++ consumer already has a toolchain, and a
committed copy could only rot.
## Regenerating
```bash
pip install grpcio-tools==1.82.1 # bundles protoc; no system protoc needed
python scripts/generate_native_protocol.py # rewrite the Python stubs
python scripts/generate_native_protocol.py --check # fail if they drifted
python scripts/generate_protocol_goldens.py --check # fail if the vectors drifted
```
Both `--check` modes run in CI via `tests/test_native_shard_protocol.py`, so a
schema edit that is not accompanied by regenerated output fails the suite rather
than shipping stubs that disagree with the schema they claim to implement.
## Building and running the C++ conformance test
If the machine has no protobuf C++ toolchain:
```bash
scripts/bootstrap_native_toolchain.sh build/native-toolchain
```
Then:
```bash
cmake -S packages/node/native -B build/native \
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain"
cmake --build build/native -j
ctest --test-dir build/native --output-on-failure
```
gRPC C++ is optional: without it, CMake builds the message types only, which is
all the conformance test needs. When gRPC C++ *is* found, the `ShardRuntime`
service stubs are built too and exported as `shard_runtime_grpc` for the worker
(DGR-008) to link.
## How the cross-language check actually proves something
Two codecs that each round-trip their own output prove only that each is
self-consistent. Instead:
1. Python builds the canonical message and commits its bytes to `testdata/`.
2. The C++ test parses *those* bytes, asserts every field, independently
recomputes the CRC32C from the polynomial, and re-serializes to
`cpp_roundtrip.binpb` in the build tree.
3. `test_cpp_and_python_agree_byte_for_byte` compares that file to the golden.
Byte equality across the two implementations is the claim; anything less is two
parallel test suites that can drift apart.

592
packages/node/native/proto/shard_runtime.proto Normal file
View File

@@ -0,0 +1,592 @@
// The native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020).
//
// This schema — not any Python or C++ type — is the semantic contract between a
// Meshnet node and a Shard worker. Direct hops speak gRPC. When direct
// connectivity is unavailable the existing relay carries these exact serialized
// frames as opaque binary, which is why every deadline, cancellation and
// identity field is carried *in the schema* rather than left to gRPC metadata:
// a relayed frame has no HTTP/2 context to inherit them from.
//
// Meshnet remains the only control plane. Nothing here selects routes, prices
// work, or authenticates peers; the worker executes a layer range it has been
// admitted for and reports what it did.
syntax = "proto3";
package meshnet.shard.v1;
option cc_enable_arenas = true;
option go_package = "github.com/meshnet/shard/v1;shardv1";
// ---------------------------------------------------------------------------
// Versioning
// ---------------------------------------------------------------------------
// Wire-compatibility generation of this schema. A worker rejects a peer whose
// SCHEMA_VERSION it cannot satisfy instead of guessing at field meaning.
//
// This is deliberately NOT the `X-Meshnet-Wire` version of the legacy HTTP
// activation format (currently "2", see ADR-0008). The native protocol is a
// separate contract with its own generation counter and starts at 1.
enum SchemaVersion {
SCHEMA_VERSION_UNSPECIFIED = 0;
SCHEMA_VERSION_1 = 1;
}
// ---------------------------------------------------------------------------
// Tensor bundle — the public activation boundary
// ---------------------------------------------------------------------------
// Element type of a tensor payload.
//
// bfloat16 is the canonical activation dtype at every Shard boundary regardless
// of the weight quantization a node uses internally (ADR-0008): weights may be
// NF4 or INT8, compute upcasts, and the boundary stays bfloat16. The other
// values exist for auxiliary tensors (position ids, attention masks) and for
// architectures whose boundary is not a plain hidden state.
enum DType {
DTYPE_UNSPECIFIED = 0;
DTYPE_BFLOAT16 = 1;
DTYPE_FLOAT16 = 2;
DTYPE_FLOAT32 = 3;
DTYPE_INT32 = 4;
DTYPE_INT64 = 5;
DTYPE_UINT8 = 6;
DTYPE_INT8 = 7;
DTYPE_BOOL = 8;
}
// Byte order of a tensor payload. Little-endian is canonical on the wire; the
// field is explicit so a big-endian peer is rejected loudly rather than
// silently reading byte-swapped activations.
enum ByteOrder {
BYTE_ORDER_UNSPECIFIED = 0;
BYTE_ORDER_LITTLE_ENDIAN = 1;
BYTE_ORDER_BIG_ENDIAN = 2;
}
// Payload compression. Mirrors the policy-driven zstd decision the existing
// activation seam already makes (`activation_compression.py`): compression is a
// transport optimisation and NONE is always a legal choice for any payload.
enum Compression {
COMPRESSION_UNSPECIFIED = 0;
COMPRESSION_NONE = 1;
COMPRESSION_ZSTD = 2;
}
// Integrity algorithm covering a payload.
enum ChecksumAlgorithm {
CHECKSUM_ALGORITHM_UNSPECIFIED = 0;
CHECKSUM_ALGORITHM_NONE = 1;
CHECKSUM_ALGORITHM_CRC32C = 2;
}
// An integrity check over the *uncompressed* canonical payload bytes.
//
// Checksumming the uncompressed bytes (not the compressed frame) means the
// value is stable whether a hop compressed the payload or not, so a relay may
// re-frame or a peer may decline compression without invalidating it.
message Checksum {
ChecksumAlgorithm algorithm = 1;
// Big-endian encoding of the checksum value.
bytes value = 2;
}
// One slice of a tensor's wire payload.
//
// Fragments bound individual payload pieces and make coverage independently
// verifiable. The negotiated `FlowControl.max_chunk_bytes` bounds the enclosing
// serialized stream message; `byte_offset` proves the pieces tile the tensor
// exactly — no hole, no overlap — instead of trusting arrival order.
//
// Offsets and payloads describe the *wire* body, which is the compressed frame
// when the parent tensor declares a compression. A zstd frame is not decodable
// per fragment, so a receiver reassembles first and decompresses once. The
// uncompressed size is not repeated here: `NamedTensor.total_bytes` is the
// single source of truth, and a second copy could only ever disagree with it.
message TensorFragment {
uint32 fragment_index = 1;
uint32 fragment_count = 2;
// Offset of this fragment within the tensor's wire body.
uint64 byte_offset = 3;
// Fragment bytes, compressed iff the parent tensor declares a compression.
bytes payload = 4;
reserved 5;
reserved "uncompressed_size";
}
// One named tensor at an architecture boundary.
message NamedTensor {
// Architecture-defined boundary name, e.g. "hidden_states", "position_ids".
// A boundary may need more than one tensor, which is why the payload is a
// named bundle rather than a bare buffer (ADR-0020).
string name = 1;
repeated int64 shape = 2;
DType dtype = 3;
ByteOrder byte_order = 4;
// Total uncompressed payload size. A receiver sizes its buffer from this
// before reading fragments and refuses a bundle that exceeds its budget.
uint64 total_bytes = 5;
Compression compression = 6;
// Over the uncompressed payload; see Checksum.
Checksum checksum = 7;
// Ordered fragments. A tensor small enough for one message carries exactly
// one fragment with fragment_count == 1.
repeated TensorFragment fragments = 8;
}
// A versioned named-tensor bundle: the public activation boundary payload.
message TensorBundle {
// Generation of the bundle layout itself, independent of SchemaVersion so a
// boundary payload can evolve without a whole-protocol version bump.
uint32 bundle_version = 1;
repeated NamedTensor tensors = 2;
}
// ---------------------------------------------------------------------------
// Identity: what work this is, for which route, against which artifact
// ---------------------------------------------------------------------------
// Exact identity of the executable thing a Shard runs.
//
// Both fingerprints must match end to end across a route. Two nodes agreeing on
// a model name but not on quantization, kernel or tail-norm placement would
// produce silently wrong tokens, so identity is a digest, not a label.
message Fingerprint {
// Digest of the exact Model Artifact (weights + config), e.g. the GGUF hash.
string model_artifact_digest = 1;
// Digest of the exact runtime recipe (backend, quantization, kernels, dtype).
string runtime_recipe_digest = 2;
// Human-readable recipe identity, mirroring the capability report an admitted
// node already registers with (ADR-0023). Labels are for diagnosis; the
// digests above are what a peer actually compares.
string recipe_id = 3;
string recipe_version = 4;
string catalogue_version = 5;
}
// The contiguous transformer layer range a Shard owns.
message ShardRange {
// Registered range, inclusive start, exclusive end.
uint32 start_layer = 1;
uint32 end_layer = 2;
// The layer this Shard must actually begin at for this route (ADR-0012).
//
// Shard ranges may overlap; the Tracker resolves the overlap when it builds
// the route and every hop is told where the previous hop stopped. Executing
// from `start_layer` when `effective_start_layer` is higher would re-apply
// layers already applied to the incoming activation and silently corrupt the
// result. A worker executes [effective_start_layer, end_layer).
uint32 effective_start_layer = 3;
}
// Which part of generation a message belongs to.
enum Phase {
PHASE_UNSPECIFIED = 0;
PHASE_PREFILL = 1;
PHASE_DECODE = 2;
PHASE_RELEASE = 3;
PHASE_CANCEL = 4;
}
// Token span carried by one chunk.
message PositionSpan {
// Absolute position of the first token in this chunk within the sequence.
uint64 first_position = 1;
// Number of token positions in this chunk. A decode step carries 1.
uint32 token_count = 2;
}
// Bounded chunking for a prefill.
//
// A long prompt is split into token-aligned chunks before the first forward
// pass (ADR-0008) so peak transfer per boundary stays bounded regardless of
// prompt length. Splits never fall mid-token.
message ChunkInfo {
uint32 chunk_index = 1;
uint32 chunk_count = 2;
// True on the final chunk of a prefill. A receiver that has not seen a final
// chunk knows the prefill is still incomplete.
bool final_chunk = 3;
}
// How the sender expects the receiver's Hot KV State to be positioned.
enum CacheMode {
CACHE_MODE_UNSPECIFIED = 0;
// No session state; the payload carries everything needed. Always a legal
// fallback and the recovery path after a miss.
CACHE_MODE_STATELESS = 1;
// Establish fresh session state for this Shard's layer range.
CACHE_MODE_PREFILL = 2;
// Continue existing session state. `expected_past_len` must match exactly.
CACHE_MODE_DECODE = 3;
}
// The sender's expectation about receiver-side cache state (ADR-0022).
//
// A mismatch is always a declared cache miss, never a silent stateless forward:
// running a single-token decode payload without the matching cache would emit
// plausible garbage. The receiver answers a miss with ShardError.CACHE_MISS and
// the head re-prefills the whole sequence under the same Route Session ID.
message CacheExpectation {
CacheMode mode = 1;
// Decode only: the number of tokens the receiver's session cache must already
// hold for this Shard's layer range.
uint64 expected_past_len = 2;
}
// What the receiver's cache actually did.
message CacheResult {
CacheMode mode = 1;
// Tokens held for this Shard's range after the step completed.
uint64 past_len = 2;
// True when session state was reused rather than rebuilt.
bool cache_hit = 3;
}
// Everything required to interpret one unit of work, independent of transport.
//
// Carried on every request and response because a relayed frame arrives as
// opaque binary with no gRPC metadata, no deadline and no channel identity.
message Envelope {
SchemaVersion schema_version = 1;
// Unique id for this unit of work; also the unit of billing attribution.
string work_id = 2;
// The Route Session this work belongs to. Together with `route_epoch` it maps
// to exactly one isolated llama sequence / bounded context (ADR-0020).
string route_session_id = 3;
// Bumped by the control plane whenever the route changes. A worker refuses
// work from a stale epoch rather than mixing it into live session state.
uint64 route_epoch = 4;
Fingerprint fingerprint = 5;
ShardRange shard_range = 6;
Phase phase = 7;
PositionSpan position = 8;
// Monotonic step within (route_session_id, route_epoch).
//
// Idempotency key: a retried or duplicated step carries the same value and
// must be acknowledged, not re-applied. Re-applying a decode step would
// advance the KV cache twice and desynchronise the route.
uint64 idempotency_step = 9;
CacheExpectation cache_expectation = 10;
// Absolute deadline. Carried in-band so a relayed frame keeps its deadline;
// on a direct hop it is set consistently with the gRPC deadline.
int64 deadline_unix_nanos = 11;
// Chunking, when this message is part of a bounded prefill.
ChunkInfo chunk = 12;
}
// ---------------------------------------------------------------------------
// Structured errors
// ---------------------------------------------------------------------------
// Why a unit of work could not be executed as asked.
//
// These are semantic outcomes, distinct from transport failure. They travel as
// a ShardStatus on the stream (and as google.rpc.Status details on unary RPCs)
// so a relayed frame carries the same diagnosis a direct gRPC hop would.
enum ErrorCode {
ERROR_CODE_UNSPECIFIED = 0;
// Peer cannot satisfy the requested SchemaVersion.
ERROR_CODE_SCHEMA_UNSUPPORTED = 1;
// Model artifact or runtime recipe digest differs from this worker's.
ERROR_CODE_FINGERPRINT_MISMATCH = 2;
// Work arrived for a route epoch the worker has already moved past.
ERROR_CODE_EPOCH_STALE = 3;
// Requested layer range is not the range this worker serves.
ERROR_CODE_SHARD_RANGE_MISMATCH = 4;
// Session state absent or positioned differently than expected (ADR-0022).
// The head recovers with one full re-prefill; this is not a fatal error.
ERROR_CODE_CACHE_MISS = 5;
// Admission budget (weights, KV, scratch, queue depth) would be exceeded.
ERROR_CODE_RESOURCE_EXHAUSTED = 6;
// Payload failed its checksum, or fragments did not cover the tensor.
ERROR_CODE_PAYLOAD_CORRUPT = 7;
// Work was cancelled by the control plane or the peer.
ERROR_CODE_CANCELLED = 8;
// Deadline in the envelope had already passed when work was dequeued.
ERROR_CODE_DEADLINE_EXCEEDED = 9;
// Sender exceeded its granted flow-control credit.
ERROR_CODE_FLOW_CONTROL_VIOLATION = 10;
// The worker failed while executing; detail is sanitized.
ERROR_CODE_INTERNAL = 11;
}
message ShardError {
ErrorCode code = 1;
// Sanitized, operator-facing explanation. Never a raw exception, file path or
// credential (ADR-0023 sanitization rule).
string detail = 2;
// True when the same work may be retried unchanged. A CACHE_MISS is not
// retryable unchanged — the head must re-prefill first.
bool retryable = 3;
// Set on CACHE_MISS so the head knows where the receiver actually is.
uint64 actual_past_len = 4;
}
// ---------------------------------------------------------------------------
// Flow control
// ---------------------------------------------------------------------------
// Application-level credit, layered on top of HTTP/2 flow control.
//
// HTTP/2 bounds bytes in flight; it does not bound how much *work* a worker has
// queued, and a relayed frame gets no HTTP/2 window at all. Credits bound the
// number of un-acked chunks a sender may have outstanding, which is what keeps
// worker queues and KV pressure bounded and lets prefill avoid starving decode.
message FlowControl {
// Additional chunks the receiver is willing to accept beyond those already
// granted. Purely additive: a grant never reduces outstanding credit.
uint32 credits_granted = 1;
// Hard ceiling on un-acked chunks, independent of granted credit.
uint32 max_inflight_chunks = 2;
// Hard ceiling on the serialized size of one chunk message.
uint64 max_chunk_bytes = 3;
// Hard ceiling on token positions per prefill chunk.
uint32 max_prefill_chunk_tokens = 4;
}
// ---------------------------------------------------------------------------
// Session stream messages
// ---------------------------------------------------------------------------
// Opens one long-lived bidirectional stream for one Route Session Activation
// Seam. The handshake settles version, identity and the initial flow-control
// window before any activation is sent, so an incompatible peer fails at open
// rather than mid-generation.
message SessionOpen {
// Highest schema version the initiator supports; the acceptor replies with
// the version actually negotiated.
SchemaVersion schema_version = 1;
string route_session_id = 2;
uint64 route_epoch = 3;
Fingerprint fingerprint = 4;
ShardRange shard_range = 5;
// Flow-control limits the initiator can honour. The acceptor replies with the
// limits that actually apply.
FlowControl proposed_flow_control = 6;
// Compressions the initiator can decode. NONE is implicitly always supported.
repeated Compression accepted_compression = 7;
}
message SessionAccepted {
// The version both peers will use for the life of this stream.
SchemaVersion schema_version = 1;
string route_session_id = 2;
uint64 route_epoch = 3;
// Authoritative limits. The initiator must not exceed these.
FlowControl flow_control = 4;
repeated Compression accepted_compression = 5;
// Fingerprint the worker actually serves, so a mismatch is visible at open.
Fingerprint fingerprint = 6;
}
// One unit of activation work: a bounded prefill chunk or a decode step.
message ActivationChunk {
Envelope envelope = 1;
TensorBundle bundle = 2;
}
// The small decode fast path.
//
// A decode step is one token: the envelope's identity fields are already fixed
// for the life of the stream, so repeating them per token is pure overhead on
// the hottest path. A DecodeStep carries only what changes — the step, the
// position, and one tensor — and inherits the rest from the SessionOpen
// handshake. A peer may always fall back to ActivationChunk with PHASE_DECODE.
message DecodeStep {
// Idempotency step within the session; also orders the stream.
uint64 idempotency_step = 1;
// Absolute position of this token.
uint64 position = 2;
// Tokens the receiver's cache must already hold. A mismatch is a CACHE_MISS.
uint64 expected_past_len = 3;
// The single boundary tensor for this token, typically [1, 1, hidden].
NamedTensor tensor = 4;
// Work id for attribution; the session/epoch/fingerprint/range are inherited.
string work_id = 5;
int64 deadline_unix_nanos = 6;
}
// Drop session state for a Route Session. Bounded memory does not depend on
// this arriving — workers also evict by TTL and LRU (ADR-0022) — but an
// explicit release returns KV immediately instead of holding it for the TTL.
message ReleaseSignal {
string route_session_id = 1;
uint64 route_epoch = 2;
string work_id = 3;
}
// Abandon in-flight work.
//
// Cancellation must remain possible when the stream is wedged behind flow
// control, which is why Cancel also exists as a unary RPC on a fresh call.
message CancelSignal {
string route_session_id = 1;
uint64 route_epoch = 2;
// Cancel one unit of work; empty cancels every in-flight unit for the session.
string work_id = 3;
string reason = 4;
}
// Acknowledges a chunk or decode step, releasing its flow-control credit and
// recording where the receiver's cache now stands.
message Ack {
string work_id = 1;
uint64 idempotency_step = 2;
CacheResult cache_result = 3;
// True when the step was recognised as an already-applied duplicate and was
// therefore acknowledged without being re-applied.
bool duplicate = 4;
// Observed execution time, for Generation Telemetry.
uint64 execution_nanos = 5;
}
// A terminal outcome for one unit of work, or for the session.
message ShardStatus {
string work_id = 1;
string route_session_id = 2;
uint64 idempotency_step = 3;
ShardError error = 4;
// True when the stream itself is finished and no further work will be served.
bool terminal = 5;
}
message SessionRequest {
oneof kind {
SessionOpen open = 1;
ActivationChunk chunk = 2;
DecodeStep decode = 3;
FlowControl flow_control = 4;
ReleaseSignal release = 5;
CancelSignal cancel = 6;
}
}
message SessionResponse {
oneof kind {
SessionAccepted accepted = 1;
// Result activations forwarded toward the next Shard, or to the head.
ActivationChunk chunk = 2;
Ack ack = 3;
FlowControl flow_control = 4;
ShardStatus status = 5;
}
}
// ---------------------------------------------------------------------------
// Capability and health
// ---------------------------------------------------------------------------
message CapabilityRequest {
SchemaVersion schema_version = 1;
}
// What this worker can actually execute, proven by a bounded real forward
// (ADR-0023). Detected hardware is not a capability; only a validated recipe is.
message CapabilityReport {
SchemaVersion schema_version = 1;
Fingerprint fingerprint = 2;
ShardRange shard_range = 3;
// Backend/device identity, e.g. "rocm:gfx1151", "cpu:avx512".
string backend = 4;
string device = 5;
// True only when a bounded real forward has passed for exactly this
// (artifact, range, recipe, device). Never set from hardware detection alone.
bool validated = 6;
// Sanitized reason when `validated` is false.
string detail = 7;
// Admission budgets this worker will enforce.
uint64 max_concurrent_sessions = 8;
uint64 max_context_tokens = 9;
FlowControl flow_control = 10;
repeated Compression accepted_compression = 11;
repeated SchemaVersion supported_schema_versions = 12;
int64 validated_at_unix_nanos = 13;
}
message HealthRequest {
SchemaVersion schema_version = 1;
}
enum ServingState {
SERVING_STATE_UNSPECIFIED = 0;
// Accepting new Route Sessions.
SERVING_STATE_SERVING = 1;
// Alive but refusing new sessions; existing sessions continue draining.
SERVING_STATE_DRAINING = 2;
// Alive but not routable — e.g. registered-but-dark pending certification.
SERVING_STATE_NOT_SERVING = 3;
}
// Live load, for backpressure and Generation Telemetry.
message HealthReport {
SchemaVersion schema_version = 1;
ServingState state = 2;
uint32 active_sessions = 3;
uint32 queued_chunks = 4;
// Occupancy of the continuous decode batch.
uint32 batch_occupancy = 5;
// Fraction of the KV budget in use, 0..1.
float kv_pressure = 6;
uint64 resident_bytes = 7;
string detail = 8;
}
message ReleaseRequest {
SchemaVersion schema_version = 1;
string route_session_id = 2;
uint64 route_epoch = 3;
}
message ReleaseResponse {
// True when session state existed and was dropped; false when there was
// nothing to drop, which is a success, not an error — release is idempotent.
bool released = 1;
ShardError error = 2;
}
message CancelRequest {
SchemaVersion schema_version = 1;
string route_session_id = 2;
uint64 route_epoch = 3;
// Empty cancels every in-flight unit for the session.
string work_id = 4;
string reason = 5;
}
message CancelResponse {
uint32 cancelled_work_items = 1;
ShardError error = 2;
}
// ---------------------------------------------------------------------------
// Service
// ---------------------------------------------------------------------------
service ShardRuntime {
// What this worker can execute. Read before a route is built.
rpc GetCapability(CapabilityRequest) returns (CapabilityReport);
// Live load and serving state.
rpc Health(HealthRequest) returns (HealthReport);
// One long-lived bidirectional stream per Route Session Activation Seam.
//
// The stream opens with SessionOpen/SessionAccepted, then carries bounded
// prefill chunks and decode steps in both directions for the life of the
// session. Per-token channel creation is a non-goal: the handshake cost is
// paid once and the hot path carries only what changes.
rpc Session(stream SessionRequest) returns (stream SessionResponse);
// Drop session state out of band. Idempotent.
rpc Release(ReleaseRequest) returns (ReleaseResponse);
// Cancel out of band, on a fresh call.
//
// In-band CancelSignal is preferred, but a sender that is blocked on flow
// control cannot write one — a cancel that can only travel down a wedged
// stream is not a cancel. This RPC always has a path to the worker.
rpc Cancel(CancelRequest) returns (CancelResponse);
}

2
packages/node/native/testdata/capability_report_golden.binpb vendored Normal file
View File

@@ -0,0 +1,2 @@
F
sha256:1f0c9d2esha256:ab77e410llama-gguf-q4km-rocm"3* 2026.07.1 "rocm*gfx11510@H€@R €€€ €Zbh€€Ð<E282AC>éθý

BIN
packages/node/native/testdata/session_request_golden.binpb vendored Normal file
View File

Binary file not shown.

337
packages/node/native/tests/test_shard_protocol_conformance.cpp Normal file
View File

@@ -0,0 +1,337 @@
// C++ conformance test for the native Shard protocol.
//
// This test does not check that C++ can round-trip its own output — that would
// only prove C++ is self-consistent. It parses the *Python-produced* committed
// vectors, asserts every field the protocol promises to carry, independently
// recomputes the CRC32C over the reassembled tensor, and re-serializes the
// message back out for the Python test to compare byte-for-byte. Only that
// closes the loop: both languages agree on the same bytes and the same meaning.
//
// Run via ctest; see packages/node/native/CMakeLists.txt.
#include "shard_runtime.pb.h"
#include <cstdint>
#include <cstdio>
#include <filesystem>
#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
#include <vector>
// `pb` is already taken by protobuf's own generated headers.
namespace sp = meshnet::shard::v1;
namespace {
int g_failures = 0;
#define CHECK(cond) \
do { \
if (!(cond)) { \
std::cerr << "FAIL " << __FILE__ << ":" << __LINE__ << ": " << #cond \
<< "\n"; \
++g_failures; \
} \
} while (0)
#define CHECK_EQ(actual, expected) \
do { \
auto &&a_ = (actual); \
auto &&e_ = (expected); \
if (!(a_ == e_)) { \
std::cerr << "FAIL " << __FILE__ << ":" << __LINE__ << ": " << #actual \
<< " == " << #expected << "\n actual: " << a_ \
<< "\n expected: " << e_ << "\n"; \
++g_failures; \
} \
} while (0)
// Independent CRC32C (Castagnoli). Written from the polynomial rather than
// shared with the Python side on purpose: a checksum that both languages
// compute with the *same* code proves nothing about interoperability.
uint32_t Crc32c(const std::string &data) {
static uint32_t table[256];
static bool built = false;
if (!built) {
for (uint32_t i = 0; i < 256; ++i) {
uint32_t c = i;
for (int k = 0; k < 8; ++k) {
c = (c & 1) ? (c >> 1) ^ 0x82F63B78u : (c >> 1);
}
table[i] = c;
}
built = true;
}
uint32_t crc = 0xFFFFFFFFu;
for (unsigned char byte : data) {
crc = (crc >> 8) ^ table[(crc ^ byte) & 0xFF];
}
return crc ^ 0xFFFFFFFFu;
}
std::string ReadFile(const std::filesystem::path &path) {
std::ifstream in(path, std::ios::binary);
if (!in) {
std::cerr << "FAIL cannot read " << path << "\n";
++g_failures;
return {};
}
std::ostringstream buffer;
buffer << in.rdbuf();
return buffer.str();
}
// Reassemble a tensor's fragments and validate coverage, exactly as a worker
// must before it feeds the bytes to a forward pass.
std::string ReassembleUncompressed(const sp::NamedTensor &tensor) {
std::string body;
std::vector<const sp::TensorFragment *> fragments;
for (const auto &fragment : tensor.fragments()) {
fragments.push_back(&fragment);
}
CHECK(!fragments.empty());
for (uint32_t index = 0; index < fragments.size(); ++index) {
const sp::TensorFragment *found = nullptr;
for (const auto *fragment : fragments) {
if (fragment->fragment_index() == index) {
found = fragment;
break;
}
}
CHECK(found != nullptr);
if (found == nullptr) {
return {};
}
CHECK_EQ(found->fragment_count(),
static_cast<uint32_t>(fragments.size()));
// Offsets must tile the wire body exactly: no hole, no overlap.
CHECK_EQ(found->byte_offset(), static_cast<uint64_t>(body.size()));
body.append(found->payload());
}
return body;
}
void CheckFingerprint(const sp::Fingerprint &fingerprint) {
CHECK_EQ(fingerprint.model_artifact_digest(), std::string("sha256:1f0c9d2e"));
CHECK_EQ(fingerprint.runtime_recipe_digest(), std::string("sha256:ab77e410"));
CHECK_EQ(fingerprint.recipe_id(), std::string("llama-gguf-q4km-rocm"));
CHECK_EQ(fingerprint.recipe_version(), std::string("3"));
CHECK_EQ(fingerprint.catalogue_version(), std::string("2026.07.1"));
}
// The canonical session request, as produced by Python.
void TestSessionRequestVector(const std::string &bytes) {
sp::SessionRequest request;
CHECK(request.ParseFromString(bytes));
CHECK(request.kind_case() == sp::SessionRequest::kChunk);
const sp::ActivationChunk &chunk = request.chunk();
const sp::Envelope &envelope = chunk.envelope();
// Every field the protocol promises to carry (acceptance criterion 4).
CHECK_EQ(envelope.schema_version(), sp::SCHEMA_VERSION_1);
CHECK_EQ(envelope.work_id(), std::string("work-7f3a"));
CHECK_EQ(envelope.route_session_id(), std::string("rs-2b91"));
CHECK_EQ(envelope.route_epoch(), 7u);
CheckFingerprint(envelope.fingerprint());
CHECK_EQ(envelope.shard_range().start_layer(), 12u);
CHECK_EQ(envelope.shard_range().end_layer(), 24u);
// Overlap-safe effective start (ADR-0012): a worker that begins at
// start_layer instead would re-apply layers 12..15.
CHECK_EQ(envelope.shard_range().effective_start_layer(), 16u);
CHECK_EQ(envelope.phase(), sp::PHASE_PREFILL);
CHECK_EQ(envelope.position().first_position(), 256u);
CHECK_EQ(envelope.position().token_count(), 128u);
CHECK_EQ(envelope.idempotency_step(), 42u);
CHECK_EQ(envelope.cache_expectation().mode(), sp::CACHE_MODE_PREFILL);
CHECK_EQ(envelope.cache_expectation().expected_past_len(), 256u);
CHECK_EQ(envelope.deadline_unix_nanos(), 1800000000000000000LL);
CHECK_EQ(envelope.chunk().chunk_index(), 1u);
CHECK_EQ(envelope.chunk().chunk_count(), 3u);
CHECK_EQ(envelope.chunk().final_chunk(), false);
// The versioned named-tensor bundle (acceptance criterion 5).
const sp::TensorBundle &bundle = chunk.bundle();
CHECK_EQ(bundle.bundle_version(), 1u);
CHECK_EQ(bundle.tensors_size(), 1);
const sp::NamedTensor &tensor = bundle.tensors(0);
CHECK_EQ(tensor.name(), std::string("hidden_states"));
CHECK_EQ(tensor.shape_size(), 3);
CHECK_EQ(tensor.shape(0), 1);
CHECK_EQ(tensor.shape(1), 128);
CHECK_EQ(tensor.shape(2), 8);
CHECK_EQ(tensor.dtype(), sp::DTYPE_BFLOAT16);
CHECK_EQ(tensor.byte_order(), sp::BYTE_ORDER_LITTLE_ENDIAN);
CHECK_EQ(tensor.total_bytes(), 1u * 128u * 8u * 2u);
CHECK_EQ(tensor.compression(), sp::COMPRESSION_NONE);
// Multi-fragment on purpose: reassembly is what a worker actually does.
CHECK(tensor.fragments_size() > 1);
const std::string payload = ReassembleUncompressed(tensor);
CHECK_EQ(payload.size(), static_cast<size_t>(tensor.total_bytes()));
// The payload Python generated, recomputed here from its rule.
std::string expected;
expected.reserve(payload.size());
for (size_t i = 0; i < payload.size(); ++i) {
expected.push_back(static_cast<char>((i * 7 + 11) % 256));
}
CHECK(payload == expected);
// Checksum: computed by Python, verified by an independent C++ CRC32C.
CHECK_EQ(tensor.checksum().algorithm(), sp::CHECKSUM_ALGORITHM_CRC32C);
const std::string &checksum = tensor.checksum().value();
CHECK_EQ(checksum.size(), 4u);
if (checksum.size() == 4) {
const uint32_t actual = Crc32c(payload);
const uint32_t declared =
(static_cast<uint32_t>(static_cast<unsigned char>(checksum[0])) << 24) |
(static_cast<uint32_t>(static_cast<unsigned char>(checksum[1])) << 16) |
(static_cast<uint32_t>(static_cast<unsigned char>(checksum[2])) << 8) |
static_cast<uint32_t>(static_cast<unsigned char>(checksum[3]));
CHECK_EQ(actual, declared);
}
}
void TestCapabilityReportVector(const std::string &bytes) {
sp::CapabilityReport report;
CHECK(report.ParseFromString(bytes));
CHECK_EQ(report.schema_version(), sp::SCHEMA_VERSION_1);
CheckFingerprint(report.fingerprint());
CHECK_EQ(report.backend(), std::string("rocm"));
CHECK_EQ(report.device(), std::string("gfx1151"));
CHECK_EQ(report.validated(), true);
CHECK_EQ(report.max_concurrent_sessions(), 4u);
CHECK_EQ(report.max_context_tokens(), 8192u);
CHECK_EQ(report.flow_control().max_prefill_chunk_tokens(), 128u);
CHECK_EQ(report.flow_control().max_chunk_bytes(), 4u * 1024u * 1024u);
CHECK_EQ(report.accepted_compression_size(), 2);
CHECK_EQ(report.supported_schema_versions_size(), 1);
}
// Forward compatibility: a field this build has never heard of must survive a
// parse/serialize cycle untouched. Without this, an old Shard silently strips
// fields a newer peer depends on as it forwards activations down the route.
void TestUnknownFieldsArePreserved(const std::string &bytes) {
// Field 9999, wire type 0 (varint), value 12345 — not in this schema.
std::string forward = bytes;
forward.push_back(static_cast<char>(0xB8)); // tag: (9999 << 3) | 0
forward.push_back(static_cast<char>(0xE0));
forward.push_back(static_cast<char>(0x04));
forward.push_back(static_cast<char>(0xB9)); // value: 12345
forward.push_back(static_cast<char>(0x60));
sp::SessionRequest request;
CHECK(request.ParseFromString(forward));
// Known fields still parse.
CHECK_EQ(request.chunk().envelope().work_id(), std::string("work-7f3a"));
// And the unknown field is retained rather than dropped.
CHECK(!request.GetReflection()->GetUnknownFields(request).empty());
std::string reserialized;
CHECK(request.SerializeToString(&reserialized));
CHECK_EQ(reserialized.size(), forward.size());
sp::SessionRequest reparsed;
CHECK(reparsed.ParseFromString(reserialized));
CHECK(!reparsed.GetReflection()->GetUnknownFields(reparsed).empty());
}
// Backward compatibility: a message from a peer that sets none of the optional
// groups must still parse, yielding proto3 defaults rather than an error.
void TestSparseMessageParses() {
sp::SessionRequest minimal;
minimal.mutable_chunk()->mutable_envelope()->set_work_id("w");
std::string bytes;
CHECK(minimal.SerializeToString(&bytes));
sp::SessionRequest parsed;
CHECK(parsed.ParseFromString(bytes));
CHECK_EQ(parsed.chunk().envelope().work_id(), std::string("w"));
CHECK_EQ(parsed.chunk().envelope().route_epoch(), 0u);
CHECK_EQ(parsed.chunk().envelope().phase(), sp::PHASE_UNSPECIFIED);
CHECK_EQ(parsed.chunk().bundle().tensors_size(), 0);
}
// The decode fast path must stay small: repeating the full envelope per token
// is pure overhead on the hottest path in the system.
void TestDecodeFastPathIsSmall() {
sp::SessionRequest decode;
sp::DecodeStep *step = decode.mutable_decode();
step->set_idempotency_step(9);
step->set_position(1024);
step->set_expected_past_len(1024);
step->set_work_id("work-7f3a");
sp::NamedTensor *tensor = step->mutable_tensor();
tensor->set_name("hidden_states");
tensor->add_shape(1);
tensor->add_shape(1);
tensor->add_shape(8);
tensor->set_dtype(sp::DTYPE_BFLOAT16);
tensor->set_byte_order(sp::BYTE_ORDER_LITTLE_ENDIAN);
tensor->set_total_bytes(16);
tensor->set_compression(sp::COMPRESSION_NONE);
sp::TensorFragment *fragment = tensor->add_fragments();
fragment->set_fragment_index(0);
fragment->set_fragment_count(1);
fragment->set_byte_offset(0);
fragment->set_payload(std::string(16, '\x01'));
std::string bytes;
CHECK(decode.SerializeToString(&bytes));
// Payload is 16 bytes; the framing around it must stay well under the
// envelope-carrying prefill path.
CHECK(bytes.size() < 96);
sp::SessionRequest parsed;
CHECK(parsed.ParseFromString(bytes));
CHECK(parsed.kind_case() == sp::SessionRequest::kDecode);
CHECK_EQ(parsed.decode().position(), 1024u);
CHECK_EQ(parsed.decode().tensor().total_bytes(), 16u);
}
} // namespace
int main(int argc, char **argv) {
GOOGLE_PROTOBUF_VERIFY_VERSION;
if (argc < 3) {
std::cerr << "usage: " << argv[0] << " <testdata-dir> <output-dir>\n";
return 2;
}
const std::filesystem::path testdata(argv[1]);
const std::filesystem::path output(argv[2]);
const std::string session_bytes =
ReadFile(testdata / "session_request_golden.binpb");
const std::string capability_bytes =
ReadFile(testdata / "capability_report_golden.binpb");
TestSessionRequestVector(session_bytes);
TestCapabilityReportVector(capability_bytes);
TestUnknownFieldsArePreserved(session_bytes);
TestSparseMessageParses();
TestDecodeFastPathIsSmall();
// Re-serialize the canonical message from the C++ object model and hand it
// back for Python to compare against the golden bytes. If the two languages
// disagreed about any field's encoding, this file would differ.
sp::SessionRequest request;
if (request.ParseFromString(session_bytes)) {
std::string reserialized;
if (request.SerializeToString(&reserialized)) {
std::ofstream out(output / "cpp_roundtrip.binpb", std::ios::binary);
out.write(reserialized.data(),
static_cast<std::streamsize>(reserialized.size()));
}
}
if (g_failures != 0) {
std::cerr << g_failures << " check(s) failed\n";
return 1;
}
std::cout << "all C++ conformance checks passed\n";
return 0;
}

11
packages/node/pyproject.toml
View File

@@ -13,6 +13,11 @@ dependencies = [
"huggingface-hub>=0.20",
"accelerate>=0.28",
"bitsandbytes>=0.43",
# Keep these floors aligned with the committed grpcio-tools 1.82.1 output.
# Older runtimes satisfy broad constraints but fail while importing the
# generated modules before a node can report a useful protocol error.
"grpcio>=1.82.1",
"protobuf>=7.35.0",
"rich>=13",
"safetensors>=0.4",
"torch>=2.1",
@@ -23,6 +28,11 @@ dependencies = [
"kernels>=0.11.1,<0.16",
]
[project.optional-dependencies]
# Regenerating the native Shard protocol stubs. Not needed to run a node: the
# generated modules are committed, and `grpcio-tools` bundles its own protoc.
proto = ["grpcio-tools==1.82.1"]
[project.scripts]
meshnet-node = "meshnet_node.cli:main"
@@ -32,3 +42,4 @@ include = ["meshnet_node*"]
[tool.setuptools.package-data]
meshnet_node = ["*.json"]
"meshnet_node.glm_alpha" = ["data/*.json"]

102
packages/tracker/meshnet_tracker/capability.py
View File

@@ -30,6 +30,13 @@ import time
from dataclasses import dataclass, replace
from typing import Any, Callable, Mapping
from .recipe import (
CertificationLedger,
FingerprintMismatch,
RecipeIdentityError,
parse_identity,
)
# The capability report layout this tracker reads (meshnet_node.capability).
SUPPORTED_SCHEMA_VERSION = 1
@@ -58,6 +65,12 @@ STATE_MODEL_MISMATCH = "model-mismatch"
STATE_SHARD_MISMATCH = "shard-mismatch"
STATE_RECIPE_MISMATCH = "recipe-mismatch"
STATE_CATALOGUE_INCOMPATIBLE = "catalogue-incompatible"
# The node presented a DGR-003 identity block whose declared fingerprint is not
# the digest of the axes it declared. Identity is derived, never asserted.
STATE_FINGERPRINT_MISMATCH = "fingerprint-mismatch"
# Registered-but-dark: a known recipe no real distributed forward has certified.
# Visible to an operator, never routable for user traffic.
STATE_UNCERTIFIED = "uncertified"
ALL_STATES = (
STATE_ADMITTED,
@@ -69,6 +82,8 @@ ALL_STATES = (
STATE_SHARD_MISMATCH,
STATE_RECIPE_MISMATCH,
STATE_CATALOGUE_INCOMPATIBLE,
STATE_FINGERPRINT_MISMATCH,
STATE_UNCERTIFIED,
)
# --- Compatibility policy for nodes that predate the capability protocol. ---
@@ -165,12 +180,25 @@ class CapabilityState:
recorded_at: float = 0.0
schema_version: int | None = None
diagnostics: tuple[str, ...] = ()
# The DGR-003 compatibility fingerprint, *re-derived* by this tracker from
# the axes the node declared — never copied from what the node claimed.
# Absent for a node that predates DGR-003.
model_artifact_digest: str | None = None
runtime_recipe_digest: str | None = None
certification: str | None = None
@property
def proven(self) -> bool:
"""The presented proof covers exactly what the node advertised."""
return self.state == STATE_ADMITTED
@property
def fingerprint(self) -> tuple[str, str] | None:
"""What route formation compares. `None` when the node declares no identity."""
if self.model_artifact_digest is None or self.runtime_recipe_digest is None:
return None
return (self.model_artifact_digest, self.runtime_recipe_digest)
def routable_under(self, policy: str) -> bool:
if self.proven:
return True
@@ -197,6 +225,9 @@ class CapabilityState:
"recorded_at": self.recorded_at,
"schema_version": self.schema_version,
"diagnostics": list(self.diagnostics),
"model_artifact_digest": self.model_artifact_digest,
"runtime_recipe_digest": self.runtime_recipe_digest,
"certification": self.certification,
}
@@ -224,6 +255,7 @@ def evaluate_report(
declared_recipe_version: str | None = None,
now: float | None = None,
max_age_seconds: float = DEFAULT_MAX_REPORT_AGE_SECONDS,
ledger: CertificationLedger | None = None,
) -> CapabilityState:
"""Judge the proof a node presented against what that node is advertising.
@@ -308,6 +340,67 @@ def evaluate_report(
f"the node declared v{declared_recipe_version}",
)
identity = None
if report.get("identity") is not None:
try:
identity = parse_identity(report["identity"])
except FingerprintMismatch as exc:
return base.with_state(STATE_FINGERPRINT_MISMATCH, str(exc))
except RecipeIdentityError as exc:
return base.with_state(
STATE_INVALID, f"capability identity block is unusable: {exc}"
)
# The report's `shard` range is inclusive/inclusive (the CLI and backend
# convention); the identity's is inclusive/exclusive (the protocol's, and
# ADR-0012's). A node whose two halves disagree has not proven the range
# it claims, whichever one is right.
if (identity.shard_start, identity.shard_end) != (
base.shard_start,
(base.shard_end or 0) + 1,
):
return base.with_state(
STATE_SHARD_MISMATCH,
f"identity covers layers {identity.shard_start}–{identity.shard_end} "
f"(end-exclusive), but the proof is for layers {base.shard_start}–"
f"{base.shard_end} (end-inclusive)",
)
if not model_matches(identity.artifact_id):
return base.with_state(
STATE_MODEL_MISMATCH,
f"identity is for artifact {identity.artifact_id!r}, but the node "
f"registered {advertised_model!r}",
)
if (
identity.recipe_id != base.recipe_id
or identity.recipe_version != base.recipe_version
or identity.catalogue_version != base.catalogue_version
):
return base.with_state(
STATE_RECIPE_MISMATCH,
"identity recipe labels do not match the capability proof",
)
if identity.axes["backend_id"] != base.backend_id:
return base.with_state(
STATE_RECIPE_MISMATCH,
"identity backend does not match the capability proof",
)
if (
base.quantization is not None
and identity.axes["weight_quantization"] != base.quantization
):
return base.with_state(
STATE_RECIPE_MISMATCH,
"identity weight quantization does not match the capability proof",
)
base = replace(
base,
model_artifact_digest=identity.model_artifact_digest,
runtime_recipe_digest=identity.runtime_recipe_digest,
)
if status != STATUS_PASSED:
return base.with_state(
STATE_FAILED,
@@ -328,6 +421,15 @@ def evaluate_report(
f"proof is timestamped {-age:.0f}s in the future; check the node's clock",
)
# The digest only establishes that this report is self-consistent. It does
# not authenticate a node or prove a distributed forward. Exact recipes are
# therefore registered-but-dark until tracker-owned certification records
# that forward.
if identity is not None:
recipe_status = (ledger or CertificationLedger()).register(identity)
if not recipe_status.may_serve:
return base.with_state(STATE_UNCERTIFIED, recipe_status.detail)
return base.with_state(
STATE_ADMITTED,
f"{base.model_id} layers {base.shard_start}–{base.shard_end} proven on "

615
packages/tracker/meshnet_tracker/recipe.py Normal file
View File

@@ -0,0 +1,615 @@
"""Tracker-side artifact and runtime recipe identity (DGR-003).
The node computes a compatibility fingerprint in `meshnet_node.runtime_recipe`.
This module recomputes it, and the recomputation is the entire point.
A fingerprint that arrives on the wire is a **claim**. If the tracker stored
what a node asserted, a node could assert the digest of a certified recipe while
running an uncertified one — and admission, route selection, and the gRPC
handshake would all wave it through, because they all compare the digest it
handed them. So the tracker derives the digest from the *axes* the node
declared, and refuses any report whose claim does not match the derivation. A
node can lie about its axes, and a real forward will catch that; it cannot lie
about the digest of the axes it declared.
This module deliberately does **not** import `meshnet_node`: the tracker package
does not depend on the node package, and an admission gate that shares an
implementation with the thing it admits is not an independent check. The two
implementations are pinned together by a committed conformance vector
(``tests/data/recipe_fingerprint_vectors.json``). If they drift, a test fails —
rather than a route quietly failing to form, or worse, quietly forming.
Recipes arrive **dark**: registered, visible to an operator, and not routable for
user traffic. Only a real distributed forward — at least two distinct nodes,
covering the whole model, emitting real tokens — takes a recipe out of the dark
(:class:`CertificationLedger`).
"""
from __future__ import annotations
import hashlib
import json
import re
import time
from dataclasses import dataclass, field
from typing import Any, Iterable, Mapping, Sequence
# Layout of the identity block this tracker reads (meshnet_node.runtime_recipe).
RECIPE_IDENTITY_SCHEMA_VERSION = 1
# Domain separation, byte-for-byte identical to the node's. These strings are
# part of the wire contract, not an implementation detail: changing one here
# without changing it there silently partitions every route.
ARTIFACT_DIGEST_DOMAIN = "meshnet.model-artifact.v1"
RECIPE_DIGEST_DOMAIN = "meshnet.runtime-recipe.v1"
# The axes a recipe digest commits to. Order is irrelevant (the canonical JSON
# sorts keys); membership is not — an axis missing here is an axis the tracker
# would let a node change without changing its identity.
RECIPE_AXES: tuple[str, ...] = (
"weight_quantization",
"activation_dtype",
"compute_dtype",
"kv_dtype",
"kv_layout",
"tokenizer_revision",
"architecture_adapter",
"backend_id",
"runtime_version",
"boundary_schema_version",
"protocol_schema_version",
)
_INT_AXES = frozenset({"boundary_schema_version", "protocol_schema_version"})
MISMATCH_ARTIFACT = "artifact"
MISMATCH_TOKENIZER = "tokenizer"
MISMATCH_ARCHITECTURE = "architecture"
MISMATCH_RANGE = "range"
MISMATCH_BOUNDARY_SCHEMA = "boundary-schema"
MISMATCH_ACTIVATION_RECIPE = "activation-recipe"
MISMATCH_CACHE_LAYOUT = "cache-layout"
MISMATCH_FINGERPRINT = "fingerprint"
MISMATCH_UNCERTIFIED = "uncertified"
STATUS_UNKNOWN = "unknown"
STATUS_DARK = "dark"
STATUS_CERTIFIED = "certified"
MIN_CERTIFYING_NODES = 2
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
_MOVING_REFS = frozenset({"main", "master", "head", "latest", "dev", "trunk"})
class RecipeIdentityError(ValueError):
"""A presented identity block is malformed or internally inconsistent."""
class FingerprintMismatch(RecipeIdentityError):
"""A supplied digest is inconsistent with its identity declaration.
A digest is an integrity and compatibility claim, not an authenticated
statement about what a node is actually executing. Distributed
certification remains the trust boundary.
"""
def canonical_sha256(value: Any) -> str:
payload = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _digest(domain: str, body: Mapping[str, Any]) -> str:
return canonical_sha256({"domain": domain, "body": dict(body)})
def _text(value: Any, what: str) -> str:
if not isinstance(value, str) or not value.strip():
raise RecipeIdentityError(f"{what!r} must be a non-empty string")
return value
def _hex64(value: Any, what: str) -> str:
text = _text(value, what)
if not _HEX64.match(text):
raise RecipeIdentityError(f"{what!r} must be a SHA-256 hex digest")
return text
def _integer(value: Any, what: str, minimum: int) -> int:
if isinstance(value, bool) or not isinstance(value, int):
raise RecipeIdentityError(f"{what!r} must be an integer")
if value < minimum:
raise RecipeIdentityError(f"{what!r} must be >= {minimum}")
return value
def _pin(value: Any, what: str) -> str:
text = _text(value, what)
lowered = text.strip().lower()
if lowered in _MOVING_REFS or lowered.startswith("refs/"):
raise RecipeIdentityError(
f"{what!r} is a moving reference, not an exact revision pin"
)
return text
def _mapping(value: Any, what: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise RecipeIdentityError(f"{what!r} must be a JSON object")
return value
def artifact_digest(
*,
source_digest: str,
architecture: str,
architecture_digest: str,
layer_count: int,
) -> str:
"""The artifact half of the fingerprint, derived exactly as the node derives it."""
return _digest(
ARTIFACT_DIGEST_DOMAIN,
{
"source_digest": source_digest,
"architecture": architecture,
"architecture_digest": architecture_digest,
"layer_count": layer_count,
},
)
def recipe_digest(axes: Mapping[str, Any]) -> str:
"""The recipe half of the fingerprint, derived exactly as the node derives it."""
missing = [axis for axis in RECIPE_AXES if axis not in axes]
if missing:
raise RecipeIdentityError(
"recipe is missing required axes: " + ", ".join(missing)
)
return _digest(RECIPE_DIGEST_DOMAIN, {axis: axes[axis] for axis in RECIPE_AXES})
@dataclass(frozen=True)
class PresentedIdentity:
"""One node's declared artifact/recipe identity, with digests re-derived here.
`model_artifact_digest` and `runtime_recipe_digest` are computed by this
tracker from the declared axes. They are never copied from the report.
"""
artifact_id: str
revision: str
source_digest: str
architecture: str
architecture_digest: str
layer_count: int
is_derivative: bool
shard_start: int
shard_end: int
axes: Mapping[str, Any]
recipe_id: str
recipe_version: str
catalogue_version: str
@property
def model_artifact_digest(self) -> str:
return artifact_digest(
source_digest=self.source_digest,
architecture=self.architecture,
architecture_digest=self.architecture_digest,
layer_count=self.layer_count,
)
@property
def runtime_recipe_digest(self) -> str:
return recipe_digest(self.axes)
@property
def key(self) -> tuple[str, str]:
return (self.model_artifact_digest, self.runtime_recipe_digest)
@property
def tokenizer_revision(self) -> str:
return str(self.axes["tokenizer_revision"])
@property
def architecture_adapter(self) -> str:
return str(self.axes["architecture_adapter"])
def fingerprint_dict(self) -> dict:
return {
"model_artifact_digest": self.model_artifact_digest,
"runtime_recipe_digest": self.runtime_recipe_digest,
"recipe_id": self.recipe_id,
"recipe_version": self.recipe_version,
"catalogue_version": self.catalogue_version,
}
def parse_identity(data: Any) -> PresentedIdentity:
"""Parse and *verify* a node's identity block.
Raises when the block is malformed, when a split artifact does not name the
exact source it was cut from, when a Shard advertises layers its artifact
does not contain, or when the declared fingerprint does not match the digest
of the axes it was declared with.
"""
doc = _mapping(data, "identity")
schema_version = doc.get("schema_version")
if schema_version != RECIPE_IDENTITY_SCHEMA_VERSION:
raise RecipeIdentityError(
f"identity block declares schema version {schema_version!r}; this tracker "
f"reads version {RECIPE_IDENTITY_SCHEMA_VERSION}"
)
artifact = _mapping(doc.get("artifact"), "identity.artifact")
recipe = _mapping(doc.get("recipe"), "identity.recipe")
layer_count = _integer(artifact.get("layer_count"), "artifact.layer_count", 1)
content_digest = _hex64(artifact.get("content_digest"), "artifact.content_digest")
raw_binding = artifact.get("derived_from")
if raw_binding is None:
source_digest = content_digest
binding: tuple[int, int] | None = None
else:
derived = _mapping(raw_binding, "artifact.derived_from")
source_digest = _hex64(
derived.get("source_artifact_digest"),
"artifact.derived_from.source_artifact_digest",
)
binding = (
_integer(derived.get("shard_start"), "derived_from.shard_start", 0),
_integer(derived.get("shard_end"), "derived_from.shard_end", 1),
)
if binding[1] <= binding[0]:
raise RecipeIdentityError(
"'derived_from' covers no layers; an empty split proves nothing"
)
if binding[1] > layer_count:
raise RecipeIdentityError(
f"'derived_from' claims layers {binding[0]}–{binding[1]}, but the "
f"source model has only {layer_count} layers"
)
if source_digest == content_digest:
raise RecipeIdentityError(
"a split artifact's source digest equals its own content digest; "
"an artifact is not a split of itself"
)
shard_start = _integer(doc.get("shard_start"), "shard_start", 0)
shard_end = _integer(doc.get("shard_end"), "shard_end", 1)
if shard_end <= shard_start:
raise RecipeIdentityError("a Shard owning no layer computes nothing")
if shard_end > layer_count:
raise RecipeIdentityError(
f"Shard owns layers {shard_start}–{shard_end}, but the artifact has only "
f"{layer_count} layers"
)
if binding is not None and not (
binding[0] <= shard_start and shard_end <= binding[1]
):
raise RecipeIdentityError(
f"Shard advertises layers {shard_start}–{shard_end}, but its split "
f"artifact only contains layers {binding[0]}–{binding[1]}"
)
axes: dict[str, Any] = {}
for axis in RECIPE_AXES:
if axis not in recipe:
raise RecipeIdentityError(
f"recipe is missing axis {axis!r}; an unstated axis cannot default"
)
value = recipe[axis]
if axis in _INT_AXES:
axes[axis] = _integer(value, f"recipe.{axis}", 1)
else:
axes[axis] = _text(value, f"recipe.{axis}")
_pin(axes["tokenizer_revision"], "recipe.tokenizer_revision")
identity = PresentedIdentity(
artifact_id=_text(artifact.get("artifact_id"), "artifact.artifact_id"),
revision=_pin(artifact.get("revision"), "artifact.revision"),
source_digest=source_digest,
architecture=_text(artifact.get("architecture"), "artifact.architecture"),
architecture_digest=_hex64(
artifact.get("architecture_digest"), "artifact.architecture_digest"
),
layer_count=layer_count,
is_derivative=binding is not None,
shard_start=shard_start,
shard_end=shard_end,
axes=axes,
recipe_id=_text(recipe.get("recipe_id"), "recipe.recipe_id"),
recipe_version=_text(recipe.get("recipe_version"), "recipe.recipe_version"),
catalogue_version=_text(
recipe.get("catalogue_version"), "recipe.catalogue_version"
),
)
declared = doc.get("fingerprint")
if declared is not None:
claim = _mapping(declared, "identity.fingerprint")
claimed_artifact = _hex64(
claim.get("model_artifact_digest"), "fingerprint.model_artifact_digest"
)
claimed_recipe = _hex64(
claim.get("runtime_recipe_digest"), "fingerprint.runtime_recipe_digest"
)
if (claimed_artifact, claimed_recipe) != identity.key:
raise FingerprintMismatch(
"declared fingerprint is inconsistent with the artifact and recipe "
"claim; the tracker recomputes compatibility digests"
)
return identity
@dataclass(frozen=True)
class RouteMismatch:
reason: str
detail: str
def describe(self) -> str:
return f"{self.reason}: {self.detail}"
def compare(
route: PresentedIdentity, candidate: PresentedIdentity
) -> tuple[RouteMismatch, ...]:
"""Why `candidate` may not join a route already running `route`."""
if route.key == candidate.key:
return ()
reasons: list[RouteMismatch] = []
if route.source_digest != candidate.source_digest:
reasons.append(
RouteMismatch(
MISMATCH_ARTIFACT,
f"serves Model Artifact {candidate.source_digest[:12]}…, the route "
f"runs {route.source_digest[:12]}…",
)
)
if route.architecture_digest != candidate.architecture_digest:
reasons.append(
RouteMismatch(
MISMATCH_ARCHITECTURE,
"architecture/config snapshot differs from the route's",
)
)
if route.layer_count != candidate.layer_count:
reasons.append(
RouteMismatch(
MISMATCH_ARTIFACT,
f"artifact has {candidate.layer_count} layers, the route's has "
f"{route.layer_count}",
)
)
axis_reason = {
"tokenizer_revision": MISMATCH_TOKENIZER,
"architecture_adapter": MISMATCH_ARCHITECTURE,
"activation_dtype": MISMATCH_ACTIVATION_RECIPE,
"compute_dtype": MISMATCH_ACTIVATION_RECIPE,
"kv_dtype": MISMATCH_CACHE_LAYOUT,
"kv_layout": MISMATCH_CACHE_LAYOUT,
"boundary_schema_version": MISMATCH_BOUNDARY_SCHEMA,
"protocol_schema_version": MISMATCH_BOUNDARY_SCHEMA,
}
for axis in RECIPE_AXES:
mine = route.axes.get(axis)
theirs = candidate.axes.get(axis)
if mine != theirs:
reasons.append(
RouteMismatch(
axis_reason.get(axis, MISMATCH_FINGERPRINT),
f"{axis} is {theirs!r}, the route runs {mine!r}",
)
)
if not reasons:
reasons.append(
RouteMismatch(
MISMATCH_FINGERPRINT,
"fingerprints differ on a field this comparison does not cover",
)
)
return tuple(reasons)
def coverage_gap(
ranges: Iterable[tuple[int, int]], layer_count: int
) -> str | None:
"""Why `ranges` fail to tile ``[0, layer_count)``, or None when they do.
Overlaps are legal — ADR-0012 lets the Tracker resolve one by telling a hop
where the previous hop stopped. A hole is not: its layers are never computed,
and the route emits fluent tokens from a truncated model.
"""
ordered = sorted(ranges)
if not ordered:
return "the route owns no layers"
if ordered[0][0] != 0:
return f"layers 0–{ordered[0][0]} are owned by no Shard on the route"
covered = 0
for start, end in ordered:
if start > covered:
return f"layers {covered}–{start} are owned by no Shard on the route"
covered = max(covered, end)
if covered < layer_count:
return f"layers {covered}–{layer_count} are owned by no Shard on the route"
if covered > layer_count:
return (
f"the route claims {covered} layers, but the model has only {layer_count}"
)
return None
@dataclass(frozen=True)
class DistributedForwardEvidence:
"""A real distributed forward — the only thing that certifies a recipe."""
route_session_id: str
route_epoch: int
node_ids: tuple[str, ...]
shard_ranges: tuple[tuple[int, int], ...]
tokens_generated: int
layer_count: int
fingerprint: tuple[str, str]
synthetic: bool = False
certified_at: float = field(default_factory=time.time)
def rejection(self) -> str | None:
if self.synthetic:
return (
"evidence comes from a synthetic worker; only a real distributed "
"forward certifies a recipe"
)
distinct = set(self.node_ids)
if len(distinct) < MIN_CERTIFYING_NODES:
return (
f"evidence covers {len(distinct)} distinct node(s); a distributed "
f"forward requires at least {MIN_CERTIFYING_NODES}"
)
if len(distinct) != len(self.node_ids):
return "the same node is counted more than once in the certifying route"
if self.tokens_generated < 1:
return "the certifying forward generated no tokens"
return coverage_gap(self.shard_ranges, self.layer_count)
@dataclass(frozen=True)
class RecipeStatus:
status: str
detail: str = ""
certified_at: float | None = None
@property
def may_serve(self) -> bool:
return self.status == STATUS_CERTIFIED
@property
def may_certify(self) -> bool:
"""A dark recipe may be routed to *certify* it, and for nothing else.
Without this, certification is unreachable by construction: serving needs
certification, certification needs a real distributed forward, and a real
distributed forward needs a route.
"""
return self.status in (STATUS_DARK, STATUS_CERTIFIED)
def to_dict(self) -> dict:
return {
"status": self.status,
"detail": self.detail,
"certified_at": self.certified_at,
}
_DARK_DETAIL = "registered; dark until a real distributed forward certifies it"
_UNKNOWN_DETAIL = "this recipe has never been registered with the tracker"
class CertificationLedger:
"""Which recipes the tracker has seen, and which a real forward has proven."""
def __init__(self) -> None:
self._dark: set[tuple[str, str]] = set()
self._certified: dict[tuple[str, str], RecipeStatus] = {}
def register(self, identity: PresentedIdentity) -> RecipeStatus:
key = identity.key
if key in self._certified:
return self._certified[key]
self._dark.add(key)
return RecipeStatus(STATUS_DARK, _DARK_DETAIL)
def status(self, identity: PresentedIdentity | tuple[str, str]) -> RecipeStatus:
key = identity if isinstance(identity, tuple) else identity.key
if key in self._certified:
return self._certified[key]
if key in self._dark:
return RecipeStatus(STATUS_DARK, _DARK_DETAIL)
return RecipeStatus(STATUS_UNKNOWN, _UNKNOWN_DETAIL)
def certify(
self,
identity: PresentedIdentity | tuple[str, str],
evidence: DistributedForwardEvidence,
) -> RecipeStatus:
"""Promote a recipe out of the dark, or raise saying why the evidence is short."""
key = identity if isinstance(identity, tuple) else identity.key
if key not in self._dark and key not in self._certified:
raise RecipeIdentityError(
"this fingerprint is not registered; unknown recipes cannot be certified"
)
if evidence.fingerprint != key:
raise RecipeIdentityError(
"certification evidence fingerprint does not match the recipe being promoted"
)
rejection = evidence.rejection()
if rejection is not None:
raise RecipeIdentityError(f"this evidence does not certify: {rejection}")
status = RecipeStatus(
STATUS_CERTIFIED,
(
f"certified by route session {evidence.route_session_id} across "
f"{len(set(evidence.node_ids))} nodes"
),
certified_at=evidence.certified_at,
)
self._certified[key] = status
self._dark.discard(key)
return status
def to_dict(self) -> dict:
return {
"dark": [list(key) for key in sorted(self._dark)],
"certified": {
"/".join(key): status.to_dict()
for key, status in sorted(self._certified.items())
},
}
def admit_route(
identities: Sequence[PresentedIdentity],
*,
ledger: CertificationLedger | None = None,
for_certification: bool = False,
) -> tuple[RouteMismatch, ...]:
"""Every reason these Shards may not form one Inference Route; empty when they may.
Fail-closed by construction: an empty route, a hole in the coverage, one
differing dtype, and an uncertified recipe each produce a reason, so a caller
that proceeds only on an empty result cannot admit any of them.
"""
if not identities:
return (RouteMismatch(MISMATCH_RANGE, "a route needs at least one Shard"),)
head = identities[0]
reasons: list[RouteMismatch] = []
for candidate in identities[1:]:
reasons.extend(compare(head, candidate))
gap = coverage_gap(
[(i.shard_start, i.shard_end) for i in identities], head.layer_count
)
if gap is not None:
reasons.append(RouteMismatch(MISMATCH_RANGE, gap))
if ledger is not None:
status = ledger.status(head)
allowed = status.may_certify if for_certification else status.may_serve
if not allowed:
reasons.append(
RouteMismatch(MISMATCH_UNCERTIFIED, f"{status.status}: {status.detail}")
)
return tuple(reasons)

9
packages/tracker/meshnet_tracker/server.py
View File

@@ -84,6 +84,7 @@ from .routing_stats import (
)
from .model_files import files_for_layer_range, snapshot_dir_for_repo
from .raft import RaftNode
from .recipe import CertificationLedger
_CONSOLE_LIMIT = 300
@@ -792,6 +793,7 @@ def _capability_from_registration(
hf_repo: str | None,
shard_start: int | None,
shard_end: int | None,
recipe_certifications: CertificationLedger,
) -> CapabilityState:
"""The tracker's verdict on the proof carried by one registration payload.
@@ -811,6 +813,7 @@ def _capability_from_registration(
declared_recipe_version=(
recipe_version if isinstance(recipe_version, str) else None
),
ledger=recipe_certifications,
)
@@ -2847,9 +2850,11 @@ class _TrackerHTTPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
relay_status: dict | None = None,
test_runner: "TestRunManager | None" = None,
capability_policy: str | None = None,
recipe_certifications: CertificationLedger | None = None,
) -> None:
super().__init__(addr, handler)
self.registry = registry
self.recipe_certifications = recipe_certifications or CertificationLedger()
self.capability_policy = normalize_policy(
capability_policy if capability_policy is not None else policy_from_env()
)
@@ -4555,6 +4560,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
hf_repo=hf_repo,
shard_start=shard_start,
shard_end=shard_end,
recipe_certifications=server.recipe_certifications,
)
node_id = _node_id_for_registration(
@@ -6531,6 +6537,7 @@ class TrackerServer:
self._embedded_relay: Any | None = None
self._embedded_relay_actual_port: int | None = None
self._registry: dict[str, _NodeEntry] = {}
self._recipe_certifications = CertificationLedger()
self._lock = threading.Lock()
self._server: _TrackerHTTPServer | None = None
self._thread: threading.Thread | None = None
@@ -6725,6 +6732,7 @@ class TrackerServer:
relay_status=http_relay_status,
test_runner=self._test_runner,
capability_policy=self._capability_policy,
recipe_certifications=self._recipe_certifications,
)
self.port = self._server.server_address[1]
@@ -6983,6 +6991,7 @@ class TrackerServer:
hf_repo=payload.get("hf_repo"),
shard_start=shard_start,
shard_end=shard_end,
recipe_certifications=self._recipe_certifications,
),
)
with self._lock:

66
scripts/bootstrap_native_toolchain.sh Normal file
View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Build a protobuf C++ toolchain for the native Shard protocol.
#
# The Python side needs nothing beyond `pip install grpcio-tools` — it bundles
# protoc. The C++ side needs libprotobuf headers and a protoc binary, and a
# machine that has neither (no protobuf-devel, no cmake, no system protoc) can
# still get a working one from source with this script. It is the exact recipe
# DGR-002 used to build and run the C++ conformance test.
#
# gRPC C++ is deliberately NOT built here. The conformance test only needs
# message types, so verifying the schema does not require the whole gRPC stack.
# The worker (DGR-008) will need gRPC C++ and should extend this script then.
#
# Usage:
# scripts/bootstrap_native_toolchain.sh [install-prefix]
#
# Then:
# cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=<prefix>
# cmake --build build/native -j
# ctest --test-dir build/native --output-on-failure
set -euo pipefail
PREFIX="${1:-${PWD}/build/native-toolchain}"
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
# Pinned: the C++ runtime a generated stub is compiled against must be a version
# that stub is allowed to use, so these are exact, not floating.
PROTOBUF_VERSION="33.1"
ABSEIL_VERSION="20250814.1"
command -v cmake >/dev/null || {
echo "cmake is required (pip install cmake==4.4.0)" >&2
exit 1
}
echo "--- fetching protobuf ${PROTOBUF_VERSION} and abseil ${ABSEIL_VERSION}"
cd "${WORK}"
curl -sfL -o protobuf.tar.gz \
"https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protobuf-${PROTOBUF_VERSION}.tar.gz"
tar xzf protobuf.tar.gz
# The protobuf release tarball ships utf8_range but not abseil, and its default
# CMake provider expects abseil as a submodule, so vendor it into place.
curl -sfL -o abseil.tar.gz \
"https://github.com/abseil/abseil-cpp/releases/download/${ABSEIL_VERSION}/abseil-cpp-${ABSEIL_VERSION}.tar.gz"
tar xzf abseil.tar.gz
rm -rf "protobuf-${PROTOBUF_VERSION}/third_party/abseil-cpp"
mv "abseil-cpp-${ABSEIL_VERSION}" "protobuf-${PROTOBUF_VERSION}/third_party/abseil-cpp"
echo "--- building protobuf into ${PREFIX}"
cmake -S "protobuf-${PROTOBUF_VERSION}" -B build \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="${PREFIX}" \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-Dprotobuf_ABSL_PROVIDER=module \
-Dprotobuf_BUILD_TESTS=OFF \
-Dprotobuf_BUILD_SHARED_LIBS=OFF \
-DABSL_PROPAGATE_CXX_STD=ON
cmake --build build -j"$(nproc)"
cmake --install build
echo "--- done"
"${PREFIX}/bin/protoc" --version
echo "configure the protocol build with: -DCMAKE_PREFIX_PATH=${PREFIX}"

125
scripts/generate_native_protocol.py Normal file
View File

@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""Generate the Python Shard-protocol stubs from `shard_runtime.proto`.
The `.proto` file is the contract; the Python modules under
`meshnet_node/native_protocol/generated/` are build output that happens to be
committed. They are committed so that installing the node does not require a
protoc toolchain, and `--check` exists so a committed stub can never silently
drift from the schema it claims to implement.
Usage::
python scripts/generate_native_protocol.py # regenerate in place
python scripts/generate_native_protocol.py --check # fail if out of date
C++ stubs are *not* generated here. They are build artifacts produced by CMake
(`packages/node/native/CMakeLists.txt`) into the build tree, because a C++ build
already requires a toolchain and nothing is gained by committing them.
"""
from __future__ import annotations
import argparse
import pathlib
import shutil
import subprocess
import sys
import tempfile
REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent
PROTO_DIR = REPO_ROOT / "packages/node/native/proto"
PROTO_FILE = PROTO_DIR / "shard_runtime.proto"
OUT_DIR = REPO_ROOT / "packages/node/meshnet_node/native_protocol/generated"
# Regenerating with a different protoc emits different gencode headers, so the
# generator version is part of the contract and `--check` would catch a drift.
REQUIRED_GRPCIO_TOOLS = "1.82.1"
_HEADER = "# Generated by scripts/generate_native_protocol.py. Do not edit.\n"
def _generate(into: pathlib.Path) -> None:
"""Run protoc, writing generated modules into `into`."""
try:
from grpc_tools import protoc
except ImportError: # pragma: no cover - exercised only without the toolchain
sys.exit(
"grpc_tools is required to generate stubs:\n"
f" pip install grpcio-tools=={REQUIRED_GRPCIO_TOOLS}"
)
into.mkdir(parents=True, exist_ok=True)
# grpc_tools bundles protoc and the well-known types, so generation needs no
# system protoc and produces identical output on any machine.
well_known = pathlib.Path(protoc.__file__).parent / "_proto"
args = [
"protoc",
f"--proto_path={PROTO_DIR}",
f"--proto_path={well_known}",
f"--python_out={into}",
f"--pyi_out={into}",
f"--grpc_python_out={into}",
str(PROTO_FILE),
]
if protoc.main(args) != 0:
sys.exit("protoc failed")
# protoc emits `import shard_runtime_pb2` — a bare top-level import that only
# resolves if the generated directory happens to be on sys.path. Rewrite it
# to a relative import so the package works as an installed package.
grpc_module = into / "shard_runtime_pb2_grpc.py"
text = grpc_module.read_text()
text = text.replace(
"import shard_runtime_pb2 as shard__runtime__pb2",
"from . import shard_runtime_pb2 as shard__runtime__pb2",
)
grpc_module.write_text(text)
(into / "__init__.py").write_text(
_HEADER + '"""Generated protobuf/gRPC stubs for the native Shard protocol."""\n'
)
def _tracked_files(directory: pathlib.Path) -> dict[str, bytes]:
return {
path.name: path.read_bytes()
for path in sorted(directory.iterdir())
if path.is_file() and path.suffix in {".py", ".pyi"}
}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--check",
action="store_true",
help="verify the committed stubs match the .proto instead of rewriting them",
)
args = parser.parse_args()
if args.check:
with tempfile.TemporaryDirectory() as tmp:
fresh = pathlib.Path(tmp) / "generated"
_generate(fresh)
if not OUT_DIR.is_dir():
print(f"generated stubs are missing: {OUT_DIR}", file=sys.stderr)
return 1
if _tracked_files(fresh) != _tracked_files(OUT_DIR):
print(
"generated stubs are out of date with shard_runtime.proto.\n"
"Run: python scripts/generate_native_protocol.py",
file=sys.stderr,
)
return 1
print("generated stubs are up to date")
return 0
if OUT_DIR.exists():
shutil.rmtree(OUT_DIR)
_generate(OUT_DIR)
print(f"wrote {OUT_DIR.relative_to(REPO_ROOT)}")
return 0
if __name__ == "__main__":
raise SystemExit(main())

71
scripts/generate_protocol_goldens.py Normal file
View File

@@ -0,0 +1,71 @@
#!/usr/bin/env python3
"""Write the committed cross-language conformance vectors.
The bytes under `packages/node/native/testdata/` are the reference both the
Python and the C++ conformance tests assert against. They are committed so the
C++ test can run without a Python step, and `--check` exists so they can never
drift from the schema unnoticed: if a schema edit changes the canonical
message's encoding, `--check` fails and the change has to be acknowledged.
Usage::
python scripts/generate_protocol_goldens.py # rewrite vectors
python scripts/generate_protocol_goldens.py --check # fail if stale
"""
from __future__ import annotations
import argparse
import pathlib
import sys
REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO_ROOT / "packages/node"))
from meshnet_node.native_protocol import conformance # noqa: E402
def _vectors() -> dict[str, bytes]:
return {
conformance.GOLDEN_SESSION_REQUEST: conformance.serialize(
conformance.canonical_session_request()
),
conformance.GOLDEN_CAPABILITY_REPORT: conformance.serialize(
conformance.canonical_capability_report()
),
}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--check", action="store_true")
args = parser.parse_args()
out_dir = conformance.TESTDATA_DIR
out_dir.mkdir(parents=True, exist_ok=True)
stale = []
for name, payload in _vectors().items():
path = out_dir / name
if args.check:
if not path.is_file() or path.read_bytes() != payload:
stale.append(name)
continue
path.write_bytes(payload)
print(f"wrote {path.relative_to(REPO_ROOT)} ({len(payload)} bytes)")
if stale:
print(
"conformance vectors are stale: " + ", ".join(stale) + "\n"
"The canonical message no longer encodes to the committed bytes. If "
"that is intended, run: python scripts/generate_protocol_goldens.py",
file=sys.stderr,
)
return 1
if args.check:
print("conformance vectors are up to date")
return 0
if __name__ == "__main__":
raise SystemExit(main())

254
scripts/refresh_glm_target_manifest.py Normal file
View File

@@ -0,0 +1,254 @@
#!/usr/bin/env python3
"""Refresh (or check) the pinned GLM-5.2 target manifest and architecture snapshot.
Resolves revisions, shard sizes, and LFS SHA-256 digests from the Hugging Face
metadata API. It never downloads a weight payload: sizes and digests come from
``/api/models/.../paths-info``, which returns the LFS pointer metadata, and the only
files fetched in full are the small config/tokenizer/chat-template documents whose
bytes the snapshot hashes.
Usage::
python scripts/refresh_glm_target_manifest.py --check # CI: pinned bytes still resolve?
python scripts/refresh_glm_target_manifest.py --write # re-pin HEAD after human review
``--check`` validates the already locked revisions through Hugging Face's
revision-specific API. A normal new upstream commit does not mutate the target.
``--write`` intentionally follows moving HEAD and is therefore a reviewed target
change, never an automatic refresh.
This script requires network access and is not part of the default test suite.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import sys
import urllib.request
from pathlib import Path
from typing import Any
DATA_DIR = Path(__file__).resolve().parent.parent / "packages/node/meshnet_node/glm_alpha/data"
MANIFEST_PATH = DATA_DIR / "target-manifest.json"
SNAPSHOT_PATH = DATA_DIR / "architecture-snapshot.json"
SOURCE_REPO = "zai-org/GLM-5.2"
GGUF_REPO = "unsloth/GLM-5.2-GGUF"
QUANT = "UD-IQ1_S"
FALLBACK_QUANT = "UD-IQ1_M"
SHARD_COUNT = 6
SNAPSHOT_FILES = (
"config.json",
"chat_template.jinja",
"generation_config.json",
"tokenizer_config.json",
)
TIMEOUT = 60
def _get(url: str) -> bytes:
with urllib.request.urlopen(url, timeout=TIMEOUT) as response: # noqa: S310 - fixed HTTPS host
return response.read()
def _get_json(url: str) -> Any:
return json.loads(_get(url))
def _post_json(url: str, payload: dict) -> Any:
request = urllib.request.Request(
url,
data=json.dumps(payload).encode("utf-8"),
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(request, timeout=TIMEOUT) as response: # noqa: S310
return json.loads(response.read())
def _shard_paths(quant: str) -> list[str]:
return [
f"{quant}/GLM-5.2-{quant}-{index:05d}-of-{SHARD_COUNT:05d}.gguf"
for index in range(1, SHARD_COUNT + 1)
]
def _resolve_shards(revision: str, quant: str) -> list[dict]:
info = _post_json(
f"https://huggingface.co/api/models/{GGUF_REPO}/paths-info/{revision}",
{"paths": _shard_paths(quant)},
)
by_path = {entry["path"]: entry for entry in info}
shards = []
for index, path in enumerate(_shard_paths(quant), start=1):
entry = by_path.get(path)
if entry is None:
raise SystemExit(f"upstream is missing shard {path} at revision {revision}")
lfs = entry.get("lfs") or {}
oid = lfs.get("oid")
if not oid:
raise SystemExit(
f"{path} has no LFS oid; a non-LFS shard is not the published artifact"
)
shards.append(
{
"index": index,
"path": path,
"size_bytes": int(entry["size"]),
"sha256": oid,
"url": f"https://huggingface.co/{GGUF_REPO}/resolve/{revision}/{path}",
}
)
return shards
def build_documents(
*,
source_revision: str | None = None,
gguf_revision: str | None = None,
) -> tuple[dict, dict]:
"""Resolve documents at explicit pins, or at current HEAD for reviewed re-pinning."""
source_api = f"https://huggingface.co/api/models/{SOURCE_REPO}"
gguf_api = f"https://huggingface.co/api/models/{GGUF_REPO}"
if source_revision is not None:
source_api += f"/revision/{source_revision}"
if gguf_revision is not None:
gguf_api += f"/revision/{gguf_revision}"
source_info = _get_json(source_api)
gguf_info = _get_json(gguf_api)
source_rev = source_info["sha"]
gguf_rev = gguf_info["sha"]
if source_revision is not None and source_rev != source_revision:
raise SystemExit(
f"source revision endpoint returned {source_rev}, expected {source_revision}"
)
if gguf_revision is not None and gguf_rev != gguf_revision:
raise SystemExit(f"GGUF revision endpoint returned {gguf_rev}, expected {gguf_revision}")
shards = _resolve_shards(gguf_rev, QUANT)
total = sum(shard["size_bytes"] for shard in shards)
fallback = _resolve_shards(gguf_rev, FALLBACK_QUANT)
fallback_total = sum(shard["size_bytes"] for shard in fallback)
manifest = json.loads(MANIFEST_PATH.read_text(encoding="utf-8"))
manifest["source_model"]["revision"] = source_rev
manifest["source_model"]["last_modified"] = source_info.get("lastModified")
manifest["source_model"]["revision_url"] = (
f"https://huggingface.co/{SOURCE_REPO}/tree/{source_rev}"
)
manifest["gguf_artifact"]["revision"] = gguf_rev
manifest["gguf_artifact"]["last_modified"] = gguf_info.get("lastModified")
manifest["gguf_artifact"]["revision_url"] = (
f"https://huggingface.co/{GGUF_REPO}/tree/{gguf_rev}"
)
manifest["gguf_artifact"]["shards"] = shards
manifest["gguf_artifact"]["total_bytes"] = total
manifest["gguf_artifact"]["total_gib"] = round(total / 1024**3, 3)
manifest["gguf_artifact"]["total_gb"] = round(total / 1000**3, 3)
manifest["diagnostic_fallback"]["total_bytes"] = fallback_total
manifest["diagnostic_fallback"]["total_gib"] = round(fallback_total / 1024**3, 3)
manifest["diagnostic_fallback"]["total_gb"] = round(fallback_total / 1000**3, 3)
snapshot = json.loads(SNAPSHOT_PATH.read_text(encoding="utf-8"))
snapshot["source_revision"] = source_rev
source_files = []
config: dict[str, Any] = {}
for name in SNAPSHOT_FILES:
url = f"https://huggingface.co/{SOURCE_REPO}/resolve/{source_rev}/{name}"
body = _get(url)
source_files.append(
{
"path": name,
"size_bytes": len(body),
"sha256": hashlib.sha256(body).hexdigest(),
"url": url,
}
)
if name == "config.json":
config = json.loads(body)
snapshot["source_files"] = source_files
indexer_types = config["indexer_types"]
arch = snapshot["architecture"]
arch["num_hidden_layers"] = config["num_hidden_layers"]
arch["num_nextn_predict_layers"] = config["num_nextn_predict_layers"]
arch["total_artifact_layers"] = config["num_hidden_layers"] + config["num_nextn_predict_layers"]
arch["hidden_size"] = config["hidden_size"]
arch["n_routed_experts"] = config["n_routed_experts"]
arch["num_experts_per_tok"] = config["num_experts_per_tok"]
arch["n_shared_experts"] = config["n_shared_experts"]
arch["index_topk"] = config["index_topk"]
arch["index_head_dim"] = config["index_head_dim"]
arch["kv_lora_rank"] = config["kv_lora_rank"]
arch["qk_rope_head_dim"] = config["qk_rope_head_dim"]
arch["mla_cached_values_per_token_per_layer"] = (
config["kv_lora_rank"] + config["qk_rope_head_dim"]
)
arch["indexer_full_layers"] = sum(1 for role in indexer_types if role == "full")
arch["indexer_shared_layers"] = sum(1 for role in indexer_types if role == "shared")
arch["indexer_types_sha256"] = hashlib.sha256(
json.dumps(indexer_types, separators=(",", ":")).encode("utf-8")
).hexdigest()
arch["max_position_embeddings"] = config["max_position_embeddings"]
arch["vocab_size"] = config["vocab_size"]
arch["first_k_dense_replace"] = config["first_k_dense_replace"]
arch["dense_layers"] = config["first_k_dense_replace"]
arch["sparse_moe_layers"] = config["num_hidden_layers"] - config["first_k_dense_replace"]
return manifest, snapshot
def _dump(document: dict) -> str:
return json.dumps(document, indent=2, ensure_ascii=False) + "\n"
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("--check", action="store_true", help="fail if the pins have drifted")
group.add_argument("--write", action="store_true", help="re-pin from upstream")
args = parser.parse_args()
if args.check:
pinned = json.loads(MANIFEST_PATH.read_text(encoding="utf-8"))
manifest, snapshot = build_documents(
source_revision=pinned["source_model"]["revision"],
gguf_revision=pinned["gguf_artifact"]["revision"],
)
else:
# --write intentionally follows current HEAD and therefore requires review.
manifest, snapshot = build_documents()
if args.write:
MANIFEST_PATH.write_text(_dump(manifest), encoding="utf-8")
SNAPSHOT_PATH.write_text(_dump(snapshot), encoding="utf-8")
print(f"wrote {MANIFEST_PATH}")
print(f"wrote {SNAPSHOT_PATH}")
print("Re-pinning changes the alpha target. Update the alpha contract under review.")
return 0
drifted = False
for path, fresh in ((MANIFEST_PATH, manifest), (SNAPSHOT_PATH, snapshot)):
current = json.loads(path.read_text(encoding="utf-8"))
if current != fresh:
drifted = True
print(f"DRIFT: {path.name} no longer matches upstream", file=sys.stderr)
if drifted:
print(
"\nPinned revision metadata no longer matches the locked files. Treat this as "
"artifact-integrity drift; do not heal or re-pin without human review.",
file=sys.stderr,
)
return 1
print("target manifest and architecture snapshot match upstream")
return 0
if __name__ == "__main__":
raise SystemExit(main())

52
tests/data/recipe_fingerprint_vectors.json Normal file
View File

@@ -0,0 +1,52 @@
{
"schema_version": 1,
"vectors": [
{
"fingerprint": {
"catalogue_version": "2026.07.1",
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
},
"identity": {
"artifact": {
"architecture": "dense-llama",
"architecture_digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"artifact_id": "example/model",
"content_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"derived_from": null,
"layer_count": 8,
"revision": "0123456789abcdef"
},
"fingerprint": {
"catalogue_version": "2026.07.1",
"model_artifact_digest": "8a0f43d6aa49d77834bdb47bcae9f42c886b7ccfe0ac014932b2a2b38697a47b",
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_recipe_digest": "9b14d70b0835a6428457e4888d453649dd0d2e41fc8ac9d84d232c8c237e68fa"
},
"recipe": {
"activation_dtype": "bfloat16",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"boundary_schema_version": 1,
"catalogue_version": "2026.07.1",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"protocol_schema_version": 1,
"recipe_id": "example-gguf",
"recipe_version": "1",
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
"tokenizer_revision": "0123456789abcdef",
"weight_quantization": "Q4_K_M"
},
"schema_version": 1,
"shard_end": 4,
"shard_start": 0
},
"name": "example-v1"
}
]
}

860
tests/test_glm_alpha_target.py Normal file
View File

@@ -0,0 +1,860 @@
"""DGR-017 — the locked GLM-5.2 Max target, resource plan, and alpha contract.
These tests are deterministic, offline, GPU-free, and download-free. They assert
against the *pinned* manifest, so they fail if a later agent swaps the artifact,
loosens the memory accounting, or moves a threshold after seeing a result.
The planner tests are written as a reproduction of the roadmap's published tables.
That is the point: if the arithmetic here ever stops reproducing them, either the
roadmap or the planner is lying, and the test says which numbers changed.
"""
from __future__ import annotations
import copy
import json
from pathlib import Path
import pytest
from meshnet_node.glm_alpha import (
AGGREGATE_HARD_FIT_FLOOR_GIB,
ALPHA_CONTRACT_ID,
ALPHA_QUANTIZATION,
ALPHA_SHARD_COUNT,
MIN_LINK_RATE_GBPS,
RECOMMENDED_LINK_RATE_GBPS,
RESERVE_FLOOR_GIB,
AlphaContractError,
GlmTargetError,
NodeMemory,
ResourcePlanError,
compute_contract_digest,
kv_bytes,
load_alpha_contract,
load_architecture_snapshot,
load_locked_target,
load_target_manifest,
parse_alpha_contract,
parse_architecture_snapshot,
parse_target_manifest,
plan_route,
plan_seams,
plan_topology,
require_contract_target,
require_pinned_target,
seal_contract,
)
from meshnet_node.glm_alpha.manifest import GIB
# The revisions observed and pinned by DGR-017 on 2026-07-13.
SOURCE_REVISION = "b4734de4facf877f85769a911abafc5283eab3d9"
GGUF_REVISION = "abc55e72527792c6e77069c99b4cb7de16fa9f23"
TOTAL_BYTES = 216_715_360_960
@pytest.fixture(scope="module")
def manifest():
return load_target_manifest()
@pytest.fixture(scope="module")
def snapshot():
return load_architecture_snapshot()
@pytest.fixture(scope="module")
def contract():
return load_alpha_contract()
@pytest.fixture
def manifest_doc(manifest):
return copy.deepcopy(dict(manifest.raw))
@pytest.fixture
def snapshot_doc(snapshot):
return copy.deepcopy(dict(snapshot.raw))
@pytest.fixture
def contract_doc(contract):
return contract.to_dict()
# --------------------------------------------------------------------------
# Identity: the exact artifact, pinned
# --------------------------------------------------------------------------
def test_manifest_pins_both_repositories_by_exact_revision(manifest):
assert manifest.source_repo_id == "zai-org/GLM-5.2"
assert manifest.source_revision == SOURCE_REVISION
assert manifest.gguf_repo_id == "unsloth/GLM-5.2-GGUF"
assert manifest.gguf_revision == GGUF_REVISION
assert manifest.quantization == ALPHA_QUANTIZATION == "UD-IQ1_S"
assert manifest.source_license == "mit"
assert manifest.gguf_license == "mit"
def test_manifest_resolves_all_six_shards_with_sizes_hashes_and_urls(manifest):
assert len(manifest.shards) == ALPHA_SHARD_COUNT == 6
assert [shard.index for shard in manifest.shards] == [1, 2, 3, 4, 5, 6]
for shard in manifest.shards:
assert shard.size_bytes > 0
assert len(shard.sha256) == 64
assert shard.url.startswith(f"https://huggingface.co/{manifest.gguf_repo_id}/resolve/")
assert GGUF_REVISION in shard.url, "a shard URL must resolve at the pinned revision"
digests = {shard.sha256 for shard in manifest.shards}
assert len(digests) == 6, "six distinct shards must have six distinct content digests"
def test_manifest_aggregate_bytes_are_exact_and_self_consistent(manifest):
assert manifest.total_bytes == TOTAL_BYTES
assert sum(shard.size_bytes for shard in manifest.shards) == TOTAL_BYTES
assert round(manifest.total_gib, 3) == 201.832
assert round(manifest.total_gb, 3) == 216.715
def test_manifest_records_the_iq1_m_diagnostic_fallback_without_promoting_it(manifest):
fallback = manifest.raw["diagnostic_fallback"]
assert fallback["quantization"] == "UD-IQ1_M"
assert fallback["total_bytes"] == 228_492_966_624
assert fallback["total_bytes"] > manifest.total_bytes
assert "does not satisfy" in fallback["policy"]
def test_manifest_forbids_home_storage(manifest):
assert manifest.raw["storage"]["mounted_storage_only"] is True
assert "/home" in manifest.raw["storage"]["forbidden_path_prefixes"]
# --------------------------------------------------------------------------
# Identity: what the manifest must reject
# --------------------------------------------------------------------------
def test_a_changed_source_revision_is_rejected(manifest, snapshot, manifest_doc):
manifest_doc["source_model"]["revision"] = "0" * 40
swapped = parse_target_manifest(manifest_doc)
with pytest.raises(GlmTargetError, match="does not match the locked alpha revision"):
require_pinned_target(
swapped,
snapshot,
expected_source_revision=SOURCE_REVISION,
expected_gguf_revision=GGUF_REVISION,
)
def test_a_changed_gguf_revision_is_rejected(manifest, snapshot, manifest_doc):
manifest_doc["gguf_artifact"]["revision"] = "1" * 40
swapped = parse_target_manifest(manifest_doc)
with pytest.raises(GlmTargetError, match="does not match the locked alpha revision"):
require_pinned_target(
swapped,
snapshot,
expected_source_revision=SOURCE_REVISION,
expected_gguf_revision=GGUF_REVISION,
)
def test_the_pinned_target_passes_its_own_revision_check(manifest, snapshot):
require_pinned_target(
manifest,
snapshot,
expected_source_revision=SOURCE_REVISION,
expected_gguf_revision=GGUF_REVISION,
)
def test_config_metadata_from_a_different_revision_than_the_weights_is_rejected(
manifest, snapshot_doc
):
snapshot_doc["source_revision"] = "2" * 40
drifted = parse_architecture_snapshot(snapshot_doc)
with pytest.raises(GlmTargetError, match="must come from one revision"):
require_pinned_target(
manifest,
drifted,
expected_source_revision=SOURCE_REVISION,
expected_gguf_revision=GGUF_REVISION,
)
def test_a_branch_name_is_not_an_acceptable_revision_pin(manifest_doc):
manifest_doc["gguf_artifact"]["revision"] = "main"
with pytest.raises(GlmTargetError, match="not an immutable pin"):
parse_target_manifest(manifest_doc)
def test_a_missing_shard_is_rejected(manifest_doc):
manifest_doc["gguf_artifact"]["shards"].pop()
with pytest.raises(GlmTargetError, match="exactly 6 shards"):
parse_target_manifest(manifest_doc)
def test_a_shard_replaced_by_a_duplicate_of_another_is_rejected(manifest_doc):
shards = manifest_doc["gguf_artifact"]["shards"]
# Keep the count at six and the byte total consistent, but drop shard 6's
# identity — the shape a lazy "just re-download it" repair takes.
shards[5]["index"] = 5
with pytest.raises(GlmTargetError, match="duplicate shard index"):
parse_target_manifest(manifest_doc)
def test_two_shards_claiming_the_same_content_digest_are_rejected(manifest_doc):
shards = manifest_doc["gguf_artifact"]["shards"]
shards[5]["sha256"] = shards[4]["sha256"]
with pytest.raises(GlmTargetError, match="same content digest"):
parse_target_manifest(manifest_doc)
def test_an_inconsistent_aggregate_byte_total_is_rejected(manifest_doc):
manifest_doc["gguf_artifact"]["total_bytes"] = TOTAL_BYTES - 1
with pytest.raises(GlmTargetError, match="not self-consistent"):
parse_target_manifest(manifest_doc)
def test_a_shard_size_edited_to_make_the_model_look_smaller_is_rejected(manifest_doc):
# The aggregate is now internally inconsistent, which is exactly the tell.
manifest_doc["gguf_artifact"]["shards"][2]["size_bytes"] = 1_000_000
with pytest.raises(GlmTargetError, match="not self-consistent"):
parse_target_manifest(manifest_doc)
def test_swapping_in_a_different_quantization_is_rejected(manifest_doc):
manifest_doc["alpha_quantization"] = "UD-IQ1_M"
with pytest.raises(GlmTargetError, match="requires a human contract change"):
parse_target_manifest(manifest_doc)
def test_a_truncated_sha256_is_rejected(manifest_doc):
manifest_doc["gguf_artifact"]["shards"][0]["sha256"] = "abc123"
with pytest.raises(GlmTargetError, match="64-character hex SHA-256"):
parse_target_manifest(manifest_doc)
# --------------------------------------------------------------------------
# Architecture snapshot
# --------------------------------------------------------------------------
def test_snapshot_captures_the_architecture_critical_metadata(snapshot):
assert snapshot["model_type"] == "glm_moe_dsa"
assert snapshot["num_hidden_layers"] == 78
assert snapshot["num_nextn_predict_layers"] == 1
assert snapshot["total_artifact_layers"] == 79
assert snapshot["dense_layers"] == 3
assert snapshot["sparse_moe_layers"] == 75
assert snapshot["hidden_size"] == 6144
assert snapshot["n_routed_experts"] == 256
assert snapshot["num_experts_per_tok"] == 8
assert snapshot["n_shared_experts"] == 1
assert snapshot["index_topk"] == 2048
assert snapshot["index_head_dim"] == 128
assert snapshot["max_position_embeddings"] == 1_048_576
assert snapshot["vocab_size"] == 154_880
def test_snapshot_records_indexshare_roles_for_every_layer(snapshot):
assert snapshot["indexer_full_layers"] == 21
assert snapshot["indexer_shared_layers"] == 57
assert snapshot["indexer_full_layers"] + snapshot["indexer_shared_layers"] == 78
def test_mla_cache_width_is_derived_not_asserted(snapshot):
assert snapshot["kv_lora_rank"] == 512
assert snapshot["qk_rope_head_dim"] == 64
assert snapshot["mla_cached_values_per_token_per_layer"] == 576
def test_snapshot_hashes_the_config_and_chat_template_bytes(snapshot):
assert len(snapshot.file_sha256("config.json")) == 64
assert len(snapshot.file_sha256("chat_template.jinja")) == 64
assert len(snapshot.file_sha256("tokenizer_config.json")) == 64
assert len(snapshot["indexer_types_sha256"]) == 64
assert len(snapshot.digest) == 64
def test_reasoning_effort_max_is_locked_as_an_observable_rendered_marker(snapshot):
reasoning = snapshot.reasoning_effort
assert reasoning["alpha_mode"] == "max"
assert reasoning["rendered_marker"] == "<|system|>Reasoning Effort: Max"
# The template's only non-max level is 'high'; everything else renders Max. So
# "the request carried reasoning_effort=max" proves nothing on its own.
assert reasoning["default_is_max"] is True
def test_folding_the_nextn_layer_into_the_backbone_is_rejected(snapshot_doc):
snapshot_doc["architecture"]["total_artifact_layers"] = 78
with pytest.raises(GlmTargetError, match="NextN layer must be counted"):
parse_architecture_snapshot(snapshot_doc)
def test_indexshare_roles_that_do_not_cover_every_layer_are_rejected(snapshot_doc):
snapshot_doc["architecture"]["indexer_full_layers"] = 20
with pytest.raises(GlmTargetError, match="exactly one IndexShare role"):
parse_architecture_snapshot(snapshot_doc)
def test_a_route_with_no_full_indexer_producer_is_rejected(snapshot_doc):
snapshot_doc["architecture"]["indexer_full_layers"] = 0
snapshot_doc["architecture"]["indexer_shared_layers"] = 78
with pytest.raises(GlmTargetError, match="no index for its Shared consumers"):
parse_architecture_snapshot(snapshot_doc)
def test_a_contradictory_mla_width_is_rejected(snapshot_doc):
snapshot_doc["architecture"]["mla_cached_values_per_token_per_layer"] = 512
with pytest.raises(GlmTargetError, match="kv_lora_rank"):
parse_architecture_snapshot(snapshot_doc)
def test_a_snapshot_missing_an_architecture_critical_field_is_rejected(snapshot_doc):
del snapshot_doc["architecture"]["n_routed_experts"]
with pytest.raises(GlmTargetError, match="architecture-critical field"):
parse_architecture_snapshot(snapshot_doc)
def test_a_snapshot_that_drops_reasoning_effort_max_is_rejected(snapshot_doc):
snapshot_doc["reasoning_effort"]["alpha_mode"] = "high"
with pytest.raises(GlmTargetError, match="does not lock reasoning_effort=max"):
parse_architecture_snapshot(snapshot_doc)
# --------------------------------------------------------------------------
# KV planning
# --------------------------------------------------------------------------
@pytest.mark.parametrize(
("context", "mla_only", "optimized", "conservative", "conservative_f16"),
[
(16_384, 0.73, 0.77, 0.89, 1.68),
(131_072, 5.83, 6.18, 7.12, 13.41),
(1_048_576, 46.62, 49.41, 56.98, 107.25),
],
)
def test_kv_planner_reproduces_the_published_roadmap_table(
snapshot, context, mla_only, optimized, conservative, conservative_f16
):
def gib(**kwargs):
return kv_bytes(snapshot, context_tokens=context, concurrency=1, **kwargs) / GIB
assert round(gib(include_indexer=False), 2) == mla_only
assert round(gib(indexer_layout="optimized"), 2) == optimized
assert round(gib(indexer_layout="conservative"), 2) == conservative
assert round(gib(indexer_layout="conservative", dtype="F16"), 2) == conservative_f16
def test_alpha_budgets_the_conservative_indexer_layout(snapshot):
"""Alpha must budget the implementation it may actually get, not the ideal one."""
optimized = kv_bytes(snapshot, indexer_layout="optimized")
conservative = kv_bytes(snapshot, indexer_layout="conservative")
assert conservative > optimized
assert kv_bytes(snapshot) == conservative, "the default must be the conservative layout"
def test_kv_scales_with_concurrency(snapshot):
assert kv_bytes(snapshot, concurrency=2) == 2 * kv_bytes(snapshot, concurrency=1)
def test_an_unlocked_kv_dtype_is_rejected(snapshot):
with pytest.raises(ResourcePlanError, match="alpha locks Q8_0"):
kv_bytes(snapshot, dtype="Q4_0")
@pytest.mark.parametrize("bad", [0, -1, True, 1.5])
def test_kv_rejects_non_positive_or_non_integer_dimensions(snapshot, bad):
with pytest.raises(ResourcePlanError, match="positive integers"):
kv_bytes(snapshot, context_tokens=bad)
# --------------------------------------------------------------------------
# Memory accounting: unified memory is one pool
# --------------------------------------------------------------------------
def test_unified_memory_is_counted_once():
node = NodeMemory.from_host("strix-halo", system_ram_gib=128.0, unified=True)
assert node.physical_usable_gib == 128.0, "integrated-GPU memory is not extra memory"
def test_adding_integrated_gpu_memory_to_system_ram_is_rejected():
with pytest.raises(ResourcePlanError, match="double-counts one pool"):
NodeMemory.from_host(
"strix-halo",
system_ram_gib=128.0,
gpu_memory_gib=96.0,
unified=True,
)
def test_a_discrete_gpu_does_add_its_own_memory():
node = NodeMemory.from_host(
"workstation", system_ram_gib=64.0, gpu_memory_gib=24.0, unified=False
)
assert node.physical_usable_gib == 88.0
def test_the_same_machine_counted_twice_in_a_route_is_rejected(manifest, snapshot):
twin = [
NodeMemory.from_host("box-a", system_ram_gib=128.0, unified=True),
NodeMemory.from_host("box-a", system_ram_gib=128.0, unified=True),
]
with pytest.raises(ResourcePlanError, match="counted twice"):
plan_route(manifest, snapshot, twin)
@pytest.mark.parametrize("bad", [float("nan"), float("inf"), -1.0, True])
def test_node_memory_rejects_non_finite_or_invalid_capacity(bad):
with pytest.raises(ResourcePlanError, match="finite positive"):
NodeMemory(name="bad-host", physical_usable_gib=bad, unified=True)
# --------------------------------------------------------------------------
# Reserve and topology
# --------------------------------------------------------------------------
def test_the_reserve_floor_binds_on_small_nodes():
small = NodeMemory(name="32", physical_usable_gib=32.0, unified=True)
assert small.reserve_gib == RESERVE_FLOOR_GIB == 8.0 # 20% of 32 is only 6.4
assert small.placement_budget_gib == 24.0
def test_the_reserve_fraction_binds_on_large_nodes():
large = NodeMemory(name="128", physical_usable_gib=128.0, unified=True)
assert large.reserve_gib == 25.6 # 20% of 128 clears the 8 GiB floor
assert large.placement_budget_gib == 102.4
@pytest.mark.parametrize(
("tier", "reserve", "budget", "arithmetic_minimum", "recommended"),
[
(32.0, 8.0, 24.0, 9, 10),
(48.0, 9.6, 38.4, 6, 6),
(64.0, 12.8, 51.2, 4, 5),
(96.0, 19.2, 76.8, 3, 3),
(128.0, 25.6, 102.4, 2, 3),
],
)
def test_topology_planner_reproduces_the_published_tier_table(
manifest, snapshot, tier, reserve, budget, arithmetic_minimum, recommended
):
plan = plan_topology(manifest, snapshot, physical_usable_gib=tier)
assert round(plan.reserve_gib, 2) == reserve
assert round(plan.placement_budget_gib, 2) == budget
assert plan.arithmetic_minimum_nodes == arithmetic_minimum
assert plan.recommended_nodes == recommended
assert round(plan.weight_gib, 3) == 201.832
assert round(plan.kv_gib, 2) == 0.89 # 16K, concurrency 1, Q8_0, conservative DSA
def test_the_recommended_topologies_are_five_by_64_or_three_by_96_or_128(manifest, snapshot):
assert plan_topology(manifest, snapshot, physical_usable_gib=64.0).recommended_nodes == 5
assert plan_topology(manifest, snapshot, physical_usable_gib=96.0).recommended_nodes == 3
assert plan_topology(manifest, snapshot, physical_usable_gib=128.0).recommended_nodes == 3
def test_two_by_128_is_an_arithmetic_minimum_not_a_recommendation(manifest, snapshot):
plan = plan_topology(manifest, snapshot, physical_usable_gib=128.0)
assert plan.arithmetic_minimum_nodes == 2
assert plan.recommended_nodes == 3
assert not plan.is_arithmetic_minimum_topology, (
"2x128 GiB leaves no room for endpoint/tensor imbalance; it is a fit probe that "
"requires measured placement evidence, not the alpha recommendation"
)
@pytest.mark.parametrize("bad", [0.9, float("nan"), float("inf"), True])
def test_an_invalid_placement_imbalance_is_rejected(manifest, snapshot, bad):
with pytest.raises(ResourcePlanError, match="less than an equal share"):
plan_topology(manifest, snapshot, physical_usable_gib=64.0, imbalance_factor=bad)
# --------------------------------------------------------------------------
# Route fit
# --------------------------------------------------------------------------
def test_the_recommended_five_by_64_route_fits(manifest, snapshot):
nodes = [
NodeMemory.from_host(f"node-{i}", system_ram_gib=64.0, unified=True) for i in range(5)
]
fit = plan_route(manifest, snapshot, nodes)
assert fit.fits
assert fit.meets_hard_fit_floor
assert fit.no_single_node_can_admit_target
assert fit.reasons == ()
def test_224_gib_aggregate_is_a_hard_fit_floor_not_an_operational_envelope(manifest, snapshot):
"""Exactly 224 GiB of aggregate memory clears the floor and still does not fit.
This is the whole reason the roadmap calls 224 GiB an *experimental hard-fit
floor*. It is the number you get by ignoring the per-node reserve — and once
the reserve is applied, the weights alone no longer have anywhere to go.
"""
nodes = [
NodeMemory.from_host(f"node-{i}", system_ram_gib=112.0, unified=True) for i in range(2)
]
fit = plan_route(manifest, snapshot, nodes)
assert fit.aggregate_usable_gib == AGGREGATE_HARD_FIT_FLOOR_GIB == 224.0
assert fit.meets_hard_fit_floor
assert not fit.fits, "224 GiB aggregate does not fit the target once reserves are honoured"
assert any("below the" in reason for reason in fit.reasons)
def test_a_route_too_small_for_the_target_does_not_fit(manifest, snapshot):
nodes = [
NodeMemory.from_host(f"node-{i}", system_ram_gib=64.0, unified=True) for i in range(3)
]
fit = plan_route(manifest, snapshot, nodes)
assert not fit.fits
assert not fit.meets_hard_fit_floor
def test_a_route_where_one_node_could_hold_everything_is_not_distributed_alpha(
manifest, snapshot
):
nodes = [
NodeMemory.from_host(f"node-{i}", system_ram_gib=512.0, unified=True) for i in range(2)
]
fit = plan_route(manifest, snapshot, nodes)
assert fit.fits
assert not fit.no_single_node_can_admit_target
assert any("single-host run" in reason for reason in fit.reasons)
def test_a_single_node_is_not_a_route(manifest, snapshot):
with pytest.raises(ResourcePlanError, match="at least two physical nodes"):
plan_route(
manifest,
snapshot,
[NodeMemory.from_host("solo", system_ram_gib=512.0, unified=True)],
)
# --------------------------------------------------------------------------
# Seams and network
# --------------------------------------------------------------------------
def test_seam_bytes_match_the_published_activation_arithmetic(snapshot):
plan = plan_seams(snapshot, node_count=4, context_tokens=16_384)
assert plan.seam_count == 3, "four nodes imply three serial seams"
assert plan.bytes_per_token_per_seam == 12_288 # 6144 x bf16
assert plan.prefill_bytes_per_seam == 192 * 1024**2 # 192 MiB
assert plan.decode_bytes_per_seam_per_token == 12 * 1024 # 12 KiB
assert plan.dsa_sideband_bytes_per_query == 8 * 1024 # 2048 int32 = 8 KiB
def test_2_5_gbe_is_the_alpha_minimum_and_10_gbe_is_recommended(snapshot):
gigabit = plan_seams(snapshot, node_count=3, link_rate_gbps=1.0)
minimum = plan_seams(snapshot, node_count=3, link_rate_gbps=MIN_LINK_RATE_GBPS)
recommended = plan_seams(snapshot, node_count=3, link_rate_gbps=RECOMMENDED_LINK_RATE_GBPS)
assert not gigabit.meets_alpha_minimum, "1 GbE is fit-only evidence, not an alpha route"
assert minimum.meets_alpha_minimum
assert not minimum.is_recommended_link
assert recommended.meets_alpha_minimum
assert recommended.is_recommended_link
def test_serial_seam_latency_is_modelled_separately_from_bandwidth(snapshot):
"""Decode moves 12 KiB a token. What it pays is hops, not bytes."""
fast_link = plan_seams(snapshot, node_count=5, link_rate_gbps=10.0, per_hop_latency_ms=2.0)
slow_link = plan_seams(snapshot, node_count=5, link_rate_gbps=2.5, per_hop_latency_ms=2.0)
# Quadrupling the link rate barely touches decode: the payload is tiny.
assert fast_link.decode_bandwidth_share_ms_per_token < 0.05
assert slow_link.decode_bandwidth_share_ms_per_token < 0.2
# Latency is unchanged by link rate and scales with the number of serial seams.
assert fast_link.decode_latency_ms_per_token == slow_link.decode_latency_ms_per_token == 8.0
# A 10 GbE claim cannot buy back a hop.
assert fast_link.decode_latency_ms_per_token > fast_link.decode_bandwidth_share_ms_per_token
def test_adding_a_node_adds_a_serial_seam_to_every_decoded_token(snapshot):
three = plan_seams(snapshot, node_count=3, per_hop_latency_ms=1.0)
five = plan_seams(snapshot, node_count=5, per_hop_latency_ms=1.0)
assert three.decode_latency_ms_per_token == 2.0
assert five.decode_latency_ms_per_token == 4.0
@pytest.mark.parametrize("bad", [float("nan"), float("inf"), 0.0, True])
def test_seam_planner_rejects_invalid_link_telemetry(snapshot, bad):
with pytest.raises(ResourcePlanError, match="finite and positive"):
plan_seams(snapshot, node_count=3, link_rate_gbps=bad)
# --------------------------------------------------------------------------
# The immutable alpha contract
# --------------------------------------------------------------------------
def test_the_packaged_contract_is_sealed_and_verifies(contract):
assert contract.contract_id == ALPHA_CONTRACT_ID
assert contract.contract_version == 1
assert contract.locked_by == "DGR-017"
assert contract.raw["locked_before_target_execution"] is True
assert contract.digest == compute_contract_digest(contract.raw)
def test_the_contract_locks_the_same_target_as_the_manifest(contract, manifest, snapshot):
assert contract.target["source_revision"] == manifest.source_revision
assert contract.target["gguf_revision"] == manifest.gguf_revision
assert contract.target["quantization"] == manifest.quantization
assert contract.target["total_bytes"] == manifest.total_bytes
assert contract.target["target_manifest_sha256"] == manifest.digest
assert contract.target["architecture_snapshot_sha256"] == snapshot.digest
assert contract.target["reasoning_effort"] == "max"
def test_the_packaged_target_documents_are_cross_bound_to_the_contract():
contract, manifest, snapshot = load_locked_target()
assert contract.target["target_manifest_sha256"] == manifest.digest
assert contract.target["architecture_snapshot_sha256"] == snapshot.digest
def test_coordinated_shard_hash_substitution_is_rejected_by_contract(
contract, manifest_doc, snapshot
):
manifest_doc["gguf_artifact"]["shards"][0]["sha256"] = "0" * 64
substituted = parse_target_manifest(manifest_doc)
with pytest.raises(AlphaContractError, match="target_manifest_sha256"):
require_contract_target(contract, substituted, snapshot)
def test_internally_consistent_architecture_substitution_is_rejected_by_contract(
contract, manifest, snapshot_doc
):
snapshot_doc["architecture"]["hidden_size"] = 8192
substituted = parse_architecture_snapshot(snapshot_doc)
with pytest.raises(AlphaContractError, match="architecture_snapshot_sha256"):
require_contract_target(contract, manifest, substituted)
def test_contract_id_cannot_be_changed_even_when_resealed(contract_doc):
contract_doc["contract_id"] = "glm-5.2-max-alpha/v2"
contract_doc = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="contract_id"):
parse_alpha_contract(contract_doc)
def test_max_reasoning_mode_cannot_be_changed_even_when_resealed(contract_doc):
contract_doc["target"]["reasoning_effort"] = "high"
contract_doc = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="reasoning_effort='max'"):
parse_alpha_contract(contract_doc)
@pytest.mark.parametrize("field", ["schema_version", "contract_version"])
def test_boolean_contract_versions_are_rejected_even_when_resealed(contract_doc, field):
contract_doc[field] = True
contract_doc = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="version"):
parse_alpha_contract(contract_doc)
def test_wheel_metadata_includes_nested_glm_alpha_json_resources():
pyproject = Path(__file__).parents[1] / "packages/node/pyproject.toml"
text = pyproject.read_text(encoding="utf-8")
assert '"meshnet_node.glm_alpha" = ["data/*.json"]' in text
def test_the_contract_locks_every_roadmap_acceptance_section(contract):
for section in (
"identity_and_fit",
"semantic_correctness",
"target_run",
"performance",
"reliability",
"storage",
):
assert contract.section(section)
def test_the_contract_locks_the_roadmap_thresholds(contract):
assert contract.threshold("identity_and_fit", "min_node_reserve_fraction") == 0.20
assert contract.threshold("identity_and_fit", "min_node_reserve_gib") == 8.0
assert contract.threshold("identity_and_fit", "aggregate_hard_fit_floor_gib") == 224.0
assert (
contract.threshold("identity_and_fit", "aggregate_floor_class")
== "experimental_hard_fit_floor"
)
assert contract.threshold("identity_and_fit", "forbid_double_counted_unified_memory") is True
assert contract.threshold("semantic_correctness", "min_greedy_token_agreement") == 0.90
assert contract.threshold("semantic_correctness", "min_mean_state_cosine_similarity") == 0.999
assert contract.threshold("semantic_correctness", "dense_attention_fallback_satisfies_alpha") is False
assert contract.threshold("target_run", "context_tokens") == 16_384
assert contract.threshold("target_run", "kv_dtype") == "Q8_0"
assert contract.threshold("target_run", "min_link_rate_gbps") == MIN_LINK_RATE_GBPS
assert contract.threshold("target_run", "recommended_link_rate_gbps") == RECOMMENDED_LINK_RATE_GBPS
assert contract.threshold("target_run", "min_output_tokens") == 512
assert contract.threshold("performance", "min_median_decode_tokens_per_second") == 0.5
assert contract.threshold("performance", "max_ttft_seconds_at_4096_prompt") == 600
assert contract.threshold("performance", "quality_pass_with_speed_fail_verdict") == "stop"
assert contract.threshold("reliability", "synthetic_workers_satisfy_alpha") is False
assert contract.threshold("storage", "forbidden_path_prefixes") == ("/home",)
def test_the_contract_offers_only_alpha_or_stop(contract):
assert sorted(contract.verdicts) == ["alpha", "stop"]
def test_lowering_the_speed_floor_after_seeing_a_result_is_rejected(contract_doc):
"""The exact move the contract exists to prevent."""
contract_doc["performance"]["min_median_decode_tokens_per_second"] = 0.05
with pytest.raises(AlphaContractError, match="modified since it was locked"):
parse_alpha_contract(contract_doc)
def test_relabelling_a_speed_failure_as_a_pass_is_rejected(contract_doc):
contract_doc["performance"]["quality_pass_with_speed_fail_verdict"] = "alpha"
with pytest.raises(AlphaContractError, match="modified since it was locked"):
parse_alpha_contract(contract_doc)
def test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected(contract_doc):
contract_doc["semantic_correctness"]["dense_attention_fallback_satisfies_alpha"] = True
with pytest.raises(AlphaContractError, match="modified since it was locked"):
parse_alpha_contract(contract_doc)
def test_relaxing_the_per_node_reserve_after_the_fact_is_rejected(contract_doc):
contract_doc["identity_and_fit"]["min_node_reserve_gib"] = 1.0
with pytest.raises(AlphaContractError, match="modified since it was locked"):
parse_alpha_contract(contract_doc)
def test_re_pointing_the_contract_at_a_different_artifact_is_rejected(contract_doc):
contract_doc["target"]["gguf_revision"] = "9" * 40
with pytest.raises(AlphaContractError, match="modified since it was locked"):
parse_alpha_contract(contract_doc)
def test_an_unsealed_contract_is_rejected(contract_doc):
del contract_doc["contract_sha256"]
with pytest.raises(AlphaContractError, match="cannot prove it predates"):
parse_alpha_contract(contract_doc)
def test_a_contract_not_locked_before_execution_is_rejected(contract_doc):
contract_doc["locked_before_target_execution"] = False
contract_doc = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="not a contract"):
parse_alpha_contract(contract_doc)
def test_a_third_verdict_is_rejected(contract_doc):
contract_doc["verdicts"] = ["alpha", "stop", "partial"]
contract_doc = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="third outcome"):
parse_alpha_contract(contract_doc)
def test_a_dropped_acceptance_section_is_rejected(contract_doc):
del contract_doc["reliability"]
contract_doc = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="missing locked acceptance section"):
parse_alpha_contract(contract_doc)
def test_resealing_a_mutated_v1_contract_is_rejected(contract, contract_doc):
contract_doc["performance"]["min_median_decode_tokens_per_second"] = 0.05
resealed = seal_contract(contract_doc)
with pytest.raises(AlphaContractError, match="trusted pre-execution digest"):
parse_alpha_contract(resealed)
assert resealed["contract_sha256"] != contract.digest
def test_parsed_contract_nested_state_is_immutable(contract):
with pytest.raises(TypeError):
contract.raw["performance"]["min_median_decode_tokens_per_second"] = 0.05
with pytest.raises(TypeError):
contract.target["reasoning_effort"] = "high"
def test_contract_to_dict_returns_an_isolated_mutable_copy(contract):
copied = contract.to_dict()
copied["performance"]["min_median_decode_tokens_per_second"] = 0.05
assert contract.threshold("performance", "min_median_decode_tokens_per_second") == 0.5
def test_an_unknown_threshold_cannot_be_invented_at_read_time(contract):
with pytest.raises(AlphaContractError, match="is not locked"):
contract.threshold("performance", "min_decode_tokens_per_second_v2")
# --------------------------------------------------------------------------
# The tests themselves stay offline
# --------------------------------------------------------------------------
def test_the_packaged_data_files_are_valid_json_and_load_offline(manifest, snapshot, contract):
"""No network, no model, no GPU: the whole target contract is reviewable offline."""
assert json.dumps(manifest.to_dict())
assert json.dumps(snapshot.to_dict())
assert json.dumps(contract.to_dict())
assert len(manifest.digest) == 64

577
tests/test_native_shard_protocol.py Normal file
View File

@@ -0,0 +1,577 @@
"""Conformance tests for the native Shard protocol (ADR-0020, DGR-002).
Three layers are tested, and they are not the same thing:
1. The *schema* — asserted against the descriptor, not against the Python
helpers. If a field the protocol promises to carry were dropped from the
`.proto`, a test that only exercised the codec would still pass.
2. The *codec* — that a payload which is corrupt, short, holed, or byte-swapped
is rejected rather than fed to a forward pass.
3. *Compatibility* — that an old build preserves fields a newer peer added, and
that the committed cross-language vectors still encode as promised.
"""
from __future__ import annotations
import pathlib
import subprocess
import sys
import pytest
pytest.importorskip("google.protobuf", reason="protobuf runtime is required")
from google.protobuf import descriptor_pb2
from meshnet_node.activation_compression import CompressionPolicy
from meshnet_node.native_protocol import (
DEFAULT_MAX_CHUNK_BYTES,
DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
DEFAULT_MAX_PREFILL_CHUNK_TOKENS,
DEFAULT_MAX_TENSORS_PER_BUNDLE,
HIDDEN_STATES,
PayloadCorrupt,
ProtocolError,
checksum_of,
decode_bundle,
decode_tensor,
default_flow_control,
encode_bundle,
encode_tensor,
negotiate_flow_control,
pb,
plan_prefill_chunks,
validate_session_message_size,
)
from meshnet_node.native_protocol import conformance
REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent
CPP_ROUNDTRIP = REPO_ROOT / "build/native" / conformance.CPP_ROUNDTRIP
# --- The schema itself ------------------------------------------------------
def test_service_exposes_capability_health_session_release_and_cancel():
service = pb.DESCRIPTOR.services_by_name["ShardRuntime"]
assert set(service.methods_by_name) == {
"GetCapability",
"Health",
"Session",
"Release",
"Cancel",
}
def test_session_is_one_long_lived_bidirectional_stream():
session = pb.DESCRIPTOR.services_by_name["ShardRuntime"].methods_by_name["Session"]
assert session.client_streaming, "the seam must stream requests"
assert session.server_streaming, "the seam must stream responses"
# Cancellation must not have to travel down a stream that flow control has
# wedged, so it also exists as its own unary call.
cancel = pb.DESCRIPTOR.services_by_name["ShardRuntime"].methods_by_name["Cancel"]
assert not cancel.client_streaming and not cancel.server_streaming
def test_envelope_carries_every_field_the_protocol_promises():
# Asserted against the descriptor: this is the acceptance criterion, and it
# must fail if the .proto drops a field, not merely if the codec stops
# setting one.
fields = set(pb.Envelope.DESCRIPTOR.fields_by_name)
assert {
"schema_version",
"work_id",
"route_session_id",
"route_epoch",
"fingerprint",
"shard_range",
"phase",
"position",
"idempotency_step",
"cache_expectation",
"deadline_unix_nanos",
"chunk",
} <= fields
assert {"model_artifact_digest", "runtime_recipe_digest"} <= set(
pb.Fingerprint.DESCRIPTOR.fields_by_name
)
# Overlap-safe start (ADR-0012) is a distinct field from the registered one.
assert {"start_layer", "end_layer", "effective_start_layer"} <= set(
pb.ShardRange.DESCRIPTOR.fields_by_name
)
assert {"mode", "expected_past_len"} <= set(
pb.CacheExpectation.DESCRIPTOR.fields_by_name
)
def test_named_tensor_bundle_is_versioned_and_fully_described():
assert "bundle_version" in pb.TensorBundle.DESCRIPTOR.fields_by_name
assert {"name", "shape", "dtype", "byte_order", "compression", "checksum", "fragments"} <= set(
pb.NamedTensor.DESCRIPTOR.fields_by_name
)
def test_phases_and_error_codes_cover_the_lifecycle():
assert {"PHASE_PREFILL", "PHASE_DECODE", "PHASE_RELEASE", "PHASE_CANCEL"} <= set(
pb.Phase.keys()
)
# A cache miss is a first-class, recoverable outcome (ADR-0022), not a crash.
assert {
"ERROR_CODE_CACHE_MISS",
"ERROR_CODE_FINGERPRINT_MISMATCH",
"ERROR_CODE_EPOCH_STALE",
"ERROR_CODE_SCHEMA_UNSUPPORTED",
"ERROR_CODE_DEADLINE_EXCEEDED",
"ERROR_CODE_FLOW_CONTROL_VIOLATION",
} <= set(pb.ErrorCode.keys())
# --- Tensor bundle round-trip ----------------------------------------------
def test_tensor_round_trips_through_fragments():
payload = bytes(range(256)) * 4
tensor = encode_tensor(
HIDDEN_STATES, payload, [1, 64, 8], pb.DTYPE_BFLOAT16, max_fragment_bytes=100
)
assert len(tensor.fragments) > 1, "the bound must actually split the payload"
assert tensor.total_bytes == len(payload)
assert decode_tensor(tensor) == payload
def test_bundle_round_trips_multiple_named_tensors():
# An architecture boundary may need more than one tensor; that is why the
# payload is a named bundle rather than a bare buffer.
hidden = encode_tensor(HIDDEN_STATES, b"\x01\x02" * 8, [1, 8, 1], pb.DTYPE_BFLOAT16)
positions = encode_tensor(
"position_ids", (7).to_bytes(4, "little") * 8, [8], pb.DTYPE_INT32
)
bundle = encode_bundle([hidden, positions])
restored = decode_bundle(pb.TensorBundle.FromString(bundle.SerializeToString()))
assert restored == {
HIDDEN_STATES: b"\x01\x02" * 8,
"position_ids": (7).to_bytes(4, "little") * 8,
}
def test_compressed_tensor_round_trips_and_keeps_its_uncompressed_checksum():
pytest.importorskip("zstandard")
# Highly compressible, and over the policy's minimum input size.
payload = b"\x00" * 65536
always = CompressionPolicy(min_input_bytes=0, min_savings_bytes=0, min_savings_ratio=0.0)
tensor = encode_tensor(
HIDDEN_STATES, payload, [1, 4096, 8], pb.DTYPE_BFLOAT16, policy=always
)
assert tensor.compression == pb.COMPRESSION_ZSTD
assert sum(len(f.payload) for f in tensor.fragments) < len(payload)
# The checksum covers the uncompressed bytes, so it stays valid whether or
# not a hop chose to compress.
assert tensor.checksum == checksum_of(payload)
assert decode_tensor(tensor) == payload
# --- The codec refuses what it cannot account for --------------------------
def test_corrupt_payload_is_rejected_by_checksum():
tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16)
# Flip one byte, as a lossy relay or a bad NIC would.
tensor.fragments[0].payload = b"\xab" + tensor.fragments[0].payload[1:]
with pytest.raises(PayloadCorrupt, match="CRC32C"):
decode_tensor(tensor)
def test_missing_fragment_is_rejected_rather_than_silently_truncated():
tensor = encode_tensor(
HIDDEN_STATES, b"\xaa" * 64, [1, 32, 1], pb.DTYPE_BFLOAT16, max_fragment_bytes=16
)
del tensor.fragments[1]
with pytest.raises(PayloadCorrupt):
decode_tensor(tensor)
def test_fragment_hole_is_rejected():
tensor = encode_tensor(
HIDDEN_STATES, b"\xaa" * 64, [1, 32, 1], pb.DTYPE_BFLOAT16, max_fragment_bytes=16
)
# A gap in coverage: offsets no longer tile the body.
tensor.fragments[2].byte_offset += 4
with pytest.raises(PayloadCorrupt, match="expected"):
decode_tensor(tensor)
def test_shape_that_disagrees_with_payload_is_rejected_at_encode():
with pytest.raises(ProtocolError, match="carries"):
encode_tensor(HIDDEN_STATES, b"\x01\x02", [1, 8, 8], pb.DTYPE_BFLOAT16)
def test_shape_that_disagrees_with_declared_bytes_is_rejected_at_decode():
tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16)
# A peer claiming a larger tensor than its bytes describe.
tensor.shape[1] = 32
with pytest.raises(PayloadCorrupt, match="implies"):
decode_tensor(tensor)
def test_big_endian_tensor_is_rejected_loudly():
tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16)
tensor.byte_order = pb.BYTE_ORDER_BIG_ENDIAN
# Byte-swapped activations would be plausible-looking garbage, so this is an
# error rather than a best-effort read.
with pytest.raises(ProtocolError, match="big-endian"):
decode_tensor(tensor)
def test_bundle_from_a_newer_layout_is_refused():
bundle = encode_bundle([])
bundle.bundle_version = 99
with pytest.raises(ProtocolError, match="not supported"):
decode_bundle(bundle)
def test_bundle_with_unspecified_version_is_refused():
with pytest.raises(ProtocolError, match="version 0"):
decode_bundle(pb.TensorBundle())
def test_tensor_with_unspecified_compression_is_refused():
tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16)
tensor.compression = pb.COMPRESSION_UNSPECIFIED
with pytest.raises(ProtocolError, match="unspecified"):
decode_tensor(tensor)
def test_tensor_with_unspecified_checksum_is_refused():
tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16)
tensor.checksum.algorithm = pb.CHECKSUM_ALGORITHM_UNSPECIFIED
with pytest.raises(ProtocolError, match="unspecified"):
decode_tensor(tensor)
def test_declared_tensor_size_cannot_exceed_negotiated_chunk_bound():
tensor = pb.NamedTensor(
name=HIDDEN_STATES,
shape=[1],
dtype=pb.DTYPE_UINT8,
byte_order=pb.BYTE_ORDER_LITTLE_ENDIAN,
total_bytes=DEFAULT_MAX_CHUNK_BYTES + 1,
compression=pb.COMPRESSION_NONE,
)
with pytest.raises(ProtocolError, match="negotiated chunk bound"):
decode_tensor(tensor)
def test_stricter_session_negotiation_overrides_global_defaults():
tensor = encode_tensor(
HIDDEN_STATES,
b"\xaa" * 32,
[32],
pb.DTYPE_UINT8,
max_fragment_bytes=32,
)
with pytest.raises(ProtocolError, match="16-byte negotiated"):
decode_tensor(tensor, max_chunk_bytes=16, max_fragment_bytes=16)
with pytest.raises(ProtocolError, match="16-byte"):
decode_bundle(
encode_bundle([tensor]),
max_chunk_bytes=16,
max_fragment_bytes=16,
)
def test_fragment_cannot_exceed_protocol_fragment_bound():
tensor = encode_tensor(
HIDDEN_STATES,
b"\xaa" * (1024 * 1024 + 1),
[1024 * 1024 + 1],
pb.DTYPE_UINT8,
max_fragment_bytes=1024 * 1024 + 1,
)
with pytest.raises(PayloadCorrupt, match="fragment larger"):
decode_tensor(tensor)
def test_fragment_count_is_bounded_before_sorting_or_allocation():
tensor = encode_tensor(
HIDDEN_STATES,
b"\xaa" * (DEFAULT_MAX_FRAGMENTS_PER_TENSOR + 1),
[DEFAULT_MAX_FRAGMENTS_PER_TENSOR + 1],
pb.DTYPE_UINT8,
max_fragment_bytes=1,
max_fragments=DEFAULT_MAX_FRAGMENTS_PER_TENSOR + 1,
)
with pytest.raises(PayloadCorrupt, match="fragments.*exceeding"):
decode_tensor(tensor)
def test_tensor_count_is_bounded_before_bundle_decoding():
tensors = [
pb.NamedTensor(name=f"t-{index}")
for index in range(DEFAULT_MAX_TENSORS_PER_BUNDLE + 1)
]
with pytest.raises(ProtocolError, match="tensors.*exceeding"):
encode_bundle(tensors)
@pytest.mark.parametrize(
("shape", "match"),
[
([1] * 9, "rank"),
([-1], "dimension outside"),
([1 << 31], "dimension outside"),
],
)
def test_shape_rank_and_dimensions_are_bounded_before_payload_work(shape, match):
with pytest.raises(ProtocolError, match=match):
encode_tensor(HIDDEN_STATES, b"\x00", shape, pb.DTYPE_UINT8)
def test_complete_session_message_includes_envelope_in_byte_bound():
small = pb.SessionRequest(chunk=pb.ActivationChunk(bundle=encode_bundle([])))
assert validate_session_message_size(small, max_chunk_bytes=64) == small.ByteSize()
oversized = pb.SessionRequest(
chunk=pb.ActivationChunk(
envelope=pb.Envelope(work_id="w" * 128),
bundle=encode_bundle([]),
)
)
with pytest.raises(ProtocolError, match="serialized session message"):
validate_session_message_size(oversized, max_chunk_bytes=64)
def test_bundle_serialized_size_is_bounded():
payload = b"\xaa" * (DEFAULT_MAX_CHUNK_BYTES // 2)
first = encode_tensor("first", payload, [len(payload)], pb.DTYPE_UINT8)
second = encode_tensor("second", payload, [len(payload)], pb.DTYPE_UINT8)
with pytest.raises(ProtocolError, match="serialized tensor bundle"):
encode_bundle([first, second])
def test_compressed_tensor_cannot_expand_past_declared_size():
pytest.importorskip("zstandard")
payload = b"\x00" * 65536
always = CompressionPolicy(min_input_bytes=0, min_savings_bytes=0, min_savings_ratio=0.0)
tensor = encode_tensor(
HIDDEN_STATES, payload, [len(payload)], pb.DTYPE_UINT8, policy=always
)
assert tensor.compression == pb.COMPRESSION_ZSTD
# Simulate a hostile frame that advertises a small output while expanding to
# the original 64 KiB. Shape and total_bytes agree, so only the bounded
# decompressor protects the receiver.
tensor.shape[:] = [1024]
tensor.total_bytes = 1024
with pytest.raises(PayloadCorrupt, match="bounded zstd"):
decode_tensor(tensor)
def test_generated_runtime_floors_match_committed_stubs():
metadata = (REPO_ROOT / "packages/node/pyproject.toml").read_text()
assert '"grpcio>=1.82.1"' in metadata
assert '"protobuf>=7.35.0"' in metadata
# --- Bounded prefill chunking and the decode fast path ----------------------
def test_prefill_is_split_into_bounded_token_aligned_chunks():
chunks = plan_prefill_chunks(2048)
assert all(c.token_count <= DEFAULT_MAX_PREFILL_CHUNK_TOKENS for c in chunks)
assert sum(c.token_count for c in chunks) == 2048
# Contiguous, token-aligned, and no split falls mid-token.
assert [c.first_position for c in chunks] == [
i * DEFAULT_MAX_PREFILL_CHUNK_TOKENS for i in range(len(chunks))
]
assert [c.final_chunk for c in chunks] == [False] * (len(chunks) - 1) + [True]
def test_final_prefill_chunk_carries_the_remainder():
chunks = plan_prefill_chunks(300, max_tokens=128)
assert [c.token_count for c in chunks] == [128, 128, 44]
assert chunks[-1].chunk_info().final_chunk
assert chunks[0].position().first_position == 0
assert chunks[-1].position().first_position == 256
def test_empty_prefill_is_refused():
with pytest.raises(ProtocolError):
plan_prefill_chunks(0)
def test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk():
hidden = b"\x01\x02" * 8 # one token, hidden=8, bfloat16
tensor = encode_tensor(HIDDEN_STATES, hidden, [1, 1, 8], pb.DTYPE_BFLOAT16)
fast = pb.SessionRequest(
decode=pb.DecodeStep(
idempotency_step=9,
position=1024,
expected_past_len=1024,
tensor=tensor,
work_id="work-7f3a",
)
)
# The same single token carried the long way, repeating identity that the
# handshake already fixed for the life of the stream.
full = pb.SessionRequest(
chunk=pb.ActivationChunk(
envelope=conformance.canonical_session_request().chunk.envelope,
bundle=encode_bundle([tensor]),
)
)
assert len(fast.SerializeToString()) * 2 < len(full.SerializeToString())
assert decode_tensor(fast.decode.tensor) == hidden
def test_flow_control_defaults_bound_the_queue_and_the_message():
limits = default_flow_control()
assert limits.max_prefill_chunk_tokens == DEFAULT_MAX_PREFILL_CHUNK_TOKENS
assert limits.max_chunk_bytes == DEFAULT_MAX_CHUNK_BYTES
assert limits.max_inflight_chunks > 0
def test_flow_control_negotiation_takes_the_strictest_bound():
proposed = pb.FlowControl(
credits_granted=64, max_inflight_chunks=64, max_chunk_bytes=64 << 20,
max_prefill_chunk_tokens=1024,
)
settled = negotiate_flow_control(proposed, default_flow_control())
# A sender cannot talk a worker into unbounded queues by proposing a large
# window: neither peer can raise the other's ceiling.
assert settled.max_inflight_chunks == default_flow_control().max_inflight_chunks
assert settled.credits_granted <= settled.max_inflight_chunks
assert settled.max_chunk_bytes == DEFAULT_MAX_CHUNK_BYTES
assert settled.max_prefill_chunk_tokens == DEFAULT_MAX_PREFILL_CHUNK_TOKENS
def test_initial_credits_never_exceed_negotiated_inflight_limit():
proposed = pb.FlowControl(credits_granted=64, max_inflight_chunks=64)
limits = pb.FlowControl(credits_granted=64, max_inflight_chunks=8)
settled = negotiate_flow_control(proposed, limits)
assert settled.credits_granted == 8
assert settled.max_inflight_chunks == 8
# --- Compatibility ----------------------------------------------------------
def test_committed_vectors_still_encode_as_promised():
# The C++ test asserts against these exact bytes. If a schema change alters
# the canonical encoding, it must be acknowledged by regenerating them.
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes()
assert conformance.serialize(conformance.canonical_session_request()) == golden
report = (conformance.TESTDATA_DIR / conformance.GOLDEN_CAPABILITY_REPORT).read_bytes()
assert conformance.serialize(conformance.canonical_capability_report()) == report
def test_golden_session_request_round_trips_with_every_field_intact():
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes()
request = pb.SessionRequest.FromString(golden)
envelope = request.chunk.envelope
assert envelope.work_id == conformance.WORK_ID
assert envelope.route_session_id == conformance.ROUTE_SESSION_ID
assert envelope.route_epoch == conformance.ROUTE_EPOCH
assert envelope.idempotency_step == conformance.IDEMPOTENCY_STEP
assert envelope.shard_range.effective_start_layer == conformance.EFFECTIVE_START_LAYER
assert envelope.phase == pb.PHASE_PREFILL
assert envelope.cache_expectation.mode == pb.CACHE_MODE_PREFILL
assert envelope.deadline_unix_nanos == conformance.DEADLINE_UNIX_NANOS
assert decode_bundle(request.chunk.bundle) == {
HIDDEN_STATES: conformance.canonical_payload()
}
assert request.SerializeToString(deterministic=True) == golden
def test_unknown_fields_from_a_newer_peer_survive_a_forwarding_hop():
# A Shard forwards activations onward. If it silently dropped fields a newer
# peer added, it would corrupt a route it is merely a waypoint on.
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes()
future_field = b"\xb8\xe0\x04\xb9\x60" # field 9999, varint 12345
request = pb.SessionRequest.FromString(golden + future_field)
assert request.chunk.envelope.work_id == conformance.WORK_ID
assert request.SerializeToString() == golden + future_field
def test_a_message_missing_newer_field_groups_still_parses():
sparse = pb.SessionRequest(chunk=pb.ActivationChunk(envelope=pb.Envelope(work_id="w")))
parsed = pb.SessionRequest.FromString(sparse.SerializeToString())
assert parsed.chunk.envelope.work_id == "w"
assert parsed.chunk.envelope.route_epoch == 0
assert parsed.chunk.envelope.phase == pb.PHASE_UNSPECIFIED
def test_retired_fragment_field_stays_reserved():
# `uncompressed_size` (field 5) was removed because NamedTensor.total_bytes
# is the single source of truth. The number stays reserved so it can never
# be recycled for a different meaning — a recycled number is the one schema
# change that old and new peers cannot detect, because the bytes still parse.
descriptor = descriptor_pb2.DescriptorProto()
pb.TensorFragment.DESCRIPTOR.CopyToProto(descriptor)
assert 5 not in {field.number for field in descriptor.field}
assert any(r.start <= 5 < r.end for r in descriptor.reserved_range)
assert "uncompressed_size" in descriptor.reserved_name
def test_a_peer_still_sending_the_retired_field_does_not_corrupt_the_tensor():
# An older peer that still sets field 5 must be parsed, not rejected: the
# value lands in unknown fields and the payload is unaffected.
tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16)
wire = tensor.SerializeToString()
fragment = pb.TensorFragment.FromString(tensor.fragments[0].SerializeToString() + b"\x28\x20")
assert fragment.payload == tensor.fragments[0].payload
assert decode_tensor(pb.NamedTensor.FromString(wire)) == b"\xaa" * 32
def test_generated_python_stubs_match_the_proto():
pytest.importorskip("grpc_tools", reason="protoc toolchain is required to verify")
result = subprocess.run(
[sys.executable, str(REPO_ROOT / "scripts/generate_native_protocol.py"), "--check"],
capture_output=True,
text=True,
)
assert result.returncode == 0, result.stdout + result.stderr
@pytest.mark.skipif(
not CPP_ROUNDTRIP.is_file(),
reason="build the C++ conformance test to check cross-language agreement",
)
def test_cpp_and_python_agree_byte_for_byte():
# Written by the C++ conformance test: it parsed the golden bytes into its
# own object model and serialized them back. Byte equality means both
# languages encode every field of this schema identically.
golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes()
assert CPP_ROUNDTRIP.read_bytes() == golden

931
tests/test_recipe_benchmark.py Normal file
View File

@@ -0,0 +1,931 @@
"""The recipe benchmark's measurement core, driven by a scripted fake runtime.
These tests never load a model, touch a GPU, or open a socket: the core is
deliberately runtime-free so the arithmetic and the lane rules can be pinned
down exactly, and the real drivers only have to be honest about what they
report.
"""
from __future__ import annotations
import base64
import copy
import hashlib
import json
import time
from dataclasses import replace
from pathlib import Path
import pytest
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from meshnet_node import recipe_benchmark as recipe_benchmark_module
from meshnet_node import recipe_drivers as recipe_drivers_module
from meshnet_node.performance_contract import (
PROVENANCE_SCHEMA_VERSION,
REAL_REPORT_PRODUCER,
STOP_CONDITION,
ContractThresholds,
PerformanceContract,
PerformanceContractError,
_canonical_sha256,
evaluate_contract,
parse_contract,
report_signing_payload,
)
from meshnet_node.recipe_drivers import (
CONTRACT_V1_PROFILE,
GPU_DIAGNOSTIC_PROFILE,
GPU_DIAGNOSTIC_REPORT_PRODUCER,
_artifact_sha256,
_gpu_offload_evidence,
_gpu_layer_config_detail,
_producer_for_profile,
_validate_config,
build_driver,
require_real_inference,
)
from meshnet_node.recipe_benchmark import (
BenchmarkError,
BenchmarkPlan,
GenerationSample,
Lane,
LoadStats,
PromptSpec,
RecipeSpec,
SamplingPolicy,
build_report,
compute_drift,
measure_recipe,
summarize_concurrency,
RequestOutcome,
)
PROMPTS = (
PromptSpec(id="short", text="Say hello.", context_class="short"),
PromptSpec(id="long", text="Summarize the following. " * 40, context_class="long"),
)
def plan(**overrides) -> BenchmarkPlan:
defaults = dict(
plan_id="test-plan",
model_id="test/model",
model_revision="revision-1",
prompts=PROMPTS,
sampling=SamplingPolicy(max_output_tokens=8),
concurrency_levels=(1, 4),
repeats=1,
warmup_requests=0,
)
defaults.update(overrides)
return BenchmarkPlan(**defaults)
class FakeDriver:
"""A runtime with fixed timings, so every metric below has one right answer."""
def __init__(
self,
*,
decode_ms_per_token: float = 10.0,
prefill_ms: float = 100.0,
artifact_bytes: int = 1_000_000,
rss_bytes: int = 4_000_000,
vram_bytes: int = 0,
texts: dict[str, str] | None = None,
fail_at_concurrency: int | None = None,
decode_tokens: int = 8,
generation_delay_s: float = 0.0,
) -> None:
self.decode_ms_per_token = decode_ms_per_token
self.prefill_ms = prefill_ms
self.artifact_bytes = artifact_bytes
self.rss_bytes = rss_bytes
self.vram_bytes = vram_bytes
self.texts = texts or {}
self.fail_at_concurrency = fail_at_concurrency
self.decode_tokens = decode_tokens
self.generation_delay_s = generation_delay_s
self.in_flight = 0
self.max_in_flight = 0
self.loads = 0
self.closes = 0
self.generations = 0
def load(self) -> LoadStats:
self.loads += 1
return LoadStats(
artifact_bytes=self.artifact_bytes, load_ms=50.0,
rss_bytes=self.rss_bytes, vram_bytes=self.vram_bytes,
)
def generate(self, prompt: str, sampling: SamplingPolicy) -> GenerationSample:
self.in_flight += 1
self.max_in_flight = max(self.max_in_flight, self.in_flight)
try:
if self.generation_delay_s:
time.sleep(self.generation_delay_s)
if self.fail_at_concurrency and self.in_flight >= self.fail_at_concurrency:
raise RuntimeError("slot exhausted")
self.generations += 1
decode_ms = self.decode_ms_per_token * self.decode_tokens
return GenerationSample(
text=self.texts.get(prompt, "hello world"),
prompt_tokens=10,
decode_tokens=self.decode_tokens,
ttft_ms=self.prefill_ms,
prefill_ms=self.prefill_ms,
decode_ms=decode_ms,
total_ms=self.prefill_ms + decode_ms,
)
finally:
self.in_flight -= 1
def memory_probe(self) -> tuple[int, int]:
return self.rss_bytes, self.vram_bytes
def close(self) -> None:
self.closes += 1
def recipe(recipe_id: str, lane: Lane, *, reference: bool = False, device: str = "cpu") -> RecipeSpec:
return RecipeSpec(
id=recipe_id, runtime="fake", weight_format="fake", weight_quantization="bf16",
lane=lane, device=device, is_reference=reference,
)
def test_plan_rejects_an_experiment_it_cannot_run():
with pytest.raises(BenchmarkError):
plan(prompts=())
with pytest.raises(BenchmarkError):
plan(concurrency_levels=(0,))
with pytest.raises(BenchmarkError):
plan(repeats=0)
def test_measure_runs_every_prompt_at_every_concurrency_level():
driver = FakeDriver()
measurement = measure_recipe(driver, recipe("r", Lane.QUALITY, reference=True), plan())
# 2 prompts x (1 + 4) requests-per-level.
assert len(measurement.outcomes) == 2 * 1 + 2 * 4
assert sorted(measurement.metrics) == [1, 4]
assert driver.loads == 1
assert driver.closes == 1
assert measurement.available
def test_concurrency_level_actually_overlaps_requests():
driver = FakeDriver(decode_ms_per_token=5.0, generation_delay_s=0.02)
measure_recipe(driver, recipe("r", Lane.QUALITY, reference=True), plan(concurrency_levels=(1, 4)))
assert driver.max_in_flight > 1, "concurrency 4 must run requests in parallel, not serially"
def test_driver_is_closed_even_when_every_request_fails():
driver = FakeDriver(fail_at_concurrency=1)
measurement = measure_recipe(driver, recipe("r", Lane.QUALITY, reference=True), plan())
assert driver.closes == 1
assert all(not outcome.ok for outcome in measurement.outcomes)
assert measurement.metrics[1].failures == 2
assert measurement.metrics[1].failure_reasons == ("RuntimeError: slot exhausted",)
def test_failed_requests_are_reported_not_raised():
driver = FakeDriver(fail_at_concurrency=4, generation_delay_s=0.02)
measurement = measure_recipe(driver, recipe("r", Lane.QUALITY, reference=True), plan())
assert measurement.metrics[1].failures == 0
assert measurement.metrics[4].failures > 0
assert measurement.metrics[4].requests == 8
def test_summary_arithmetic_is_exact():
outcomes = [
RequestOutcome(
recipe_id="r", concurrency=2, prompt_id="p", repeat=0, ok=True,
latency_ms=200.0, ttft_ms=100.0, prefill_ms=100.0, decode_ms=100.0,
prompt_tokens=10, decode_tokens=10,
),
RequestOutcome(
recipe_id="r", concurrency=2, prompt_id="p", repeat=1, ok=True,
latency_ms=400.0, ttft_ms=200.0, prefill_ms=200.0, decode_ms=200.0,
prompt_tokens=10, decode_tokens=10,
),
]
metrics = summarize_concurrency(
outcomes, concurrency=2, wall_ms=1000.0, peak_rss_bytes=7, peak_vram_bytes=9
)
assert metrics.latency_p50_ms == 200.0
assert metrics.latency_p95_ms == 400.0
# 10 tok / 0.1 s = 100 tok/s and 10 tok / 0.2 s = 50 tok/s, averaged.
assert metrics.decode_tokens_per_sec == 75.0
# 20 decoded tokens over a 1 s wall clock, regardless of per-request rates.
assert metrics.aggregate_decode_tokens_per_sec == 20.0
assert (metrics.peak_rss_bytes, metrics.peak_vram_bytes) == (7, 9)
def test_aggregate_throughput_credits_overlap_but_per_request_rate_does_not():
"""Two runtimes with identical per-request speed must be told apart by overlap."""
serial = summarize_concurrency(
[
RequestOutcome(recipe_id="s", concurrency=4, prompt_id="p", repeat=i, ok=True,
latency_ms=100.0, decode_ms=100.0, decode_tokens=10)
for i in range(4)
],
concurrency=4, wall_ms=400.0, peak_rss_bytes=0, peak_vram_bytes=0,
)
batched = summarize_concurrency(
[
RequestOutcome(recipe_id="b", concurrency=4, prompt_id="p", repeat=i, ok=True,
latency_ms=100.0, decode_ms=100.0, decode_tokens=10)
for i in range(4)
],
concurrency=4, wall_ms=100.0, peak_rss_bytes=0, peak_vram_bytes=0,
)
assert serial.decode_tokens_per_sec == batched.decode_tokens_per_sec == 100.0
assert serial.aggregate_decode_tokens_per_sec == 100.0
assert batched.aggregate_decode_tokens_per_sec == 400.0
def test_drift_against_the_reference_is_exact_for_an_identical_runtime():
texts = {prompt.text: f"answer for {prompt.id}" for prompt in PROMPTS}
reference = measure_recipe(
FakeDriver(texts=texts), recipe("ref", Lane.QUALITY, reference=True), plan()
)
twin = measure_recipe(FakeDriver(texts=texts), recipe("twin", Lane.QUALITY), plan())
drift = compute_drift(twin, reference)
assert drift.compared_prompts == 2
assert drift.exact_match_rate == 1.0
assert drift.mean_similarity == 1.0
assert drift.advisory is False
def test_quantized_drift_is_advisory_and_never_an_equivalence_claim():
reference = measure_recipe(
FakeDriver(texts={prompt.text: "the capital is Paris" for prompt in PROMPTS}),
recipe("ref", Lane.QUALITY, reference=True), plan(),
)
quantized = measure_recipe(
FakeDriver(texts={prompt.text: "the capital is Lyon" for prompt in PROMPTS}),
recipe("q4", Lane.PERFORMANCE_FIT), plan(),
)
drift = compute_drift(quantized, reference)
assert drift.advisory is True, "a quantized recipe's drift must be advisory"
assert drift.exact_match_rate == 0.0
assert 0.0 < drift.mean_similarity < 1.0
assert drift.per_prompt[0]["first_divergence_char"] > 0
def test_report_needs_exactly_one_quality_lane_reference():
measurement = measure_recipe(FakeDriver(), recipe("a", Lane.QUALITY, reference=True), plan())
second = measure_recipe(FakeDriver(), recipe("b", Lane.QUALITY, reference=True), plan())
quantized = measure_recipe(FakeDriver(), recipe("q", Lane.PERFORMANCE_FIT), plan())
with pytest.raises(BenchmarkError, match="exactly one reference"):
build_report(plan(), [measurement, second], host={}, evidence_class="synthetic")
with pytest.raises(BenchmarkError, match="exactly one reference"):
build_report(plan(), [quantized], host={}, evidence_class="synthetic")
def test_reference_recipe_may_not_be_quantized():
quantized_reference = measure_recipe(
FakeDriver(), recipe("q", Lane.PERFORMANCE_FIT, reference=True), plan()
)
with pytest.raises(BenchmarkError, match="quality lane"):
build_report(plan(), [quantized_reference], host={}, evidence_class="synthetic")
def test_report_must_declare_how_it_was_produced():
measurement = measure_recipe(FakeDriver(), recipe("a", Lane.QUALITY, reference=True), plan())
with pytest.raises(BenchmarkError, match="evidence class"):
build_report(plan(), [measurement], host={}, evidence_class="probably-real")
def test_report_carries_every_metric_the_contract_reads():
reference = measure_recipe(FakeDriver(), recipe("ref", Lane.QUALITY, reference=True), plan())
quantized = measure_recipe(
FakeDriver(decode_ms_per_token=4.0, artifact_bytes=400_000, rss_bytes=1_000_000),
recipe("q4", Lane.PERFORMANCE_FIT), plan(),
)
report = build_report(
plan(), [reference, quantized], host={"cpu": "test"}, evidence_class="synthetic"
)
assert report["schema_version"] == 1
assert report["reference_recipe_id"] == "ref"
entry = next(e for e in report["recipes"] if e["recipe"]["id"] == "q4")
cell = entry["concurrency"]["1"]
for metric in (
"ttft_p50_ms", "ttft_p95_ms", "latency_p50_ms", "latency_p95_ms",
"prefill_tokens_per_sec", "decode_tokens_per_sec", "aggregate_decode_tokens_per_sec",
"peak_rss_bytes", "peak_vram_bytes", "failures",
):
assert metric in cell, f"the contract reads {metric}, so the report must carry it"
assert entry["load"]["artifact_bytes"] == 400_000
assert [d["recipe_id"] for d in report["drift"]] == ["q4"]
def test_unavailable_recipes_are_recorded_rather_than_dropped():
from meshnet_node.recipe_benchmark import RecipeMeasurement
reference = measure_recipe(FakeDriver(), recipe("ref", Lane.QUALITY, reference=True), plan())
missing = RecipeMeasurement(
recipe=recipe("q4", Lane.PERFORMANCE_FIT),
load=LoadStats(artifact_bytes=0, load_ms=0.0),
unavailable_reason="BenchmarkError: GGUF artifact not found",
)
report = build_report(plan(), [reference, missing], host={}, evidence_class="synthetic")
entry = next(e for e in report["recipes"] if e["recipe"]["id"] == "q4")
assert entry["available"] is False
assert "not found" in entry["unavailable_reason"]
assert report["drift"] == [], "an unmeasured recipe has no drift to report"
def test_contract_requires_a_quality_lane_then_allows_quantized_fit_benefit():
texts = {prompt.text: "same greedy answer" for prompt in PROMPTS}
reference = measure_recipe(
FakeDriver(texts=texts, rss_bytes=4_000_000), recipe("safetensors", Lane.QUALITY, reference=True), plan()
)
quality = measure_recipe(
FakeDriver(texts=texts), recipe("gguf-f16", Lane.QUALITY), plan()
)
q4 = measure_recipe(
FakeDriver(texts={prompt.text: "different quantized answer" for prompt in PROMPTS},
rss_bytes=1_000_000, decode_ms_per_token=20.0),
recipe("gguf-q4", Lane.PERFORMANCE_FIT), plan()
)
report = build_report(plan(), [reference, quality, q4], host={}, evidence_class="synthetic")
reference_entry = next(
entry for entry in report["recipes"] if entry["recipe"]["id"] == "safetensors"
)
q4_entry = next(
entry for entry in report["recipes"] if entry["recipe"]["id"] == "gguf-q4"
)
for level, q4_cell in q4_entry["concurrency"].items():
reference_cell = reference_entry["concurrency"][level]
q4_cell["decode_tokens_per_sec"] = reference_cell["decode_tokens_per_sec"] / 2
q4_cell["aggregate_decode_tokens_per_sec"] = (
reference_cell["aggregate_decode_tokens_per_sec"] / 2
)
q4_cell["ttft_p50_ms"] = reference_cell["ttft_p50_ms"] * 2
contract = PerformanceContract(
contract_version=1, locked_at="2026-07-13T00:00:00Z", locked_by="test",
plan_id="test-plan", thresholds=ContractThresholds(), baseline={}, stop_condition="test",
)
evaluation = evaluate_contract(contract, report)
assert evaluation.quality_lane_pass is True
assert evaluation.fit_benefit is True
assert evaluation.verdict == "optimize"
def test_peak_memory_is_sampled_while_requests_are_in_flight():
class TransientMemoryDriver(FakeDriver):
def memory_probe(self) -> tuple[int, int]:
return (99_000_000 if self.in_flight else 1_000_000), 0
measurement = measure_recipe(
TransientMemoryDriver(generation_delay_s=0.05),
recipe("transient", Lane.QUALITY, reference=True),
plan(warmup_requests=0),
)
assert measurement.metrics[1].peak_rss_bytes == 99_000_000
assert measurement.metrics[4].peak_rss_bytes == 99_000_000
def test_real_inference_requires_explicit_opt_in(monkeypatch):
monkeypatch.delenv("MESHNET_ENABLE_REAL_INFERENCE_TESTS", raising=False)
with pytest.raises(BenchmarkError, match="opt-in"):
require_real_inference()
def test_config_rejects_an_artifact_digest_mismatch(tmp_path: Path):
artifact = tmp_path / "model.gguf"
artifact.write_bytes(b"real model bytes")
config = {
"artifact_storage_root": str(tmp_path),
"plan": {
"model_id": "test/model",
"model_revision": "revision-1",
"prompts": [
{"id": "p1", "text": "one"},
{"id": "p2", "text": "two"},
{"id": "p3", "text": "three"},
],
"concurrency_levels": [1, 4],
"repeats": 3,
"warmup_requests": 1,
"sampling": {
"temperature": 0.0,
"top_k": 1,
"top_p": 1.0,
"max_output_tokens": 32,
},
},
"recipes": [{
"id": "recipe",
"source_model_id": "test/model",
"source_model_revision": "revision-1",
"artifact_path": str(artifact),
"artifact_sha256": _artifact_sha256(artifact),
"device": "cpu",
"driver": {
"type": "llama-cpp-server",
"binary": str(artifact),
"binary_sha256": _artifact_sha256(artifact),
"gguf_path": str(artifact),
"device": "cpu",
"threads": 8,
"n_parallel": 4,
},
}],
}
_validate_config(config)
nested_digest = copy.deepcopy(config)
nested_digest["recipes"][0]["driver"]["artifact_sha256"] = "f" * 64
with pytest.raises(BenchmarkError, match="artifact_sha256 is forbidden"):
_validate_config(nested_digest)
driver = build_driver(nested_digest["recipes"][0], plan())
assert driver.artifact_sha256 == config["recipes"][0]["artifact_sha256"]
cuda_transformers = copy.deepcopy(config)
cuda_transformers["recipes"][0]["device"] = "cuda"
cuda_transformers["recipes"][0]["driver"] = {
"type": "transformers",
"model_path": str(artifact),
"device": "cuda",
"threads": 8,
}
with pytest.raises(BenchmarkError, match="every recipe to run on CPU"):
_validate_config(cuda_transformers)
config["recipes"][0]["artifact_sha256"] = "0" * 64
with pytest.raises(BenchmarkError, match="digest mismatch"):
_validate_config(config)
config["recipes"][0]["artifact_sha256"] = _artifact_sha256(artifact)
config["recipes"][0]["driver"]["binary_sha256"] = "0" * 64
with pytest.raises(BenchmarkError, match="binary SHA-256 mismatch"):
_validate_config(config)
config["recipes"][0]["driver"]["binary_sha256"] = _artifact_sha256(artifact)
config["recipes"][0]["driver"]["n_gpu_layers"] = 1
with pytest.raises(BenchmarkError, match="CPU-only"):
_validate_config(config)
config["recipes"][0]["device"] = "cuda"
config["recipes"][0]["driver"]["device"] = "cuda"
with pytest.raises(BenchmarkError, match="host marker"):
_validate_config(config, profile=GPU_DIAGNOSTIC_PROFILE)
config["host"] = []
with pytest.raises(BenchmarkError, match="host metadata must be an object"):
_validate_config(config, profile=GPU_DIAGNOSTIC_PROFILE)
config["host"] = {"benchmark_lane": "rocm-gpu-diagnostic"}
_validate_config(config, profile=GPU_DIAGNOSTIC_PROFILE)
config["recipes"][0]["driver"]["n_gpu_layers"] = 0
with pytest.raises(BenchmarkError, match="positive n_gpu_layers"):
_validate_config(config, profile=GPU_DIAGNOSTIC_PROFILE)
def test_gpu_offload_evidence_requires_measured_rocm_placement():
assert _gpu_layer_config_detail("cpu", 0) == "gpu layers 0"
assert _gpu_layer_config_detail("cuda", 99) == "requested gpu layers 99"
measured = """
common_param: - ROCm0 : Radeon 8060S Graphics (63963 MiB, 33058 MiB free)
load_tensors: offloaded 25/25 layers to GPU
"""
assert _gpu_offload_evidence(measured, 99) == (
"measured accelerator ROCm0: Radeon 8060S Graphics; "
"measured offload 25/25 layers"
)
with pytest.raises(BenchmarkError, match="no measured ROCm device"):
_gpu_offload_evidence("offloaded 25/25 layers to GPU", 99)
with pytest.raises(BenchmarkError, match="no measured layer offload"):
_gpu_offload_evidence(
"- ROCm0 : Radeon 8060S Graphics (63963 MiB, 33058 MiB free)", 99
)
with pytest.raises(BenchmarkError, match="measured only 4/25"):
_gpu_offload_evidence(measured.replace("25/25", "4/25"), 99)
def test_profiles_derive_fixed_producers_and_cli_dispatch(monkeypatch, tmp_path: Path):
assert _producer_for_profile(CONTRACT_V1_PROFILE) == REAL_REPORT_PRODUCER
assert _producer_for_profile(GPU_DIAGNOSTIC_PROFILE) == GPU_DIAGNOSTIC_REPORT_PRODUCER
with pytest.raises(BenchmarkError, match="unknown benchmark validation profile"):
_producer_for_profile("caller-selected-producer")
profiled_calls = []
def profiled_runner(config, *, profile):
profiled_calls.append((config, profile))
return {"profile": profile}
monkeypatch.setattr(recipe_drivers_module, "_run_profiled_benchmark", profiled_runner)
assert recipe_drivers_module.run_configured_benchmark({"source": "v1"}) == {
"profile": CONTRACT_V1_PROFILE
}
assert recipe_drivers_module.run_configured_gpu_diagnostic({"source": "gpu"}) == {
"profile": GPU_DIAGNOSTIC_PROFILE
}
assert profiled_calls == [
({"source": "v1"}, CONTRACT_V1_PROFILE),
({"source": "gpu"}, GPU_DIAGNOSTIC_PROFILE),
]
config_path = tmp_path / "config.json"
config_path.write_text(json.dumps({"test": "config"}), encoding="utf-8")
calls = []
def contract_runner(config):
calls.append((CONTRACT_V1_PROFILE, config))
return {"profile": CONTRACT_V1_PROFILE}
def gpu_runner(config):
calls.append((GPU_DIAGNOSTIC_PROFILE, config))
return {"profile": GPU_DIAGNOSTIC_PROFILE}
monkeypatch.setattr(recipe_drivers_module, "run_configured_benchmark", contract_runner)
monkeypatch.setattr(recipe_drivers_module, "run_configured_gpu_diagnostic", gpu_runner)
monkeypatch.setattr(
recipe_benchmark_module, "format_summary", lambda report: report["profile"]
)
recipe_benchmark_module.main(["--config", str(config_path)])
recipe_benchmark_module.main(
["--profile", GPU_DIAGNOSTIC_PROFILE, "--config", str(config_path)]
)
assert calls == [
(CONTRACT_V1_PROFILE, {"test": "config"}),
(GPU_DIAGNOSTIC_PROFILE, {"test": "config"}),
]
def test_committed_config_digest_matches_immutable_contract():
evidence = (
Path(__file__).parents[1]
/ ".scratch"
/ "distributed-gguf-runtime"
/ "evidence"
/ "DGR-001"
)
contract = parse_contract(
json.loads((evidence / "performance-contract.json").read_text())
)
config = json.loads((evidence / "benchmark-config.json").read_text())
assert _canonical_sha256(config) == contract.baseline["required_config_sha256"]
trusted = json.loads(
(evidence.parents[1] / "trusted-evidence-signers.json").read_text()
)
public_key = base64.b64decode(contract.baseline["required_signer_public_key"])
fingerprint = hashlib.sha256(public_key).hexdigest()
assert any(
signer["algorithm"] == "ed25519"
and signer["fingerprint_sha256"] == fingerprint
and signer["status"] == "active"
for signer in trusted["signers"]
)
expected_cpu_gpu_detail = _gpu_layer_config_detail("cpu", 0)
llama_backends = [
detail
for recipe_id, detail in contract.baseline["required_backend_detail"].items()
if recipe_id.startswith("llama-cpp-")
]
assert llama_backends
assert all(expected_cpu_gpu_detail in detail for detail in llama_backends)
assert all("requested gpu layers" not in detail for detail in llama_backends)
_TEST_SIGNING_KEY = Ed25519PrivateKey.generate()
def _resign_test_report(report: dict) -> None:
report["provenance"].pop("signature", None)
report["provenance"]["signature"] = base64.b64encode(
_TEST_SIGNING_KEY.sign(report_signing_payload(report))
).decode("ascii")
def _lock_real_report(report: dict) -> PerformanceContract:
report["evidence_class"] = "local-real"
report["host"] = {
"hostname": "test-host",
"platform": "test-platform",
"python": "3.12",
"cpu_count": 8,
}
for entry in report["recipes"]:
entry["recipe"]["source_model_id"] = report["plan"]["model_id"]
entry["recipe"]["source_model_revision"] = report["plan"]["model_revision"]
entry["recipe"]["artifact_sha256"] = "a" * 64
entry["load"]["backend_detail"] = f"fake-backend-{entry['recipe']['id']}"
public_key = _TEST_SIGNING_KEY.public_key().public_bytes(
serialization.Encoding.Raw, serialization.PublicFormat.Raw
)
config_sha256 = "c" * 64
report["provenance"] = {
"schema_version": PROVENANCE_SCHEMA_VERSION,
"producer": REAL_REPORT_PRODUCER,
"run_id": "test-run-id",
"started_at": "2026-07-13T00:00:00Z",
"completed_at": "2026-07-13T00:01:00Z",
"config_sha256": config_sha256,
"signature_algorithm": "ed25519",
"signer_public_key_sha256": hashlib.sha256(public_key).hexdigest(),
}
contract = PerformanceContract(
contract_version=1,
locked_at="2026-07-13T00:00:00Z",
locked_by="test",
plan_id=report["plan"]["plan_id"],
thresholds=ContractThresholds(),
baseline={
"required_evidence_class": "local-real",
"required_recipes": [entry["recipe"]["id"] for entry in report["recipes"]],
"required_concurrency_levels": [1, 4],
"required_config_sha256": config_sha256,
"required_signer_public_key": base64.b64encode(public_key).decode("ascii"),
"required_artifact_sha256": {
entry["recipe"]["id"]: entry["recipe"]["artifact_sha256"]
for entry in report["recipes"]
},
"required_recipe_runtime": {
entry["recipe"]["id"]: {
field: entry["recipe"].get(field)
for field in ("runtime", "weight_format", "weight_quantization", "device")
}
for entry in report["recipes"]
},
"required_backend_detail": {
entry["recipe"]["id"]: entry["load"]["backend_detail"]
for entry in report["recipes"]
},
"required_host_identity": {"python": "3.12"},
},
stop_condition="test",
)
_resign_test_report(report)
return contract
def test_locked_contract_rejects_synthetic_evidence():
reference = measure_recipe(
FakeDriver(), recipe("ref", Lane.QUALITY, reference=True), plan()
)
quality = measure_recipe(FakeDriver(), recipe("quality", Lane.QUALITY), plan())
q4 = measure_recipe(FakeDriver(), recipe("q4", Lane.PERFORMANCE_FIT), plan())
report = build_report(
plan(), [reference, quality, q4], host={}, evidence_class="synthetic"
)
contract = _lock_real_report(report)
report["evidence_class"] = "synthetic"
with pytest.raises(PerformanceContractError, match="evidence class"):
evaluate_contract(contract, report)
def test_non_synthetic_report_requires_canonical_provenance():
reference = measure_recipe(
FakeDriver(), recipe("ref", Lane.QUALITY, reference=True), plan()
)
with pytest.raises(BenchmarkError, match="signed provenance"):
build_report(
plan(), [reference], host={}, evidence_class="local-real"
)
def test_signed_real_report_rejects_tampering_and_rebinding():
texts = {prompt.text: "same greedy answer" for prompt in PROMPTS}
measurements = [
measure_recipe(FakeDriver(texts=texts), recipe("ref", Lane.QUALITY, reference=True), plan()),
measure_recipe(FakeDriver(texts=texts), recipe("quality", Lane.QUALITY), plan()),
measure_recipe(FakeDriver(texts=texts), recipe("q4", Lane.PERFORMANCE_FIT), plan()),
]
report = build_report(plan(), measurements, host={}, evidence_class="synthetic")
contract = _lock_real_report(report)
assert evaluate_contract(contract, report).verdict in {"promote", "optimize", "stop"}
unsigned = copy.deepcopy(report)
unsigned["provenance"].pop("signature")
with pytest.raises(PerformanceContractError, match="signature"):
evaluate_contract(contract, unsigned)
gpu_diagnostic = copy.deepcopy(report)
gpu_diagnostic["provenance"]["producer"] = GPU_DIAGNOSTIC_REPORT_PRODUCER
_resign_test_report(gpu_diagnostic)
with pytest.raises(PerformanceContractError, match="canonical real runner"):
evaluate_contract(contract, gpu_diagnostic)
tampered = copy.deepcopy(report)
tampered["recipes"][0]["recipe"]["artifact_sha256"] = "b" * 64
with pytest.raises(PerformanceContractError, match="signature verification failed"):
evaluate_contract(contract, tampered)
rebound_artifact = copy.deepcopy(tampered)
_resign_test_report(rebound_artifact)
with pytest.raises(PerformanceContractError, match="artifact digest"):
evaluate_contract(contract, rebound_artifact)
rebound_config = copy.deepcopy(report)
rebound_config["provenance"]["config_sha256"] = "d" * 64
_resign_test_report(rebound_config)
with pytest.raises(PerformanceContractError, match="config digest"):
evaluate_contract(contract, rebound_config)
rebound_runtime = copy.deepcopy(report)
rebound_runtime["recipes"][1]["recipe"]["runtime"] = "other-runtime"
_resign_test_report(rebound_runtime)
with pytest.raises(PerformanceContractError, match="runtime identity"):
evaluate_contract(contract, rebound_runtime)
rebound_backend = copy.deepcopy(report)
rebound_backend["recipes"][1]["load"]["backend_detail"] = "other-backend"
_resign_test_report(rebound_backend)
with pytest.raises(PerformanceContractError, match="backend identity"):
evaluate_contract(contract, rebound_backend)
rebound_host = copy.deepcopy(report)
rebound_host["host"]["python"] = "9.9"
_resign_test_report(rebound_host)
with pytest.raises(PerformanceContractError, match="host/runtime field"):
evaluate_contract(contract, rebound_host)
def test_quality_lane_requires_every_prompt_to_be_compared():
texts = {prompt.text: "same greedy answer" for prompt in PROMPTS}
reference = measure_recipe(
FakeDriver(texts=texts), recipe("ref", Lane.QUALITY, reference=True), plan()
)
quality = measure_recipe(
FakeDriver(texts=texts), recipe("quality", Lane.QUALITY), plan()
)
q4 = measure_recipe(
FakeDriver(texts=texts, rss_bytes=1_000_000),
recipe("q4", Lane.PERFORMANCE_FIT),
plan(),
)
report = build_report(
plan(), [reference, quality, q4], host={}, evidence_class="synthetic"
)
contract = _lock_real_report(report)
incomplete = copy.deepcopy(report)
quality_drift = next(
drift for drift in incomplete["drift"] if drift["recipe_id"] == "quality"
)
quality_drift["compared_prompts"] = 1
quality_drift["per_prompt"] = quality_drift["per_prompt"][:1]
_resign_test_report(incomplete)
evaluation = evaluate_contract(contract, incomplete)
assert evaluation.quality_lane_pass is False
assert evaluation.verdict == "stop"
def test_failed_request_reaches_zero_tolerance_gate_instead_of_validation_error():
texts = {prompt.text: "same greedy answer" for prompt in PROMPTS}
measurements = [
measure_recipe(
FakeDriver(texts=texts), recipe("ref", Lane.QUALITY, reference=True), plan()
),
measure_recipe(FakeDriver(texts=texts), recipe("quality", Lane.QUALITY), plan()),
measure_recipe(FakeDriver(texts=texts), recipe("q4", Lane.PERFORMANCE_FIT), plan()),
]
report = build_report(plan(), measurements, host={}, evidence_class="synthetic")
contract = _lock_real_report(report)
failed = copy.deepcopy(report)
quality_entry = next(
entry for entry in failed["recipes"] if entry["recipe"]["id"] == "quality"
)
outcome = quality_entry["outcomes"][0]
outcome["ok"] = False
outcome["error"] = "simulated runtime failure"
cell_key = next(
key for key in quality_entry["concurrency"] if int(key) == outcome["concurrency"]
)
quality_entry["concurrency"][cell_key]["failures"] = 1
_resign_test_report(failed)
evaluation = evaluate_contract(contract, failed)
quality_evaluation = next(
item for item in evaluation.recipes if item.recipe_id == "quality"
)
assert quality_evaluation.failures == 1
assert quality_evaluation.quality_pass is False
assert evaluation.verdict == "stop"
permissive_contract = replace(
contract,
thresholds=replace(contract.thresholds, max_failure_rate=0.01),
)
with pytest.raises(PerformanceContractError, match="explicit failed-request token policy"):
evaluate_contract(permissive_contract, failed)
inconsistent = copy.deepcopy(failed)
quality_entry = next(
entry for entry in inconsistent["recipes"] if entry["recipe"]["id"] == "quality"
)
quality_entry["concurrency"][cell_key]["failures"] = 0
_resign_test_report(inconsistent)
with pytest.raises(PerformanceContractError, match="failures do not match raw outcomes"):
evaluate_contract(contract, inconsistent)
wrong_requests = copy.deepcopy(report)
quality_entry = next(
entry for entry in wrong_requests["recipes"] if entry["recipe"]["id"] == "quality"
)
quality_entry["concurrency"][cell_key]["requests"] += 1
_resign_test_report(wrong_requests)
with pytest.raises(PerformanceContractError, match="requests do not match raw outcomes"):
evaluate_contract(contract, wrong_requests)
misidentified = copy.deepcopy(report)
quality_entry = next(
entry for entry in misidentified["recipes"] if entry["recipe"]["id"] == "quality"
)
quality_entry["outcomes"][0]["recipe_id"] = "other-recipe"
_resign_test_report(misidentified)
with pytest.raises(PerformanceContractError, match="contains an outcome for"):
evaluate_contract(contract, misidentified)
missing = copy.deepcopy(failed)
quality_entry = next(
entry for entry in missing["recipes"] if entry["recipe"]["id"] == "quality"
)
quality_entry["outcomes"].pop()
_resign_test_report(missing)
with pytest.raises(PerformanceContractError, match="complete request coverage"):
evaluate_contract(contract, missing)
def test_locked_contract_rejects_changed_plan_and_token_counts():
texts = {prompt.text: "same greedy answer" for prompt in PROMPTS}
reference = measure_recipe(
FakeDriver(texts=texts), recipe("ref", Lane.QUALITY, reference=True), plan()
)
quality = measure_recipe(
FakeDriver(texts=texts), recipe("quality", Lane.QUALITY), plan()
)
q4 = measure_recipe(
FakeDriver(texts=texts), recipe("q4", Lane.PERFORMANCE_FIT), plan()
)
report = build_report(
plan(), [reference, quality, q4], host={}, evidence_class="synthetic"
)
contract = _lock_real_report(report)
contract.baseline["required_plan_sha256"] = _canonical_sha256(report["plan"])
changed_plan = copy.deepcopy(report)
changed_plan["plan"]["prompts"][0]["text"] = "changed after locking"
_resign_test_report(changed_plan)
with pytest.raises(PerformanceContractError, match="plan digest"):
evaluate_contract(contract, changed_plan)
changed_tokens = copy.deepcopy(report)
quality_entry = next(
entry for entry in changed_tokens["recipes"] if entry["recipe"]["id"] == "quality"
)
quality_entry["outcomes"][0]["decode_tokens"] += 1
_resign_test_report(changed_tokens)
with pytest.raises(PerformanceContractError, match="different prompt/decode token counts"):
evaluate_contract(contract, changed_tokens)
def test_v1_contract_thresholds_and_stop_condition_are_immutable():
raw = PerformanceContract(
contract_version=1,
locked_at="2026-07-13T00:00:00Z",
locked_by="test",
plan_id="test-plan",
thresholds=ContractThresholds(),
baseline={},
stop_condition=STOP_CONDITION,
).to_dict()
parse_contract(raw)
changed_threshold = copy.deepcopy(raw)
changed_threshold["thresholds"]["min_decode_speedup"] = 1.01
with pytest.raises(PerformanceContractError, match="immutable v1 thresholds"):
parse_contract(changed_threshold)
changed_stop = copy.deepcopy(raw)
changed_stop["stop_condition"] = "promote everything"
with pytest.raises(PerformanceContractError, match="stop condition differs"):
parse_contract(changed_stop)

270
tests/test_runtime_recipe_identity.py Normal file
View File

@@ -0,0 +1,270 @@
"""Deterministic DGR-003 identity and tracker-admission conformance tests."""
from __future__ import annotations
from dataclasses import replace
import json
from pathlib import Path
import time
import pytest
from meshnet_node.runtime_recipe import (
ArtifactIdentity,
CompatibilityFingerprint,
DerivativeBinding,
RecipeIdentityError,
RuntimeRecipe,
ShardIdentity,
check_handshake,
check_route,
handshake_error,
)
from meshnet_tracker.capability import (
STATE_ADMITTED,
STATE_FINGERPRINT_MISMATCH,
STATE_MODEL_MISMATCH,
STATE_RECIPE_MISMATCH,
STATE_UNCERTIFIED,
evaluate_report,
)
from meshnet_tracker.server import TrackerServer, _capability_from_registration
from meshnet_tracker.recipe import (
CertificationLedger,
DistributedForwardEvidence,
parse_identity,
)
def _digest(char: str) -> str:
return char * 64
def _identity(start: int = 0, end: int = 4, **recipe_changes: object) -> ShardIdentity:
recipe_fields: dict[str, object] = {
"weight_quantization": "Q4_K_M",
"activation_dtype": "bfloat16",
"compute_dtype": "float32",
"kv_dtype": "q8_0",
"kv_layout": "paged-v1",
"tokenizer_revision": "0123456789abcdef",
"architecture_adapter": "llama/range-v1",
"backend_id": "llama.cpp",
"runtime_version": "llama.cpp@deadbeef+meshnet.1",
"recipe_id": "example-gguf",
"recipe_version": "1",
"catalogue_version": "2026.07.1",
}
recipe_fields.update(recipe_changes)
recipe = RuntimeRecipe(**recipe_fields) # type: ignore[arg-type]
return ShardIdentity(
ArtifactIdentity(
"example/model",
"0123456789abcdef",
_digest("a"),
"dense-llama",
_digest("b"),
8,
),
recipe,
start,
end,
)
def _report(identity: ShardIdentity) -> dict:
return {
"schema_version": 1,
"model": {"model_id": "example/model"},
"shard": {"start": identity.shard_start, "end": identity.shard_end - 1},
"recipe": identity.recipe.to_dict() | {
"recipe_id": identity.recipe.recipe_id,
"recipe_version": identity.recipe.recipe_version,
"catalogue_version": identity.recipe.catalogue_version,
},
"backend": {"backend_id": "llama.cpp", "device": "test"},
"status": "passed",
"validated_at": 100.0,
"duration_ms": 1,
"diagnostics": [],
"identity": identity.to_dict(),
}
def _evaluate(report: dict, **kwargs: object):
return evaluate_report(
report,
model_matches=lambda model: model == "example/model",
advertised_model="example/model",
shard_start=0,
shard_end=3,
now=100.0,
**kwargs,
)
def test_node_and_tracker_share_the_committed_canonical_fingerprint_vector():
vector_path = Path(__file__).parent / "data" / "recipe_fingerprint_vectors.json"
vector = json.loads(vector_path.read_text(encoding="utf-8"))["vectors"][0]
expected = vector["fingerprint"]
identity = ShardIdentity.from_dict(vector["identity"])
assert identity.fingerprint.to_dict() == expected
assert parse_identity(vector["identity"]).fingerprint_dict() == expected
assert (
CompatibilityFingerprint.from_proto(identity.fingerprint.to_proto()).to_dict()
== expected
)
@pytest.mark.parametrize(
"axis,value",
[
("weight_quantization", "Q5_K_M"),
("activation_dtype", "float16"),
("compute_dtype", "float16"),
("kv_dtype", "float16"),
("kv_layout", "contiguous-v2"),
("tokenizer_revision", "fedcba9876543210"),
("architecture_adapter", "llama/range-v2"),
("backend_id", "other-backend"),
("runtime_version", "llama.cpp@other+meshnet.1"),
("boundary_schema_version", 2),
("protocol_schema_version", 2),
],
)
def test_every_recipe_axis_changes_the_fingerprint_and_blocks_route(axis, value):
left = _identity()
right = _identity(4, 8, **{axis: value})
assert left.fingerprint.key != right.fingerprint.key
assert check_route([left, right])
def test_split_artifact_is_bound_to_exact_source_and_owned_range():
source = _identity()
split = ShardIdentity(
ArtifactIdentity(
"example/model",
"0123456789abcdef",
_digest("c"),
"dense-llama",
_digest("b"),
8,
DerivativeBinding(_digest("a"), 4, 8),
),
source.recipe,
4,
8,
)
assert source.fingerprint.matches(split.fingerprint)
with pytest.raises(RecipeIdentityError):
ShardIdentity(split.artifact, split.recipe, 3, 8)
def test_declared_digest_mismatch_is_recomputed_and_rejected_not_authenticated():
report = _report(_identity())
report["identity"]["fingerprint"]["runtime_recipe_digest"] = _digest("f")
assert _evaluate(report).state == STATE_FINGERPRINT_MISMATCH
def test_exact_recipe_registers_dark_until_tracker_certification():
identity = _identity()
report = _report(identity)
ledger = CertificationLedger()
dark = _evaluate(report, ledger=ledger)
assert dark.state == STATE_UNCERTIFIED
# Unit fixtures cannot promote a recipe: this is the explicit trust boundary.
with pytest.raises(ValueError, match="synthetic"):
ledger.certify(
parse_identity(identity.to_dict()),
DistributedForwardEvidence(
"session",
1,
("a", "b"),
((0, 4), (4, 8)),
1,
8,
identity.fingerprint.key,
synthetic=True,
),
)
with pytest.raises(ValueError, match="fingerprint does not match"):
ledger.certify(
parse_identity(identity.to_dict()),
DistributedForwardEvidence(
"session",
1,
("a", "b"),
((0, 4), (4, 8)),
1,
8,
(_digest("e"), _digest("f")),
),
)
def test_capability_identity_must_match_the_proof_labels():
identity = _identity()
wrong_model = _report(identity)
wrong_model["identity"]["artifact"]["artifact_id"] = "other/model"
wrong_model["identity"]["fingerprint"] = None
assert _evaluate(wrong_model).state == STATE_MODEL_MISMATCH
wrong_recipe = _report(identity)
wrong_recipe["recipe"]["recipe_id"] = "other-recipe"
assert _evaluate(wrong_recipe).state == STATE_RECIPE_MISMATCH
wrong_backend = _report(identity)
wrong_backend["backend"]["backend_id"] = "other-backend"
assert _evaluate(wrong_backend).state == STATE_RECIPE_MISMATCH
wrong_quantization = _report(identity)
wrong_quantization["backend"]["quantization"] = "Q5_K_M"
assert _evaluate(wrong_quantization).state == STATE_RECIPE_MISMATCH
def test_tracker_server_owns_the_ledger_used_by_registration():
identity = _identity()
report = _report(identity)
report["validated_at"] = time.time()
payload = {"capability_report": report}
tracker = TrackerServer()
def evaluate():
return _capability_from_registration(
payload,
model="example/model",
hf_repo=None,
shard_start=0,
shard_end=3,
recipe_certifications=tracker._recipe_certifications,
)
assert evaluate().state == STATE_UNCERTIFIED
tracker._recipe_certifications.certify(
parse_identity(identity.to_dict()),
DistributedForwardEvidence(
"session",
1,
("physical-a", "physical-b"),
((0, 4), (4, 8)),
1,
8,
identity.fingerprint.key,
),
)
assert evaluate().state == STATE_ADMITTED
def test_grpc_handshake_uses_the_dgr_002_fingerprint_and_fails_closed():
local = _identity()
remote = replace(local.fingerprint, runtime_recipe_digest=_digest("f")).to_proto()
mismatches = check_handshake(local, remote)
assert mismatches
assert handshake_error(mismatches).code != 0