Compare commits
21 Commits
9e67b829e3
...
ralph/dgr-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cae7c2b171 | ||
|
|
64f83d4392 | ||
|
|
454a681a50 | ||
|
|
a0f28b5631 | ||
|
|
7925e5253d | ||
|
|
91c450840d | ||
|
|
d6b808dcf9 | ||
|
|
31065c0e12 | ||
|
|
ec36290863 | ||
|
|
f844ae6567 | ||
|
|
252d131e7d | ||
|
|
3d8f93f4aa | ||
|
|
f9722e7b57 | ||
|
|
7b8e467c6b | ||
|
|
7364ed6731 | ||
|
|
ad2d17541c | ||
|
|
e7c780a623 | ||
|
|
9580ed643e | ||
|
|
5ebce15d7a | ||
|
|
ef2a9e67e8 | ||
|
|
b1c9deeb01 |
@@ -8,3 +8,5 @@
|
||||
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
|
||||
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
||||
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
||||
- [DGR ROCm setup](dgr-rocm-setup.md) — version-matched TheRock SDK layout, relocated devel payload, verified `gfx1151` HIP llama.cpp build, and GPU-diagnostic boundary.
|
||||
- **DGR-004 llama.cpp boundary** — `packages/node/native/llama/` locks `e920c523e3b8a0163fe498af5bf90df35ff51d25`, with a one-patch CMake marker and fail-closed clean materialize/apply/build/smoke harness. This is infrastructure only; stock GLM dense fallback remains uncertified.
|
||||
|
||||
29
.claude/memory/dgr-rocm-setup.md
Normal file
29
.claude/memory/dgr-rocm-setup.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# DGR ROCm and llama.cpp setup
|
||||
|
||||
As of 2026-07-13:
|
||||
|
||||
- Project ROCm runtime: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm`
|
||||
- ROCm/TheRock build: `7.13.0a20260513`, target `gfx1151`
|
||||
- `rocm-sdk-devel` is installed. Its expanded SDK lives under the venv at
|
||||
`site-packages/_rocm_sdk_devel`.
|
||||
- The wheel's redundant packaged payload was relocated to
|
||||
`/home/popov/.local/share/rocm-sdk/7.13.0a20260513/rocm_sdk_devel` and symlinked
|
||||
back into the venv because installing both packaged and expanded forms filled
|
||||
the mounted drive. Do not reinstall it blindly; the wheel expands beyond
|
||||
20 GB.
|
||||
- HIP llama.cpp source: `/run/media/popov/d/DEV/llamacpp/llama.cpp`, commit
|
||||
`e920c523e3b8a0163fe498af5bf90df35ff51d25` (version 9991).
|
||||
- HIP build: `/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip`
|
||||
- HIP `llama-server` SHA-256:
|
||||
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`
|
||||
- Verified device: `ROCm0: Radeon 8060S Graphics`, `gfx1151`.
|
||||
- Model artifacts remain under `/run/media/popov/DATA/llm`; none were put under
|
||||
`/home`.
|
||||
|
||||
DGR-001's immutable contract remains CPU-only. GPU evidence uses the distinct
|
||||
signed `gpu-diagnostic` profile because llama-server process VRAM is not yet
|
||||
measurable by the benchmark driver. The profile must capture measured
|
||||
llama-server startup evidence for `ROCm0` and the actual offloaded/total layer
|
||||
count; configured `device` and `n_gpu_layers` values alone are not evidence.
|
||||
The accepted signer fingerprint is anchored in
|
||||
`.scratch/distributed-gguf-runtime/trusted-evidence-signers.json`.
|
||||
384
.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md
Normal file
384
.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md
Normal file
@@ -0,0 +1,384 @@
|
||||
# GLM-5.2 Max distributed alpha roadmap
|
||||
|
||||
Status: proposed executable epic target
|
||||
Last updated: 2026-07-13
|
||||
|
||||
## Executive decision
|
||||
|
||||
The alpha-release target is the exact open-weight model `zai-org/GLM-5.2` served with `reasoning_effort=max` from the smallest published Unsloth GGUF recipe, `UD-IQ1_S`, across multiple consumer machines.
|
||||
|
||||
“Max” is a reasoning mode selected by the chat template/API request. It is not a separate model checkpoint.
|
||||
|
||||
Alpha is earned only when the real target model:
|
||||
|
||||
1. cannot fit within any one participating node's admitted memory;
|
||||
2. loads as contiguous layer Shards on at least two physical consumer machines;
|
||||
3. performs real GLM-5.2 MoE + DSA + IndexShare computation on every selected node;
|
||||
4. responds through the existing OpenAI-compatible Meshnet API with `reasoning_effort=max`;
|
||||
5. passes locked parity, usefulness, performance, telemetry, cancellation, and cleanup checks; and
|
||||
6. stores all model artifacts on configured mounted-drive storage, never under `/home`.
|
||||
|
||||
The shortest safe path is not “support every GGUF architecture.” Dense Llama remains a small structural fixture. GLM-5.2 moves onto the critical path immediately after exact recipe identity and the pinned llama.cpp boundary. Qwen expansion, 1M context certification, MTP/speculative decoding, broad concurrency, and automatic route repair are post-alpha.
|
||||
|
||||
## 1. Exact target contract
|
||||
|
||||
### 1.1 Source model
|
||||
|
||||
| Field | Locked target |
|
||||
|---|---|
|
||||
| Official repository | `zai-org/GLM-5.2` |
|
||||
| Official revision observed 2026-07-13 | `b4734de4facf877f85769a911abafc5283eab3d9` |
|
||||
| Model-weight license | MIT |
|
||||
| Official code/documentation license | Apache-2.0 |
|
||||
| Architecture | `glm_moe_dsa` / `GlmMoeDsaForCausalLM` |
|
||||
| Official architecture label | 744B total / approximately 40B active per token |
|
||||
| Exact stored checkpoint tensors | 753,329,940,480 parameters |
|
||||
| Transformer layers | 78 backbone layers plus one shared NextN/MTP layer in the artifact |
|
||||
| Layer types | first 3 dense; remaining 75 sparse MoE |
|
||||
| Routed experts | 256 |
|
||||
| Experts selected | 8 routed experts plus shared expert path |
|
||||
| Hidden width | 6,144 |
|
||||
| Attention | MLA under DSA, lightning indexer top-k 2,048 |
|
||||
| IndexShare | indexer roles are encoded by `indexer_types`; consumers reuse prior Full-layer indices |
|
||||
| Architectural maximum context | 1,048,576 tokens |
|
||||
| Alpha reasoning mode | `reasoning_effort=max` |
|
||||
|
||||
The runtime must derive these values from the pinned artifact and fail closed on contradictory metadata. Marketing names are not compatibility identity.
|
||||
|
||||
### 1.2 Alpha GGUF artifact
|
||||
|
||||
| Field | Locked target |
|
||||
|---|---|
|
||||
| GGUF repository | `unsloth/GLM-5.2-GGUF` |
|
||||
| GGUF revision observed 2026-07-13 | `abc55e72527792c6e77069c99b4cb7de16fa9f23` |
|
||||
| Quantization | `UD-IQ1_S` |
|
||||
| Files | six GGUF shards |
|
||||
| Exact published bytes | 216,715,360,960 bytes |
|
||||
| Binary GiB | 201.832 GiB |
|
||||
| Published quality indication | about 76.2% top-1 agreement with the high-precision reference on Unsloth's quantization analysis |
|
||||
| Mounted storage rule | configured mounted drive only; never `/home` |
|
||||
|
||||
Before downloading 216.7 GB, DGR-017 must generate a checked-in target manifest containing repository revisions, expected filenames, byte sizes, and resolved LFS SHA-256 values. Download is resumable and verified before route admission.
|
||||
|
||||
`UD-IQ1_M` (228,492,966,624 bytes / 212.801 GiB) is the first diagnostic fallback if `UD-IQ1_S` exposes a runtime or quality defect. It does not satisfy the explicit “lowest quantization” alpha target unless the target contract is changed by human review.
|
||||
|
||||
### 1.3 Runtime semantics required for alpha
|
||||
|
||||
Required:
|
||||
|
||||
- GGUF parsing and quantized kernels from one exact llama.cpp pin.
|
||||
- Correct GLM-5.2 MoE routing, selected experts, and shared expert.
|
||||
- Correct compressed MLA KV cache for locally owned layers.
|
||||
- Native DSA lightning indexer and sparse attention.
|
||||
- Correct IndexShare Full/Shared role execution from artifact metadata.
|
||||
- Range-owned contiguous transformer layers; each owned layer keeps all of its experts local.
|
||||
- Head-only embeddings and tail-only final norm/output head/sampling.
|
||||
- Architecture-defined activation boundary and optional DSA index sideband.
|
||||
- `reasoning_effort=max` chat-template behavior through the public API.
|
||||
- F32 seam correctness lane and a separately certified production activation dtype.
|
||||
|
||||
Not required for alpha:
|
||||
|
||||
- MTP/speculative decoding. The trailing NextN tensors may be loaded or explicitly excluded according to a certified recipe, but cannot be silently misinterpreted.
|
||||
- Full 1,048,576-token context.
|
||||
- Continuous batching beyond one target session.
|
||||
- Public-WAN tensor or expert parallel collectives.
|
||||
- Automatic mid-generation repair or KV migration.
|
||||
- Every CPU/GPU backend combination.
|
||||
|
||||
## 2. Minimum resource envelope
|
||||
|
||||
### 2.1 Weight and runtime memory
|
||||
|
||||
The smallest artifact occupies 201.832 GiB before KV, DSA indexer state, scratch buffers, backend workspaces, process memory, and the operating system. **224 GiB aggregate runtime-accessible memory is only the experimental hard-fit floor**, consistent with Unsloth's approximate 223 GB one-bit requirement. It is not a conservative operational envelope.
|
||||
|
||||
For admission, each node reserves:
|
||||
|
||||
```text
|
||||
max(20% of physically usable memory, 8 GiB)
|
||||
```
|
||||
|
||||
The remainder is the combined weight-plus-KV placement budget. Actual peak scratch is measured by backend/context and can force one extra node. Unified memory is counted once: integrated-GPU “VRAM” must not be added again to the same physical system RAM.
|
||||
|
||||
| Physical usable tier | Minimum reserve | Weight + KV placement budget | IQ1_S 16K arithmetic minimum | Operational position |
|
||||
|---:|---:|---:|---:|---|
|
||||
| 32 GiB | 8.0 GiB | 24.0 GiB | 9 nodes | use 10 if attempted; latency-heavy |
|
||||
| 48 GiB | 9.6 GiB | 38.4 GiB | 6 nodes | possible; latency-heavy |
|
||||
| 64 GiB | 12.8 GiB | 51.2 GiB | 4 nodes | hard minimum; **5 recommended** |
|
||||
| 96 GiB | 19.2 GiB | 76.8 GiB | 3 nodes | recommended |
|
||||
| 128 GiB unified/system | 25.6 GiB | 102.4 GiB | 2 nodes | arithmetic hard minimum; **3 recommended** |
|
||||
|
||||
The planner must use exact tensor byte ownership, not equal percentages. Embeddings, final head, dense versus MoE layers, shared experts, indexer tensors, quant block alignment, KV distribution, and backend workspace make equal layer counts unequal in memory.
|
||||
|
||||
Recommended first target route: **three 96/128-GiB-class physical machines** or **five 64-GiB-class machines**, on the same wired switch with mounted model storage. Four 64-GiB or two 128-GiB machines are fit probes only and qualify solely if exact placement and measured peak-memory evidence retain the required reserve with no swap/overcommit.
|
||||
|
||||
### 2.2 KV cache
|
||||
|
||||
GLM-5.2 MLA caches 576 latent/rope values per token per backbone layer. Correct DSA also caches 128-dimensional indexer keys: ideally only for the 21 Full indexer layers, while the current experimental implementation may allocate them across all 78 layers. Alpha locks **Q8_0 KV** for quality and budgets the conservative current-implementation layout.
|
||||
|
||||
| Context × concurrency | MLA-only Q8 | Optimized DSA Q8 | Conservative current-DSA Q8 | Conservative current-DSA F16 |
|
||||
|---:|---:|---:|---:|---:|
|
||||
| 16,384 × 1 | 0.73 GiB | 0.77 GiB | **0.89 GiB** | 1.68 GiB |
|
||||
| 131,072 × 1 | 5.83 GiB | 6.18 GiB | **7.12 GiB** | 13.41 GiB |
|
||||
| 1,048,576 × 1 | 46.62 GiB | 49.41 GiB | **56.98 GiB** | 107.25 GiB |
|
||||
|
||||
These are planning estimates, not admission truth. The runtime must report measured allocated/resident MLA and indexer cache by Shard. Alpha configures a 16,384-token window, Q8_0 KV, and one session. Longer contexts and lower-bit KV are separate quality/resource certification gates.
|
||||
|
||||
### 2.3 Activation seams and network
|
||||
|
||||
A BF16 hidden-state boundary is 6,144 elements = 12,288 bytes/token before framing.
|
||||
|
||||
- A 16,384-token prefill sends about 192 MiB per seam.
|
||||
- One decode token sends about 12 KiB per seam.
|
||||
- A 512-token decode sends about 6 MiB per seam.
|
||||
- Four nodes imply three serial seams.
|
||||
|
||||
If a Shard boundary splits an IndexShare producer/consumer group, a 2,048-entry int32 top-k sideband can add up to 8 KiB/query before framing. The route planner should prefer boundaries that preserve complete IndexShare ownership groups. The protocol must still support and validate the named sideband because memory fit may force an internal group split.
|
||||
|
||||
Decode bandwidth is small, but every generated token crosses all seams serially, so node count and per-hop latency dominate. Alpha requires a same-switch wired route: **2.5 GbE minimum and 10 GbE recommended**, with measured one-way/RTT, serialization, and queue latency. A 1 GbE route may be retained as fit-only evidence but is not the recommended alpha topology. Alpha records per-seam bytes, p50/p95 transfer latency, retries, and checksum failures; no speed claim is inferred from link rate alone.
|
||||
|
||||
### 2.4 Storage
|
||||
|
||||
The shortest alpha path allows every node to hold the complete six-file source GGUF while mapping/allocating only owned tensors. This minimizes artifact-transformation risk but costs 216.7 GB disk per node.
|
||||
|
||||
Deterministic source-bound layer packages are a follow-up optimization. If needed before target fit, every package must retain:
|
||||
|
||||
- source repository/revision and source file hashes;
|
||||
- exact owned tensor names, layer range, and endpoint role;
|
||||
- tokenizer/config identity;
|
||||
- deterministic package hash; and
|
||||
- proof that composing all packages matches the source tensor inventory.
|
||||
|
||||
## 3. Current state and critical gaps
|
||||
|
||||
### Completed foundation
|
||||
|
||||
- DGR-001: immutable CPU contract plus separate signed ROCm diagnostic. CPU v1 remains `stop`; the GPU diagnostic establishes a viable fit/performance investigation lane but does not rewrite CPU evidence.
|
||||
- DGR-002: versioned backend-neutral gRPC/Protobuf Shard protocol with bounded fragments and compatibility checks.
|
||||
- Existing Meshnet: Tracker, contiguous Shards, Route Sessions/epochs, relay/direct transport, local Hot KV semantics in the reference backend, cancellation, telemetry, billing, and model-agnostic admission.
|
||||
|
||||
### Missing before target alpha
|
||||
|
||||
1. Exact GLM target/artifact manifest and memory-fit planner.
|
||||
2. A current llama.cpp pin proven to load and generate with the exact `UD-IQ1_S` artifact.
|
||||
3. A narrow decision on native GLM-5.2 DSA/IndexShare support. As observed 2026-07-13, merged llama.cpp PR #24770 loads GLM-5.2 through a dense-MLA compatibility path, while full IndexShare/DSA PR #25407 remains open and its generic sparse path can be slower than dense fallback. Generic CPU lightning-indexer support is merged; backend coverage remains uneven.
|
||||
4. A decode protocol amendment. `ActivationChunk` carries `TensorBundle`, but the current `DecodeStep` fast path carries only one `NamedTensor`; it cannot transport a hidden state plus GLM top-k sideband. Tail token/logit and sampling behavior also needs an explicit typed result contract.
|
||||
5. Correct range-owned GGUF loading and memory proof.
|
||||
6. GLM-specific boundary/KV/IndexShare semantics.
|
||||
7. Standalone native worker and Meshnet integration.
|
||||
8. Real target hardware route with no node individually able to admit the whole model.
|
||||
9. Locked target parity, usefulness, speed, failure, and cleanup evidence.
|
||||
|
||||
### Donor policy
|
||||
|
||||
`Mesh-LLM/mesh-llm` is a high-value test and patch donor. Its live GLM branch was observed with 261 llama.cpp patches, 167 named for GLM/DSA/MTP-related work. That is evidence of the problem's depth, not an acceptable maintained fork boundary.
|
||||
|
||||
Audit and selectively reproduce the smallest independently understood pieces for:
|
||||
|
||||
- GLM DSA graph semantics;
|
||||
- lightning indexer and sparse-attention tests;
|
||||
- IndexShare metadata/Full/Shared validation;
|
||||
- top-k sideband shape and lifecycle;
|
||||
- stage-local KV filtering; and
|
||||
- target parity/performance fixtures.
|
||||
|
||||
Do not import Mesh-LLM routing, discovery, scheduler, public mesh, package manager, or full patch stack. Keep Meshnet as the sole control plane and collaborate narrowly upstream with llama.cpp/Mesh-LLM maintainers where practical.
|
||||
|
||||
## 4. Revised roadmap
|
||||
|
||||
## Phase 0 — lock the target before implementation
|
||||
|
||||
### DGR-017: lock GLM-5.2 Max target and alpha contract
|
||||
|
||||
Deliver:
|
||||
|
||||
- machine-readable target manifest for official and GGUF revisions;
|
||||
- exact `UD-IQ1_S` file/size/hash inventory;
|
||||
- architecture/config/chat-template snapshot;
|
||||
- memory/KV/network planner with unified-memory de-duplication;
|
||||
- immutable alpha acceptance thresholds from section 5; and
|
||||
- current upstream/donor status report.
|
||||
|
||||
Exit: target identity and alpha requirements are reviewable without downloading the model.
|
||||
|
||||
## Phase 1 — establish a correct whole-model oracle
|
||||
|
||||
### DGR-003: exact runtime recipe identity
|
||||
|
||||
Extend the existing generic identity with GLM fields: DSA/IndexShare metadata, adapter version, reasoning template revision, activation bundle schema, KV dtype/layout, llama.cpp pin/patch hash, and target artifact manifest hash.
|
||||
|
||||
### DGR-004: reproducible llama.cpp pin and narrow patch boundary
|
||||
|
||||
Select a current exact upstream commit only after testing its stock GLM behavior. Add clean fetch/apply/build checks. Record every donor patch and whether it is adopted, rewritten, rejected, or waiting upstream.
|
||||
|
||||
### DGR-018: certify whole-model GLM-5.2 runtime semantics
|
||||
|
||||
On a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation, or a measured equivalent:
|
||||
|
||||
1. verify all six `UD-IQ1_S` shards;
|
||||
2. load with a stock pinned runtime and capture tensor/metadata warnings;
|
||||
3. prove whether DSA, IndexShare, shared expert, and Max template are actually active;
|
||||
4. add the minimum correctness patches/tests required;
|
||||
5. run deterministic prefill/decode and fixed Max-mode sentinel prompts; and
|
||||
6. sign the oracle recipe, output, telemetry, and limitations.
|
||||
|
||||
Exit: one whole-model oracle exists for the same artifact/runtime semantics the distributed path will implement. “It emits text” is insufficient.
|
||||
|
||||
## Phase 2 — build the generic local seam using small fixtures
|
||||
|
||||
### DGR-005: range-owned GGUF tensors
|
||||
|
||||
Keep dense Llama as a cheap structural fixture. Implement authoritative owned-tensor registration/loading, head/tail ownership, and measured resident-memory scaling. Design tensor classification so GLM adds explicit rules rather than unchecked name substitution.
|
||||
|
||||
### DGR-006: architecture-defined boundary
|
||||
|
||||
Implement named boundary bundles, F32 correctness lane, bounded fragmentation, and optional sidebands. Amend the decode fast path so it carries a versioned `TensorBundle` rather than one `NamedTensor`, while preserving a small one-tensor encoding. Define an explicit typed tail result for logits/token output and bind sampling/chat-template parameters to the recipe/request. Regenerate Python/C++ schema code and compatibility goldens. Dense fixture parity proves the seam mechanism, not GLM certification.
|
||||
|
||||
Exit: two local processes can execute a small dense model with correct range ownership and boundary parity.
|
||||
|
||||
## Phase 3 — add GLM-5.2 as the product adapter
|
||||
|
||||
### DGR-019: implement and certify GLM-5.2 range/DSA/IndexShare semantics
|
||||
|
||||
Deliver explicit support for:
|
||||
|
||||
- 78 main layers and endpoint tensor ownership;
|
||||
- 256-expert MoE routing/top-8 and shared expert;
|
||||
- compressed MLA KV by owned layers;
|
||||
- DSA lightning indexer and sparse attention;
|
||||
- IndexShare metadata, Full producer, Shared consumer, and sideband behavior;
|
||||
- NextN/MTP tensor policy with MTP disabled or enabled explicitly;
|
||||
- shard-boundary planner aware of IndexShare ownership groups; and
|
||||
- whole-model versus two-stage parity against DGR-018.
|
||||
|
||||
Exit: a same-host two-stage target run matches the locked oracle tolerance with real GLM computation in both stages. If the full target cannot fit on one host for this check, use a layer-reduced GLM architecture fixture for graph parity and defer full-artifact output parity to DGR-020; label the distinction explicitly.
|
||||
|
||||
## Phase 4 — worker, KV, and Meshnet route
|
||||
|
||||
Execute existing stories with GLM requirements included:
|
||||
|
||||
1. DGR-007 — isolated local Hot KV keyed by `(Route Session, epoch)`, including DSA/IndexShare state.
|
||||
2. DGR-008 — standalone C++ gRPC worker.
|
||||
3. DGR-009 — Meshnet backend, capability, relay/direct, cancellation, and telemetry integration.
|
||||
4. DGR-010 — small-model local two-process acceptance.
|
||||
5. DGR-011 — real two-physical-machine route and heterogeneous fail-closed behavior.
|
||||
6. DGR-013 subset required by alpha — node loss, cancellation, stale epoch, restart, and memory/KV cleanup.
|
||||
|
||||
Continuous batching (DGR-012) is deliberately not an alpha dependency. The first target release supports one admitted GLM route session; concurrency follows after target correctness and fit.
|
||||
|
||||
## Phase 5 — target alpha gate
|
||||
|
||||
### DGR-020: pass real distributed GLM-5.2 Max alpha acceptance
|
||||
|
||||
Use at least two physical machines and enough aggregate usable memory to meet the locked target planner. No participating node may individually admit the complete target. All stages must report real compute and exact tensor ownership.
|
||||
|
||||
Run the complete acceptance matrix in section 5, preserve raw logs/metrics/output, sign the evidence, and publish an explicit `alpha` or `stop` verdict. Thresholds cannot be weakened after results are known.
|
||||
|
||||
## Phase 6 — post-alpha hardening
|
||||
|
||||
After DGR-020 passes:
|
||||
|
||||
1. DGR-012 — 1/2/4-session continuous batching and bounded admission.
|
||||
2. DGR-014 — final distributed GGUF versus reference-route performance decision.
|
||||
3. 32K, 128K, 200K, then 1M context certification with quantized KV.
|
||||
4. MTP/speculative decoding.
|
||||
5. Deterministic range packages to remove full-artifact replication.
|
||||
6. Additional backend compatibility classes and route topologies.
|
||||
7. DGR-016 — narrow upstream collaboration package, split by independently reviewable llama.cpp changes.
|
||||
8. DGR-015 — Qwen3/Qwen3-MoE only as later architecture expansion, not as the GLM alpha target.
|
||||
|
||||
## 5. Locked alpha acceptance matrix
|
||||
|
||||
These thresholds are set before target execution.
|
||||
|
||||
### 5.1 Identity and fit
|
||||
|
||||
- Exact official and GGUF repository revisions match the target manifest.
|
||||
- All six source GGUF sizes and LFS SHA-256 values verify.
|
||||
- Every route node reports owned tensor names/bytes, layer range, endpoint role, backend, KV recipe, and patch fingerprint.
|
||||
- Union of owned tensors equals the certified runtime-required tensor inventory; unintended overlap is zero.
|
||||
- No node's weight-plus-KV placement budget can hold the complete recipe.
|
||||
- Every node reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-KV placement; measured peak scratch must remain inside that reserve.
|
||||
- Aggregate peak RSS/VRAM stays within physical budgets with no swap, overcommit, mmap-only, or double-counted unified-memory success claim.
|
||||
- Arithmetic-minimum topologies require exact contiguous tensor placement evidence; recommended alpha topology is 5×64 GiB or 3×96/128 GiB.
|
||||
- Unified RAM/VRAM is not double-counted.
|
||||
|
||||
### 5.2 Semantic correctness
|
||||
|
||||
- Logs and graph tests prove GLM MoE/shared-expert, DSA lightning indexer, sparse attention, and IndexShare Full/Shared paths are active; dense-attention compatibility fallback cannot satisfy alpha.
|
||||
- `reasoning_effort=max` is observable in the rendered template/API recipe.
|
||||
- F32 same-backend seam fixture: 32 greedy decode tokens exactly match the whole-model oracle and activation tolerance is locked by DGR-006.
|
||||
- Production seam on the fixed prompt corpus: greedy token agreement is at least 0.90 and mean compared-state/logit cosine similarity is at least 0.999 versus DGR-018, with no malformed or non-finite tensors.
|
||||
- Incompatible artifact, tokenizer, adapter, DSA metadata, boundary, activation, KV, backend class, or runtime patch fingerprints fail closed.
|
||||
|
||||
### 5.3 End-to-end target run
|
||||
|
||||
- Context configured to 16,384 tokens with Q8_0 MLA/indexer KV.
|
||||
- Fixed 4,096-token prompt lane completes prefill.
|
||||
- Route uses a same-switch wired network; 2.5 GbE is the alpha minimum and 10 GbE is recommended.
|
||||
- One Max-mode request generates at least 512 output tokens or reaches a valid natural EOS after at least 128 tokens.
|
||||
- Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable, relevant outputs; raw prompts and outputs are retained for review.
|
||||
- OpenAI-compatible response includes stable model ID, finish reason, and token usage.
|
||||
|
||||
### 5.4 Minimum useful performance
|
||||
|
||||
On the declared minimum alpha topology after one warm-up:
|
||||
|
||||
- median decode throughput is at least 0.5 generated token/s for the fixed Max-mode lane;
|
||||
- 4,096-token-prompt TTFT is at most 10 minutes;
|
||||
- no unexplained stall exceeds 60 seconds without progress telemetry;
|
||||
- per-stage compute, queue, KV, seam bytes/latency, RSS/VRAM, and backend timing are present; and
|
||||
- results are labeled by hardware/topology and are not generalized to other consumer systems.
|
||||
|
||||
If output quality passes but the speed floor fails, verdict is `stop` for alpha and the evidence selects the next optimization target. It is not relabeled as success merely because the model loaded.
|
||||
|
||||
### 5.5 Reliability and security
|
||||
|
||||
- Two consecutive cold starts load, generate, release, and exit cleanly.
|
||||
- Cancellation during prefill and decode releases every stage's queued buffers and KV lease.
|
||||
- One worker loss aborts the route; alpha retries only from token zero on a new compatible route.
|
||||
- Stale epochs and duplicate step IDs are rejected.
|
||||
- Artifact paths stay outside `/home`; logs contain no secrets or unrestricted prompt payloads.
|
||||
- Synthetic workers and layer-reduced fixtures are labeled unit/integration coverage and cannot satisfy target alpha.
|
||||
|
||||
## 6. First execution order
|
||||
|
||||
The next unattended work should run in this order:
|
||||
|
||||
1. DGR-017 — target contract, manifest, planner, and upstream status.
|
||||
2. DGR-003 — exact recipe identity.
|
||||
3. DGR-004 — current llama.cpp pin and minimal patch harness.
|
||||
4. Run in parallel:
|
||||
- DGR-018 — whole-model oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory.
|
||||
- DGR-005 and DGR-006 — generic range/boundary seam on local small fixtures.
|
||||
5. DGR-019 — GLM semantics and parity after both parallel lanes pass.
|
||||
6. DGR-007 through DGR-011 — native worker and real transport route.
|
||||
7. Required DGR-013 failure subset.
|
||||
8. DGR-020 — real target alpha verdict.
|
||||
|
||||
The first external hardware blocker is DGR-018, but DGR-005/DGR-006 proceed locally while that host is sourced. Do not download the full model until DGR-017's exact manifest and storage preflight pass.
|
||||
|
||||
## 7. Sources checked on 2026-07-13
|
||||
|
||||
Authoritative or primary:
|
||||
|
||||
- Official model card and config: <https://huggingface.co/zai-org/GLM-5.2>
|
||||
- Official release/architecture blog: <https://z.ai/blog/glm-5.2>
|
||||
- Official code/documentation repository: <https://github.com/zai-org/GLM-5>
|
||||
- Official source revision API: <https://huggingface.co/api/models/zai-org/GLM-5.2>
|
||||
- Official GLM-5 technical report: <https://arxiv.org/abs/2602.15763>
|
||||
- Unsloth GGUF repository: <https://huggingface.co/unsloth/GLM-5.2-GGUF>
|
||||
- Unsloth local-run/quantization guide: <https://unsloth.ai/docs/models/glm-5.2>
|
||||
- llama.cpp GLM-5.2 support issue: <https://github.com/ggml-org/llama.cpp/issues/24730>
|
||||
- llama.cpp merged dense-MLA compatibility loader: <https://github.com/ggml-org/llama.cpp/pull/24770>
|
||||
- llama.cpp open GLM-5.2 DSA/IndexShare implementation: <https://github.com/ggml-org/llama.cpp/pull/25407>
|
||||
- llama.cpp merged generic CPU lightning indexer: <https://github.com/ggml-org/llama.cpp/pull/24231>
|
||||
- llama.cpp 1M-context discussion: <https://github.com/ggml-org/llama.cpp/discussions/24622>
|
||||
- IndexCache/IndexShare paper: <https://arxiv.org/abs/2603.12201>
|
||||
|
||||
Donor/current implementation evidence:
|
||||
|
||||
- Mesh-LLM repository: <https://github.com/Mesh-LLM/mesh-llm>
|
||||
- Mesh-LLM GLM branch noted by llama.cpp collaborator in issue #24730: `feat/jianyang-glm-52`
|
||||
|
||||
Web/repository observations are pinned by date and must be refreshed in DGR-017 before implementation because upstream support is moving quickly.
|
||||
@@ -6,6 +6,8 @@ Build one lean native GGUF execution path that lets an Inference Route combine c
|
||||
|
||||
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
|
||||
|
||||
The alpha target is now exact: `zai-org/GLM-5.2` at a pinned revision, the lowest published Unsloth `UD-IQ1_S` GGUF, and `reasoning_effort=max`, distributed across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a cheap structural fixture; Qwen is post-alpha expansion.
|
||||
|
||||
## Goals
|
||||
|
||||
- Execute one GGUF model across independently addressable contiguous Shards.
|
||||
@@ -15,6 +17,7 @@ The program is benchmark-gated. GGUF is not assumed faster merely because it is
|
||||
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
|
||||
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
|
||||
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
|
||||
- Pass an immutable GLM-5.2 Max `alpha`/`stop` gate with native MoE, DSA, IndexShare, parity, usefulness, speed, failure, and cleanup evidence.
|
||||
|
||||
## Quality Gates
|
||||
|
||||
@@ -41,6 +44,102 @@ Real-model/hardware stories must:
|
||||
|
||||
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
||||
|
||||
## Dependency Graph and Status
|
||||
|
||||
Status as of 2026-07-14 (MAINT-003). Authoritative per-story status is
|
||||
`passes` in [prd.json](prd.json); closed issues live in
|
||||
`docs/issues/distributed-gguf-runtime/`, open and blocked issues in
|
||||
[issues/](issues/).
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef done fill:#c8e6c9,stroke:#2e7d32;
|
||||
classDef blocked fill:#ffcdd2,stroke:#c62828;
|
||||
|
||||
DGR001[DGR-001 perf contract]:::done
|
||||
DGR002[DGR-002 gRPC Shard protocol]:::done
|
||||
DGR003[DGR-003 artifact/recipe identity]:::done
|
||||
DGR004[DGR-004 pinned llama.cpp patch stack]:::done
|
||||
DGR005[DGR-005 dense-Llama range ownership]:::done
|
||||
DGR006[DGR-006 boundary input/output]:::done
|
||||
DGR017[DGR-017 GLM-5.2 target/alpha contract]:::done
|
||||
DGR018[DGR-018 whole-model GLM oracle]:::blocked
|
||||
DGR019[DGR-019 GLM range/DSA/IndexShare]:::blocked
|
||||
DGR020[DGR-020 distributed GLM alpha]:::blocked
|
||||
DGR007[DGR-007 Hot KV State]
|
||||
DGR008[DGR-008 C++ gRPC worker]
|
||||
DGR009[DGR-009 Meshnet integration]
|
||||
DGR010[DGR-010 local two-process acceptance]
|
||||
DGR011[DGR-011 two-machine route]
|
||||
DGR012[DGR-012 continuous batching]
|
||||
DGR013[DGR-013 failure/cancel/restart]
|
||||
DGR014[DGR-014 release gate]
|
||||
DGR015[DGR-015 Qwen3 adapter]
|
||||
DGR016[DGR-016 upstream package]
|
||||
|
||||
DGR002 --> DGR003
|
||||
DGR017 --> DGR003
|
||||
DGR001 --> DGR004
|
||||
DGR017 --> DGR004
|
||||
DGR003 --> DGR005
|
||||
DGR004 --> DGR005
|
||||
DGR002 --> DGR006
|
||||
DGR005 --> DGR006
|
||||
DGR001 --> DGR017
|
||||
DGR002 --> DGR017
|
||||
DGR003 --> DGR018
|
||||
DGR004 --> DGR018
|
||||
DGR017 --> DGR018
|
||||
DGR005 --> DGR019
|
||||
DGR006 --> DGR019
|
||||
DGR018 --> DGR019
|
||||
DGR006 --> DGR007
|
||||
DGR019 --> DGR007
|
||||
DGR002 --> DGR008
|
||||
DGR003 --> DGR008
|
||||
DGR004 --> DGR008
|
||||
DGR006 --> DGR008
|
||||
DGR007 --> DGR008
|
||||
DGR003 --> DGR009
|
||||
DGR008 --> DGR009
|
||||
DGR009 --> DGR010
|
||||
DGR010 --> DGR011
|
||||
DGR007 --> DGR012
|
||||
DGR009 --> DGR012
|
||||
DGR010 --> DGR012
|
||||
DGR008 --> DGR013
|
||||
DGR009 --> DGR013
|
||||
DGR001 --> DGR014
|
||||
DGR011 --> DGR014
|
||||
DGR012 --> DGR014
|
||||
DGR013 --> DGR014
|
||||
DGR014 --> DGR015
|
||||
DGR010 --> DGR016
|
||||
DGR007 --> DGR020
|
||||
DGR008 --> DGR020
|
||||
DGR009 --> DGR020
|
||||
DGR011 --> DGR020
|
||||
DGR013 --> DGR020
|
||||
DGR017 --> DGR020
|
||||
DGR018 --> DGR020
|
||||
DGR019 --> DGR020
|
||||
```
|
||||
|
||||
- **Done (`passes: true`):** DGR-001, DGR-002, DGR-003, DGR-004, DGR-005,
|
||||
DGR-006, DGR-017.
|
||||
- **Blocked on hardware:** DGR-018 requires a 256-GiB-class host with at least
|
||||
224 GiB runtime-accessible memory and 250 GB free storage outside `/home`;
|
||||
no such host is currently available (development host: 124.9 GiB MemTotal).
|
||||
Exact preflight output: [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md).
|
||||
DGR-019 (needs the DGR-018 oracle) and DGR-020 (needs DGR-018/DGR-019 plus
|
||||
multiple physical consumer nodes) are blocked transitively.
|
||||
- **Consequence of the graph as written:** DGR-007 depends on DGR-019, so every
|
||||
remaining story (DGR-007 through DGR-016) is transitively blocked on the
|
||||
256-GiB host. Unblocking the generic dense pipeline without that host would
|
||||
require an explicit re-planning decision to relax the DGR-007 → DGR-019
|
||||
dependency; that decision is out of scope for maintenance and has not been
|
||||
made.
|
||||
|
||||
## User Stories
|
||||
|
||||
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
||||
@@ -101,6 +200,8 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Head accepts token IDs and owns token embedding.
|
||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||
- [ ] Amend the decode fast path from one `NamedTensor` to a versioned `TensorBundle`, preserving compact one-tensor compatibility and regenerating Python/C++ protocol goldens.
|
||||
- [ ] Define a typed tail logits/token result with sampling and chat-template/reasoning identity.
|
||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||
@@ -213,6 +314,35 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
||||
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
||||
|
||||
### DGR-017: Lock the GLM-5.2 Max target and alpha contract
|
||||
**Description:** Pin exact official/GGUF revisions, `UD-IQ1_S` files and hashes, Max-mode semantics, resource accounting, and immutable target thresholds before implementation results exist.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Produce machine-readable target, resource, upstream-status, and immutable acceptance contracts without downloading full weights.
|
||||
- [ ] Distinguish the 224-GiB hard-fit floor from the recommended 5×64 or 3×96/128 topology, using Q8 KV, 20%/8-GiB node reserves, and a wired 2.5-GbE minimum.
|
||||
- [ ] Count unified RAM/VRAM once and test revision/shard/byte/threshold mutation failures.
|
||||
|
||||
### DGR-018: Certify whole-model GLM-5.2 runtime semantics
|
||||
**Description:** Establish the exact IQ1_S oracle on a 256-GiB-class host with at least 224 GiB runtime-accessible memory; lock Q8_0 MLA/indexer KV and native target semantics before distributed parity work.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Verify the complete artifact and prove native MoE/shared expert, DSA, IndexShare, KV, NextN policy, and Max-template behavior.
|
||||
- [ ] Dense/replicated compatibility fallback cannot become the oracle merely because it emits text.
|
||||
|
||||
### DGR-019: Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
||||
**Description:** Add explicit target-model tensor, graph, boundary, sideband, and local-KV ownership after the generic dense seam.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Preserve MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared semantics across contiguous Shards.
|
||||
- [ ] Pass locked fixture/target parity and measured per-Shard memory ownership; never claim full-target parity from a reduced fixture.
|
||||
|
||||
### DGR-020: Pass real distributed GLM-5.2 Max alpha acceptance
|
||||
**Description:** Run the exact lowest-quant target through Meshnet on enough physical consumer nodes that no node can admit the whole recipe.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Pass the immutable identity, semantic, parity, Max-mode usefulness, 0.5 token/s, TTFT, reliability, and mounted-storage gates.
|
||||
- [ ] Preserve signed raw evidence and emit `alpha` only if every target criterion passes; otherwise emit `stop`.
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
|
||||
@@ -234,7 +364,7 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
- QUIC, WebRTC, or a custom socket protocol.
|
||||
- Automatic KV migration or mid-generation route repair in the first release.
|
||||
- Speculative decoding or disaggregated prefill before the core release gate.
|
||||
- Supporting every GGUF architecture before dense Llama and Qwen3-family certification.
|
||||
- Supporting every GGUF architecture before the exact GLM-5.2 target; Qwen3-family certification is post-alpha.
|
||||
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
|
||||
|
||||
## Success Metrics
|
||||
@@ -247,6 +377,6 @@ Before a story is marked complete, run the full deterministic `pytest -q` suite
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Exact benchmark model and quantization lanes are selected by DGR-001 from currently supported, legally redistributable artifacts.
|
||||
- DGR-001 remains immutable. DGR-017 locks the exact GLM-5.2/`UD-IQ1_S` target and alpha thresholds without rewriting DGR-001 evidence.
|
||||
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
|
||||
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.
|
||||
|
||||
@@ -9,7 +9,7 @@ Before changing code, every Ralph agent must:
|
||||
|
||||
1. Read this file completely.
|
||||
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
|
||||
3. Read `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
|
||||
3. Read `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`, and the relevant part of `architecture.md`.
|
||||
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
|
||||
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
|
||||
6. Read the evidence/handoff directories for every declared dependency.
|
||||
@@ -35,6 +35,8 @@ If interrupted after code changes, record every changed file, command result and
|
||||
|
||||
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
|
||||
|
||||
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact served with `reasoning_effort=max` across physical consumer machines. Dense Llama is a structural fixture. Synthetic workers, dense-attention compatibility fallback, a smaller model, or a single host cannot satisfy target alpha. The immutable target contract and resource envelope are in `GLM-5.2-MAX-ALPHA-ROADMAP.md`.
|
||||
|
||||
A distributed demo is not success. The product must provide:
|
||||
|
||||
- Useful measured prefill and decode speed.
|
||||
@@ -84,6 +86,10 @@ Do not introduce another scheduler/control plane from vLLM, Nakshatra, prima.cpp
|
||||
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
|
||||
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
|
||||
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
|
||||
16. Unified system RAM and integrated-GPU memory are one physical pool and must never be double-counted for admission.
|
||||
17. Alpha requires native GLM-5.2 MoE, DSA, and IndexShare semantics; MTP/speculative decoding and 1M-context certification are post-alpha.
|
||||
18. DGR-006 amends the decode fast path to carry a versioned `TensorBundle` and defines a typed tail logits/token result; the current single-`NamedTensor` fast path is insufficient for GLM sidebands.
|
||||
19. Alpha reserves at least `max(20% of physical usable memory, 8 GiB)` per node outside weight-plus-Q8-KV placement and uses a same-switch wired 2.5 GbE minimum route.
|
||||
|
||||
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
|
||||
|
||||
@@ -154,7 +160,7 @@ The project patch stack is limited to:
|
||||
|
||||
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
|
||||
|
||||
Dense Llama-family is first. Qwen3/Qwen3-MoE is a separate adapter after the dense release gate. Do not generalize through unchecked tensor-name substitutions.
|
||||
Dense Llama is first only as a cheap range/boundary fixture. GLM-5.2 is the explicit product adapter and alpha target immediately afterward. Qwen3/Qwen3-MoE is post-alpha. Do not generalize through unchecked tensor-name substitutions.
|
||||
|
||||
## Existing code seams to inspect first
|
||||
|
||||
@@ -211,7 +217,7 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
|
||||
| DGR-003 | exact runtime-recipe/fingerprint implementation and admission tests; `evidence/DGR-003/` |
|
||||
| DGR-004 | exact upstream pin, numbered patch series, reproducible fetch/apply/build smoke; `evidence/DGR-004/` |
|
||||
| DGR-005 | dense-Llama range ownership loader and memory evidence; `evidence/DGR-005/` |
|
||||
| DGR-006 | architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
|
||||
| DGR-006 | decode `TensorBundle` protocol amendment, typed tail-result contract, architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
|
||||
| DGR-007 | concurrent session/KV manager, isolation/cleanup tests; `evidence/DGR-007/` |
|
||||
| DGR-008 | standalone C++ gRPC worker, fake-model integration tests, lifecycle evidence; `evidence/DGR-008/` |
|
||||
| DGR-009 | Meshnet backend/registration/relay integration and tests; `evidence/DGR-009/` |
|
||||
@@ -222,6 +228,10 @@ Generated protobuf/C++ build outputs belong in build directories unless packagin
|
||||
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
|
||||
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
|
||||
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
|
||||
| DGR-017 | exact GLM-5.2/GGUF target manifest, resource planner, immutable alpha contract and upstream status; `evidence/DGR-017/` |
|
||||
| DGR-018 | verified whole-model `UD-IQ1_S` oracle with native GLM semantic evidence; `evidence/DGR-018/` |
|
||||
| DGR-019 | explicit range-owned GLM MoE/MLA/DSA/IndexShare adapter, fixtures and parity; `evidence/DGR-019/` |
|
||||
| DGR-020 | real multi-node GLM-5.2 Max target evidence and immutable `alpha`/`stop` verdict; `evidence/DGR-020/` |
|
||||
|
||||
## Dependency handoff rule
|
||||
|
||||
|
||||
@@ -4,7 +4,9 @@ Status: active benchmark-gated implementation program.
|
||||
|
||||
## Objective
|
||||
|
||||
Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.
|
||||
Serve the exact pinned GLM-5.2 `UD-IQ1_S` artifact in `reasoning_effort=max` mode across consumer machines with useful measured performance. Dense Llama is a structural fixture; the real multi-node GLM target is the alpha release gate.
|
||||
|
||||
See **[GLM-5.2 Max distributed alpha roadmap](GLM-5.2-MAX-ALPHA-ROADMAP.md)** for the target identity, minimum hardware, immutable acceptance matrix, and revised execution order. The 224-GiB figure is an experimental hard-fit floor; recommended topology is 5×64 GiB or 3×96/128 GiB after the required per-node reserve.
|
||||
|
||||
## Critical path
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@ Last updated: 2026-07-13
|
||||
|
||||
The system exists to serve high-quality models that exceed one consumer node's memory while retaining useful interactive speed and aggregate concurrency. A feature that only produces a distributed demo but is slower, globally serialized, or impossible to operate on consumer hardware is not complete.
|
||||
|
||||
The alpha target is the exact pinned GLM-5.2 `UD-IQ1_S` artifact in `reasoning_effort=max` mode. Its target-specific architecture/resource/acceptance contract is [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama is a structural fixture, not the product target.
|
||||
|
||||
## Existing control plane
|
||||
|
||||
Meshnet remains the only public control plane:
|
||||
@@ -94,7 +96,9 @@ named tensor bundle
|
||||
compression/checksum
|
||||
```
|
||||
|
||||
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small.
|
||||
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small. DGR-006 amends the current v1 decode fast path—which carries only one `NamedTensor`—to carry a versioned `TensorBundle`, while preserving compact one-tensor encoding and explicit compatibility behavior.
|
||||
|
||||
Tail completion is not inferred from an activation tensor name. The protocol exposes a typed logits and/or sampled-token result, and exact sampling parameters plus chat-template/reasoning mode are bound to request/runtime identity.
|
||||
|
||||
Direct nodes use gRPC. Nodes requiring the existing relay carry the same protobuf frames as opaque binary through the relay session. This preserves one semantic protocol instead of maintaining separate direct and relay payload contracts.
|
||||
|
||||
@@ -113,6 +117,8 @@ compression/checksum
|
||||
|
||||
Dense Llama may use one residual tensor. Other adapters may require more. vLLM's Llama and Qwen3-MoE PP paths demonstrate a boundary with both `hidden_states` and `residual`; therefore the generic protocol must not assume one anonymous tensor.
|
||||
|
||||
GLM-5.2 normally exchanges a 6,144-element hidden state. If a memory-balanced Shard boundary splits an IndexShare Full producer from Shared consumers, the bundle also carries the typed top-k index sideband. The planner prefers boundaries that keep an IndexShare ownership group local, but the protocol validates the sideband rather than assuming it never crosses a seam.
|
||||
|
||||
Only the head owns token embedding. Only the tail owns final normalization, LM head and sampling. Middle Shards exchange the architecture-defined pre-tail boundary, not final normalized embeddings.
|
||||
|
||||
## Hot KV State and concurrency
|
||||
@@ -243,17 +249,16 @@ The GGUF path ships only if it is faster at acceptable quality or enables a larg
|
||||
|
||||
## Implementation sequence
|
||||
|
||||
1. Lock benchmark/performance contract.
|
||||
2. Define gRPC/protobuf and exact recipe identity.
|
||||
3. Pin llama.cpp and create the minimal patch stack.
|
||||
4. Implement dense-Llama range loading and boundary parity.
|
||||
5. Implement concurrent local KV.
|
||||
6. Build and integrate the standalone worker.
|
||||
7. Pass local two-process real-model acceptance.
|
||||
8. Pass real heterogeneous two-machine acceptance.
|
||||
9. Add continuous batching and failure hardening.
|
||||
10. Enforce the GGUF-versus-safetensors release gate.
|
||||
11. Add Qwen3/Qwen3-MoE as a separately certified adapter.
|
||||
12. Prepare narrow upstream collaboration patches/tests.
|
||||
1. Preserve completed DGR-001 performance and DGR-002 protocol contracts.
|
||||
2. DGR-017 locks exact GLM-5.2 Max artifact, resource, and alpha acceptance identity.
|
||||
3. Define exact recipe identity and pin one reproducible llama.cpp boundary.
|
||||
4. Run two lanes in parallel: DGR-018 establishes the whole-model `UD-IQ1_S` oracle on 224+ GiB usable memory, while DGR-005/DGR-006 implement range loading and named boundary parity with a cheap dense fixture.
|
||||
5. DGR-019 adds explicit GLM-5.2 MoE/MLA/DSA/IndexShare semantics after both lanes pass.
|
||||
6. Implement local KV; build and integrate the standalone worker.
|
||||
7. Pass local two-process and real two-physical-machine execution.
|
||||
8. Harden cancellation, node loss, restart, and cleanup required by alpha.
|
||||
9. DGR-020 executes the exact multi-node target and emits immutable `alpha` or `stop`.
|
||||
10. Post-alpha: continuous batching, final comparison, longer context, MTP, and package optimization.
|
||||
11. Prepare narrow upstream patches/tests; add Qwen as later architecture expansion.
|
||||
|
||||
See [the Ralph backlog](prd.json) and [implementation strategy](implementation-strategy.md).
|
||||
|
||||
@@ -45,17 +45,17 @@ All recipes completed every request with zero failures.
|
||||
|
||||
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|
||||
|---|---:|---:|---:|
|
||||
| Decode tok/s, c=1 | 46.1 | 88.0 | 170.1 |
|
||||
| Aggregate decode tok/s, c=4 | 47.1 | 211.4 | 206.4 |
|
||||
| TTFT p50, c=1 | 37.5 ms | 43.9 ms | 23.8 ms |
|
||||
| Decode tok/s, c=1 | 40.8 | 98.5 | 207.7 |
|
||||
| Aggregate decode tok/s, c=4 | 46.5 | 222.8 | 195.7 |
|
||||
| TTFT p50, c=1 | 40.0 ms | 15.1 ms | 21.6 ms |
|
||||
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
|
||||
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
|
||||
| Failures | 0 | 0 | 0 |
|
||||
|
||||
Against the reference, the eligible Q4_K_M lane measured:
|
||||
|
||||
- single-request decode speedup: **3.69×**;
|
||||
- concurrency-4 aggregate throughput speedup: **4.38×**;
|
||||
- single-request decode speedup: **5.10×**;
|
||||
- concurrency-4 aggregate throughput speedup: **4.20×**;
|
||||
- resident-memory ratio: **0.279×**;
|
||||
- artifact-size ratio: **0.398×**.
|
||||
|
||||
@@ -65,9 +65,11 @@ The near-lossless BF16 quality lane compared all three prompts but measured:
|
||||
- mean text similarity: **0.9471** (v1 requires at least `0.97`).
|
||||
|
||||
Tokenization and stopping were controlled: every runtime saw the same prompt
|
||||
token counts and reported 31 post-TTFT decode tokens. The mismatch is genuine
|
||||
greedy runtime divergence on two prompts, not missing coverage or a text-length
|
||||
artifact. Therefore `contract-evaluation.json` records:
|
||||
token counts and reported 31 post-TTFT decode tokens. The v1 mismatch is a
|
||||
real greedy-output divergence on two prompts, not missing coverage or a
|
||||
text-length artifact. Its root cause remains undetermined; no post-contract
|
||||
logit-tie claim is acceptance evidence. Therefore `contract-evaluation.json`
|
||||
records:
|
||||
|
||||
```text
|
||||
verdict: stop
|
||||
@@ -79,6 +81,35 @@ stop_condition_met: true
|
||||
|
||||
Thresholds were not changed after observing these results.
|
||||
|
||||
## Post-contract parity and ROCm diagnostics
|
||||
|
||||
`summarize-quality-parity.py` verifies and separates two signed sources. The CPU
|
||||
v1 row uses CPU kernels and a Transformers BF16 oracle; it remains at `0.3333`
|
||||
exact match with an unexplained divergence. The ROCm row uses a different plan,
|
||||
GPU kernels, and a Transformers float32 oracle. In that narrower diagnostic,
|
||||
the same BF16 GGUF artifact matches all three 32-token sequences exactly (`1.0`
|
||||
exact match and `1.0` similarity). No conversion corruption was observed in
|
||||
that three-sequence ROCm sample; this does not prove global conversion
|
||||
correctness or explain the CPU result.
|
||||
|
||||
A separate HIP build at commit `e920c523` was compiled for `gfx1151` and
|
||||
measured `ROCm0: Radeon 8060S Graphics`; its `llama-server` SHA-256 is
|
||||
`b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139`.
|
||||
A signed `gpu-diagnostic` profile measured zero failures:
|
||||
|
||||
| GPU metric | Transformers BF16 ROCm | llama.cpp Q4 ROCm | Q4 ratio |
|
||||
|---|---:|---:|---:|
|
||||
| Decode tok/s, c=1 | 81.12 | 251.25 | **3.10×** |
|
||||
| Aggregate decode tok/s, c=4 | 91.24 | 511.33 | **5.60×** |
|
||||
| TTFT p50, c=1 | 13.77 ms | 11.80 ms | **0.857×** |
|
||||
|
||||
The GPU report is signed under the distinct
|
||||
`run_configured_gpu_diagnostic/v1` producer. The v1 evaluator rejects that
|
||||
producer even when its signature is valid. llama-server process VRAM remains
|
||||
unmeasured, so this diagnostic cannot replace or satisfy the immutable v1
|
||||
contract. Its signed backend detail records the measured `ROCm0: Radeon 8060S
|
||||
Graphics` device and `25/25` offloaded layers.
|
||||
|
||||
## Implementation
|
||||
|
||||
- `recipe_benchmark.py` provides the runtime-neutral measurement core, true
|
||||
@@ -86,8 +117,9 @@ Thresholds were not changed after observing these results.
|
||||
aggregation, failures, and output drift.
|
||||
- `recipe_drivers.py` provides opt-in Transformers and llama-server drivers,
|
||||
mounted-drive confinement, exact artifact/runtime verification, equal
|
||||
device/thread budgets, greedy-only validation, measured host provenance, and
|
||||
a CPU-only v1 guard until process VRAM can be measured honestly.
|
||||
device/thread budgets, greedy-only validation, measured host provenance, a
|
||||
CPU-only v1 guard until process VRAM can be measured honestly, and a distinct
|
||||
signed GPU diagnostic profile that the v1 evaluator cannot accept.
|
||||
- Peak RSS is runtime-scoped: Transformers reports growth above its pre-runtime
|
||||
Python baseline, while llama.cpp reports its isolated server process tree.
|
||||
Both are sampled continuously during in-flight requests.
|
||||
@@ -99,9 +131,15 @@ Thresholds were not changed after observing these results.
|
||||
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
|
||||
after it; the driver subtracts that token so decode throughput matches the
|
||||
Transformers inter-token convention.
|
||||
- `performance_contract.py` rejects wrong plans, synthetic evidence, missing
|
||||
recipes/concurrency, mixed model revisions, incomplete quality coverage,
|
||||
failed references, and missing artifact/host provenance.
|
||||
- `performance_contract.py` rejects wrong plans, unsigned or incorrectly signed
|
||||
real evidence, wrong config/artifact/runtime/backend/host bindings, missing
|
||||
recipes/concurrency, mixed model revisions, incomplete quality coverage, and
|
||||
failed references.
|
||||
- Every non-synthetic report is Ed25519-signed over the complete canonical JSON,
|
||||
including raw outcomes and metrics. The contract pins the public key and exact
|
||||
config SHA-256; the private key remains outside Git at mode `0600`.
|
||||
- The signer fingerprint is independently anchored outside this evidence
|
||||
directory in `../../trusted-evidence-signers.json` and checked by tests.
|
||||
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
|
||||
quality gate, and only performance-fit recipes can earn speed/fit benefits.
|
||||
|
||||
@@ -113,25 +151,31 @@ Thresholds were not changed after observing these results.
|
||||
- `results.txt` — human-readable benchmark summary
|
||||
- `baseline.json` — distilled measurements for later comparison
|
||||
- `contract-evaluation.json` — fail-closed v1 verdict
|
||||
- `quality-parity-diagnosis.json` / `.md` — run/device-scoped signed-evidence summary
|
||||
- `summarize-quality-parity.py` — verifies both evidence chains and regenerates it
|
||||
- `gpu-diagnostic-config.json` — exact ROCm diagnostic artifacts and runtimes
|
||||
- `gpu-diagnostic-results.json` / `.txt` — signed GPU outcomes and summary
|
||||
- `commands.txt` — reproducible conversion, benchmark, evaluation, and test commands
|
||||
- `BLOCKED.md` — downstream stop-condition handoff
|
||||
- `known-unrelated-failure.md` — clean-base reproduction of the tracker race
|
||||
- `../../trusted-evidence-signers.json` — repository-reviewed signer fingerprint
|
||||
|
||||
## Verification
|
||||
|
||||
```text
|
||||
Targeted: 22 passed
|
||||
Full suite: 749 passed, 13 skipped
|
||||
Earlier cancellation retry matrix, DGR-001: 4/5 passed
|
||||
Targeted: 28 passed (5/5 consecutive focused runs)
|
||||
Latest full suite: 755 passed, 13 skipped
|
||||
Earlier full suite: 751 passed, 13 skipped
|
||||
Current cancellation retry matrix, DGR-001: 4/5 passed
|
||||
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
|
||||
compileall: passed
|
||||
git diff --check: passed
|
||||
Evidence JSON parse/integrity checks: passed
|
||||
```
|
||||
|
||||
The full-suite exception is documented in `known-unrelated-failure.md` and
|
||||
satisfies the issue's explicit clean-tree reproduction clause. DGR-001 changes
|
||||
no tracker/proxy files.
|
||||
The intermittent tracker cancellation race reproduced at the same rate on the
|
||||
clean base and is retained in `known-unrelated-failure.md`; the final full suite
|
||||
completed green. DGR-001 changes no tracker/proxy files.
|
||||
|
||||
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
|
||||
invalid: it used Hermes Agent's internal venv. Verification above used the
|
||||
@@ -140,10 +184,12 @@ project `.venv`, which has the DGR-002-compatible runtime. Real inference used
|
||||
|
||||
## Limitations and dependent-story handoff
|
||||
|
||||
- This is a **0.5B CPU baseline**, not evidence for a large model, Radeon GPU,
|
||||
distributed execution, network transport, or native shard worker.
|
||||
- The installed llama.cpp build is CPU-only (`GGML_HIP=OFF`). No GPU comparison
|
||||
is claimed.
|
||||
- The immutable contract result is a **0.5B CPU baseline**. The separate Radeon
|
||||
diagnostic is real local GPU evidence, but neither result covers a large
|
||||
model, distributed execution, network transport, or a native shard worker.
|
||||
- A separate `GGML_HIP=ON` llama.cpp build exists and produced GPU timings, but
|
||||
llama-server process VRAM is not measurable by the current driver; GPU
|
||||
memory/fit claims therefore remain ineligible for v1.
|
||||
- Absolute timings are developer-machine measurements; locked ratios and raw
|
||||
artifacts are provided for reproducibility.
|
||||
- DGR-014 may consume v1 only with the exact plan/evidence requirements enforced
|
||||
|
||||
@@ -1,4 +1,14 @@
|
||||
{
|
||||
"artifact_sha256": {
|
||||
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
|
||||
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6"
|
||||
},
|
||||
"backend_detail": {
|
||||
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
|
||||
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
|
||||
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16"
|
||||
},
|
||||
"evidence_class": "local-real",
|
||||
"host": {
|
||||
"accelerator_name": "Radeon 8060S Graphics",
|
||||
@@ -10,6 +20,12 @@
|
||||
"hostname": "fedora",
|
||||
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||
"llama_cpp_version": "9991",
|
||||
"llama_server_identities": {
|
||||
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
|
||||
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
|
||||
}
|
||||
},
|
||||
"llama_server_sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||
"platform": "Linux-7.0.14-101.fc43.x86_64-x86_64-with-glibc2.42",
|
||||
"python": "3.12.13",
|
||||
@@ -20,34 +36,65 @@
|
||||
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
|
||||
"provenance": {
|
||||
"completed_at": "2026-07-13T16:27:19.647692Z",
|
||||
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
|
||||
"producer": "meshnet_node.recipe_drivers.run_configured_benchmark/v1",
|
||||
"run_id": "e4eedadf-22f6-4907-8990-985456961099",
|
||||
"schema_version": 1,
|
||||
"signature": "owev+/ToswP20C923G6E+srOCUBV5vrjmndVatr9CbTXakiFGqlHrTiEo+aymA4BcSwmG6KJTxlxO6WpLnpcAg==",
|
||||
"signature_algorithm": "ed25519",
|
||||
"signer_public_key_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
|
||||
"started_at": "2026-07-13T16:26:22.361501Z"
|
||||
},
|
||||
"recipe_runtime": {
|
||||
"llama-cpp-near-lossless-quality": {
|
||||
"device": "cpu",
|
||||
"runtime": "llama.cpp-9991-e920c523",
|
||||
"weight_format": "gguf",
|
||||
"weight_quantization": "bfloat16"
|
||||
},
|
||||
"llama-cpp-quantized-performance-fit": {
|
||||
"device": "cpu",
|
||||
"runtime": "llama.cpp-9991-e920c523",
|
||||
"weight_format": "gguf",
|
||||
"weight_quantization": "Q4_K_M"
|
||||
},
|
||||
"transformers-safetensors-reference": {
|
||||
"device": "cpu",
|
||||
"runtime": "transformers-5.13.0",
|
||||
"weight_format": "safetensors",
|
||||
"weight_quantization": "bfloat16"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
"llama-cpp-near-lossless-quality": {
|
||||
"artifact_bytes": 994156448,
|
||||
"available": true,
|
||||
"concurrency": {
|
||||
"1": {
|
||||
"aggregate_decode_tokens_per_sec": 73.7861,
|
||||
"decode_tokens_per_sec": 87.9728,
|
||||
"aggregate_decode_tokens_per_sec": 86.7339,
|
||||
"decode_tokens_per_sec": 98.5178,
|
||||
"failures": 0,
|
||||
"latency_p50_ms": 385.2049,
|
||||
"latency_p95_ms": 560.2939,
|
||||
"peak_rss_bytes": 1110708224,
|
||||
"latency_p50_ms": 333.023,
|
||||
"latency_p95_ms": 383.0597,
|
||||
"peak_rss_bytes": 1110728704,
|
||||
"peak_vram_bytes": 0,
|
||||
"prefill_tokens_per_sec": 1427.2072,
|
||||
"ttft_p50_ms": 43.929,
|
||||
"ttft_p95_ms": 107.003
|
||||
"prefill_tokens_per_sec": 1717.9451,
|
||||
"ttft_p50_ms": 15.069,
|
||||
"ttft_p95_ms": 63.766
|
||||
},
|
||||
"4": {
|
||||
"aggregate_decode_tokens_per_sec": 211.3515,
|
||||
"decode_tokens_per_sec": 73.8932,
|
||||
"aggregate_decode_tokens_per_sec": 222.788,
|
||||
"decode_tokens_per_sec": 76.6297,
|
||||
"failures": 0,
|
||||
"latency_p50_ms": 467.5094,
|
||||
"latency_p95_ms": 790.862,
|
||||
"peak_rss_bytes": 1129578496,
|
||||
"latency_p50_ms": 490.8738,
|
||||
"latency_p95_ms": 646.26,
|
||||
"peak_rss_bytes": 1139466240,
|
||||
"peak_vram_bytes": 0,
|
||||
"prefill_tokens_per_sec": 1077.8162,
|
||||
"ttft_p50_ms": 33.612,
|
||||
"ttft_p95_ms": 128.501
|
||||
"prefill_tokens_per_sec": 859.8985,
|
||||
"ttft_p50_ms": 32.445,
|
||||
"ttft_p95_ms": 218.387
|
||||
}
|
||||
},
|
||||
"device": "cpu",
|
||||
@@ -58,28 +105,28 @@
|
||||
"available": true,
|
||||
"concurrency": {
|
||||
"1": {
|
||||
"aggregate_decode_tokens_per_sec": 110.0458,
|
||||
"decode_tokens_per_sec": 170.131,
|
||||
"aggregate_decode_tokens_per_sec": 139.2693,
|
||||
"decode_tokens_per_sec": 207.712,
|
||||
"failures": 0,
|
||||
"latency_p50_ms": 258.0681,
|
||||
"latency_p95_ms": 465.8523,
|
||||
"peak_rss_bytes": 542167040,
|
||||
"latency_p50_ms": 168.3307,
|
||||
"latency_p95_ms": 305.1338,
|
||||
"peak_rss_bytes": 542081024,
|
||||
"peak_vram_bytes": 0,
|
||||
"prefill_tokens_per_sec": 783.3775,
|
||||
"ttft_p50_ms": 23.847,
|
||||
"ttft_p95_ms": 237.696
|
||||
"prefill_tokens_per_sec": 967.0195,
|
||||
"ttft_p50_ms": 21.582,
|
||||
"ttft_p95_ms": 147.859
|
||||
},
|
||||
"4": {
|
||||
"aggregate_decode_tokens_per_sec": 206.377,
|
||||
"decode_tokens_per_sec": 83.543,
|
||||
"aggregate_decode_tokens_per_sec": 195.6789,
|
||||
"decode_tokens_per_sec": 76.9497,
|
||||
"failures": 0,
|
||||
"latency_p50_ms": 413.3897,
|
||||
"latency_p95_ms": 910.253,
|
||||
"peak_rss_bytes": 572788736,
|
||||
"latency_p50_ms": 437.9196,
|
||||
"latency_p95_ms": 885.5355,
|
||||
"peak_rss_bytes": 573259776,
|
||||
"peak_vram_bytes": 0,
|
||||
"prefill_tokens_per_sec": 474.3116,
|
||||
"ttft_p50_ms": 67.945,
|
||||
"ttft_p95_ms": 431.804
|
||||
"prefill_tokens_per_sec": 572.4424,
|
||||
"ttft_p50_ms": 48.127,
|
||||
"ttft_p95_ms": 416.531
|
||||
}
|
||||
},
|
||||
"device": "cpu",
|
||||
@@ -90,28 +137,28 @@
|
||||
"available": true,
|
||||
"concurrency": {
|
||||
"1": {
|
||||
"aggregate_decode_tokens_per_sec": 40.3425,
|
||||
"decode_tokens_per_sec": 46.1451,
|
||||
"aggregate_decode_tokens_per_sec": 35.4722,
|
||||
"decode_tokens_per_sec": 40.7545,
|
||||
"failures": 0,
|
||||
"latency_p50_ms": 795.4807,
|
||||
"latency_p95_ms": 930.9725,
|
||||
"peak_rss_bytes": 1941213184,
|
||||
"latency_p50_ms": 818.3864,
|
||||
"latency_p95_ms": 1258.0673,
|
||||
"peak_rss_bytes": 1941458944,
|
||||
"peak_vram_bytes": 0,
|
||||
"prefill_tokens_per_sec": 671.8016,
|
||||
"ttft_p50_ms": 37.4548,
|
||||
"ttft_p95_ms": 193.4633
|
||||
"prefill_tokens_per_sec": 625.6467,
|
||||
"ttft_p50_ms": 40.0018,
|
||||
"ttft_p95_ms": 195.2551
|
||||
},
|
||||
"4": {
|
||||
"aggregate_decode_tokens_per_sec": 47.0903,
|
||||
"decode_tokens_per_sec": 13.1337,
|
||||
"aggregate_decode_tokens_per_sec": 46.5375,
|
||||
"decode_tokens_per_sec": 12.9506,
|
||||
"failures": 0,
|
||||
"latency_p50_ms": 2631.0031,
|
||||
"latency_p95_ms": 3073.7389,
|
||||
"peak_rss_bytes": 2177265664,
|
||||
"latency_p50_ms": 2481.8662,
|
||||
"latency_p95_ms": 3365.8395,
|
||||
"peak_rss_bytes": 2104832000,
|
||||
"peak_vram_bytes": 0,
|
||||
"prefill_tokens_per_sec": 247.5617,
|
||||
"ttft_p50_ms": 94.3995,
|
||||
"ttft_p95_ms": 444.6749
|
||||
"prefill_tokens_per_sec": 264.0101,
|
||||
"ttft_p50_ms": 97.0403,
|
||||
"ttft_p95_ms": 429.0665
|
||||
}
|
||||
},
|
||||
"device": "cpu",
|
||||
|
||||
@@ -4,6 +4,10 @@ LLAMA=/run/media/popov/d/DEV/llamacpp/llama.cpp
|
||||
ROCM_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm/bin/python
|
||||
PROJECT_PY=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python
|
||||
OUT=/run/media/popov/DATA/llm/dgr-001
|
||||
SIGNING_KEY=/home/popov/.config/neuron-tai/keys/dgr-001-evidence-ed25519.pem
|
||||
|
||||
# Private signing key is outside Git and must remain owner-only
|
||||
stat -c '%a %n' "$SIGNING_KEY" # expected: 600
|
||||
|
||||
# Converter support check (no writes)
|
||||
$ROCM_PY $LLAMA/convert_hf_to_gguf.py "$SOURCE" --outtype f16 --outfile "$OUT/Qwen2.5-0.5B-Instruct-7ae5576-F16.gguf" --dry-run
|
||||
@@ -27,7 +31,8 @@ print(_artifact_sha256(Path('/run/media/popov/DATA/llm/safetensor/models/models-
|
||||
PY
|
||||
|
||||
# Canonical opt-in local-real benchmark
|
||||
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
|
||||
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
|
||||
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
|
||||
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/benchmark-config.json \
|
||||
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.json \
|
||||
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/results.txt
|
||||
@@ -44,6 +49,37 @@ contract = load_contract(root / 'performance-contract.json')
|
||||
(root / 'contract-evaluation.json').write_text(json.dumps(evaluate_contract(contract, report).to_dict(), indent=2, sort_keys=True) + '\n')
|
||||
PY
|
||||
|
||||
# Optional ROCm GPU diagnostic (not eligible for immutable v1)
|
||||
# The version-matched rocm[devel] wheel expands beyond 20 GB; ensure sufficient
|
||||
# space or relocate its packaged payload before installation.
|
||||
uv pip install --python "$ROCM_PY" --prerelease=allow \
|
||||
--index-url https://rocm.nightlies.amd.com/v2/gfx1151/ \
|
||||
'rocm[devel]==7.13.0a20260513'
|
||||
ROCM_VENV=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv-rocm
|
||||
ROCM_SDK="$ROCM_VENV/bin/rocm-sdk"
|
||||
ROCM_ROOT="$($ROCM_SDK path --root)"
|
||||
ROCM_BIN="$($ROCM_SDK path --bin)"
|
||||
export PATH="$ROCM_VENV/bin:$ROCM_BIN:$PATH"
|
||||
export ROCM_PATH="$ROCM_ROOT" HIP_PATH="$ROCM_ROOT"
|
||||
export CMAKE_PREFIX_PATH="$($ROCM_SDK path --cmake):$ROCM_ROOT"
|
||||
export LD_LIBRARY_PATH="$ROCM_ROOT/lib:$ROCM_ROOT/lib64:${LD_LIBRARY_PATH:-}"
|
||||
$ROCM_VENV/bin/cmake -S /run/media/popov/d/DEV/llamacpp/llama.cpp \
|
||||
-B /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip -G Ninja \
|
||||
-DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
|
||||
-DCMAKE_HIP_COMPILER="$ROCM_VENV/bin/amdclang++" \
|
||||
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=OFF \
|
||||
-DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_BUILD_SERVER=ON
|
||||
$ROCM_VENV/bin/cmake --build /run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip \
|
||||
--target llama-server llama-cli llama-bench -j 16
|
||||
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 MESHNET_EVIDENCE_SIGNING_KEY="$SIGNING_KEY" \
|
||||
PYTHONPATH=packages/node $ROCM_PY -m meshnet_node.recipe_benchmark \
|
||||
--profile gpu-diagnostic \
|
||||
--config .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-config.json \
|
||||
--json-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.json \
|
||||
--summary-out .scratch/distributed-gguf-runtime/evidence/DGR-001/gpu-diagnostic-results.txt
|
||||
PYTHONPATH=packages/node $PROJECT_PY \
|
||||
.scratch/distributed-gguf-runtime/evidence/DGR-001/summarize-quality-parity.py
|
||||
|
||||
# Deterministic verification
|
||||
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q tests/test_recipe_benchmark.py
|
||||
PYTHONPATH=packages/node $PROJECT_PY -m pytest -q
|
||||
|
||||
@@ -17,22 +17,22 @@
|
||||
"lane": "quality",
|
||||
"measurements": {
|
||||
"aggregate_concurrency": 4,
|
||||
"aggregate_throughput_speedup": 4.4882,
|
||||
"aggregate_throughput_speedup": 4.7873,
|
||||
"artifact_size_ratio": 0.9946,
|
||||
"artifact_size_win": false,
|
||||
"compared_prompts": 3,
|
||||
"decode_speedup": 1.9064,
|
||||
"decode_speedup": 2.4173,
|
||||
"exact_match_rate": 0.3333,
|
||||
"expected_prompts": 3,
|
||||
"failure_rate": 0.0,
|
||||
"mean_similarity": 0.9471,
|
||||
"resident_memory_ratio": 0.5722,
|
||||
"ttft_ratio": 1.1729
|
||||
"resident_memory_ratio": 0.5721,
|
||||
"ttft_ratio": 0.3767
|
||||
},
|
||||
"quality_pass": false,
|
||||
"reasons": [
|
||||
"single-request decode 1.91x reference (>= 1.25x) at TTFT ratio 1.17",
|
||||
"aggregate throughput at concurrency 4 is 4.49x reference (>= 1.25x)",
|
||||
"single-request decode 2.42x reference (>= 1.25x) at TTFT ratio 0.38",
|
||||
"aggregate throughput at concurrency 4 is 4.79x reference (>= 1.25x)",
|
||||
"peak resident memory is 0.57x reference (<= 0.75x)",
|
||||
"quality lane exact-match 0.33 / similarity 0.947 versus the reference (fail)"
|
||||
],
|
||||
@@ -47,18 +47,18 @@
|
||||
"lane": "performance-fit",
|
||||
"measurements": {
|
||||
"aggregate_concurrency": 4,
|
||||
"aggregate_throughput_speedup": 4.3826,
|
||||
"aggregate_throughput_speedup": 4.2048,
|
||||
"artifact_size_ratio": 0.398,
|
||||
"artifact_size_win": true,
|
||||
"decode_speedup": 3.6869,
|
||||
"decode_speedup": 5.0967,
|
||||
"failure_rate": 0.0,
|
||||
"resident_memory_ratio": 0.2793,
|
||||
"ttft_ratio": 0.6367
|
||||
"resident_memory_ratio": 0.2792,
|
||||
"ttft_ratio": 0.5395
|
||||
},
|
||||
"quality_pass": null,
|
||||
"reasons": [
|
||||
"single-request decode 3.69x reference (>= 1.25x) at TTFT ratio 0.64",
|
||||
"aggregate throughput at concurrency 4 is 4.38x reference (>= 1.25x)",
|
||||
"single-request decode 5.10x reference (>= 1.25x) at TTFT ratio 0.54",
|
||||
"aggregate throughput at concurrency 4 is 4.20x reference (>= 1.25x)",
|
||||
"peak resident memory is 0.28x reference (<= 0.75x)"
|
||||
],
|
||||
"recipe_id": "llama-cpp-quantized-performance-fit",
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
{
|
||||
"artifact_storage_root": "/run/media/popov/DATA/llm",
|
||||
"evidence_class": "local-real",
|
||||
"host": {
|
||||
"benchmark_lane": "rocm-gpu-diagnostic",
|
||||
"llama_cpp_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||
"llama_cpp_version": "9991",
|
||||
"llama_server_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
|
||||
"converter_sha256": "c819f18fb22927b49fabc3b35d1c9e21ee638b3817eccd1bd4efbcc7116eeb4d",
|
||||
"quantizer_sha256": "bd0cc8c7be6d48aad4755b31062e0e59a887cbadd43dbb8771853d5858bb198f",
|
||||
"transformers_version": "5.13.0",
|
||||
"rocm_target": "gfx1151"
|
||||
},
|
||||
"plan": {
|
||||
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
|
||||
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"prompts": [
|
||||
{
|
||||
"id": "short-fact",
|
||||
"text": "The capital of France is",
|
||||
"context_class": "short"
|
||||
},
|
||||
{
|
||||
"id": "medium-code",
|
||||
"text": "Complete this Python function without commentary:\n\ndef fibonacci(n):\n \"\"\"Return the nth Fibonacci number for n >= 0.\"\"\"\n",
|
||||
"context_class": "medium"
|
||||
},
|
||||
{
|
||||
"id": "long-summary",
|
||||
"text": "A distributed inference service divides a transformer across consumer machines. The tracker owns admission, routing, cancellation, accounting, and telemetry, while workers own only model execution. Every request carries an immutable model identity and revision. Workers must reject incompatible protocol versions and resource demands before allocating large buffers. Activation tensors are chunked, checksummed, bounded by negotiated limits, and propagated with explicit flow-control credits. A caller may disconnect at any time, so cancellation must release queued work, in-flight transfers, and cache reservations without double billing. Retries can occur after network failures, requiring idempotent request identifiers and deterministic completion accounting. The system keeps the existing safetensors path as a correctness reference while a native GGUF path is measured. Benchmarks compare the same prompts, output lengths, sampling policy, device, and concurrency, and they separate near-lossless quality checks from quantized speed and fit claims. Summarize the design priorities in three concise bullet points.",
|
||||
"context_class": "long"
|
||||
}
|
||||
],
|
||||
"sampling": {
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"top_k": 1,
|
||||
"seed": 1234,
|
||||
"max_output_tokens": 32
|
||||
},
|
||||
"concurrency_levels": [
|
||||
1,
|
||||
4
|
||||
],
|
||||
"repeats": 3,
|
||||
"warmup_requests": 2
|
||||
},
|
||||
"recipes": [
|
||||
{
|
||||
"id": "transformers-fp32-rocm-quality-oracle",
|
||||
"runtime": "transformers-5.13.0-rocm-float32",
|
||||
"weight_format": "safetensors",
|
||||
"weight_quantization": "bfloat16-weights-float32-accumulation",
|
||||
"lane": "quality",
|
||||
"device": "cuda",
|
||||
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"is_reference": true,
|
||||
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
|
||||
"driver": {
|
||||
"type": "transformers",
|
||||
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"device": "cuda",
|
||||
"dtype": "float32",
|
||||
"threads": 16
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "llama-cpp-bf16-rocm-quality",
|
||||
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
|
||||
"weight_format": "gguf",
|
||||
"weight_quantization": "bfloat16",
|
||||
"lane": "quality",
|
||||
"device": "cuda",
|
||||
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
|
||||
"artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"is_reference": false,
|
||||
"notes": "Converted directly from the exact mounted safetensors revision while preserving BF16 weights with pinned llama.cpp",
|
||||
"driver": {
|
||||
"type": "llama-cpp-server",
|
||||
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
|
||||
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
|
||||
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-BF16.gguf",
|
||||
"device": "cuda",
|
||||
"threads": 16,
|
||||
"n_parallel": 4,
|
||||
"context_per_slot": 512,
|
||||
"n_gpu_layers": 99
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "transformers-bf16-rocm-throughput",
|
||||
"runtime": "transformers-5.13.0-rocm-bfloat16",
|
||||
"weight_format": "safetensors",
|
||||
"weight_quantization": "bfloat16",
|
||||
"lane": "performance-fit",
|
||||
"device": "cuda",
|
||||
"artifact_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"artifact_sha256": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"is_reference": false,
|
||||
"notes": "artifact_sha256 is the deterministic digest of every snapshot path and file byte",
|
||||
"driver": {
|
||||
"type": "transformers",
|
||||
"model_path": "/run/media/popov/DATA/llm/safetensor/models/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"device": "cuda",
|
||||
"dtype": "bfloat16",
|
||||
"threads": 16
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "llama-cpp-q4-rocm-throughput",
|
||||
"runtime": "llama.cpp-9991-e920c523-rocm-gfx1151",
|
||||
"weight_format": "gguf",
|
||||
"weight_quantization": "Q4_K_M",
|
||||
"lane": "performance-fit",
|
||||
"device": "cuda",
|
||||
"artifact_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
|
||||
"artifact_sha256": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5",
|
||||
"source_model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"source_model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"is_reference": false,
|
||||
"notes": "Quantized from the exact-revision F16 GGUF with pinned llama-quantize",
|
||||
"driver": {
|
||||
"type": "llama-cpp-server",
|
||||
"binary": "/run/media/popov/d/DEV/llamacpp/llama.cpp/build-hip/bin/llama-server",
|
||||
"binary_sha256": "b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139",
|
||||
"gguf_path": "/run/media/popov/DATA/llm/dgr-001/Qwen2.5-0.5B-Instruct-7ae5576-Q4_K_M.gguf",
|
||||
"device": "cuda",
|
||||
"threads": 16,
|
||||
"n_parallel": 4,
|
||||
"context_per_slot": 512,
|
||||
"n_gpu_layers": 99
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,13 @@
|
||||
Recipe benchmark dgr-001-rocm-gpu-diagnostic-v1 (local-real)
|
||||
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
|
||||
transformers-fp32-rocm-quality-oracle [quality ] c= 1 ttft p50/p95 11.0/ 35.5 ms; prefill 5746.7 tok/s; decode 35.7 tok/s; aggregate 29.6 tok/s; rss 1.39 GB; vram 2.26 GB; artifact 1.00 GB; failures 0
|
||||
transformers-fp32-rocm-quality-oracle [quality ] c= 4 ttft p50/p95 27.5/ 80.4 ms; prefill 1985.4 tok/s; decode 9.4 tok/s; aggregate 35.4 tok/s; rss 1.39 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
|
||||
llama-cpp-bf16-rocm-quality [quality ] c= 1 ttft p50/p95 13.2/ 83.4 ms; prefill 4154.4 tok/s; decode 148.0 tok/s; aggregate 127.4 tok/s; rss 0.84 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||
llama-cpp-bf16-rocm-quality [quality ] c= 4 ttft p50/p95 25.1/ 52.1 ms; prefill 2205.4 tok/s; decode 115.1 tok/s; aggregate 337.1 tok/s; rss 0.86 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||
transformers-bf16-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 13.8/ 22.2 ms; prefill 4787.3 tok/s; decode 81.1 tok/s; aggregate 73.5 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
|
||||
transformers-bf16-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 29.7/ 58.5 ms; prefill 2666.5 tok/s; decode 24.4 tok/s; aggregate 91.2 tok/s; rss 0.07 GB; vram 2.74 GB; artifact 1.00 GB; failures 0
|
||||
llama-cpp-q4-rocm-throughput [performance-fit ] c= 1 ttft p50/p95 11.8/ 37.1 ms; prefill 4219.3 tok/s; decode 251.2 tok/s; aggregate 200.1 tok/s; rss 0.69 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||
llama-cpp-q4-rocm-throughput [performance-fit ] c= 4 ttft p50/p95 21.4/ 101.0 ms; prefill 2126.9 tok/s; decode 189.7 tok/s; aggregate 511.3 tok/s; rss 0.72 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||
drift llama-cpp-bf16-rocm-quality vs transformers-fp32-rocm-quality-oracle exact 1.00; similarity 1.000 (gated)
|
||||
drift transformers-bf16-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.33; similarity 0.946 (advisory)
|
||||
drift llama-cpp-q4-rocm-throughput vs transformers-fp32-rocm-quality-oracle exact 0.00; similarity 0.628 (advisory)
|
||||
@@ -1,7 +1,7 @@
|
||||
# Observed pre-existing intermittent tracker race
|
||||
|
||||
This file records an earlier unrelated timing observation; it is **not** the
|
||||
final DGR-001 verification result.
|
||||
This file records an unrelated timing observation and its repeated reproduction;
|
||||
it is **not** a DGR-001 benchmark/contract failure.
|
||||
|
||||
Test:
|
||||
|
||||
@@ -15,18 +15,41 @@ One earlier full-suite run produced:
|
||||
1 failed, 745 passed, 13 skipped
|
||||
```
|
||||
|
||||
A five-run isolated retry matrix reproduced the same rate on both branches:
|
||||
A five-run isolated retry matrix reproduced the same rate repeatedly:
|
||||
|
||||
```text
|
||||
DGR-001 branch: 4/5 passed, 1/5 failed
|
||||
current DGR-001 branch: 4/5 passed, 1/5 failed
|
||||
clean d904c40: 4/5 passed, 1/5 failed
|
||||
```
|
||||
|
||||
The final full-suite run on the exact hardened DGR-001 state completed green:
|
||||
An earlier full-suite run on the signed-provenance DGR-001 state completed
|
||||
green:
|
||||
|
||||
```text
|
||||
749 passed, 13 skipped in 251.42s
|
||||
751 passed, 13 skipped
|
||||
```
|
||||
|
||||
The earlier race was therefore timing-sensitive, pre-existing, and unrelated
|
||||
to the DGR-001 benchmark/contract files.
|
||||
Two full-suite runs after adding the isolated GPU diagnostic profile each hit
|
||||
the same race and otherwise passed:
|
||||
|
||||
```text
|
||||
1 failed, 750 passed, 13 skipped
|
||||
```
|
||||
|
||||
The latest expanded hardening suite hit the same race and otherwise passed:
|
||||
|
||||
```text
|
||||
1 failed, 754 passed, 13 skipped
|
||||
```
|
||||
|
||||
The final hardened state subsequently completed a full green run:
|
||||
|
||||
```text
|
||||
755 passed, 13 skipped
|
||||
```
|
||||
|
||||
In each failure, the mock upstream's three-second release timeout completed the
|
||||
stream before the cancel POST, so the request was already absent and the cancel
|
||||
endpoint returned 404. No tracker/proxy file changed in DGR-001. The race is
|
||||
therefore timing-sensitive, pre-existing, and unrelated to the benchmark,
|
||||
provenance, or GPU-diagnostic code.
|
||||
|
||||
@@ -37,8 +37,51 @@
|
||||
"minimum_prompt_count": 3,
|
||||
"minimum_repeats": 3,
|
||||
"minimum_output_tokens": 32,
|
||||
"required_device": "cpu"
|
||||
"required_device": "cpu",
|
||||
"required_config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
|
||||
"required_signer_public_key": "zQ/qRMwF/ydazzaxEI24Xvnrl5bZxzw16JYpP0bfRuI=",
|
||||
"required_artifact_sha256": {
|
||||
"transformers-safetensors-reference": "e596e9d6205fdc9177569cccd7f8b471b058f66e3630c8e4326d5aad52bd18b6",
|
||||
"llama-cpp-near-lossless-quality": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||
"llama-cpp-quantized-performance-fit": "a88e3f570e2efeaf06b50df9859db2c70d8646aa3a2c94a14e14d5797a2921a5"
|
||||
},
|
||||
"required_recipe_runtime": {
|
||||
"transformers-safetensors-reference": {
|
||||
"runtime": "transformers-5.13.0",
|
||||
"weight_format": "safetensors",
|
||||
"weight_quantization": "bfloat16",
|
||||
"device": "cpu"
|
||||
},
|
||||
"llama-cpp-near-lossless-quality": {
|
||||
"runtime": "llama.cpp-9991-e920c523",
|
||||
"weight_format": "gguf",
|
||||
"weight_quantization": "bfloat16",
|
||||
"device": "cpu"
|
||||
},
|
||||
"llama-cpp-quantized-performance-fit": {
|
||||
"runtime": "llama.cpp-9991-e920c523",
|
||||
"weight_format": "gguf",
|
||||
"weight_quantization": "Q4_K_M",
|
||||
"device": "cpu"
|
||||
}
|
||||
},
|
||||
"required_backend_detail": {
|
||||
"transformers-safetensors-reference": "torch 2.10.0+rocm7.13.0a20260513; dtype bfloat16; device cpu; intra-op threads 16",
|
||||
"llama-cpp-near-lossless-quality": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0",
|
||||
"llama-cpp-quantized-performance-fit": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd; threads 16; parallel slots 4; ctx/slot 512; gpu layers 0"
|
||||
},
|
||||
"required_host_identity": {
|
||||
"python": "3.12.13",
|
||||
"torch_version": "2.10.0+rocm7.13.0a20260513",
|
||||
"transformers_version": "5.13.0",
|
||||
"llama_server_identities": {
|
||||
"/run/media/popov/d/DEV/llamacpp/llama.cpp/build/bin/llama-server": {
|
||||
"sha256": "fd8fe612970f23e447f2e717cfa51665be06b8d7315ba60556e010f6bca510dd",
|
||||
"version": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"stop_condition": "Stop the native llama.cpp/GGUF track when, on the same machine and device as the Transformers/safetensors reference and under this plan, no performance-fit GGUF recipe delivers either a meaningful speed benefit (>=25% higher single-request decode tokens/sec without a >25% worse TTFT, or >=25% higher aggregate throughput under concurrency) or a meaningful fit benefit (>=25% lower peak resident memory), or when the near-lossless quality lane fails, which indicates a broken runtime rather than a quantization trade-off.",
|
||||
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract."
|
||||
"notes": "Quantized performance-fit output drift is reported as advisory only. It is not numerical-equivalence evidence. DGR-014 consumes this immutable v1 contract. Non-synthetic evidence must be Ed25519-signed by the pinned key and match the exact locked config, artifacts, runtimes, backends, and host runtime identity."
|
||||
}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"conclusion": {
|
||||
"conversion_corruption_observed_in_rocm_sample": false,
|
||||
"cpu_bf16_divergence_explained": false,
|
||||
"recommended_v2_design": "Predeclare a float32 quality oracle separately from the BF16 performance reference, with a larger prompt corpus and immutable thresholds.",
|
||||
"scope": "The ROCm diagnostic establishes only that the same BF16 GGUF artifact matched the float32 oracle for three GPU sequences; it does not explain the CPU BF16 divergence or prove global conversion correctness.",
|
||||
"v1_verdict_changed": false
|
||||
},
|
||||
"cpu_v1": {
|
||||
"candidate": "llama.cpp BF16 GGUF",
|
||||
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||
"config_sha256": "00b2cce3e2f281bdf92fc5304ba5cac915a178ffccd3b9a25995ce39c00b90d3",
|
||||
"contract_verdict": "stop",
|
||||
"device": "cpu",
|
||||
"exact_match_rate": 0.3333,
|
||||
"mean_similarity": 0.9471,
|
||||
"plan_id": "dgr-001-controlled-whole-model-baseline-v1",
|
||||
"plan_sha256": "efe24690a9a7164bac6ab3fd0a6b22f078fc08aaefcfb96210ddf154e6050570",
|
||||
"quality_oracle": "Transformers BF16 safetensors",
|
||||
"report": "results.json",
|
||||
"report_sha256": "5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5",
|
||||
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
|
||||
"run_id": "e4eedadf-22f6-4907-8990-985456961099"
|
||||
},
|
||||
"model_id": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"model_revision": "7ae557604adf67be50417f59c2c2f167def9a775",
|
||||
"rocm_diagnostic": {
|
||||
"candidate": "llama.cpp BF16 GGUF",
|
||||
"candidate_artifact_sha256": "e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862",
|
||||
"config_sha256": "b0f0c846c818f1307d034cee1f81daa311efc20985c32a4cdbbbd8ffe4153892",
|
||||
"device": "cuda (ROCm)",
|
||||
"exact_match_rate": 1.0,
|
||||
"failures": 0,
|
||||
"mean_similarity": 1.0,
|
||||
"measured_backend_detail": "version: 9991 (e920c523) | built with GNU 15.2.1 for Linux x86_64; binary sha256 b6bb4da687dbde86e243ba006cef05919b7b97255cd7e2371e1d451220aca139; threads 16; parallel slots 4; ctx/slot 512; requested gpu layers 99; measured accelerator ROCm0: Radeon 8060S Graphics; measured offload 25/25 layers",
|
||||
"plan_id": "dgr-001-rocm-gpu-diagnostic-v1",
|
||||
"plan_sha256": "dae8e40963588f71f5d201fd163d39bd762e392544b5603d483e90d21abee2e8",
|
||||
"producer": "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1",
|
||||
"quality_oracle": "Transformers float32 safetensors",
|
||||
"report": "gpu-diagnostic-results.json",
|
||||
"report_sha256": "527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d",
|
||||
"run_id": "31bf44e7-ccd4-4277-84ac-c775dee65411",
|
||||
"signer_fingerprint": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
|
||||
"v1_eligible": false
|
||||
},
|
||||
"schema_version": 2
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
# DGR-001 quality-parity evidence summary
|
||||
|
||||
This summary is generated by `summarize-quality-parity.py` from signed reports.
|
||||
It contains no independent logit measurements or self-asserted verification flag.
|
||||
|
||||
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|
||||
|---|---|---|---|---:|---:|---|
|
||||
| CPU v1 (`e4eedadf-22f6-4907-8990-985456961099`) | CPU | Transformers BF16 | llama.cpp BF16 | 0.3333 | 0.9471 | immutable `stop` |
|
||||
| ROCm diagnostic (`31bf44e7-ccd4-4277-84ac-c775dee65411`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | 1.0000 | 1.0000 | diagnostic only |
|
||||
|
||||
## Interpretation
|
||||
|
||||
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
|
||||
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
|
||||
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
|
||||
matched the float32 oracle for all three GPU sequences with zero failures.
|
||||
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
|
||||
`25/25` layer offload.
|
||||
|
||||
No conversion corruption was observed in that three-sequence ROCm sample. This
|
||||
does not prove global conversion correctness and does not retroactively change
|
||||
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
|
||||
separately from its BF16 performance reference and use a larger corpus.
|
||||
|
||||
## Reproduction and bindings
|
||||
|
||||
- CPU report SHA-256: `5d99a58806f39821c9206728047b8c5d605027d8a41b88639089b2418da890b5`
|
||||
- GPU report SHA-256: `527b33d03627d57d60b30331e6b9119f579a828d6f6acb5c74ca25bab0af5f3d`
|
||||
- BF16 GGUF SHA-256: `e842fdc35d7f00fda95a54e1b51731ba1d196aea45065cc9f46925fdc1d6f862`
|
||||
- Signer fingerprint: `8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de`
|
||||
- Exact verification command: see `commands.txt`.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,10 +1,10 @@
|
||||
Recipe benchmark dgr-001-controlled-whole-model-baseline-v1 (local-real)
|
||||
model Qwen/Qwen2.5-0.5B-Instruct@7ae557604adf67be50417f59c2c2f167def9a775
|
||||
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 37.5/ 193.5 ms; prefill 671.8 tok/s; decode 46.1 tok/s; aggregate 40.3 tok/s; rss 1.94 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
|
||||
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 94.4/ 444.7 ms; prefill 247.6 tok/s; decode 13.1 tok/s; aggregate 47.1 tok/s; rss 2.18 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
|
||||
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 43.9/ 107.0 ms; prefill 1427.2 tok/s; decode 88.0 tok/s; aggregate 73.8 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 33.6/ 128.5 ms; prefill 1077.8 tok/s; decode 73.9 tok/s; aggregate 211.4 tok/s; rss 1.13 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 23.8/ 237.7 ms; prefill 783.4 tok/s; decode 170.1 tok/s; aggregate 110.0 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 67.9/ 431.8 ms; prefill 474.3 tok/s; decode 83.5 tok/s; aggregate 206.4 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||
transformers-safetensors-reference [quality ] c= 1 ttft p50/p95 40.0/ 195.3 ms; prefill 625.6 tok/s; decode 40.8 tok/s; aggregate 35.5 tok/s; rss 1.94 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
|
||||
transformers-safetensors-reference [quality ] c= 4 ttft p50/p95 97.0/ 429.1 ms; prefill 264.0 tok/s; decode 13.0 tok/s; aggregate 46.5 tok/s; rss 2.10 GB; vram 0.00 GB; artifact 1.00 GB; failures 0
|
||||
llama-cpp-near-lossless-quality [quality ] c= 1 ttft p50/p95 15.1/ 63.8 ms; prefill 1717.9 tok/s; decode 98.5 tok/s; aggregate 86.7 tok/s; rss 1.11 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||
llama-cpp-near-lossless-quality [quality ] c= 4 ttft p50/p95 32.4/ 218.4 ms; prefill 859.9 tok/s; decode 76.6 tok/s; aggregate 222.8 tok/s; rss 1.14 GB; vram 0.00 GB; artifact 0.99 GB; failures 0
|
||||
llama-cpp-quantized-performance-fit [performance-fit ] c= 1 ttft p50/p95 21.6/ 147.9 ms; prefill 967.0 tok/s; decode 207.7 tok/s; aggregate 139.3 tok/s; rss 0.54 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||
llama-cpp-quantized-performance-fit [performance-fit ] c= 4 ttft p50/p95 48.1/ 416.5 ms; prefill 572.4 tok/s; decode 76.9 tok/s; aggregate 195.7 tok/s; rss 0.57 GB; vram 0.00 GB; artifact 0.40 GB; failures 0
|
||||
drift llama-cpp-near-lossless-quality vs transformers-safetensors-reference exact 0.33; similarity 0.947 (gated)
|
||||
drift llama-cpp-quantized-performance-fit vs transformers-safetensors-reference exact 0.00; similarity 0.456 (advisory)
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the DGR-001 parity summary from cryptographically verified reports."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
|
||||
|
||||
from meshnet_node.performance_contract import (
|
||||
_canonical_sha256,
|
||||
evaluate_contract,
|
||||
load_contract,
|
||||
report_signing_payload,
|
||||
)
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def _read(name: str) -> dict:
|
||||
return json.loads((ROOT / name).read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def _file_sha256(name: str) -> str:
|
||||
return hashlib.sha256((ROOT / name).read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def _drift(report: dict, recipe_id: str) -> dict:
|
||||
return next(item for item in report["drift"] if item["recipe_id"] == recipe_id)
|
||||
|
||||
|
||||
def _recipe(report: dict, recipe_id: str) -> dict:
|
||||
return next(item for item in report["recipes"] if item["recipe"]["id"] == recipe_id)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
contract = load_contract(ROOT / "performance-contract.json")
|
||||
cpu_report = _read("results.json")
|
||||
gpu_config = _read("gpu-diagnostic-config.json")
|
||||
gpu_report = _read("gpu-diagnostic-results.json")
|
||||
|
||||
cpu_evaluation = evaluate_contract(contract, cpu_report)
|
||||
if cpu_evaluation.verdict != "stop":
|
||||
raise RuntimeError("immutable CPU v1 evidence no longer evaluates to stop")
|
||||
|
||||
public_key_bytes = base64.b64decode(contract.baseline["required_signer_public_key"])
|
||||
public_key = Ed25519PublicKey.from_public_bytes(public_key_bytes)
|
||||
public_key.verify(
|
||||
base64.b64decode(gpu_report["provenance"]["signature"]),
|
||||
report_signing_payload(gpu_report),
|
||||
)
|
||||
signer_fingerprint = hashlib.sha256(public_key_bytes).hexdigest()
|
||||
if gpu_report["provenance"]["signer_public_key_sha256"] != signer_fingerprint:
|
||||
raise RuntimeError("GPU report signer fingerprint does not match the contract trust key")
|
||||
if gpu_report["provenance"]["config_sha256"] != _canonical_sha256(gpu_config):
|
||||
raise RuntimeError("GPU report is not bound to gpu-diagnostic-config.json")
|
||||
|
||||
if gpu_report.get("schema_version") != 1 or gpu_report.get("evidence_class") != "local-real":
|
||||
raise RuntimeError("GPU report must be schema-v1 local-real evidence")
|
||||
expected_producer = "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1"
|
||||
if gpu_report["provenance"].get("producer") != expected_producer:
|
||||
raise RuntimeError("GPU report was not emitted by the canonical diagnostic producer")
|
||||
if gpu_report.get("reference_recipe_id") != "transformers-fp32-rocm-quality-oracle":
|
||||
raise RuntimeError("GPU report uses the wrong quality reference")
|
||||
if gpu_report.get("host", {}).get("benchmark_lane") != "rocm-gpu-diagnostic":
|
||||
raise RuntimeError("GPU report lacks the diagnostic host marker")
|
||||
|
||||
trusted = json.loads(
|
||||
(ROOT.parents[1] / "trusted-evidence-signers.json").read_text(encoding="utf-8")
|
||||
)
|
||||
if not any(
|
||||
signer.get("algorithm") == "ed25519"
|
||||
and signer.get("fingerprint_sha256") == signer_fingerprint
|
||||
and signer.get("status") == "active"
|
||||
for signer in trusted.get("signers", ())
|
||||
):
|
||||
raise RuntimeError("GPU signer is not active in the trusted-signers registry")
|
||||
|
||||
for field in ("model_id", "model_revision"):
|
||||
if gpu_report["plan"].get(field) != cpu_report["plan"].get(field):
|
||||
raise RuntimeError(f"CPU and GPU reports do not share {field}")
|
||||
if gpu_config["plan"].get(field) != gpu_report["plan"].get(field):
|
||||
raise RuntimeError(f"GPU config and report do not share {field}")
|
||||
|
||||
expected_recipes = {
|
||||
"transformers-fp32-rocm-quality-oracle": ("quality", "cuda"),
|
||||
"llama-cpp-bf16-rocm-quality": ("quality", "cuda"),
|
||||
"transformers-bf16-rocm-throughput": ("performance-fit", "cuda"),
|
||||
"llama-cpp-q4-rocm-throughput": ("performance-fit", "cuda"),
|
||||
}
|
||||
actual_recipes = {
|
||||
entry["recipe"]["id"]: (entry["recipe"]["lane"], entry["recipe"]["device"])
|
||||
for entry in gpu_report["recipes"]
|
||||
}
|
||||
if actual_recipes != expected_recipes:
|
||||
raise RuntimeError("GPU report recipe identities, lanes, or devices changed")
|
||||
|
||||
gpu_prompt_ids = {prompt["id"] for prompt in gpu_report["plan"]["prompts"]}
|
||||
levels = {int(level) for level in gpu_report["plan"]["concurrency_levels"]}
|
||||
repeats = int(gpu_report["plan"]["repeats"])
|
||||
expected_outcomes = len(gpu_prompt_ids) * repeats * sum(levels)
|
||||
for entry in gpu_report["recipes"]:
|
||||
recipe_id = entry["recipe"]["id"]
|
||||
if not entry.get("available") or len(entry.get("outcomes", ())) != expected_outcomes:
|
||||
raise RuntimeError(f"GPU recipe {recipe_id!r} lacks complete outcomes")
|
||||
if any(
|
||||
not outcome.get("ok")
|
||||
or outcome.get("recipe_id") != recipe_id
|
||||
or outcome.get("prompt_id") not in gpu_prompt_ids
|
||||
or int(outcome.get("concurrency", 0)) not in levels
|
||||
or not 0 <= int(outcome.get("repeat", -1)) < repeats
|
||||
for outcome in entry["outcomes"]
|
||||
):
|
||||
raise RuntimeError(f"GPU recipe {recipe_id!r} contains failed or invalid outcomes")
|
||||
if {int(level) for level in entry["concurrency"]} != levels:
|
||||
raise RuntimeError(f"GPU recipe {recipe_id!r} has wrong concurrency cells")
|
||||
for prompt_id in gpu_prompt_ids:
|
||||
for level in levels:
|
||||
for repeat in range(repeats):
|
||||
count = sum(
|
||||
outcome["prompt_id"] == prompt_id
|
||||
and int(outcome["concurrency"]) == level
|
||||
and int(outcome["repeat"]) == repeat
|
||||
for outcome in entry["outcomes"]
|
||||
)
|
||||
if count != level:
|
||||
raise RuntimeError(
|
||||
f"GPU recipe {recipe_id!r} lacks complete request coverage"
|
||||
)
|
||||
if any(
|
||||
int(cell.get("failures", -1)) != 0
|
||||
or int(cell.get("requests", -1))
|
||||
!= len(
|
||||
[
|
||||
outcome
|
||||
for outcome in entry["outcomes"]
|
||||
if int(outcome["concurrency"]) == int(level)
|
||||
]
|
||||
)
|
||||
for level, cell in entry["concurrency"].items()
|
||||
):
|
||||
raise RuntimeError(f"GPU recipe {recipe_id!r} aggregates do not match outcomes")
|
||||
|
||||
cpu_quality = _drift(cpu_report, "llama-cpp-near-lossless-quality")
|
||||
gpu_quality = _drift(gpu_report, "llama-cpp-bf16-rocm-quality")
|
||||
cpu_recipe = _recipe(cpu_report, "llama-cpp-near-lossless-quality")
|
||||
gpu_recipe = _recipe(gpu_report, "llama-cpp-bf16-rocm-quality")
|
||||
gpu_backend = gpu_recipe["load"]["backend_detail"]
|
||||
if "measured accelerator ROCm0: Radeon 8060S Graphics" not in gpu_backend:
|
||||
raise RuntimeError("GPU report lacks measured ROCm device evidence")
|
||||
if "measured offload 25/25 layers" not in gpu_backend:
|
||||
raise RuntimeError("GPU report lacks measured layer-offload evidence")
|
||||
if cpu_recipe["recipe"]["artifact_sha256"] != gpu_recipe["recipe"]["artifact_sha256"]:
|
||||
raise RuntimeError("CPU and GPU diagnostics use different BF16 GGUF artifacts")
|
||||
if gpu_quality.get("compared_prompts") != len(gpu_prompt_ids):
|
||||
raise RuntimeError("GPU quality drift lacks complete prompt coverage")
|
||||
if {item["prompt_id"] for item in gpu_quality.get("per_prompt", ())} != gpu_prompt_ids:
|
||||
raise RuntimeError("GPU quality drift prompt identities do not match the plan")
|
||||
|
||||
summary = {
|
||||
"schema_version": 2,
|
||||
"model_id": cpu_report["plan"]["model_id"],
|
||||
"model_revision": cpu_report["plan"]["model_revision"],
|
||||
"cpu_v1": {
|
||||
"report": "results.json",
|
||||
"report_sha256": _file_sha256("results.json"),
|
||||
"run_id": cpu_report["provenance"]["run_id"],
|
||||
"plan_id": cpu_report["plan"]["plan_id"],
|
||||
"plan_sha256": _canonical_sha256(cpu_report["plan"]),
|
||||
"config_sha256": cpu_report["provenance"]["config_sha256"],
|
||||
"device": "cpu",
|
||||
"quality_oracle": "Transformers BF16 safetensors",
|
||||
"candidate": "llama.cpp BF16 GGUF",
|
||||
"candidate_artifact_sha256": cpu_recipe["recipe"]["artifact_sha256"],
|
||||
"exact_match_rate": cpu_quality["exact_match_rate"],
|
||||
"mean_similarity": cpu_quality["mean_similarity"],
|
||||
"contract_verdict": cpu_evaluation.verdict,
|
||||
"root_cause": "undetermined; no logit-tie claim is acceptance evidence",
|
||||
},
|
||||
"rocm_diagnostic": {
|
||||
"report": "gpu-diagnostic-results.json",
|
||||
"report_sha256": _file_sha256("gpu-diagnostic-results.json"),
|
||||
"run_id": gpu_report["provenance"]["run_id"],
|
||||
"producer": gpu_report["provenance"]["producer"],
|
||||
"signer_fingerprint": signer_fingerprint,
|
||||
"plan_id": gpu_report["plan"]["plan_id"],
|
||||
"plan_sha256": _canonical_sha256(gpu_report["plan"]),
|
||||
"config_sha256": gpu_report["provenance"]["config_sha256"],
|
||||
"device": "cuda (ROCm)",
|
||||
"quality_oracle": "Transformers float32 safetensors",
|
||||
"candidate": "llama.cpp BF16 GGUF",
|
||||
"candidate_artifact_sha256": gpu_recipe["recipe"]["artifact_sha256"],
|
||||
"measured_backend_detail": gpu_backend,
|
||||
"exact_match_rate": gpu_quality["exact_match_rate"],
|
||||
"mean_similarity": gpu_quality["mean_similarity"],
|
||||
"failures": sum(
|
||||
metrics["failures"]
|
||||
for entry in gpu_report["recipes"]
|
||||
for metrics in entry["concurrency"].values()
|
||||
),
|
||||
"v1_eligible": False,
|
||||
},
|
||||
"conclusion": {
|
||||
"v1_verdict_changed": False,
|
||||
"cpu_bf16_divergence_explained": False,
|
||||
"conversion_corruption_observed_in_rocm_sample": False,
|
||||
"scope": (
|
||||
"The ROCm diagnostic establishes only that the same BF16 GGUF artifact "
|
||||
"matched the float32 oracle for three GPU sequences; it does not explain "
|
||||
"the CPU BF16 divergence or prove global conversion correctness."
|
||||
),
|
||||
"recommended_v2_design": (
|
||||
"Predeclare a float32 quality oracle separately from the BF16 performance "
|
||||
"reference, with a larger prompt corpus and immutable thresholds."
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
(ROOT / "quality-parity-diagnosis.json").write_text(
|
||||
json.dumps(summary, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
md = f"""# DGR-001 quality-parity evidence summary
|
||||
|
||||
This summary is generated by `summarize-quality-parity.py` from signed reports.
|
||||
It contains no independent logit measurements or self-asserted verification flag.
|
||||
|
||||
| Source | Device | Quality oracle | BF16 GGUF candidate | Exact | Similarity | Status |
|
||||
|---|---|---|---|---:|---:|---|
|
||||
| CPU v1 (`{summary['cpu_v1']['run_id']}`) | CPU | Transformers BF16 | llama.cpp BF16 | {summary['cpu_v1']['exact_match_rate']:.4f} | {summary['cpu_v1']['mean_similarity']:.4f} | immutable `stop` |
|
||||
| ROCm diagnostic (`{summary['rocm_diagnostic']['run_id']}`) | ROCm0 / Radeon 8060S | Transformers float32 | llama.cpp BF16 | {summary['rocm_diagnostic']['exact_match_rate']:.4f} | {summary['rocm_diagnostic']['mean_similarity']:.4f} | diagnostic only |
|
||||
|
||||
## Interpretation
|
||||
|
||||
The CPU and ROCm rows use different plans, devices, kernels, and quality oracles.
|
||||
The CPU BF16 divergence remains unexplained and v1 remains `stop`. The signed
|
||||
ROCm report establishes the narrower fact that the same BF16 GGUF artifact
|
||||
matched the float32 oracle for all three GPU sequences with zero failures.
|
||||
Its signed backend detail records `ROCm0: Radeon 8060S Graphics` and measured
|
||||
`25/25` layer offload.
|
||||
|
||||
No conversion corruption was observed in that three-sequence ROCm sample. This
|
||||
does not prove global conversion correctness and does not retroactively change
|
||||
or explain the CPU result. A future v2 should predeclare a float32 quality oracle
|
||||
separately from its BF16 performance reference and use a larger corpus.
|
||||
|
||||
## Reproduction and bindings
|
||||
|
||||
- CPU report SHA-256: `{summary['cpu_v1']['report_sha256']}`
|
||||
- GPU report SHA-256: `{summary['rocm_diagnostic']['report_sha256']}`
|
||||
- BF16 GGUF SHA-256: `{summary['rocm_diagnostic']['candidate_artifact_sha256']}`
|
||||
- Signer fingerprint: `{signer_fingerprint}`
|
||||
- Exact verification command: see `commands.txt`.
|
||||
"""
|
||||
(ROOT / "quality-parity-diagnosis.md").write_text(md, encoding="utf-8")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
186
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md
Normal file
186
.scratch/distributed-gguf-runtime/evidence/DGR-003/README.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# DGR-003 — exact Artifact and runtime recipe identity
|
||||
|
||||
Evidence class: deterministic offline/unit. No model payload, GPU, external API,
|
||||
network node, or API credit is required or claimed.
|
||||
|
||||
## Result — delayed-review repair, 2026-07-14
|
||||
|
||||
DGR-003 defines and tests an exact, model-agnostic compatibility identity and
|
||||
connects it to DGR-002's gRPC `Fingerprint` plus tracker parsing, admission,
|
||||
route partitioning, and certification. It is **not complete**: the existing
|
||||
production doctor/backend path still emits the legacy capability report without
|
||||
constructing a `ShardIdentity` from authoritative loaded artifact/runtime state.
|
||||
No exact recipe is therefore claimed live or routable from that path; supplied
|
||||
exact identities remain dark until tracker-owned certification.
|
||||
|
||||
A matching digest proves canonical consistency, **not node authenticity or real
|
||||
execution**. Tracker-owned certification of a fingerprint by a non-synthetic,
|
||||
complete, multi-node distributed forward is the execution trust boundary.
|
||||
|
||||
## Implementation
|
||||
|
||||
- `ArtifactIdentity` binds artifact ID/revision, exact content digest,
|
||||
architecture/config digest, layer count, and optional derivative binding.
|
||||
- `DerivativeBinding` binds a split artifact to the exact source artifact digest
|
||||
and its end-exclusive layer range. A Shard cannot advertise outside that range.
|
||||
- `RuntimeRecipe` keeps these canonical axes separate rather than hiding them in
|
||||
a backend label:
|
||||
- weight quantization;
|
||||
- activation and compute dtypes;
|
||||
- KV dtype and layout;
|
||||
- tokenizer revision;
|
||||
- architecture adapter;
|
||||
- backend and runtime version;
|
||||
- boundary and protocol schema versions;
|
||||
- recipe ID/version and catalogue version.
|
||||
- `CompatibilityFingerprint` populates the existing DGR-002 Protobuf
|
||||
`Fingerprint`; `check_session_open()` fails closed on schema, fingerprint,
|
||||
advertised/effective range, non-empty route session, positive route epoch,
|
||||
and (when supplied) exact tracker route-session/epoch assignment.
|
||||
- Node and tracker implementations independently canonicalize the declaration.
|
||||
This is intentional: the tracker must not trust a digest copied from a node,
|
||||
and future native/C++ workers also need an independent implementation. Their
|
||||
behavior is pinned by `tests/data/recipe_fingerprint_vectors.json`.
|
||||
- Tracker admission cross-checks the exact identity against the capability
|
||||
proof's model, range, recipe labels, backend, and weight quantization. Any
|
||||
disagreement fails closed.
|
||||
- `TrackerServer` owns the sole live certification ledger and passes it through
|
||||
direct and replicated registration paths. A known exact recipe is
|
||||
`uncertified` and dark for user traffic until the same exact fingerprint is
|
||||
certified. Restart fails closed; durable/cluster-wide certification events
|
||||
require the later real-forward control path and are not claimed here.
|
||||
- Certification evidence is bound to the promoted fingerprint, requires at
|
||||
least two distinct nodes, complete layer coverage, generated tokens, and
|
||||
`synthetic=false`. Unknown or mismatched fingerprints cannot be promoted.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `packages/node/meshnet_node/runtime_recipe.py`
|
||||
- `packages/tracker/meshnet_tracker/recipe.py`
|
||||
- `packages/tracker/meshnet_tracker/capability.py`
|
||||
- `packages/tracker/meshnet_tracker/server.py`
|
||||
- `tests/data/recipe_fingerprint_vectors.json`
|
||||
- `tests/test_runtime_recipe_identity.py`
|
||||
- this evidence directory, issue state, and DGR-003 PRD state
|
||||
|
||||
A late review of dependency DGR-017 also found and fixed two genuine contract
|
||||
continuity defects during delayed DGR-003 review: v1 now has an independently
|
||||
trusted digest and recursively immutable parsed state. Those changes and tests
|
||||
are recorded in DGR-017 evidence rather than claimed as DGR-003 functionality.
|
||||
|
||||
## Verification
|
||||
|
||||
Exact commands and outcomes are in `commands.txt`.
|
||||
|
||||
Observed final results:
|
||||
|
||||
- DGR-003 identity + node/tracker capability suites: **126 passed**.
|
||||
- DGR-017 focused dependency repair suite: **99 passed**.
|
||||
- Tracker routing suite: **93 passed**.
|
||||
- First delayed-review integrated run: **898 passed, 13 skipped, 1 failed** on
|
||||
the pre-existing tracker-cancellation race.
|
||||
- Final delayed-review integrated rerun: **899 passed, 13 skipped** in
|
||||
**253.64s**; Hermes controller acceptance rerun: **899 passed, 13 skipped**
|
||||
in **252.66s**.
|
||||
- `python -m compileall -q packages tests`: pass.
|
||||
- `git diff --check`: pass.
|
||||
- Ruff on the changed identity, capability, contract, and test modules: pass.
|
||||
- `server.py` has 8 pre-existing Ruff findings at both pushed baseline and the
|
||||
current tree; DGR-003 added no finding.
|
||||
|
||||
The first integrated full-suite run produced **871 passed, 13 skipped, 1 failed**
|
||||
on the known unrelated
|
||||
`test_tracker_dashboard_can_cancel_inflight_proxy` timing race. Its fixture
|
||||
completed after three seconds just before cancellation, so the cancel endpoint
|
||||
returned 404. In this delayed repair it again produced a 404 after the stream
|
||||
finished (first integrated run: **898 passed, 13 skipped, 1 failed**); three
|
||||
immediate isolated repeats passed before a fourth reproduced the same race.
|
||||
No cancellation-test code was changed. The final complete integrated rerun
|
||||
passed **899/899** tests.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Certification state is process-local in this story. The same running tracker
|
||||
reuses it across registrations, but durable/cluster-wide certification-event
|
||||
persistence belongs with the later real distributed-forward control path.
|
||||
Restart or failover therefore returns exact recipes to the safe dark state;
|
||||
it never makes an unsupported recipe routable.
|
||||
- The node module has no certification ledger or admission policy; it holds only
|
||||
identity construction and handshake validation. The Tracker is the sole
|
||||
promotion authority.
|
||||
- **Completion blocker:** `doctor._validate_recipe()` calls
|
||||
`build_capability_report()` without `identity=`, because the legacy
|
||||
Transformers backend does not expose an immutable artifact-content pin and
|
||||
full runtime recipe axes authoritative enough to build one. Adding a guessed
|
||||
identity would weaken this contract. Production emission must be added with
|
||||
the authoritative native worker/backend loading seam; until then the issue and
|
||||
PRD deliberately remain incomplete.
|
||||
- This story proves identity and admission behavior with deterministic fixtures.
|
||||
It does not claim a real GLM forward or hardware certification.
|
||||
|
||||
## Compatibility
|
||||
|
||||
- Capability report identity is additive. Legacy reports without the new block
|
||||
retain ADR-0023's explicit compatibility-policy behavior.
|
||||
- Reports that opt into exact identity are held to it and fail closed on malformed,
|
||||
inconsistent, unknown, dark, or mismatched declarations.
|
||||
- No new wire identity was invented; DGR-002's `Fingerprint` remains the gRPC
|
||||
representation.
|
||||
|
||||
## Handoff
|
||||
|
||||
DGR-004 and native workers must build `ShardIdentity` from the actual immutable
|
||||
artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout,
|
||||
schema versions, and owned range. At `SessionOpen`, compare its
|
||||
`CompatibilityFingerprint` and return DGR-002's
|
||||
`ERROR_CODE_FINGERPRINT_MISMATCH` on any mismatch.
|
||||
|
||||
A digest match is not certification. Only tracker-recorded evidence from the
|
||||
same exact fingerprint and a real complete distributed forward can move that
|
||||
recipe out of dark status.
|
||||
|
||||
## Native emission closure — 2026-07-14
|
||||
|
||||
Status: **done**. DGR-004/DGR-005's native loaded-artifact seam now reaches the
|
||||
production capability-report path through `NativeWorkerBackendAdapter`.
|
||||
|
||||
### Files changed
|
||||
|
||||
- `packages/node/meshnet_node/native_backend.py` — immutable loaded-GGUF report,
|
||||
immutable artifact and numerical pins, exact identity derivation, and the
|
||||
SessionOpen boundary.
|
||||
- `packages/node/meshnet_node/doctor.py` — includes exact identity only for the
|
||||
native adapter and derives all matching capability-proof fields from it.
|
||||
- `tests/test_native_identity_emission.py` — deterministic native report,
|
||||
immutable-pin, SessionOpen, capability emission, legacy-dark, and
|
||||
tracker-uncertified tests.
|
||||
- This issue, `prd.json`, and this evidence directory.
|
||||
|
||||
### Correctness and trust boundary
|
||||
|
||||
The native report carries the end-exclusive owned range, mapped/resident/
|
||||
registered bytes, GGUF architecture metadata digest, and layer count. The
|
||||
adapter constructs `ShardIdentity` only from that report plus immutable artifact
|
||||
pin, tokenizer revision, and numerical recipe inputs. It does not accept a
|
||||
caller-supplied shard range.
|
||||
|
||||
`on_session_open()` calls `check_session_open()` before returning
|
||||
`SessionAccepted`, preserving fingerprint, schema, range, tracker-session, and
|
||||
epoch fail-closed behavior. The legacy Transformers backend is deliberately not
|
||||
an adapter and its doctor report remains identity-free.
|
||||
|
||||
The tracker evaluates a self-consistent native report as `uncertified`: digest
|
||||
equality is canonical consistency, not node authenticity. Only its owned
|
||||
certification ledger can promote a real distributed forward.
|
||||
|
||||
### Verification
|
||||
|
||||
- Focused/adversarial DGR-003, node/tracker capability, doctor, and native
|
||||
dependency suites: **171 passed, 1 skipped**.
|
||||
- Native protocol CMake configure/build plus CTest: **1/1 passed**.
|
||||
- `compileall`, Ruff, and `git diff --check`: pass.
|
||||
- Full deterministic suite: **902 passed, 13 skipped** (255.01s).
|
||||
|
||||
No model payload, GPU, external API, network node, or real distributed forward
|
||||
was run or claimed. The standalone gRPC process remains DGR-008 work; this
|
||||
story supplies its exact native identity and fail-closed SessionOpen contract.
|
||||
108
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt
Normal file
108
.scratch/distributed-gguf-runtime/evidence/DGR-003/commands.txt
Normal file
@@ -0,0 +1,108 @@
|
||||
# DGR-003 final verification — 2026-07-14
|
||||
|
||||
# Native emission closure — 2026-07-14
|
||||
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_native_identity_emission.py tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py tests/test_node_doctor.py tests/test_llama_cpp_dependency.py
|
||||
# result: 171 passed, 1 skipped in 7.07s
|
||||
|
||||
ruff check packages/node/meshnet_node/native_backend.py packages/node/meshnet_node/doctor.py tests/test_native_identity_emission.py
|
||||
# result: All checks passed
|
||||
|
||||
git diff --check
|
||||
# result: pass
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall packages tests
|
||||
# result: pass
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake -S packages/node/native -B build/dgr-003-native-protocol -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake --build build/dgr-003-native-protocol -j2
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/ctest --test-dir build/dgr-003-native-protocol --output-on-failure
|
||||
# result: configured and built shard_protocol_conformance; 1/1 CTest passed
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||
# result: 902 passed, 13 skipped in 255.01s
|
||||
|
||||
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||
# result: 99 passed in 4.76s
|
||||
|
||||
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
|
||||
# result: 99 passed in 0.15s
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||
# first integrated result: 871 passed, 13 skipped, 1 failed in 258.18s
|
||||
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
# fixture completed at ~3s before cancellation; cancel endpoint returned 404
|
||||
|
||||
for i in 1 2 3 4 5; do
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
done
|
||||
# result: 5/5 passed (1.14s, 1.14s, 1.26s, 1.14s, 1.64s)
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||
# final integrated result: 872 passed, 13 skipped in 253.46s
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
|
||||
# result: pass
|
||||
|
||||
git diff --check
|
||||
# result: pass
|
||||
|
||||
ruff check packages/node/meshnet_node/glm_alpha/contract.py packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_glm_alpha_target.py tests/test_runtime_recipe_identity.py
|
||||
# result: All checks passed!
|
||||
|
||||
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
|
||||
ruff check /tmp/dgr003-server-base.py
|
||||
ruff check packages/tracker/meshnet_tracker/server.py
|
||||
# result: both baseline and current server.py report the same 8 pre-existing findings
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Delayed-review repair continuation — 2026-07-14
|
||||
# No model payload, GPU, external API, or real inference was run.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PYTHONPATH=packages/node:packages/tracker:packages/contracts /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_runtime_recipe_identity.py tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||
# result: 126 passed in 4.77s
|
||||
# includes adversarial certification binding, unknown participant, mutation-atomicity,
|
||||
# report/identity revision+config, route partition, golden-vector, and SessionOpen tests
|
||||
|
||||
PYTHONPATH=packages/node:packages/tracker /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python scripts/gen_recipe_fingerprint_vectors.py --check
|
||||
# result: tests/data/recipe_fingerprint_vectors.json matches the identity implementation
|
||||
|
||||
PYTHONPATH=packages/node /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_glm_alpha_target.py
|
||||
# result: 99 passed in 0.11s
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py
|
||||
# result: 93 passed in 46.83s
|
||||
# there is no separate tests/test_tracker_server.py in this repository
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m compileall -q packages tests
|
||||
# result: pass
|
||||
|
||||
ruff check packages/node/meshnet_node/runtime_recipe.py packages/tracker/meshnet_tracker/recipe.py packages/tracker/meshnet_tracker/capability.py tests/test_runtime_recipe_identity.py scripts/gen_recipe_fingerprint_vectors.py
|
||||
# result: All checks passed!
|
||||
|
||||
git show e7c780a:packages/tracker/meshnet_tracker/server.py > /tmp/dgr003-server-base.py
|
||||
ruff check /tmp/dgr003-server-base.py
|
||||
ruff check packages/tracker/meshnet_tracker/server.py
|
||||
# result: baseline has 8 pre-existing findings; current has 7 because DGR-003 now
|
||||
# uses the previously unused STATE_ADMITTED import. No new server.py finding.
|
||||
|
||||
git diff --check
|
||||
# result: pass
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||
# result: 898 passed, 13 skipped, 1 failed in 255.43s
|
||||
# sole failure: tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
# the fixture completed its three-second stream before the cancel request, so cancel returned 404
|
||||
|
||||
for i in 1 2 3 4 5; do
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy || exit 1
|
||||
done
|
||||
# result: first 3 passed (1.16s, 1.65s, 1.64s); attempt 4 reproduced the same 404 race.
|
||||
# The test was not modified because it is outside the current DGR-003 P1 repair.
|
||||
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||
# result: 899 passed, 13 skipped in 253.64s (0:04:13)
|
||||
|
||||
# Hermes controller acceptance rerun after agent completion
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q
|
||||
# result: 899 passed, 13 skipped in 252.66s (0:04:12)
|
||||
@@ -0,0 +1,42 @@
|
||||
# DGR-004 verification blocker — 2026-07-14
|
||||
|
||||
## Verified state
|
||||
|
||||
The pre-existing DGR-004 boundary is present and its lock data is internally
|
||||
consistent:
|
||||
|
||||
- `scripts/llama_cpp_dependency.py inspect` reports pin
|
||||
`e920c523e3b8a0163fe498af5bf90df35ff51d25`, one patch, no model downloads,
|
||||
and no semantic certification.
|
||||
- The existing clean cached checkout at `build/dgr-004-final/source` is at the
|
||||
locked commit/tree and contains only the expected staged patch changes.
|
||||
- The existing `llama-gguf-hash --help` smoke binary runs successfully.
|
||||
- `python -m compileall -q packages tests`, Ruff on the DGR-004 Python files,
|
||||
and `git diff --check` pass.
|
||||
|
||||
## Blocker
|
||||
|
||||
The verification environment no longer contains the `.venv` recorded in
|
||||
`commands.txt`, nor a `cmake` executable on `PATH`. The available global
|
||||
pytest environment cannot import the native protocol because its protobuf
|
||||
runtime is 6.33.6 while the checked-in generated code requires 7.35.0. This
|
||||
causes both `tests/test_llama_cpp_dependency.py` and the native protocol suite
|
||||
to fail during the repository-wide autouse fixture setup, before their tests
|
||||
run.
|
||||
|
||||
This prevents the required fresh focused test and native CTest verification.
|
||||
No DGR-004 completion state, commit, or push is claimed from this worktree.
|
||||
|
||||
## Continuation
|
||||
|
||||
1. Restore the project test environment used by the prior evidence (including
|
||||
protobuf >= 7.35.0 and CMake), without changing DGR-004 source files.
|
||||
2. Run the exact focused test command from `commands.txt` and the clean
|
||||
`reproduce` command using the local llama.cpp object cache.
|
||||
3. Re-run compileall, Ruff, diff check, and the deterministic full suite.
|
||||
4. Only then apply the supervising engine's commit policy and unblock DGR-005.
|
||||
|
||||
## Dependency graph
|
||||
|
||||
`DGR-004 verification -> DGR-005 range-aware GGUF ownership -> DGR-003 live
|
||||
ShardIdentity emission`. DGR-005 and DGR-003-emission were not modified.
|
||||
31
.scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
Normal file
31
.scratch/distributed-gguf-runtime/evidence/DGR-004/README.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# DGR-004 — Reproducible pinned llama.cpp patch stack
|
||||
|
||||
Status: **done**. This is reproducible native-build infrastructure evidence, not model execution evidence.
|
||||
|
||||
## Delivered boundary
|
||||
|
||||
- Pin: `ggml-org/llama.cpp` at `e920c523e3b8a0163fe498af5bf90df35ff51d25` (tree `6c91a11407a3a3fb160f5dac705f9c59718f54f1`).
|
||||
- Ordered patch: `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch`, SHA-256 `1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75`.
|
||||
- The sole patch adds an interface-library CMake marker. It adds no model execution/loading, networking, Tracker, relay, gRPC, billing, or authentication code.
|
||||
- `scripts/llama_cpp_dependency.py` makes a fresh checkout, validates commit/tree/baseline blob, validates patch order/digests/context, applies the series, and verifies the exact resulting Git index tree. It rejects stale destinations, upstream drift, changed patches, untracked files, and local edits.
|
||||
|
||||
## Build and smoke result
|
||||
|
||||
The clean build cloned only the already-present exact Git object cache as a read-only source and did not trust its worktree. CMake 4.4.0 and GCC 15.2.1 built `llama-gguf-hash` with the locked Release/CPU flags in `UPSTREAM_LOCK.json`; `llama-gguf-hash --help` passed with no model download or load.
|
||||
|
||||
llama.cpp tests are intentionally off for this small no-model smoke target, so no upstream CTest applies. Meshnet's focused native protocol suite passed independently. Exact results are in `commands.txt` and `results.json`.
|
||||
|
||||
## License, compatibility, and handoff
|
||||
|
||||
llama.cpp is MIT licensed. The materializer requires upstream `LICENSE`, preserves all upstream notices, and `THIRD_PARTY_NOTICES.md` requires including them in redistribution. No Mesh-LLM code or patch was adopted.
|
||||
|
||||
The lock records the patched upstream blob and resulting patched tree. Pin updates must intentionally revise those values, the patch digest/order, toolchain metadata, and evidence.
|
||||
|
||||
This stock/native build is **infrastructure evidence only**: not a standalone Meshnet worker (DGR-008), GLM semantic acceptance, DSA/IndexShare proof, numerical equivalence, performance success, model-fit evidence, or route certification. The stock dense-MLA fallback remains explicitly uncertified. DGR-001 CPU v1 remains `stop`; DGR-017 is a separate target contract. DGR-005 may consume this dense-Llama structural boundary; DGR-018/DGR-019 must prove GLM semantics.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `packages/node/native/llama/*`
|
||||
- `scripts/llama_cpp_dependency.py`
|
||||
- `tests/test_llama_cpp_dependency.py`
|
||||
- this evidence directory, the DGR-004 issue, and `prd.json`
|
||||
@@ -0,0 +1,28 @@
|
||||
# DGR-004 commands and real results — 2026-07-14
|
||||
|
||||
```text
|
||||
$ .venv/bin/python -m pytest -q tests/test_llama_cpp_dependency.py tests/test_native_shard_protocol.py
|
||||
47 passed, 1 skipped in 0.59s
|
||||
|
||||
$ .venv/bin/python scripts/llama_cpp_dependency.py reproduce --work-dir build/dgr-004-smoke --source-repository /run/media/popov/d/DEV/llamacpp/llama.cpp
|
||||
llama-gguf-hash --help -> exit 0; output contains "Hash a GGUF file"
|
||||
|
||||
$ touch build/dgr-004-drift/source/DGR-004-local-edit
|
||||
$ .venv/bin/python scripts/llama_cpp_dependency.py apply --source-dir build/dgr-004-drift/source
|
||||
DGR-004 dependency error: local edits detected in materialized llama.cpp checkout
|
||||
exit 2
|
||||
|
||||
$ .venv/bin/python -m compileall -q packages tests
|
||||
exit 0
|
||||
|
||||
$ ruff check scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
|
||||
All checks passed!
|
||||
|
||||
$ git diff --check
|
||||
exit 0
|
||||
|
||||
$ .venv/bin/python -m pytest -q --cache-clear
|
||||
902 passed, 13 skipped in 255.01s (0:04:15)
|
||||
```
|
||||
|
||||
The source-cache command avoids transient network availability only. The script defaults to the public upstream URL and verifies the exact object/tree, not external worktree state.
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"evidence_class": "native build infrastructure",
|
||||
"llama_cpp": {
|
||||
"upstream": "https://github.com/ggml-org/llama.cpp.git",
|
||||
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
|
||||
"patched_tree": "4a37c06fac668834435b803caa59ba272bdace5c",
|
||||
"patch_sha256": "1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75"
|
||||
},
|
||||
"toolchain": {"cmake": "4.4.0", "cxx": "GCC 15.2.1", "generator": "Unix Makefiles", "target": "llama-gguf-hash", "configure_flags": ["-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_TESTS=OFF", "-DLLAMA_BUILD_EXAMPLES=ON", "-DLLAMA_BUILD_SERVER=OFF", "-DLLAMA_BUILD_TOOLS=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_CURL=OFF"]},
|
||||
"checks": {"clean_materialize_apply_build_smoke": "passed", "local_edit_detection": "passed (exit 2)", "focused_pytest": "47 passed, 1 skipped", "compileall": "passed", "ruff": "passed", "git_diff_check": "passed", "full_pytest": "902 passed, 13 skipped"},
|
||||
"model_downloads": false,
|
||||
"model_loaded": false,
|
||||
"inference_run": false,
|
||||
"glm_semantic_certification": false,
|
||||
"performance_certification": false,
|
||||
"route_certification": false
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
# DGR-005 decomposition — 2026-07-14
|
||||
|
||||
## Verified starting point
|
||||
|
||||
- The mandated environment is present: project Python 3.14.6, CMake 4.4.0,
|
||||
and protobuf 7.35.1.
|
||||
- DGR-003's focused identity/capability tests and DGR-004's dependency tests
|
||||
pass together: `95 passed`.
|
||||
- The DGR-004 materialized source at the pinned commit is available for source
|
||||
inspection. It contains only the DGR-004 CMake-marker patch.
|
||||
|
||||
## Why this chain cannot safely claim DGR-005 yet
|
||||
|
||||
At the locked llama.cpp revision, `llama_model_base::load_tensors()`:
|
||||
|
||||
1. sizes `layers` to `hparams.n_layer_all`;
|
||||
2. calls every architecture loader, which registers each architecture's layer
|
||||
tensors; and
|
||||
3. runs a generic optional-scale pass over the full layer count before creating
|
||||
mmap/backend buffers.
|
||||
|
||||
Filtering names after this point does not meet the ownership contract: it
|
||||
leaves full-model model/graph assumptions and can make a middle Shard silently
|
||||
look valid while it lacks the endpoint and boundary semantics needed by the
|
||||
next story. A generic `blk.N.*` filter alone is also not an architecture
|
||||
adapter, which violates ADR-0020's fail-closed dense-Llama-first rule.
|
||||
|
||||
## Required child slices
|
||||
|
||||
1. **DGR-005A — native dense-Llama ownership API and loader**
|
||||
- Add an explicit end-exclusive owned range to the project-owned native
|
||||
interface and validate it against immutable GGUF layer metadata.
|
||||
- Restrict registration, optional scales, allocation and mmap ranges to the
|
||||
owned `blk.N.*` tensors.
|
||||
- Record authoritative loaded start/end and mapped/resident byte counters
|
||||
from the instantiated model, not command-line input.
|
||||
- Add a deterministic synthetic dense-Llama GGUF fixture plus native tests
|
||||
for head, middle and tail ranges.
|
||||
|
||||
2. **DGR-005B — endpoint ownership and graph guard**
|
||||
- Load token embeddings only for the head, and final norm/output head only
|
||||
for the tail, including tied embeddings.
|
||||
- Make the dense-Llama graph fail closed when an endpoint-required tensor is
|
||||
absent; do not infer endpoint ownership from an empty pointer.
|
||||
- Prove that split ranges map fewer bytes than the whole-model fixture and
|
||||
that the loaded range report matches actual registered tensors.
|
||||
|
||||
3. **DGR-003-emission follow-up**
|
||||
- Expose the resulting immutable native loaded-artifact report to a native
|
||||
worker/backend adapter.
|
||||
- Construct `ShardIdentity` only from that report plus the immutable
|
||||
artifact, tokenizer and numerical-recipe inputs. The legacy Transformers
|
||||
doctor path must remain identity-free rather than fabricate a pin.
|
||||
- Wire `check_session_open()` at the worker SessionOpen boundary; current
|
||||
unit coverage already verifies its fail-closed fingerprint, range,
|
||||
session and epoch behavior.
|
||||
|
||||
## Handoff and non-claims
|
||||
|
||||
No DGR-005 source patch, identity-emission code, issue status, or `prd.json`
|
||||
pass state was changed. No model was loaded, downloaded, benchmarked, or
|
||||
certified. This document is a supervised-review handoff, not DGR-005 evidence
|
||||
of completion.
|
||||
79
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
79
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# DGR-005 — dense-Llama range-aware GGUF ownership
|
||||
|
||||
Evidence class: deterministic offline/unit (synthetic fixture) plus
|
||||
real-model integration (TinyLlama 1.1B, opt-in via MESHNET_ENABLE_REAL_INFERENCE_TESTS=1).
|
||||
|
||||
## Result
|
||||
|
||||
All six acceptance criteria pass:
|
||||
|
||||
1. **Range-aware tensor ownership**: native C++ patch (`0002-dense-llama-owned-range-loader.patch`,
|
||||
169 lines as merged — DGR-005A's original 365-line version was slimmed by DGR-005B)
|
||||
adds `llama_model_params.meshnet_owned_layer_start/end`, `llama_meshnet_range_report`,
|
||||
and restricts `blk.N.*` registration to the owned range.
|
||||
2. **Head/tail embedding loading**: head loads `token_embd.weight`; tail loads `output_norm`/`output`
|
||||
(with tied-embedding dedup). Middle shards load zero endpoint tensors.
|
||||
3. **Mapped/resident memory scales with owned tensors**: proven with TinyLlama 1.1B Q4_K_M.
|
||||
4. **Targeted pytest tests**: `tests/test_llama_cpp_dependency.py` (3 tests — lock/patch
|
||||
manifest consistency, offline dependency report, control-plane-code scan; re-verified
|
||||
2026-07-14: `3 passed, 6 skipped` together with the opt-in integration file), native CTest
|
||||
(`test-meshnet-range-ownership` synthetic fixture, added by the 0002 patch).
|
||||
5. **compileall, ruff, git diff --check, full pytest**: all pass.
|
||||
6. **Integration test**: `tests/test_gguf_distributed_load.py` (6/6, opt-in real model).
|
||||
|
||||
## Files changed (vs HEAD at DGR-004)
|
||||
|
||||
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch` — 169-line native patch (as merged)
|
||||
- `packages/node/native/llama/patches/SHA256SUMS` — updated hash
|
||||
- `packages/node/native/llama/patches/series` — added patch to series
|
||||
- `packages/node/native/llama/UPSTREAM_LOCK.json` — updated patched_tree, serial number
|
||||
- `scripts/llama_cpp_dependency.py` — `inspect` report for 2-patch stack
|
||||
- `tests/test_llama_cpp_dependency.py` — patch_count 2
|
||||
- `packages/node/native/llama/meshnet-range-loader.cpp` — C CLI wrapper
|
||||
- `tests/test_gguf_distributed_load.py` — real-model integration test
|
||||
|
||||
## Commands
|
||||
|
||||
```text
|
||||
# Build patched llama.cpp + range loader
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||
-S build/dgr-004-final/source -B build/dgr-004-final/build \
|
||||
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_EXAMPLES=ON \
|
||||
-DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=OFF \
|
||||
-DLLAMA_BUILD_TOOLS=ON -DLLAMA_CURL=OFF
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||
--build build/dgr-004-final/build --target llama-simple -j$(nproc)
|
||||
g++ -std=c++17 -Ibuild/dgr-004-final/source -Ibuild/dgr-004-final/source/include \
|
||||
-Ibuild/dgr-004-final/source/ggml/include -Lbuild/dgr-004-final/build/bin \
|
||||
packages/node/native/llama/meshnet-range-loader.cpp -lllama \
|
||||
-Wl,-rpath,build/dgr-004-final/build/bin \
|
||||
-o build/dgr-004-final/build/bin/meshnet-range-loader
|
||||
|
||||
# Focused tests (no model download)
|
||||
PYTHONPATH=packages/node:packages/tracker:packages/contracts
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||
-m pytest -q tests/test_llama_cpp_dependency.py
|
||||
|
||||
# Real-model integration test (opt-in, downloads ~670 MB)
|
||||
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=... \
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||
-m pytest -q tests/test_gguf_distributed_load.py
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Dense-Llama architecture only (LLM_ARCH_LLAMA). GLM/MoE/MLA is DGR-006+.
|
||||
- Graph-level endpoint assertions (`has_token_embeddings`, `has_output_head`) were
|
||||
simplified to only `start_layer`/`end_layer`/`mapped_bytes`/`resident_bytes` in
|
||||
the patch as merged. Full endpoint tracking is available via the integration test
|
||||
by observing which tensors are registered per shard.
|
||||
- Loading the full `llama-simple` CLI requires reconfiguring with `-DLLAMA_BUILD_EXAMPLES=ON`.
|
||||
The smoke-only build (`llama-gguf-hash`) is sufficient for patch verification.
|
||||
- TinyLlama 1.1B is a baseline dense-Llama architecture only.
|
||||
|
||||
## Commits
|
||||
|
||||
- `252d131` feat: DGR-005A dense Llama owned range loader
|
||||
- `f844ae6` feat: DGR-005B endpoint ownership and graph guard
|
||||
- `31065c0` feat: distributed GGUF shard load integration test with TinyLlama 1.1B
|
||||
- `d6b808d` chore: mark DGR-005 passes:true in PRD
|
||||
71
.scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
Normal file
71
.scratch/distributed-gguf-runtime/evidence/DGR-006/README.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# DGR-006 — architecture-defined boundary input/output
|
||||
|
||||
Status: complete deterministic/offline contract and dense-fixture evidence.
|
||||
|
||||
## Result
|
||||
|
||||
The native protocol now carries a versioned `TensorBundle` on the decode fast
|
||||
path. It includes explicit architecture and boundary-point metadata. Its legacy
|
||||
`NamedTensor` field remains a compact one-tensor encoding for certified dense
|
||||
boundaries; the writer deliberately selects it only for a one-tensor bundle and
|
||||
new readers wrap that representation into a bundle. The bundle is authoritative
|
||||
when present, allowing MoE/MLA sidebands without a second transport contract.
|
||||
|
||||
`architecture_boundary.py` is the fail-closed adapter boundary. Dense head
|
||||
Shards accept token IDs and own embedding. Middle/tail Shards accept only a
|
||||
validated bundle. Dense, MoE, and MLA route through explicit adapters; unknown
|
||||
architectures are rejected. The dense F32 fixture proves whole-model versus
|
||||
two-range boundary parity without model downloads or real inference.
|
||||
|
||||
Tail output is explicit in the schema: `TailResult` contains either logits or a
|
||||
sampled token and binds sampling parameters plus request ID, runtime recipe,
|
||||
chat template/version, reasoning mode, and architecture identity. The adapter
|
||||
builds and validates the serialized protobuf result before returning it.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `packages/node/native/proto/shard_runtime.proto`
|
||||
- `packages/node/meshnet_node/native_protocol/{codec.py,__init__.py,conformance.py,generated/*}`
|
||||
- `packages/node/native/testdata/decode_step_golden.binpb`
|
||||
- `packages/node/native/tests/test_shard_protocol_conformance.cpp`
|
||||
- `packages/node/meshnet_node/architecture_boundary.py`
|
||||
- `tests/test_architecture_boundary.py`
|
||||
- `tests/test_native_shard_protocol.py`
|
||||
- `packages/node/native/README.md`
|
||||
|
||||
## Commands and results
|
||||
|
||||
All Python commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python`.
|
||||
All native commands used `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake`.
|
||||
|
||||
```text
|
||||
python scripts/generate_native_protocol.py --check -> passed
|
||||
python scripts/generate_protocol_goldens.py --check -> passed
|
||||
pytest -q tests/test_architecture_boundary.py \
|
||||
tests/test_native_shard_protocol.py tests/test_llama_cpp_dependency.py
|
||||
-> 59 passed
|
||||
cmake -S packages/node/native -B build/native \
|
||||
-DCMAKE_PREFIX_PATH=/tmp/pbsrc/install -> configured
|
||||
cmake --build build/native -j$(nproc) -> built shard_protocol_conformance
|
||||
ctest --test-dir build/native --output-on-failure -> 1/1 passed
|
||||
python -m compileall -q packages tests -> passed
|
||||
git diff --check -> passed
|
||||
pytest -q -> 917 passed, 18 skipped
|
||||
```
|
||||
|
||||
## Compatibility and limitations
|
||||
|
||||
- Existing Nodes that send `DecodeStep.tensor` are accepted. New multi-tensor
|
||||
Nodes require the versioned bundle and older Nodes safely preserve it as an
|
||||
unknown field rather than interpreting it as a single tensor.
|
||||
- The committed C++ conformance vector covers the multi-tensor decode path.
|
||||
- The dense parity result is a deterministic F32 structural fixture, not real
|
||||
GGUF inference or GLM certification. No real inference was run.
|
||||
- MoE and MLA adapters define and validate their sideband contracts but are not
|
||||
architecture certifications. DGR-019 owns GLM MoE/MLA/DSA/IndexShare semantics.
|
||||
|
||||
## Handoff
|
||||
|
||||
DGR-007 can key its Hot KV state to the validated decoded bundle. DGR-008 can
|
||||
translate the generated `TailResult` and decode bundle over gRPC. DGR-019 must
|
||||
replace the generic MoE/MLA sideband names with exact certified GLM semantics.
|
||||
275
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
Normal file
275
.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md
Normal file
@@ -0,0 +1,275 @@
|
||||
# DGR-017 — Lock the GLM-5.2 Max target and alpha contract
|
||||
|
||||
Status: **done**. Every acceptance criterion is met with real command output.
|
||||
|
||||
Evidence class: **real upstream metadata + deterministic arithmetic**. No weight
|
||||
payload was downloaded, no model was loaded, no GPU was used, and no benchmark was
|
||||
run — and none is claimed. This story makes the target *reviewable before* the
|
||||
216.7 GB download, which is exactly its job.
|
||||
|
||||
## 1. Summary
|
||||
|
||||
The alpha target is now pinned, planned, and sealed:
|
||||
|
||||
- **Identity.** `zai-org/GLM-5.2` @ `b4734de4facf877f85769a911abafc5283eab3d9` and
|
||||
`unsloth/GLM-5.2-GGUF` @ `abc55e72527792c6e77069c99b4cb7de16fa9f23`, quantization
|
||||
`UD-IQ1_S`, six shards, 216,715,360,960 bytes, every shard's LFS SHA-256 resolved.
|
||||
- **Architecture.** The config/tokenizer/chat-template metadata the runtime cannot
|
||||
shard without, hashed at the pinned revision.
|
||||
- **Resources.** A deterministic planner that counts unified memory once, applies the
|
||||
`max(20% , 8 GiB)` reserve, and reports the arithmetic minimum and the recommended
|
||||
node count as two different numbers.
|
||||
- **Contract.** The roadmap's section-5 acceptance matrix as a machine-readable,
|
||||
digest-sealed document, locked before the target ever runs and cross-bound to the
|
||||
exact manifest and architecture-snapshot digests.
|
||||
- **Upstream.** A refreshed llama.cpp/donor status report.
|
||||
|
||||
Three findings are worth a reader's attention.
|
||||
|
||||
**Every number in the roadmap reproduced from primary sources.** The 216,715,360,960
|
||||
byte total, the 201.832 GiB figure, the whole KV table (0.73 / 0.77 / 0.89 / 1.68 GiB
|
||||
at 16K, through 46.62 / 49.41 / 56.98 / 107.25 GiB at 1M), and the whole tier table
|
||||
(9 / 6 / 4 / 3 / 2 arithmetic minimum nodes) fall out of the exact config and the
|
||||
exact shard bytes. The roadmap was not approximating. The planner is written as a
|
||||
*reproduction* of those tables, so if the arithmetic ever stops matching, a test says
|
||||
which numbers moved.
|
||||
|
||||
**The roadmap's "recommended" column is an imbalance factor of exactly 1.10.** Nodes
|
||||
= `ceil(total x 1.10 / budget)` yields 10 / 6 / 5 / 3 / 3 for the 32 / 48 / 64 / 96 /
|
||||
128 GiB tiers — precisely the roadmap's recommendations. That constant is now named
|
||||
(`PLACEMENT_IMBALANCE_FACTOR`) and documented as a placeholder for measured
|
||||
per-tensor placement, not a fudge factor to be tuned once results are in.
|
||||
|
||||
**224 GiB aggregate does not actually fit.** Two 112 GiB nodes hit the 224 GiB
|
||||
"hard-fit floor" exactly, and still come up **23.5 GiB short** once each node honours
|
||||
its reserve. That is what makes 224 GiB an *experimental floor* rather than an
|
||||
envelope, and it is now a test, not a caveat in prose. Relatedly, the 2×128 and 4×64
|
||||
"fit probe" topologies fit with only **2.08 GiB of headroom across the entire route** —
|
||||
which is why they require measured placement evidence and are not the recommendation.
|
||||
|
||||
## 2. Files changed
|
||||
|
||||
New — runtime-loadable package (single source of truth):
|
||||
|
||||
| Path | What |
|
||||
|---|---|
|
||||
| `packages/node/meshnet_node/glm_alpha/__init__.py` | Public surface |
|
||||
| `packages/node/meshnet_node/glm_alpha/manifest.py` | Target manifest + architecture snapshot; fail-closed identity |
|
||||
| `packages/node/meshnet_node/glm_alpha/planner.py` | Memory / KV / seam planner; unified-memory de-duplication |
|
||||
| `packages/node/meshnet_node/glm_alpha/contract.py` | Immutable, digest-sealed alpha acceptance contract |
|
||||
| `packages/node/meshnet_node/glm_alpha/data/target-manifest.json` | The six pinned shards, sizes, SHA-256, URLs, licenses |
|
||||
| `packages/node/meshnet_node/glm_alpha/data/architecture-snapshot.json` | Pinned architecture + config/template hashes |
|
||||
| `packages/node/meshnet_node/glm_alpha/data/alpha-contract.json` | Sealed acceptance thresholds (`aab23220…`) |
|
||||
| `scripts/refresh_glm_target_manifest.py` | Re-resolve/verify pins from upstream metadata (`--check` / `--write`) |
|
||||
| `tests/test_glm_alpha_target.py` | 97 deterministic offline tests (99 after the late-review repair — see §4a) |
|
||||
|
||||
New — evidence:
|
||||
|
||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md` (this file)
|
||||
- `.../commands.txt` — exact commands and real results
|
||||
- `.../resource-plan.json` — generated tier/route/seam/KV plan
|
||||
- `.../upstream-status.json` — refreshed llama.cpp and donor status
|
||||
|
||||
Modified:
|
||||
|
||||
- `.scratch/distributed-gguf-runtime/issues/17-...md` — `Status: done`
|
||||
- `.scratch/distributed-gguf-runtime/prd.json` — DGR-017 `passes: true` (this story only)
|
||||
- `.ralph-tui/progress.md` — learnings
|
||||
|
||||
The data files live **in the package**, not in evidence, because the runtime must load
|
||||
them (DGR-018 verifies downloads against these digests; DGR-003 folds the manifest
|
||||
digest into the recipe fingerprint). Duplicating them into evidence would create two
|
||||
sources of truth that could drift.
|
||||
|
||||
## 3. Acceptance criteria
|
||||
|
||||
| Criterion | Where it is proven |
|
||||
|---|---|
|
||||
| Pin both repos by exact observed revision; `UD-IQ1_S` is the alpha quant | `target-manifest.json`; `test_manifest_pins_both_repositories_by_exact_revision` |
|
||||
| Six filenames, exact bytes, LFS SHA-256, aggregate GB/GiB, license, URLs, no payload download | HF `paths-info` API (LFS pointer metadata); `test_manifest_resolves_all_six_shards…`, `test_manifest_aggregate_bytes_are_exact_and_self_consistent` |
|
||||
| Snapshot + hash architecture-critical config/tokenizer/chat-template metadata | `architecture-snapshot.json`; `test_snapshot_captures_the_architecture_critical_metadata`, `test_snapshot_hashes_the_config_and_chat_template_bytes` |
|
||||
| Deterministic minimum-node calc from exact bytes, Q8_0 KV @16K/c1, imbalance, reserve | `planner.plan_topology`; `test_topology_planner_reproduces_the_published_tier_table` |
|
||||
| 224 GiB is a hard-fit floor, not an envelope; recommend 5×64 or 3×96/128 | `test_224_gib_aggregate_is_a_hard_fit_floor_not_an_operational_envelope`, `test_the_recommended_topologies_are_five_by_64_or_three_by_96_or_128` |
|
||||
| Unified memory counted once; additive RAM+VRAM rejected | `NodeMemory.from_host`; `test_adding_integrated_gpu_memory_to_system_ram_is_rejected`, `test_unified_memory_is_counted_once` |
|
||||
| 2.5 GbE minimum / 10 GbE recommended; serial latency modelled apart from bandwidth | `planner.plan_seams`; `test_2_5_gbe_is_the_alpha_minimum_and_10_gbe_is_recommended`, `test_serial_seam_latency_is_modelled_separately_from_bandwidth` |
|
||||
| Identity/semantic/target-run/performance/reliability/storage criteria locked before execution | `alpha-contract.json` (sealed, `locked_before_target_execution: true`); `test_the_contract_locks_every_roadmap_acceptance_section` |
|
||||
| Refresh upstream llama.cpp + donor status; no broad fork/scheduler | `upstream-status.json`; `adoption_state: none adopted` |
|
||||
| Tests reject changed revisions, missing shards, coordinated digest/config substitutions, inconsistent bytes, duplicate unified memory, malformed telemetry, and post-result threshold mutation | 97 tests; see §5 |
|
||||
| Targeted pytest passes | `97 passed` |
|
||||
| Installed wheel includes and loads all locked JSON resources | Real wheel build/install plus `load_locked_target()` outside the source tree |
|
||||
| `compileall packages tests` | exit 0 |
|
||||
| `git diff --check` | exit 0 |
|
||||
| Default tests deterministic, download-free, credit-free, GPU-free | Pure JSON + arithmetic; the only network code is an opt-in script excluded from the suite |
|
||||
| Full deterministic `pytest -q` | **852 passed, 13 skipped** on final rerun |
|
||||
|
||||
## 4. Real results
|
||||
|
||||
```
|
||||
scripts/refresh_glm_target_manifest.py --check -> match upstream (exit 0)
|
||||
pytest -q tests/test_glm_alpha_target.py -> 97 passed
|
||||
build + install wheel; load_locked_target() -> INSTALLED_WHEEL_PASS
|
||||
compileall -q packages tests -> exit 0
|
||||
git diff --check -> exit 0
|
||||
pytest -q -> 852 passed, 13 skipped (253.30s)
|
||||
```
|
||||
|
||||
Controller review found and fixed four gaps before the commit was accepted:
|
||||
|
||||
1. the initial wheel omitted `glm_alpha/data/*.json`, despite source-tree tests passing;
|
||||
2. the self-sealed contract did not bind the manifest/snapshot digests, so coordinated
|
||||
shard-hash or architecture substitutions could be accepted;
|
||||
3. `--check` followed moving repository HEAD instead of validating immutable pins;
|
||||
4. non-finite resource telemetry could bypass ordinary range checks.
|
||||
|
||||
Each failure now has either an adversarial regression test or a real installed-wheel /
|
||||
live-metadata gate. None of the provisional agent results were accepted on trust.
|
||||
|
||||
The intermittent tracker cancellation race that DGR-001 and DGR-002 both recorded as
|
||||
flaky on a clean tree failed once in the final validation (`404` after the request
|
||||
completed), then passed **5/5** in isolation and passed in the integrated full-suite
|
||||
rerun above. This story touches no tracker code; the failed run is retained in
|
||||
`commands.txt` rather than hidden.
|
||||
|
||||
### 4a. Late independent-review repair (2026-07-14)
|
||||
|
||||
During delayed DGR-003 review, two contract-continuity defects were found and
|
||||
fixed here: v1 now has an independently trusted digest pinned in code
|
||||
(`test_resealing_a_mutated_v1_contract_is_rejected`) and parsed nested contract
|
||||
state is recursively immutable. This added two tests; the suite is now
|
||||
**99 passed** (`commands.txt` §7 records the exact runs). All "97" figures
|
||||
elsewhere in this README describe the suite at original completion.
|
||||
|
||||
Planner output (`resource-plan.json`):
|
||||
|
||||
| Route | Fits | Headroom |
|
||||
|---|---|---:|
|
||||
| 5×64 GiB unified (recommended) | yes | +53.28 GiB |
|
||||
| 3×96 GiB unified (recommended) | yes | +27.68 GiB |
|
||||
| 3×128 GiB unified (recommended) | yes | +104.48 GiB |
|
||||
| 4×64 GiB (fit probe) | yes | **+2.08 GiB** |
|
||||
| 2×128 GiB (fit probe) | yes | **+2.08 GiB** |
|
||||
| 2×112 GiB (= 224 GiB floor) | **no** | −23.52 GiB |
|
||||
| 3×64 GiB | no | −49.12 GiB |
|
||||
|
||||
## 5. How the "no silent swap" claim is earned
|
||||
|
||||
The story's whole purpose is to stop a later agent from changing the target after
|
||||
seeing a result. Each of those moves now has a test that names it:
|
||||
|
||||
- swap the artifact → `test_a_changed_gguf_revision_is_rejected`
|
||||
- pin a branch instead of a commit → `test_a_branch_name_is_not_an_acceptable_revision_pin`
|
||||
- drop a shard → `test_a_missing_shard_is_rejected`
|
||||
- shrink a shard so it "fits" → `test_a_shard_size_edited_to_make_the_model_look_smaller_is_rejected`
|
||||
- change a valid-looking shard SHA without changing bytes → `test_coordinated_shard_hash_substitution_is_rejected_by_contract`
|
||||
- change internally consistent architecture metadata → `test_internally_consistent_architecture_substitution_is_rejected_by_contract`
|
||||
- take the bigger quant quietly → `test_swapping_in_a_different_quantization_is_rejected`
|
||||
- add iGPU "VRAM" to system RAM → `test_adding_integrated_gpu_memory_to_system_ram_is_rejected`
|
||||
- count one machine twice → `test_the_same_machine_counted_twice_in_a_route_is_rejected`
|
||||
- lower the speed floor → `test_lowering_the_speed_floor_after_seeing_a_result_is_rejected`
|
||||
- call a slow pass an alpha → `test_relabelling_a_speed_failure_as_a_pass_is_rejected`
|
||||
- admit the dense fallback → `test_admitting_the_dense_attention_fallback_after_the_fact_is_rejected`
|
||||
- shrink the reserve → `test_relaxing_the_per_node_reserve_after_the_fact_is_rejected`
|
||||
|
||||
**Contract continuity is fail-closed.** The document’s `contract_sha256` detects
|
||||
accidental edits, and the approved v1 digest is pinned independently in code. A caller
|
||||
that changes a threshold and re-seals it under `glm-5.2-max-alpha/v1` is rejected by
|
||||
`test_resealing_a_mutated_v1_contract_is_rejected`; amendments require a new supported
|
||||
contract identity under human review. Parsed nested state is recursively immutable,
|
||||
so thresholds cannot change between validation and use; `to_dict()` returns an isolated
|
||||
copy rather than exposing the validated object.
|
||||
|
||||
## 6. Upstream status — the gating risk for DGR-004/DGR-018
|
||||
|
||||
Refreshed against live GitHub on 2026-07-13. One item **changed** since the roadmap:
|
||||
|
||||
- **#24231 is now MERGED** (2026-07-11) — a generic `GGML_OP_LIGHTNING_INDEXER` exists.
|
||||
- #24770 MERGED (2026-06-20) — GLM-5.2 loads via a **dense-MLA compatibility path**.
|
||||
- **#25407 still OPEN** (updated today) — the real GLM DSA/IndexShare wiring.
|
||||
- #24730 still OPEN — the umbrella GLM-5.2 support request.
|
||||
|
||||
**No released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare today.** A
|
||||
stock pin taken now would load the artifact and emit text through the dense fallback —
|
||||
which the alpha contract explicitly refuses (`dense_attention_fallback_satisfies_alpha:
|
||||
false`). DGR-018 must therefore prove those paths are *active*, not that output appeared.
|
||||
|
||||
Donor policy holds, and the evidence now supports it more strongly than before: PR
|
||||
#25407 is **12 files, +414/−7**. The semantics alpha needs are small enough to track
|
||||
and reproduce upstream. That is the argument against adopting Mesh-LLM's 261-patch
|
||||
fork — recorded as a donor (Apache-2.0, branch head `9bd18f15`, 2026-07-12), nothing
|
||||
adopted here.
|
||||
|
||||
## 7. Limitations and deferred work
|
||||
|
||||
- **No artifact was downloaded or loaded.** Sizes and SHA-256 come from Hugging Face
|
||||
LFS pointer metadata. DGR-018 must verify the digests against the real files on
|
||||
mounted storage before route admission. A matching size with a wrong hash is exactly
|
||||
the failure this manifest exists to catch, and only a local verify can catch it.
|
||||
- **`PLACEMENT_IMBALANCE_FACTOR = 1.10` is a planning assumption, not a measurement.**
|
||||
It reproduces the roadmap's recommendations, but the real per-node share depends on
|
||||
exact tensor bytes (embeddings, output head, dense vs MoE layers, shared experts,
|
||||
indexer tensors, quant block alignment). DGR-019 must replace it with measured
|
||||
placement. Until then, arithmetic-minimum topologies (2×128, 4×64) stay fit probes.
|
||||
- **KV numbers are planning estimates, not admission truth.** The planner deliberately
|
||||
budgets the *conservative* indexer layout (keys across all 78 layers, not just the 21
|
||||
Full ones) so a route admitted here cannot be surprised by the implementation it
|
||||
actually gets. The runtime must still report measured allocated/resident MLA and
|
||||
indexer cache per shard.
|
||||
- **Peak scratch is unmodelled.** The reserve exists precisely because backend
|
||||
workspaces and graph scratch are not predictable from the artifact; measured peak
|
||||
must land inside the reserve, and the contract requires that evidence.
|
||||
- **Upstream is moving fast.** #25407 was updated the same day it was observed. Refresh
|
||||
`upstream-status.json` before DGR-004 picks a llama.cpp pin. The manifest script's
|
||||
`--check` deliberately validates the immutable Hugging Face pins, not moving HEAD.
|
||||
|
||||
## 8. Compatibility and migration notes
|
||||
|
||||
- Purely additive. No existing module, wire format, or test changed. Nothing in this
|
||||
story is on a live request path.
|
||||
- `meshnet_node.glm_alpha` has **no heavy imports** — no torch, no transformers, no
|
||||
network at import time — so a tracker or planner can read the target contract without
|
||||
paying for a model runtime.
|
||||
- Re-pinning is deliberately awkward: `--write` follows current HEAD and leaves the
|
||||
existing contract binding invalid until a new contract is reviewed and sealed.
|
||||
`--check` uses revision-specific APIs and exits non-zero rather than healing any
|
||||
integrity drift in the already locked target.
|
||||
|
||||
## 9. Handoff to dependent stories
|
||||
|
||||
**DGR-003 (recipe identity):** fold `TargetManifest.digest`
|
||||
(`0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`) and
|
||||
`ArchitectureSnapshot.digest` (`253fbd94…`) into the runtime recipe fingerprint. The
|
||||
GLM fields the roadmap asks you to add (DSA/IndexShare metadata, context max, expert
|
||||
counts) are already resolved in `architecture-snapshot.json` — read them, do not
|
||||
re-derive them by hand. Populate the DGR-002 `Fingerprint` message; do not invent a
|
||||
second identity struct.
|
||||
|
||||
**DGR-004 (llama.cpp pin):** read `upstream-status.json` first. Any pin taken before
|
||||
#25407 merges gives you the dense-MLA fallback, which cannot satisfy alpha. Track
|
||||
#25407 (12 files) as a numbered patch; do not adopt the Mesh-LLM fork.
|
||||
|
||||
**DGR-018 (whole-model oracle):** the six digests in `target-manifest.json` are what you
|
||||
verify the download against. Your host needs ≥224 GiB runtime-accessible memory — and
|
||||
note that 224 GiB is the *floor*, not a comfortable target (see §1). Prove DSA,
|
||||
IndexShare, shared expert, and the Max template are **active**; the contract's
|
||||
`require_rendered_reasoning_effort_marker` is `<|system|>Reasoning Effort: Max`. Assert
|
||||
the rendered marker, not the request field: the template's only non-max level is
|
||||
`'high'`, so *every other value — including an absent one — renders Max*, and "the
|
||||
request said max" proves nothing.
|
||||
|
||||
**DGR-019 (GLM semantics):** the IndexShare split is **21 Full producer layers and 57
|
||||
Shared consumers**, in a `[full, full, full] + repeating [shared, shared, shared, full]`
|
||||
pattern. Prefer shard boundaries that keep an ownership group whole; the 8 KiB
|
||||
(2048 × int32) top-k sideband is the cost when you cannot. Replace
|
||||
`PLACEMENT_IMBALANCE_FACTOR` with measured per-tensor placement.
|
||||
|
||||
**DGR-020 (alpha verdict):** load the target with `load_locked_target()`. It verifies
|
||||
the contract seal and cross-binds the manifest and architecture snapshot before
|
||||
returning them. Judge against
|
||||
`contract.threshold(section, key)` — an unlocked threshold raises rather than
|
||||
defaulting, so a criterion cannot be invented at read time. The verdict is `alpha` or
|
||||
`stop`; there is no third outcome, and a quality pass with a speed failure is `stop`.
|
||||
|
||||
**Everyone:** unified system RAM and integrated-GPU memory are one pool. Build nodes
|
||||
with `NodeMemory.from_host(..., unified=True)` and it is impossible to write the
|
||||
double-count; pass a GPU size alongside `unified=True` and it raises rather than
|
||||
silently ignoring the argument.
|
||||
115
.scratch/distributed-gguf-runtime/evidence/DGR-017/commands.txt
Normal file
115
.scratch/distributed-gguf-runtime/evidence/DGR-017/commands.txt
Normal file
@@ -0,0 +1,115 @@
|
||||
# DGR-017 — exact commands and real results (2026-07-13)
|
||||
# Project venv is used explicitly. NOTE: bare `pytest` on this machine resolves to
|
||||
# Hermes Agent's internal venv (/home/popov/.hermes/...), which DGR-001 already
|
||||
# recorded as the cause of a bogus "suite is blocked" claim. Always use $VP.
|
||||
VP=/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python # Python 3.14.6
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Resolve the target from upstream metadata ONLY. No weight payload downloaded.
|
||||
# Sizes and SHA-256 come from the HF LFS pointer metadata (paths-info), not the blobs.
|
||||
# ---------------------------------------------------------------------------
|
||||
curl -sS "https://huggingface.co/api/models/zai-org/GLM-5.2"
|
||||
-> sha b4734de4facf877f85769a911abafc5283eab3d9 (matches the roadmap pin)
|
||||
-> license mit, lastModified 2026-07-02T08:08:14.000Z
|
||||
|
||||
curl -sS "https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF"
|
||||
-> sha abc55e72527792c6e77069c99b4cb7de16fa9f23 (matches the roadmap pin)
|
||||
-> license mit, lastModified 2026-06-23T15:18:23.000Z
|
||||
-> six UD-IQ1_S shards present
|
||||
|
||||
curl -sS -X POST -d '{"paths": [<6 UD-IQ1_S shards>]}' \
|
||||
"https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF/paths-info/abc55e7..."
|
||||
-> all six shards resolved with exact size + LFS oid (sha256)
|
||||
-> sum = 216,715,360,960 bytes = 201.832 GiB = 216.715 GB
|
||||
-> matches the roadmap's published byte total EXACTLY
|
||||
-> UD-IQ1_M fallback = 228,492,966,624 bytes = 212.801 GiB (also matches)
|
||||
|
||||
curl -sS ".../resolve/b4734de4.../{config.json,chat_template.jinja,
|
||||
generation_config.json,tokenizer_config.json}"
|
||||
-> config.json 3732 B sha256 185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a
|
||||
-> chat_template.jinja 5076 B sha256 172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679
|
||||
-> generation_config.json 194 B sha256 ac76b43d8683d3b930126870fc8be73d8679308fe752fa1f381096d8354f6a55
|
||||
-> tokenizer_config.json 761 B sha256 98b1271574f41abf89427ae2dda030d94dc9478f0edc5a8bd240db213c6fd5fc
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Verify the checked-in pins still match live upstream (reproducible, no weights)
|
||||
# ---------------------------------------------------------------------------
|
||||
$VP scripts/refresh_glm_target_manifest.py --check
|
||||
-> "target manifest and architecture snapshot match upstream"
|
||||
-> exit 0
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Upstream llama.cpp / donor status refresh (GitHub REST API, read-only)
|
||||
# ---------------------------------------------------------------------------
|
||||
curl -sS "https://api.github.com/repos/ggml-org/llama.cpp/issues/{24730,24770,25407,24231}"
|
||||
-> #24730 issue OPEN "Feature Request: Support for GLM 5.2"
|
||||
-> #24770 PR MERGED 2026-06-20 dense-MLA compatibility loader (DSA tensors optional)
|
||||
-> #24231 PR MERGED 2026-07-11 generic GGML_OP_LIGHTNING_INDEXER [CHANGED since roadmap]
|
||||
-> #25407 PR OPEN updated 2026-07-13, non-draft, 12 files, +414/-7 GLM 5.2 Indexer support
|
||||
curl -sS "https://api.github.com/repos/Mesh-LLM/mesh-llm{,/branches/feat%2Fjianyang-glm-52}"
|
||||
-> Apache-2.0, 2048 stars, branch head 9bd18f1509dff7fac21578635084035b3ba90a38 (2026-07-12)
|
||||
-> recorded as donor only; nothing forked, nothing adopted
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Seal the alpha contract (digest over its own canonical content)
|
||||
# ---------------------------------------------------------------------------
|
||||
$VP -c "seal_contract(...)" -> contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Generate the machine-readable resource plan from the pinned artifact
|
||||
# ---------------------------------------------------------------------------
|
||||
PYTHONPATH=packages/node $VP <generate resource-plan.json>
|
||||
-> manifest_sha256 0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962
|
||||
-> architecture_sha256 253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1
|
||||
-> alpha_contract_sha256 aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9
|
||||
-> tier arithmetic minimum 32:9 48:6 64:4 96:3 128:2 (reproduces the roadmap table)
|
||||
-> tier recommended 32:10 48:6 64:5 96:3 128:3 (reproduces the roadmap table)
|
||||
-> 5x64 GiB unified fits, +53.28 GiB headroom
|
||||
-> 3x96 GiB unified fits, +27.68 GiB headroom
|
||||
-> 2x128 / 4x64 (fit probes) fit with only +2.08 GiB headroom across the WHOLE route
|
||||
-> 2x112 GiB (= 224 GiB, the hard-fit floor) DOES NOT FIT: -23.52 GiB
|
||||
-> 3x64 GiB does not fit: -49.12 GiB
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. Quality gates (project .venv, deterministic, offline, GPU-free)
|
||||
# ---------------------------------------------------------------------------
|
||||
$VP -m pytest -q tests/test_glm_alpha_target.py
|
||||
-> 97 passed in 0.12s
|
||||
-> includes coordinated shard/config substitution, malformed telemetry, and
|
||||
contract-ID reseal rejection tests added during controller review
|
||||
|
||||
$VP -m pip wheel --no-deps packages/node -w /tmp/dgr017-wheel
|
||||
$VP -m pip install --no-deps --target /tmp/dgr017-install /tmp/dgr017-wheel/*.whl
|
||||
$VP -I -c "... from meshnet_node.glm_alpha import load_locked_target ..."
|
||||
-> INSTALLED_WHEEL_PASS
|
||||
-> packaged alpha-contract.json, target-manifest.json, and architecture-snapshot.json
|
||||
load and cross-bind successfully outside the source tree
|
||||
|
||||
$VP -m compileall -q packages tests
|
||||
-> exit 0
|
||||
|
||||
git diff --check
|
||||
-> exit 0
|
||||
|
||||
$VP -m pytest -q # full deterministic suite
|
||||
-> first final run: 1 failed, 851 passed, 13 skipped; only the tracker cancellation
|
||||
race already documented by DGR-001/DGR-002 failed
|
||||
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
-> 1 passed, repeated 5/5 in isolation
|
||||
$VP -m pytest -q # integrated rerun
|
||||
-> 852 passed, 13 skipped in 253.30s (0:04:13)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. Late independent-review repair (2026-07-14)
|
||||
# ---------------------------------------------------------------------------
|
||||
PYTHONPATH=packages/node $VP -m pytest -q tests/test_glm_alpha_target.py
|
||||
-> 99 passed in 0.15s
|
||||
-> adds trusted-v1-digest rejection after coordinated mutation + reseal
|
||||
-> adds nested parsed-state immutability and isolated to_dict() coverage
|
||||
|
||||
$VP -m pytest -q # after DGR-003 integration and DGR-017 repair
|
||||
-> first run: 871 passed, 13 skipped, 1 known cancellation-race failure
|
||||
$VP -m pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
-> 5/5 passed in isolation
|
||||
$VP -m pytest -q # integrated rerun
|
||||
-> 872 passed, 13 skipped in 253.46s (0:04:13)
|
||||
@@ -0,0 +1,255 @@
|
||||
{
|
||||
"generated_by": "DGR-017 meshnet_node.glm_alpha.planner",
|
||||
"target": {
|
||||
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||
"quantization": "UD-IQ1_S",
|
||||
"total_bytes": 216715360960,
|
||||
"total_gib": 201.832,
|
||||
"total_gb": 216.715
|
||||
},
|
||||
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962",
|
||||
"architecture_snapshot_sha256": "253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1",
|
||||
"alpha_contract_sha256": "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9",
|
||||
"kv_assumptions": {
|
||||
"dtype": "Q8_0",
|
||||
"bytes_per_value": 1.0625,
|
||||
"context_tokens": 16384,
|
||||
"concurrency": 1,
|
||||
"indexer_layout": "conservative",
|
||||
"mla_values_per_token_per_layer": 576,
|
||||
"backbone_layers": 78,
|
||||
"indexer_full_layers": 21,
|
||||
"note": "Alpha budgets indexer keys across all 78 layers (current experimental DSA layout), not only the 21 Full layers."
|
||||
},
|
||||
"kv_table_gib": {
|
||||
"16384": {
|
||||
"mla_only_q8_gib": 0.73,
|
||||
"optimized_dsa_q8_gib": 0.77,
|
||||
"conservative_dsa_q8_gib": 0.89,
|
||||
"conservative_dsa_f16_gib": 1.68
|
||||
},
|
||||
"131072": {
|
||||
"mla_only_q8_gib": 5.83,
|
||||
"optimized_dsa_q8_gib": 6.18,
|
||||
"conservative_dsa_q8_gib": 7.12,
|
||||
"conservative_dsa_f16_gib": 13.41
|
||||
},
|
||||
"1048576": {
|
||||
"mla_only_q8_gib": 46.62,
|
||||
"optimized_dsa_q8_gib": 49.41,
|
||||
"conservative_dsa_q8_gib": 56.98,
|
||||
"conservative_dsa_f16_gib": 107.25
|
||||
}
|
||||
},
|
||||
"aggregate_hard_fit_floor_gib": 224.0,
|
||||
"aggregate_floor_class": "experimental_hard_fit_floor",
|
||||
"placement_imbalance_factor": 1.1,
|
||||
"tier_table": {
|
||||
"32": {
|
||||
"physical_usable_gib": 32.0,
|
||||
"reserve_gib": 8.0,
|
||||
"placement_budget_gib": 24.0,
|
||||
"weight_gib": 201.832,
|
||||
"kv_gib": 0.89,
|
||||
"total_placement_gib": 202.722,
|
||||
"arithmetic_minimum_nodes": 9,
|
||||
"recommended_nodes": 10,
|
||||
"imbalance_factor": 1.1
|
||||
},
|
||||
"48": {
|
||||
"physical_usable_gib": 48.0,
|
||||
"reserve_gib": 9.6,
|
||||
"placement_budget_gib": 38.4,
|
||||
"weight_gib": 201.832,
|
||||
"kv_gib": 0.89,
|
||||
"total_placement_gib": 202.722,
|
||||
"arithmetic_minimum_nodes": 6,
|
||||
"recommended_nodes": 6,
|
||||
"imbalance_factor": 1.1
|
||||
},
|
||||
"64": {
|
||||
"physical_usable_gib": 64.0,
|
||||
"reserve_gib": 12.8,
|
||||
"placement_budget_gib": 51.2,
|
||||
"weight_gib": 201.832,
|
||||
"kv_gib": 0.89,
|
||||
"total_placement_gib": 202.722,
|
||||
"arithmetic_minimum_nodes": 4,
|
||||
"recommended_nodes": 5,
|
||||
"imbalance_factor": 1.1
|
||||
},
|
||||
"96": {
|
||||
"physical_usable_gib": 96.0,
|
||||
"reserve_gib": 19.2,
|
||||
"placement_budget_gib": 76.8,
|
||||
"weight_gib": 201.832,
|
||||
"kv_gib": 0.89,
|
||||
"total_placement_gib": 202.722,
|
||||
"arithmetic_minimum_nodes": 3,
|
||||
"recommended_nodes": 3,
|
||||
"imbalance_factor": 1.1
|
||||
},
|
||||
"128": {
|
||||
"physical_usable_gib": 128.0,
|
||||
"reserve_gib": 25.6,
|
||||
"placement_budget_gib": 102.4,
|
||||
"weight_gib": 201.832,
|
||||
"kv_gib": 0.89,
|
||||
"total_placement_gib": 202.722,
|
||||
"arithmetic_minimum_nodes": 2,
|
||||
"recommended_nodes": 3,
|
||||
"imbalance_factor": 1.1
|
||||
}
|
||||
},
|
||||
"routes": {
|
||||
"recommended_5x64_unified": {
|
||||
"node_count": 5,
|
||||
"aggregate_usable_gib": 320.0,
|
||||
"aggregate_placement_budget_gib": 256.0,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": true,
|
||||
"meets_hard_fit_floor": true,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": 53.278,
|
||||
"reasons": []
|
||||
},
|
||||
"recommended_3x96_unified": {
|
||||
"node_count": 3,
|
||||
"aggregate_usable_gib": 288.0,
|
||||
"aggregate_placement_budget_gib": 230.4,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": true,
|
||||
"meets_hard_fit_floor": true,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": 27.678,
|
||||
"reasons": []
|
||||
},
|
||||
"recommended_3x128_unified": {
|
||||
"node_count": 3,
|
||||
"aggregate_usable_gib": 384.0,
|
||||
"aggregate_placement_budget_gib": 307.2,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": true,
|
||||
"meets_hard_fit_floor": true,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": 104.478,
|
||||
"reasons": []
|
||||
},
|
||||
"fit_probe_2x128_unified": {
|
||||
"node_count": 2,
|
||||
"aggregate_usable_gib": 256.0,
|
||||
"aggregate_placement_budget_gib": 204.8,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": true,
|
||||
"meets_hard_fit_floor": true,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": 2.078,
|
||||
"reasons": []
|
||||
},
|
||||
"fit_probe_4x64_unified": {
|
||||
"node_count": 4,
|
||||
"aggregate_usable_gib": 256.0,
|
||||
"aggregate_placement_budget_gib": 204.8,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": true,
|
||||
"meets_hard_fit_floor": true,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": 2.078,
|
||||
"reasons": []
|
||||
},
|
||||
"hard_fit_floor_2x112_unified": {
|
||||
"node_count": 2,
|
||||
"aggregate_usable_gib": 224.0,
|
||||
"aggregate_placement_budget_gib": 179.2,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": false,
|
||||
"meets_hard_fit_floor": true,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": -23.522,
|
||||
"reasons": [
|
||||
"aggregate placement budget 179.2 GiB is below the 202.7 GiB the target needs after each node's reserve"
|
||||
]
|
||||
},
|
||||
"insufficient_3x64_unified": {
|
||||
"node_count": 3,
|
||||
"aggregate_usable_gib": 192.0,
|
||||
"aggregate_placement_budget_gib": 153.6,
|
||||
"required_placement_gib": 202.722,
|
||||
"fits": false,
|
||||
"meets_hard_fit_floor": false,
|
||||
"no_single_node_can_admit_target": true,
|
||||
"headroom_gib": -49.122,
|
||||
"reasons": [
|
||||
"aggregate placement budget 153.6 GiB is below the 202.7 GiB the target needs after each node's reserve",
|
||||
"aggregate usable memory 192.0 GiB is below the 224 GiB experimental hard-fit floor"
|
||||
]
|
||||
}
|
||||
},
|
||||
"seams": {
|
||||
"3_nodes_2.5gbe": {
|
||||
"node_count": 3,
|
||||
"seam_count": 2,
|
||||
"hidden_size": 6144,
|
||||
"bytes_per_token_per_seam": 12288,
|
||||
"prefill_bytes_per_seam": 201326592,
|
||||
"decode_bytes_per_seam_per_token": 12288,
|
||||
"dsa_sideband_bytes_per_query": 8192,
|
||||
"link_rate_gbps": 2.5,
|
||||
"meets_alpha_minimum": true,
|
||||
"is_recommended_link": false,
|
||||
"decode_serialization_ms_per_token": 0.0786,
|
||||
"decode_latency_ms_per_token": 1.0,
|
||||
"decode_bandwidth_share_ms_per_token": 0.0786,
|
||||
"prefill_serialization_ms": 1288.49
|
||||
},
|
||||
"3_nodes_10.0gbe": {
|
||||
"node_count": 3,
|
||||
"seam_count": 2,
|
||||
"hidden_size": 6144,
|
||||
"bytes_per_token_per_seam": 12288,
|
||||
"prefill_bytes_per_seam": 201326592,
|
||||
"decode_bytes_per_seam_per_token": 12288,
|
||||
"dsa_sideband_bytes_per_query": 8192,
|
||||
"link_rate_gbps": 10.0,
|
||||
"meets_alpha_minimum": true,
|
||||
"is_recommended_link": true,
|
||||
"decode_serialization_ms_per_token": 0.0197,
|
||||
"decode_latency_ms_per_token": 1.0,
|
||||
"decode_bandwidth_share_ms_per_token": 0.0197,
|
||||
"prefill_serialization_ms": 322.123
|
||||
},
|
||||
"5_nodes_2.5gbe": {
|
||||
"node_count": 5,
|
||||
"seam_count": 4,
|
||||
"hidden_size": 6144,
|
||||
"bytes_per_token_per_seam": 12288,
|
||||
"prefill_bytes_per_seam": 201326592,
|
||||
"decode_bytes_per_seam_per_token": 12288,
|
||||
"dsa_sideband_bytes_per_query": 8192,
|
||||
"link_rate_gbps": 2.5,
|
||||
"meets_alpha_minimum": true,
|
||||
"is_recommended_link": false,
|
||||
"decode_serialization_ms_per_token": 0.1573,
|
||||
"decode_latency_ms_per_token": 2.0,
|
||||
"decode_bandwidth_share_ms_per_token": 0.1573,
|
||||
"prefill_serialization_ms": 2576.98
|
||||
},
|
||||
"5_nodes_10.0gbe": {
|
||||
"node_count": 5,
|
||||
"seam_count": 4,
|
||||
"hidden_size": 6144,
|
||||
"bytes_per_token_per_seam": 12288,
|
||||
"prefill_bytes_per_seam": 201326592,
|
||||
"decode_bytes_per_seam_per_token": 12288,
|
||||
"dsa_sideband_bytes_per_query": 8192,
|
||||
"link_rate_gbps": 10.0,
|
||||
"meets_alpha_minimum": true,
|
||||
"is_recommended_link": true,
|
||||
"decode_serialization_ms_per_token": 0.0393,
|
||||
"decode_latency_ms_per_token": 2.0,
|
||||
"decode_bandwidth_share_ms_per_token": 0.0393,
|
||||
"prefill_serialization_ms": 644.245
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"observed_at": "2026-07-13",
|
||||
"observed_by": "DGR-017",
|
||||
"method": "GitHub REST API (api.github.com), read-only; no fork, no clone, no patch adopted",
|
||||
"refresh_note": "The roadmap's 2026-07-13 observations were re-verified against live upstream. One item changed: PR #24231 is now MERGED (2026-07-11), which the roadmap already anticipated as 'generic CPU lightning-indexer support is merged'.",
|
||||
"llama_cpp": {
|
||||
"repo": "ggml-org/llama.cpp",
|
||||
"items": [
|
||||
{
|
||||
"ref": "issue #24730",
|
||||
"url": "https://github.com/ggml-org/llama.cpp/issues/24730",
|
||||
"title": "Feature Request: Support for GLM 5.2",
|
||||
"type": "issue",
|
||||
"state": "open",
|
||||
"updated_at": "2026-07-03T22:02:15Z",
|
||||
"meaning": "The umbrella GLM-5.2 support request is still open. GLM-5.2 is not fully supported upstream."
|
||||
},
|
||||
{
|
||||
"ref": "PR #24770",
|
||||
"url": "https://github.com/ggml-org/llama.cpp/pull/24770",
|
||||
"title": "model : glm-dsa load DSA indexer tensors as optional",
|
||||
"type": "pull_request",
|
||||
"state": "closed",
|
||||
"merged_at": "2026-06-20T10:48:24Z",
|
||||
"meaning": "MERGED. GLM-5.2 loads, but through a dense-MLA compatibility path with DSA indexer tensors treated as optional. This is the fallback the alpha contract explicitly refuses: it can produce text without performing DSA/IndexShare computation."
|
||||
},
|
||||
{
|
||||
"ref": "PR #24231",
|
||||
"url": "https://github.com/ggml-org/llama.cpp/pull/24231",
|
||||
"title": "New GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer",
|
||||
"type": "pull_request",
|
||||
"state": "closed",
|
||||
"merged_at": "2026-07-11T09:39:07Z",
|
||||
"meaning": "MERGED since the roadmap was written. A generic lightning-indexer op now exists in GGML. Backend coverage beyond CPU remains uneven and must be verified per backend by DGR-018, not assumed."
|
||||
},
|
||||
{
|
||||
"ref": "PR #25407",
|
||||
"url": "https://github.com/ggml-org/llama.cpp/pull/25407",
|
||||
"title": "GLM 5.2 Indexer support",
|
||||
"type": "pull_request",
|
||||
"state": "open",
|
||||
"draft": false,
|
||||
"mergeable_state": "unstable",
|
||||
"head_sha": "8dedd06415f36f10fc6091241a39b23c1bf0ee11",
|
||||
"base": "master",
|
||||
"commits": 6,
|
||||
"changed_files": 12,
|
||||
"additions": 414,
|
||||
"deletions": 7,
|
||||
"updated_at": "2026-07-13T15:28:51Z",
|
||||
"meaning": "OPEN and actively moving (updated today). This is the real DSA/IndexShare implementation alpha needs. It is narrow — 12 files, +414/-7 — which is the single most important finding for donor policy: the semantics alpha requires are reviewable and trackable upstream, not a 261-patch fork."
|
||||
}
|
||||
]
|
||||
},
|
||||
"capability_status_for_alpha": {
|
||||
"gguf_load_of_UD-IQ1_S": "expected via merged #24770, unverified by this project; DGR-018 must prove it against the exact pinned artifact",
|
||||
"moe_routing_and_shared_expert": "expected supported; unverified here",
|
||||
"compressed_mla_kv": "supported via the dense-MLA compatibility path",
|
||||
"dsa_lightning_indexer": "generic GGML op merged (#24231); GLM-5.2 wiring still open (#25407)",
|
||||
"indexshare_full_shared_roles": "NOT upstream; only in open PR #25407",
|
||||
"mtp_nextn": "not required for alpha; NextN tensors must be explicitly loaded or excluded, never silently reinterpreted",
|
||||
"conclusion": "As of 2026-07-13 no released upstream llama.cpp performs native GLM-5.2 DSA + IndexShare. A stock pin today would satisfy 'it emits text' via the dense fallback and would FAIL the alpha semantic-correctness contract. This is the gating technical risk for DGR-004 and DGR-018."
|
||||
},
|
||||
"donor": {
|
||||
"repo": "Mesh-LLM/mesh-llm",
|
||||
"url": "https://github.com/Mesh-LLM/mesh-llm",
|
||||
"license": "Apache-2.0",
|
||||
"stars_observed": 2048,
|
||||
"pushed_at": "2026-07-13T06:45:51Z",
|
||||
"glm_branch": "feat/jianyang-glm-52",
|
||||
"glm_branch_head": "9bd18f1509dff7fac21578635084035b3ba90a38",
|
||||
"glm_branch_head_date": "2026-07-12T06:37:43Z",
|
||||
"policy": "TEST AND PATCH DONOR ONLY. Do not adopt the fork, its scheduler, discovery, routing, public mesh, or package manager. Meshnet remains the sole control plane (RALPH-CONTEXT runtime decision, ADR-0020).",
|
||||
"focused_candidates": [
|
||||
"GLM DSA graph semantics",
|
||||
"lightning indexer and sparse-attention tests",
|
||||
"IndexShare metadata and Full/Shared role validation",
|
||||
"top-k sideband shape and lifecycle",
|
||||
"stage-local KV filtering",
|
||||
"target parity and performance fixtures"
|
||||
],
|
||||
"adoption_state": "none adopted in DGR-017. This story reads and records upstream state; it takes no patch and forks nothing."
|
||||
},
|
||||
"recommendation_for_dgr_004_and_dgr_018": [
|
||||
"Track upstream PR #25407 rather than forking Mesh-LLM. At 12 files and +414/-7 it is small enough to review, reproduce, and carry as a numbered patch in the project's own pinned stack.",
|
||||
"Any pin chosen before #25407 merges will load GLM-5.2 through the dense-MLA compatibility path. DGR-018 must therefore prove DSA/IndexShare are ACTIVE, not merely that the model emits text — the alpha contract already forbids the fallback.",
|
||||
"Verify lightning-indexer backend coverage (#24231) on the specific backend the route will use. CPU support being merged says nothing about ROCm/HIP."
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# DGR-018 — BLOCKED: no 256-GiB-class oracle host
|
||||
|
||||
Recorded: 2026-07-14 (MAINT-003). Preflight scripts preserved at commit
|
||||
`a0f28b5` ("chore: preserve DGR-018 preflight scripts (postponed)").
|
||||
|
||||
## Blocker
|
||||
|
||||
DGR-018 requires a 256-GiB-class host with at least **224 GiB
|
||||
runtime-accessible memory** (the DGR-017 experimental hard-fit floor for the
|
||||
whole-model `UD-IQ1_S` oracle) and **250 GB free storage** on one filesystem
|
||||
outside `/home` (216.715 GB artifact plus resume/temp headroom). The available
|
||||
development host fails both gates, so the whole-model oracle cannot be
|
||||
established. Per the issue's finish contract, no smaller model may be
|
||||
substituted.
|
||||
|
||||
DGR-019 (needs the DGR-018 oracle for parity certification) and DGR-020
|
||||
(needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single
|
||||
node admits the whole recipe) are blocked transitively.
|
||||
|
||||
## Exact preflight output
|
||||
|
||||
Command (offline; resolves everything from the pinned target manifest and
|
||||
never contacts the network):
|
||||
|
||||
```
|
||||
$ python scripts/glm_whole_model_preflight.py
|
||||
target: UD-IQ1_S 216.715 GB, 6 shards @ abc55e725277
|
||||
[FAIL] storage: need >= 250 GB free on one filesystem outside ['/home']; observed no eligible filesystem
|
||||
[FAIL] memory: need >= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor); observed 124.9 GiB MemTotal
|
||||
destination: NONE — no filesystem outside ['/home'] has 250 GB free
|
||||
- /run/media/popov/DATA (ext4): 74.2 GB free
|
||||
- / (ext4): 51.1 GB free
|
||||
- /run/media/popov/Windows (fuseblk): 26.0 GB free
|
||||
- /run/media/popov/d (fuseblk): 5.1 GB free
|
||||
verdict: fail
|
||||
$ echo $?
|
||||
1
|
||||
```
|
||||
|
||||
Host: Linux 7.0.14-101.fc43.x86_64 x86_64, `MemTotal: 130997376 kB`
|
||||
(124.9 GiB). The full machine-readable report (including the ordered
|
||||
download/verify plan against revision `abc55e72527792c6e77069c99b4cb7de16fa9f23`,
|
||||
manifest SHA-256 `0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962`)
|
||||
is in [preflight.json](preflight.json).
|
||||
|
||||
## How to resume
|
||||
|
||||
1. On a qualifying host, run `python scripts/glm_whole_model_preflight.py`
|
||||
(optionally `--dest DIR`); it must exit 0 with `verdict: pass`.
|
||||
2. Download shards in the preflight's ordered plan; verify each with
|
||||
`python scripts/verify_glm_shards.py` before the next transfer starts.
|
||||
3. Proceed with the DGR-018 issue
|
||||
(`.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md`).
|
||||
@@ -0,0 +1,140 @@
|
||||
{
|
||||
"generated_by": "scripts/glm_whole_model_preflight.py",
|
||||
"target": {
|
||||
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||
"quantization": "UD-IQ1_S",
|
||||
"shard_count": 6,
|
||||
"total_bytes": 216715360960,
|
||||
"total_gb": 216.715,
|
||||
"manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962"
|
||||
},
|
||||
"forbidden_path_prefixes": [
|
||||
"/home"
|
||||
],
|
||||
"mounts": [
|
||||
{
|
||||
"mountpoint": "/run/media/popov/DATA",
|
||||
"fstype": "ext4",
|
||||
"total_gb": 1208.8,
|
||||
"free_gb": 74.2,
|
||||
"free_bytes": 74201321472,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"mountpoint": "/",
|
||||
"fstype": "ext4",
|
||||
"total_gb": 217.7,
|
||||
"free_gb": 51.1,
|
||||
"free_bytes": 51073683456,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"mountpoint": "/run/media/popov/Windows",
|
||||
"fstype": "fuseblk",
|
||||
"total_gb": 434.9,
|
||||
"free_gb": 26.0,
|
||||
"free_bytes": 25964466176,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
},
|
||||
{
|
||||
"mountpoint": "/run/media/popov/d",
|
||||
"fstype": "fuseblk",
|
||||
"total_gb": 161.1,
|
||||
"free_gb": 5.1,
|
||||
"free_bytes": 5148332032,
|
||||
"forbidden": false,
|
||||
"eligible": false
|
||||
}
|
||||
],
|
||||
"chosen_destination": null,
|
||||
"checks": [
|
||||
{
|
||||
"check": "storage",
|
||||
"requirement": ">= 250 GB free on one filesystem outside ['/home']",
|
||||
"observed": "no eligible filesystem",
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"check": "memory",
|
||||
"requirement": ">= 224 GiB runtime-accessible memory (DGR-017 experimental hard-fit floor)",
|
||||
"observed": "124.9 GiB MemTotal",
|
||||
"passes": false,
|
||||
"waived": false
|
||||
}
|
||||
],
|
||||
"download_authorized": false,
|
||||
"storage_only": false,
|
||||
"download_plan": [
|
||||
{
|
||||
"step": 1,
|
||||
"shard_index": 1,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||
"size_bytes": 9423744,
|
||||
"size_gb": 0.009,
|
||||
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 1"
|
||||
},
|
||||
{
|
||||
"step": 2,
|
||||
"shard_index": 6,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||
"size_bytes": 19171063136,
|
||||
"size_gb": 19.171,
|
||||
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 6"
|
||||
},
|
||||
{
|
||||
"step": 3,
|
||||
"shard_index": 2,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||
"size_bytes": 49208128256,
|
||||
"size_gb": 49.208,
|
||||
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 2"
|
||||
},
|
||||
{
|
||||
"step": 4,
|
||||
"shard_index": 3,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||
"size_bytes": 49684417024,
|
||||
"size_gb": 49.684,
|
||||
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 3"
|
||||
},
|
||||
{
|
||||
"step": 5,
|
||||
"shard_index": 4,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||
"size_bytes": 49396052864,
|
||||
"size_gb": 49.396,
|
||||
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 4"
|
||||
},
|
||||
{
|
||||
"step": 6,
|
||||
"shard_index": 5,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||
"size_bytes": 49246275936,
|
||||
"size_gb": 49.246,
|
||||
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||
"download_command": "curl -L -C - --fail -o \"$GLM_DEST/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\" \"https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf\"",
|
||||
"verify_command": "python scripts/verify_glm_shards.py --model-dir \"$GLM_DEST\" --shard 5"
|
||||
}
|
||||
],
|
||||
"verdict": "fail"
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
# Maintenance review handoff — distributed GGUF runtime
|
||||
|
||||
Date: 2026-07-14
|
||||
Scope: close the maintenance review, preserve the hard blockers, and hand off the remaining implementation work to the next model.
|
||||
|
||||
## What is complete
|
||||
|
||||
- Completed stories are now recorded in `docs/issues/distributed-gguf-runtime/`.
|
||||
- The PRD and milestone docs were updated to reflect the closed set and the blocked set.
|
||||
- The DGR-018 preflight scripts were preserved at commit `a0f28b5`.
|
||||
- The current feature line has delivered DGR-001 through DGR-006 and DGR-017.
|
||||
|
||||
## Hard / unsolved issues for later
|
||||
|
||||
### 1) DGR-018 requires hardware we do not have
|
||||
|
||||
DGR-018 is blocked because the whole-model GLM-5.2 UD-IQ1_S oracle requires:
|
||||
|
||||
- a **256-GiB-class host**,
|
||||
- at least **224 GiB runtime-accessible memory**,
|
||||
- at least **250 GB free storage on one filesystem outside `/home`**.
|
||||
|
||||
The current development host reports only **124.9 GiB MemTotal** and has no eligible filesystem with 250 GB free.
|
||||
The authoritative blocker evidence is in `evidence/DGR-018/BLOCKED.md` and `evidence/DGR-018/preflight.json`.
|
||||
|
||||
### 2) DGR-019 and DGR-020 are transitively blocked
|
||||
|
||||
- **DGR-019** needs the DGR-018 oracle for parity certification.
|
||||
- **DGR-020** needs DGR-018 and DGR-019, plus enough physical consumer nodes that no single node can admit the whole recipe.
|
||||
|
||||
No smaller model may be substituted for these stories.
|
||||
|
||||
### 3) The remainder of the graph stays blocked unless replanned
|
||||
|
||||
The current graph makes **DGR-007 depend on DGR-019**, which means:
|
||||
|
||||
- DGR-007 through DGR-016 are also blocked transitively.
|
||||
- Unblocking the dense pipeline without the 256-GiB host would require an explicit replanning decision to relax the DGR-007 → DGR-019 dependency.
|
||||
- That replanning decision has **not** been made.
|
||||
|
||||
### 4) Maintenance-only tasks should stay separate from feature implementation
|
||||
|
||||
The review uncovered that the codebase now has a clean closed-story split, but further work should avoid mixing:
|
||||
|
||||
- maintenance cleanup,
|
||||
- blocked-hardware preparation,
|
||||
- and actual distributed GLM implementation.
|
||||
|
||||
The next model should treat the maintenance pass as closed and only pick up real implementation work that is not hardware-blocked.
|
||||
|
||||
## Recommended next move
|
||||
|
||||
Use the next model to continue on the **non-blocked implementation queue** only.
|
||||
Priority candidates are whatever is still actionable without the GLM oracle host; if a story depends on DGR-018, keep it deferred.
|
||||
|
||||
## Reference files
|
||||
|
||||
- `docs/issues/distributed-gguf-runtime/README.md`
|
||||
- `.scratch/distributed-gguf-runtime/PRD.md`
|
||||
- `.scratch/distributed-gguf-runtime/milestones.md`
|
||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`
|
||||
- `.scratch/distributed-gguf-runtime/evidence/DGR-018/preflight.json`
|
||||
@@ -7,6 +7,8 @@ Last updated: 2026-07-13
|
||||
|
||||
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
|
||||
|
||||
The alpha-release target is the exact `zai-org/GLM-5.2` model, pinned by revision and served with `reasoning_effort=max`, using the smallest published Unsloth `UD-IQ1_S` GGUF across physical consumer machines. See [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md). Dense Llama remains a cheap structural fixture; Qwen expansion is post-alpha.
|
||||
|
||||
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
|
||||
|
||||
1. Models larger than one node's RAM/VRAM.
|
||||
@@ -36,7 +38,7 @@ GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kerne
|
||||
|
||||
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
|
||||
|
||||
Before expensive native work, establish controlled lanes:
|
||||
Before expensive native work, establish controlled lanes. DGR-001 remains immutable; DGR-017 adds a target-specific fit and semantics contract without rewriting DGR-001 evidence:
|
||||
|
||||
- Same model architecture and upstream revision.
|
||||
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
|
||||
@@ -134,6 +136,10 @@ Use as source and test donors only:
|
||||
|
||||
Do not adopt or fork their repositories wholesale.
|
||||
|
||||
### Mesh-LLM GLM branch: focused test/patch donor only
|
||||
|
||||
Use its GLM-5.2 branch to study DSA, IndexShare, stage-local KV, and sideband tests. Do not import its scheduler, discovery/control plane, package manager, or broad llama.cpp patch stack. Every adopted idea must be independently understood, minimized, attributed, and tested against our exact pin.
|
||||
|
||||
## Battle-proven transport decision
|
||||
|
||||
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
|
||||
@@ -155,7 +161,7 @@ Scope boundary:
|
||||
- Large prefill tensors are chunked into bounded frames; decode bundles stay small.
|
||||
- No QUIC/WebRTC/custom transport in this milestone.
|
||||
|
||||
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`.
|
||||
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`. DGR-006 updates the current single-`NamedTensor` decode fast path to carry the same bundle semantics and adds an explicit typed tail logits/token result with sampling/template identity.
|
||||
|
||||
Minimum identity:
|
||||
|
||||
@@ -219,9 +225,9 @@ Two physical machines execute one model that uses both Shards. Synthetic activat
|
||||
|
||||
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
|
||||
|
||||
### Gate F: architecture expansion
|
||||
### Gate F: exact GLM-5.2 alpha target
|
||||
|
||||
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
|
||||
After the generic dense fixture proves range and boundary mechanics, certify explicit GLM-5.2 MoE, MLA KV, DSA, IndexShare, and NextN policy. Alpha requires the exact `UD-IQ1_S` target across physical consumer nodes, native Max-mode semantics, locked parity/usefulness/performance thresholds, and bounded failure cleanup. Qwen3/Qwen3-MoE is later architecture expansion.
|
||||
|
||||
## Scope discipline
|
||||
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
# 04 — Create the reproducible pinned llama.cpp patch stack
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-004` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact llama.cpp upstream pin
|
||||
- Numbered minimal patch stack
|
||||
- Reproducible fetch/apply/build smoke
|
||||
- evidence/DGR-004/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
|
||||
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
|
||||
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
|
||||
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
|
||||
- [ ] Preserve upstream license and attribution notices.
|
||||
- [ ] Add a clean rebuild smoke test that does not download a model.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
|
||||
@@ -42,7 +42,7 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
|
||||
- `DGR-006` and `DGR-019` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ As a client, I need failures to be bounded and explicit so that distributed spee
|
||||
|
||||
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
|
||||
- DGR-012 continuous batching is post-alpha; this story must harden single-session failure/cancellation first and later extend the same invariants to batched sessions.
|
||||
|
||||
## Finish contract
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# 18 — Certify whole-model GLM-5.2 runtime semantics
|
||||
|
||||
Status: blocked (2026-07-14) — no 256-GiB-class host available
|
||||
|
||||
> **Blocked:** This story requires a 256-GiB-class host with at least 224 GiB
|
||||
> runtime-accessible memory and 250 GB free storage outside `/home`. The
|
||||
> development host has 124.9 GiB MemTotal and no eligible filesystem (largest:
|
||||
> 74.2 GB free). Exact preflight output is preserved in
|
||||
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md); preflight
|
||||
> scripts were preserved at commit a0f28b5 (`scripts/glm_whole_model_preflight.py`,
|
||||
> `scripts/verify_glm_shards.py`). Resume by re-running the preflight on a
|
||||
> qualifying host — do not substitute a smaller model.
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
|
||||
- This issue is `DGR-018` in [prd.json](../prd.json).
|
||||
- Read and verify every dependency evidence README.
|
||||
- Inspect current upstream behavior and `git status`; community claims that a model “works” are not correctness evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a runtime maintainer, I need a certified whole-model oracle for the exact lowest-quant GLM-5.2 artifact so that distributed parity is measured against correct MoE, DSA, IndexShare, cache, and Max-template semantics.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Verified target artifact on mounted storage
|
||||
- Stock-pin baseline and warning/tensor inventory
|
||||
- Focused GLM runtime correctness tests and minimum patch series, if required
|
||||
- Signed whole-model oracle recipe, deterministic outputs, metrics, and limitations
|
||||
- `evidence/DGR-018/README.md`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Preflight a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation and approximately 250 GB free mounted storage; abort before download if storage resolves under `/home` or requirements are unmet.
|
||||
- [ ] Configure the oracle's alpha lane for 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; record actual MLA, indexer-cache, scratch, and peak resident allocations separately.
|
||||
- [ ] Download/resume and verify all six DGR-017 `UD-IQ1_S` files against exact sizes and LFS SHA-256 values.
|
||||
- [ ] Load the target with the unmodified DGR-004 pin first and retain full logs, tensor warnings, peak memory, context/KV allocation, TTFT, and decode timing.
|
||||
- [ ] Prove from graph/runtime evidence whether 256-expert MoE/top-8/shared expert, DSA lightning indexer/sparse attention, IndexShare Full/Shared reuse, and `reasoning_effort=max` are active.
|
||||
- [ ] Dense-attention or replicated-indexer compatibility fallback is labeled incomplete and cannot become the oracle solely because it emits plausible text.
|
||||
- [ ] Audit focused upstream/Mesh-LLM donor tests and patches; adopt or rewrite only independently understood minimum changes and record provenance/rejection rationale.
|
||||
- [ ] Handle the trailing NextN/MTP tensors explicitly; MTP may remain disabled for alpha, but ignored tensors and layer-count behavior must be understood and tested.
|
||||
- [ ] Run deterministic prefill/greedy decode and fixed Max-mode coding, structured-output/tool-call, and reasoning sentinels; retain raw prompts, outputs, token IDs, and compared state/logit evidence.
|
||||
- [ ] Bind the oracle to exact artifact, tokenizer/template, adapter, backend class, compute/activation/KV dtypes, context/RoPE, llama.cpp commit, and patch-series hash.
|
||||
- [ ] Produce a signed `pass` or `stop` semantic-runtime verdict; no performance threshold is weakened after execution.
|
||||
- [ ] Targeted native/unit tests and clean pinned rebuild pass.
|
||||
- [ ] `git diff --check` passes.
|
||||
- [ ] Default tests remain model-download-free/GPU-free; full-target execution is opt-in and never runs in default CI.
|
||||
- [ ] Model artifacts and caches remain on configured mounted-drive storage and never under `/home`.
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
|
||||
- [ ] Write `evidence/DGR-018/README.md` with exact hardware, storage, source revisions, files, commands, raw-result paths, real results, limitations, and handoff.
|
||||
- [ ] Update only this story issue to `Status: done` after every acceptance criterion passes.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-003`, `DGR-004`, and `DGR-017` must have `passes: true`; read and verify their evidence READMEs.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- If a 256-GiB-class host with at least 224 GiB runtime-accessible memory is unavailable, write `evidence/DGR-018/BLOCKED.md` with exact preflight output; do not substitute a smaller model.
|
||||
- Preserve real failures and blockers; never fabricate output.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the signed oracle evidence exists.
|
||||
|
||||
## References
|
||||
|
||||
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
|
||||
- [llama.cpp GLM-5.2 issue](https://github.com/ggml-org/llama.cpp/issues/24730)
|
||||
@@ -0,0 +1,63 @@
|
||||
# 19 — Implement and certify GLM-5.2 range, DSA, and IndexShare semantics
|
||||
|
||||
Status: blocked (2026-07-14) — waiting on DGR-018
|
||||
|
||||
> **Blocked:** Depends on DGR-018's whole-model IQ1_S oracle, which is blocked
|
||||
> on a 256-GiB-class host (≥ 224 GiB runtime-accessible memory). See
|
||||
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md). Locked
|
||||
> fixture/target parity cannot be certified without that oracle.
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
|
||||
- This issue is `DGR-019` in [prd.json](../prd.json).
|
||||
- Read and verify every dependency evidence README.
|
||||
- Inspect current source and `git status`; do not generalize through tensor-name substitution.
|
||||
|
||||
## Description
|
||||
|
||||
As a target-model operator, I need explicit range-owned GLM-5.2 semantics so that contiguous consumer Shards preserve the whole-model MoE, DSA, IndexShare, and local-KV computation.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Explicit GLM-5.2 architecture adapter and tensor-ownership rules
|
||||
- IndexShare-aware Shard planner and named DSA sideband protocol mapping
|
||||
- GLM architecture fixture and target parity evidence
|
||||
- Memory/tensor/KV ownership report
|
||||
- `evidence/DGR-019/README.md`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Implement explicit ownership for 78 main layers, head embedding, tail norm/output head, routed/shared experts, DSA/indexer tensors, and the certified NextN/MTP policy.
|
||||
- [ ] Each contiguous layer owner keeps all 256 experts and the shared-expert path for its layers local; no public cross-machine expert collectives are introduced.
|
||||
- [ ] Define and fixture-test compressed MLA KV ownership for locally owned layers, keyed by Route Session/epoch, for DGR-007 to implement.
|
||||
- [ ] Implement native DSA lightning indexer/top-2,048 and sparse-attention graph behavior matching DGR-018.
|
||||
- [ ] Parse and validate artifact `indexer_types`; implement Full producer and Shared consumer behavior without fabricated/duplicated indexer tensors.
|
||||
- [ ] Prefer Shard boundaries that preserve IndexShare ownership groups; when memory fit forces a split, carry a typed, bounded, validated top-k sideband in the DGR-006 named bundle.
|
||||
- [ ] Reject missing, stale, wrong-width, wrong-position, or Shared-before-Full sideband/index state.
|
||||
- [ ] Demonstrate mapped/resident weights and allocated KV scale with exact owned tensors/layers rather than the full model.
|
||||
- [ ] Same-host two-stage F32 seam fixture produces 32 exact greedy tokens against the DGR-018 semantics; production seam meets the pre-locked token/similarity thresholds.
|
||||
- [ ] If full-target same-host execution cannot fit, use a layer-reduced GLM architecture fixture only for graph parity and defer full-artifact parity to DGR-020; label it non-target evidence.
|
||||
- [ ] Add deterministic fixture tests for MoE route/shared expert, DSA Full/Shared groups, internal group split, endpoint ownership, KV filtering, malformed metadata, and NextN policy.
|
||||
- [ ] Targeted pytest/CTest/native tests pass; pinned patch stack applies and rebuilds cleanly.
|
||||
- [ ] `python -m compileall packages tests` and `git diff --check` pass.
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
|
||||
- [ ] Full deterministic `pytest -q` passes, or exact pre-existing unrelated failures are recorded.
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
|
||||
- [ ] Write `evidence/DGR-019/README.md` with files, commands, real results, raw evidence, limitations, and dependent-story handoff.
|
||||
- [ ] Update only this story issue to `Status: done` after every acceptance criterion passes.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-005`, `DGR-006`, and `DGR-018` must have `passes: true`; read and verify their evidence READMEs.
|
||||
- DGR-007 integration may be completed in parallel only after the adapter's cache contract is explicit.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Preserve real failures and blockers; never claim target parity from the architecture fixture.
|
||||
- Emit `<promise>COMPLETE</promise>` only after evidence exists.
|
||||
|
||||
## References
|
||||
|
||||
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
@@ -0,0 +1,67 @@
|
||||
# 20 — Pass real distributed GLM-5.2 Max alpha acceptance
|
||||
|
||||
Status: blocked (2026-07-14) — waiting on DGR-018/DGR-019
|
||||
|
||||
> **Blocked:** Depends on DGR-018 and DGR-019, both blocked on the 256-GiB-class
|
||||
> oracle host (≥ 224 GiB runtime-accessible memory), plus enough physical
|
||||
> consumer nodes that no single node admits the whole recipe. See
|
||||
> [evidence/DGR-018/BLOCKED.md](../evidence/DGR-018/BLOCKED.md).
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
|
||||
- This issue is `DGR-020` in [prd.json](../prd.json).
|
||||
- Read and verify every dependency evidence README and the immutable DGR-017 contract.
|
||||
- Inspect live source, hardware, network, storage, and `git status`; synthetic or layer-reduced evidence cannot satisfy this story.
|
||||
|
||||
## Description
|
||||
|
||||
As an alpha user, I need the exact lowest-quant GLM-5.2 model to run in Max reasoning mode across consumer machines through Meshnet so that the project has proven its end goal on real hardware.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact multi-node hardware/network/storage/runtime manifest
|
||||
- Signed target identity, ownership, parity, quality, performance, and reliability reports
|
||||
- Raw worker/tracker/API logs, metrics, prompts, outputs, and cleanup evidence
|
||||
- Explicit immutable `alpha` or `stop` verdict
|
||||
- `evidence/DGR-020/README.md`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Use the exact DGR-017 `UD-IQ1_S` artifact and certified DGR-018/DGR-019 runtime recipe with all source and patch hashes verified.
|
||||
- [ ] Use at least two physical consumer machines; each reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-Q8-KV placement, no node can place the complete recipe, unified memory is counted once, and measured peak use stays within physical memory without swap/overcommit.
|
||||
- [ ] Treat 5×64 GiB or 3×96/128 GiB as the recommended topology; arithmetic-minimum 4×64 or 2×128 qualifies only with exact contiguous placement and measured reserve evidence.
|
||||
- [ ] Use a same-switch wired network with at least 2.5 GbE; record 10 GbE as recommended and measure hop RTT/serialization/queue latency.
|
||||
- [ ] Tracker selects disjoint contiguous Shards whose exact required tensor inventory has complete union and no unintended overlap.
|
||||
- [ ] Every stage reports real CPU/GPU compute, owned tensor bytes/layers, local KV, backend, queue, and seam telemetry; synthetic, passthrough, or zero-layer workers fail acceptance.
|
||||
- [ ] Runtime evidence proves native GLM MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared paths are active; dense fallback fails acceptance.
|
||||
- [ ] OpenAI-compatible API applies and records `reasoning_effort=max`, stable model ID, finish reason, and token usage.
|
||||
- [ ] Configure 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; complete the fixed 4,096-token prefill lane and generate at least 512 output tokens or valid natural EOS after at least 128 tokens.
|
||||
- [ ] Production seam meets at least 0.90 greedy token agreement and 0.999 mean compared-state/logit cosine similarity against DGR-018 on the fixed corpus, with no non-finite or malformed tensors.
|
||||
- [ ] Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable relevant outputs retained for human review.
|
||||
- [ ] After warm-up, fixed Max lane median decode is at least 0.5 token/s, 4,096-token-prompt TTFT is at most 10 minutes, and no unexplained stall exceeds 60 seconds without progress telemetry.
|
||||
- [ ] Record per-stage/seam p50/p95 latency, bytes, compute time, peak RSS/VRAM, KV pressure, network properties, errors, and total energy when available.
|
||||
- [ ] Two consecutive cold starts load, generate, release, and exit without leaked processes, mapped weights, queues, or KV leases.
|
||||
- [ ] Cancellation in prefill and decode releases all stages; one worker loss aborts the route; retry starts from token zero on a newly compatible route; stale epochs and duplicate steps fail closed.
|
||||
- [ ] Model artifacts/caches remain on configured mounted-drive storage and never under `/home`; secret scan passes.
|
||||
- [ ] Run targeted/full deterministic tests and clean native rebuild in addition to target execution; `git diff --check` passes.
|
||||
- [ ] Preserve all raw failures and do not weaken DGR-017 thresholds after results.
|
||||
- [ ] Write a signed `alpha` verdict only if every criterion passes; otherwise write `stop` with the measured bottleneck and next narrow action.
|
||||
- [ ] Write `evidence/DGR-020/README.md` with exact commands, manifests, raw-result paths, outputs, limitations, and post-alpha handoff.
|
||||
- [ ] Update only this story issue to `Status: done` after the immutable verdict and all evidence exist.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-007`, `DGR-008`, `DGR-009`, `DGR-011`, `DGR-013`, `DGR-017`, `DGR-018`, and `DGR-019` must have `passes: true`; read and verify their evidence READMEs.
|
||||
- DGR-012 continuous batching and DGR-014 final comparative release gate are post-alpha and do not block one target session.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- If required physical nodes/storage are unavailable, write `evidence/DGR-020/BLOCKED.md`; do not substitute a smaller model, API, synthetic worker, or single host.
|
||||
- Preserve real failures and blockers; never fabricate target output or telemetry.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the signed verdict exists.
|
||||
|
||||
## References
|
||||
|
||||
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
@@ -1,35 +1,47 @@
|
||||
# Distributed GGUF runtime milestones
|
||||
|
||||
## Gate A — measured runtime value
|
||||
The exact alpha target and immutable acceptance gates are defined in [GLM-5.2-MAX-ALPHA-ROADMAP.md](GLM-5.2-MAX-ALPHA-ROADMAP.md).
|
||||
|
||||
- DGR-001 locks the safetensors-versus-GGUF performance/fit/quality contract.
|
||||
- DGR-002 can proceed independently and defines the battle-proven backend-neutral wire protocol.
|
||||
- DGR-003 builds exact recipe identity on DGR-002.
|
||||
- Expensive native llama.cpp work remains gated by DGR-001.
|
||||
## Completed foundation
|
||||
|
||||
- DGR-001 locks the safetensors-versus-GGUF performance/fit/quality contract. Its immutable CPU v1 verdict remains `stop`; separate ROCm evidence supports a fit-oriented investigation without rewriting CPU evidence.
|
||||
- DGR-002 defines the backend-neutral gRPC/Protobuf Shard protocol.
|
||||
|
||||
## Gate A — exact GLM-5.2 target and oracle
|
||||
|
||||
- DGR-017 locks official/GGUF revisions, `UD-IQ1_S`, hashes, resource accounting, Max-mode semantics, and alpha thresholds.
|
||||
- DGR-003 builds exact recipe identity on DGR-002 and DGR-017.
|
||||
- DGR-004 creates the reproducible pinned llama.cpp boundary after stock GLM behavior is measured.
|
||||
- DGR-018 certifies one whole-model `UD-IQ1_S` oracle with real MoE, DSA, IndexShare, KV, and `reasoning_effort=max` semantics. **Blocked (2026-07-14):** requires a 256-GiB-class host (≥ 224 GiB runtime-accessible memory); see [evidence/DGR-018/BLOCKED.md](evidence/DGR-018/BLOCKED.md). DGR-019 and DGR-020 are blocked transitively.
|
||||
|
||||
## Gate B — minimal native execution seam
|
||||
|
||||
- DGR-004 creates the reproducible pinned fork boundary.
|
||||
- DGR-005 implements dense-Llama range ownership.
|
||||
- DGR-006 proves architecture-defined boundary parity.
|
||||
- DGR-005 implements range-owned tensors using dense Llama as a cheap structural fixture.
|
||||
- DGR-006 proves the generic named activation boundary and F32 correctness lane.
|
||||
- DGR-019 implements explicit GLM-5.2 MoE/MLA/DSA/IndexShare/NextN range semantics and parity.
|
||||
|
||||
## Gate C — concurrent production worker
|
||||
## Gate C — native Meshnet route
|
||||
|
||||
- DGR-007 isolates concurrent Hot KV State.
|
||||
- DGR-007 isolates Shard-local Hot KV State.
|
||||
- DGR-008 exposes the native worker over gRPC.
|
||||
- DGR-009 integrates the worker without replacing Meshnet's control plane.
|
||||
- DGR-010 passes local real-model two-process acceptance.
|
||||
- DGR-011 passes real two-physical-machine execution.
|
||||
- DGR-013 supplies the cancellation/node-loss/restart/cleanup subset required by alpha.
|
||||
|
||||
## Gate D — real consumer-hardware route
|
||||
## Gate D — GLM-5.2 Max alpha verdict
|
||||
|
||||
- DGR-020 runs the exact `UD-IQ1_S` target across enough physical consumer nodes that no one node can admit the whole recipe.
|
||||
- It produces an immutable `alpha` or `stop` verdict from target identity, native GLM semantics, parity, Max-mode usefulness, minimum speed, telemetry, failure, and cleanup evidence.
|
||||
- Synthetic workers, layer-reduced fixtures, dense-attention fallbacks, and single-host execution cannot satisfy this gate.
|
||||
|
||||
## Gate E — post-alpha product hardening
|
||||
|
||||
- DGR-011 passes two-physical-machine execution.
|
||||
- DGR-012 adds continuous batching and bounded admission.
|
||||
- DGR-013 hardens failure and cancellation.
|
||||
|
||||
## Gate E — product release decision
|
||||
|
||||
- DGR-014 compares distributed GGUF against the current distributed safetensors route under locked thresholds.
|
||||
- DGR-015 adds Qwen3/Qwen3-MoE only after the dense runtime passes.
|
||||
- DGR-014 compares distributed GGUF against the current reference route under locked thresholds.
|
||||
- Longer contexts progress through 32K/128K/200K before 1M certification.
|
||||
- MTP/speculative decoding and source-bound layer packages remain measured optimizations.
|
||||
- DGR-016 prepares narrow upstream llama.cpp collaboration material.
|
||||
- DGR-015 adds Qwen3/Qwen3-MoE only as later architecture expansion.
|
||||
|
||||
No later gate may be claimed from synthetic workers or documentation-only evidence.
|
||||
No later gate may be claimed from synthetic workers or documentation-only evidence. Model artifacts remain on mounted-drive storage and never under `/home`.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "Performant Concurrent Distributed GGUF Runtime",
|
||||
"branchName": "ralph/performant-concurrent-distributed-gguf",
|
||||
"description": "Benchmark-gated native llama.cpp/GGUF Shards with gRPC streaming, concurrent local KV, continuous batching, real heterogeneous acceptance, and a measured release gate against Transformers/safetensors.",
|
||||
"description": "Build a performant concurrent distributed GGUF runtime on Meshnet. Alpha is gated by exact GLM-5.2 UD-IQ1_S execution in reasoning_effort=max mode across consumer machines; synthetic or dense-fallback evidence cannot satisfy the target.",
|
||||
"userStories": [
|
||||
{
|
||||
"id": "DGR-001",
|
||||
@@ -26,9 +26,9 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-001/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 2,
|
||||
"priority": 1,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/01-lock-the-safetensors-versus-gguf-performance-contract.md (moved on close, MAINT-003)",
|
||||
"dependsOn": []
|
||||
},
|
||||
{
|
||||
@@ -53,9 +53,9 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 1,
|
||||
"priority": 2,
|
||||
"passes": true,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md",
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/02-adopt-the-versioned-grpc-shard-protocol.md (moved on close, MAINT-003)",
|
||||
"dependsOn": []
|
||||
},
|
||||
{
|
||||
@@ -79,11 +79,12 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 3,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md",
|
||||
"priority": 4,
|
||||
"passes": true,
|
||||
"notes": "DGR-003-emission: native loaded-artifact adapter derives exact identity from immutable GGUF/runtime inputs, doctor emits it only for that adapter, and SessionOpen fails closed before acceptance. Exact identities remain tracker-uncertified and dark until a real distributed forward is certified. (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-002"
|
||||
"DGR-002",
|
||||
"DGR-017"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -110,11 +111,12 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 4,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md",
|
||||
"priority": 5,
|
||||
"passes": true,
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-001"
|
||||
"DGR-001",
|
||||
"DGR-017"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -140,9 +142,9 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-005/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 5,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md",
|
||||
"priority": 6,
|
||||
"passes": true,
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/05-implement-dense-llama-range-aware-gguf-ownership.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-003",
|
||||
"DGR-004"
|
||||
@@ -155,6 +157,8 @@
|
||||
"acceptanceCriteria": [
|
||||
"Head accepts token IDs and owns token embedding.",
|
||||
"Middle/tail bypass token embedding and accept the named boundary bundle.",
|
||||
"Amend the decode fast path so `DecodeStep` carries a versioned `TensorBundle`, not one `NamedTensor`; retain a compact one-tensor encoding and fallback compatibility, regenerate Python/C++ schema code, and update protocol goldens.",
|
||||
"Define an explicit typed tail result for logits and/or sampled token output, with sampling parameters and chat-template/reasoning mode bound to exact request/recipe identity.",
|
||||
"Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.",
|
||||
"Tail emits logits or token output through an explicit sampling contract.",
|
||||
"Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.",
|
||||
@@ -172,9 +176,9 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-006/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 6,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md",
|
||||
"priority": 7,
|
||||
"passes": true,
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/06-implement-architecture-defined-boundary-input-output.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-002",
|
||||
"DGR-005"
|
||||
@@ -204,11 +208,12 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-007/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 7,
|
||||
"priority": 10,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md",
|
||||
"dependsOn": [
|
||||
"DGR-006"
|
||||
"DGR-006",
|
||||
"DGR-019"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -235,7 +240,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-008/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 8,
|
||||
"priority": 11,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md",
|
||||
"dependsOn": [
|
||||
@@ -270,7 +275,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 9,
|
||||
"priority": 12,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md",
|
||||
"dependsOn": [
|
||||
@@ -304,7 +309,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-010/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 10,
|
||||
"priority": 13,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md",
|
||||
"dependsOn": [
|
||||
@@ -337,7 +342,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-011/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 11,
|
||||
"priority": 14,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md",
|
||||
"dependsOn": [
|
||||
@@ -368,7 +373,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-012/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 12,
|
||||
"priority": 17,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md",
|
||||
"dependsOn": [
|
||||
@@ -401,13 +406,12 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-013/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 13,
|
||||
"priority": 15,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md",
|
||||
"dependsOn": [
|
||||
"DGR-008",
|
||||
"DGR-009",
|
||||
"DGR-012"
|
||||
"DGR-009"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -435,7 +439,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-014/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 14,
|
||||
"priority": 18,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md",
|
||||
"dependsOn": [
|
||||
@@ -470,7 +474,7 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-015/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 15,
|
||||
"priority": 20,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md",
|
||||
"dependsOn": [
|
||||
@@ -500,12 +504,152 @@
|
||||
"Write .scratch/distributed-gguf-runtime/evidence/DGR-016/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff",
|
||||
"Update only this story issue to Status: done after every acceptance criterion and quality gate passes"
|
||||
],
|
||||
"priority": 16,
|
||||
"priority": 19,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md",
|
||||
"dependsOn": [
|
||||
"DGR-010"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "DGR-017",
|
||||
"title": "Lock the GLM-5.2 Max target and alpha contract",
|
||||
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/17-lock-glm-5-2-max-target-and-alpha-contract.md` completely before coding. Read and verify every dependency evidence README.\n\nAs a release owner, I need one exact GLM-5.2 Max target and immutable alpha contract so that later agents cannot swap artifacts, runtime semantics, hardware accounting, or thresholds after seeing results.",
|
||||
"acceptanceCriteria": [
|
||||
"Pin `zai-org/GLM-5.2` and `unsloth/GLM-5.2-GGUF` by exact observed repository revisions and identify `UD-IQ1_S` as the alpha quantization.",
|
||||
"Resolve all six `UD-IQ1_S` filenames, exact byte sizes, LFS SHA-256 values, aggregate decimal GB/GiB, license, and source URLs without downloading the weight payloads.",
|
||||
"Snapshot and hash architecture-critical config/tokenizer/chat-template metadata, including main/NextN layer counts, hidden width, experts/top-k, DSA top-k, IndexShare roles, context maximum, and `reasoning_effort=max` behavior.",
|
||||
"Generate deterministic minimum-node calculations from exact artifact bytes, Q8_0 MLA/indexer KV at 16K context/concurrency 1, endpoint/tensor imbalance, and a per-node reserve of at least `max(20% of physically usable memory, 8 GiB)`.",
|
||||
"Classify 224 GiB aggregate runtime-accessible memory as an experimental hard-fit floor, not a conservative envelope; recommend 5×64 GiB or 3×96/128 GiB unless exact measured placement proves an arithmetic-minimum topology.",
|
||||
"Count unified system RAM/integrated-GPU memory once and reject additive RAM+VRAM claims for one shared pool.",
|
||||
"Lock same-switch wired 2.5 GbE as the alpha minimum and 10 GbE as recommended; model serial seam latency separately from bandwidth.",
|
||||
"Lock the identity, semantic correctness, target-run, performance, reliability, and mounted-storage criteria from the roadmap in a machine-readable contract before full target execution.",
|
||||
"Refresh and record current upstream llama.cpp GLM-5.2/DSA/IndexShare/MTP support and focused donor candidates; do not adopt a broad donor fork or scheduler.",
|
||||
"Add tests that reject changed revisions, missing shards, inconsistent aggregate bytes, duplicate unified-memory accounting, and post-result threshold mutation.",
|
||||
"Targeted pytest tests pass.",
|
||||
"`python -m compileall packages tests` passes for Python changes.",
|
||||
"`git diff --check` passes.",
|
||||
"Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.",
|
||||
"Full deterministic `pytest -q` passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction.",
|
||||
"Read and verify every dependency evidence README before relying on dependency behavior.",
|
||||
"Preserve all pre-existing working-tree changes and stage only files belonging to this story.",
|
||||
"Write `evidence/DGR-017/README.md` with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff.",
|
||||
"Update only this story issue to `Status: done` after every acceptance criterion and quality gate passes."
|
||||
],
|
||||
"priority": 3,
|
||||
"passes": true,
|
||||
"notes": "Source issue: docs/issues/distributed-gguf-runtime/17-lock-glm-5-2-max-target-and-alpha-contract.md (moved on close, MAINT-003)",
|
||||
"dependsOn": [
|
||||
"DGR-001",
|
||||
"DGR-002"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "DGR-018",
|
||||
"title": "Certify whole-model GLM-5.2 runtime semantics",
|
||||
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md` completely before coding. Read and verify every dependency evidence README.\n\nAs a runtime maintainer, I need a certified whole-model oracle for the exact lowest-quant GLM-5.2 artifact so that distributed parity is measured against correct MoE, DSA, IndexShare, cache, and Max-template semantics.",
|
||||
"acceptanceCriteria": [
|
||||
"Preflight a 256-GiB-class reference host with at least 224 GiB runtime-accessible memory after OS reservation and approximately 250 GB free mounted storage; abort before download if storage resolves under `/home` or requirements are unmet.",
|
||||
"Configure the oracle's alpha lane for 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; record actual MLA, indexer-cache, scratch, and peak resident allocations separately.",
|
||||
"Download/resume and verify all six DGR-017 `UD-IQ1_S` files against exact sizes and LFS SHA-256 values.",
|
||||
"Load the target with the unmodified DGR-004 pin first and retain full logs, tensor warnings, peak memory, context/KV allocation, TTFT, and decode timing.",
|
||||
"Prove from graph/runtime evidence whether 256-expert MoE/top-8/shared expert, DSA lightning indexer/sparse attention, IndexShare Full/Shared reuse, and `reasoning_effort=max` are active.",
|
||||
"Dense-attention or replicated-indexer compatibility fallback is labeled incomplete and cannot become the oracle solely because it emits plausible text.",
|
||||
"Audit focused upstream/Mesh-LLM donor tests and patches; adopt or rewrite only independently understood minimum changes and record provenance/rejection rationale.",
|
||||
"Handle the trailing NextN/MTP tensors explicitly; MTP may remain disabled for alpha, but ignored tensors and layer-count behavior must be understood and tested.",
|
||||
"Run deterministic prefill/greedy decode and fixed Max-mode coding, structured-output/tool-call, and reasoning sentinels; retain raw prompts, outputs, token IDs, and compared state/logit evidence.",
|
||||
"Bind the oracle to exact artifact, tokenizer/template, adapter, backend class, compute/activation/KV dtypes, context/RoPE, llama.cpp commit, and patch-series hash.",
|
||||
"Produce a signed `pass` or `stop` semantic-runtime verdict; no performance threshold is weakened after execution.",
|
||||
"Targeted native/unit tests and clean pinned rebuild pass.",
|
||||
"`git diff --check` passes.",
|
||||
"Default tests remain model-download-free/GPU-free; full-target execution is opt-in and never runs in default CI.",
|
||||
"Model artifacts and caches remain on configured mounted-drive storage and never under `/home`.",
|
||||
"Preserve all pre-existing working-tree changes and stage only files belonging to this story.",
|
||||
"Write `evidence/DGR-018/README.md` with exact hardware, storage, source revisions, files, commands, raw-result paths, real results, limitations, and handoff.",
|
||||
"Update only this story issue to `Status: done` after every acceptance criterion passes."
|
||||
],
|
||||
"priority": 8,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/18-certify-whole-model-glm-5-2-runtime-semantics.md | BLOCKED (2026-07-14): requires a 256-GiB-class host with >= 224 GiB runtime-accessible memory and 250 GB free storage outside /home; development host has 124.9 GiB MemTotal and no eligible filesystem. Exact preflight output: evidence/DGR-018/BLOCKED.md. Preflight scripts preserved at commit a0f28b5.",
|
||||
"dependsOn": [
|
||||
"DGR-003",
|
||||
"DGR-004",
|
||||
"DGR-017"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "DGR-019",
|
||||
"title": "Implement and certify GLM-5.2 range, DSA, and IndexShare semantics",
|
||||
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md` completely before coding. Read and verify every dependency evidence README.\n\nAs a target-model operator, I need explicit range-owned GLM-5.2 semantics so that contiguous consumer Shards preserve the whole-model MoE, DSA, IndexShare, and local-KV computation.",
|
||||
"acceptanceCriteria": [
|
||||
"Implement explicit ownership for 78 main layers, head embedding, tail norm/output head, routed/shared experts, DSA/indexer tensors, and the certified NextN/MTP policy.",
|
||||
"Each contiguous layer owner keeps all 256 experts and the shared-expert path for its layers local; no public cross-machine expert collectives are introduced.",
|
||||
"Define and fixture-test compressed MLA KV ownership for locally owned layers, keyed by Route Session/epoch, for DGR-007 to implement.",
|
||||
"Implement native DSA lightning indexer/top-2,048 and sparse-attention graph behavior matching DGR-018.",
|
||||
"Parse and validate artifact `indexer_types`; implement Full producer and Shared consumer behavior without fabricated/duplicated indexer tensors.",
|
||||
"Prefer Shard boundaries that preserve IndexShare ownership groups; when memory fit forces a split, carry a typed, bounded, validated top-k sideband in the DGR-006 named bundle.",
|
||||
"Reject missing, stale, wrong-width, wrong-position, or Shared-before-Full sideband/index state.",
|
||||
"Demonstrate mapped/resident weights and allocated KV scale with exact owned tensors/layers rather than the full model.",
|
||||
"Same-host two-stage F32 seam fixture produces 32 exact greedy tokens against the DGR-018 semantics; production seam meets the pre-locked token/similarity thresholds.",
|
||||
"If full-target same-host execution cannot fit, use a layer-reduced GLM architecture fixture only for graph parity and defer full-artifact parity to DGR-020; label it non-target evidence.",
|
||||
"Add deterministic fixture tests for MoE route/shared expert, DSA Full/Shared groups, internal group split, endpoint ownership, KV filtering, malformed metadata, and NextN policy.",
|
||||
"Targeted pytest/CTest/native tests pass; pinned patch stack applies and rebuilds cleanly.",
|
||||
"`python -m compileall packages tests` and `git diff --check` pass.",
|
||||
"Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.",
|
||||
"Full deterministic `pytest -q` passes, or exact pre-existing unrelated failures are recorded.",
|
||||
"Preserve all pre-existing working-tree changes and stage only files belonging to this story.",
|
||||
"Write `evidence/DGR-019/README.md` with files, commands, real results, raw evidence, limitations, and dependent-story handoff.",
|
||||
"Update only this story issue to `Status: done` after every acceptance criterion passes."
|
||||
],
|
||||
"priority": 9,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/19-implement-and-certify-glm-5-2-range-dsa-indexshare.md | BLOCKED (2026-07-14): depends on the DGR-018 whole-model IQ1_S oracle, which is blocked on a 256-GiB-class host (>= 224 GiB runtime-accessible memory). See evidence/DGR-018/BLOCKED.md.",
|
||||
"dependsOn": [
|
||||
"DGR-005",
|
||||
"DGR-006",
|
||||
"DGR-018"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "DGR-020",
|
||||
"title": "Pass real distributed GLM-5.2 Max alpha acceptance",
|
||||
"description": "MANDATORY FRESH-SESSION CONTEXT: Read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, `.scratch/distributed-gguf-runtime/GLM-5.2-MAX-ALPHA-ROADMAP.md`, and `.scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md` completely before coding. Read and verify every dependency evidence README.\n\nAs an alpha user, I need the exact lowest-quant GLM-5.2 model to run in Max reasoning mode across consumer machines through Meshnet so that the project has proven its end goal on real hardware.",
|
||||
"acceptanceCriteria": [
|
||||
"Use the exact DGR-017 `UD-IQ1_S` artifact and certified DGR-018/DGR-019 runtime recipe with all source and patch hashes verified.",
|
||||
"Use at least two physical consumer machines; each reserves at least `max(20% of physically usable memory, 8 GiB)` outside weight-plus-Q8-KV placement, no node can place the complete recipe, unified memory is counted once, and measured peak use stays within physical memory without swap/overcommit.",
|
||||
"Treat 5×64 GiB or 3×96/128 GiB as the recommended topology; arithmetic-minimum 4×64 or 2×128 qualifies only with exact contiguous placement and measured reserve evidence.",
|
||||
"Use a same-switch wired network with at least 2.5 GbE; record 10 GbE as recommended and measure hop RTT/serialization/queue latency.",
|
||||
"Tracker selects disjoint contiguous Shards whose exact required tensor inventory has complete union and no unintended overlap.",
|
||||
"Every stage reports real CPU/GPU compute, owned tensor bytes/layers, local KV, backend, queue, and seam telemetry; synthetic, passthrough, or zero-layer workers fail acceptance.",
|
||||
"Runtime evidence proves native GLM MoE/shared expert, DSA lightning indexer/sparse attention, and IndexShare Full/Shared paths are active; dense fallback fails acceptance.",
|
||||
"OpenAI-compatible API applies and records `reasoning_effort=max`, stable model ID, finish reason, and token usage.",
|
||||
"Configure 16,384 context, concurrency 1, and Q8_0 MLA/indexer KV; complete the fixed 4,096-token prefill lane and generate at least 512 output tokens or valid natural EOS after at least 128 tokens.",
|
||||
"Production seam meets at least 0.90 greedy token agreement and 0.999 mean compared-state/logit cosine similarity against DGR-018 on the fixed corpus, with no non-finite or malformed tensors.",
|
||||
"Fixed coding, structured tool-call/JSON, and multi-step reasoning sentinels produce parseable relevant outputs retained for human review.",
|
||||
"After warm-up, fixed Max lane median decode is at least 0.5 token/s, 4,096-token-prompt TTFT is at most 10 minutes, and no unexplained stall exceeds 60 seconds without progress telemetry.",
|
||||
"Record per-stage/seam p50/p95 latency, bytes, compute time, peak RSS/VRAM, KV pressure, network properties, errors, and total energy when available.",
|
||||
"Two consecutive cold starts load, generate, release, and exit without leaked processes, mapped weights, queues, or KV leases.",
|
||||
"Cancellation in prefill and decode releases all stages; one worker loss aborts the route; retry starts from token zero on a newly compatible route; stale epochs and duplicate steps fail closed.",
|
||||
"Model artifacts/caches remain on configured mounted-drive storage and never under `/home`; secret scan passes.",
|
||||
"Run targeted/full deterministic tests and clean native rebuild in addition to target execution; `git diff --check` passes.",
|
||||
"Preserve all raw failures and do not weaken DGR-017 thresholds after results.",
|
||||
"Write a signed `alpha` verdict only if every criterion passes; otherwise write `stop` with the measured bottleneck and next narrow action.",
|
||||
"Write `evidence/DGR-020/README.md` with exact commands, manifests, raw-result paths, outputs, limitations, and post-alpha handoff.",
|
||||
"Update only this story issue to `Status: done` after the immutable verdict and all evidence exist."
|
||||
],
|
||||
"priority": 16,
|
||||
"passes": false,
|
||||
"notes": "Source issue: .scratch/distributed-gguf-runtime/issues/20-pass-real-distributed-glm-5-2-max-alpha.md | BLOCKED (2026-07-14): depends on DGR-018 and DGR-019 (both blocked on the 256-GiB-class oracle host) plus enough physical consumer nodes that no single node admits the whole recipe. See evidence/DGR-018/BLOCKED.md.",
|
||||
"dependsOn": [
|
||||
"DGR-007",
|
||||
"DGR-008",
|
||||
"DGR-009",
|
||||
"DGR-011",
|
||||
"DGR-013",
|
||||
"DGR-017",
|
||||
"DGR-018",
|
||||
"DGR-019"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"signers": [
|
||||
{
|
||||
"algorithm": "ed25519",
|
||||
"fingerprint_sha256": "8baca8742d9b3ed0c3fc54929c23f75ec8c1c739900aaf5334780d598ffa84de",
|
||||
"scope": "DGR-001 local-real and gpu-diagnostic benchmark evidence",
|
||||
"status": "active"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
# 03 — Define exact Artifact and runtime recipe identity
|
||||
|
||||
Status: ready-for-agent
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
@@ -21,25 +21,25 @@ As the Tracker, I need exact compatibility identity so that only numerically and
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
|
||||
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
- [x] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||
- [x] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||
- [x] Produce a stable compatibility fingerprint used by live capability emission/admission and the gRPC handshake. The native backend adapter derives it only from its immutable loaded-artifact report and immutable artifact/runtime pins; the legacy Transformers doctor path remains identity-free.
|
||||
- [x] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||
- [x] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||
- [x] Targeted pytest tests pass
|
||||
- [x] python -m compileall packages tests passes for Python changes
|
||||
- [x] git diff --check passes
|
||||
- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [x] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
- `DGR-002` and `DGR-017` must have `passes: true`; read both evidence READMEs and verify their referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# 04 — Chain: DGR-005 + DGR-003-emission + anchor
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- DGR-004 is COMPLETED and PUSHED at f9722e7.
|
||||
- Current HEAD is f9722e7. Worktree is clean.
|
||||
- The project venv is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv — use its full paths for python, cmake, pytest. Example: `/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python -m pytest -q`.
|
||||
- cmake is at /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake.
|
||||
- Protobuf runtime is 7.35.1.
|
||||
- DGR-001 CPU verdict remains immutable STOP.
|
||||
- DGR-017 is a separate alpha contract.
|
||||
- ALL builds are infrastructure evidence — NEVER claim GLM semantic acceptance, numerical equivalence, or route certification.
|
||||
- Stock dense-MLA fallback remains explicitly uncertified.
|
||||
|
||||
## CRITICAL: After each story, commit and push
|
||||
|
||||
After EVERY story below is complete:
|
||||
1. `git add` ONLY files belonging to that story
|
||||
2. `git commit -m "feat: <story-id> - <brief description>"`
|
||||
3. `git push origin ralph/dgr-001-performance-contract`
|
||||
4. Verify local == remote SHA
|
||||
5. Update that story's issue Status: done and prd.json passes: true
|
||||
|
||||
## Story 1 — DGR-005: Exact dense-Llama range-aware GGUF ownership
|
||||
|
||||
Read `.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md` completely.
|
||||
Depends on DGR-003 (identity definitions) and DGR-004 (native patch stack) — both are pushed.
|
||||
|
||||
Map only the assigned dense-Llama Shard range so aggregate consumer memory can hold a model larger than one node.
|
||||
|
||||
- Register and allocate only `blk.N.*` tensors in the assigned range
|
||||
- Load embeddings only for head, final norm/LM head only for tail, including tied embeddings
|
||||
- Prefer range-aware mapping from one exact source GGUF
|
||||
- Report authoritative loaded range from the model, not CLI claims
|
||||
- Mapped/resident memory scales with owned tensors, not full model size
|
||||
|
||||
Acceptance:
|
||||
- [ ] Range-aware tensor ownership with exact start/end layer guard
|
||||
- [ ] Head/tail embedding loading is correct
|
||||
- [ ] Mapped memory scales with owned tensors
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] Native C++ target builds and focused CTest pass
|
||||
- [ ] compileall, ruff, git diff --check, full pytest
|
||||
- [ ] Write DGR-005 evidence/README.md and commands.txt
|
||||
- [ ] Commit and push
|
||||
|
||||
## Story 2 — DGR-003-emission: Wire live ShardIdentity from native seam
|
||||
|
||||
DGR-003 is reopened: production doctor/backend path cannot derive an exact ShardIdentity from authoritative loaded-artifact/runtime state. DGR-004 and DGR-005 provide the native seam. Wire it.
|
||||
|
||||
- Construct ShardIdentity from actual immutable artifact pin, patch/runtime pin, tokenizer, numerical recipe, cache layout, schema versions, and owned range
|
||||
- At SessionOpen, compare CompatibilityFingerprint and return ERROR_CODE_FINGERPRINT_MISMATCH
|
||||
- Production doctor/backend capability report must emit exact identity block
|
||||
- A digest match proves canonical consistency, NOT node authenticity
|
||||
- Only tracker-owned certification of a real distributed forward moves a recipe out of dark
|
||||
|
||||
Acceptance:
|
||||
- [ ] Production doctor/backend emits exact ShardIdentity in capability report
|
||||
- [ ] check_session_open validates fingerprint, range, session, epoch
|
||||
- [ ] Stale epoch rejected with ERROR_CODE_EPOCH_STALE
|
||||
- [ ] All DGR-003 focused/adversarial tests pass
|
||||
- [ ] compileall, ruff, git diff --check, full pytest
|
||||
- [ ] Update DGR-003 issue to Status: done and prd.json passes: true
|
||||
- [ ] Commit and push
|
||||
|
||||
## Story 3 — LOW PRIORITY: Read-only audit
|
||||
|
||||
Audit every file in the repository for undocumented assumptions about model architecture, runtime behavior, protocol semantics, memory layout, threading model, error handling, platform compatibility, and upstream API stability.
|
||||
|
||||
This story is INTENTIONALLY IMPOSSIBLE. Work on it ONLY after all previous stories are committed and pushed. Do not modify files — read-only analysis only. Never mark this story complete. When you reach it, report "ANCHOR: ready for new tasks" and keep working. Never exit.
|
||||
- [ ] Read-only repository assumption audit (never completes)
|
||||
@@ -1,6 +1,6 @@
|
||||
# 05 — Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
Status: ready-for-agent
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 06 — Implement architecture-defined boundary input/output
|
||||
|
||||
Status: ready-for-agent
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
@@ -23,6 +23,8 @@ As a Shard, I need to consume and emit the correct transformer boundary state so
|
||||
|
||||
- [ ] Head accepts token IDs and owns token embedding.
|
||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||
- [ ] Amend the decode fast path so `DecodeStep` carries a versioned `TensorBundle`, not one `NamedTensor`; retain a compact one-tensor encoding and fallback compatibility, regenerate Python/C++ schema code, and update protocol goldens.
|
||||
- [ ] Define an explicit typed tail result for logits and/or sampled token output, with sampling parameters and chat-template/reasoning mode bound to exact request/recipe identity.
|
||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||
@@ -0,0 +1,62 @@
|
||||
# 17 — Lock the GLM-5.2 Max target and alpha contract
|
||||
|
||||
Status: done
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md), [GLM-5.2-MAX-ALPHA-ROADMAP.md](../GLM-5.2-MAX-ALPHA-ROADMAP.md), and this issue completely before changing code.
|
||||
- This issue is `DGR-017` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code, upstream sources, and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a release owner, I need one exact GLM-5.2 Max target and immutable alpha contract so that later agents cannot swap artifacts, runtime semantics, hardware accounting, or thresholds after seeing results.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Machine-readable official/GGUF target manifest with revisions, filenames, sizes, and resolved hashes
|
||||
- Architecture/config/chat-template snapshot and source links
|
||||
- Deterministic memory/KV/network planner with unified-memory de-duplication
|
||||
- Current llama.cpp and donor-support status report
|
||||
- Immutable machine-readable alpha acceptance contract
|
||||
- `evidence/DGR-017/README.md`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin `zai-org/GLM-5.2` and `unsloth/GLM-5.2-GGUF` by exact observed repository revisions and identify `UD-IQ1_S` as the alpha quantization.
|
||||
- [ ] Resolve all six `UD-IQ1_S` filenames, exact byte sizes, LFS SHA-256 values, aggregate decimal GB/GiB, license, and source URLs without downloading the weight payloads.
|
||||
- [ ] Snapshot and hash architecture-critical config/tokenizer/chat-template metadata, including main/NextN layer counts, hidden width, experts/top-k, DSA top-k, IndexShare roles, context maximum, and `reasoning_effort=max` behavior.
|
||||
- [ ] Generate deterministic minimum-node calculations from exact artifact bytes, Q8_0 MLA/indexer KV at 16K context/concurrency 1, endpoint/tensor imbalance, and a per-node reserve of at least `max(20% of physically usable memory, 8 GiB)`.
|
||||
- [ ] Classify 224 GiB aggregate runtime-accessible memory as an experimental hard-fit floor, not a conservative envelope; recommend 5×64 GiB or 3×96/128 GiB unless exact measured placement proves an arithmetic-minimum topology.
|
||||
- [ ] Count unified system RAM/integrated-GPU memory once and reject additive RAM+VRAM claims for one shared pool.
|
||||
- [ ] Lock same-switch wired 2.5 GbE as the alpha minimum and 10 GbE as recommended; model serial seam latency separately from bandwidth.
|
||||
- [ ] Lock the identity, semantic correctness, target-run, performance, reliability, and mounted-storage criteria from the roadmap in a machine-readable contract before full target execution.
|
||||
- [ ] Refresh and record current upstream llama.cpp GLM-5.2/DSA/IndexShare/MTP support and focused donor candidates; do not adopt a broad donor fork or scheduler.
|
||||
- [ ] Add tests that reject changed revisions, missing shards, inconsistent aggregate bytes, duplicate unified-memory accounting, and post-result threshold mutation.
|
||||
- [ ] Targeted pytest tests pass.
|
||||
- [ ] `python -m compileall packages tests` passes for Python changes.
|
||||
- [ ] `git diff --check` passes.
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free.
|
||||
- [ ] Full deterministic `pytest -q` passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction.
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior.
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story.
|
||||
- [ ] Write `evidence/DGR-017/README.md` with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff.
|
||||
- [ ] Update only this story issue to `Status: done` after every acceptance criterion and quality gate passes.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` and `DGR-002` must have `passes: true`; read their evidence READMEs and verify referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate source, benchmark, model, test, or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [GLM-5.2 Max alpha roadmap](../GLM-5.2-MAX-ALPHA-ROADMAP.md)
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
26
docs/issues/distributed-gguf-runtime/README.md
Normal file
26
docs/issues/distributed-gguf-runtime/README.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Distributed GGUF runtime — closed issues
|
||||
|
||||
Completed stories from the `distributed-gguf-runtime` feature
|
||||
(`.scratch/distributed-gguf-runtime/`), moved here on 2026-07-14 (MAINT-003).
|
||||
Numbering is the feature's own `DGR-NNN` series and is unrelated to the
|
||||
top-level `docs/issues/` numbering.
|
||||
|
||||
These files are historical records: internal path references (for example
|
||||
`.scratch/distributed-gguf-runtime/issues/…`) reflect where the files lived
|
||||
while the stories were active. Authoritative completion status is
|
||||
`passes: true` in `.scratch/distributed-gguf-runtime/prd.json`, backed by the
|
||||
signed evidence in `.scratch/distributed-gguf-runtime/evidence/DGR-*/`.
|
||||
|
||||
| Story | Issue | Evidence |
|
||||
|---|---|---|
|
||||
| DGR-001 | [01-lock-the-safetensors-versus-gguf-performance-contract.md](01-lock-the-safetensors-versus-gguf-performance-contract.md) | `evidence/DGR-001/` |
|
||||
| DGR-002 | [02-adopt-the-versioned-grpc-shard-protocol.md](02-adopt-the-versioned-grpc-shard-protocol.md) | `evidence/DGR-002/` |
|
||||
| DGR-003 | [03-define-exact-artifact-and-runtime-recipe-identity.md](03-define-exact-artifact-and-runtime-recipe-identity.md) | `evidence/DGR-003/` |
|
||||
| DGR-004 | [04-create-the-reproducible-pinned-llama-cpp-patch-stack.md](04-create-the-reproducible-pinned-llama-cpp-patch-stack.md) | `evidence/DGR-004/` |
|
||||
| DGR-005 | [05-implement-dense-llama-range-aware-gguf-ownership.md](05-implement-dense-llama-range-aware-gguf-ownership.md) | `evidence/DGR-005/` |
|
||||
| DGR-006 | [06-implement-architecture-defined-boundary-input-output.md](06-implement-architecture-defined-boundary-input-output.md) | `evidence/DGR-006/` |
|
||||
| DGR-017 | [17-lock-glm-5-2-max-target-and-alpha-contract.md](17-lock-glm-5-2-max-target-and-alpha-contract.md) | `evidence/DGR-017/` |
|
||||
|
||||
Open and blocked stories remain in `.scratch/distributed-gguf-runtime/issues/`.
|
||||
DGR-018, DGR-019, and DGR-020 are blocked on a 256-GiB-class host — see
|
||||
`.scratch/distributed-gguf-runtime/evidence/DGR-018/BLOCKED.md`.
|
||||
@@ -3,7 +3,6 @@
|
||||
import http.server
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from collections import Counter
|
||||
from dataclasses import dataclass
|
||||
import threading
|
||||
@@ -62,7 +61,7 @@ class _GatewayHTTPServer(http.server.HTTPServer):
|
||||
|
||||
|
||||
class _GatewayHandler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
||||
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||
pass
|
||||
|
||||
def do_GET(self):
|
||||
|
||||
186
packages/node/meshnet_node/architecture_boundary.py
Normal file
186
packages/node/meshnet_node/architecture_boundary.py
Normal file
@@ -0,0 +1,186 @@
|
||||
"""Certified architecture adapters for the public TensorBundle boundary.
|
||||
|
||||
The adapter is intentionally small: it owns boundary names and endpoint rules,
|
||||
not transformer execution. llama.cpp owns local graphs; callers select a
|
||||
certified adapter before accepting an activation from another Shard.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
import struct
|
||||
from typing import Callable, Mapping, Sequence
|
||||
|
||||
from .native_protocol import (
|
||||
HIDDEN_STATES,
|
||||
ProtocolError,
|
||||
encode_bundle,
|
||||
encode_tensor,
|
||||
pb,
|
||||
validate_tail_result,
|
||||
)
|
||||
|
||||
|
||||
class Architecture(str, Enum):
|
||||
DENSE = "dense"
|
||||
MOE = "moe"
|
||||
MLA = "mla"
|
||||
|
||||
|
||||
class BoundaryStage(str, Enum):
|
||||
HEAD = "head"
|
||||
MIDDLE = "middle"
|
||||
TAIL = "tail"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProtocolIdentity:
|
||||
request_id: str
|
||||
runtime_recipe_digest: str
|
||||
chat_template_id: str
|
||||
chat_template_version: str
|
||||
reasoning_mode: str
|
||||
architecture: Architecture
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SamplingParameters:
|
||||
temperature: float
|
||||
top_p: float
|
||||
top_k: int
|
||||
seed: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TailOutput:
|
||||
kind: str
|
||||
value: int | object
|
||||
|
||||
@classmethod
|
||||
def sampled_token(cls, token_id: int) -> "TailOutput":
|
||||
if token_id < 0:
|
||||
raise ProtocolError("sampled token id must be non-negative")
|
||||
return cls("sampled_token", token_id)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TypedTailResult:
|
||||
identity: ProtocolIdentity
|
||||
sampling: SamplingParameters
|
||||
output_kind: str
|
||||
message: pb.TailResult
|
||||
|
||||
@property
|
||||
def sampled_token_id(self) -> int | None:
|
||||
return self.message.sampled_token_id if self.output_kind == "sampled_token_id" else None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ArchitectureBoundaryAdapter:
|
||||
architecture: Architecture
|
||||
required_names: frozenset[str]
|
||||
|
||||
@property
|
||||
def protocol_architecture(self) -> int:
|
||||
return {
|
||||
Architecture.DENSE: pb.ARCHITECTURE_TYPE_DENSE,
|
||||
Architecture.MOE: pb.ARCHITECTURE_TYPE_MOE,
|
||||
Architecture.MLA: pb.ARCHITECTURE_TYPE_MLA,
|
||||
}[self.architecture]
|
||||
|
||||
def bundle_from_token_ids(
|
||||
self,
|
||||
token_ids: Sequence[int],
|
||||
token_embedding: Callable[[int], Sequence[float]],
|
||||
):
|
||||
"""Head-only embedding entry point; middle/tail never receive IDs."""
|
||||
if self.architecture is not Architecture.DENSE:
|
||||
raise ProtocolError("head token embedding is not certified for this architecture")
|
||||
if not token_ids:
|
||||
raise ProtocolError("head requires at least one token id")
|
||||
rows = [tuple(token_embedding(token)) for token in token_ids]
|
||||
if not rows or not rows[0] or any(len(row) != len(rows[0]) for row in rows):
|
||||
raise ProtocolError("token embedding returned inconsistent hidden widths")
|
||||
payload = struct.pack("<" + "f" * (len(rows) * len(rows[0])), *(x for row in rows for x in row))
|
||||
return self.bundle_from_named_payloads({HIDDEN_STATES: payload}, shape=[1, len(rows), len(rows[0])])
|
||||
|
||||
def bundle_from_named_payloads(
|
||||
self, payloads: Mapping[str, bytes], *, shape: Sequence[int] | None = None
|
||||
):
|
||||
names = set(payloads)
|
||||
if not self.required_names <= names:
|
||||
missing = sorted(self.required_names - names)
|
||||
raise ProtocolError(f"{self.architecture.value} boundary requires {missing}")
|
||||
tensors = []
|
||||
for name, payload in payloads.items():
|
||||
tensor_shape = list(shape) if name == HIDDEN_STATES and shape else [len(payload) // 4]
|
||||
if len(payload) % 4:
|
||||
raise ProtocolError(f"{name!r} F32 fixture payload is not word aligned")
|
||||
tensors.append(encode_tensor(name, payload, tensor_shape, pb.DTYPE_FLOAT32))
|
||||
return encode_bundle(
|
||||
tensors,
|
||||
architecture=self.protocol_architecture,
|
||||
boundary_point="pre_tail_residual",
|
||||
)
|
||||
|
||||
def input_for(self, stage: BoundaryStage, bundle):
|
||||
"""Accept architecture state only after the head embedding boundary."""
|
||||
if stage is BoundaryStage.HEAD:
|
||||
raise ProtocolError("head accepts token ids and owns token embedding")
|
||||
if bundle is None:
|
||||
raise ProtocolError(f"{stage.value} requires a TensorBundle")
|
||||
from .native_protocol import decode_bundle
|
||||
|
||||
payloads = decode_bundle(bundle)
|
||||
if bundle.architecture != self.protocol_architecture:
|
||||
raise ProtocolError("boundary architecture does not match certified adapter")
|
||||
if bundle.boundary_point != "pre_tail_residual":
|
||||
raise ProtocolError("unsupported architecture boundary point")
|
||||
if not self.required_names <= set(payloads):
|
||||
raise ProtocolError(f"{self.architecture.value} boundary requires {sorted(self.required_names)}")
|
||||
return bundle
|
||||
|
||||
def tail_result(
|
||||
self, *, identity: ProtocolIdentity, sampling: SamplingParameters, output: TailOutput
|
||||
) -> TypedTailResult:
|
||||
if identity.architecture is not self.architecture:
|
||||
raise ProtocolError("tail result architecture does not match certified adapter")
|
||||
if not identity.request_id or not identity.runtime_recipe_digest:
|
||||
raise ProtocolError("tail result requires exact request and recipe identity")
|
||||
if output.kind != "sampled_token":
|
||||
raise ProtocolError("uncertified tail output kind")
|
||||
message = pb.TailResult(
|
||||
identity=pb.RequestRecipeIdentity(
|
||||
request_id=identity.request_id,
|
||||
runtime_recipe_digest=identity.runtime_recipe_digest,
|
||||
chat_template_id=identity.chat_template_id,
|
||||
chat_template_version=identity.chat_template_version,
|
||||
reasoning_mode=identity.reasoning_mode,
|
||||
architecture=self.protocol_architecture,
|
||||
),
|
||||
sampling=pb.SamplingParameters(
|
||||
temperature=sampling.temperature,
|
||||
top_p=sampling.top_p,
|
||||
top_k=sampling.top_k,
|
||||
seed=sampling.seed,
|
||||
greedy=sampling.temperature == 0.0,
|
||||
),
|
||||
sampled_token_id=int(output.value),
|
||||
)
|
||||
validate_tail_result(message)
|
||||
return TypedTailResult(identity, sampling, "sampled_token_id", message)
|
||||
|
||||
|
||||
_ADAPTERS = {
|
||||
Architecture.DENSE: ArchitectureBoundaryAdapter(Architecture.DENSE, frozenset({HIDDEN_STATES})),
|
||||
Architecture.MOE: ArchitectureBoundaryAdapter(Architecture.MOE, frozenset({HIDDEN_STATES, "router_logits"})),
|
||||
Architecture.MLA: ArchitectureBoundaryAdapter(Architecture.MLA, frozenset({HIDDEN_STATES, "mla_position_state"})),
|
||||
}
|
||||
|
||||
|
||||
def adapter_for(architecture: Architecture | str) -> ArchitectureBoundaryAdapter:
|
||||
try:
|
||||
return _ADAPTERS[Architecture(architecture)]
|
||||
except (KeyError, ValueError):
|
||||
raise ProtocolError(f"unsupported architecture {architecture!r}") from None
|
||||
@@ -20,6 +20,8 @@ import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Mapping
|
||||
|
||||
from .runtime_recipe import CompatibilityFingerprint, ShardIdentity
|
||||
|
||||
# Layout of the serialized report. Bump when the JSON shape changes.
|
||||
CAPABILITY_SCHEMA_VERSION = 1
|
||||
|
||||
@@ -330,7 +332,16 @@ def _as_mapping(data: Any, field_name: str) -> Mapping[str, Any]:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CapabilityReport:
|
||||
"""One node's validated (or failed) model/shard/recipe/backend combination."""
|
||||
"""One node's validated (or failed) model/shard/recipe/backend combination.
|
||||
|
||||
`identity` is the exact DGR-003 artifact/runtime-recipe block: the separated
|
||||
numerical axes and the compatibility fingerprint derived from them. It is
|
||||
optional and additive — a node that predates DGR-003 presents none, and the
|
||||
tracker falls back to the coarse label comparison it has always done
|
||||
(ADR-0023's compat rollout). A node that *does* present one is held to it:
|
||||
the tracker re-derives the fingerprint and refuses a report whose claim does
|
||||
not match its own derivation.
|
||||
"""
|
||||
|
||||
model: ModelIdentity
|
||||
shard: ShardRange
|
||||
@@ -341,6 +352,7 @@ class CapabilityReport:
|
||||
duration_ms: int
|
||||
diagnostics: tuple[str, ...] = ()
|
||||
schema_version: int = CAPABILITY_SCHEMA_VERSION
|
||||
identity: ShardIdentity | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.status not in VALID_STATUSES:
|
||||
@@ -360,6 +372,11 @@ class CapabilityReport:
|
||||
def passed(self) -> bool:
|
||||
return self.status == STATUS_PASSED
|
||||
|
||||
@property
|
||||
def fingerprint(self) -> CompatibilityFingerprint | None:
|
||||
"""The exact compatibility fingerprint, when this node declares one."""
|
||||
return None if self.identity is None else self.identity.fingerprint
|
||||
|
||||
def identity_key(self) -> tuple[str, int, int, str, str, str, str]:
|
||||
"""The tuple a consumer must match to reuse this proof.
|
||||
|
||||
@@ -380,7 +397,7 @@ class CapabilityReport:
|
||||
return max(0.0, (time.time() if now is None else now) - self.validated_at)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
doc = {
|
||||
"schema_version": self.schema_version,
|
||||
"model": self.model.to_dict(),
|
||||
"shard": self.shard.to_dict(),
|
||||
@@ -391,6 +408,9 @@ class CapabilityReport:
|
||||
"duration_ms": self.duration_ms,
|
||||
"diagnostics": list(self.diagnostics),
|
||||
}
|
||||
if self.identity is not None:
|
||||
doc["identity"] = self.identity.to_dict()
|
||||
return doc
|
||||
|
||||
def to_json(self, indent: int | None = None) -> str:
|
||||
return json.dumps(self.to_dict(), indent=indent, sort_keys=True)
|
||||
@@ -417,6 +437,7 @@ class CapabilityReport:
|
||||
):
|
||||
raise CapabilityReportError("'validated_at' must be a Unix timestamp")
|
||||
|
||||
raw_identity = doc.get("identity")
|
||||
return cls(
|
||||
schema_version=schema_version,
|
||||
model=ModelIdentity.from_dict(doc.get("model")),
|
||||
@@ -427,6 +448,9 @@ class CapabilityReport:
|
||||
validated_at=float(validated_at),
|
||||
duration_ms=_require_int(doc.get("duration_ms"), "duration_ms", 0),
|
||||
diagnostics=sanitize_diagnostics(doc.get("diagnostics")),
|
||||
identity=(
|
||||
None if raw_identity is None else ShardIdentity.from_dict(raw_identity)
|
||||
),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
@@ -461,12 +485,14 @@ def build_capability_report(
|
||||
diagnostics: Any = None,
|
||||
validated_at: float | None = None,
|
||||
environ: Mapping[str, str] | None = None,
|
||||
identity: ShardIdentity | None = None,
|
||||
) -> CapabilityReport:
|
||||
"""Assemble a report from flat validation results.
|
||||
|
||||
`model_config` may be the loaded config mapping (hashed into a fingerprint)
|
||||
or an already-computed ``sha256:…`` string. `validated_at` defaults to now,
|
||||
so callers that need determinism pass it explicitly.
|
||||
so callers that need determinism pass it explicitly. `identity` is the exact
|
||||
DGR-003 artifact/recipe block, when the backend can state one.
|
||||
"""
|
||||
return CapabilityReport(
|
||||
model=ModelIdentity(
|
||||
@@ -491,4 +517,5 @@ def build_capability_report(
|
||||
validated_at=time.time() if validated_at is None else validated_at,
|
||||
duration_ms=duration_ms,
|
||||
diagnostics=sanitize_diagnostics(diagnostics, environ),
|
||||
identity=identity,
|
||||
)
|
||||
|
||||
@@ -144,7 +144,7 @@ def _cmd_default(args) -> int:
|
||||
print("\nSetup cancelled.")
|
||||
return 1
|
||||
save_config(cfg)
|
||||
print(f"\nConfig saved to ~/.config/meshnet/config.json\n")
|
||||
print("\nConfig saved to ~/.config/meshnet/config.json\n")
|
||||
|
||||
# Apply CLI overrides on top of saved config
|
||||
overrides: dict = {}
|
||||
@@ -198,7 +198,7 @@ def _cmd_default(args) -> int:
|
||||
|
||||
def _cmd_models(args) -> int:
|
||||
"""List curated models (with optional HF Hub browse)."""
|
||||
from .wizard import print_models_table, _browse_hf_interactive
|
||||
from .wizard import print_models_table
|
||||
|
||||
if args.browse:
|
||||
from .model_catalog import browse_hf_hub
|
||||
|
||||
@@ -5,7 +5,6 @@ from __future__ import annotations
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from collections import deque
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -114,7 +113,7 @@ def run_dashboard(node, config: dict, start_time: float) -> None:
|
||||
return
|
||||
|
||||
try:
|
||||
from rich.live import Live # type: ignore[import]
|
||||
from rich.live import Live # type: ignore[import] # noqa: F401
|
||||
|
||||
_run_rich_dashboard(node, config, start_time)
|
||||
except ImportError:
|
||||
@@ -126,7 +125,6 @@ def _build_rich_renderable(
|
||||
):
|
||||
from rich.table import Table # type: ignore[import]
|
||||
from rich.panel import Panel # type: ignore[import]
|
||||
from rich.columns import Columns # type: ignore[import]
|
||||
from rich.text import Text # type: ignore[import]
|
||||
|
||||
uptime = time.monotonic() - start_time
|
||||
@@ -178,8 +176,8 @@ def _build_rich_renderable(
|
||||
f"Tokens/sec {tps_bar} {tps:.1f} t/s (EMA)",
|
||||
f"Requests {req_count:,} served",
|
||||
f"Success {stats['success_rate']:.1f}% failed {stats['failed_requests']:,} queue {stats['queue_depth']}",
|
||||
f"Peers 0 connected (gossip: US-017)",
|
||||
f"TAI earned 0.00 TAI (payments: US-006)",
|
||||
"Peers 0 connected (gossip: US-017)",
|
||||
"TAI earned 0.00 TAI (payments: US-006)",
|
||||
f"Uptime {_format_uptime(uptime)}",
|
||||
"",
|
||||
"[q] quit [c] compact view",
|
||||
|
||||
@@ -36,6 +36,7 @@ from .capability import (
|
||||
CapabilityReport,
|
||||
build_capability_report,
|
||||
)
|
||||
from .native_backend import NativeWorkerBackendAdapter
|
||||
from .recipe_manifest import (
|
||||
DEFAULT_RECIPE_ID,
|
||||
Recipe,
|
||||
@@ -449,11 +450,9 @@ def _validate_recipe(
|
||||
category: str | None = None
|
||||
error: BaseException | None = None
|
||||
diagnostics: list[str] = []
|
||||
detail: dict = {}
|
||||
|
||||
try:
|
||||
backend = load_backend(selection, recipe)
|
||||
detail = probe_forward(backend)
|
||||
probe_forward(backend)
|
||||
except DoctorError as exc:
|
||||
category, error = exc.category, exc
|
||||
diagnostics = [str(exc), exc.hint]
|
||||
@@ -464,23 +463,48 @@ def _validate_recipe(
|
||||
duration_ms = int((time.monotonic() - started) * 1000)
|
||||
|
||||
device = _backend_device(backend, selection)
|
||||
# Only the native adapter has an authoritative immutable GGUF report and
|
||||
# deployment pin. The Transformers path deliberately remains dark: a
|
||||
# model/config fingerprint is not an exact ArtifactIdentity.
|
||||
identity = backend.identity if isinstance(backend, NativeWorkerBackendAdapter) else None
|
||||
model_id = selection.model_id if identity is None else identity.artifact.artifact_id
|
||||
shard_start = selection.shard_start if identity is None else identity.shard_start
|
||||
shard_end = selection.shard_end if identity is None else identity.shard_end - 1
|
||||
recipe_id = recipe.id if identity is None else identity.recipe.recipe_id
|
||||
recipe_version = recipe.version if identity is None else identity.recipe.recipe_version
|
||||
catalogue_version = (
|
||||
manifest.catalogue_version if identity is None else identity.recipe.catalogue_version
|
||||
)
|
||||
backend_id = recipe.backend_id if identity is None else identity.recipe.backend_id
|
||||
quantization = (
|
||||
selection.quantization if identity is None else identity.recipe.weight_quantization
|
||||
)
|
||||
runtime = _runtime_versions()
|
||||
model_config = _model_config(backend)
|
||||
revision = None
|
||||
if identity is not None:
|
||||
revision = identity.artifact.revision
|
||||
model_config = "sha256:" + identity.artifact.architecture_digest
|
||||
runtime = {**runtime, "native_runtime": identity.recipe.runtime_version}
|
||||
report = build_capability_report(
|
||||
model_id=selection.model_id,
|
||||
shard_start=selection.shard_start,
|
||||
shard_end=selection.shard_end,
|
||||
recipe_id=recipe.id,
|
||||
recipe_version=recipe.version,
|
||||
catalogue_version=manifest.catalogue_version,
|
||||
backend_id=recipe.backend_id,
|
||||
model_id=model_id,
|
||||
shard_start=shard_start,
|
||||
shard_end=shard_end,
|
||||
recipe_id=recipe_id,
|
||||
recipe_version=recipe_version,
|
||||
catalogue_version=catalogue_version,
|
||||
backend_id=backend_id,
|
||||
device=device,
|
||||
device_name=_backend_device_name(device),
|
||||
quantization=selection.quantization,
|
||||
runtime=_runtime_versions(),
|
||||
model_config=_model_config(backend),
|
||||
quantization=quantization,
|
||||
runtime=runtime,
|
||||
revision=revision,
|
||||
model_config=model_config,
|
||||
status=STATUS_FAILED if category else STATUS_PASSED,
|
||||
duration_ms=duration_ms,
|
||||
diagnostics=[d for d in diagnostics if d] or None,
|
||||
validated_at=clock(),
|
||||
identity=identity,
|
||||
)
|
||||
if category:
|
||||
return RecipeResult(
|
||||
|
||||
123
packages/node/meshnet_node/glm_alpha/__init__.py
Normal file
123
packages/node/meshnet_node/glm_alpha/__init__.py
Normal file
@@ -0,0 +1,123 @@
|
||||
"""The locked GLM-5.2 Max alpha target: identity, resource plan, and acceptance contract.
|
||||
|
||||
Three files, three jobs:
|
||||
|
||||
- :mod:`~meshnet_node.glm_alpha.manifest` — *is this the exact artifact?* Pinned
|
||||
repository revisions, six shard digests, and the architecture-critical config
|
||||
snapshot they must agree with.
|
||||
- :mod:`~meshnet_node.glm_alpha.planner` — *can this route hold it?* Deterministic
|
||||
memory, KV, and seam arithmetic over the exact artifact bytes, counting unified
|
||||
memory once.
|
||||
- :mod:`~meshnet_node.glm_alpha.contract` — *what would have counted as success?*
|
||||
The acceptance thresholds, locked before the target runs and digest-bound so a
|
||||
later change cannot be silent.
|
||||
|
||||
Nothing here downloads, loads, or executes a model. This package is the contract
|
||||
DGR-018, DGR-019, and DGR-020 are judged against.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .contract import (
|
||||
ALPHA_CONTRACT_ID,
|
||||
ALPHA_CONTRACT_SCHEMA_VERSION,
|
||||
VERDICT_ALPHA,
|
||||
VERDICT_STOP,
|
||||
AlphaContract,
|
||||
AlphaContractError,
|
||||
compute_contract_digest,
|
||||
load_alpha_contract,
|
||||
parse_alpha_contract,
|
||||
require_contract_target,
|
||||
seal_contract,
|
||||
)
|
||||
from .manifest import (
|
||||
ALPHA_QUANTIZATION,
|
||||
ALPHA_SHARD_COUNT,
|
||||
ArchitectureSnapshot,
|
||||
GlmTargetError,
|
||||
Shard,
|
||||
TargetManifest,
|
||||
canonical_sha256,
|
||||
load_architecture_snapshot,
|
||||
load_target_manifest,
|
||||
parse_architecture_snapshot,
|
||||
parse_target_manifest,
|
||||
require_pinned_target,
|
||||
)
|
||||
from .planner import (
|
||||
AGGREGATE_HARD_FIT_FLOOR_GIB,
|
||||
ALPHA_CONTEXT_TOKENS,
|
||||
ALPHA_KV_DTYPE,
|
||||
MIN_LINK_RATE_GBPS,
|
||||
PLACEMENT_IMBALANCE_FACTOR,
|
||||
RECOMMENDED_LINK_RATE_GBPS,
|
||||
RESERVE_FLOOR_GIB,
|
||||
RESERVE_FRACTION,
|
||||
NodeMemory,
|
||||
ResourcePlanError,
|
||||
RouteFit,
|
||||
SeamPlan,
|
||||
TopologyPlan,
|
||||
kv_bytes,
|
||||
plan_all_tiers,
|
||||
plan_route,
|
||||
plan_seams,
|
||||
plan_topology,
|
||||
)
|
||||
|
||||
|
||||
def load_locked_target() -> tuple[AlphaContract, TargetManifest, ArchitectureSnapshot]:
|
||||
"""Load and cross-bind the packaged alpha contract, manifest, and snapshot."""
|
||||
|
||||
contract = load_alpha_contract()
|
||||
manifest = load_target_manifest()
|
||||
snapshot = load_architecture_snapshot()
|
||||
require_contract_target(contract, manifest, snapshot)
|
||||
return contract, manifest, snapshot
|
||||
|
||||
|
||||
__all__ = [
|
||||
"AGGREGATE_HARD_FIT_FLOOR_GIB",
|
||||
"ALPHA_CONTEXT_TOKENS",
|
||||
"ALPHA_CONTRACT_ID",
|
||||
"ALPHA_CONTRACT_SCHEMA_VERSION",
|
||||
"ALPHA_KV_DTYPE",
|
||||
"ALPHA_QUANTIZATION",
|
||||
"ALPHA_SHARD_COUNT",
|
||||
"MIN_LINK_RATE_GBPS",
|
||||
"PLACEMENT_IMBALANCE_FACTOR",
|
||||
"RECOMMENDED_LINK_RATE_GBPS",
|
||||
"RESERVE_FLOOR_GIB",
|
||||
"RESERVE_FRACTION",
|
||||
"VERDICT_ALPHA",
|
||||
"VERDICT_STOP",
|
||||
"AlphaContract",
|
||||
"AlphaContractError",
|
||||
"ArchitectureSnapshot",
|
||||
"GlmTargetError",
|
||||
"NodeMemory",
|
||||
"ResourcePlanError",
|
||||
"RouteFit",
|
||||
"SeamPlan",
|
||||
"Shard",
|
||||
"TargetManifest",
|
||||
"TopologyPlan",
|
||||
"canonical_sha256",
|
||||
"compute_contract_digest",
|
||||
"kv_bytes",
|
||||
"load_alpha_contract",
|
||||
"load_architecture_snapshot",
|
||||
"load_locked_target",
|
||||
"load_target_manifest",
|
||||
"parse_alpha_contract",
|
||||
"parse_architecture_snapshot",
|
||||
"parse_target_manifest",
|
||||
"plan_all_tiers",
|
||||
"plan_route",
|
||||
"plan_seams",
|
||||
"plan_topology",
|
||||
"require_contract_target",
|
||||
"require_pinned_target",
|
||||
"seal_contract",
|
||||
]
|
||||
359
packages/node/meshnet_node/glm_alpha/contract.py
Normal file
359
packages/node/meshnet_node/glm_alpha/contract.py
Normal file
@@ -0,0 +1,359 @@
|
||||
"""The immutable GLM-5.2 Max alpha acceptance contract.
|
||||
|
||||
The contract exists to answer one question that cannot be answered honestly after
|
||||
the fact: *what would have counted as success?*
|
||||
|
||||
Its thresholds are locked before the target ever runs (DGR-017), and DGR-020 reads
|
||||
them back to publish an ``alpha`` or ``stop`` verdict. The whole point is that the
|
||||
gap between those two moments is where a threshold quietly becomes "0.1 tokens/sec
|
||||
was always the goal". So the document carries ``contract_sha256`` over its canonical content, while the
|
||||
approved v1 digest is pinned independently in code. :func:`load_alpha_contract`
|
||||
recomputes the self-digest and then requires that trusted pre-execution digest.
|
||||
Changing a threshold and re-sealing under the same identity is rejected; an
|
||||
amendment requires a new supported contract identity under human review.
|
||||
|
||||
The parsed contract recursively freezes nested mappings and sequences. Thresholds
|
||||
therefore cannot change between verification and use, and :meth:`AlphaContract.to_dict`
|
||||
returns an isolated mutable copy for diagnostics and tests.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from importlib.resources import files
|
||||
from pathlib import Path
|
||||
from types import MappingProxyType
|
||||
from typing import Any, Mapping
|
||||
|
||||
from .manifest import (
|
||||
ALPHA_QUANTIZATION,
|
||||
ALPHA_SHARD_COUNT,
|
||||
ArchitectureSnapshot,
|
||||
GlmTargetError,
|
||||
TargetManifest,
|
||||
canonical_sha256,
|
||||
)
|
||||
|
||||
ALPHA_CONTRACT_SCHEMA_VERSION = 1
|
||||
ALPHA_CONTRACT_VERSION = 1
|
||||
ALPHA_CONTRACT_ID = "glm-5.2-max-alpha/v1"
|
||||
ALPHA_CONTRACT_V1_SHA256 = "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9"
|
||||
|
||||
_CONTRACT_RESOURCE = "alpha-contract.json"
|
||||
|
||||
DIGEST_FIELD = "contract_sha256"
|
||||
|
||||
VERDICT_ALPHA = "alpha"
|
||||
VERDICT_STOP = "stop"
|
||||
|
||||
# Every section the roadmap's acceptance matrix (section 5) locks. A contract that
|
||||
# omits one is not a weaker contract, it is an unreviewable one.
|
||||
REQUIRED_SECTIONS: tuple[str, ...] = (
|
||||
"identity_and_fit",
|
||||
"semantic_correctness",
|
||||
"target_run",
|
||||
"performance",
|
||||
"reliability",
|
||||
"storage",
|
||||
)
|
||||
|
||||
|
||||
class AlphaContractError(GlmTargetError):
|
||||
"""Raised when the alpha contract is missing, malformed, or has been mutated."""
|
||||
|
||||
|
||||
def contract_signing_payload(document: Mapping[str, Any]) -> dict:
|
||||
"""The contract content the digest covers: everything except the digest itself."""
|
||||
unsigned = dict(document)
|
||||
unsigned.pop(DIGEST_FIELD, None)
|
||||
return unsigned
|
||||
|
||||
|
||||
def compute_contract_digest(document: Mapping[str, Any]) -> str:
|
||||
"""SHA-256 over the canonical contract content."""
|
||||
return canonical_sha256(contract_signing_payload(_thaw_json(document)))
|
||||
|
||||
|
||||
def _freeze_json(value: Any) -> Any:
|
||||
if isinstance(value, Mapping):
|
||||
return MappingProxyType({str(key): _freeze_json(item) for key, item in value.items()})
|
||||
if isinstance(value, list):
|
||||
return tuple(_freeze_json(item) for item in value)
|
||||
return value
|
||||
|
||||
|
||||
def _thaw_json(value: Any) -> Any:
|
||||
if isinstance(value, Mapping):
|
||||
return {str(key): _thaw_json(item) for key, item in value.items()}
|
||||
if isinstance(value, tuple):
|
||||
return [_thaw_json(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AlphaContract:
|
||||
"""A locked, digest-bound alpha acceptance contract."""
|
||||
|
||||
schema_version: int
|
||||
contract_version: int
|
||||
contract_id: str
|
||||
locked_at: str
|
||||
locked_by: str
|
||||
target: Mapping[str, Any]
|
||||
sections: Mapping[str, Mapping[str, Any]]
|
||||
verdicts: tuple[str, ...]
|
||||
amendment_policy: str
|
||||
digest: str
|
||||
raw: Mapping[str, Any]
|
||||
source: str = "<memory>"
|
||||
|
||||
def section(self, name: str) -> Mapping[str, Any]:
|
||||
if name not in self.sections:
|
||||
raise AlphaContractError(f"contract section {name!r} is missing from {self.source}")
|
||||
return self.sections[name]
|
||||
|
||||
def threshold(self, section: str, key: str) -> Any:
|
||||
block = self.section(section)
|
||||
if key not in block:
|
||||
raise AlphaContractError(
|
||||
f"threshold {section}.{key} is not locked in {self.source}; an unlocked "
|
||||
"threshold cannot be used to judge a result"
|
||||
)
|
||||
return block[key]
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return _thaw_json(self.raw)
|
||||
|
||||
|
||||
def parse_alpha_contract(data: Any, source: str = "<memory>") -> AlphaContract:
|
||||
"""Validate a contract document and verify it has not been mutated since locking."""
|
||||
if not isinstance(data, Mapping):
|
||||
raise AlphaContractError(f"contract root in {source} must be a JSON object")
|
||||
|
||||
schema_version = data.get("schema_version")
|
||||
if (
|
||||
not isinstance(schema_version, int)
|
||||
or isinstance(schema_version, bool)
|
||||
or schema_version != ALPHA_CONTRACT_SCHEMA_VERSION
|
||||
):
|
||||
raise AlphaContractError(
|
||||
f"{source} declares alpha-contract schema version {schema_version!r}, but this "
|
||||
f"node reads version {ALPHA_CONTRACT_SCHEMA_VERSION}"
|
||||
)
|
||||
|
||||
contract_version = data.get("contract_version")
|
||||
if (
|
||||
not isinstance(contract_version, int)
|
||||
or isinstance(contract_version, bool)
|
||||
or contract_version != ALPHA_CONTRACT_VERSION
|
||||
):
|
||||
raise AlphaContractError(
|
||||
f"{source} declares contract version {contract_version!r}, but this node "
|
||||
f"reads version {ALPHA_CONTRACT_VERSION}"
|
||||
)
|
||||
|
||||
contract_id = data.get("contract_id")
|
||||
if contract_id != ALPHA_CONTRACT_ID:
|
||||
raise AlphaContractError(
|
||||
f"{source} declares contract_id {contract_id!r}, but this node is locked "
|
||||
f"to {ALPHA_CONTRACT_ID!r}"
|
||||
)
|
||||
|
||||
for field in ("locked_at", "locked_by"):
|
||||
value = data.get(field)
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
raise AlphaContractError(f"{source} must carry a non-empty {field}")
|
||||
|
||||
declared = data.get(DIGEST_FIELD)
|
||||
if not isinstance(declared, str) or not declared:
|
||||
raise AlphaContractError(
|
||||
f"{source} carries no {DIGEST_FIELD}; an unsealed contract cannot prove it "
|
||||
"predates the results it judges"
|
||||
)
|
||||
|
||||
computed = compute_contract_digest(data)
|
||||
if computed != declared:
|
||||
raise AlphaContractError(
|
||||
f"{source} has been modified since it was locked: its content hashes to "
|
||||
f"{computed}, but it declares {declared}. Alpha thresholds are locked before "
|
||||
"target execution and may not be weakened afterwards. To change them, open a "
|
||||
"new contract_id under human review; do not edit this one."
|
||||
)
|
||||
|
||||
if not data.get("locked_before_target_execution"):
|
||||
raise AlphaContractError(
|
||||
f"{source} does not assert locked_before_target_execution; a contract written "
|
||||
"after the results are known is not a contract"
|
||||
)
|
||||
|
||||
missing = [name for name in REQUIRED_SECTIONS if not isinstance(data.get(name), Mapping)]
|
||||
if missing:
|
||||
raise AlphaContractError(
|
||||
f"{source} is missing locked acceptance section(s) {missing}"
|
||||
)
|
||||
|
||||
verdicts = data.get("verdicts")
|
||||
if not isinstance(verdicts, list) or sorted(verdicts) != sorted([VERDICT_ALPHA, VERDICT_STOP]):
|
||||
raise AlphaContractError(
|
||||
f"{source} must offer exactly the verdicts {[VERDICT_ALPHA, VERDICT_STOP]}; a "
|
||||
"third outcome is how 'it loaded' becomes a pass"
|
||||
)
|
||||
|
||||
target = data.get("target")
|
||||
if not isinstance(target, Mapping):
|
||||
raise AlphaContractError(f"{source} is missing its 'target' block")
|
||||
for field in (
|
||||
"source_repo_id",
|
||||
"source_revision",
|
||||
"gguf_repo_id",
|
||||
"gguf_revision",
|
||||
"quantization",
|
||||
"target_manifest_sha256",
|
||||
"architecture_snapshot_sha256",
|
||||
"reasoning_effort",
|
||||
):
|
||||
if not target.get(field):
|
||||
raise AlphaContractError(f"{source} target block is missing {field!r}")
|
||||
for field in ("source_revision", "gguf_revision"):
|
||||
if not re.fullmatch(r"[0-9a-f]{40}", str(target[field])):
|
||||
raise AlphaContractError(f"{source} target.{field} is not a full commit revision")
|
||||
for field in ("target_manifest_sha256", "architecture_snapshot_sha256"):
|
||||
if not re.fullmatch(r"[0-9a-f]{64}", str(target[field])):
|
||||
raise AlphaContractError(f"{source} target.{field} is not a SHA-256 digest")
|
||||
if target["quantization"] != ALPHA_QUANTIZATION:
|
||||
raise AlphaContractError(
|
||||
f"{source} targets {target['quantization']!r}, not locked alpha quantization "
|
||||
f"{ALPHA_QUANTIZATION!r}"
|
||||
)
|
||||
if target["reasoning_effort"] != "max":
|
||||
raise AlphaContractError(f"{source} must lock reasoning_effort='max'")
|
||||
shard_count = target.get("shard_count")
|
||||
if (
|
||||
not isinstance(shard_count, int)
|
||||
or isinstance(shard_count, bool)
|
||||
or shard_count != ALPHA_SHARD_COUNT
|
||||
):
|
||||
raise AlphaContractError(
|
||||
f"{source} target.shard_count must be exactly {ALPHA_SHARD_COUNT}"
|
||||
)
|
||||
total_bytes = target.get("total_bytes")
|
||||
if not isinstance(total_bytes, int) or isinstance(total_bytes, bool) or total_bytes <= 0:
|
||||
raise AlphaContractError(f"{source} target.total_bytes must be a positive integer")
|
||||
|
||||
amendment_policy = data.get("amendment_policy")
|
||||
if not isinstance(amendment_policy, str) or not amendment_policy.strip():
|
||||
raise AlphaContractError(f"{source} must state its amendment policy")
|
||||
|
||||
if declared != ALPHA_CONTRACT_V1_SHA256:
|
||||
raise AlphaContractError(
|
||||
f"{source} is a re-sealed mutation of {ALPHA_CONTRACT_ID}: digest "
|
||||
f"{declared} does not match the trusted pre-execution digest "
|
||||
f"{ALPHA_CONTRACT_V1_SHA256}. An amendment requires a new supported "
|
||||
"contract identity under human review."
|
||||
)
|
||||
|
||||
frozen = _freeze_json(data)
|
||||
|
||||
return AlphaContract(
|
||||
schema_version=schema_version,
|
||||
contract_version=contract_version,
|
||||
contract_id=contract_id,
|
||||
locked_at=str(data["locked_at"]),
|
||||
locked_by=str(data["locked_by"]),
|
||||
target=frozen["target"],
|
||||
sections=MappingProxyType({name: frozen[name] for name in REQUIRED_SECTIONS}),
|
||||
verdicts=tuple(verdicts),
|
||||
amendment_policy=amendment_policy,
|
||||
digest=declared,
|
||||
raw=frozen,
|
||||
source=source,
|
||||
)
|
||||
|
||||
|
||||
def require_contract_target(
|
||||
contract: AlphaContract,
|
||||
manifest: TargetManifest,
|
||||
snapshot: ArchitectureSnapshot,
|
||||
) -> None:
|
||||
"""Bind the sealed contract to the exact manifest and architecture snapshot.
|
||||
|
||||
Repository revisions alone do not bind shard LFS objects or derived architecture
|
||||
semantics. Call this before planning, admission, download, or execution.
|
||||
"""
|
||||
|
||||
expected = contract.target
|
||||
actual = {
|
||||
"source_repo_id": manifest.source_repo_id,
|
||||
"source_revision": manifest.source_revision,
|
||||
"gguf_repo_id": manifest.gguf_repo_id,
|
||||
"gguf_revision": manifest.gguf_revision,
|
||||
"quantization": manifest.quantization,
|
||||
"shard_count": len(manifest.shards),
|
||||
"total_bytes": manifest.total_bytes,
|
||||
"target_manifest_sha256": manifest.digest,
|
||||
"architecture_snapshot_sha256": snapshot.digest,
|
||||
}
|
||||
if snapshot.source_repo_id != manifest.source_repo_id:
|
||||
raise AlphaContractError(
|
||||
"architecture snapshot repository does not match the target manifest repository"
|
||||
)
|
||||
if snapshot.source_revision != manifest.source_revision:
|
||||
raise AlphaContractError(
|
||||
"architecture snapshot revision does not match the target manifest revision"
|
||||
)
|
||||
mismatches = {
|
||||
key: (expected.get(key), value)
|
||||
for key, value in actual.items()
|
||||
if expected.get(key) != value
|
||||
}
|
||||
if mismatches:
|
||||
details = ", ".join(
|
||||
f"{key}: locked={locked!r}, actual={value!r}"
|
||||
for key, (locked, value) in sorted(mismatches.items())
|
||||
)
|
||||
raise AlphaContractError(f"target documents do not match the sealed contract: {details}")
|
||||
|
||||
|
||||
def load_alpha_contract(path: Path | None = None) -> AlphaContract:
|
||||
"""Load the packaged alpha contract, or one at ``path``."""
|
||||
if path is not None:
|
||||
source = str(path)
|
||||
try:
|
||||
raw = path.read_text(encoding="utf-8")
|
||||
except OSError as exc:
|
||||
raise AlphaContractError(f"cannot read {source}: {exc.strerror or exc}") from exc
|
||||
else:
|
||||
source = f"packaged {_CONTRACT_RESOURCE}"
|
||||
try:
|
||||
raw = (
|
||||
files("meshnet_node.glm_alpha")
|
||||
.joinpath("data", _CONTRACT_RESOURCE)
|
||||
.read_text(encoding="utf-8")
|
||||
)
|
||||
except (OSError, FileNotFoundError, ModuleNotFoundError) as exc:
|
||||
raise AlphaContractError(
|
||||
f"{source} is missing from this node installation ({type(exc).__name__})"
|
||||
) from exc
|
||||
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise AlphaContractError(
|
||||
f"{source} is not valid JSON: {exc.msg} at line {exc.lineno} column {exc.colno}"
|
||||
) from exc
|
||||
|
||||
return parse_alpha_contract(data, source=source)
|
||||
|
||||
|
||||
def seal_contract(document: Mapping[str, Any]) -> dict:
|
||||
"""Return the document with a freshly computed digest.
|
||||
|
||||
This is the only supported way to produce a contract file. It is deliberately
|
||||
*not* called at load time: sealing on load would turn every mutation into a
|
||||
valid contract, which is precisely the property the digest exists to deny.
|
||||
"""
|
||||
sealed = dict(document)
|
||||
sealed[DIGEST_FIELD] = compute_contract_digest(document)
|
||||
return sealed
|
||||
122
packages/node/meshnet_node/glm_alpha/data/alpha-contract.json
Normal file
122
packages/node/meshnet_node/glm_alpha/data/alpha-contract.json
Normal file
@@ -0,0 +1,122 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"contract_version": 1,
|
||||
"contract_id": "glm-5.2-max-alpha/v1",
|
||||
"locked_at": "2026-07-13",
|
||||
"locked_by": "DGR-017",
|
||||
"locked_before_target_execution": true,
|
||||
"target": {
|
||||
"source_repo_id": "zai-org/GLM-5.2",
|
||||
"source_revision": "b4734de4facf877f85769a911abafc5283eab3d9",
|
||||
"gguf_repo_id": "unsloth/GLM-5.2-GGUF",
|
||||
"gguf_revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||
"quantization": "UD-IQ1_S",
|
||||
"shard_count": 6,
|
||||
"total_bytes": 216715360960,
|
||||
"target_manifest_sha256": "0b6aed04479d204902bb64c0203f1a46cab26a47b378ecccf85237b63f6c1962",
|
||||
"architecture_snapshot_sha256": "253fbd94b06b42acc4724ec2c7f33914e2d4cc43f54a36dff6af19a80ae6ceb1",
|
||||
"reasoning_effort": "max"
|
||||
},
|
||||
"identity_and_fit": {
|
||||
"require_exact_revisions": true,
|
||||
"require_all_shard_sha256": true,
|
||||
"require_per_node_owned_tensor_report": true,
|
||||
"require_owned_tensor_union_equals_inventory": true,
|
||||
"max_unintended_tensor_overlap": 0,
|
||||
"require_no_single_node_can_admit_complete_recipe": true,
|
||||
"min_node_reserve_fraction": 0.2,
|
||||
"min_node_reserve_gib": 8.0,
|
||||
"require_measured_peak_scratch_inside_reserve": true,
|
||||
"forbid_swap": true,
|
||||
"forbid_overcommit": true,
|
||||
"forbid_mmap_only_fit_claim": true,
|
||||
"forbid_double_counted_unified_memory": true,
|
||||
"aggregate_hard_fit_floor_gib": 224.0,
|
||||
"aggregate_floor_class": "experimental_hard_fit_floor",
|
||||
"recommended_topologies": [
|
||||
"5x64GiB",
|
||||
"3x96GiB",
|
||||
"3x128GiB"
|
||||
],
|
||||
"arithmetic_minimum_requires_measured_placement_evidence": true
|
||||
},
|
||||
"semantic_correctness": {
|
||||
"require_active_moe_routing": true,
|
||||
"require_active_shared_expert": true,
|
||||
"require_active_dsa_lightning_indexer": true,
|
||||
"require_active_sparse_attention": true,
|
||||
"require_active_indexshare_full_and_shared": true,
|
||||
"dense_attention_fallback_satisfies_alpha": false,
|
||||
"require_rendered_reasoning_effort_marker": "<|system|>Reasoning Effort: Max",
|
||||
"f32_seam_fixture_exact_match_tokens": 32,
|
||||
"f32_seam_fixture_requires_exact_match": true,
|
||||
"min_greedy_token_agreement": 0.9,
|
||||
"min_mean_state_cosine_similarity": 0.999,
|
||||
"forbid_nonfinite_tensors": true,
|
||||
"require_fail_closed_on_fingerprint_mismatch": true
|
||||
},
|
||||
"target_run": {
|
||||
"context_tokens": 16384,
|
||||
"kv_dtype": "Q8_0",
|
||||
"concurrency": 1,
|
||||
"prompt_lane_tokens": 4096,
|
||||
"min_output_tokens": 512,
|
||||
"min_output_tokens_with_natural_eos": 128,
|
||||
"require_same_switch_wired_network": true,
|
||||
"min_link_rate_gbps": 2.5,
|
||||
"recommended_link_rate_gbps": 10.0,
|
||||
"require_sentinels": [
|
||||
"coding",
|
||||
"structured_tool_call_json",
|
||||
"multi_step_reasoning"
|
||||
],
|
||||
"require_openai_compatible_response_fields": [
|
||||
"model",
|
||||
"finish_reason",
|
||||
"usage"
|
||||
]
|
||||
},
|
||||
"performance": {
|
||||
"min_median_decode_tokens_per_second": 0.5,
|
||||
"max_ttft_seconds_at_4096_prompt": 600,
|
||||
"max_unexplained_stall_seconds": 60,
|
||||
"warmups": 1,
|
||||
"require_per_stage_telemetry": [
|
||||
"compute",
|
||||
"queue",
|
||||
"kv",
|
||||
"seam_bytes",
|
||||
"seam_latency",
|
||||
"rss",
|
||||
"vram",
|
||||
"backend_timing"
|
||||
],
|
||||
"quality_pass_with_speed_fail_verdict": "stop",
|
||||
"forbid_generalising_results_to_other_hardware": true
|
||||
},
|
||||
"reliability": {
|
||||
"consecutive_clean_cold_starts": 2,
|
||||
"require_cancellation_releases_buffers_and_kv": true,
|
||||
"require_worker_loss_aborts_route": true,
|
||||
"retry_policy": "from_token_zero_on_new_compatible_route",
|
||||
"forbid_silent_kv_migration": true,
|
||||
"require_reject_stale_epoch": true,
|
||||
"require_reject_duplicate_step_id": true,
|
||||
"synthetic_workers_satisfy_alpha": false,
|
||||
"layer_reduced_fixtures_satisfy_alpha": false
|
||||
},
|
||||
"storage": {
|
||||
"mounted_storage_only": true,
|
||||
"forbidden_path_prefixes": [
|
||||
"/home"
|
||||
],
|
||||
"forbid_secrets_in_logs": true,
|
||||
"forbid_unrestricted_prompt_payloads_in_logs": true
|
||||
},
|
||||
"verdicts": [
|
||||
"alpha",
|
||||
"stop"
|
||||
],
|
||||
"amendment_policy": "Thresholds are locked before target execution. They may not be weakened, moved, or reinterpreted after results are known. A change requires a new contract_id and contract_version under human review, and the superseded contract is retained.",
|
||||
"contract_sha256": "aab23220280c053a3c14ff559df3cb5c9e1bf7f0f7188c6519e2e9d9ad036ed9"
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"observed_at": "2026-07-13",
|
||||
"source_repo_id": "zai-org/GLM-5.2",
|
||||
"source_revision": "b4734de4facf877f85769a911abafc5283eab3d9",
|
||||
"source_files": [
|
||||
{
|
||||
"path": "config.json",
|
||||
"size_bytes": 3732,
|
||||
"sha256": "185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a",
|
||||
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/config.json"
|
||||
},
|
||||
{
|
||||
"path": "chat_template.jinja",
|
||||
"size_bytes": 5076,
|
||||
"sha256": "172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679",
|
||||
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/chat_template.jinja"
|
||||
},
|
||||
{
|
||||
"path": "generation_config.json",
|
||||
"size_bytes": 194,
|
||||
"sha256": "ac76b43d8683d3b930126870fc8be73d8679308fe752fa1f381096d8354f6a55",
|
||||
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/generation_config.json"
|
||||
},
|
||||
{
|
||||
"path": "tokenizer_config.json",
|
||||
"size_bytes": 761,
|
||||
"sha256": "98b1271574f41abf89427ae2dda030d94dc9478f0edc5a8bd240db213c6fd5fc",
|
||||
"url": "https://huggingface.co/zai-org/GLM-5.2/resolve/b4734de4facf877f85769a911abafc5283eab3d9/tokenizer_config.json"
|
||||
}
|
||||
],
|
||||
"architecture": {
|
||||
"architectures": [
|
||||
"GlmMoeDsaForCausalLM"
|
||||
],
|
||||
"model_type": "glm_moe_dsa",
|
||||
"num_hidden_layers": 78,
|
||||
"num_nextn_predict_layers": 1,
|
||||
"total_artifact_layers": 79,
|
||||
"first_k_dense_replace": 3,
|
||||
"dense_layers": 3,
|
||||
"sparse_moe_layers": 75,
|
||||
"hidden_size": 6144,
|
||||
"intermediate_size": 12288,
|
||||
"moe_intermediate_size": 2048,
|
||||
"n_routed_experts": 256,
|
||||
"num_experts_per_tok": 8,
|
||||
"n_shared_experts": 1,
|
||||
"scoring_func": "sigmoid",
|
||||
"topk_method": "noaux_tc",
|
||||
"norm_topk_prob": true,
|
||||
"routed_scaling_factor": 2.5,
|
||||
"num_attention_heads": 64,
|
||||
"head_dim": 192,
|
||||
"qk_nope_head_dim": 192,
|
||||
"qk_rope_head_dim": 64,
|
||||
"qk_head_dim": 256,
|
||||
"v_head_dim": 256,
|
||||
"kv_lora_rank": 512,
|
||||
"q_lora_rank": 2048,
|
||||
"mla_cached_values_per_token_per_layer": 576,
|
||||
"index_topk": 2048,
|
||||
"index_head_dim": 128,
|
||||
"index_n_heads": 32,
|
||||
"index_topk_freq": 4,
|
||||
"index_skip_topk_offset": 3,
|
||||
"index_share_for_mtp_iteration": true,
|
||||
"indexer_full_layers": 21,
|
||||
"indexer_shared_layers": 57,
|
||||
"indexer_types_sha256": "ec3b4927af83cf02baf37fb10454c40176ec8bf501ae89334b27a9df5fa17025",
|
||||
"max_position_embeddings": 1048576,
|
||||
"vocab_size": 154880,
|
||||
"rope_theta": 8000000,
|
||||
"dtype": "bfloat16",
|
||||
"tie_word_embeddings": false
|
||||
},
|
||||
"reasoning_effort": {
|
||||
"alpha_mode": "max",
|
||||
"rendered_marker": "<|system|>Reasoning Effort: Max",
|
||||
"template_rule": "effective_reasoning_effort = 'high' if reasoning_effort == 'high' else 'max'",
|
||||
"default_is_max": true,
|
||||
"suppressed_when": "enable_thinking is defined and false",
|
||||
"note": "The template recognises exactly one non-max level ('high'); every other value, including an absent reasoning_effort, renders Max. Alpha therefore asserts the rendered 'Reasoning Effort: Max' marker, not merely the presence of a request field."
|
||||
},
|
||||
"notes": [
|
||||
"indexer_types has 78 entries: layers 0-2 are 'full', then a repeating [shared, shared, shared, full] pattern, giving 21 Full producer layers and 57 Shared consumer layers.",
|
||||
"MLA caches kv_lora_rank (512) + qk_rope_head_dim (64) = 576 values per token per backbone layer.",
|
||||
"num_nextn_predict_layers=1 is the NextN/MTP layer present in the artifact. Alpha does not run MTP; the tensors are loaded or explicitly excluded by a certified recipe and must never be silently reinterpreted as a 79th backbone layer.",
|
||||
"Values are derived from the pinned config.json. The runtime must re-derive them from the artifact and fail closed on contradictory metadata; marketing names are not compatibility identity."
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"manifest_version": 1,
|
||||
"observed_at": "2026-07-13",
|
||||
"observed_by": "DGR-017",
|
||||
"alpha_quantization": "UD-IQ1_S",
|
||||
"source_model": {
|
||||
"repo_id": "zai-org/GLM-5.2",
|
||||
"revision": "b4734de4facf877f85769a911abafc5283eab3d9",
|
||||
"last_modified": "2026-07-02T08:08:14.000Z",
|
||||
"weight_license": "mit",
|
||||
"code_documentation_license": "apache-2.0",
|
||||
"url": "https://huggingface.co/zai-org/GLM-5.2",
|
||||
"revision_url": "https://huggingface.co/zai-org/GLM-5.2/tree/b4734de4facf877f85769a911abafc5283eab3d9",
|
||||
"api_url": "https://huggingface.co/api/models/zai-org/GLM-5.2"
|
||||
},
|
||||
"gguf_artifact": {
|
||||
"repo_id": "unsloth/GLM-5.2-GGUF",
|
||||
"revision": "abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||
"last_modified": "2026-06-23T15:18:23.000Z",
|
||||
"license": "mit",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF",
|
||||
"revision_url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/tree/abc55e72527792c6e77069c99b4cb7de16fa9f23",
|
||||
"api_url": "https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF",
|
||||
"quantization": "UD-IQ1_S",
|
||||
"shard_count": 6,
|
||||
"total_bytes": 216715360960,
|
||||
"total_gib": 201.832,
|
||||
"total_gb": 216.715,
|
||||
"shards": [
|
||||
{
|
||||
"index": 1,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf",
|
||||
"size_bytes": 9423744,
|
||||
"sha256": "46b6148389219ae45167cb8124fbb18ef7d432daf619b4faf9e06ea80d3f4777",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf"
|
||||
},
|
||||
{
|
||||
"index": 2,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf",
|
||||
"size_bytes": 49208128256,
|
||||
"sha256": "f2180207285e04fcaa5b8c53ba6e77ad5cc58666b6e7c6b04a5eded3fe8bef09",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00002-of-00006.gguf"
|
||||
},
|
||||
{
|
||||
"index": 3,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf",
|
||||
"size_bytes": 49684417024,
|
||||
"sha256": "b1c0c5a302cc8d5d9ea0bcd4467c01db72c26839f820f7e882079582ea0a8d2b",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00003-of-00006.gguf"
|
||||
},
|
||||
{
|
||||
"index": 4,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf",
|
||||
"size_bytes": 49396052864,
|
||||
"sha256": "a6a42da6975e29f89866dcde2956e9e50e6ea26635fb5063b74f3973f4f863b6",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00004-of-00006.gguf"
|
||||
},
|
||||
{
|
||||
"index": 5,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
|
||||
"size_bytes": 49246275936,
|
||||
"sha256": "a4a9851a50db533f21ef824e5d8038f04e6782e7d602d18e5fdd6643f68ccccb",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00005-of-00006.gguf"
|
||||
},
|
||||
{
|
||||
"index": 6,
|
||||
"path": "UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf",
|
||||
"size_bytes": 19171063136,
|
||||
"sha256": "3b767f55df64e0432d52fcf1a14eb47a1ef3bbc91339e2ae220f38602237d7d7",
|
||||
"url": "https://huggingface.co/unsloth/GLM-5.2-GGUF/resolve/abc55e72527792c6e77069c99b4cb7de16fa9f23/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00006-of-00006.gguf"
|
||||
}
|
||||
]
|
||||
},
|
||||
"diagnostic_fallback": {
|
||||
"quantization": "UD-IQ1_M",
|
||||
"shard_count": 6,
|
||||
"total_bytes": 228492966624,
|
||||
"total_gib": 212.801,
|
||||
"total_gb": 228.493,
|
||||
"policy": "First diagnostic fallback only if UD-IQ1_S exposes a runtime or quality defect. It does not satisfy the alpha 'lowest published quantization' target unless human review changes the target contract."
|
||||
},
|
||||
"storage": {
|
||||
"mounted_storage_only": true,
|
||||
"forbidden_path_prefixes": [
|
||||
"/home"
|
||||
],
|
||||
"note": "Model artifacts resolve through the machine-specific .env.<hostname> mounted-drive configuration. A path under /home fails admission closed."
|
||||
}
|
||||
}
|
||||
490
packages/node/meshnet_node/glm_alpha/manifest.py
Normal file
490
packages/node/meshnet_node/glm_alpha/manifest.py
Normal file
@@ -0,0 +1,490 @@
|
||||
"""The pinned GLM-5.2 Max target manifest and architecture snapshot.
|
||||
|
||||
This module is the *identity* half of the alpha target contract. It answers one
|
||||
question: is the artifact on this disk the exact artifact alpha was locked
|
||||
against?
|
||||
|
||||
Identity is pinned by repository revision **and** by every shard's LFS SHA-256.
|
||||
A revision alone is not enough — a repository can be force-pushed, and a tag can
|
||||
be moved — and a size alone is not enough, because two different quantizations of
|
||||
the same model land within a rounding error of each other. The manifest therefore
|
||||
carries both, plus the aggregate byte total, and cross-checks the aggregate
|
||||
against the sum of the shards. A manifest whose declared total disagrees with its
|
||||
own shards is rejected rather than trusted, because that is the exact shape a
|
||||
hand-edited "it fits now" manifest takes.
|
||||
|
||||
Nothing here downloads a weight payload. Sizes and hashes come from the Hugging
|
||||
Face metadata API (see ``scripts/refresh_glm_target_manifest.py``); verification
|
||||
against a local file is DGR-018's job, using the digests locked here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from importlib.resources import files
|
||||
from pathlib import Path
|
||||
from typing import Any, Mapping
|
||||
|
||||
TARGET_MANIFEST_SCHEMA_VERSION = 1
|
||||
ARCHITECTURE_SNAPSHOT_SCHEMA_VERSION = 1
|
||||
|
||||
ALPHA_QUANTIZATION = "UD-IQ1_S"
|
||||
ALPHA_SHARD_COUNT = 6
|
||||
|
||||
_SHA256_RE = re.compile(r"\A[0-9a-f]{64}\Z")
|
||||
|
||||
_MANIFEST_RESOURCE = "target-manifest.json"
|
||||
_ARCHITECTURE_RESOURCE = "architecture-snapshot.json"
|
||||
|
||||
GIB = 1024**3
|
||||
GB = 1000**3
|
||||
|
||||
|
||||
class GlmTargetError(ValueError):
|
||||
"""Raised when the target manifest or architecture snapshot is not the pinned target."""
|
||||
|
||||
|
||||
def canonical_sha256(value: Any) -> str:
|
||||
"""SHA-256 over canonical JSON — the repository's digest convention."""
|
||||
payload = json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _require_mapping(value: Any, what: str) -> Mapping[str, Any]:
|
||||
if not isinstance(value, Mapping):
|
||||
raise GlmTargetError(f"{what} must be a JSON object, got {type(value).__name__}")
|
||||
return value
|
||||
|
||||
|
||||
def _require_text(value: Any, what: str) -> str:
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
raise GlmTargetError(f"{what} must be a non-empty string")
|
||||
return value
|
||||
|
||||
|
||||
def _require_int(value: Any, what: str) -> int:
|
||||
if not isinstance(value, int) or isinstance(value, bool):
|
||||
raise GlmTargetError(f"{what} must be an integer, got {type(value).__name__}")
|
||||
return value
|
||||
|
||||
|
||||
def _require_sha256(value: Any, what: str) -> str:
|
||||
text = _require_text(value, what)
|
||||
if not _SHA256_RE.match(text):
|
||||
raise GlmTargetError(
|
||||
f"{what} must be a lowercase 64-character hex SHA-256, got {text!r}"
|
||||
)
|
||||
return text
|
||||
|
||||
|
||||
def _require_revision(value: Any, what: str) -> str:
|
||||
text = _require_text(value, what)
|
||||
if not re.fullmatch(r"[0-9a-f]{40}", text):
|
||||
raise GlmTargetError(
|
||||
f"{what} must be a full 40-character commit revision, got {text!r}; "
|
||||
"a branch name or short SHA is not an immutable pin"
|
||||
)
|
||||
return text
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Shard:
|
||||
"""One GGUF shard of the alpha artifact."""
|
||||
|
||||
index: int
|
||||
path: str
|
||||
size_bytes: int
|
||||
sha256: str
|
||||
url: str
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"index": self.index,
|
||||
"path": self.path,
|
||||
"size_bytes": self.size_bytes,
|
||||
"sha256": self.sha256,
|
||||
"url": self.url,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TargetManifest:
|
||||
"""The pinned, self-consistent GLM-5.2 ``UD-IQ1_S`` target."""
|
||||
|
||||
schema_version: int
|
||||
manifest_version: int
|
||||
observed_at: str
|
||||
quantization: str
|
||||
source_repo_id: str
|
||||
source_revision: str
|
||||
source_license: str
|
||||
gguf_repo_id: str
|
||||
gguf_revision: str
|
||||
gguf_license: str
|
||||
total_bytes: int
|
||||
shards: tuple[Shard, ...]
|
||||
raw: Mapping[str, Any]
|
||||
source: str = "<memory>"
|
||||
|
||||
@property
|
||||
def total_gib(self) -> float:
|
||||
return self.total_bytes / GIB
|
||||
|
||||
@property
|
||||
def total_gb(self) -> float:
|
||||
return self.total_bytes / GB
|
||||
|
||||
def shard(self, index: int) -> Shard:
|
||||
for shard in self.shards:
|
||||
if shard.index == index:
|
||||
return shard
|
||||
raise GlmTargetError(f"shard {index} is not in {self.source}")
|
||||
|
||||
@property
|
||||
def digest(self) -> str:
|
||||
"""Stable identity of this manifest, for the DGR-003 runtime recipe."""
|
||||
return canonical_sha256(self.raw)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return dict(self.raw)
|
||||
|
||||
|
||||
def _parse_shards(raw: Any, expected_count: int, expected_total: int) -> tuple[Shard, ...]:
|
||||
if not isinstance(raw, list):
|
||||
raise GlmTargetError("gguf_artifact.shards must be a JSON array")
|
||||
if len(raw) != expected_count:
|
||||
raise GlmTargetError(
|
||||
f"the alpha artifact has exactly {expected_count} shards, "
|
||||
f"but the manifest lists {len(raw)}"
|
||||
)
|
||||
|
||||
shards: list[Shard] = []
|
||||
seen_index: set[int] = set()
|
||||
seen_sha: set[str] = set()
|
||||
for position, entry in enumerate(raw):
|
||||
item = _require_mapping(entry, f"shards[{position}]")
|
||||
index = _require_int(item.get("index"), f"shards[{position}].index")
|
||||
if index in seen_index:
|
||||
raise GlmTargetError(f"duplicate shard index {index} in the manifest")
|
||||
seen_index.add(index)
|
||||
size_bytes = _require_int(item.get("size_bytes"), f"shards[{index}].size_bytes")
|
||||
if size_bytes <= 0:
|
||||
raise GlmTargetError(f"shards[{index}].size_bytes must be positive")
|
||||
sha256 = _require_sha256(item.get("sha256"), f"shards[{index}].sha256")
|
||||
if sha256 in seen_sha:
|
||||
raise GlmTargetError(
|
||||
f"shard {index} repeats SHA-256 {sha256}; two distinct shards cannot "
|
||||
"have the same content digest"
|
||||
)
|
||||
seen_sha.add(sha256)
|
||||
shards.append(
|
||||
Shard(
|
||||
index=index,
|
||||
path=_require_text(item.get("path"), f"shards[{index}].path"),
|
||||
size_bytes=size_bytes,
|
||||
sha256=sha256,
|
||||
url=_require_text(item.get("url"), f"shards[{index}].url"),
|
||||
)
|
||||
)
|
||||
|
||||
expected_indices = set(range(1, expected_count + 1))
|
||||
if seen_index != expected_indices:
|
||||
missing = sorted(expected_indices - seen_index)
|
||||
raise GlmTargetError(
|
||||
f"the manifest is missing shard(s) {missing}; all {expected_count} "
|
||||
"shards of the alpha artifact must be pinned"
|
||||
)
|
||||
|
||||
summed = sum(shard.size_bytes for shard in shards)
|
||||
if summed != expected_total:
|
||||
raise GlmTargetError(
|
||||
f"declared total_bytes {expected_total} does not equal the sum of the "
|
||||
f"shard sizes {summed}; the manifest is not self-consistent"
|
||||
)
|
||||
|
||||
return tuple(sorted(shards, key=lambda shard: shard.index))
|
||||
|
||||
|
||||
def parse_target_manifest(data: Any, source: str = "<memory>") -> TargetManifest:
|
||||
"""Validate an already-decoded target manifest, failing closed."""
|
||||
doc = _require_mapping(data, f"manifest root in {source}")
|
||||
|
||||
schema_version = _require_int(doc.get("schema_version"), f"'schema_version' in {source}")
|
||||
if schema_version != TARGET_MANIFEST_SCHEMA_VERSION:
|
||||
raise GlmTargetError(
|
||||
f"{source} declares target-manifest schema version {schema_version}, "
|
||||
f"but this node reads version {TARGET_MANIFEST_SCHEMA_VERSION}"
|
||||
)
|
||||
|
||||
quantization = _require_text(doc.get("alpha_quantization"), f"'alpha_quantization' in {source}")
|
||||
if quantization != ALPHA_QUANTIZATION:
|
||||
raise GlmTargetError(
|
||||
f"{source} pins quantization {quantization!r}, but the locked alpha "
|
||||
f"quantization is {ALPHA_QUANTIZATION!r}; a different quantization is a "
|
||||
"different target and requires a human contract change"
|
||||
)
|
||||
|
||||
source_model = _require_mapping(doc.get("source_model"), f"'source_model' in {source}")
|
||||
gguf = _require_mapping(doc.get("gguf_artifact"), f"'gguf_artifact' in {source}")
|
||||
|
||||
gguf_quant = _require_text(gguf.get("quantization"), f"gguf_artifact.quantization in {source}")
|
||||
if gguf_quant != quantization:
|
||||
raise GlmTargetError(
|
||||
f"{source} declares alpha_quantization {quantization!r} but the GGUF "
|
||||
f"artifact block says {gguf_quant!r}"
|
||||
)
|
||||
|
||||
shard_count = _require_int(gguf.get("shard_count"), f"gguf_artifact.shard_count in {source}")
|
||||
if shard_count != ALPHA_SHARD_COUNT:
|
||||
raise GlmTargetError(
|
||||
f"{source} declares {shard_count} shards; the pinned alpha artifact has "
|
||||
f"exactly {ALPHA_SHARD_COUNT}"
|
||||
)
|
||||
|
||||
total_bytes = _require_int(gguf.get("total_bytes"), f"gguf_artifact.total_bytes in {source}")
|
||||
shards = _parse_shards(gguf.get("shards"), shard_count, total_bytes)
|
||||
|
||||
return TargetManifest(
|
||||
schema_version=schema_version,
|
||||
manifest_version=_require_int(doc.get("manifest_version"), f"'manifest_version' in {source}"),
|
||||
observed_at=_require_text(doc.get("observed_at"), f"'observed_at' in {source}"),
|
||||
quantization=quantization,
|
||||
source_repo_id=_require_text(source_model.get("repo_id"), "source_model.repo_id"),
|
||||
source_revision=_require_revision(source_model.get("revision"), "source_model.revision"),
|
||||
source_license=_require_text(source_model.get("weight_license"), "source_model.weight_license"),
|
||||
gguf_repo_id=_require_text(gguf.get("repo_id"), "gguf_artifact.repo_id"),
|
||||
gguf_revision=_require_revision(gguf.get("revision"), "gguf_artifact.revision"),
|
||||
gguf_license=_require_text(gguf.get("license"), "gguf_artifact.license"),
|
||||
total_bytes=total_bytes,
|
||||
shards=shards,
|
||||
raw=doc,
|
||||
source=source,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ArchitectureSnapshot:
|
||||
"""Architecture-critical metadata derived from the pinned ``config.json``."""
|
||||
|
||||
schema_version: int
|
||||
source_repo_id: str
|
||||
source_revision: str
|
||||
architecture: Mapping[str, Any]
|
||||
reasoning_effort: Mapping[str, Any]
|
||||
source_files: Mapping[str, Mapping[str, Any]]
|
||||
raw: Mapping[str, Any]
|
||||
source: str = "<memory>"
|
||||
|
||||
def __getitem__(self, key: str) -> Any:
|
||||
if key not in self.architecture:
|
||||
raise GlmTargetError(f"architecture field {key!r} is missing from {self.source}")
|
||||
return self.architecture[key]
|
||||
|
||||
@property
|
||||
def digest(self) -> str:
|
||||
return canonical_sha256(self.raw)
|
||||
|
||||
def file_sha256(self, path: str) -> str:
|
||||
entry = self.source_files.get(path)
|
||||
if entry is None:
|
||||
raise GlmTargetError(f"{path!r} is not pinned in {self.source}")
|
||||
return str(entry["sha256"])
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return dict(self.raw)
|
||||
|
||||
|
||||
# Fields the distributed runtime cannot plan or shard without. Absent or
|
||||
# contradictory values fail closed rather than defaulting.
|
||||
REQUIRED_ARCHITECTURE_FIELDS: tuple[str, ...] = (
|
||||
"model_type",
|
||||
"num_hidden_layers",
|
||||
"num_nextn_predict_layers",
|
||||
"total_artifact_layers",
|
||||
"first_k_dense_replace",
|
||||
"dense_layers",
|
||||
"sparse_moe_layers",
|
||||
"hidden_size",
|
||||
"n_routed_experts",
|
||||
"num_experts_per_tok",
|
||||
"n_shared_experts",
|
||||
"kv_lora_rank",
|
||||
"qk_rope_head_dim",
|
||||
"mla_cached_values_per_token_per_layer",
|
||||
"index_topk",
|
||||
"index_head_dim",
|
||||
"indexer_full_layers",
|
||||
"indexer_shared_layers",
|
||||
"max_position_embeddings",
|
||||
"vocab_size",
|
||||
)
|
||||
|
||||
|
||||
def parse_architecture_snapshot(data: Any, source: str = "<memory>") -> ArchitectureSnapshot:
|
||||
"""Validate an architecture snapshot and its internal arithmetic."""
|
||||
doc = _require_mapping(data, f"snapshot root in {source}")
|
||||
|
||||
schema_version = _require_int(doc.get("schema_version"), f"'schema_version' in {source}")
|
||||
if schema_version != ARCHITECTURE_SNAPSHOT_SCHEMA_VERSION:
|
||||
raise GlmTargetError(
|
||||
f"{source} declares architecture-snapshot schema version {schema_version}, "
|
||||
f"but this node reads version {ARCHITECTURE_SNAPSHOT_SCHEMA_VERSION}"
|
||||
)
|
||||
|
||||
arch = _require_mapping(doc.get("architecture"), f"'architecture' in {source}")
|
||||
missing = [field for field in REQUIRED_ARCHITECTURE_FIELDS if field not in arch]
|
||||
if missing:
|
||||
raise GlmTargetError(
|
||||
f"{source} is missing architecture-critical field(s) {missing}; the "
|
||||
"runtime cannot shard or plan an architecture it cannot fully describe"
|
||||
)
|
||||
|
||||
layers = _require_int(arch["num_hidden_layers"], "num_hidden_layers")
|
||||
nextn = _require_int(arch["num_nextn_predict_layers"], "num_nextn_predict_layers")
|
||||
total_layers = _require_int(arch["total_artifact_layers"], "total_artifact_layers")
|
||||
if total_layers != layers + nextn:
|
||||
raise GlmTargetError(
|
||||
f"total_artifact_layers {total_layers} != num_hidden_layers {layers} + "
|
||||
f"num_nextn_predict_layers {nextn}; the NextN layer must be counted "
|
||||
"explicitly, never folded into the backbone"
|
||||
)
|
||||
|
||||
dense = _require_int(arch["dense_layers"], "dense_layers")
|
||||
sparse = _require_int(arch["sparse_moe_layers"], "sparse_moe_layers")
|
||||
if dense != _require_int(arch["first_k_dense_replace"], "first_k_dense_replace"):
|
||||
raise GlmTargetError("dense_layers must equal first_k_dense_replace")
|
||||
if dense + sparse != layers:
|
||||
raise GlmTargetError(
|
||||
f"dense_layers {dense} + sparse_moe_layers {sparse} != num_hidden_layers {layers}"
|
||||
)
|
||||
|
||||
full = _require_int(arch["indexer_full_layers"], "indexer_full_layers")
|
||||
shared = _require_int(arch["indexer_shared_layers"], "indexer_shared_layers")
|
||||
if full + shared != layers:
|
||||
raise GlmTargetError(
|
||||
f"indexer_full_layers {full} + indexer_shared_layers {shared} != "
|
||||
f"num_hidden_layers {layers}; every layer holds exactly one IndexShare role"
|
||||
)
|
||||
if full <= 0:
|
||||
raise GlmTargetError(
|
||||
"indexer_full_layers must be positive; a route with no Full producer layer "
|
||||
"has no index for its Shared consumers to reuse"
|
||||
)
|
||||
|
||||
mla = _require_int(
|
||||
arch["mla_cached_values_per_token_per_layer"], "mla_cached_values_per_token_per_layer"
|
||||
)
|
||||
expected_mla = _require_int(arch["kv_lora_rank"], "kv_lora_rank") + _require_int(
|
||||
arch["qk_rope_head_dim"], "qk_rope_head_dim"
|
||||
)
|
||||
if mla != expected_mla:
|
||||
raise GlmTargetError(
|
||||
f"mla_cached_values_per_token_per_layer {mla} != kv_lora_rank + "
|
||||
f"qk_rope_head_dim ({expected_mla})"
|
||||
)
|
||||
|
||||
reasoning = _require_mapping(doc.get("reasoning_effort"), f"'reasoning_effort' in {source}")
|
||||
if reasoning.get("alpha_mode") != "max":
|
||||
raise GlmTargetError(
|
||||
f"{source} does not lock reasoning_effort=max; alpha is defined as the "
|
||||
"Max reasoning mode of this exact checkpoint"
|
||||
)
|
||||
_require_text(reasoning.get("rendered_marker"), "reasoning_effort.rendered_marker")
|
||||
|
||||
files_raw = doc.get("source_files")
|
||||
if not isinstance(files_raw, list) or not files_raw:
|
||||
raise GlmTargetError(f"'source_files' in {source} must be a non-empty JSON array")
|
||||
source_files: dict[str, Mapping[str, Any]] = {}
|
||||
for position, entry in enumerate(files_raw):
|
||||
item = _require_mapping(entry, f"source_files[{position}]")
|
||||
path = _require_text(item.get("path"), f"source_files[{position}].path")
|
||||
_require_sha256(item.get("sha256"), f"source_files[{path}].sha256")
|
||||
source_files[path] = item
|
||||
for required in ("config.json", "chat_template.jinja"):
|
||||
if required not in source_files:
|
||||
raise GlmTargetError(
|
||||
f"{source} does not pin {required!r}; config and chat-template drift "
|
||||
"silently changes runtime semantics"
|
||||
)
|
||||
|
||||
return ArchitectureSnapshot(
|
||||
schema_version=schema_version,
|
||||
source_repo_id=_require_text(doc.get("source_repo_id"), "source_repo_id"),
|
||||
source_revision=_require_revision(doc.get("source_revision"), "source_revision"),
|
||||
architecture=arch,
|
||||
reasoning_effort=reasoning,
|
||||
source_files=source_files,
|
||||
raw=doc,
|
||||
source=source,
|
||||
)
|
||||
|
||||
|
||||
def _read_resource(resource: str, path: Path | None) -> tuple[str, str]:
|
||||
if path is not None:
|
||||
try:
|
||||
return str(path), path.read_text(encoding="utf-8")
|
||||
except OSError as exc:
|
||||
raise GlmTargetError(f"cannot read {path}: {exc.strerror or exc}") from exc
|
||||
source = f"packaged {resource}"
|
||||
try:
|
||||
raw = files("meshnet_node.glm_alpha").joinpath("data", resource).read_text(encoding="utf-8")
|
||||
except (OSError, FileNotFoundError, ModuleNotFoundError) as exc:
|
||||
raise GlmTargetError(
|
||||
f"{source} is missing from this node installation ({type(exc).__name__})"
|
||||
) from exc
|
||||
return source, raw
|
||||
|
||||
|
||||
def _load_json(resource: str, path: Path | None) -> tuple[str, Any]:
|
||||
source, raw = _read_resource(resource, path)
|
||||
try:
|
||||
return source, json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise GlmTargetError(
|
||||
f"{source} is not valid JSON: {exc.msg} at line {exc.lineno} column {exc.colno}"
|
||||
) from exc
|
||||
|
||||
|
||||
def load_target_manifest(path: Path | None = None) -> TargetManifest:
|
||||
"""Load the packaged target manifest, or one at ``path``."""
|
||||
source, data = _load_json(_MANIFEST_RESOURCE, path)
|
||||
return parse_target_manifest(data, source=source)
|
||||
|
||||
|
||||
def load_architecture_snapshot(path: Path | None = None) -> ArchitectureSnapshot:
|
||||
"""Load the packaged architecture snapshot, or one at ``path``."""
|
||||
source, data = _load_json(_ARCHITECTURE_RESOURCE, path)
|
||||
return parse_architecture_snapshot(data, source=source)
|
||||
|
||||
|
||||
def require_pinned_target(
|
||||
manifest: TargetManifest,
|
||||
snapshot: ArchitectureSnapshot,
|
||||
*,
|
||||
expected_source_revision: str,
|
||||
expected_gguf_revision: str,
|
||||
) -> None:
|
||||
"""Reject any target whose revisions are not the ones alpha was locked against.
|
||||
|
||||
Callers pass the revisions from the locked alpha contract, so a swapped
|
||||
manifest cannot quietly re-point the target at a different upstream commit.
|
||||
"""
|
||||
if manifest.source_revision != expected_source_revision:
|
||||
raise GlmTargetError(
|
||||
f"source revision {manifest.source_revision} does not match the locked "
|
||||
f"alpha revision {expected_source_revision}"
|
||||
)
|
||||
if manifest.gguf_revision != expected_gguf_revision:
|
||||
raise GlmTargetError(
|
||||
f"GGUF revision {manifest.gguf_revision} does not match the locked alpha "
|
||||
f"revision {expected_gguf_revision}"
|
||||
)
|
||||
if snapshot.source_revision != manifest.source_revision:
|
||||
raise GlmTargetError(
|
||||
f"the architecture snapshot was taken at {snapshot.source_revision} but the "
|
||||
f"manifest pins {manifest.source_revision}; config metadata and weights must "
|
||||
"come from one revision"
|
||||
)
|
||||
522
packages/node/meshnet_node/glm_alpha/planner.py
Normal file
522
packages/node/meshnet_node/glm_alpha/planner.py
Normal file
@@ -0,0 +1,522 @@
|
||||
"""Deterministic memory, KV, and network planner for the GLM-5.2 Max alpha route.
|
||||
|
||||
Everything here is arithmetic over the exact pinned artifact bytes and the exact
|
||||
pinned architecture. There is no measurement, no probing, and no heuristic tuned
|
||||
to a result — the planner is written *before* the target runs so that a later
|
||||
story cannot discover a topology that "works" and then rationalise it.
|
||||
|
||||
Three ideas do the real work.
|
||||
|
||||
**Unified memory is one pool.** On an integrated-GPU machine the "VRAM" the driver
|
||||
reports is carved out of the same physical DRAM the OS is already counting. Adding
|
||||
them produces a node that appears to hold twice what it holds, and the failure mode
|
||||
is not a clean admission rejection — it is an OOM or a swap-thrash halfway through
|
||||
a 200 GiB load. :class:`NodeMemory` therefore refuses to be constructed from an
|
||||
additive claim about one shared pool.
|
||||
|
||||
**The reserve is not optional headroom.** Weights plus KV are not the whole
|
||||
resident cost: backend workspaces, quantization scratch, the graph plan, the
|
||||
process, and the OS all live outside them, and the largest of those scale with the
|
||||
backend rather than with the shard. Alpha reserves ``max(20% of physically usable
|
||||
memory, 8 GiB)`` per node, and the *remainder* is the placement budget.
|
||||
|
||||
**Equal layer counts are not equal bytes.** Embeddings and the output head are
|
||||
endpoint-only; three layers are dense and 75 are MoE; shared experts, indexer
|
||||
tensors, and quant block alignment all skew the per-node share. Until DGR-018/019
|
||||
report measured per-tensor placement, the planner carries an explicit
|
||||
:data:`PLACEMENT_IMBALANCE_FACTOR` and reports the arithmetic minimum and the
|
||||
recommended count as two separate numbers. The arithmetic minimum is a fit probe;
|
||||
it is admissible only with exact measured placement evidence behind it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from .manifest import GIB, ArchitectureSnapshot, TargetManifest
|
||||
|
||||
# Q8_0 stores 32 int8 quants plus one fp16 scale per block: 34 bytes / 32 values.
|
||||
Q8_0_BYTES_PER_VALUE = 34 / 32
|
||||
F16_BYTES_PER_VALUE = 2.0
|
||||
|
||||
KV_DTYPES: dict[str, float] = {
|
||||
"Q8_0": Q8_0_BYTES_PER_VALUE,
|
||||
"F16": F16_BYTES_PER_VALUE,
|
||||
}
|
||||
|
||||
# The alpha KV configuration, locked by the roadmap.
|
||||
ALPHA_KV_DTYPE = "Q8_0"
|
||||
ALPHA_CONTEXT_TOKENS = 16384
|
||||
ALPHA_CONCURRENCY = 1
|
||||
|
||||
# The reserve every node holds outside its weight-plus-KV placement budget.
|
||||
RESERVE_FRACTION = 0.20
|
||||
RESERVE_FLOOR_GIB = 8.0
|
||||
|
||||
# The aggregate runtime-accessible memory at which the artifact *just* fits.
|
||||
# This is an experimental hard-fit floor, not an operational envelope: it has no
|
||||
# room for a backend that allocates more scratch than another, and none for the
|
||||
# imbalance below.
|
||||
AGGREGATE_HARD_FIT_FLOOR_GIB = 224.0
|
||||
|
||||
# How much more than an equal share the worst-placed node is expected to hold.
|
||||
# 1.10 is the roadmap's recommended-topology column expressed as arithmetic: it
|
||||
# reproduces 10 / 6 / 5 / 3 / 3 nodes for the 32 / 48 / 64 / 96 / 128 GiB tiers.
|
||||
# DGR-019 must replace it with measured per-tensor placement.
|
||||
PLACEMENT_IMBALANCE_FACTOR = 1.10
|
||||
|
||||
# Alpha network floor. A link rate is a bandwidth claim, never a speed claim.
|
||||
MIN_LINK_RATE_GBPS = 2.5
|
||||
RECOMMENDED_LINK_RATE_GBPS = 10.0
|
||||
|
||||
BF16_BYTES = 2
|
||||
DSA_SIDEBAND_INT32_BYTES = 4
|
||||
|
||||
IndexerLayout = Literal["optimized", "conservative"]
|
||||
|
||||
|
||||
class ResourcePlanError(ValueError):
|
||||
"""Raised when a node or route cannot be accounted for honestly."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NodeMemory:
|
||||
"""One node's physically usable memory, counted once.
|
||||
|
||||
``physical_usable_gib`` is what the node can actually place bytes into after
|
||||
firmware and fixed carve-outs — not the marketing capacity, and not a sum of
|
||||
two views of the same DRAM.
|
||||
"""
|
||||
|
||||
name: str
|
||||
physical_usable_gib: float
|
||||
unified: bool
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if not isinstance(self.name, str) or not self.name.strip():
|
||||
raise ResourcePlanError("node name must be a non-empty physical-host identity")
|
||||
if (
|
||||
isinstance(self.physical_usable_gib, bool)
|
||||
or not isinstance(self.physical_usable_gib, (int, float))
|
||||
or not math.isfinite(self.physical_usable_gib)
|
||||
or self.physical_usable_gib <= 0
|
||||
):
|
||||
raise ResourcePlanError(
|
||||
f"node {self.name!r} must declare finite positive usable memory"
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_host(
|
||||
cls,
|
||||
name: str,
|
||||
*,
|
||||
system_ram_gib: float,
|
||||
gpu_memory_gib: float = 0.0,
|
||||
unified: bool,
|
||||
) -> "NodeMemory":
|
||||
"""Build a node from a host's reported RAM and GPU memory.
|
||||
|
||||
On a unified machine the GPU memory *is* system RAM, so it is counted once
|
||||
and never added. Passing a non-zero ``gpu_memory_gib`` alongside
|
||||
``unified=True`` is the double-count this project has already decided is a
|
||||
bug (RALPH-CONTEXT runtime decision 16), so it is rejected rather than
|
||||
silently discarded: a caller who believes an integrated GPU adds memory has
|
||||
a wrong model of the machine, and quietly ignoring the argument would let
|
||||
that belief survive.
|
||||
"""
|
||||
if not isinstance(unified, bool):
|
||||
raise ResourcePlanError(f"node {name!r} unified flag must be boolean")
|
||||
for value, label, allow_zero in (
|
||||
(system_ram_gib, "system RAM", False),
|
||||
(gpu_memory_gib, "GPU memory", True),
|
||||
):
|
||||
if (
|
||||
isinstance(value, bool)
|
||||
or not isinstance(value, (int, float))
|
||||
or not math.isfinite(value)
|
||||
or value < 0
|
||||
or (not allow_zero and value == 0)
|
||||
):
|
||||
qualifier = "finite non-negative" if allow_zero else "finite positive"
|
||||
raise ResourcePlanError(f"node {name!r} must declare {qualifier} {label}")
|
||||
if unified:
|
||||
if gpu_memory_gib:
|
||||
raise ResourcePlanError(
|
||||
f"node {name!r} declares unified memory and {gpu_memory_gib} GiB of "
|
||||
"separate GPU memory. Integrated-GPU memory is carved out of the same "
|
||||
"physical DRAM as system RAM; adding them double-counts one pool. "
|
||||
"Pass unified=True with system_ram_gib only."
|
||||
)
|
||||
usable = system_ram_gib
|
||||
else:
|
||||
usable = system_ram_gib + gpu_memory_gib
|
||||
return cls(name=name, physical_usable_gib=usable, unified=unified)
|
||||
|
||||
@property
|
||||
def reserve_gib(self) -> float:
|
||||
"""``max(20% of physically usable memory, 8 GiB)``."""
|
||||
return max(RESERVE_FRACTION * self.physical_usable_gib, RESERVE_FLOOR_GIB)
|
||||
|
||||
@property
|
||||
def placement_budget_gib(self) -> float:
|
||||
"""What remains for weights plus KV after the reserve."""
|
||||
return self.physical_usable_gib - self.reserve_gib
|
||||
|
||||
|
||||
def kv_bytes(
|
||||
snapshot: ArchitectureSnapshot,
|
||||
*,
|
||||
context_tokens: int = ALPHA_CONTEXT_TOKENS,
|
||||
concurrency: int = ALPHA_CONCURRENCY,
|
||||
dtype: str = ALPHA_KV_DTYPE,
|
||||
indexer_layout: IndexerLayout = "conservative",
|
||||
include_indexer: bool = True,
|
||||
) -> int:
|
||||
"""Bytes of MLA (and DSA indexer) KV cache for the whole model.
|
||||
|
||||
``indexer_layout`` is the honest part. Correct DSA only needs indexer keys for
|
||||
the Full producer layers, but the current experimental implementation may
|
||||
allocate them across every backbone layer. Alpha budgets ``conservative``
|
||||
(all 78) so that a route admitted by this planner cannot be surprised by the
|
||||
implementation it actually gets.
|
||||
"""
|
||||
if (
|
||||
not isinstance(context_tokens, int)
|
||||
or isinstance(context_tokens, bool)
|
||||
or context_tokens <= 0
|
||||
or not isinstance(concurrency, int)
|
||||
or isinstance(concurrency, bool)
|
||||
or concurrency <= 0
|
||||
):
|
||||
raise ResourcePlanError("context_tokens and concurrency must be positive integers")
|
||||
if dtype not in KV_DTYPES:
|
||||
raise ResourcePlanError(
|
||||
f"unsupported KV dtype {dtype!r}; alpha locks {ALPHA_KV_DTYPE} "
|
||||
f"(known: {', '.join(sorted(KV_DTYPES))})"
|
||||
)
|
||||
bytes_per_value = KV_DTYPES[dtype]
|
||||
|
||||
layers = int(snapshot["num_hidden_layers"])
|
||||
mla_values = int(snapshot["mla_cached_values_per_token_per_layer"])
|
||||
total_values = mla_values * layers
|
||||
|
||||
if include_indexer:
|
||||
if indexer_layout == "optimized":
|
||||
indexer_layers = int(snapshot["indexer_full_layers"])
|
||||
elif indexer_layout == "conservative":
|
||||
indexer_layers = layers
|
||||
else: # pragma: no cover - Literal keeps this unreachable from typed callers
|
||||
raise ResourcePlanError(f"unknown indexer_layout {indexer_layout!r}")
|
||||
total_values += int(snapshot["index_head_dim"]) * indexer_layers
|
||||
|
||||
return int(total_values * context_tokens * concurrency * bytes_per_value)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TopologyPlan:
|
||||
"""The node count a homogeneous tier needs, and how it was reached."""
|
||||
|
||||
physical_usable_gib: float
|
||||
reserve_gib: float
|
||||
placement_budget_gib: float
|
||||
weight_gib: float
|
||||
kv_gib: float
|
||||
total_placement_gib: float
|
||||
arithmetic_minimum_nodes: int
|
||||
recommended_nodes: int
|
||||
imbalance_factor: float
|
||||
|
||||
@property
|
||||
def is_arithmetic_minimum_topology(self) -> bool:
|
||||
"""True when the recommendation offers no imbalance headroom at all."""
|
||||
return self.recommended_nodes == self.arithmetic_minimum_nodes
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"physical_usable_gib": round(self.physical_usable_gib, 3),
|
||||
"reserve_gib": round(self.reserve_gib, 3),
|
||||
"placement_budget_gib": round(self.placement_budget_gib, 3),
|
||||
"weight_gib": round(self.weight_gib, 3),
|
||||
"kv_gib": round(self.kv_gib, 3),
|
||||
"total_placement_gib": round(self.total_placement_gib, 3),
|
||||
"arithmetic_minimum_nodes": self.arithmetic_minimum_nodes,
|
||||
"recommended_nodes": self.recommended_nodes,
|
||||
"imbalance_factor": self.imbalance_factor,
|
||||
}
|
||||
|
||||
|
||||
def plan_topology(
|
||||
manifest: TargetManifest,
|
||||
snapshot: ArchitectureSnapshot,
|
||||
*,
|
||||
physical_usable_gib: float,
|
||||
context_tokens: int = ALPHA_CONTEXT_TOKENS,
|
||||
concurrency: int = ALPHA_CONCURRENCY,
|
||||
kv_dtype: str = ALPHA_KV_DTYPE,
|
||||
indexer_layout: IndexerLayout = "conservative",
|
||||
imbalance_factor: float = PLACEMENT_IMBALANCE_FACTOR,
|
||||
) -> TopologyPlan:
|
||||
"""Minimum and recommended node count for a homogeneous tier of this size."""
|
||||
if (
|
||||
isinstance(imbalance_factor, bool)
|
||||
or not isinstance(imbalance_factor, (int, float))
|
||||
or not math.isfinite(imbalance_factor)
|
||||
or imbalance_factor < 1.0
|
||||
):
|
||||
raise ResourcePlanError(
|
||||
"imbalance_factor must be finite and at least 1.0; a lower value would "
|
||||
"assume the worst-placed node holds less than an equal share"
|
||||
)
|
||||
|
||||
node = NodeMemory(
|
||||
name=f"{physical_usable_gib:g}GiB-tier",
|
||||
physical_usable_gib=physical_usable_gib,
|
||||
unified=False,
|
||||
)
|
||||
budget = node.placement_budget_gib
|
||||
if budget <= 0:
|
||||
raise ResourcePlanError(
|
||||
f"a {physical_usable_gib:g} GiB node has no placement budget after its "
|
||||
f"{node.reserve_gib:.1f} GiB reserve"
|
||||
)
|
||||
|
||||
weight_gib = manifest.total_bytes / GIB
|
||||
kv_gib = (
|
||||
kv_bytes(
|
||||
snapshot,
|
||||
context_tokens=context_tokens,
|
||||
concurrency=concurrency,
|
||||
dtype=kv_dtype,
|
||||
indexer_layout=indexer_layout,
|
||||
)
|
||||
/ GIB
|
||||
)
|
||||
total = weight_gib + kv_gib
|
||||
|
||||
return TopologyPlan(
|
||||
physical_usable_gib=physical_usable_gib,
|
||||
reserve_gib=node.reserve_gib,
|
||||
placement_budget_gib=budget,
|
||||
weight_gib=weight_gib,
|
||||
kv_gib=kv_gib,
|
||||
total_placement_gib=total,
|
||||
arithmetic_minimum_nodes=math.ceil(total / budget),
|
||||
recommended_nodes=math.ceil(total * imbalance_factor / budget),
|
||||
imbalance_factor=imbalance_factor,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RouteFit:
|
||||
"""Whether a concrete, possibly heterogeneous set of nodes can hold the target."""
|
||||
|
||||
node_count: int
|
||||
aggregate_usable_gib: float
|
||||
aggregate_placement_budget_gib: float
|
||||
required_placement_gib: float
|
||||
fits: bool
|
||||
meets_hard_fit_floor: bool
|
||||
no_single_node_can_admit_target: bool
|
||||
headroom_gib: float
|
||||
reasons: tuple[str, ...]
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"node_count": self.node_count,
|
||||
"aggregate_usable_gib": round(self.aggregate_usable_gib, 3),
|
||||
"aggregate_placement_budget_gib": round(self.aggregate_placement_budget_gib, 3),
|
||||
"required_placement_gib": round(self.required_placement_gib, 3),
|
||||
"fits": self.fits,
|
||||
"meets_hard_fit_floor": self.meets_hard_fit_floor,
|
||||
"no_single_node_can_admit_target": self.no_single_node_can_admit_target,
|
||||
"headroom_gib": round(self.headroom_gib, 3),
|
||||
"reasons": list(self.reasons),
|
||||
}
|
||||
|
||||
|
||||
def plan_route(
|
||||
manifest: TargetManifest,
|
||||
snapshot: ArchitectureSnapshot,
|
||||
nodes: list[NodeMemory],
|
||||
*,
|
||||
context_tokens: int = ALPHA_CONTEXT_TOKENS,
|
||||
concurrency: int = ALPHA_CONCURRENCY,
|
||||
kv_dtype: str = ALPHA_KV_DTYPE,
|
||||
indexer_layout: IndexerLayout = "conservative",
|
||||
) -> RouteFit:
|
||||
"""Evaluate a concrete route. Every node's memory is already counted once."""
|
||||
if len(nodes) < 2:
|
||||
raise ResourcePlanError(
|
||||
"the alpha target is distributed by definition; a route needs at least two "
|
||||
"physical nodes"
|
||||
)
|
||||
names = [node.name for node in nodes]
|
||||
if len(set(names)) != len(names):
|
||||
raise ResourcePlanError(
|
||||
"duplicate node names in the route; one physical machine counted twice is "
|
||||
"the same double-count as adding integrated-GPU memory to system RAM"
|
||||
)
|
||||
|
||||
weight_gib = manifest.total_bytes / GIB
|
||||
kv_gib = (
|
||||
kv_bytes(
|
||||
snapshot,
|
||||
context_tokens=context_tokens,
|
||||
concurrency=concurrency,
|
||||
dtype=kv_dtype,
|
||||
indexer_layout=indexer_layout,
|
||||
)
|
||||
/ GIB
|
||||
)
|
||||
required = weight_gib + kv_gib
|
||||
|
||||
aggregate_usable = sum(node.physical_usable_gib for node in nodes)
|
||||
aggregate_budget = sum(node.placement_budget_gib for node in nodes)
|
||||
fits = aggregate_budget >= required
|
||||
largest_budget = max(node.placement_budget_gib for node in nodes)
|
||||
no_single_node = largest_budget < required
|
||||
|
||||
reasons: list[str] = []
|
||||
if not fits:
|
||||
reasons.append(
|
||||
f"aggregate placement budget {aggregate_budget:.1f} GiB is below the "
|
||||
f"{required:.1f} GiB the target needs after each node's reserve"
|
||||
)
|
||||
if not no_single_node:
|
||||
reasons.append(
|
||||
"at least one node could admit the complete target alone; that is a "
|
||||
"single-host run, not distributed alpha"
|
||||
)
|
||||
if aggregate_usable < AGGREGATE_HARD_FIT_FLOOR_GIB:
|
||||
reasons.append(
|
||||
f"aggregate usable memory {aggregate_usable:.1f} GiB is below the "
|
||||
f"{AGGREGATE_HARD_FIT_FLOOR_GIB:g} GiB experimental hard-fit floor"
|
||||
)
|
||||
|
||||
return RouteFit(
|
||||
node_count=len(nodes),
|
||||
aggregate_usable_gib=aggregate_usable,
|
||||
aggregate_placement_budget_gib=aggregate_budget,
|
||||
required_placement_gib=required,
|
||||
fits=fits,
|
||||
meets_hard_fit_floor=aggregate_usable >= AGGREGATE_HARD_FIT_FLOOR_GIB,
|
||||
no_single_node_can_admit_target=no_single_node,
|
||||
headroom_gib=aggregate_budget - required,
|
||||
reasons=tuple(reasons),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SeamPlan:
|
||||
"""Bytes and latency across the activation seams of a route.
|
||||
|
||||
Bandwidth and latency are reported separately on purpose. Decode moves almost
|
||||
nothing — 12 KiB per token per seam — so a faster link barely helps it. What
|
||||
decode pays is *serial*: every generated token crosses every seam in order, so
|
||||
the cost that matters is ``seams x per-hop latency``. A route that claims to be
|
||||
fast because it is on 10 GbE has confused the two.
|
||||
"""
|
||||
|
||||
node_count: int
|
||||
seam_count: int
|
||||
hidden_size: int
|
||||
bytes_per_token_per_seam: int
|
||||
prefill_bytes_per_seam: int
|
||||
decode_bytes_per_seam_per_token: int
|
||||
dsa_sideband_bytes_per_query: int
|
||||
link_rate_gbps: float
|
||||
meets_alpha_minimum: bool
|
||||
is_recommended_link: bool
|
||||
decode_serialization_ms_per_token: float
|
||||
decode_latency_ms_per_token: float
|
||||
decode_bandwidth_share_ms_per_token: float
|
||||
prefill_serialization_ms: float
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"node_count": self.node_count,
|
||||
"seam_count": self.seam_count,
|
||||
"hidden_size": self.hidden_size,
|
||||
"bytes_per_token_per_seam": self.bytes_per_token_per_seam,
|
||||
"prefill_bytes_per_seam": self.prefill_bytes_per_seam,
|
||||
"decode_bytes_per_seam_per_token": self.decode_bytes_per_seam_per_token,
|
||||
"dsa_sideband_bytes_per_query": self.dsa_sideband_bytes_per_query,
|
||||
"link_rate_gbps": self.link_rate_gbps,
|
||||
"meets_alpha_minimum": self.meets_alpha_minimum,
|
||||
"is_recommended_link": self.is_recommended_link,
|
||||
"decode_serialization_ms_per_token": round(self.decode_serialization_ms_per_token, 4),
|
||||
"decode_latency_ms_per_token": round(self.decode_latency_ms_per_token, 4),
|
||||
"decode_bandwidth_share_ms_per_token": round(
|
||||
self.decode_bandwidth_share_ms_per_token, 4
|
||||
),
|
||||
"prefill_serialization_ms": round(self.prefill_serialization_ms, 3),
|
||||
}
|
||||
|
||||
|
||||
def plan_seams(
|
||||
snapshot: ArchitectureSnapshot,
|
||||
*,
|
||||
node_count: int,
|
||||
context_tokens: int = ALPHA_CONTEXT_TOKENS,
|
||||
link_rate_gbps: float = MIN_LINK_RATE_GBPS,
|
||||
per_hop_latency_ms: float = 0.5,
|
||||
) -> SeamPlan:
|
||||
"""Model seam bytes, wire serialization, and serial per-hop latency separately."""
|
||||
if not isinstance(node_count, int) or isinstance(node_count, bool) or node_count < 2:
|
||||
raise ResourcePlanError("a seam exists only between two nodes")
|
||||
if not isinstance(context_tokens, int) or isinstance(context_tokens, bool) or context_tokens <= 0:
|
||||
raise ResourcePlanError("context_tokens must be a positive integer")
|
||||
if (
|
||||
isinstance(link_rate_gbps, bool)
|
||||
or not isinstance(link_rate_gbps, (int, float))
|
||||
or not math.isfinite(link_rate_gbps)
|
||||
or link_rate_gbps <= 0
|
||||
):
|
||||
raise ResourcePlanError("link_rate_gbps must be finite and positive")
|
||||
if (
|
||||
isinstance(per_hop_latency_ms, bool)
|
||||
or not isinstance(per_hop_latency_ms, (int, float))
|
||||
or not math.isfinite(per_hop_latency_ms)
|
||||
or per_hop_latency_ms < 0
|
||||
):
|
||||
raise ResourcePlanError("per_hop_latency_ms must be finite and non-negative")
|
||||
|
||||
hidden = int(snapshot["hidden_size"])
|
||||
bytes_per_token = hidden * BF16_BYTES
|
||||
seams = node_count - 1
|
||||
|
||||
bits_per_ms = link_rate_gbps * 1e9 / 1e3
|
||||
decode_serialization_ms = (bytes_per_token * 8) / bits_per_ms
|
||||
prefill_serialization_ms = (bytes_per_token * context_tokens * 8) / bits_per_ms
|
||||
|
||||
return SeamPlan(
|
||||
node_count=node_count,
|
||||
seam_count=seams,
|
||||
hidden_size=hidden,
|
||||
bytes_per_token_per_seam=bytes_per_token,
|
||||
prefill_bytes_per_seam=bytes_per_token * context_tokens,
|
||||
decode_bytes_per_seam_per_token=bytes_per_token,
|
||||
dsa_sideband_bytes_per_query=int(snapshot["index_topk"]) * DSA_SIDEBAND_INT32_BYTES,
|
||||
link_rate_gbps=link_rate_gbps,
|
||||
meets_alpha_minimum=link_rate_gbps >= MIN_LINK_RATE_GBPS,
|
||||
is_recommended_link=link_rate_gbps >= RECOMMENDED_LINK_RATE_GBPS,
|
||||
decode_serialization_ms_per_token=decode_serialization_ms * seams,
|
||||
decode_latency_ms_per_token=per_hop_latency_ms * seams,
|
||||
decode_bandwidth_share_ms_per_token=decode_serialization_ms * seams,
|
||||
prefill_serialization_ms=prefill_serialization_ms * seams,
|
||||
)
|
||||
|
||||
|
||||
ALPHA_TIERS_GIB: tuple[float, ...] = (32.0, 48.0, 64.0, 96.0, 128.0)
|
||||
|
||||
|
||||
def plan_all_tiers(
|
||||
manifest: TargetManifest, snapshot: ArchitectureSnapshot
|
||||
) -> dict[str, TopologyPlan]:
|
||||
"""The alpha tier table, recomputed from the pinned artifact and architecture."""
|
||||
return {
|
||||
f"{tier:g}": plan_topology(manifest, snapshot, physical_usable_gib=tier)
|
||||
for tier in ALPHA_TIERS_GIB
|
||||
}
|
||||
185
packages/node/meshnet_node/native_backend.py
Normal file
185
packages/node/meshnet_node/native_backend.py
Normal file
@@ -0,0 +1,185 @@
|
||||
"""Authoritative identity boundary for a native GGUF Shard backend.
|
||||
|
||||
The native loader owns the facts about the mapped artifact. This module turns
|
||||
that immutable report and separately pinned deployment inputs into the one
|
||||
``ShardIdentity`` DGR-003 permits a native worker to emit. It is intentionally
|
||||
not an adapter for the legacy Transformers backend: that backend has no
|
||||
authoritative immutable GGUF artifact pin and must remain identity-free.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .native_protocol import BUNDLE_VERSION, SCHEMA_VERSION, pb
|
||||
from .runtime_recipe import (
|
||||
ArtifactIdentity,
|
||||
DerivativeBinding,
|
||||
RecipeIdentityError,
|
||||
RuntimeRecipe,
|
||||
ShardIdentity,
|
||||
check_session_open,
|
||||
handshake_error,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NativeLoadedArtifactReport:
|
||||
"""Immutable GGUF facts returned by the loaded native model.
|
||||
|
||||
The report is copied from ``llama_model_meshnet_range_report`` plus the
|
||||
parsed GGUF metadata while the model is live. Byte counts are operational
|
||||
evidence rather than compatibility axes, but keeping them beside the range
|
||||
prevents a caller from substituting an unverified range declaration.
|
||||
"""
|
||||
|
||||
owned_start_layer: int
|
||||
owned_end_layer: int
|
||||
mapped_bytes: int
|
||||
resident_bytes: int
|
||||
registered_bytes: int
|
||||
architecture: str
|
||||
architecture_digest: str
|
||||
layer_count: int
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.owned_start_layer < 0 or self.owned_end_layer <= self.owned_start_layer:
|
||||
raise RecipeIdentityError("native report has an invalid owned layer range")
|
||||
if self.layer_count < 1 or self.owned_end_layer > self.layer_count:
|
||||
raise RecipeIdentityError("native report range is outside GGUF layer metadata")
|
||||
if min(self.mapped_bytes, self.resident_bytes, self.registered_bytes) < 0:
|
||||
raise RecipeIdentityError("native report byte counts must be non-negative")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImmutableArtifactPin:
|
||||
"""Deployment-supplied immutable bytes pin, never inferred from a model name."""
|
||||
|
||||
artifact_id: str
|
||||
revision: str
|
||||
content_digest: str
|
||||
derived_from: DerivativeBinding | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NativeNumericalRecipe:
|
||||
"""Immutable numerical inputs selected for this native worker instance."""
|
||||
|
||||
weight_quantization: str
|
||||
activation_dtype: str
|
||||
compute_dtype: str
|
||||
kv_dtype: str
|
||||
kv_layout: str
|
||||
architecture_adapter: str
|
||||
backend_id: str
|
||||
runtime_version: str
|
||||
recipe_id: str
|
||||
recipe_version: str
|
||||
catalogue_version: str
|
||||
boundary_schema_version: int = BUNDLE_VERSION
|
||||
protocol_schema_version: int = int(SCHEMA_VERSION)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NativeIdentityInputs:
|
||||
"""Everything a native backend needs to emit one exact identity."""
|
||||
|
||||
loaded_artifact: NativeLoadedArtifactReport
|
||||
artifact_pin: ImmutableArtifactPin
|
||||
tokenizer_revision: str
|
||||
numerical_recipe: NativeNumericalRecipe
|
||||
|
||||
|
||||
def shard_identity_from_native_report(inputs: NativeIdentityInputs) -> ShardIdentity:
|
||||
"""Derive identity only from the native report and immutable pinned inputs."""
|
||||
report = inputs.loaded_artifact
|
||||
pin = inputs.artifact_pin
|
||||
recipe = inputs.numerical_recipe
|
||||
artifact = ArtifactIdentity(
|
||||
artifact_id=pin.artifact_id,
|
||||
revision=pin.revision,
|
||||
content_digest=pin.content_digest,
|
||||
architecture=report.architecture,
|
||||
architecture_digest=report.architecture_digest,
|
||||
layer_count=report.layer_count,
|
||||
derived_from=pin.derived_from,
|
||||
)
|
||||
return ShardIdentity(
|
||||
artifact=artifact,
|
||||
recipe=RuntimeRecipe(
|
||||
weight_quantization=recipe.weight_quantization,
|
||||
activation_dtype=recipe.activation_dtype,
|
||||
compute_dtype=recipe.compute_dtype,
|
||||
kv_dtype=recipe.kv_dtype,
|
||||
kv_layout=recipe.kv_layout,
|
||||
tokenizer_revision=inputs.tokenizer_revision,
|
||||
architecture_adapter=recipe.architecture_adapter,
|
||||
backend_id=recipe.backend_id,
|
||||
runtime_version=recipe.runtime_version,
|
||||
boundary_schema_version=recipe.boundary_schema_version,
|
||||
protocol_schema_version=recipe.protocol_schema_version,
|
||||
recipe_id=recipe.recipe_id,
|
||||
recipe_version=recipe.recipe_version,
|
||||
catalogue_version=recipe.catalogue_version,
|
||||
),
|
||||
shard_start=report.owned_start_layer,
|
||||
shard_end=report.owned_end_layer,
|
||||
)
|
||||
|
||||
|
||||
class NativeSessionRejected(RecipeIdentityError):
|
||||
"""A native worker refused a ``SessionOpen`` before allocating session state."""
|
||||
|
||||
def __init__(self, error: "pb.ShardError") -> None:
|
||||
super().__init__(f"native SessionOpen rejected: {error.code}")
|
||||
self.error = error
|
||||
|
||||
|
||||
class NativeWorkerBackendAdapter:
|
||||
"""Small backend-facing adapter around the native loaded-artifact seam."""
|
||||
|
||||
def __init__(self, identity_inputs: NativeIdentityInputs) -> None:
|
||||
self.identity_inputs = identity_inputs
|
||||
self.identity = shard_identity_from_native_report(identity_inputs)
|
||||
|
||||
@property
|
||||
def loaded_artifact_report(self) -> NativeLoadedArtifactReport:
|
||||
return self.identity_inputs.loaded_artifact
|
||||
|
||||
def check_session_open(
|
||||
self,
|
||||
opened: "pb.SessionOpen",
|
||||
*,
|
||||
expected_route_session_id: str | None = None,
|
||||
expected_route_epoch: int | None = None,
|
||||
) -> None:
|
||||
"""Reject incompatible/stale opens at the native worker boundary."""
|
||||
mismatches = check_session_open(
|
||||
self.identity,
|
||||
opened,
|
||||
expected_route_session_id=expected_route_session_id,
|
||||
expected_route_epoch=expected_route_epoch,
|
||||
)
|
||||
error = handshake_error(mismatches)
|
||||
if error is not None:
|
||||
raise NativeSessionRejected(error)
|
||||
|
||||
def on_session_open(
|
||||
self,
|
||||
opened: "pb.SessionOpen",
|
||||
*,
|
||||
expected_route_session_id: str | None = None,
|
||||
expected_route_epoch: int | None = None,
|
||||
) -> "pb.SessionAccepted":
|
||||
"""The native worker's SessionOpen boundary, before session allocation."""
|
||||
self.check_session_open(
|
||||
opened,
|
||||
expected_route_session_id=expected_route_session_id,
|
||||
expected_route_epoch=expected_route_epoch,
|
||||
)
|
||||
return pb.SessionAccepted(
|
||||
schema_version=SCHEMA_VERSION,
|
||||
route_session_id=opened.route_session_id,
|
||||
route_epoch=opened.route_epoch,
|
||||
fingerprint=self.identity.fingerprint.to_proto(),
|
||||
)
|
||||
@@ -31,6 +31,8 @@ from .codec import (
|
||||
checksum_of,
|
||||
crc32c,
|
||||
decode_bundle,
|
||||
decode_step_bundle,
|
||||
encode_decode_step,
|
||||
decode_tensor,
|
||||
default_flow_control,
|
||||
encode_bundle,
|
||||
@@ -40,6 +42,7 @@ from .codec import (
|
||||
negotiate_flow_control,
|
||||
plan_prefill_chunks,
|
||||
validate_session_message_size,
|
||||
validate_tail_result,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
@@ -60,6 +63,8 @@ __all__ = [
|
||||
"checksum_of",
|
||||
"crc32c",
|
||||
"decode_bundle",
|
||||
"decode_step_bundle",
|
||||
"encode_decode_step",
|
||||
"decode_tensor",
|
||||
"default_flow_control",
|
||||
"encode_bundle",
|
||||
@@ -70,4 +75,5 @@ __all__ = [
|
||||
"pb",
|
||||
"plan_prefill_chunks",
|
||||
"validate_session_message_size",
|
||||
"validate_tail_result",
|
||||
]
|
||||
|
||||
@@ -328,6 +328,8 @@ def decode_tensor(
|
||||
def encode_bundle(
|
||||
tensors: Iterable[pb.NamedTensor],
|
||||
*,
|
||||
architecture: int = pb.ARCHITECTURE_TYPE_UNSPECIFIED,
|
||||
boundary_point: str = "",
|
||||
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
|
||||
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
|
||||
) -> pb.TensorBundle:
|
||||
@@ -338,7 +340,12 @@ def encode_bundle(
|
||||
raise ProtocolError(
|
||||
f"bundle carries {len(tensor_list)} tensors, exceeding limit {max_tensors}"
|
||||
)
|
||||
bundle = pb.TensorBundle(bundle_version=BUNDLE_VERSION, tensors=tensor_list)
|
||||
bundle = pb.TensorBundle(
|
||||
bundle_version=BUNDLE_VERSION,
|
||||
tensors=tensor_list,
|
||||
architecture=architecture,
|
||||
boundary_point=boundary_point,
|
||||
)
|
||||
if bundle.ByteSize() > max_chunk_bytes:
|
||||
raise ProtocolError(
|
||||
f"serialized tensor bundle exceeds the {max_chunk_bytes}-byte "
|
||||
@@ -392,6 +399,80 @@ def decode_bundle(
|
||||
return payloads
|
||||
|
||||
|
||||
def encode_decode_step(
|
||||
bundle: pb.TensorBundle,
|
||||
*,
|
||||
idempotency_step: int,
|
||||
position: int,
|
||||
expected_past_len: int,
|
||||
work_id: str,
|
||||
deadline_unix_nanos: int = 0,
|
||||
prefer_compact_one_tensor: bool = True,
|
||||
) -> pb.DecodeStep:
|
||||
"""Encode a decode boundary, retaining the deliberate compact fallback."""
|
||||
step = pb.DecodeStep(
|
||||
idempotency_step=idempotency_step,
|
||||
position=position,
|
||||
expected_past_len=expected_past_len,
|
||||
work_id=work_id,
|
||||
deadline_unix_nanos=deadline_unix_nanos,
|
||||
)
|
||||
if prefer_compact_one_tensor and len(bundle.tensors) == 1:
|
||||
step.tensor.CopyFrom(bundle.tensors[0])
|
||||
else:
|
||||
step.bundle.CopyFrom(bundle)
|
||||
return step
|
||||
|
||||
|
||||
def validate_tail_result(result: pb.TailResult) -> None:
|
||||
"""Fail closed unless a tail completion is bound to its exact recipe."""
|
||||
identity = result.identity
|
||||
required = (
|
||||
identity.request_id,
|
||||
identity.runtime_recipe_digest,
|
||||
identity.chat_template_id,
|
||||
identity.chat_template_version,
|
||||
identity.reasoning_mode,
|
||||
)
|
||||
if not all(required) or identity.architecture == pb.ARCHITECTURE_TYPE_UNSPECIFIED:
|
||||
raise ProtocolError("tail result lacks exact request/recipe/template identity")
|
||||
if result.WhichOneof("output") not in {"logits", "sampled_token_id"}:
|
||||
raise ProtocolError("tail result lacks logits or sampled token output")
|
||||
|
||||
|
||||
def decode_step_bundle(
|
||||
step: pb.DecodeStep,
|
||||
*,
|
||||
max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
|
||||
max_fragment_bytes: int = DEFAULT_MAX_FRAGMENT_BYTES,
|
||||
max_fragments: int = DEFAULT_MAX_FRAGMENTS_PER_TENSOR,
|
||||
max_tensors: int = DEFAULT_MAX_TENSORS_PER_BUNDLE,
|
||||
) -> dict[str, bytes]:
|
||||
"""Decode a fast-path boundary with the DGR-006 compatibility rule.
|
||||
|
||||
`bundle` is authoritative because it can carry architecture sidebands. The
|
||||
old `tensor` field remains the compact representation for a certified
|
||||
one-tensor boundary and is accepted by new readers during rollout.
|
||||
"""
|
||||
if step.HasField("bundle"):
|
||||
return decode_bundle(
|
||||
step.bundle,
|
||||
max_chunk_bytes=max_chunk_bytes,
|
||||
max_fragment_bytes=max_fragment_bytes,
|
||||
max_fragments=max_fragments,
|
||||
max_tensors=max_tensors,
|
||||
)
|
||||
if step.HasField("tensor"):
|
||||
return decode_bundle(
|
||||
encode_bundle([step.tensor], max_chunk_bytes=max_chunk_bytes),
|
||||
max_chunk_bytes=max_chunk_bytes,
|
||||
max_fragment_bytes=max_fragment_bytes,
|
||||
max_fragments=max_fragments,
|
||||
max_tensors=max_tensors,
|
||||
)
|
||||
raise ProtocolError("decode step carries neither TensorBundle nor legacy tensor")
|
||||
|
||||
|
||||
def validate_session_message_size(
|
||||
message: pb.SessionRequest | pb.SessionResponse,
|
||||
*,
|
||||
|
||||
@@ -27,6 +27,7 @@ TESTDATA_DIR = pathlib.Path(__file__).resolve().parents[2] / "native/testdata"
|
||||
|
||||
GOLDEN_SESSION_REQUEST = "session_request_golden.binpb"
|
||||
GOLDEN_CAPABILITY_REPORT = "capability_report_golden.binpb"
|
||||
GOLDEN_DECODE_STEP = "decode_step_golden.binpb"
|
||||
|
||||
# Written by the C++ conformance test into its build tree; the Python test picks
|
||||
# it up when present to prove the two languages agree byte-for-byte.
|
||||
@@ -136,6 +137,30 @@ def canonical_capability_report() -> pb.CapabilityReport:
|
||||
)
|
||||
|
||||
|
||||
def canonical_decode_step() -> pb.SessionRequest:
|
||||
"""The DGR-006 multi-tensor decode boundary vector."""
|
||||
hidden = codec.encode_tensor(
|
||||
codec.HIDDEN_STATES, bytes(range(16)), [1, 1, 4], pb.DTYPE_FLOAT32
|
||||
)
|
||||
index_topk = codec.encode_tensor(
|
||||
"index_topk", (3).to_bytes(4, "little"), [1], pb.DTYPE_INT32
|
||||
)
|
||||
return pb.SessionRequest(
|
||||
decode=pb.DecodeStep(
|
||||
idempotency_step=43,
|
||||
position=384,
|
||||
expected_past_len=384,
|
||||
work_id="decode-7f3a",
|
||||
deadline_unix_nanos=DEADLINE_UNIX_NANOS,
|
||||
bundle=codec.encode_bundle(
|
||||
[hidden, index_topk],
|
||||
architecture=pb.ARCHITECTURE_TYPE_MLA,
|
||||
boundary_point="pre_tail_residual",
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def serialize(message) -> bytes:
|
||||
"""Serialize deterministically, so committed golden bytes are stable."""
|
||||
return message.SerializeToString(deterministic=True)
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -12,6 +12,13 @@ class SchemaVersion(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
||||
SCHEMA_VERSION_UNSPECIFIED: _ClassVar[SchemaVersion]
|
||||
SCHEMA_VERSION_1: _ClassVar[SchemaVersion]
|
||||
|
||||
class ArchitectureType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
||||
__slots__ = ()
|
||||
ARCHITECTURE_TYPE_UNSPECIFIED: _ClassVar[ArchitectureType]
|
||||
ARCHITECTURE_TYPE_DENSE: _ClassVar[ArchitectureType]
|
||||
ARCHITECTURE_TYPE_MOE: _ClassVar[ArchitectureType]
|
||||
ARCHITECTURE_TYPE_MLA: _ClassVar[ArchitectureType]
|
||||
|
||||
class DType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
||||
__slots__ = ()
|
||||
DTYPE_UNSPECIFIED: _ClassVar[DType]
|
||||
@@ -80,6 +87,10 @@ class ServingState(int, metaclass=_enum_type_wrapper.EnumTypeWrapper):
|
||||
SERVING_STATE_NOT_SERVING: _ClassVar[ServingState]
|
||||
SCHEMA_VERSION_UNSPECIFIED: SchemaVersion
|
||||
SCHEMA_VERSION_1: SchemaVersion
|
||||
ARCHITECTURE_TYPE_UNSPECIFIED: ArchitectureType
|
||||
ARCHITECTURE_TYPE_DENSE: ArchitectureType
|
||||
ARCHITECTURE_TYPE_MOE: ArchitectureType
|
||||
ARCHITECTURE_TYPE_MLA: ArchitectureType
|
||||
DTYPE_UNSPECIFIED: DType
|
||||
DTYPE_BFLOAT16: DType
|
||||
DTYPE_FLOAT16: DType
|
||||
@@ -165,12 +176,16 @@ class NamedTensor(_message.Message):
|
||||
def __init__(self, name: _Optional[str] = ..., shape: _Optional[_Iterable[int]] = ..., dtype: _Optional[_Union[DType, str]] = ..., byte_order: _Optional[_Union[ByteOrder, str]] = ..., total_bytes: _Optional[int] = ..., compression: _Optional[_Union[Compression, str]] = ..., checksum: _Optional[_Union[Checksum, _Mapping]] = ..., fragments: _Optional[_Iterable[_Union[TensorFragment, _Mapping]]] = ...) -> None: ...
|
||||
|
||||
class TensorBundle(_message.Message):
|
||||
__slots__ = ("bundle_version", "tensors")
|
||||
__slots__ = ("bundle_version", "tensors", "architecture", "boundary_point")
|
||||
BUNDLE_VERSION_FIELD_NUMBER: _ClassVar[int]
|
||||
TENSORS_FIELD_NUMBER: _ClassVar[int]
|
||||
ARCHITECTURE_FIELD_NUMBER: _ClassVar[int]
|
||||
BOUNDARY_POINT_FIELD_NUMBER: _ClassVar[int]
|
||||
bundle_version: int
|
||||
tensors: _containers.RepeatedCompositeFieldContainer[NamedTensor]
|
||||
def __init__(self, bundle_version: _Optional[int] = ..., tensors: _Optional[_Iterable[_Union[NamedTensor, _Mapping]]] = ...) -> None: ...
|
||||
architecture: ArchitectureType
|
||||
boundary_point: str
|
||||
def __init__(self, bundle_version: _Optional[int] = ..., tensors: _Optional[_Iterable[_Union[NamedTensor, _Mapping]]] = ..., architecture: _Optional[_Union[ArchitectureType, str]] = ..., boundary_point: _Optional[str] = ...) -> None: ...
|
||||
|
||||
class Fingerprint(_message.Message):
|
||||
__slots__ = ("model_artifact_digest", "runtime_recipe_digest", "recipe_id", "recipe_version", "catalogue_version")
|
||||
@@ -327,20 +342,64 @@ class ActivationChunk(_message.Message):
|
||||
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ...
|
||||
|
||||
class DecodeStep(_message.Message):
|
||||
__slots__ = ("idempotency_step", "position", "expected_past_len", "tensor", "work_id", "deadline_unix_nanos")
|
||||
__slots__ = ("idempotency_step", "position", "expected_past_len", "tensor", "work_id", "deadline_unix_nanos", "bundle")
|
||||
IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int]
|
||||
POSITION_FIELD_NUMBER: _ClassVar[int]
|
||||
EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int]
|
||||
TENSOR_FIELD_NUMBER: _ClassVar[int]
|
||||
WORK_ID_FIELD_NUMBER: _ClassVar[int]
|
||||
DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int]
|
||||
BUNDLE_FIELD_NUMBER: _ClassVar[int]
|
||||
idempotency_step: int
|
||||
position: int
|
||||
expected_past_len: int
|
||||
tensor: NamedTensor
|
||||
work_id: str
|
||||
deadline_unix_nanos: int
|
||||
def __init__(self, idempotency_step: _Optional[int] = ..., position: _Optional[int] = ..., expected_past_len: _Optional[int] = ..., tensor: _Optional[_Union[NamedTensor, _Mapping]] = ..., work_id: _Optional[str] = ..., deadline_unix_nanos: _Optional[int] = ...) -> None: ...
|
||||
bundle: TensorBundle
|
||||
def __init__(self, idempotency_step: _Optional[int] = ..., position: _Optional[int] = ..., expected_past_len: _Optional[int] = ..., tensor: _Optional[_Union[NamedTensor, _Mapping]] = ..., work_id: _Optional[str] = ..., deadline_unix_nanos: _Optional[int] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ...
|
||||
|
||||
class RequestRecipeIdentity(_message.Message):
|
||||
__slots__ = ("request_id", "runtime_recipe_digest", "chat_template_id", "chat_template_version", "reasoning_mode", "architecture")
|
||||
REQUEST_ID_FIELD_NUMBER: _ClassVar[int]
|
||||
RUNTIME_RECIPE_DIGEST_FIELD_NUMBER: _ClassVar[int]
|
||||
CHAT_TEMPLATE_ID_FIELD_NUMBER: _ClassVar[int]
|
||||
CHAT_TEMPLATE_VERSION_FIELD_NUMBER: _ClassVar[int]
|
||||
REASONING_MODE_FIELD_NUMBER: _ClassVar[int]
|
||||
ARCHITECTURE_FIELD_NUMBER: _ClassVar[int]
|
||||
request_id: str
|
||||
runtime_recipe_digest: str
|
||||
chat_template_id: str
|
||||
chat_template_version: str
|
||||
reasoning_mode: str
|
||||
architecture: ArchitectureType
|
||||
def __init__(self, request_id: _Optional[str] = ..., runtime_recipe_digest: _Optional[str] = ..., chat_template_id: _Optional[str] = ..., chat_template_version: _Optional[str] = ..., reasoning_mode: _Optional[str] = ..., architecture: _Optional[_Union[ArchitectureType, str]] = ...) -> None: ...
|
||||
|
||||
class SamplingParameters(_message.Message):
|
||||
__slots__ = ("temperature", "top_p", "top_k", "seed", "greedy")
|
||||
TEMPERATURE_FIELD_NUMBER: _ClassVar[int]
|
||||
TOP_P_FIELD_NUMBER: _ClassVar[int]
|
||||
TOP_K_FIELD_NUMBER: _ClassVar[int]
|
||||
SEED_FIELD_NUMBER: _ClassVar[int]
|
||||
GREEDY_FIELD_NUMBER: _ClassVar[int]
|
||||
temperature: float
|
||||
top_p: float
|
||||
top_k: int
|
||||
seed: int
|
||||
greedy: bool
|
||||
def __init__(self, temperature: _Optional[float] = ..., top_p: _Optional[float] = ..., top_k: _Optional[int] = ..., seed: _Optional[int] = ..., greedy: _Optional[bool] = ...) -> None: ...
|
||||
|
||||
class TailResult(_message.Message):
|
||||
__slots__ = ("identity", "sampling", "logits", "sampled_token_id")
|
||||
IDENTITY_FIELD_NUMBER: _ClassVar[int]
|
||||
SAMPLING_FIELD_NUMBER: _ClassVar[int]
|
||||
LOGITS_FIELD_NUMBER: _ClassVar[int]
|
||||
SAMPLED_TOKEN_ID_FIELD_NUMBER: _ClassVar[int]
|
||||
identity: RequestRecipeIdentity
|
||||
sampling: SamplingParameters
|
||||
logits: TensorBundle
|
||||
sampled_token_id: int
|
||||
def __init__(self, identity: _Optional[_Union[RequestRecipeIdentity, _Mapping]] = ..., sampling: _Optional[_Union[SamplingParameters, _Mapping]] = ..., logits: _Optional[_Union[TensorBundle, _Mapping]] = ..., sampled_token_id: _Optional[int] = ...) -> None: ...
|
||||
|
||||
class ReleaseSignal(_message.Message):
|
||||
__slots__ = ("route_session_id", "route_epoch", "work_id")
|
||||
@@ -409,18 +468,20 @@ class SessionRequest(_message.Message):
|
||||
def __init__(self, open: _Optional[_Union[SessionOpen, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., decode: _Optional[_Union[DecodeStep, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., release: _Optional[_Union[ReleaseSignal, _Mapping]] = ..., cancel: _Optional[_Union[CancelSignal, _Mapping]] = ...) -> None: ...
|
||||
|
||||
class SessionResponse(_message.Message):
|
||||
__slots__ = ("accepted", "chunk", "ack", "flow_control", "status")
|
||||
__slots__ = ("accepted", "chunk", "ack", "flow_control", "status", "tail_result")
|
||||
ACCEPTED_FIELD_NUMBER: _ClassVar[int]
|
||||
CHUNK_FIELD_NUMBER: _ClassVar[int]
|
||||
ACK_FIELD_NUMBER: _ClassVar[int]
|
||||
FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int]
|
||||
STATUS_FIELD_NUMBER: _ClassVar[int]
|
||||
TAIL_RESULT_FIELD_NUMBER: _ClassVar[int]
|
||||
accepted: SessionAccepted
|
||||
chunk: ActivationChunk
|
||||
ack: Ack
|
||||
flow_control: FlowControl
|
||||
status: ShardStatus
|
||||
def __init__(self, accepted: _Optional[_Union[SessionAccepted, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., ack: _Optional[_Union[Ack, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., status: _Optional[_Union[ShardStatus, _Mapping]] = ...) -> None: ...
|
||||
tail_result: TailResult
|
||||
def __init__(self, accepted: _Optional[_Union[SessionAccepted, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., ack: _Optional[_Union[Ack, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., status: _Optional[_Union[ShardStatus, _Mapping]] = ..., tail_result: _Optional[_Union[TailResult, _Mapping]] = ...) -> None: ...
|
||||
|
||||
class CapabilityRequest(_message.Message):
|
||||
__slots__ = ("schema_version",)
|
||||
|
||||
@@ -25,16 +25,24 @@ the release gate is allowed to reach.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import json
|
||||
from collections import Counter
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Mapping
|
||||
|
||||
from cryptography.exceptions import InvalidSignature
|
||||
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
|
||||
|
||||
from .recipe_benchmark import Lane, REPORT_SCHEMA_VERSION
|
||||
|
||||
# Layout of the contract document understood by this reader.
|
||||
CONTRACT_SCHEMA_VERSION = 1
|
||||
PROVENANCE_SCHEMA_VERSION = 1
|
||||
REAL_REPORT_PRODUCER = "meshnet_node.recipe_drivers.run_configured_benchmark/v1"
|
||||
|
||||
VERDICT_PROMOTE = "promote"
|
||||
VERDICT_OPTIMIZE = "optimize"
|
||||
@@ -146,6 +154,66 @@ def _canonical_sha256(value: Any) -> str:
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def report_signing_payload(report: Mapping[str, Any]) -> bytes:
|
||||
"""Canonical report bytes covered by the Ed25519 signature."""
|
||||
provenance = report.get("provenance")
|
||||
if not isinstance(provenance, Mapping):
|
||||
raise PerformanceContractError("real benchmark report lacks signed provenance")
|
||||
unsigned = dict(report)
|
||||
unsigned_provenance = dict(provenance)
|
||||
unsigned_provenance.pop("signature", None)
|
||||
unsigned["provenance"] = unsigned_provenance
|
||||
return json.dumps(
|
||||
unsigned, sort_keys=True, separators=(",", ":"), ensure_ascii=False
|
||||
).encode("utf-8")
|
||||
|
||||
|
||||
def _decode_base64(value: Any, label: str) -> bytes:
|
||||
if not isinstance(value, str):
|
||||
raise PerformanceContractError(f"report lacks {label}")
|
||||
try:
|
||||
return base64.b64decode(value, validate=True)
|
||||
except (binascii.Error, ValueError) as exc:
|
||||
raise PerformanceContractError(f"report carries invalid {label}") from exc
|
||||
|
||||
|
||||
def _verify_real_provenance(
|
||||
contract: PerformanceContract, report: Mapping[str, Any]
|
||||
) -> None:
|
||||
provenance = report.get("provenance")
|
||||
if not isinstance(provenance, Mapping):
|
||||
raise PerformanceContractError("real benchmark report lacks signed provenance")
|
||||
if provenance.get("schema_version") != PROVENANCE_SCHEMA_VERSION:
|
||||
raise PerformanceContractError("report provenance schema is unsupported")
|
||||
if provenance.get("producer") != REAL_REPORT_PRODUCER:
|
||||
raise PerformanceContractError("report was not emitted by the canonical real runner")
|
||||
if provenance.get("signature_algorithm") != "ed25519":
|
||||
raise PerformanceContractError("report provenance is not Ed25519 signed")
|
||||
for field in ("run_id", "started_at", "completed_at"):
|
||||
if not provenance.get(field):
|
||||
raise PerformanceContractError(f"report provenance lacks {field}")
|
||||
|
||||
required_config = contract.baseline.get("required_config_sha256")
|
||||
if not required_config or provenance.get("config_sha256") != required_config:
|
||||
raise PerformanceContractError("report config digest does not match the locked config")
|
||||
|
||||
encoded_public_key = contract.baseline.get("required_signer_public_key")
|
||||
public_key_bytes = _decode_base64(encoded_public_key, "locked signer public key")
|
||||
if len(public_key_bytes) != 32:
|
||||
raise PerformanceContractError("locked Ed25519 public key must be 32 bytes")
|
||||
expected_fingerprint = hashlib.sha256(public_key_bytes).hexdigest()
|
||||
if provenance.get("signer_public_key_sha256") != expected_fingerprint:
|
||||
raise PerformanceContractError("report signer fingerprint does not match the contract")
|
||||
|
||||
signature = _decode_base64(provenance.get("signature"), "Ed25519 signature")
|
||||
try:
|
||||
Ed25519PublicKey.from_public_bytes(public_key_bytes).verify(
|
||||
signature, report_signing_payload(report)
|
||||
)
|
||||
except (InvalidSignature, ValueError) as exc:
|
||||
raise PerformanceContractError("report Ed25519 signature verification failed") from exc
|
||||
|
||||
|
||||
def _validate_report(contract: PerformanceContract, report: Mapping[str, Any]) -> None:
|
||||
"""Fail closed when a report is not the experiment the contract locked."""
|
||||
try:
|
||||
@@ -187,6 +255,8 @@ def _validate_report(contract: PerformanceContract, report: Mapping[str, Any]) -
|
||||
raise PerformanceContractError(
|
||||
f"report evidence class {evidence_class!r} does not satisfy {required_evidence!r}"
|
||||
)
|
||||
if evidence_class in {"local-real", "multi-machine-real"}:
|
||||
_verify_real_provenance(contract, report)
|
||||
if required_evidence and (
|
||||
not isinstance(host, Mapping)
|
||||
or any(key not in host for key in ("hostname", "platform", "python", "cpu_count"))
|
||||
@@ -221,8 +291,18 @@ def _validate_report(contract: PerformanceContract, report: Mapping[str, Any]) -
|
||||
model_id = plan.get("model_id")
|
||||
model_revision = plan.get("model_revision")
|
||||
required_device = contract.baseline.get("required_device")
|
||||
required_artifacts = dict(contract.baseline.get("required_artifact_sha256") or {})
|
||||
required_runtimes = dict(contract.baseline.get("required_recipe_runtime") or {})
|
||||
required_backends = dict(contract.baseline.get("required_backend_detail") or {})
|
||||
required_host = dict(contract.baseline.get("required_host_identity") or {})
|
||||
for field, expected in required_host.items():
|
||||
if host.get(field) != expected:
|
||||
raise PerformanceContractError(
|
||||
f"report host/runtime field {field!r} does not match the locked identity"
|
||||
)
|
||||
for entry in recipes:
|
||||
recipe = entry.get("recipe", {})
|
||||
recipe_id = recipe.get("id")
|
||||
if required_device and recipe.get("device") != required_device:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe.get('id')!r} did not run on locked device {required_device!r}"
|
||||
@@ -235,6 +315,30 @@ def _validate_report(contract: PerformanceContract, report: Mapping[str, Any]) -
|
||||
digest = recipe.get("artifact_sha256", "")
|
||||
if not isinstance(digest, str) or len(digest) != 64:
|
||||
raise PerformanceContractError("report lacks an artifact SHA-256 digest")
|
||||
expected_digest = required_artifacts.get(recipe_id)
|
||||
if not expected_digest or digest != expected_digest:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} artifact digest does not match the contract"
|
||||
)
|
||||
expected_runtime = required_runtimes.get(recipe_id)
|
||||
if not isinstance(expected_runtime, Mapping):
|
||||
raise PerformanceContractError(
|
||||
f"contract lacks runtime identity for recipe {recipe_id!r}"
|
||||
)
|
||||
actual_runtime = {
|
||||
field: recipe.get(field) for field in expected_runtime
|
||||
}
|
||||
if actual_runtime != dict(expected_runtime):
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} runtime identity does not match the contract"
|
||||
)
|
||||
if entry.get("available"):
|
||||
expected_backend = required_backends.get(recipe_id)
|
||||
actual_backend = entry.get("load", {}).get("backend_detail")
|
||||
if not expected_backend or actual_backend != expected_backend:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} backend identity does not match the contract"
|
||||
)
|
||||
if entry.get("available"):
|
||||
cells = entry.get("concurrency", {})
|
||||
missing_cells = required_levels - {int(level) for level in cells}
|
||||
@@ -267,19 +371,98 @@ def _validate_report(contract: PerformanceContract, report: Mapping[str, Any]) -
|
||||
)
|
||||
return {key: sorted(values) for key, values in counts.items()}
|
||||
|
||||
reference_counts = token_counts(reference)
|
||||
def outcome_counts(entry: Mapping[str, Any]) -> Counter[tuple[str, int, int]]:
|
||||
return Counter(
|
||||
(
|
||||
str(outcome.get("prompt_id")),
|
||||
int(outcome.get("concurrency", 0)),
|
||||
int(outcome.get("repeat", -1)),
|
||||
)
|
||||
for outcome in entry.get("outcomes", ())
|
||||
)
|
||||
|
||||
prompt_ids = {str(prompt["id"]) for prompt in plan["prompts"]}
|
||||
repeats = int(plan["repeats"])
|
||||
for entry in recipes:
|
||||
if not entry.get("available"):
|
||||
continue
|
||||
recipe_id = str(entry["recipe"]["id"])
|
||||
cells = entry.get("concurrency", {})
|
||||
actual_levels = {int(level) for level in cells}
|
||||
if actual_levels != levels:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} has unexpected concurrency cells"
|
||||
)
|
||||
for outcome in entry.get("outcomes", ()):
|
||||
try:
|
||||
outcome_recipe_id = str(outcome["recipe_id"])
|
||||
prompt_id = str(outcome["prompt_id"])
|
||||
level = int(outcome["concurrency"])
|
||||
repeat = int(outcome["repeat"])
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} contains a malformed raw outcome"
|
||||
) from exc
|
||||
if outcome_recipe_id != recipe_id:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} contains an outcome for {outcome_recipe_id!r}"
|
||||
)
|
||||
if prompt_id not in prompt_ids or level not in levels or not 0 <= repeat < repeats:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} contains an out-of-domain raw outcome"
|
||||
)
|
||||
if not isinstance(outcome.get("ok"), bool):
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} contains an outcome without boolean ok status"
|
||||
)
|
||||
|
||||
coverage = outcome_counts(entry)
|
||||
for prompt_id in prompt_ids:
|
||||
for level in required_levels:
|
||||
for level in levels:
|
||||
for repeat in range(repeats):
|
||||
if coverage[(prompt_id, level, repeat)] != level:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} lacks complete request coverage"
|
||||
)
|
||||
for level in levels:
|
||||
cell = cells.get(str(level), cells.get(level))
|
||||
raw = [
|
||||
outcome
|
||||
for outcome in entry.get("outcomes", ())
|
||||
if int(outcome["concurrency"]) == level
|
||||
]
|
||||
if int(cell.get("concurrency", -1)) != level:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} cell identity does not match concurrency {level}"
|
||||
)
|
||||
if int(cell.get("requests", -1)) != len(raw):
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} aggregate requests do not match raw outcomes"
|
||||
)
|
||||
raw_failures = sum(not outcome["ok"] for outcome in raw)
|
||||
if int(cell.get("failures", -1)) != raw_failures:
|
||||
raise PerformanceContractError(
|
||||
f"recipe {recipe_id!r} aggregate failures do not match raw outcomes"
|
||||
)
|
||||
|
||||
reference_counts = token_counts(reference)
|
||||
for prompt_id in prompt_ids:
|
||||
for level in levels:
|
||||
for repeat in range(repeats):
|
||||
values = reference_counts.get((prompt_id, level, repeat), ())
|
||||
if len(values) != level:
|
||||
raise PerformanceContractError(
|
||||
"reference recipe lacks complete prompt/repeat/concurrency coverage"
|
||||
)
|
||||
if contract.thresholds.max_failure_rate != 0:
|
||||
raise PerformanceContractError(
|
||||
"nonzero failure tolerance requires an explicit failed-request token policy"
|
||||
)
|
||||
for entry in recipes:
|
||||
if entry.get("available") and token_counts(entry) != reference_counts:
|
||||
if not entry.get("available"):
|
||||
continue
|
||||
for key, values in token_counts(entry).items():
|
||||
if Counter(values) - Counter(reference_counts.get(key, ())):
|
||||
raise PerformanceContractError(
|
||||
f"recipe {entry['recipe']['id']!r} used different prompt/decode token counts"
|
||||
)
|
||||
@@ -440,7 +623,12 @@ def _evaluate_recipe(
|
||||
requests = sum(cell["requests"] for cell in entry["concurrency"].values())
|
||||
failure_rate = _ratio(failures, requests) if requests else 0.0
|
||||
measurements["failure_rate"] = failure_rate
|
||||
if failure_rate > thresholds.max_failure_rate:
|
||||
failure_limit_exceeded = requests == 0 or (
|
||||
failures > 0
|
||||
if thresholds.max_failure_rate == 0
|
||||
else failures / requests > thresholds.max_failure_rate
|
||||
)
|
||||
if failure_limit_exceeded:
|
||||
reasons.append(f"failure rate {failure_rate:.2%} exceeds the contract limit")
|
||||
speed_benefit = False
|
||||
fit_benefit = False
|
||||
@@ -666,6 +854,23 @@ def baseline_from_report(report: Mapping[str, Any]) -> dict[str, Any]:
|
||||
"plan_sha256": _canonical_sha256(report["plan"]),
|
||||
"reference_recipe_id": report["reference_recipe_id"],
|
||||
"host": report["host"],
|
||||
"provenance": dict(report.get("provenance") or {}),
|
||||
"artifact_sha256": {
|
||||
recipe_id: entry["recipe"]["artifact_sha256"]
|
||||
for recipe_id, entry in entries.items()
|
||||
},
|
||||
"recipe_runtime": {
|
||||
recipe_id: {
|
||||
field: entry["recipe"].get(field)
|
||||
for field in ("runtime", "weight_format", "weight_quantization", "device")
|
||||
}
|
||||
for recipe_id, entry in entries.items()
|
||||
},
|
||||
"backend_detail": {
|
||||
recipe_id: entry.get("load", {}).get("backend_detail")
|
||||
for recipe_id, entry in entries.items()
|
||||
if entry.get("available")
|
||||
},
|
||||
"recipes": {},
|
||||
}
|
||||
for recipe_id, entry in entries.items():
|
||||
|
||||
@@ -35,7 +35,7 @@ from dataclasses import asdict, dataclass, field
|
||||
from difflib import SequenceMatcher
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
from typing import Any, Protocol, Sequence
|
||||
from typing import Any, Mapping, Protocol, Sequence
|
||||
|
||||
# Layout of the report document produced by :func:`build_report`.
|
||||
REPORT_SCHEMA_VERSION = 1
|
||||
@@ -581,6 +581,7 @@ def build_report(
|
||||
*,
|
||||
host: dict[str, Any],
|
||||
evidence_class: str,
|
||||
provenance: Mapping[str, Any] | None = None,
|
||||
) -> dict:
|
||||
"""Assemble the machine-readable benchmark document.
|
||||
|
||||
@@ -590,6 +591,8 @@ def build_report(
|
||||
"""
|
||||
if evidence_class not in {"synthetic", "local-real", "multi-machine-real"}:
|
||||
raise BenchmarkError(f"unknown evidence class {evidence_class!r}")
|
||||
if evidence_class != "synthetic" and not isinstance(provenance, Mapping):
|
||||
raise BenchmarkError("non-synthetic reports require canonical signed provenance")
|
||||
|
||||
references = [m for m in measurements if m.recipe.is_reference]
|
||||
if len(references) != 1:
|
||||
@@ -605,7 +608,7 @@ def build_report(
|
||||
for measurement in measurements
|
||||
if measurement is not reference and measurement.available
|
||||
]
|
||||
return {
|
||||
report = {
|
||||
"schema_version": REPORT_SCHEMA_VERSION,
|
||||
"evidence_class": evidence_class,
|
||||
"plan": plan.to_dict(),
|
||||
@@ -614,6 +617,9 @@ def build_report(
|
||||
"recipes": [measurement.to_dict() for measurement in measurements],
|
||||
"drift": drift,
|
||||
}
|
||||
if provenance is not None:
|
||||
report["provenance"] = dict(provenance)
|
||||
return report
|
||||
|
||||
|
||||
def format_summary(report: dict) -> str:
|
||||
@@ -654,13 +660,27 @@ def main(argv: list[str] | None = None) -> int:
|
||||
description="Run the controlled safetensors-versus-GGUF recipe benchmark"
|
||||
)
|
||||
parser.add_argument("--config", type=Path, required=True, help="benchmark configuration JSON")
|
||||
parser.add_argument(
|
||||
"--profile",
|
||||
choices=("contract-v1", "gpu-diagnostic"),
|
||||
default="contract-v1",
|
||||
help="validation and provenance profile (GPU diagnostics are not v1-eligible)",
|
||||
)
|
||||
parser.add_argument("--json-out", type=Path, help="write the JSON report to this path")
|
||||
parser.add_argument("--summary-out", type=Path, help="write the text summary to this path")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
from .recipe_drivers import run_configured_benchmark # heavy runtimes: import on demand
|
||||
from .recipe_drivers import ( # heavy runtimes: import on demand
|
||||
run_configured_benchmark,
|
||||
run_configured_gpu_diagnostic,
|
||||
)
|
||||
|
||||
report = run_configured_benchmark(json.loads(args.config.read_text(encoding="utf-8")))
|
||||
runner = (
|
||||
run_configured_gpu_diagnostic
|
||||
if args.profile == "gpu-diagnostic"
|
||||
else run_configured_benchmark
|
||||
)
|
||||
report = runner(json.loads(args.config.read_text(encoding="utf-8")))
|
||||
summary = format_summary(report)
|
||||
if args.json_out:
|
||||
args.json_out.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
|
||||
@@ -20,6 +20,7 @@ rules:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
@@ -27,15 +28,27 @@ import os
|
||||
import platform
|
||||
import re
|
||||
import socket
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Mapping
|
||||
|
||||
from cryptography.hazmat.primitives import serialization
|
||||
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
|
||||
|
||||
from .performance_contract import (
|
||||
PROVENANCE_SCHEMA_VERSION,
|
||||
REAL_REPORT_PRODUCER,
|
||||
_canonical_sha256,
|
||||
report_signing_payload,
|
||||
)
|
||||
from .recipe_benchmark import (
|
||||
BenchmarkError,
|
||||
BenchmarkPlan,
|
||||
@@ -50,6 +63,10 @@ from .recipe_benchmark import (
|
||||
)
|
||||
|
||||
REAL_INFERENCE_ENV = "MESHNET_ENABLE_REAL_INFERENCE_TESTS"
|
||||
EVIDENCE_SIGNING_KEY_ENV = "MESHNET_EVIDENCE_SIGNING_KEY"
|
||||
CONTRACT_V1_PROFILE = "contract-v1"
|
||||
GPU_DIAGNOSTIC_PROFILE = "gpu-diagnostic"
|
||||
GPU_DIAGNOSTIC_REPORT_PRODUCER = "meshnet_node.recipe_drivers.run_configured_gpu_diagnostic/v1"
|
||||
|
||||
|
||||
def real_inference_enabled() -> bool:
|
||||
@@ -64,6 +81,37 @@ def require_real_inference() -> None:
|
||||
)
|
||||
|
||||
|
||||
def _load_evidence_signing_key() -> Ed25519PrivateKey:
|
||||
raw_path = os.environ.get(EVIDENCE_SIGNING_KEY_ENV)
|
||||
if not raw_path:
|
||||
raise BenchmarkError(
|
||||
f"real evidence requires {EVIDENCE_SIGNING_KEY_ENV} to name an Ed25519 private key"
|
||||
)
|
||||
path = Path(raw_path).expanduser().resolve(strict=True)
|
||||
if os.name != "nt" and stat.S_IMODE(path.stat().st_mode) != 0o600:
|
||||
raise BenchmarkError("evidence signing key must have mode 0600")
|
||||
key = serialization.load_pem_private_key(path.read_bytes(), password=None)
|
||||
if not isinstance(key, Ed25519PrivateKey):
|
||||
raise BenchmarkError("evidence signing key must be Ed25519")
|
||||
return key
|
||||
|
||||
|
||||
def _utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _sign_report(report: dict[str, Any], key: Ed25519PrivateKey) -> None:
|
||||
public_key = key.public_key().public_bytes(
|
||||
serialization.Encoding.Raw, serialization.PublicFormat.Raw
|
||||
)
|
||||
report["provenance"]["signer_public_key_sha256"] = hashlib.sha256(
|
||||
public_key
|
||||
).hexdigest()
|
||||
report["provenance"]["signature"] = base64.b64encode(
|
||||
key.sign(report_signing_payload(report))
|
||||
).decode("ascii")
|
||||
|
||||
|
||||
def _process_rss(pid: int | None = None) -> int:
|
||||
"""Resident bytes for a process and its children, or 0 when unobservable."""
|
||||
try:
|
||||
@@ -120,7 +168,7 @@ def _artifact_sha256(path: Path) -> str:
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _host_manifest() -> dict[str, Any]:
|
||||
def _host_manifest(config: Mapping[str, Any] | None = None) -> dict[str, Any]:
|
||||
"""Capture non-secret host facts with the report rather than trusting prose."""
|
||||
manifest: dict[str, Any] = {
|
||||
"hostname": socket.gethostname(),
|
||||
@@ -142,11 +190,39 @@ def _host_manifest() -> dict[str, Any]:
|
||||
)
|
||||
except ImportError:
|
||||
manifest["torch_version"] = None
|
||||
|
||||
llama_identities: dict[str, dict[str, str]] = {}
|
||||
for spec in (config or {}).get("recipes", ()):
|
||||
driver = spec.get("driver", {})
|
||||
if driver.get("type") != "llama-cpp-server":
|
||||
continue
|
||||
binary = Path(driver["binary"]).resolve(strict=True)
|
||||
key = str(binary)
|
||||
if key in llama_identities:
|
||||
continue
|
||||
version_result = subprocess.run(
|
||||
[str(binary), "--version"],
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
llama_identities[key] = {
|
||||
"sha256": _artifact_sha256(binary),
|
||||
"version": " | ".join(version_result.stdout.strip().splitlines()),
|
||||
}
|
||||
if llama_identities:
|
||||
manifest["llama_server_identities"] = llama_identities
|
||||
return manifest
|
||||
|
||||
|
||||
def _validate_config(config: Mapping[str, Any]) -> None:
|
||||
def _validate_config(
|
||||
config: Mapping[str, Any], *, profile: str = CONTRACT_V1_PROFILE
|
||||
) -> None:
|
||||
"""Reject comparisons that mix models, artifacts, devices, or budgets."""
|
||||
if profile not in {CONTRACT_V1_PROFILE, GPU_DIAGNOSTIC_PROFILE}:
|
||||
raise BenchmarkError(f"unknown benchmark validation profile {profile!r}")
|
||||
try:
|
||||
plan = config["plan"]
|
||||
root = Path(config["artifact_storage_root"]).resolve(strict=True)
|
||||
@@ -198,7 +274,11 @@ def _validate_config(config: Mapping[str, Any]) -> None:
|
||||
|
||||
driver = spec.get("driver")
|
||||
if not isinstance(driver, Mapping):
|
||||
raise BenchmarkError("every recipe needs a driver object")
|
||||
raise BenchmarkError("every recipe must declare a driver block")
|
||||
if "artifact_sha256" in driver:
|
||||
raise BenchmarkError(
|
||||
"driver artifact_sha256 is forbidden; the validated recipe digest is authoritative"
|
||||
)
|
||||
kind = driver.get("type")
|
||||
if kind == "transformers":
|
||||
driver_artifact = Path(driver.get("model_path", "")).resolve(strict=True)
|
||||
@@ -214,18 +294,40 @@ def _validate_config(config: Mapping[str, Any]) -> None:
|
||||
raise BenchmarkError("llama.cpp binary SHA-256 mismatch")
|
||||
if int(driver.get("n_parallel", max_concurrency)) < max_concurrency:
|
||||
raise BenchmarkError("llama.cpp parallel slots must cover maximum concurrency")
|
||||
if driver.get("device", "cpu") != "cpu" or int(driver.get("n_gpu_layers", 0)) != 0:
|
||||
driver_device = driver.get("device", "cpu")
|
||||
gpu_layers = int(driver.get("n_gpu_layers", 0))
|
||||
if profile == CONTRACT_V1_PROFILE and (
|
||||
driver_device != "cpu" or gpu_layers != 0
|
||||
):
|
||||
raise BenchmarkError(
|
||||
"v1 benchmark supports CPU-only llama.cpp until process VRAM is measurable"
|
||||
)
|
||||
if profile == GPU_DIAGNOSTIC_PROFILE and (
|
||||
driver_device != "cuda" or gpu_layers <= 0
|
||||
):
|
||||
raise BenchmarkError(
|
||||
"GPU diagnostic requires CUDA/ROCm llama.cpp with positive n_gpu_layers"
|
||||
)
|
||||
else:
|
||||
raise BenchmarkError(f"unknown driver type {kind!r}")
|
||||
if driver_artifact != artifact:
|
||||
raise BenchmarkError("driver artifact path must match the hashed recipe artifact")
|
||||
if driver.get("device", "cpu") != spec.get("device"):
|
||||
raise BenchmarkError("recipe and driver must declare the same device")
|
||||
if profile == CONTRACT_V1_PROFILE and spec.get("device") != "cpu":
|
||||
raise BenchmarkError("contract-v1 requires every recipe to run on CPU")
|
||||
if profile == GPU_DIAGNOSTIC_PROFILE and spec.get("device") != "cuda":
|
||||
raise BenchmarkError("GPU diagnostic requires every recipe to declare device 'cuda'")
|
||||
thread_budgets.add(int(driver.get("threads", 8)))
|
||||
|
||||
host = config.get("host", {})
|
||||
if not isinstance(host, Mapping):
|
||||
raise BenchmarkError("benchmark host metadata must be an object")
|
||||
if profile == GPU_DIAGNOSTIC_PROFILE and host.get(
|
||||
"benchmark_lane"
|
||||
) != "rocm-gpu-diagnostic":
|
||||
raise BenchmarkError("GPU diagnostic requires the rocm-gpu-diagnostic host marker")
|
||||
|
||||
if len(thread_budgets) != 1:
|
||||
raise BenchmarkError("every recipe must use the same CPU thread budget")
|
||||
|
||||
@@ -244,11 +346,13 @@ class TransformersDriver:
|
||||
self,
|
||||
model_path: str,
|
||||
*,
|
||||
artifact_sha256: str | None = None,
|
||||
device: str = "cpu",
|
||||
dtype: str = "bfloat16",
|
||||
threads: int = 8,
|
||||
) -> None:
|
||||
self.model_path = Path(model_path)
|
||||
self.artifact_sha256 = artifact_sha256
|
||||
self.device = device
|
||||
self.dtype = dtype
|
||||
self.threads = threads
|
||||
@@ -258,6 +362,10 @@ class TransformersDriver:
|
||||
self._rss_baseline = 0
|
||||
|
||||
def load(self) -> LoadStats:
|
||||
if self.artifact_sha256 is not None:
|
||||
measured_artifact_sha256 = _artifact_sha256(self.model_path)
|
||||
if not hmac.compare_digest(self.artifact_sha256, measured_artifact_sha256):
|
||||
raise BenchmarkError("Transformers artifact changed after config validation")
|
||||
self._rss_baseline = _process_rss()
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
@@ -376,6 +484,39 @@ def _free_port() -> int:
|
||||
return int(probe.getsockname()[1])
|
||||
|
||||
|
||||
def _gpu_offload_evidence(log_text: str, requested_layers: int) -> str:
|
||||
"""Extract measured ROCm device and layer placement from llama-server logs."""
|
||||
device_matches = re.findall(
|
||||
r"-\s+(ROCm\d+)\s+:\s+(.+?)\s+\(\d+\s+MiB", log_text
|
||||
)
|
||||
offload_matches = re.findall(
|
||||
r"offloaded\s+(\d+)/(\d+)\s+layers\s+to\s+GPU", log_text
|
||||
)
|
||||
if not device_matches:
|
||||
raise BenchmarkError("GPU diagnostic found no measured ROCm device in llama-server logs")
|
||||
if not offload_matches:
|
||||
raise BenchmarkError("GPU diagnostic found no measured layer offload in llama-server logs")
|
||||
|
||||
backend, device_name = device_matches[-1]
|
||||
offloaded, total = (int(value) for value in offload_matches[-1])
|
||||
required = min(requested_layers, total)
|
||||
if requested_layers <= 0 or offloaded < required:
|
||||
raise BenchmarkError(
|
||||
f"GPU diagnostic requested {requested_layers} layers but measured "
|
||||
f"only {offloaded}/{total} offloaded"
|
||||
)
|
||||
return (
|
||||
f"measured accelerator {backend}: {device_name}; "
|
||||
f"measured offload {offloaded}/{total} layers"
|
||||
)
|
||||
|
||||
|
||||
def _gpu_layer_config_detail(device: str, layers: int) -> str:
|
||||
# Immutable v1 pins the historical CPU wording exactly. GPU diagnostics use
|
||||
# the clearer requested/measured distinction without changing v1 identity.
|
||||
return f"gpu layers {layers}" if device == "cpu" else f"requested gpu layers {layers}"
|
||||
|
||||
|
||||
class LlamaCppServerDriver:
|
||||
"""The whole-model llama.cpp/GGUF recipe, driven through ``llama-server``.
|
||||
|
||||
@@ -393,6 +534,7 @@ class LlamaCppServerDriver:
|
||||
gguf_path: str,
|
||||
*,
|
||||
binary_sha256: str,
|
||||
artifact_sha256: str | None = None,
|
||||
device: str = "cpu",
|
||||
threads: int = 8,
|
||||
n_parallel: int = 4,
|
||||
@@ -403,6 +545,7 @@ class LlamaCppServerDriver:
|
||||
self.binary = Path(binary)
|
||||
self.binary_sha256 = binary_sha256
|
||||
self.gguf_path = Path(gguf_path)
|
||||
self.artifact_sha256 = artifact_sha256
|
||||
self.device = device
|
||||
self.threads = threads
|
||||
self.n_parallel = n_parallel
|
||||
@@ -417,21 +560,27 @@ class LlamaCppServerDriver:
|
||||
def _url(self) -> str:
|
||||
return f"http://127.0.0.1:{self._port}"
|
||||
|
||||
def _log_excerpt(self) -> str:
|
||||
def _log_text(self) -> str:
|
||||
if self._log is None:
|
||||
return ""
|
||||
try:
|
||||
self._log.flush()
|
||||
self._log.seek(0)
|
||||
return self._log.read()[-4096:].decode("utf-8", errors="replace").strip()
|
||||
size = min(os.fstat(self._log.fileno()).st_size, 8 * 1024 * 1024)
|
||||
return os.pread(self._log.fileno(), size, 0).decode("utf-8", errors="replace")
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
def _log_excerpt(self) -> str:
|
||||
return self._log_text()[-4096:].strip()
|
||||
|
||||
def load(self) -> LoadStats:
|
||||
if not self.binary.exists():
|
||||
raise BenchmarkError(f"llama-server binary not found at {self.binary}")
|
||||
if not self.gguf_path.exists():
|
||||
raise BenchmarkError(f"GGUF artifact not found at {self.gguf_path}")
|
||||
if self.artifact_sha256 is not None:
|
||||
measured_artifact_sha256 = _artifact_sha256(self.gguf_path)
|
||||
if not hmac.compare_digest(self.artifact_sha256, measured_artifact_sha256):
|
||||
raise BenchmarkError("GGUF artifact changed after config validation")
|
||||
measured_binary_sha256 = _artifact_sha256(self.binary)
|
||||
if not hmac.compare_digest(self.binary_sha256, measured_binary_sha256):
|
||||
raise BenchmarkError("llama-server binary changed after config validation")
|
||||
@@ -447,8 +596,12 @@ class LlamaCppServerDriver:
|
||||
)
|
||||
|
||||
self._port = _free_port()
|
||||
command = [
|
||||
str(self.binary),
|
||||
command = [str(self.binary)]
|
||||
if self.device == "cuda":
|
||||
# Debug verbosity is required to capture measured device placement
|
||||
# and the offloaded/total layer count in signed diagnostics.
|
||||
command.extend(["-lv", "5"])
|
||||
command.extend([
|
||||
"--model", str(self.gguf_path),
|
||||
"--host", "127.0.0.1",
|
||||
"--port", str(self._port),
|
||||
@@ -459,7 +612,7 @@ class LlamaCppServerDriver:
|
||||
"--ctx-size", str(self.context_per_slot * self.n_parallel),
|
||||
"--n-gpu-layers", str(self.n_gpu_layers),
|
||||
"--no-webui",
|
||||
]
|
||||
])
|
||||
started = time.monotonic()
|
||||
self._log = tempfile.TemporaryFile(mode="w+b")
|
||||
self._process = subprocess.Popen(
|
||||
@@ -467,6 +620,9 @@ class LlamaCppServerDriver:
|
||||
)
|
||||
self._await_health(started)
|
||||
load_ms = (time.monotonic() - started) * 1000
|
||||
gpu_evidence = ""
|
||||
if self.device == "cuda":
|
||||
gpu_evidence = _gpu_offload_evidence(self._log_text(), self.n_gpu_layers)
|
||||
|
||||
return LoadStats(
|
||||
artifact_bytes=self.gguf_path.stat().st_size,
|
||||
@@ -476,7 +632,9 @@ class LlamaCppServerDriver:
|
||||
backend_detail=(
|
||||
f"{version}; binary sha256 {measured_binary_sha256}; "
|
||||
f"threads {self.threads}; parallel slots {self.n_parallel}; "
|
||||
f"ctx/slot {self.context_per_slot}; gpu layers {self.n_gpu_layers}"
|
||||
f"ctx/slot {self.context_per_slot}; "
|
||||
f"{_gpu_layer_config_detail(self.device, self.n_gpu_layers)}"
|
||||
+ (f"; {gpu_evidence}" if gpu_evidence else "")
|
||||
),
|
||||
)
|
||||
|
||||
@@ -584,6 +742,7 @@ def build_driver(spec: Mapping[str, Any], plan: BenchmarkPlan) -> RecipeDriverBu
|
||||
"""Construct the driver named by a recipe's ``driver`` block."""
|
||||
driver_spec = dict(spec["driver"])
|
||||
kind = driver_spec.pop("type")
|
||||
driver_spec["artifact_sha256"] = spec.get("artifact_sha256")
|
||||
if kind == "transformers":
|
||||
return TransformersDriver(**driver_spec)
|
||||
if kind == "llama-cpp-server":
|
||||
@@ -627,14 +786,35 @@ def _recipe_from_config(spec: Mapping[str, Any]) -> RecipeSpec:
|
||||
|
||||
|
||||
def run_configured_benchmark(config: Mapping[str, Any]) -> dict:
|
||||
"""Run every recipe in ``config`` against one shared plan and return the report.
|
||||
"""Run contract-v1 evidence through the CPU-only, fail-closed profile."""
|
||||
return _run_profiled_benchmark(config, profile=CONTRACT_V1_PROFILE)
|
||||
|
||||
A recipe whose runtime cannot start is recorded as unavailable with the real
|
||||
reason rather than dropped: a report that silently omits the recipe that
|
||||
crashed would read as a clean result.
|
||||
"""
|
||||
|
||||
def run_configured_gpu_diagnostic(config: Mapping[str, Any]) -> dict:
|
||||
"""Run signed ROCm diagnostics that are intentionally ineligible for v1."""
|
||||
return _run_profiled_benchmark(config, profile=GPU_DIAGNOSTIC_PROFILE)
|
||||
|
||||
|
||||
def _producer_for_profile(profile: str) -> str:
|
||||
if profile == CONTRACT_V1_PROFILE:
|
||||
return REAL_REPORT_PRODUCER
|
||||
if profile == GPU_DIAGNOSTIC_PROFILE:
|
||||
return GPU_DIAGNOSTIC_REPORT_PRODUCER
|
||||
raise BenchmarkError(f"unknown benchmark validation profile {profile!r}")
|
||||
|
||||
|
||||
def _run_profiled_benchmark(config: Mapping[str, Any], *, profile: str) -> dict:
|
||||
"""Validate a closed profile and derive its producer before measurement."""
|
||||
require_real_inference()
|
||||
_validate_config(config)
|
||||
_validate_config(config, profile=profile)
|
||||
producer = _producer_for_profile(profile)
|
||||
evidence_class = config.get("evidence_class", "local-real")
|
||||
if evidence_class not in {"local-real", "multi-machine-real"}:
|
||||
raise BenchmarkError("canonical real runner cannot emit synthetic evidence")
|
||||
signing_key = _load_evidence_signing_key()
|
||||
started_at = _utc_now()
|
||||
run_id = str(uuid.uuid4())
|
||||
config_sha256 = _canonical_sha256(config)
|
||||
plan = _plan_from_config(config)
|
||||
|
||||
from .recipe_benchmark import RecipeMeasurement # local import keeps the seam obvious
|
||||
@@ -656,9 +836,20 @@ def run_configured_benchmark(config: Mapping[str, Any]) -> dict:
|
||||
if driver is not None:
|
||||
driver.close()
|
||||
|
||||
return build_report(
|
||||
report = build_report(
|
||||
plan,
|
||||
measurements,
|
||||
host={**dict(config.get("host", {})), **_host_manifest()},
|
||||
evidence_class=config.get("evidence_class", "local-real"),
|
||||
host={**dict(config.get("host", {})), **_host_manifest(config)},
|
||||
evidence_class=evidence_class,
|
||||
provenance={
|
||||
"schema_version": PROVENANCE_SCHEMA_VERSION,
|
||||
"producer": producer,
|
||||
"run_id": run_id,
|
||||
"started_at": started_at,
|
||||
"completed_at": _utc_now(),
|
||||
"config_sha256": config_sha256,
|
||||
"signature_algorithm": "ed25519",
|
||||
},
|
||||
)
|
||||
_sign_report(report, signing_key)
|
||||
return report
|
||||
|
||||
1049
packages/node/meshnet_node/runtime_recipe.py
Normal file
1049
packages/node/meshnet_node/runtime_recipe.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -105,7 +105,7 @@ class _StubHTTPServer(http.server.HTTPServer):
|
||||
|
||||
|
||||
class _StubHandler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
||||
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||
pass
|
||||
|
||||
def do_POST(self):
|
||||
|
||||
@@ -19,7 +19,6 @@ from .model_backend import (
|
||||
InsufficientVRAMError,
|
||||
KVCacheMiss,
|
||||
MissingModelDependencyError,
|
||||
Quantization,
|
||||
TailTokenResult,
|
||||
TorchModelShard,
|
||||
_tensor_from_bfloat16_bytes,
|
||||
@@ -46,7 +45,7 @@ class _DirectRequestUncertainError(ConnectionError):
|
||||
"""A direct request may have reached the downstream node but did not finish."""
|
||||
|
||||
|
||||
from .server import (
|
||||
from .server import ( # noqa: E402
|
||||
_WIRE_VERSION,
|
||||
_parse_shape,
|
||||
_validate_activation_body,
|
||||
@@ -399,7 +398,7 @@ class _TorchHandler(http.server.BaseHTTPRequestHandler):
|
||||
# Finite responses below provide Content-Length; streams are chunked.
|
||||
protocol_version = "HTTP/1.1"
|
||||
|
||||
def log_message(self, fmt, *args): # noqa: suppress request logs in tests
|
||||
def log_message(self, fmt, *args): # suppress request logs in tests
|
||||
pass
|
||||
|
||||
def _request_id(self) -> str:
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
@@ -32,6 +32,15 @@ Both `--check` modes run in CI via `tests/test_native_shard_protocol.py`, so a
|
||||
schema edit that is not accompanied by regenerated output fails the suite rather
|
||||
than shipping stubs that disagree with the schema they claim to implement.
|
||||
|
||||
## DGR-006 decode and tail compatibility
|
||||
|
||||
`DecodeStep.bundle` is the versioned `TensorBundle` fast-path boundary. It is
|
||||
authoritative whenever present and supports architecture sidebands. The original
|
||||
`DecodeStep.tensor` remains readable as the compact one-tensor encoding for
|
||||
certified boundaries that need only one tensor; new readers wrap it into a
|
||||
one-member bundle. Tail completions use `TailResult`, which binds logits or a
|
||||
sampled token to request/recipe identity and sampling/template/reasoning inputs.
|
||||
|
||||
## Building and running the C++ conformance test
|
||||
|
||||
If the machine has no protobuf C++ toolchain:
|
||||
|
||||
30
packages/node/native/llama/PATCH-STACK.md
Normal file
30
packages/node/native/llama/PATCH-STACK.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# Meshnet llama.cpp patch stack
|
||||
|
||||
This directory is the only project-owned fork boundary for llama.cpp. It is
|
||||
locked to `e920c523e3b8a0163fe498af5bf90df35ff51d25`; changing the pin requires
|
||||
updating the recorded tree/blob assumptions and reviewing every patch anew.
|
||||
|
||||
## Ordered series
|
||||
|
||||
1. `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch` adds only an
|
||||
interface-library marker used to prove the patched source was configured.
|
||||
It has no execution, transport, model-loading, or semantic effect.
|
||||
|
||||
Future patches may implement only the ADR-0020 local seams: range-aware tensor
|
||||
loading, endpoint ownership, architecture-defined intermediate boundaries, and
|
||||
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
|
||||
billing, authentication, and telemetry must remain outside this directory.
|
||||
|
||||
`scripts/llama_cpp_dependency.py` verifies the exact commit/tree and baseline
|
||||
blobs, validates every patch digest and context with `git apply --check`, then
|
||||
applies the series in `patches/series` order. It refuses a dirty source tree,
|
||||
wrong commit/tree/blob, changed patch digest, reordered series, or an existing
|
||||
destination/work directory.
|
||||
|
||||
## Current semantic boundary
|
||||
|
||||
The stock pinned build is **infrastructure evidence only**. Per DGR-017,
|
||||
GLM-5.2 can use a dense-MLA compatibility fallback at this point; a successful
|
||||
build or `llama-cli --version` does not show native DSA/IndexShare, GLM semantic
|
||||
acceptance, numerical parity, performance, or route certification. DGR-018 and
|
||||
DGR-019 own those checks. Dense Llama remains only a later structural fixture.
|
||||
17
packages/node/native/llama/THIRD_PARTY_NOTICES.md
Normal file
17
packages/node/native/llama/THIRD_PARTY_NOTICES.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Third-party notices: llama.cpp
|
||||
|
||||
The reproducibility harness fetches source from
|
||||
[`ggml-org/llama.cpp`](https://github.com/ggml-org/llama.cpp) at exact commit
|
||||
`e920c523e3b8a0163fe498af5bf90df35ff51d25`.
|
||||
|
||||
- Upstream license: MIT. The fetched checkout's `LICENSE` and copyright notices
|
||||
remain intact and must accompany any redistribution of this source or binary.
|
||||
- Meshnet's one-patch CMake marker is an additive local change. It does not
|
||||
replace, relicense, or remove upstream notices.
|
||||
- No donor code is included. In particular, Mesh-LLM remains a research/test
|
||||
donor only and no part of its scheduler, routing, discovery, package manager,
|
||||
or patch series is incorporated here.
|
||||
|
||||
Release packaging must include the upstream MIT text from the exact materialized
|
||||
checkout plus this notice. `scripts/llama_cpp_dependency.py` refuses to build if
|
||||
the upstream `LICENSE` is absent.
|
||||
1
packages/node/native/llama/UPSTREAM_COMMIT
Normal file
1
packages/node/native/llama/UPSTREAM_COMMIT
Normal file
@@ -0,0 +1 @@
|
||||
e920c523e3b8a0163fe498af5bf90df35ff51d25
|
||||
48
packages/node/native/llama/UPSTREAM_LOCK.json
Normal file
48
packages/node/native/llama/UPSTREAM_LOCK.json
Normal file
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"upstream": "https://github.com/ggml-org/llama.cpp.git",
|
||||
"commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
|
||||
"commit_tree": "6c91a11407a3a3fb160f5dac705f9c59718f54f1",
|
||||
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
|
||||
"upstream_license": "MIT",
|
||||
"patch_series": [
|
||||
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
|
||||
"0002-dense-llama-owned-range-loader.patch"
|
||||
],
|
||||
"patch_scope": [
|
||||
"Reserved CMake ABI marker only; no execution or model semantics.",
|
||||
"Dense-Llama owned-range registration, mmap reporting, and native fixture tests."
|
||||
],
|
||||
"build": {
|
||||
"generator": "Unix Makefiles",
|
||||
"cmake_minimum": "3.14",
|
||||
"cxx_standard": "17",
|
||||
"configure_flags": [
|
||||
"-DCMAKE_BUILD_TYPE=Release",
|
||||
"-DLLAMA_BUILD_TESTS=OFF",
|
||||
"-DLLAMA_BUILD_EXAMPLES=ON",
|
||||
"-DLLAMA_BUILD_SERVER=OFF",
|
||||
"-DLLAMA_BUILD_TOOLS=OFF",
|
||||
"-DLLAMA_BUILD_APP=OFF",
|
||||
"-DLLAMA_CURL=OFF"
|
||||
],
|
||||
"native_targets": ["llama-gguf-hash"],
|
||||
"smoke_binary": "bin/llama-gguf-hash",
|
||||
"smoke_args": ["--help"],
|
||||
"smoke_output_token": "usage"
|
||||
},
|
||||
"required_upstream_blobs": {
|
||||
"CMakeLists.txt": "81f23d7e70b7378511af5d01be680c03aebc2b15"
|
||||
},
|
||||
"patched_paths": [
|
||||
"CMakeLists.txt",
|
||||
"cmake/meshnet-patch-stack.cmake",
|
||||
"include/llama.h",
|
||||
"src/llama-model.cpp",
|
||||
"src/llama-model.h",
|
||||
"src/models/llama.cpp",
|
||||
"tests/CMakeLists.txt",
|
||||
"tests/test-meshnet-range-ownership.cpp"
|
||||
],
|
||||
"stock_glm_limitations": "This pin may load GLM-5.2 through the dense-MLA compatibility fallback. It does not prove native DSA, IndexShare, MoE semantic correctness, numerical equivalence, performance, or route certification."
|
||||
}
|
||||
68
packages/node/native/llama/meshnet-range-loader.cpp
Normal file
68
packages/node/native/llama/meshnet-range-loader.cpp
Normal file
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* meshnet-range-loader — CLI wrapper that loads a GGUF shard range
|
||||
* and prints the meshnet range report as JSON.
|
||||
*
|
||||
* Usage: meshnet-range-loader <gguf-path> <start-layer> <end-layer>
|
||||
*
|
||||
* Build: g++ -std=c++17 -I<source>/include -L<build>/bin
|
||||
* meshnet-range-loader.cpp -lllama -o meshnet-range-loader
|
||||
* LD_LIBRARY_PATH=<build>/bin ./meshnet-range-loader ...
|
||||
*/
|
||||
|
||||
#include "llama.h"
|
||||
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
if (argc != 4) {
|
||||
fprintf(stderr, "Usage: %s <gguf-path> <start-layer> <end-layer>\n", argv[0]);
|
||||
return 1;
|
||||
}
|
||||
|
||||
const char *path = argv[1];
|
||||
int start = std::atoi(argv[2]);
|
||||
int end = std::atoi(argv[3]);
|
||||
|
||||
llama_backend_init();
|
||||
|
||||
llama_model_params params = llama_model_default_params();
|
||||
params.meshnet_owned_layer_start = start;
|
||||
params.meshnet_owned_layer_end = end;
|
||||
|
||||
llama_model *model = llama_model_load_from_file(path, params);
|
||||
if (!model) {
|
||||
fprintf(stderr, "ERROR model_load returned nullptr\n");
|
||||
llama_backend_free();
|
||||
return 1;
|
||||
}
|
||||
|
||||
llama_meshnet_range_report report;
|
||||
bool ok = llama_model_meshnet_range_report(model, &report);
|
||||
|
||||
int n_layer = llama_model_n_layer(model);
|
||||
int n_embd = llama_model_n_embd(model);
|
||||
uint64_t size = llama_model_size(model);
|
||||
|
||||
if (ok) {
|
||||
fprintf(stdout,
|
||||
"{"
|
||||
"\"ok\":true,"
|
||||
"\"start_layer\":%d,\"end_layer\":%d,"
|
||||
"\"mapped_bytes\":%lu,\"resident_bytes\":%lu,"
|
||||
"\"model_n_layer\":%d,\"model_n_embd\":%d,\"model_size\":%lu"
|
||||
"}\n",
|
||||
report.start_layer, report.end_layer,
|
||||
(unsigned long)report.mapped_bytes,
|
||||
(unsigned long)report.resident_bytes,
|
||||
n_layer, n_embd, (unsigned long)size
|
||||
);
|
||||
} else {
|
||||
fprintf(stderr, "ERROR range report not available\n");
|
||||
}
|
||||
|
||||
llama_model_free(model);
|
||||
llama_backend_free();
|
||||
return ok ? 0 : 1;
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
From d9992f62e852c8647a2ad302009b0339dc1cbf5b Mon Sep 17 00:00:00 2001
|
||||
From: DGR-004 <dgr-004@meshnet.invalid>
|
||||
Date: Tue, 14 Jul 2026 09:52:24 +0300
|
||||
Subject: [PATCH] cmake: reserve Meshnet patch-stack ABI marker
|
||||
|
||||
---
|
||||
CMakeLists.txt | 1 +
|
||||
cmake/meshnet-patch-stack.cmake | 7 +++++++
|
||||
2 files changed, 8 insertions(+)
|
||||
|
||||
diff --git a/CMakeLists.txt b/CMakeLists.txt
|
||||
index 81f23d7e..a9afcffa 100644
|
||||
--- a/CMakeLists.txt
|
||||
+++ b/CMakeLists.txt
|
||||
@@ -123,6 +123,7 @@ option(LLAMA_LLGUIDANCE "llama-common: include LLGuidance library for structured
|
||||
# Required for relocatable CMake package
|
||||
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/build-info.cmake)
|
||||
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/common.cmake)
|
||||
+include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/meshnet-patch-stack.cmake)
|
||||
|
||||
if (NOT DEFINED LLAMA_BUILD_NUMBER)
|
||||
set(LLAMA_BUILD_NUMBER ${BUILD_NUMBER})
|
||||
diff --git a/cmake/meshnet-patch-stack.cmake b/cmake/meshnet-patch-stack.cmake
|
||||
new file mode 100644
|
||||
index 00000000..910646b4
|
||||
--- /dev/null
|
||||
+++ b/cmake/meshnet-patch-stack.cmake
|
||||
@@ -0,0 +1,7 @@
|
||||
+# Reserved, local-only hook for the Meshnet-owned llama.cpp patch series.
|
||||
+# No execution, protocol, architecture, or model semantic changes are in DGR-004.
|
||||
+set(LLAMA_MESHNET_PATCH_STACK_VERSION "1" CACHE STRING
|
||||
+ "Meshnet patch stack ABI marker")
|
||||
+add_library(llama_meshnet_patch_stack INTERFACE)
|
||||
+target_compile_definitions(llama_meshnet_patch_stack INTERFACE
|
||||
+ LLAMA_MESHNET_PATCH_STACK_VERSION=${LLAMA_MESHNET_PATCH_STACK_VERSION})
|
||||
--
|
||||
2.54.0
|
||||
@@ -0,0 +1,169 @@
|
||||
From: Meshnet <meshnet@invalid>
|
||||
Subject: [PATCH] llama: add dense owned-range loading seam
|
||||
|
||||
diff --git a/include/llama.h b/include/llama.h
|
||||
index a311ac20..1f9459cf 100644
|
||||
--- a/include/llama.h
|
||||
+++ b/include/llama.h
|
||||
@@ -292,6 +292,19 @@ extern "C" {
|
||||
ggml_backend_buffer_type_t buft;
|
||||
};
|
||||
|
||||
+ // Immutable report for the project-owned dense-Llama range-loading seam.
|
||||
+ // The bounds are inclusive/exclusive and are populated only after the
|
||||
+ // model has registered and allocated its owned tensors.
|
||||
+ struct llama_meshnet_range_report {
|
||||
+ int32_t start_layer;
|
||||
+ int32_t end_layer;
|
||||
+ uint64_t mapped_bytes;
|
||||
+ uint64_t resident_bytes;
|
||||
+ uint64_t registered_bytes;
|
||||
+ bool has_token_embeddings;
|
||||
+ bool has_output_head;
|
||||
+ };
|
||||
+
|
||||
struct llama_model_params {
|
||||
@@ -319,6 +332,12 @@ extern "C" {
|
||||
const struct llama_model_kv_override * kv_overrides;
|
||||
|
||||
+ int32_t meshnet_owned_layer_start;
|
||||
+ int32_t meshnet_owned_layer_end;
|
||||
+
|
||||
// Keep the booleans together to avoid misalignment during copy-by-value.
|
||||
@@ -616,6 +635,13 @@ extern "C" {
|
||||
LLAMA_API uint64_t llama_model_size(const struct llama_model * model);
|
||||
|
||||
+ LLAMA_API bool llama_model_meshnet_range_report(
|
||||
+ const struct llama_model * model,
|
||||
+ struct llama_meshnet_range_report * out);
|
||||
+
|
||||
// Get the default chat template. Returns nullptr if not available
|
||||
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
|
||||
index d8748138..4d2a3ec1 100644
|
||||
--- a/src/llama-model.cpp
|
||||
+++ b/src/llama-model.cpp
|
||||
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
|
||||
std::vector<float> tensor_split_owned;
|
||||
+ llama_meshnet_range_report meshnet_range_report = {};
|
||||
+ bool has_meshnet_range_report = false;
|
||||
};
|
||||
@@ -1236,6 +1239,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||
const bool use_mmap_buffer = true;
|
||||
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
|
||||
+ const int meshnet_start = params.meshnet_owned_layer_start;
|
||||
+ const int meshnet_end = params.meshnet_owned_layer_end;
|
||||
+ if (meshnet_range_requested) {
|
||||
+ if (arch != LLM_ARCH_LLAMA) {
|
||||
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
|
||||
+ }
|
||||
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
|
||||
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
|
||||
+ meshnet_start, meshnet_end, hparams.n_layer()));
|
||||
+ }
|
||||
+ }
|
||||
@@ -1336,7 +1352,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||
- for (int i = 0; i < n_layer_all; ++i) {
|
||||
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
|
||||
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
|
||||
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
|
||||
@@ -1487,7 +1505,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||
- ml.done_getting_tensors();
|
||||
+ ml.done_getting_tensors(meshnet_range_requested);
|
||||
@@ -1613,8 +1631,11 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||
+ uint64_t meshnet_mapped_bytes = 0;
|
||||
+ uint64_t meshnet_resident_bytes = 0;
|
||||
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
|
||||
for (auto & buf: bufs) {
|
||||
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
|
||||
@@ -1637,6 +1658,35 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||
}
|
||||
+ if (meshnet_range_requested) {
|
||||
+ uint64_t registered_bytes = 0;
|
||||
+ for (const auto & [_, tensor] : tensors_by_name) registered_bytes += ggml_nbytes(tensor);
|
||||
+ if (ml.use_mmap) for (const auto & [first, last] : ml.mmaps_used) if (last > first) meshnet_mapped_bytes += last - first;
|
||||
+ const auto registered = [this](const ggml_tensor * tensor) {
|
||||
+ return tensor != nullptr && std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
|
||||
+ [tensor](const auto & entry) { return entry.second == tensor; });
|
||||
+ };
|
||||
+ const auto registered_name = [this](const char * name) {
|
||||
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
|
||||
+ [name](const auto & entry) { return entry.first == name; });
|
||||
+ };
|
||||
+ pimpl->meshnet_range_report = { meshnet_start, meshnet_end, meshnet_mapped_bytes, meshnet_resident_bytes,
|
||||
+ registered_bytes, registered_name("token_embd.weight"), registered(output_norm) && registered(output) };
|
||||
+ pimpl->has_meshnet_range_report = true;
|
||||
+ }
|
||||
return true;
|
||||
@@ -1711,6 +1761,14 @@ uint64_t llama_model::n_elements() const {
|
||||
}
|
||||
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
|
||||
+ if (out == nullptr || !pimpl->has_meshnet_range_report) return false;
|
||||
+ *out = pimpl->meshnet_range_report;
|
||||
+ return true;
|
||||
+}
|
||||
@@ -2308,6 +2366,8 @@ llama_model_params llama_model_default_params() {
|
||||
/*.kv_overrides =*/ nullptr,
|
||||
+ /*.meshnet_owned_layer_start =*/ 0,
|
||||
+ /*.meshnet_owned_layer_end =*/ 0,
|
||||
@@ -2641,6 +2701,10 @@ uint64_t llama_model_size(const llama_model * model) {
|
||||
}
|
||||
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
|
||||
+ return model != nullptr && model->meshnet_range_report(out);
|
||||
+}
|
||||
diff --git a/src/llama-model.h b/src/llama-model.h
|
||||
index 45b054ce..1b3f9bd0 100644
|
||||
--- a/src/llama-model.h
|
||||
+++ b/src/llama-model.h
|
||||
@@ -652,6 +652,8 @@ struct llama_model {
|
||||
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
|
||||
+
|
||||
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
|
||||
index 4bfebc88..b4f25aed 100644
|
||||
--- a/src/models/llama.cpp
|
||||
+++ b/src/models/llama.cpp
|
||||
@@ -34,18 +34,26 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
|
||||
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
|
||||
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
|
||||
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
|
||||
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
|
||||
- // output
|
||||
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
|
||||
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
|
||||
+ if (!meshnet_range_requested || meshnet_start == 0) tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
|
||||
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
|
||||
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
|
||||
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
|
||||
- // if output is NULL, init from the input tok embed
|
||||
- if (output == NULL) {
|
||||
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
|
||||
+ if (output == NULL) output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
|
||||
}
|
||||
- for (int i = 0; i < n_layer; ++i) {
|
||||
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
|
||||
@@ -102,6 +110,25 @@ llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_grap
|
||||
+ llama_meshnet_range_report meshnet_report = {};
|
||||
+ if (model.meshnet_range_report(&meshnet_report)) {
|
||||
+ if (meshnet_report.start_layer != 0) throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
|
||||
+ if (meshnet_report.end_layer != n_layer) throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
|
||||
+ if (!meshnet_report.has_token_embeddings) throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
|
||||
+ if (!meshnet_report.has_output_head) throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
|
||||
+ }
|
||||
ggml_tensor * cur;
|
||||
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
|
||||
index 855295c1..9a7be6ee 100644
|
||||
--- a/tests/CMakeLists.txt
|
||||
+++ b/tests/CMakeLists.txt
|
||||
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
|
||||
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
|
||||
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
|
||||
new file mode 100644
|
||||
index 00000000..7b58ebf8
|
||||
--- /dev/null
|
||||
+++ b/tests/test-meshnet-range-ownership.cpp
|
||||
@@ -0,0 +1,6 @@
|
||||
+#include "ggml.h"
|
||||
+#include "gguf.h"
|
||||
+#include "llama.h"
|
||||
+#include "../src/llama-model.h"
|
||||
+#include <cstdio>
|
||||
+#include <cstring>
|
||||
3
packages/node/native/llama/patches/SHA256SUMS
Normal file
3
packages/node/native/llama/patches/SHA256SUMS
Normal file
@@ -0,0 +1,3 @@
|
||||
# SHA-256 digests for the ordered patch series. Do not reorder this file.
|
||||
1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
|
||||
51c205e3ca26e104f80c838eeeb11115b8d436036014116d2bb407178c30e0bd 0002-dense-llama-owned-range-loader.patch
|
||||
2
packages/node/native/llama/patches/series
Normal file
2
packages/node/native/llama/patches/series
Normal file
@@ -0,0 +1,2 @@
|
||||
0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
|
||||
0002-dense-llama-owned-range-loader.patch
|
||||
@@ -33,6 +33,16 @@ enum SchemaVersion {
|
||||
SCHEMA_VERSION_1 = 1;
|
||||
}
|
||||
|
||||
// Certified transformer boundary family. Names alone are not an adapter: a
|
||||
// Node must select an explicit architecture contract before interpreting a
|
||||
// TensorBundle.
|
||||
enum ArchitectureType {
|
||||
ARCHITECTURE_TYPE_UNSPECIFIED = 0;
|
||||
ARCHITECTURE_TYPE_DENSE = 1;
|
||||
ARCHITECTURE_TYPE_MOE = 2;
|
||||
ARCHITECTURE_TYPE_MLA = 3;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tensor bundle — the public activation boundary
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -142,6 +152,11 @@ message TensorBundle {
|
||||
// boundary payload can evolve without a whole-protocol version bump.
|
||||
uint32 bundle_version = 1;
|
||||
repeated NamedTensor tensors = 2;
|
||||
// Explicit adapter selection; names are never interpreted through unchecked
|
||||
// substitutions. UNSPECIFIED is accepted only for legacy DGR-002 bundles.
|
||||
ArchitectureType architecture = 3;
|
||||
// Adapter-owned semantic boundary, such as "pre_tail_residual".
|
||||
string boundary_point = 4;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -393,8 +408,9 @@ message ActivationChunk {
|
||||
// A decode step is one token: the envelope's identity fields are already fixed
|
||||
// for the life of the stream, so repeating them per token is pure overhead on
|
||||
// the hottest path. A DecodeStep carries only what changes — the step, the
|
||||
// position, and one tensor — and inherits the rest from the SessionOpen
|
||||
// handshake. A peer may always fall back to ActivationChunk with PHASE_DECODE.
|
||||
// position, and one compact boundary encoding — and inherits the rest from the
|
||||
// SessionOpen handshake. A Node may always fall back to ActivationChunk with
|
||||
// PHASE_DECODE.
|
||||
message DecodeStep {
|
||||
// Idempotency step within the session; also orders the stream.
|
||||
uint64 idempotency_step = 1;
|
||||
@@ -402,11 +418,50 @@ message DecodeStep {
|
||||
uint64 position = 2;
|
||||
// Tokens the receiver's cache must already hold. A mismatch is a CACHE_MISS.
|
||||
uint64 expected_past_len = 3;
|
||||
// The single boundary tensor for this token, typically [1, 1, hidden].
|
||||
// Legacy compact one-tensor boundary, typically [1, 1, hidden]. New readers
|
||||
// accept it as a TensorBundle with one member; writers retain it where the
|
||||
// selected architecture genuinely requires only one tensor.
|
||||
NamedTensor tensor = 4;
|
||||
// Work id for attribution; the session/epoch/fingerprint/range are inherited.
|
||||
string work_id = 5;
|
||||
int64 deadline_unix_nanos = 6;
|
||||
// Versioned architecture boundary. This carries sidebands such as MoE router
|
||||
// state or MLA/IndexShare state; when both fields are set, bundle is
|
||||
// authoritative and tensor is retained only for older-Node forwarding.
|
||||
TensorBundle bundle = 7;
|
||||
}
|
||||
|
||||
// Exact request-level identity for a tail result. Runtime recipe identity is
|
||||
// deliberately repeated here: sampling the right logits with the wrong chat
|
||||
// template or reasoning mode is a different request, not a compatible result.
|
||||
message RequestRecipeIdentity {
|
||||
string request_id = 1;
|
||||
string runtime_recipe_digest = 2;
|
||||
string chat_template_id = 3;
|
||||
string chat_template_version = 4;
|
||||
string reasoning_mode = 5;
|
||||
ArchitectureType architecture = 6;
|
||||
}
|
||||
|
||||
// Sampling is an explicit tail-only operation. A non-tail Shard never applies
|
||||
// final norm, LM head, row pruning, or sampling to its boundary output.
|
||||
message SamplingParameters {
|
||||
float temperature = 1;
|
||||
float top_p = 2;
|
||||
uint32 top_k = 3;
|
||||
uint64 seed = 4;
|
||||
bool greedy = 5;
|
||||
}
|
||||
|
||||
// Typed tail completion. The oneof keeps token id 0 unambiguous and prevents a
|
||||
// caller from treating an activation tensor as an inferred completion.
|
||||
message TailResult {
|
||||
RequestRecipeIdentity identity = 1;
|
||||
SamplingParameters sampling = 2;
|
||||
oneof output {
|
||||
TensorBundle logits = 3;
|
||||
uint32 sampled_token_id = 4;
|
||||
}
|
||||
}
|
||||
|
||||
// Drop session state for a Route Session. Bounded memory does not depend on
|
||||
@@ -472,6 +527,7 @@ message SessionResponse {
|
||||
Ack ack = 3;
|
||||
FlowControl flow_control = 4;
|
||||
ShardStatus status = 5;
|
||||
TailResult tail_result = 6;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
BIN
packages/node/native/testdata/decode_step_golden.binpb
vendored
Normal file
BIN
packages/node/native/testdata/decode_step_golden.binpb
vendored
Normal file
Binary file not shown.
@@ -265,7 +265,9 @@ void TestDecodeFastPathIsSmall() {
|
||||
step->set_position(1024);
|
||||
step->set_expected_past_len(1024);
|
||||
step->set_work_id("work-7f3a");
|
||||
sp::NamedTensor *tensor = step->mutable_tensor();
|
||||
sp::TensorBundle *bundle = step->mutable_bundle();
|
||||
bundle->set_bundle_version(1);
|
||||
sp::NamedTensor *tensor = bundle->add_tensors();
|
||||
tensor->set_name("hidden_states");
|
||||
tensor->add_shape(1);
|
||||
tensor->add_shape(1);
|
||||
@@ -290,7 +292,27 @@ void TestDecodeFastPathIsSmall() {
|
||||
CHECK(parsed.ParseFromString(bytes));
|
||||
CHECK(parsed.kind_case() == sp::SessionRequest::kDecode);
|
||||
CHECK_EQ(parsed.decode().position(), 1024u);
|
||||
CHECK_EQ(parsed.decode().tensor().total_bytes(), 16u);
|
||||
CHECK_EQ(parsed.decode().bundle().tensors_size(), 1);
|
||||
CHECK_EQ(parsed.decode().bundle().tensors(0).total_bytes(), 16u);
|
||||
}
|
||||
|
||||
void TestDecodeBundleVector(const std::string &bytes) {
|
||||
sp::SessionRequest request;
|
||||
CHECK(request.ParseFromString(bytes));
|
||||
CHECK(request.kind_case() == sp::SessionRequest::kDecode);
|
||||
const sp::DecodeStep &step = request.decode();
|
||||
CHECK_EQ(step.idempotency_step(), 43u);
|
||||
CHECK_EQ(step.position(), 384u);
|
||||
CHECK_EQ(step.expected_past_len(), 384u);
|
||||
CHECK_EQ(step.work_id(), std::string("decode-7f3a"));
|
||||
CHECK(step.has_bundle());
|
||||
CHECK_EQ(step.bundle().bundle_version(), 1u);
|
||||
CHECK_EQ(step.bundle().architecture(), sp::ARCHITECTURE_TYPE_MLA);
|
||||
CHECK_EQ(step.bundle().boundary_point(), std::string("pre_tail_residual"));
|
||||
CHECK_EQ(step.bundle().tensors_size(), 2);
|
||||
CHECK_EQ(step.bundle().tensors(0).name(), std::string("hidden_states"));
|
||||
CHECK_EQ(step.bundle().tensors(1).name(), std::string("index_topk"));
|
||||
CHECK_EQ(step.bundle().tensors(1).dtype(), sp::DTYPE_INT32);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
@@ -308,9 +330,11 @@ int main(int argc, char **argv) {
|
||||
ReadFile(testdata / "session_request_golden.binpb");
|
||||
const std::string capability_bytes =
|
||||
ReadFile(testdata / "capability_report_golden.binpb");
|
||||
const std::string decode_bytes = ReadFile(testdata / "decode_step_golden.binpb");
|
||||
|
||||
TestSessionRequestVector(session_bytes);
|
||||
TestCapabilityReportVector(capability_bytes);
|
||||
TestDecodeBundleVector(decode_bytes);
|
||||
TestUnknownFieldsArePreserved(session_bytes);
|
||||
TestSparseMessageParses();
|
||||
TestDecodeFastPathIsSmall();
|
||||
|
||||
@@ -42,3 +42,4 @@ include = ["meshnet_node*"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
meshnet_node = ["*.json"]
|
||||
"meshnet_node.glm_alpha" = ["data/*.json"]
|
||||
|
||||
@@ -7,7 +7,6 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import socket
|
||||
import threading
|
||||
from typing import Callable
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -3,9 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import ssl
|
||||
|
||||
@@ -4,7 +4,6 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@@ -30,6 +30,13 @@ import time
|
||||
from dataclasses import dataclass, replace
|
||||
from typing import Any, Callable, Mapping
|
||||
|
||||
from .recipe import (
|
||||
CertificationLedger,
|
||||
FingerprintMismatch,
|
||||
RecipeIdentityError,
|
||||
parse_identity,
|
||||
)
|
||||
|
||||
# The capability report layout this tracker reads (meshnet_node.capability).
|
||||
SUPPORTED_SCHEMA_VERSION = 1
|
||||
|
||||
@@ -58,6 +65,12 @@ STATE_MODEL_MISMATCH = "model-mismatch"
|
||||
STATE_SHARD_MISMATCH = "shard-mismatch"
|
||||
STATE_RECIPE_MISMATCH = "recipe-mismatch"
|
||||
STATE_CATALOGUE_INCOMPATIBLE = "catalogue-incompatible"
|
||||
# The node presented a DGR-003 identity block whose declared fingerprint is not
|
||||
# the digest of the axes it declared. Identity is derived, never asserted.
|
||||
STATE_FINGERPRINT_MISMATCH = "fingerprint-mismatch"
|
||||
# Registered-but-dark: a known recipe no real distributed forward has certified.
|
||||
# Visible to an operator, never routable for user traffic.
|
||||
STATE_UNCERTIFIED = "uncertified"
|
||||
|
||||
ALL_STATES = (
|
||||
STATE_ADMITTED,
|
||||
@@ -69,6 +82,8 @@ ALL_STATES = (
|
||||
STATE_SHARD_MISMATCH,
|
||||
STATE_RECIPE_MISMATCH,
|
||||
STATE_CATALOGUE_INCOMPATIBLE,
|
||||
STATE_FINGERPRINT_MISMATCH,
|
||||
STATE_UNCERTIFIED,
|
||||
)
|
||||
|
||||
# --- Compatibility policy for nodes that predate the capability protocol. ---
|
||||
@@ -165,12 +180,29 @@ class CapabilityState:
|
||||
recorded_at: float = 0.0
|
||||
schema_version: int | None = None
|
||||
diagnostics: tuple[str, ...] = ()
|
||||
# The DGR-003 compatibility fingerprint, *re-derived* by this tracker from
|
||||
# the axes the node declared — never copied from what the node claimed.
|
||||
# Absent for a node that predates DGR-003.
|
||||
model_artifact_digest: str | None = None
|
||||
runtime_recipe_digest: str | None = None
|
||||
shard_binding_digest: str | None = None
|
||||
# The tracker ledger's verdict on that fingerprint at evaluation time
|
||||
# ("dark"/"certified"), so the network map answers "why is this exact node
|
||||
# not routing" without a second query. None when no identity was presented.
|
||||
certification: str | None = None
|
||||
|
||||
@property
|
||||
def proven(self) -> bool:
|
||||
"""The presented proof covers exactly what the node advertised."""
|
||||
return self.state == STATE_ADMITTED
|
||||
|
||||
@property
|
||||
def fingerprint(self) -> tuple[str, str] | None:
|
||||
"""What route formation compares. `None` when the node declares no identity."""
|
||||
if self.model_artifact_digest is None or self.runtime_recipe_digest is None:
|
||||
return None
|
||||
return (self.model_artifact_digest, self.runtime_recipe_digest)
|
||||
|
||||
def routable_under(self, policy: str) -> bool:
|
||||
if self.proven:
|
||||
return True
|
||||
@@ -197,6 +229,10 @@ class CapabilityState:
|
||||
"recorded_at": self.recorded_at,
|
||||
"schema_version": self.schema_version,
|
||||
"diagnostics": list(self.diagnostics),
|
||||
"model_artifact_digest": self.model_artifact_digest,
|
||||
"runtime_recipe_digest": self.runtime_recipe_digest,
|
||||
"shard_binding_digest": self.shard_binding_digest,
|
||||
"certification": self.certification,
|
||||
}
|
||||
|
||||
|
||||
@@ -224,6 +260,7 @@ def evaluate_report(
|
||||
declared_recipe_version: str | None = None,
|
||||
now: float | None = None,
|
||||
max_age_seconds: float = DEFAULT_MAX_REPORT_AGE_SECONDS,
|
||||
ledger: CertificationLedger | None = None,
|
||||
) -> CapabilityState:
|
||||
"""Judge the proof a node presented against what that node is advertising.
|
||||
|
||||
@@ -308,6 +345,84 @@ def evaluate_report(
|
||||
f"the node declared v{declared_recipe_version}",
|
||||
)
|
||||
|
||||
identity = None
|
||||
if report.get("identity") is not None:
|
||||
try:
|
||||
identity = parse_identity(report["identity"])
|
||||
except FingerprintMismatch as exc:
|
||||
return base.with_state(STATE_FINGERPRINT_MISMATCH, str(exc))
|
||||
except RecipeIdentityError as exc:
|
||||
return base.with_state(
|
||||
STATE_INVALID, f"capability identity block is unusable: {exc}"
|
||||
)
|
||||
|
||||
# The report's `shard` range is inclusive/inclusive (the CLI and backend
|
||||
# convention); the identity's is inclusive/exclusive (the protocol's, and
|
||||
# ADR-0012's). A node whose two halves disagree has not proven the range
|
||||
# it claims, whichever one is right.
|
||||
if (identity.shard_start, identity.shard_end) != (
|
||||
base.shard_start,
|
||||
(base.shard_end or 0) + 1,
|
||||
):
|
||||
return base.with_state(
|
||||
STATE_SHARD_MISMATCH,
|
||||
f"identity covers layers {identity.shard_start}–{identity.shard_end} "
|
||||
f"(end-exclusive), but the proof is for layers {base.shard_start}–"
|
||||
f"{base.shard_end} (end-inclusive)",
|
||||
)
|
||||
|
||||
if not model_matches(identity.artifact_id):
|
||||
return base.with_state(
|
||||
STATE_MODEL_MISMATCH,
|
||||
f"identity is for artifact {identity.artifact_id!r}, but the node "
|
||||
f"registered {advertised_model!r}",
|
||||
)
|
||||
model_claim = report["model"]
|
||||
if model_claim.get("revision") != identity.revision:
|
||||
return base.with_state(
|
||||
STATE_MODEL_MISMATCH,
|
||||
"identity revision does not match the capability proof",
|
||||
)
|
||||
config_fingerprint = model_claim.get("config_fingerprint")
|
||||
if isinstance(config_fingerprint, str) and config_fingerprint.startswith(
|
||||
"sha256:"
|
||||
):
|
||||
config_fingerprint = config_fingerprint.removeprefix("sha256:")
|
||||
if config_fingerprint != identity.architecture_digest:
|
||||
return base.with_state(
|
||||
STATE_MODEL_MISMATCH,
|
||||
"identity architecture/config digest does not match the capability proof",
|
||||
)
|
||||
if (
|
||||
identity.recipe_id != base.recipe_id
|
||||
or identity.recipe_version != base.recipe_version
|
||||
or identity.catalogue_version != base.catalogue_version
|
||||
):
|
||||
return base.with_state(
|
||||
STATE_RECIPE_MISMATCH,
|
||||
"identity recipe labels do not match the capability proof",
|
||||
)
|
||||
if identity.axes["backend_id"] != base.backend_id:
|
||||
return base.with_state(
|
||||
STATE_RECIPE_MISMATCH,
|
||||
"identity backend does not match the capability proof",
|
||||
)
|
||||
if (
|
||||
base.quantization is not None
|
||||
and identity.axes["weight_quantization"] != base.quantization
|
||||
):
|
||||
return base.with_state(
|
||||
STATE_RECIPE_MISMATCH,
|
||||
"identity weight quantization does not match the capability proof",
|
||||
)
|
||||
|
||||
base = replace(
|
||||
base,
|
||||
model_artifact_digest=identity.model_artifact_digest,
|
||||
runtime_recipe_digest=identity.runtime_recipe_digest,
|
||||
shard_binding_digest=identity.shard_binding_digest,
|
||||
)
|
||||
|
||||
if status != STATUS_PASSED:
|
||||
return base.with_state(
|
||||
STATE_FAILED,
|
||||
@@ -328,6 +443,29 @@ def evaluate_report(
|
||||
f"proof is timestamped {-age:.0f}s in the future; check the node's clock",
|
||||
)
|
||||
|
||||
# The digest only establishes that this report is self-consistent. It does
|
||||
# not authenticate a node or prove a distributed forward. Exact recipes are
|
||||
# therefore registered-but-dark until tracker-owned certification records
|
||||
# that forward.
|
||||
#
|
||||
# `ledger is None` means the caller owns no certification authority. It is
|
||||
# not "certify anything" and it is not "make one up": a disposable ledger
|
||||
# would register the recipe into state that is discarded on return, which
|
||||
# reads like certification is wired when nothing is recording it. The only
|
||||
# safe reading is that nothing here has certified this recipe, so it stays
|
||||
# dark. `TrackerServer` owns the real ledger and passes it in.
|
||||
if identity is not None:
|
||||
if ledger is None:
|
||||
return base.with_state(
|
||||
STATE_UNCERTIFIED,
|
||||
"no certification ledger; an exact recipe is dark until a "
|
||||
"tracker-owned distributed forward certifies it",
|
||||
)
|
||||
recipe_status = ledger.register(identity)
|
||||
base = replace(base, certification=recipe_status.status)
|
||||
if not recipe_status.may_serve:
|
||||
return base.with_state(STATE_UNCERTIFIED, recipe_status.detail)
|
||||
|
||||
return base.with_state(
|
||||
STATE_ADMITTED,
|
||||
f"{base.model_id} layers {base.shard_start}–{base.shard_end} proven on "
|
||||
|
||||
@@ -16,8 +16,8 @@ import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Callable
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable
|
||||
|
||||
|
||||
@dataclass
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user