popov/neuron-tai
1
0
Fork 0
Code Issues 42 Pull Requests Actions Packages Projects Releases Wiki Activity

story: DGR-034 Implement dense-Llama range-aware GGUF ownership

Browse Source
This commit is contained in:
Dobromir Popov
2026-08-01 01:08:28 +03:00
parent 27a0d89678
commit d339cfde25
18 changed files with 1393 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

94
.scratch/distributed-gguf-runtime/evidence/DGR-034/README.md Normal file
View File

@@ -0,0 +1,94 @@
# DGR-034 evidence — dense-Llama range-aware GGUF ownership
**Status:** implemented and live-verified on 2026-08-01. `prd.json` remains
the authority for story state.
## What changed
- The pinned llama.cpp patch stack adds `meshnet_owned_layer_start/end` and
filters dense-Llama GGUF registration to `blk.N.*` for the requested
half-open range. `token_embd.weight` belongs to the head; `output_norm` and
`output.weight` (or the tied embedding) belong to the tail.
- The load state exposes a C range report derived from the registered model
buffers, and a project-owned `meshnet-range-report` tool audits the live
registered tensor map. It rejects empty, inverted, out-of-model, missing,
outside-range, unexpected, and endpoint-inconsistent loads.
- `meshnet_node.range_report` accepts only audited tool output. It makes the
range and endpoint flags authoritative from loaded state rather than caller
assertions, and fails closed on malformed ownership or byte counts.
## Real-model memory evidence
Artifact: `Magistral-Small-2509-Q4_K_M.gguf`, 14,333,911,104 bytes, SHA-256
`a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc`.
It stayed on the configured mounted drive; no artifact was downloaded or put
under `/home`.
The direct non-mmap lane proves resident storage tracks owned tensors:
| Range | Registered tensors | Resident bytes | Process peak RSS |
| --- | ---: | ---: | ---: |
| `[10, 20)` | 90 | 3,304,898,560 | 3,298,800 KiB |
| `[0, 40)` | 363 | 14,326,026,240 | 14,061,632 KiB |
Raw reports and timings are in `runs/default-mid-a.*` and
`runs/default-full-nommap.*`. The middle range is 23.1% of the full
resident allocation and owns 24.8% of the registered tensors.
## Commands and results
```text
python3 scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
python3 scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# apply/check/reverse succeeded against e920c523e3b8a0163fe498af5bf90df35ff51d25;
# the source was then applied for the focused native checks.
(cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS)
# all six patches: OK
/home/popov/.hermes/hermes-agent/venv/bin/ctest \
--test-dir build/llama.cpp/dgr034-check \
-R '^test-meshnet-range-ownership$' --output-on-failure
# 1/1 passed
PYTHONPATH=packages/node MESHNET_RANGE_REPORT_BIN="$PWD/build/llama.cpp/dgr034-check/bin/meshnet-range-report" \
/home/popov/.hermes/hermes-agent/venv/bin/pytest -q \
tests/test_range_report.py tests/test_meshnet_range_report_tool.py \
tests/test_llama_cpp_dependency.py
# 56 passed in 0.87s
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python \
-m compileall -q packages tests
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
git diff --check && git diff --cached --check
# all exit 0; PRD validation: 55 stories validated
```
The model commands used the same `meshnet-range-report` binary with
`--no-mmap --no-extra-bufts`, first for `[10,20)` and then `[0,40)`; both
returned `ok: true` and their exact output is retained above.
## Changed files
- `packages/node/native/llama/PATCH-STACK.md`
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/patches/{series,SHA256SUMS,UPSTREAM-ASSUMPTIONS.json,0006-meshnet-range-report-tool.patch}`
- `packages/node/meshnet_node/range_report.py`
- `tests/test_range_report.py`
- `tests/test_meshnet_range_report_tool.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-034/*`
## Limitations and dependency handoff
- The mmap loader can retain broad contiguous file spans when GGUF tensor
order places a tail endpoint near the beginning of the artifact; the direct
non-mmap lane is the certified resident-memory result. The raw mmap report
is retained in `runs/default-head.json` and must not be presented as a
physical-RSS saving.
- This story proves loading/ownership only. Partial-range graph execution
remains fail-closed until DGR-035 provides typed dense boundary adapters.
- DGR-037 can bind the worker to `llama_model_meshnet_range_report` or the
strict Python consumer; it must use the reported range, not requested range,
for capability publication. DGR-051 must add its V4-specific ownership
rules separately.

1
.scratch/distributed-gguf-runtime/evidence/DGR-034/artifact-sha256.txt Normal file
View File

@@ -0,0 +1 @@
a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc Magistral-Small-2509-Q4_K_M.gguf

24
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-full-nommap.json Normal file
View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 40],
"reported_range": [0, 40],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": true,
"has_output_head": true,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 14326026240,
"registered_tensors": 363,
"registered_bytes": 14326026240,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 14392061952,
"vm_rss_bytes": 14387003392,
"vm_hwm_bytes": 14399111168
}

1
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-full-nommap.stderr Normal file
View File

@@ -0,0 +1 @@
elapsed=0:02.48 maxrss_kib=14061632 exit=0

24
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-head.json Normal file
View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 10],
"reported_range": [0, 10],
"mmap": true,
"touched": false,
"use_extra_bufts": true,
"has_token_embeddings": true,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 6219366400,
"resident_bytes": 6219366400,
"registered_tensors": 91,
"registered_bytes": 3771596800,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 16942260224,
"vm_rss_bytes": 16937005056,
"vm_hwm_bytes": 16947953664
}

0
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-head.stderr Normal file
View File

24
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-mid-a.json Normal file
View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [10, 20],
"reported_range": [10, 20],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": false,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 3304898560,
"registered_tensors": 90,
"registered_bytes": 3304898560,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 3370934272,
"vm_rss_bytes": 3365814272,
"vm_hwm_bytes": 3377971200
}

1
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-mid-a.stderr Normal file
View File

@@ -0,0 +1 @@
elapsed=0:00.82 maxrss_kib=3298800 exit=0

5
.scratch/distributed-gguf-runtime/prd.json
View File

@@ -954,13 +954,14 @@
"Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
],
"passes": false,
"passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md; prd.json is authoritative.",
"blocks": [
"DGR-035",
"DGR-037",
"DGR-051"
]
],
"completionNotes": "Dense-Llama owned-range loading is verified with strict engine-state reports, focused native/Python tests, and a real GGUF resident-memory comparison."
},
{
"id": "DGR-035",

218
packages/node/meshnet_node/range_report.py Normal file
View File

@@ -0,0 +1,218 @@
"""Authoritative dense-Llama owned-range reports from the loaded engine state.
DGR-034 loads only the tensors a shard range owns through the Meshnet
owned-range loader (``llama_model_params::meshnet_owned_layer_start/end`` in
the pinned llama.cpp patch stack). The project-owned ``meshnet-range-report``
native tool runs that load and prints a JSON document derived from the loaded
model state — the registered tensor set and the backend buffers — never from
caller-asserted values. This module is the strict consumer of that document:
it parses it into :class:`OwnedRangeReport` and fails closed on any
inconsistency, so a range or endpoint claim that the loaded engine state does
not back is rejected before it can reach identity, admission, or routing.
Ownership contract enforced here (dense Llama only):
- every registered ``blk.N.*`` tensor lies inside the half-open owned range
``[start, end)``, and every layer in that range is present — a gapped or
out-of-range registration is rejected;
- ``token_embd.weight`` is registered only by the head shard (``start == 0``),
or by a tail shard whose model ties the output head to the embedding
(``end == n_layer`` and no separate ``output.weight``);
- ``output_norm.weight`` and ``output.weight`` are registered only by the
tail shard (``end == n_layer``);
- any other registered tensor name is unexpected and rejected;
- byte counts are consistent: an mmap load maps a file span at least the
registered tensor bytes and at most the artifact size; a non-mmap load
reports a resident allocation at least the registered tensor bytes.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Mapping
class RangeReportError(ValueError):
"""A range report is malformed, or the loaded state breaks ownership."""
_DENSE_ARCHITECTURE = "llama"
_INT_FIELDS = (
"n_layer",
"file_bytes",
"mapped_bytes",
"resident_bytes",
"registered_tensors",
"registered_bytes",
)
_BOOL_FIELDS = (
"mmap",
"touched",
"has_token_embeddings",
"has_output_head",
"tied_output_head",
)
@dataclass(frozen=True)
class OwnedRangeReport:
"""One validated owned-range load, derived from loaded engine state.
``start_layer``/``end_layer`` are the authoritative half-open owned range
the engine actually registered (the tool already refused a report whose
loaded bounds differ from the requested ones). ``has_token_embeddings`` is
true for the head shard, and also for a tail shard on a tied-output model
(the embedding tensor *is* its output head); ``tied_output_head``
disambiguates those two cases. ``mapped_bytes``/``resident_bytes`` come
from the backend buffers: with mmap they are the mapped file span holding
the owned tensors, without mmap the resident allocation holding them.
"""
architecture: str
n_layer: int
start_layer: int
end_layer: int
has_token_embeddings: bool
has_output_head: bool
tied_output_head: bool
mapped_bytes: int
resident_bytes: int
registered_tensors: int
registered_bytes: int
file_bytes: int
mmap: bool
touched: bool
vm_size_bytes: int | None
vm_rss_bytes: int | None
vm_hwm_bytes: int | None
@property
def is_head(self) -> bool:
return self.start_layer == 0
@property
def is_tail(self) -> bool:
return self.end_layer == self.n_layer
def __post_init__(self) -> None:
if self.architecture != _DENSE_ARCHITECTURE:
raise RangeReportError(
f"owned-range loading supports dense Llama only, got {self.architecture!r}"
)
if isinstance(self.n_layer, bool) or self.n_layer < 1:
raise RangeReportError("report must record a positive GGUF block count")
for name in _INT_FIELDS:
value = getattr(self, name)
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
raise RangeReportError(f"report field {name!r} must be a non-negative integer")
for name in _BOOL_FIELDS:
if not isinstance(getattr(self, name), bool):
raise RangeReportError(f"report field {name!r} must be a boolean")
if not 0 <= self.start_layer < self.end_layer <= self.n_layer:
raise RangeReportError(
f"owned range [{self.start_layer}, {self.end_layer}) is empty or "
f"outside the model's {self.n_layer} layers"
)
if self.tied_output_head and not self.is_tail:
raise RangeReportError("a tied output head can only belong to the tail shard")
expected_embeddings = self.is_head or self.tied_output_head
if self.has_token_embeddings != expected_embeddings:
raise RangeReportError(
"token-embedding registration disagrees with endpoint ownership: "
"embeddings belong to the head shard (or to a tied-output tail)"
)
if self.has_output_head != self.is_tail:
raise RangeReportError(
"output-head registration disagrees with endpoint ownership: "
"the final norm and output head belong to the tail shard"
)
if self.registered_tensors < 1 or self.registered_bytes < 1:
raise RangeReportError("the owned range registered no tensors")
if self.file_bytes < 1:
raise RangeReportError("report must record the artifact size")
if self.mmap:
if self.mapped_bytes < self.registered_bytes:
raise RangeReportError(
"mapped span undercounts the registered owned tensors"
)
if self.mapped_bytes > self.file_bytes:
raise RangeReportError("mapped span exceeds the artifact size")
else:
if self.mapped_bytes != 0:
raise RangeReportError("a non-mmap load must not claim a mapped span")
if self.resident_bytes < self.registered_bytes:
raise RangeReportError(
"resident allocation undercounts the registered owned tensors"
)
for name in ("vm_size_bytes", "vm_rss_bytes", "vm_hwm_bytes"):
value = getattr(self, name)
if value is not None and (
isinstance(value, bool) or not isinstance(value, int) or value < 0
):
raise RangeReportError(f"report field {name!r} must be a non-negative integer or null")
def _require_range(doc: Mapping[str, Any], key: str) -> tuple[int, int]:
value = doc.get(key)
if (
not isinstance(value, (list, tuple))
or len(value) != 2
or any(isinstance(v, bool) or not isinstance(v, int) for v in value)
):
raise RangeReportError(f"report field {key!r} must be a [start, end] integer pair")
return value[0], value[1]
def parse_owned_range_report(doc: Mapping[str, Any]) -> OwnedRangeReport:
"""Parse and validate one ``meshnet-range-report`` JSON document.
Fails closed: a load the tool rejected (``ok: false``), a requested range
the loaded state did not match, a gapped or out-of-range registration, an
unexpected registered tensor, and any byte-count inconsistency all raise
:class:`RangeReportError` instead of producing a report.
"""
if not isinstance(doc, Mapping):
raise RangeReportError("range report must be a JSON object")
if doc.get("ok") is not True:
error = doc.get("error")
detail = f": {error}" if isinstance(error, str) and error else ""
raise RangeReportError(f"the owned-range load was rejected{detail}")
requested = _require_range(doc, "requested_range")
reported = _require_range(doc, "reported_range")
if requested != reported:
raise RangeReportError(
f"reported range {reported} does not match the requested range {requested}; "
"ownership must be derived from the loaded engine state"
)
for key in ("unexpected_registered_tensors", "missing_owned_layers"):
value = doc.get(key)
if not isinstance(value, list):
raise RangeReportError(f"report field {key!r} must be a list")
if value:
raise RangeReportError(
f"ownership audit failed: {key} is {value!r}; the registered "
"tensor set must exactly cover the owned range and its endpoints"
)
architecture = doc.get("architecture")
if not isinstance(architecture, str):
raise RangeReportError("report field 'architecture' must be a string")
fields: dict[str, Any] = {}
for name in _INT_FIELDS + _BOOL_FIELDS:
if name not in doc:
raise RangeReportError(f"range report is missing field {name!r}")
fields[name] = doc[name]
for name in ("vm_size_bytes", "vm_rss_bytes", "vm_hwm_bytes"):
fields[name] = doc.get(name)
return OwnedRangeReport(
architecture=architecture,
start_layer=reported[0],
end_layer=reported[1],
**fields,
)

6
packages/node/native/llama/PATCH-STACK.md
View File

@@ -28,6 +28,12 @@ One numbered patch per concern (ADR-0024 local seams only):
5. `0005-worker-range-report-hook.patch` (worker hooks) exposes the
`llama_model_meshnet_range_report` C API the project-owned worker binds to
and registers a model-free native fixture test for it.
6. `0006-meshnet-range-report-tool.patch` (range reporting) adds the
project-owned `meshnet-range-report` tool: it loads one GGUF artifact
through the owned-range loader and prints a JSON document derived from the
loaded model state — the owned-range report, the registered tensor set
audited against the requested ownership, and backend-buffer byte counts.
It never builds or runs a compute graph.
Meshnet routing, Tracker, gRPC, relay, billing, authentication, and telemetry
remain outside this directory; the stack is checked for such control-plane

14
packages/node/native/llama/UPSTREAM_LOCK.json
View File

@@ -10,21 +10,23 @@
"method": "git-clone-detached-commit",
"workspace": "build/llama.cpp"
},
"patched_tree": "c0045714735ae5ee7b7334a480d8ac04e03e1b18",
"patched_tree": "8f7e87fea6743f0b9744afe44f9e6f9ca3b7d08a",
"upstream_license": "MIT",
"patch_series": [
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
"0002-dense-llama-owned-range-loading.patch",
"0003-owned-range-filtered-state-report.patch",
"0004-dense-boundary-io-endpoint-guard.patch",
"0005-worker-range-report-hook.patch"
"0005-worker-range-report-hook.patch",
"0006-meshnet-range-report-tool.patch"
],
"patch_scope": [
"Reserved CMake ABI marker only; no execution or model semantics.",
"Range loading: dense-Llama owned-range params, validation, and filtered tensor registration with endpoint ownership.",
"Filtered state: owned-range report populated from registered tensors and backend buffers, derived never asserted.",
"Boundary I/O: endpoint ownership flags and a fail-closed dense graph guard until typed endpoint adapters exist.",
"Worker hooks: public C range-report API and the model-free native fixture test the project-owned worker binds to."
"Worker hooks: public C range-report API and the model-free native fixture test the project-owned worker binds to.",
"Range reporting: project-owned tool that loads one artifact through the owned-range loader and reports derived ownership and buffer-byte state as JSON."
],
"patch_assumptions": "patches/UPSTREAM-ASSUMPTIONS.json",
"build": {
@@ -46,7 +48,7 @@
"-DGGML_VULKAN=OFF",
"-DGGML_METAL=OFF"
],
"native_targets": ["llama-gguf-hash", "test-meshnet-range-ownership"],
"native_targets": ["llama-gguf-hash", "test-meshnet-range-ownership", "meshnet-range-report"],
"smoke_binary": "bin/llama-gguf-hash",
"smoke_args": ["--help"],
"smoke_output_token": "usage",
@@ -81,7 +83,9 @@
"src/llama-model.h",
"src/models/llama.cpp",
"tests/CMakeLists.txt",
"tests/test-meshnet-range-ownership.cpp"
"tests/test-meshnet-range-ownership.cpp",
"tools/meshnet-range-report/CMakeLists.txt",
"tools/meshnet-range-report/meshnet-range-report.cpp"
],
"stock_glm_limitations": "This pin may load GLM-5.2 through the dense-MLA compatibility fallback. It does not prove native DSA, IndexShare, MoE semantic correctness, numerical equivalence, performance, or route certification."
}

414
packages/node/native/llama/patches/0006-meshnet-range-report-tool.patch Normal file
View File

@@ -0,0 +1,414 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense-Llama owned-range report tool
Concern: range reporting. Adds the project-owned meshnet-range-report tool:
it loads one GGUF artifact through the Meshnet owned-range loader and prints
a JSON document derived from the loaded model state — the owned-range
report, the registered tensor set audited against the requested ownership,
and backend-buffer byte counts (optionally split from repack buffers, plus
process resident readings). It never builds or runs a compute graph and
never trusts caller-asserted range or endpoint claims.
---
diff --git a/CMakeLists.txt b/CMakeLists.txt
index a9afcff..868793b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -281,3 +281,6 @@ configure_file(cmake/llama.pc.in
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/llama.pc"
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
+
+# Meshnet-owned owned-range report tool (patch stack, range-report concern).
+add_subdirectory(tools/meshnet-range-report)
diff --git a/tools/meshnet-range-report/CMakeLists.txt b/tools/meshnet-range-report/CMakeLists.txt
new file mode 100644
index 000000000..24401007e
--- /dev/null
+++ b/tools/meshnet-range-report/CMakeLists.txt
@@ -0,0 +1,7 @@
+# Meshnet-owned dense-Llama owned-range load/report tool.
+#
+# Built unconditionally with the patched tree: it exercises the Meshnet
+# owned-range loader against real GGUF artifacts and reports only state
+# derived from the loaded model (registered tensors, backend buffers).
+add_executable(meshnet-range-report meshnet-range-report.cpp)
+target_link_libraries(meshnet-range-report PRIVATE llama)
diff --git a/tools/meshnet-range-report/meshnet-range-report.cpp b/tools/meshnet-range-report/meshnet-range-report.cpp
new file mode 100644
index 000000000..49a5eb2a0
--- /dev/null
+++ b/tools/meshnet-range-report/meshnet-range-report.cpp
@@ -0,0 +1,373 @@
+// Meshnet-owned dense-Llama owned-range load/report tool.
+//
+// Loads one GGUF artifact through the Meshnet owned-range loader
+// (llama_model_params::meshnet_owned_layer_start/end) and prints a single
+// JSON report derived from the loaded model state — registered tensors and
+// backend buffers, never caller-asserted values. The audit fails closed when
+// the registered tensor set disagrees with the requested ownership: every
+// registered per-layer tensor must lie inside [start, end), the token
+// embedding may be registered only by the head shard (start == 0) or by a
+// tail shard whose model ties the output head to the embedding, and the
+// final norm plus output head may be registered only by the tail shard
+// (end == n_layer).
+
+#include "ggml.h"
+#include "llama.h"
+
+#include "../../src/llama-model.h"
+
+#include <cstdint>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <set>
+#include <string>
+#include <sys/stat.h>
+#include <vector>
+
+namespace {
+
+constexpr int kExitUsage = 2;
+constexpr int kExitLoad = 3;
+constexpr int kExitAudit = 4;
+
+std::string g_log_tail;
+
+void capture_log(enum ggml_log_level level, const char * text, void *) {
+ if (level >= GGML_LOG_LEVEL_ERROR) {
+ g_log_tail += text;
+ if (g_log_tail.size() > 512) {
+ g_log_tail.erase(0, g_log_tail.size() - 512);
+ }
+ }
+}
+
+std::string json_escape(const std::string & value) {
+ std::string out;
+ for (const char c : value) {
+ if (c == '"' || c == '\\') {
+ out += '\\';
+ out += c;
+ } else if (c == '\n') {
+ out += "\\n";
+ } else if (c == '\r') {
+ // drop carriage returns from embedded log text
+ } else {
+ out += c;
+ }
+ }
+ return out;
+}
+
+std::string json_string_array(const std::vector<std::string> & items) {
+ std::string out = "[";
+ for (size_t i = 0; i < items.size(); ++i) {
+ if (i) {
+ out += ", ";
+ }
+ out += "\"" + json_escape(items[i]) + "\"";
+ }
+ return out + "]";
+}
+
+std::string json_int_array(const std::vector<int> & items) {
+ std::string out = "[";
+ for (size_t i = 0; i < items.size(); ++i) {
+ if (i) {
+ out += ", ";
+ }
+ out += std::to_string(items[i]);
+ }
+ return out + "]";
+}
+
+int fail(int code, const std::string & error) {
+ std::string detail = error;
+ if (!g_log_tail.empty()) {
+ detail += ": " + g_log_tail;
+ }
+ std::printf("{\"ok\": false, \"error\": \"%s\"}\n", json_escape(detail).c_str());
+ return code;
+}
+
+bool parse_nonnegative(const char * text, int & out) {
+ if (text == nullptr || *text == '\0' || *text == '-') {
+ return false;
+ }
+ char * end = nullptr;
+ const long value = std::strtol(text, &end, 10);
+ if (end == text || *end != '\0' || value > INT32_MAX) {
+ return false;
+ }
+ out = static_cast<int>(value);
+ return true;
+}
+
+uint64_t file_size(const std::string & path) {
+ struct stat st;
+ return ::stat(path.c_str(), &st) == 0 ? static_cast<uint64_t>(st.st_size) : 0;
+}
+
+struct proc_status {
+ uint64_t vm_size = 0;
+ uint64_t vm_rss = 0;
+ uint64_t vm_hwm = 0;
+ bool valid = false;
+};
+
+proc_status read_proc_status() {
+ proc_status out;
+#ifdef __linux__
+ FILE * f = std::fopen("/proc/self/status", "r");
+ if (!f) {
+ return out;
+ }
+ char line[256];
+ while (std::fgets(line, sizeof(line), f)) {
+ uint64_t kb = 0;
+ if (std::sscanf(line, "VmSize: %lu kB", &kb) == 1) {
+ out.vm_size = kb * 1024;
+ } else if (std::sscanf(line, "VmRSS: %lu kB", &kb) == 1) {
+ out.vm_rss = kb * 1024;
+ } else if (std::sscanf(line, "VmHWM: %lu kB", &kb) == 1) {
+ out.vm_hwm = kb * 1024;
+ }
+ }
+ std::fclose(f);
+ out.valid = true;
+#endif
+ return out;
+}
+
+void usage(const char * argv0) {
+ std::fprintf(stderr,
+ "usage: %s --model PATH --start N --end M [--no-mmap] [--no-extra-bufts] [--touch]\n"
+ "loads one dense-Llama GGUF through the Meshnet owned-range loader and\n"
+ "prints a JSON report derived from the loaded model state\n",
+ argv0);
+}
+
+} // namespace
+
+int main(int argc, char ** argv) {
+ std::string model_path;
+ int start = -1;
+ int end = -1;
+ bool use_mmap = true;
+ bool use_extra_bufts = true;
+ bool touch = false;
+
+ for (int i = 1; i < argc; ++i) {
+ const std::string arg = argv[i];
+ if (arg == "--model" && i + 1 < argc) {
+ model_path = argv[++i];
+ } else if (arg == "--start" && i + 1 < argc) {
+ if (!parse_nonnegative(argv[++i], start)) {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+ } else if (arg == "--end" && i + 1 < argc) {
+ if (!parse_nonnegative(argv[++i], end)) {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+ } else if (arg == "--no-mmap") {
+ use_mmap = false;
+ } else if (arg == "--no-extra-bufts") {
+ use_extra_bufts = false;
+ } else if (arg == "--touch") {
+ touch = true;
+ } else {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+ }
+ if (model_path.empty() || start < 0 || end < 0) {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+
+ llama_log_set(capture_log, nullptr);
+ llama_backend_init();
+
+ llama_model_params params = llama_model_default_params();
+ params.meshnet_owned_layer_start = start;
+ params.meshnet_owned_layer_end = end;
+ params.use_mmap = use_mmap;
+ params.use_extra_bufts = use_extra_bufts;
+ params.progress_callback = nullptr;
+
+ llama_model * model = llama_model_load_from_file(model_path.c_str(), params);
+ if (model == nullptr) {
+ return fail(kExitLoad, "owned-range load rejected the artifact or range");
+ }
+
+ llama_meshnet_range_report report = {};
+ if (!llama_model_meshnet_range_report(model, &report)) {
+ llama_model_free(model);
+ return fail(kExitLoad, "loaded model carries no owned-range report");
+ }
+
+ char arch_buf[128] = {};
+ std::string arch;
+ if (llama_model_meta_val_str(model, "general.architecture", arch_buf, sizeof(arch_buf)) >= 0) {
+ arch = arch_buf;
+ }
+ const int n_layer = llama_model_n_layer(model);
+ const uint64_t bytes_on_disk = file_size(model_path);
+
+ // Audit the registered tensor set against the requested ownership.
+ const auto & tensors = llama_internal_get_tensor_map(model);
+ bool has_embd = false;
+ bool has_out_norm = false;
+ bool has_out = false;
+ std::set<int> owned_layers;
+ std::vector<std::string> unexpected;
+ uint64_t registered_bytes = 0;
+ for (const auto & entry : tensors) {
+ const std::string & name = entry.first;
+ registered_bytes += ggml_nbytes(entry.second);
+ if (name == "token_embd.weight") {
+ has_embd = true;
+ continue;
+ }
+ if (name == "output_norm.weight") {
+ has_out_norm = true;
+ continue;
+ }
+ if (name == "output.weight") {
+ has_out = true;
+ continue;
+ }
+ int block = -1;
+ if (std::sscanf(name.c_str(), "blk.%d.", &block) == 1 && block >= 0) {
+ owned_layers.insert(block);
+ continue;
+ }
+ unexpected.push_back(name);
+ }
+
+ // A tail shard whose model ties the output head to the token embedding
+ // registers token_embd.weight as its output head instead of output.weight.
+ const bool tied_tail = end == n_layer && has_embd && !has_out;
+ const bool expect_embd = start == 0 || tied_tail;
+
+ std::vector<int> missing_layers;
+ for (int i = start; i < end; ++i) {
+ if (!owned_layers.count(i)) {
+ missing_layers.push_back(i);
+ }
+ }
+ std::vector<int> outside_layers;
+ for (const int block : owned_layers) {
+ if (block < start || block >= end) {
+ outside_layers.push_back(block);
+ }
+ }
+
+ std::vector<std::string> mismatches;
+ if (report.start_layer != start || report.end_layer != end) {
+ mismatches.push_back("reported range differs from the requested range");
+ }
+ if (has_embd != expect_embd) {
+ mismatches.push_back("token-embedding registration disagrees with endpoint ownership");
+ }
+ if ((end == n_layer) && !has_out_norm) {
+ mismatches.push_back("tail range is missing the final norm");
+ }
+ if ((end == n_layer) && !has_out && !has_embd) {
+ mismatches.push_back("tail range is missing the output head");
+ }
+ if ((end != n_layer) && (has_out_norm || has_out)) {
+ mismatches.push_back("non-tail range registered tail-only tensors");
+ }
+ if (report.has_token_embeddings != has_embd) {
+ mismatches.push_back("reported embedding ownership disagrees with registered tensors");
+ }
+ if (report.has_output_head != (end == n_layer)) {
+ mismatches.push_back("reported output-head ownership disagrees with endpoint ownership");
+ }
+ if (!missing_layers.empty()) {
+ mismatches.push_back("owned range has missing per-layer tensors");
+ }
+ if (!outside_layers.empty()) {
+ mismatches.push_back("registered per-layer tensors lie outside the owned range");
+ }
+ if (!unexpected.empty()) {
+ mismatches.push_back("registered tensors outside the dense-Llama ownership vocabulary");
+ }
+ if (use_mmap && report.mapped_bytes < registered_bytes) {
+ mismatches.push_back("mapped span undercounts the registered tensors");
+ }
+ if (!use_mmap && report.resident_bytes < registered_bytes) {
+ mismatches.push_back("resident allocation undercounts the registered tensors");
+ }
+
+ if (touch) {
+ volatile uint64_t sink = 0;
+ for (const auto & entry : tensors) {
+ const auto * data = static_cast<const volatile uint8_t *>(entry.second->data);
+ const size_t nbytes = ggml_nbytes(entry.second);
+ for (size_t i = 0; i < nbytes; i += 4096) {
+ sink += data[i];
+ }
+ }
+ (void) sink;
+ }
+
+ const proc_status proc = read_proc_status();
+
+ if (!mismatches.empty()) {
+ llama_model_free(model);
+ return fail(kExitAudit, "ownership audit failed: " + json_string_array(mismatches));
+ }
+
+ std::printf(
+ "{\n"
+ " \"ok\": true,\n"
+ " \"model\": \"%s\",\n"
+ " \"architecture\": \"%s\",\n"
+ " \"n_layer\": %d,\n"
+ " \"file_bytes\": %llu,\n"
+ " \"requested_range\": [%d, %d],\n"
+ " \"reported_range\": [%d, %d],\n"
+ " \"mmap\": %s,\n"
+ " \"touched\": %s,\n"
+ " \"use_extra_bufts\": %s,\n"
+ " \"has_token_embeddings\": %s,\n"
+ " \"has_output_head\": %s,\n"
+ " \"tied_output_head\": %s,\n"
+ " \"mapped_bytes\": %llu,\n"
+ " \"resident_bytes\": %llu,\n"
+ " \"registered_tensors\": %d,\n"
+ " \"registered_bytes\": %llu,\n"
+ " \"unexpected_registered_tensors\": [],\n"
+ " \"missing_owned_layers\": [],\n"
+ " \"vm_size_bytes\": %llu,\n"
+ " \"vm_rss_bytes\": %llu,\n"
+ " \"vm_hwm_bytes\": %llu\n"
+ "}\n",
+ json_escape(model_path).c_str(),
+ json_escape(arch).c_str(),
+ n_layer,
+ (unsigned long long) bytes_on_disk,
+ start, end,
+ report.start_layer, report.end_layer,
+ use_mmap ? "true" : "false",
+ touch ? "true" : "false",
+ use_extra_bufts ? "true" : "false",
+ report.has_token_embeddings ? "true" : "false",
+ report.has_output_head ? "true" : "false",
+ tied_tail ? "true" : "false",
+ (unsigned long long) report.mapped_bytes,
+ (unsigned long long) report.resident_bytes,
+ (int) tensors.size(),
+ (unsigned long long) registered_bytes,
+ (unsigned long long) proc.vm_size,
+ (unsigned long long) proc.vm_rss,
+ (unsigned long long) proc.vm_hwm);
+
+ llama_model_free(model);
+ llama_backend_free();
+ return 0;
+}

1
packages/node/native/llama/patches/SHA256SUMS
View File

@@ -4,3 +4,4 @@
4871a37544df658980a01b4f94151a90b609fb144c931b4a814309ee608ebb46 0003-owned-range-filtered-state-report.patch
19d451ce259150ffede793c4eb547425375c0fcd97caf326b43e8f1a204f05b6 0004-dense-boundary-io-endpoint-guard.patch
cf263357a6a8de193f710836c7c467c38cac7099975303ee2628e0609daf5a47 0005-worker-range-report-hook.patch
23b4b8c56243d52ba682f0034022a86bf8ded007885be5b659cf5158ff3eb429 0006-meshnet-range-report-tool.patch

24
packages/node/native/llama/patches/UPSTREAM-ASSUMPTIONS.json
View File

@@ -112,6 +112,30 @@
"llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h",
"gguf empty-context writer API: gguf_init_empty, gguf_add_tensor, gguf_write_to_file"
]
},
"0006-meshnet-range-report-tool.patch": {
"concern": "range-reporting",
"files": {
"CMakeLists.txt": {
"before": "a9afcffa68bed7cbd8fad39ad9f95ad784251234",
"after": "868793b826f565df7f041e7ba55820b5ad744b10"
},
"tools/meshnet-range-report/CMakeLists.txt": {
"before": null,
"after": "24401007ee85e217c2741a42c7119fad323ff08a"
},
"tools/meshnet-range-report/meshnet-range-report.cpp": {
"before": null,
"after": "49a5eb2a05bf6514e166453ea0e35b8bc9c5fdf6"
}
},
"api_assumptions": [
"llama_model_params carries meshnet_owned_layer_start/end, use_mmap, and use_extra_bufts",
"llama_model_meshnet_range_report C API and llama_meshnet_range_report fields (patch 0005)",
"llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h",
"llama_model_meta_val_str and llama_model_n_layer public accessors",
"top-level CMakeLists add_subdirectory of a project-owned tool directory after the llama target"
]
}
}
}

1
packages/node/native/llama/patches/series
View File

@@ -3,3 +3,4 @@
0003-owned-range-filtered-state-report.patch
0004-dense-boundary-io-endpoint-guard.patch
0005-worker-range-report-hook.patch
0006-meshnet-range-report-tool.patch

275
tests/test_meshnet_range_report_tool.py Normal file
View File

@@ -0,0 +1,275 @@
"""DGR-034: end-to-end owned-range loads through the native report tool.
Gated on the built ``meshnet-range-report`` binary (the deterministic
CPU-only native lane builds it from the pinned, patched llama.cpp tree); in
an environment without that build these tests skip rather than fake a pass.
When the binary is present they run real loads of a tiny synthetic
dense-Llama GGUF — no model download, no GPU — and prove the loader
registers exactly the owned tensors, reports ownership derived from the
loaded state, and rejects invalid/out-of-model ranges and missing required
tensors. The JSON is consumed through ``meshnet_node.range_report`` so the
strict project-owned contract is exercised on real tool output.
"""
from __future__ import annotations
import json
import os
import struct
import subprocess
import sys
from pathlib import Path
import pytest
from meshnet_node.range_report import RangeReportError, parse_owned_range_report
REPO_ROOT = Path(__file__).resolve().parent.parent
DEFAULT_BINARY = REPO_ROOT / "build" / "llama.cpp" / "build" / "bin" / "meshnet-range-report"
BINARY = Path(os.environ.get("MESHNET_RANGE_REPORT_BIN", DEFAULT_BINARY))
requires_range_report_tool = pytest.mark.skipif(
not BINARY.is_file(),
reason=(
"meshnet-range-report is not built; run the deterministic native lane "
"(scripts/llama_cpp_dependency.py build) to enable these tests"
),
)
# --- Minimal GGUF v3 writer, mirroring the model-free native fixture --------
K_LAYERS = 4
K_EMBD = 8
K_FFN = 16
K_VOCAB = 16
ALIGNMENT = 32
_GGUF_UINT32 = 4
_GGUF_FLOAT32 = 6
_GGUF_STRING = 8
_GGML_TYPE_F32 = 0
def _gguf_string(value: str) -> bytes:
data = value.encode("utf-8")
return struct.pack("<Q", len(data)) + data
def _metadata_entries() -> list[tuple[str, int, object]]:
return [
("general.architecture", _GGUF_STRING, "llama"),
("general.alignment", _GGUF_UINT32, ALIGNMENT),
("llama.context_length", _GGUF_UINT32, 16),
("llama.embedding_length", _GGUF_UINT32, K_EMBD),
("llama.block_count", _GGUF_UINT32, K_LAYERS),
("llama.feed_forward_length", _GGUF_UINT32, K_FFN),
("llama.attention.head_count", _GGUF_UINT32, 2),
("llama.attention.head_count_kv", _GGUF_UINT32, 2),
("llama.rope.dimension_count", _GGUF_UINT32, 4),
("llama.attention.layer_norm_rms_epsilon", _GGUF_FLOAT32, 1.0e-5),
("tokenizer.ggml.model", _GGUF_STRING, "no_vocab"),
("llama.vocab_size", _GGUF_UINT32, K_VOCAB),
]
def _fixture_tensors() -> list[tuple[str, tuple[int, ...]]]:
tensors: list[tuple[str, tuple[int, ...]]] = [
("token_embd.weight", (K_EMBD, K_VOCAB)),
("output_norm.weight", (K_EMBD,)),
("output.weight", (K_EMBD, K_VOCAB)),
]
for layer in range(K_LAYERS):
prefix = f"blk.{layer}."
tensors += [
(prefix + "attn_norm.weight", (K_EMBD,)),
(prefix + "attn_q.weight", (K_EMBD, K_EMBD)),
(prefix + "attn_k.weight", (K_EMBD, K_EMBD)),
(prefix + "attn_v.weight", (K_EMBD, K_EMBD)),
(prefix + "attn_output.weight", (K_EMBD, K_EMBD)),
(prefix + "ffn_norm.weight", (K_EMBD,)),
(prefix + "ffn_gate.weight", (K_EMBD, K_FFN)),
(prefix + "ffn_down.weight", (K_FFN, K_EMBD)),
(prefix + "ffn_up.weight", (K_EMBD, K_FFN)),
]
return tensors
def write_dense_llama_gguf(path: Path, *, drop: frozenset[str] = frozenset()) -> Path:
"""Write a tiny dense-Llama GGUF; ``drop`` omits tensors (corruption cases)."""
kvs = _metadata_entries()
tensors = [(name, dims) for name, dims in _fixture_tensors() if name not in drop]
blob = bytearray()
blob += b"GGUF" + struct.pack("<IQQ", 3, len(tensors), len(kvs))
for key, vtype, value in kvs:
blob += _gguf_string(key)
blob += struct.pack("<I", vtype)
if vtype == _GGUF_STRING:
blob += _gguf_string(value) # type: ignore[arg-type]
elif vtype == _GGUF_UINT32:
blob += struct.pack("<I", value) # type: ignore[arg-type]
elif vtype == _GGUF_FLOAT32:
blob += struct.pack("<f", value) # type: ignore[arg-type]
else: # pragma: no cover - writer guard
raise AssertionError(f"unhandled kv type {vtype}")
offset = 0
infos = bytearray()
data = bytearray()
for name, dims in tensors:
infos += _gguf_string(name)
infos += struct.pack("<I", len(dims))
for dim in dims:
infos += struct.pack("<Q", dim)
infos += struct.pack("<IQ", _GGML_TYPE_F32, offset)
size = 4
for dim in dims:
size *= dim
assert size % ALIGNMENT == 0
data += bytes(size)
offset += size
blob += infos
blob += bytes(-len(blob) % ALIGNMENT) # pad header to the data section
blob += data
path.write_bytes(bytes(blob))
return path
# --- Tool driver -------------------------------------------------------------
LAYER_BYTES = 2624 # 9 registered F32 tensors per layer, see _fixture_tensors
EMBD_BYTES = 512
OUT_NORM_BYTES = 32
OUT_BYTES = 512
def run_tool(model: Path, start: int, end: int, *extra: str) -> tuple[int, dict]:
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = f"{BINARY.parent}:{env.get('LD_LIBRARY_PATH', '')}"
completed = subprocess.run(
[
str(BINARY),
"--model", str(model),
"--start", str(start),
"--end", str(end),
*extra,
],
capture_output=True,
text=True,
env=env,
timeout=120,
)
try:
doc = json.loads(completed.stdout)
except json.JSONDecodeError as exc: # pragma: no cover - diagnostic path
raise AssertionError(
f"tool did not print a JSON report (exit {completed.returncode}): "
f"{completed.stdout!r} {completed.stderr!r}"
) from exc
return completed.returncode, doc
@pytest.fixture(scope="module")
def dense_llama_gguf(tmp_path_factory: pytest.TempPathFactory) -> Path:
return write_dense_llama_gguf(tmp_path_factory.mktemp("gguf") / "dense-llama.gguf")
@requires_range_report_tool
class TestOwnedRangeLoads:
def test_middle_range_registers_exactly_its_layers(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 1, 3, "--no-extra-bufts")
assert code == 0
report = parse_owned_range_report(doc)
assert (report.start_layer, report.end_layer) == (1, 3)
assert report.registered_tensors == 18
assert report.registered_bytes == 2 * LAYER_BYTES
# The fixture layers are contiguous in the file, so the pure mmap span
# is exactly the owned tensor bytes — scaled down from the artifact.
assert report.mapped_bytes == 2 * LAYER_BYTES
assert report.mapped_bytes < report.file_bytes
def test_head_range_owns_embeddings(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 0, 1)
assert code == 0
report = parse_owned_range_report(doc)
assert report.is_head and report.has_token_embeddings
assert not report.has_output_head
assert report.registered_tensors == 10
assert report.registered_bytes == EMBD_BYTES + LAYER_BYTES
def test_tail_range_owns_norm_and_output(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 3, 4)
assert code == 0
report = parse_owned_range_report(doc)
assert report.is_tail and report.has_output_head
assert not report.has_token_embeddings
assert report.registered_tensors == 11
assert report.registered_bytes == LAYER_BYTES + OUT_NORM_BYTES + OUT_BYTES
def test_shards_partition_the_whole_model_bytes(self, dense_llama_gguf: Path) -> None:
shards = [(0, 1), (1, 3), (3, 4)]
registered = []
for start, end in shards:
code, doc = run_tool(dense_llama_gguf, start, end)
assert code == 0
registered.append(parse_owned_range_report(doc).registered_bytes)
code, doc = run_tool(dense_llama_gguf, 0, 4)
assert code == 0
whole = parse_owned_range_report(doc)
assert whole.registered_tensors == 3 + 9 * K_LAYERS
assert sum(registered) == whole.registered_bytes
def test_non_mmap_load_scales_resident_with_the_range(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 1, 3, "--no-mmap")
assert code == 0
report = parse_owned_range_report(doc)
assert report.mapped_bytes == 0
assert report.registered_bytes == 2 * LAYER_BYTES
code, doc = run_tool(dense_llama_gguf, 0, 4, "--no-mmap")
assert code == 0
whole = parse_owned_range_report(doc)
assert report.resident_bytes < whole.resident_bytes
@requires_range_report_tool
class TestRangeRejection:
def test_out_of_model_range_is_refused(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 3, 5)
assert code == 3 and doc["ok"] is False
with pytest.raises(RangeReportError):
parse_owned_range_report(doc)
def test_empty_range_is_refused(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 2, 2)
assert code == 3 and doc["ok"] is False
def test_inverted_range_is_refused(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 3, 1)
assert code == 3 and doc["ok"] is False
def test_missing_required_owned_tensor_is_refused(self, tmp_path: Path) -> None:
corrupted = write_dense_llama_gguf(
tmp_path / "missing-tensor.gguf", drop=frozenset({"blk.1.attn_q.weight"})
)
code, doc = run_tool(corrupted, 0, 2)
assert code == 3 and doc["ok"] is False
assert "blk.1.attn_q.weight" in doc["error"]
def test_whole_model_load_still_works_through_the_range_loader(
self, dense_llama_gguf: Path
) -> None:
code, doc = run_tool(dense_llama_gguf, 0, 4)
assert code == 0
report = parse_owned_range_report(doc)
assert report.is_head and report.is_tail
assert report.has_token_embeddings and report.has_output_head
def test_tool_binary_gate_points_at_the_locked_build() -> None:
# The gate must name the deterministic lane's output, never a downloaded binary.
assert DEFAULT_BINARY.name == "meshnet-range-report"
assert "llama.cpp" in DEFAULT_BINARY.parts
assert DEFAULT_BINARY.parent.name == "bin"
assert DEFAULT_BINARY.parent.parent.name == "build"

273
tests/test_range_report.py Normal file
View File

@@ -0,0 +1,273 @@
"""DGR-034: strict consumption of owned-range reports from loaded engine state.
The ``meshnet-range-report`` native tool loads one dense-Llama GGUF through
the Meshnet owned-range loader and prints a JSON document derived from the
loaded model state. ``meshnet_node.range_report`` is the strict consumer:
it must accept exactly the documents that encode the dense-Llama ownership
contract and fail closed on everything else — invalid, empty, or
out-of-model ranges, endpoint registrations that disagree with the loaded
state, gapped or unexpected tensor registrations, and inconsistent byte
counts.
"""
from __future__ import annotations
from typing import Any
import pytest
from meshnet_node.range_report import (
OwnedRangeReport,
RangeReportError,
parse_owned_range_report,
)
N_LAYER = 40
LAYER_BYTES = 300 * 2**20
EMBD_BYTES = 360 * 2**20
OUT_BYTES = 525 * 2**20
FILE_BYTES = 13669 * 2**20
def _doc(**overrides: Any) -> dict[str, Any]:
"""A valid middle-range [10, 20) mmap report the consumer must accept."""
doc: dict[str, Any] = {
"ok": True,
"model": "/models/dense.gguf",
"architecture": "llama",
"n_layer": N_LAYER,
"file_bytes": FILE_BYTES,
"requested_range": [10, 20],
"reported_range": [10, 20],
"mmap": True,
"touched": False,
"use_extra_bufts": True,
"has_token_embeddings": False,
"has_output_head": False,
"tied_output_head": False,
"mapped_bytes": 10 * LAYER_BYTES,
"resident_bytes": 10 * LAYER_BYTES,
"registered_tensors": 90,
"registered_bytes": 10 * LAYER_BYTES,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": FILE_BYTES + 2**28,
"vm_rss_bytes": 2**28,
"vm_hwm_bytes": 2**28,
}
doc.update(overrides)
return doc
def _head_doc(**overrides: Any) -> dict[str, Any]:
base = _doc(
requested_range=[0, 10],
reported_range=[0, 10],
has_token_embeddings=True,
mapped_bytes=10 * LAYER_BYTES + EMBD_BYTES,
resident_bytes=10 * LAYER_BYTES + EMBD_BYTES,
registered_tensors=91,
registered_bytes=10 * LAYER_BYTES + EMBD_BYTES,
)
base.update(overrides)
return base
def _tail_doc(**overrides: Any) -> dict[str, Any]:
base = _doc(
requested_range=[30, 40],
reported_range=[30, 40],
has_output_head=True,
mapped_bytes=10 * LAYER_BYTES + OUT_BYTES,
resident_bytes=10 * LAYER_BYTES + OUT_BYTES,
registered_tensors=92,
registered_bytes=10 * LAYER_BYTES + OUT_BYTES,
)
base.update(overrides)
return base
class TestAcceptance:
def test_middle_range_registers_only_per_layer_tensors(self) -> None:
report = parse_owned_range_report(_doc())
assert (report.start_layer, report.end_layer) == (10, 20)
assert not report.is_head and not report.is_tail
assert not report.has_token_embeddings and not report.has_output_head
def test_head_range_owns_embeddings_only_at_the_head(self) -> None:
report = parse_owned_range_report(_head_doc())
assert report.is_head and not report.is_tail
assert report.has_token_embeddings and not report.has_output_head
def test_tail_range_owns_norm_and_output_only_at_the_tail(self) -> None:
report = parse_owned_range_report(_tail_doc())
assert report.is_tail and not report.is_head
assert report.has_output_head and not report.has_token_embeddings
def test_whole_model_range_owns_both_endpoints(self) -> None:
report = parse_owned_range_report(
_head_doc(
requested_range=[0, 40],
reported_range=[0, 40],
has_output_head=True,
mapped_bytes=FILE_BYTES,
resident_bytes=FILE_BYTES,
registered_tensors=363,
registered_bytes=N_LAYER * LAYER_BYTES + EMBD_BYTES + OUT_BYTES,
)
)
assert report.is_head and report.is_tail
assert report.has_token_embeddings and report.has_output_head
def test_tied_output_tail_registers_the_embedding_as_its_output_head(self) -> None:
report = parse_owned_range_report(
_tail_doc(
has_token_embeddings=True,
tied_output_head=True,
registered_tensors=91,
registered_bytes=10 * LAYER_BYTES + EMBD_BYTES,
mapped_bytes=10 * LAYER_BYTES + EMBD_BYTES,
resident_bytes=10 * LAYER_BYTES + EMBD_BYTES,
)
)
assert report.tied_output_head and report.has_output_head
def test_non_mmap_load_reports_resident_allocation_only(self) -> None:
report = parse_owned_range_report(
_doc(mmap=False, mapped_bytes=0, resident_bytes=10 * LAYER_BYTES)
)
assert report.mapped_bytes == 0
assert report.resident_bytes == 10 * LAYER_BYTES
def test_process_counters_may_be_absent_off_linux(self) -> None:
report = parse_owned_range_report(
_doc(vm_size_bytes=None, vm_rss_bytes=None, vm_hwm_bytes=None)
)
assert report.vm_hwm_bytes is None
class TestRangeRejection:
def test_rejected_load_fails_closed_with_the_tool_error(self) -> None:
with pytest.raises(RangeReportError, match="dense Llama only"):
parse_owned_range_report(
{"ok": False, "error": "owned-range load rejected the artifact or range: dense Llama only"}
)
def test_reported_range_must_match_the_requested_range(self) -> None:
with pytest.raises(RangeReportError, match="loaded engine state"):
parse_owned_range_report(_doc(reported_range=[10, 21]))
def test_out_of_model_range_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="outside the model"):
parse_owned_range_report(
_doc(requested_range=[30, 41], reported_range=[30, 41], has_output_head=True)
)
def test_empty_range_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="empty or"):
parse_owned_range_report(_doc(requested_range=[10, 10], reported_range=[10, 10]))
def test_inverted_range_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="empty or"):
parse_owned_range_report(_doc(requested_range=[20, 10], reported_range=[20, 10]))
def test_boolean_range_bounds_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="integer pair"):
parse_owned_range_report(_doc(reported_range=[True, 20]))
class TestEndpointRejection:
def test_embeddings_registered_below_the_head_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="embeddings belong to the head"):
parse_owned_range_report(_doc(has_token_embeddings=True))
def test_output_head_registered_above_the_tail_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="output head belong to the tail"):
parse_owned_range_report(_tail_doc(requested_range=[20, 30], reported_range=[20, 30]))
def test_tail_without_an_output_head_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="output head belong to the tail"):
parse_owned_range_report(_tail_doc(has_output_head=False))
def test_tied_output_below_the_tail_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="only belong to the tail"):
parse_owned_range_report(_doc(tied_output_head=True))
def test_unexpected_registered_tensors_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="unexpected_registered_tensors"):
parse_owned_range_report(
_doc(unexpected_registered_tensors=["blk.10.attn_q.weight.extra"])
)
def test_missing_owned_layers_are_rejected_as_gaps(self) -> None:
with pytest.raises(RangeReportError, match="missing_owned_layers"):
parse_owned_range_report(_doc(missing_owned_layers=[12]))
class TestByteCountRejection:
def test_mapped_span_must_cover_the_registered_tensors(self) -> None:
with pytest.raises(RangeReportError, match="undercounts"):
parse_owned_range_report(_doc(mapped_bytes=LAYER_BYTES))
def test_mapped_span_must_not_exceed_the_artifact(self) -> None:
with pytest.raises(RangeReportError, match="exceeds the artifact"):
parse_owned_range_report(
_tail_doc(mapped_bytes=FILE_BYTES + 1, resident_bytes=FILE_BYTES + 1)
)
def test_non_mmap_load_must_not_claim_a_mapped_span(self) -> None:
with pytest.raises(RangeReportError, match="must not claim"):
parse_owned_range_report(_doc(mmap=False, mapped_bytes=LAYER_BYTES))
def test_resident_allocation_must_cover_the_registered_tensors(self) -> None:
with pytest.raises(RangeReportError, match="undercounts"):
parse_owned_range_report(
_doc(mmap=False, mapped_bytes=0, resident_bytes=LAYER_BYTES)
)
def test_an_empty_registration_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="no tensors"):
parse_owned_range_report(_doc(registered_tensors=0, registered_bytes=0))
class TestSchemaRejection:
def test_wrong_architecture_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="dense Llama only"):
parse_owned_range_report(_doc(architecture="qwen2"))
def test_missing_field_is_rejected(self) -> None:
doc = _doc()
del doc["mapped_bytes"]
with pytest.raises(RangeReportError, match="missing field"):
parse_owned_range_report(doc)
def test_boolean_bytes_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="non-negative integer"):
parse_owned_range_report(_doc(mapped_bytes=True))
def test_non_mapping_document_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="JSON object"):
parse_owned_range_report(["not", "a", "report"]) # type: ignore[arg-type]
def test_owned_range_report_rejects_direct_construction_outside_the_contract() -> None:
with pytest.raises(RangeReportError, match="dense Llama only"):
OwnedRangeReport(
architecture="qwen2",
n_layer=N_LAYER,
start_layer=10,
end_layer=20,
has_token_embeddings=False,
has_output_head=False,
tied_output_head=False,
mapped_bytes=10 * LAYER_BYTES,
resident_bytes=10 * LAYER_BYTES,
registered_tensors=90,
registered_bytes=10 * LAYER_BYTES,
file_bytes=FILE_BYTES,
mmap=True,
touched=False,
vm_size_bytes=None,
vm_rss_bytes=None,
vm_hwm_bytes=None,
)