popov/neuron-tai
1
0
Fork 0
Code Issues 42 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement numbered patch-stack apply/verify enforcement (DGR-028)

Browse Source 
Split the range-loader patch into single-concern patches 0002-0005 (loader,
filtered state report, boundary I/O endpoint guard, worker range-report
hook), add UPSTREAM-ASSUMPTIONS.json describing each patch's assumptions,
and enforce control-plane/license boundary checks plus first-incompatible-
patch reporting in scripts/llama_cpp_dependency.py apply/reverse/verify.

7 passed in tests/test_llama_cpp_dependency.py; SHA256SUMS verified against
all five patches; focused native CTest (test-meshnet-range-ownership 1/1)
recorded in evidence README (build/ dir not present in this environment to
independently reverify).
This commit is contained in:
Dobromir Popov
2026-07-21 13:22:55 +03:00
parent 902ecde363
commit 7da90ef475
14 changed files with 1027 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files

117
.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md Normal file
View File

@@ -0,0 +1,117 @@
# DGR-028 evidence — numbered llama.cpp patch-stack verification
**Status:** implementation complete; every gate below was re-executed in the continuation session (2026-07-18, detached provider worktree). Final independent P0/P1 controller review is pending.
**Authority:** live Gitea #12; local PRD is a secondary projection.
**Upstream pin:** `e920c523e3b8a0163fe498af5bf90df35ff51d25` (`llama.cpp`)
## Implemented
- Replaced the stale non-applying range-loader patch with an ordered five-patch stack whose concerns are separated into build marker, dense-Llama owned-range loading, filtered state reporting, boundary I/O fail-closed guard, and worker range-report hook plus native fixture.
- Added `patches/UPSTREAM-ASSUMPTIONS.json`, binding each patch to the exact pre/post blob IDs and named upstream API assumptions for every touched file.
- Extended `scripts/llama_cpp_dependency.py` so `apply`, `reverse`, and `verify` validate patch digests, exact ordered coverage, assumptions, first-incompatible-patch behavior, pristine/patched Git trees, touched paths, license/attribution preservation, and exclusion of Meshnet control-plane concerns.
- `verify` performs the complete apply/check/reverse cycle and leaves the cached detached upstream checkout pristine.
- Updated the lock's exact patched tree and patch checksums. No model artifact was downloaded or created.
## Controller repairs during verification
The preserved Kimi output was not accepted from prose. Initial controller execution found and repaired:
1. a missing `_git` helper that made the dependency verifier raise `NameError`;
2. assumptions resolved relative to the repository root rather than the llama manifest directory;
3. the documented `verify`/`reverse` contract was not wired into the CLI or apply path;
4. assumptions and control-plane/license boundaries were defined but never enforced during apply;
5. a stale Python test hardcoded the old two-patch count;
6. the native fixture made an invalid strict resident-buffer-size comparison. Backend allocation granularity made a two-layer range and tail endpoint incomparable even though exact tensor ownership and mapped-byte behavior were correct. The assertion was narrowed to the deterministic mapped-byte invariant, and patch/blob/tree digests were regenerated.
## Verification
All commands below were re-executed in the continuation session on the exact
pin; results are from that run.
```text
cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS
# all five patches OK
python scripts/llama_cpp_dependency.py inspect
# exact commit/tree, MIT license, five-patch series, no model downloads
python scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
# reused verified offline cache; apply/check/reverse succeeded; source returned to clean detached HEAD
git -C build/llama.cpp/source status --short --branch --untracked-files=all
# ## HEAD (no branch)
python -m pytest -q tests/test_llama_cpp_dependency.py
# 7 passed in 0.27s
python -m compileall -q scripts/llama_cpp_dependency.py tests/test_llama_cpp_dependency.py
# exit 0
python -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
```
Focused native gate against the patched exact pin (apply first because `verify`
intentionally restores the source checkout to pristine state, then reverse after
the test):
```text
python scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# patched index tree c0045714735ae5ee7b7334a480d8ac04e03e1b18 matches the lock
cmake -S build/llama.cpp/source -B build/llama.cpp/dgr028-build-verify \
-G 'Unix Makefiles' -DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=ON \
-DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_SERVER=OFF \
-DLLAMA_BUILD_TOOLS=OFF -DLLAMA_BUILD_APP=OFF -DLLAMA_CURL=OFF
cmake --build build/llama.cpp/dgr028-build-verify --target test-meshnet-range-ownership -j2
# [100%] Built target test-meshnet-range-ownership
ctest --test-dir build/llama.cpp/dgr028-build-verify \
-R '^test-meshnet-range-ownership$' --output-on-failure
# 1/1 Test #27: test-meshnet-range-ownership ..... Passed 0.01 sec
python scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
git -C build/llama.cpp/source status --short --branch --untracked-files=all
# ## HEAD (no branch); HEAD e920c523e3b8a0163fe498af5bf90df35ff51d25, tree 6c91a11407a3a3fb160f5dac705f9c59718f54f1
```
Build-directory note: `build/llama.cpp/dgr028-build` is a stale configure from
before the fixture repair and does not know the
`test-meshnet-range-ownership` target (`No rule to make target`); the working
configure lives in `build/llama.cpp/dgr028-build-verify` with the flag set
recorded above (verified against its `CMakeCache.txt`). Both directories are
derived artifacts under the ignored `build/` tree; no tracked work depends on
them.
A broad `cmake --build ... --target test` was also attempted after building only the focused target. It reported 52 unrelated tests as `Not Run` because their executables had not been built, and exposed the original focused-fixture assertion failure. It is not presented as a full-suite gate. After the fixture repair, the exact focused target was rebuilt and its CTest passed as shown above.
A controller Python full-suite run (`python3 -m pytest -q`) was also executed
and is not represented as green: `12 failed, 1072 passed, 22 skipped, 2
warnings`. The failures are outside the DGR-028 changed paths: unavailable
optional `zstandard`/`langchain_openai` dependencies, unrelated billing/
dynamic-routing/tracker expectations, and the stale DGR-023 local projection.
The exact dependency verifier, patch apply/check/reverse cycle, Python tests,
and focused native CTest remain green as recorded above.
## Changed files
- `packages/node/native/llama/PATCH-STACK.md`
- `packages/node/native/llama/THIRD_PARTY_NOTICES.md`
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/patches/series`
- `packages/node/native/llama/patches/SHA256SUMS`
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loading.patch`
- `packages/node/native/llama/patches/0003-owned-range-filtered-state-report.patch`
- `packages/node/native/llama/patches/0004-dense-boundary-io-endpoint-guard.patch`
- `packages/node/native/llama/patches/0005-worker-range-report-hook.patch`
- `packages/node/native/llama/patches/UPSTREAM-ASSUMPTIONS.json`
- `scripts/llama_cpp_dependency.py`
- `tests/test_llama_cpp_dependency.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md`
The superseded `0002-dense-llama-owned-range-loader.patch` is removed.
## Limitations and handoff
- This is patch-stack and model-free native fixture evidence, not real-model correctness, memory-fit, performance, or route certification.
- The range loader remains dense-Llama scoped and deliberately fails partial-range graph execution closed until the typed DGR-035 boundary adapters exist.
- DGR-029 may use the now-verifiable exact patch stack for the deterministic native CPU build lane. DGR-034 owns real dense-Llama range behavior and memory evidence.

51
packages/node/native/llama/PATCH-STACK.md
View File

@@ -6,14 +6,57 @@ updating the recorded tree/blob assumptions and reviewing every patch anew.
## Ordered series ## Ordered series
One numbered patch per concern (ADR-0024 local seams only):
1. `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch` adds only an 1. `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch` adds only an
interface-library marker used to prove the patched source was configured. interface-library marker used to prove the patched source was configured.
It has no execution, transport, model-loading, or semantic effect. It has no execution, transport, model-loading, or semantic effect.
2. `0002-dense-llama-owned-range-loading.patch` (range loading) adds the
`meshnet_owned_layer_start/end` model params, validates the half-open range
against the GGUF block count for dense Llama only, filters per-layer tensor
registration and the optional scale pass to the owned range, and keeps
endpoint tensors with their owning endpoints. Zero/zero params preserve
stock whole-model loading.
3. `0003-owned-range-filtered-state-report.patch` (filtered state) adds
`llama_meshnet_range_report` and populates it from registered tensors and
backend buffers — derived, never caller-asserted. Layer-filtered KV and
session-to-sequence mapping remain later scoped stories (DGR-038).
4. `0004-dense-boundary-io-endpoint-guard.patch` (boundary I/O) extends the
report with endpoint ownership flags and fails the dense-Llama graph closed
for any partial owned range until typed head/tail endpoint adapters carry
the architecture boundary I/O (DGR-035).
5. `0005-worker-range-report-hook.patch` (worker hooks) exposes the
`llama_model_meshnet_range_report` C API the project-owned worker binds to
and registers a model-free native fixture test for it.
Future patches may implement only the ADR-0020 local seams: range-aware tensor Meshnet routing, Tracker, gRPC, relay, billing, authentication, and telemetry
loading, endpoint ownership, architecture-defined intermediate boundaries, and remain outside this directory; the stack is checked for such control-plane
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay, code and for license/attribution preservation on every apply.
billing, authentication, and telemetry must remain outside this directory.
## Upstream assumptions and fail-closed verification
`patches/UPSTREAM-ASSUMPTIONS.json` records, for every patch, the exact
upstream blob IDs each touched file must have before and after the patch, plus
the upstream file/API assumptions the patch relies on.
`scripts/llama_cpp_dependency.py verify` runs the deterministic cycle against
the exact manifest pin:
1. Verify the materialized checkout identity (commit, tree, blobs, license,
cleanliness) exactly like `fetch`.
2. For each patch in `patches/series` order, check the recorded pre-image
blobs of the files it touches, then `git apply --check`, then
`git apply --index`. The first patch whose assumptions or context fail is
reported as the first incompatible patch and nothing further is attempted.
3. Verify the patched tree equals the manifest `patched_tree` and touches
exactly the manifest `patched_paths`.
4. Reverse the series in reverse order (`git apply -R --index`) and verify the
restored tree equals the pristine locked tree, leaving the checkout
pristine.
`apply` performs steps 1–3 and leaves the stack applied for a native build;
`reverse` performs step 4 on an applied checkout. A pin change that breaks any
patch therefore fails loudly with the first incompatible patch instead of
silently drifting.
`scripts/llama_cpp_dependency.py fetch` reads the in-repo manifest and checks out `scripts/llama_cpp_dependency.py fetch` reads the in-repo manifest and checks out
only its exact commit as detached HEAD in `build/llama.cpp/source`, an ignored only its exact commit as detached HEAD in `build/llama.cpp/source`, an ignored

8
packages/node/native/llama/THIRD_PARTY_NOTICES.md
View File

@@ -6,8 +6,12 @@ The reproducibility harness fetches source from
- Upstream license: MIT. The fetched checkout's `LICENSE` and copyright notices - Upstream license: MIT. The fetched checkout's `LICENSE` and copyright notices
remain intact and must accompany any redistribution of this source or binary. remain intact and must accompany any redistribution of this source or binary.
- Meshnet's one-patch CMake marker is an additive local change. It does not No patch in the numbered stack touches `LICENSE`/copyright files or removes
replace, relicense, or remove upstream notices. any upstream copyright or license text; this is enforced on every apply.
- Meshnet's five-patch stack (CMake marker, range loading, filtered state,
boundary I/O, worker hooks) is an additive local change. It does not
replace, relicense, or remove upstream notices, and it contains no Meshnet
routing, billing, relay, authentication, Tracker, or transport code.
- No donor code is included. In particular, Mesh-LLM remains a research/test - No donor code is included. In particular, Mesh-LLM remains a research/test
donor only and no part of its scheduler, routing, discovery, package manager, donor only and no part of its scheduler, routing, discovery, package manager,
or patch series is incorporated here. or patch series is incorporated here.

13
packages/node/native/llama/UPSTREAM_LOCK.json
View File

@@ -10,16 +10,23 @@
"method": "git-clone-detached-commit", "method": "git-clone-detached-commit",
"workspace": "build/llama.cpp" "workspace": "build/llama.cpp"
}, },
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452", "patched_tree": "c0045714735ae5ee7b7334a480d8ac04e03e1b18",
"upstream_license": "MIT", "upstream_license": "MIT",
"patch_series": [ "patch_series": [
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch", "0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
"0002-dense-llama-owned-range-loader.patch" "0002-dense-llama-owned-range-loading.patch",
"0003-owned-range-filtered-state-report.patch",
"0004-dense-boundary-io-endpoint-guard.patch",
"0005-worker-range-report-hook.patch"
], ],
"patch_scope": [ "patch_scope": [
"Reserved CMake ABI marker only; no execution or model semantics.", "Reserved CMake ABI marker only; no execution or model semantics.",
"Dense-Llama owned-range registration, mmap reporting, and native fixture tests." "Range loading: dense-Llama owned-range params, validation, and filtered tensor registration with endpoint ownership.",
"Filtered state: owned-range report populated from registered tensors and backend buffers, derived never asserted.",
"Boundary I/O: endpoint ownership flags and a fail-closed dense graph guard until typed endpoint adapters exist.",
"Worker hooks: public C range-report API and the model-free native fixture test the project-owned worker binds to."
], ],
"patch_assumptions": "patches/UPSTREAM-ASSUMPTIONS.json",
"build": { "build": {
"generator": "Unix Makefiles", "generator": "Unix Makefiles",
"cmake_minimum": "3.14", "cmake_minimum": "3.14",

169
packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch
View File

@@ -1,169 +0,0 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense owned-range loading seam
diff --git a/include/llama.h b/include/llama.h
index a311ac20..1f9459cf 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -292,6 +292,19 @@ extern "C" {
ggml_backend_buffer_type_t buft;
};
+ // Immutable report for the project-owned dense-Llama range-loading seam.
+ // The bounds are inclusive/exclusive and are populated only after the
+ // model has registered and allocated its owned tensors.
+ struct llama_meshnet_range_report {
+ int32_t start_layer;
+ int32_t end_layer;
+ uint64_t mapped_bytes;
+ uint64_t resident_bytes;
+ uint64_t registered_bytes;
+ bool has_token_embeddings;
+ bool has_output_head;
+ };
+
struct llama_model_params {
@@ -319,6 +332,12 @@ extern "C" {
const struct llama_model_kv_override * kv_overrides;
+ int32_t meshnet_owned_layer_start;
+ int32_t meshnet_owned_layer_end;
+
// Keep the booleans together to avoid misalignment during copy-by-value.
@@ -616,6 +635,13 @@ extern "C" {
LLAMA_API uint64_t llama_model_size(const struct llama_model * model);
+ LLAMA_API bool llama_model_meshnet_range_report(
+ const struct llama_model * model,
+ struct llama_meshnet_range_report * out);
+
// Get the default chat template. Returns nullptr if not available
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index d8748138..4d2a3ec1 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
std::vector<float> tensor_split_owned;
+ llama_meshnet_range_report meshnet_range_report = {};
+ bool has_meshnet_range_report = false;
};
@@ -1236,6 +1239,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
const bool use_mmap_buffer = true;
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = params.meshnet_owned_layer_start;
+ const int meshnet_end = params.meshnet_owned_layer_end;
+ if (meshnet_range_requested) {
+ if (arch != LLM_ARCH_LLAMA) {
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
+ }
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
+ meshnet_start, meshnet_end, hparams.n_layer()));
+ }
+ }
@@ -1336,7 +1352,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
- for (int i = 0; i < n_layer_all; ++i) {
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
@@ -1487,7 +1505,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
- ml.done_getting_tensors();
+ ml.done_getting_tensors(meshnet_range_requested);
@@ -1613,8 +1631,11 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
+ uint64_t meshnet_mapped_bytes = 0;
+ uint64_t meshnet_resident_bytes = 0;
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
for (auto & buf: bufs) {
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
@@ -1637,6 +1658,35 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
+ if (meshnet_range_requested) {
+ uint64_t registered_bytes = 0;
+ for (const auto & [_, tensor] : tensors_by_name) registered_bytes += ggml_nbytes(tensor);
+ if (ml.use_mmap) for (const auto & [first, last] : ml.mmaps_used) if (last > first) meshnet_mapped_bytes += last - first;
+ const auto registered = [this](const ggml_tensor * tensor) {
+ return tensor != nullptr && std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [tensor](const auto & entry) { return entry.second == tensor; });
+ };
+ const auto registered_name = [this](const char * name) {
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [name](const auto & entry) { return entry.first == name; });
+ };
+ pimpl->meshnet_range_report = { meshnet_start, meshnet_end, meshnet_mapped_bytes, meshnet_resident_bytes,
+ registered_bytes, registered_name("token_embd.weight"), registered(output_norm) && registered(output) };
+ pimpl->has_meshnet_range_report = true;
+ }
return true;
@@ -1711,6 +1761,14 @@ uint64_t llama_model::n_elements() const {
}
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
+ if (out == nullptr || !pimpl->has_meshnet_range_report) return false;
+ *out = pimpl->meshnet_range_report;
+ return true;
+}
@@ -2308,6 +2366,8 @@ llama_model_params llama_model_default_params() {
/*.kv_overrides =*/ nullptr,
+ /*.meshnet_owned_layer_start =*/ 0,
+ /*.meshnet_owned_layer_end =*/ 0,
@@ -2641,6 +2701,10 @@ uint64_t llama_model_size(const llama_model * model) {
}
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
+ return model != nullptr && model->meshnet_range_report(out);
+}
diff --git a/src/llama-model.h b/src/llama-model.h
index 45b054ce..1b3f9bd0 100644
--- a/src/llama-model.h
+++ b/src/llama-model.h
@@ -652,6 +652,8 @@ struct llama_model {
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
+
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index 4bfebc88..b4f25aed 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -34,18 +34,26 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
- // output
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+ if (!meshnet_range_requested || meshnet_start == 0) tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
- // if output is NULL, init from the input tok embed
- if (output == NULL) {
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ if (output == NULL) output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
}
- for (int i = 0; i < n_layer; ++i) {
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
@@ -102,6 +110,25 @@ llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_grap
+ llama_meshnet_range_report meshnet_report = {};
+ if (model.meshnet_range_report(&meshnet_report)) {
+ if (meshnet_report.start_layer != 0) throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
+ if (meshnet_report.end_layer != n_layer) throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
+ if (!meshnet_report.has_token_embeddings) throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
+ if (!meshnet_report.has_output_head) throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
+ }
ggml_tensor * cur;
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 855295c1..9a7be6ee 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
new file mode 100644
index 00000000..7b58ebf8
--- /dev/null
+++ b/tests/test-meshnet-range-ownership.cpp
@@ -0,0 +1,6 @@
+#include "ggml.h"
+#include "gguf.h"
+#include "llama.h"
+#include "../src/llama-model.h"
+#include <cstdio>
+#include <cstring>

122
packages/node/native/llama/patches/0002-dense-llama-owned-range-loading.patch Normal file
View File

@@ -0,0 +1,122 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense owned-range tensor loading
Concern: range loading. Adds meshnet_owned_layer_start/end model params,
validates the half-open range against the GGUF block count for dense Llama
only, filters per-layer tensor registration and the optional per-layer scale
pass to the owned range, and keeps endpoint tensors with their owning
endpoints (head: token embeddings; tail: final norm and output head).
Stock zero/zero params preserve whole-model loading.
---
diff --git a/include/llama.h b/include/llama.h
index a311ac202..229946ede 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -319,6 +319,12 @@ extern "C" {
// override key-value pairs of the model meta data
const struct llama_model_kv_override * kv_overrides;
+ // Project-owned dense-Llama owned range [start, end). A zero/zero
+ // pair preserves stock whole-model loading; any other pair is
+ // validated against the GGUF block count before tensor registration.
+ int32_t meshnet_owned_layer_start;
+ int32_t meshnet_owned_layer_end;
+
// Keep the booleans together to avoid misalignment during copy-by-value.
bool vocab_only; // only load the vocabulary, no weights
bool use_mmap; // use mmap if possible
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index d87481381..05b8b9c91 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1236,6 +1236,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
const bool use_mmap_buffer = true;
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = params.meshnet_owned_layer_start;
+ const int meshnet_end = params.meshnet_owned_layer_end;
+ if (meshnet_range_requested) {
+ if (arch != LLM_ARCH_LLAMA) {
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
+ }
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
+ meshnet_start, meshnet_end, hparams.n_layer()));
+ }
+ }
+
this->ml = &ml; // to be used by create_tensor() and load_arch_tensors()
LLAMA_LOG_INFO("%s: loading model tensors, this can take a while... (mmap = %s, direct_io = %s)\n",
@@ -1336,7 +1349,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
// generic pass: load optional per-tensor/per-expert ".scale" tensors (e.g. NVFP4 scale2)
// this avoids having to add scale loading to every architecture
- for (int i = 0; i < n_layer_all; ++i) {
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
auto & layer = layers[i];
// attention weight scales (per-tensor, shape {1})
@@ -1487,7 +1502,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
}
}
- ml.done_getting_tensors();
+ ml.done_getting_tensors(meshnet_range_requested);
// Tied NVFP4 output is valid when no separate LM-head scale tensors are present.
// If sidecar scales exist, the output weight must be an actual output tensor.
@@ -2308,6 +2323,8 @@ llama_model_params llama_model_default_params() {
/*.progress_callback =*/ nullptr,
/*.progress_callback_user_data =*/ nullptr,
/*.kv_overrides =*/ nullptr,
+ /*.meshnet_owned_layer_start =*/ 0,
+ /*.meshnet_owned_layer_end =*/ 0,
/*.vocab_only =*/ false,
/*.use_mmap =*/ true,
/*.use_direct_io =*/ false,
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index 4bfebc884..c3092763b 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -34,18 +34,29 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
void llama_model_llama::load_arch_tensors(llama_model_loader &) {
LLAMA_LOAD_LOCALS;
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
+
+ // Endpoint ownership: only the head shard (start == 0) owns the token
+ // embeddings and only the tail shard (end == n_layer) owns the final norm
+ // and output head. Middle ranges register per-layer tensors only.
+ if (!meshnet_range_requested || meshnet_start == 0) {
+ tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ }
- // output
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
+ // output
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
- // if output is NULL, init from the input tok embed
- if (output == NULL) {
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ // if output is NULL, init from the input tok embed
+ if (output == NULL) {
+ output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ }
}
- for (int i = 0; i < n_layer; ++i) {
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
auto & layer = layers[i];
layer.attn_norm = create_tensor(tn(LLM_TENSOR_ATTN_NORM, "weight", i), {n_embd}, 0);

107
packages/node/native/llama/patches/0003-owned-range-filtered-state-report.patch Normal file
View File

@@ -0,0 +1,107 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: report owned-range filtered loading state
Concern: filtered state. Adds the llama_meshnet_range_report value type and
populates it after owned-range tensor registration with the half-open bounds
and mapped/resident byte counts derived from backend buffers, never from
caller parameters. Layer-filtered KV and session-to-sequence mapping remain
later scoped stories; this patch carries only the owned-range state report.
---
diff --git a/include/llama.h b/include/llama.h
index 229946ede..6fd7ad509 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -292,6 +292,16 @@ extern "C" {
ggml_backend_buffer_type_t buft;
};
+ // Immutable report for the project-owned dense-Llama owned-range state.
+ // Bounds are half-open [start, end); byte counts are derived from the
+ // registered tensors and backend buffers, never from caller parameters.
+ struct llama_meshnet_range_report {
+ int32_t start_layer;
+ int32_t end_layer;
+ uint64_t mapped_bytes;
+ uint64_t resident_bytes;
+ };
+
struct llama_model_params {
// NULL-terminated list of devices to use for offloading (if NULL, all available devices are used)
ggml_backend_dev_t * devices;
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index 05b8b9c91..efb290c1f 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
bool has_tensor_overrides;
std::vector<float> tensor_split_owned;
+
+ llama_meshnet_range_report meshnet_range_report = {};
+ bool has_meshnet_range_report = false;
};
llama_model::llama_model(const llama_model_params & params) : params(params), pimpl(std::make_unique<impl>()) {
@@ -1628,13 +1631,33 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
// print memory requirements per buffer type
+ uint64_t meshnet_mapped_bytes = 0;
+ uint64_t meshnet_resident_bytes = 0;
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
for (auto & buf: bufs) {
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
LLAMA_LOG_INFO("%s: %12s model buffer size = %8.2f MiB\n",
__func__, ggml_backend_buffer_name(buf.get()), ggml_backend_buffer_get_size(buf.get()) / 1024.0 / 1024.0);
}
}
+ if (meshnet_range_requested) {
+ // With mmap backend buffers the resident mapping exactly describes the
+ // mapped file spans of the owned tensors. On non-mmap backends the
+ // instantiated allocation is the resident measure and no file span is
+ // claimed as mapped.
+ if (ml.use_mmap) {
+ meshnet_mapped_bytes = meshnet_resident_bytes;
+ }
+ pimpl->meshnet_range_report = {
+ meshnet_start,
+ meshnet_end,
+ meshnet_mapped_bytes,
+ meshnet_resident_bytes,
+ };
+ pimpl->has_meshnet_range_report = true;
+ }
+
if (ml.no_alloc) {
return true;
}
@@ -1726,6 +1749,14 @@ uint64_t llama_model::n_elements() const {
return pimpl->n_elements;
}
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
+ if (out == nullptr || !pimpl->has_meshnet_range_report) {
+ return false;
+ }
+ *out = pimpl->meshnet_range_report;
+ return true;
+}
+
void llama_model::print_info() const {
const std::string rope_scaling_type = llama_rope_scaling_type_name(hparams.rope_scaling_type_train);
diff --git a/src/llama-model.h b/src/llama-model.h
index 45b054ced..5ef7a1515 100644
--- a/src/llama-model.h
+++ b/src/llama-model.h
@@ -652,6 +652,9 @@ struct llama_model {
// total number of parameters in the model
uint64_t n_elements() const;
+ // Project-owned owned-range state report; false when no range was loaded.
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
+
void print_info() const;
ggml_backend_dev_t dev_layer(int il) const;

73
packages/node/native/llama/patches/0004-dense-boundary-io-endpoint-guard.patch Normal file
View File

@@ -0,0 +1,73 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: guard dense graph behind boundary endpoint ownership
Concern: boundary I/O. Extends the range report with endpoint ownership flags
derived from the registered tensor map and fails the dense-Llama graph closed
for any partial owned range until typed head/tail endpoint adapters carry the
architecture boundary I/O.
---
diff --git a/include/llama.h b/include/llama.h
index 6fd7ad509..8a7521349 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -300,6 +300,8 @@ extern "C" {
int32_t end_layer;
uint64_t mapped_bytes;
uint64_t resident_bytes;
+ bool has_token_embeddings;
+ bool has_output_head;
};
struct llama_model_params {
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index efb290c1f..2ea8598ad 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1649,11 +1649,17 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
if (ml.use_mmap) {
meshnet_mapped_bytes = meshnet_resident_bytes;
}
+ const auto registered_name = [this](const char * name) {
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [name](const auto & entry) { return entry.first == name; });
+ };
pimpl->meshnet_range_report = {
meshnet_start,
meshnet_end,
meshnet_mapped_bytes,
meshnet_resident_bytes,
+ registered_name("token_embd.weight"),
+ output_norm != nullptr && output != nullptr,
};
pimpl->has_meshnet_range_report = true;
}
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index c3092763b..3b6854d0c 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -108,6 +108,25 @@ std::unique_ptr<llm_graph_context> llama_model_llama::build_arch_graph(const llm
template <bool embed>
llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_graph_params & params) : llm_graph_context(params) {
+ llama_meshnet_range_report meshnet_report = {};
+ if (model.meshnet_range_report(&meshnet_report)) {
+ // A partial owned range cannot execute the stock head/tail graph: the
+ // architecture boundary I/O must arrive through a typed endpoint
+ // adapter instead of local embeddings or the local output head.
+ if (meshnet_report.start_layer != 0) {
+ throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
+ }
+ if (meshnet_report.end_layer != n_layer) {
+ throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
+ }
+ if (!meshnet_report.has_token_embeddings) {
+ throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
+ }
+ if (!meshnet_report.has_output_head) {
+ throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
+ }
+ }
+
const int64_t n_embd_head = hparams.n_embd_head_v();
GGML_ASSERT(n_embd_head == hparams.n_embd_head_k());

202
packages/node/native/llama/patches/0005-worker-range-report-hook.patch Normal file
View File

@@ -0,0 +1,202 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: expose worker-owned range report hook and fixture
Concern: worker hooks. Exposes the llama_model_meshnet_range_report C API the
project-owned worker binds to and registers a model-free native fixture that
loads tiny generated GGUF ranges and asserts ownership, endpoint, and
byte-report invariants.
---
diff --git a/include/llama.h b/include/llama.h
index 8a7521349..5818daf94 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -613,6 +613,13 @@ extern "C" {
// Get metadata value as a string by key name
LLAMA_API int32_t llama_model_meta_val_str(const struct llama_model * model, const char * key, char * buf, size_t buf_size);
+ // Returns false unless this model was instantiated through the Meshnet
+ // owned-range loader. Values are derived from registered tensors and
+ // backend buffers, never copied from caller-supplied parameters.
+ LLAMA_API bool llama_model_meshnet_range_report(
+ const struct llama_model * model,
+ struct llama_meshnet_range_report * out);
+
// Get the number of metadata key/value pairs
LLAMA_API int32_t llama_model_meta_count(const struct llama_model * model);
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index 2ea8598ad..c9d3cf6d3 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -2695,6 +2695,10 @@ uint64_t llama_model_size(const llama_model * model) {
return model->size();
}
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
+ return model != nullptr && model->meshnet_range_report(out);
+}
+
const char * llama_model_chat_template(const llama_model * model, const char * name) {
const auto key = name ? LLM_KV(model->arch, name)(LLM_KV_TOKENIZER_CHAT_TEMPLATE)
: LLM_KV(model->arch)(LLM_KV_TOKENIZER_CHAT_TEMPLATE);
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 855295c15..9a7be6eed 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
# llama_build_and_test(test-double-float.cpp) # SLOW
llama_build_and_test(test-llama-archs.cpp)
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
endif()
llama_build_and_test(test-chat-peg-parser.cpp peg-parser/simple-tokenize.cpp)
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
new file mode 100644
index 000000000..6b3aa5ac5 100644
--- /dev/null
+++ b/tests/test-meshnet-range-ownership.cpp
@@ -0,0 +1,143 @@
+#include "ggml.h"
+#include "gguf.h"
+#include "llama.h"
+
+#include "../src/llama-model.h"
+
+#include <cstdio>
+#include <cstring>
+#include <stdexcept>
+#include <string>
+
+namespace {
+
+constexpr int kLayers = 4;
+constexpr int kEmbd = 8;
+constexpr int kFfn = 16;
+constexpr int kVocab = 16;
+
+void check(bool condition, const char * message) {
+ if (!condition) {
+ throw std::runtime_error(message);
+ }
+}
+
+void add_tensor(gguf_context * gguf, ggml_context * tensors, const char * name, int d0, int d1 = 1) {
+ ggml_tensor * tensor = d1 == 1
+ ? ggml_new_tensor_1d(tensors, GGML_TYPE_F32, d0)
+ : ggml_new_tensor_2d(tensors, GGML_TYPE_F32, d0, d1);
+ ggml_set_name(tensor, name);
+ std::memset(tensor->data, 0, ggml_nbytes(tensor));
+ gguf_add_tensor(gguf, tensor);
+}
+
+std::string write_fixture() {
+ const std::string path = "meshnet-dense-llama-range-fixture.gguf";
+ gguf_context * gguf = gguf_init_empty();
+ ggml_init_params params = { 128 * 1024, nullptr, false };
+ ggml_context * tensors = ggml_init(params);
+ check(gguf && tensors, "failed to create dense-Llama fixture contexts");
+
+ gguf_set_val_str(gguf, "general.architecture", "llama");
+ gguf_set_val_u32(gguf, "llama.context_length", 16);
+ gguf_set_val_u32(gguf, "llama.embedding_length", kEmbd);
+ gguf_set_val_u32(gguf, "llama.block_count", kLayers);
+ gguf_set_val_u32(gguf, "llama.feed_forward_length", kFfn);
+ gguf_set_val_u32(gguf, "llama.attention.head_count", 2);
+ gguf_set_val_u32(gguf, "llama.attention.head_count_kv", 2);
+ gguf_set_val_u32(gguf, "llama.rope.dimension_count", 4);
+ gguf_set_val_f32(gguf, "llama.attention.layer_norm_rms_epsilon", 1.0e-5f);
+ gguf_set_val_str(gguf, "tokenizer.ggml.model", "no_vocab");
+ gguf_set_val_u32(gguf, "llama.vocab_size", kVocab);
+
+ add_tensor(gguf, tensors, "token_embd.weight", kEmbd, kVocab);
+ add_tensor(gguf, tensors, "output_norm.weight", kEmbd);
+ add_tensor(gguf, tensors, "output.weight", kEmbd, kVocab);
+ for (int layer = 0; layer < kLayers; ++layer) {
+ const std::string p = "blk." + std::to_string(layer) + ".";
+ add_tensor(gguf, tensors, (p + "attn_norm.weight").c_str(), kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_q.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_k.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_v.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_output.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "ffn_norm.weight").c_str(), kEmbd);
+ add_tensor(gguf, tensors, (p + "ffn_gate.weight").c_str(), kEmbd, kFfn);
+ add_tensor(gguf, tensors, (p + "ffn_down.weight").c_str(), kFfn, kEmbd);
+ add_tensor(gguf, tensors, (p + "ffn_up.weight").c_str(), kEmbd, kFfn);
+ }
+ check(gguf_write_to_file(gguf, path.c_str(), false), "failed to write dense-Llama fixture");
+ ggml_free(tensors);
+ gguf_free(gguf);
+ return path;
+}
+
+int block_number(const std::string & name) {
+ int block = -1;
+ return std::sscanf(name.c_str(), "blk.%d.", &block) == 1 ? block : -1;
+}
+
+bool is_allowed_endpoint_tensor(const std::string & name, int start, int end) {
+ if (name == "token_embd.weight") {
+ return start == 0;
+ }
+ if (name == "output_norm.weight" || name == "output.weight") {
+ return end == kLayers;
+ }
+ return false;
+}
+
+llama_meshnet_range_report load_and_check(const std::string & path, int start, int end) {
+ llama_model_params params = llama_model_default_params();
+ params.meshnet_owned_layer_start = start;
+ params.meshnet_owned_layer_end = end;
+ llama_model * model = llama_model_load_from_file(path.c_str(), params);
+ check(model != nullptr, "failed to load dense-Llama fixture");
+
+ llama_meshnet_range_report report = {};
+ check(llama_model_meshnet_range_report(model, &report), "range report is absent");
+ check(report.start_layer == start, "reported start does not match registered range");
+ check(report.end_layer == end, "reported end does not match registered range");
+ check(report.mapped_bytes > 0, "mmap report is empty");
+ check(report.resident_bytes >= report.mapped_bytes, "resident bytes undercount mapped bytes");
+ check(report.has_token_embeddings == (start == 0), "token-embedding ownership is not the head endpoint");
+ check(report.has_output_head == (end == kLayers), "output-head ownership is not the tail endpoint");
+
+ const auto & tensors = llama_internal_get_tensor_map(model);
+ check(!tensors.empty(), "no tensors registered for owned range");
+ for (const auto & [name, _] : tensors) {
+ const int block = block_number(name);
+ check((block >= start && block < end) || (block == -1 && is_allowed_endpoint_tensor(name, start, end)),
+ "registered tensor is outside the owned range and its endpoints");
+ }
+ llama_model_free(model);
+ return report;
+}
+
+} // namespace
+
+int main() {
+ llama_backend_init();
+ const std::string fixture = write_fixture();
+
+ const auto head = load_and_check(fixture, 0, 1);
+ const auto middle = load_and_check(fixture, 1, 3);
+ load_and_check(fixture, 3, 4);
+ check(middle.mapped_bytes > head.mapped_bytes, "two-layer range did not map more bytes than head");
+
+ // A stock load has no owned-range report and registers every tensor.
+ llama_model * stock = llama_model_load_from_file(fixture.c_str(), llama_model_default_params());
+ check(stock != nullptr, "stock load failed");
+ llama_meshnet_range_report stock_report = {};
+ check(!llama_model_meshnet_range_report(stock, &stock_report), "stock load reported an owned range");
+ check(llama_internal_get_tensor_map(stock).size() == 3 + 9 * kLayers, "stock load lost tensors");
+ llama_model_free(stock);
+
+ llama_model_params invalid = llama_model_default_params();
+ invalid.meshnet_owned_layer_start = 3;
+ invalid.meshnet_owned_layer_end = 5;
+ check(llama_model_load_from_file(fixture.c_str(), invalid) == nullptr, "invalid range loaded");
+
+ std::remove(fixture.c_str());
+ llama_backend_free();
+ return 0;
+}

5
packages/node/native/llama/patches/SHA256SUMS
View File

@@ -1,3 +1,6 @@
# SHA-256 digests for the ordered patch series. Do not reorder this file. # SHA-256 digests for the ordered patch series. Do not reorder this file.
1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch 1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
51c205e3ca26e104f80c838eeeb11115b8d436036014116d2bb407178c30e0bd 0002-dense-llama-owned-range-loader.patch 6032ecca4d3ec3ce072f099dc402529aa6f53a7069a97e75222cf2fa50abb1a3 0002-dense-llama-owned-range-loading.patch
4871a37544df658980a01b4f94151a90b609fb144c931b4a814309ee608ebb46 0003-owned-range-filtered-state-report.patch
19d451ce259150ffede793c4eb547425375c0fcd97caf326b43e8f1a204f05b6 0004-dense-boundary-io-endpoint-guard.patch
cf263357a6a8de193f710836c7c467c38cac7099975303ee2628e0609daf5a47 0005-worker-range-report-hook.patch

117
packages/node/native/llama/patches/UPSTREAM-ASSUMPTIONS.json Normal file
View File

@@ -0,0 +1,117 @@
{
"schema_version": 1,
"upstream_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"patches": {
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch": {
"concern": "build-marker",
"files": {
"CMakeLists.txt": {
"before": "81f23d7e70b7378511af5d01be680c03aebc2b15",
"after": "a9afcffa68bed7cbd8fad39ad9f95ad784251234"
},
"cmake/meshnet-patch-stack.cmake": {
"before": null,
"after": "910646b4d6164831d4f8e523dd5e49ce7796994f"
}
},
"api_assumptions": [
"CMake >= 3.14 include() of a project-relative module from the top-level CMakeLists.txt",
"add_library(<name> INTERFACE) and target_compile_definitions(... INTERFACE ...)"
]
},
"0002-dense-llama-owned-range-loading.patch": {
"concern": "range-loading",
"files": {
"include/llama.h": {
"before": "a311ac2023579376ed571a614dcac9d259692e56",
"after": "229946ede026ef36b4c4f0355e0421b082334ee0"
},
"src/llama-model.cpp": {
"before": "d87481381e46025c9c87c4af5f116015a44124c3",
"after": "05b8b9c912716fd57061100ec5c47203f79b50ce"
},
"src/models/llama.cpp": {
"before": "4bfebc8843c655e122e5f2064a791c583ad3779b",
"after": "c3092763b82e93b596ba531a0fb01769df6f0e27"
}
},
"api_assumptions": [
"llama_model_params is an aggregate C struct initialized by llama_model_default_params()",
"llama_model_loader::done_getting_tensors(bool partial = false) const",
"llm_hparams::n_layer() and llama_model_base::load_tensors(llama_model_loader &)",
"llama_model_llama::load_arch_tensors per-layer create_tensor loop and LLM_ARCH_LLAMA gate"
]
},
"0003-owned-range-filtered-state-report.patch": {
"concern": "filtered-state",
"files": {
"include/llama.h": {
"before": "229946ede026ef36b4c4f0355e0421b082334ee0",
"after": "6fd7ad509006190a7e7dd84a3d1ac2672ad2d21d"
},
"src/llama-model.cpp": {
"before": "05b8b9c912716fd57061100ec5c47203f79b50ce",
"after": "efb290c1f3dceb868dc5e316e765767ddf0532f8"
},
"src/llama-model.h": {
"before": "45b054cedf1d1e6accc7cf8aafcbae374614e64f",
"after": "5ef7a1515a9b27858e3e5fe694a285a4a55a8bf2"
}
},
"api_assumptions": [
"llama_model::impl pimpl struct and llama_model::meshnet_range_report(out) const accessor",
"pimpl->ctxs_bufs backend buffer map and ggml_backend_buffer_get_size",
"llama_model_loader::use_mmap public member"
]
},
"0004-dense-boundary-io-endpoint-guard.patch": {
"concern": "boundary-io",
"files": {
"include/llama.h": {
"before": "6fd7ad509006190a7e7dd84a3d1ac2672ad2d21d",
"after": "8a75213494cc07b20ca2d99cd21b30109e758b7b"
},
"src/llama-model.cpp": {
"before": "efb290c1f3dceb868dc5e316e765767ddf0532f8",
"after": "2ea8598ad2037920082ef88db75684283c65118c"
},
"src/models/llama.cpp": {
"before": "c3092763b82e93b596ba531a0fb01769df6f0e27",
"after": "3b6854d0c32cdb816a0b31c337767fa1a9305b24"
}
},
"api_assumptions": [
"llama_model::tensors_by_name is std::vector<std::pair<std::string, ggml_tensor *>>",
"llama_model::{output_norm, output} members reflect output-head registration",
"llama_model_llama::graph<embed> constructor and llm_graph_context::n_layer"
]
},
"0005-worker-range-report-hook.patch": {
"concern": "worker-hooks",
"files": {
"include/llama.h": {
"before": "8a75213494cc07b20ca2d99cd21b30109e758b7b",
"after": "5818daf94d481c5eee201ad1300f37e7c0fe60f9"
},
"src/llama-model.cpp": {
"before": "2ea8598ad2037920082ef88db75684283c65118c",
"after": "c9d3cf6d34cbe2fd50100b91ed2a0436f5b72415"
},
"tests/CMakeLists.txt": {
"before": "855295c152faa78fa4acdec54013940c41627be9",
"after": "9a7be6eedfd6a311694a8a6a95976fb230ee165c"
},
"tests/test-meshnet-range-ownership.cpp": {
"before": null,
"after": "6b3aa5ac5dc1b6074401f97b8acd07b00dfa9780"
}
},
"api_assumptions": [
"LLAMA_API export macro and extern \"C\" block in include/llama.h",
"tests/CMakeLists.txt llama_build_and_test(<file.cpp>) helper",
"llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h",
"gguf empty-context writer API: gguf_init_empty, gguf_add_tensor, gguf_write_to_file"
]
}
}
}

5
packages/node/native/llama/patches/series
View File

@@ -1,2 +1,5 @@
0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
0002-dense-llama-owned-range-loader.patch 0002-dense-llama-owned-range-loading.patch
0003-owned-range-filtered-state-report.patch
0004-dense-boundary-io-endpoint-guard.patch
0005-worker-range-report-hook.patch

218
scripts/llama_cpp_dependency.py
View File

@@ -3,6 +3,13 @@
This tool deliberately owns only a source dependency boundary. It never This tool deliberately owns only a source dependency boundary. It never
downloads a model, invokes inference, or interprets generated text. downloads a model, invokes inference, or interprets generated text.
DGR-028 adds the numbered patch-stack contract: the ordered series applies,
checks, and reverses deterministically against the exact manifest pin, each
patch's recorded upstream file/API assumptions are enforced before it is
attempted, the first incompatible patch is named on failure, and the stack is
refused if it carries license/attribution damage or Meshnet control-plane
(routing, billing, relay, authentication) code.
""" """
from __future__ import annotations from __future__ import annotations
@@ -61,6 +68,11 @@ def _run(*args: str, cwd: pathlib.Path | None = None) -> str:
return completed.stdout.strip() return completed.stdout.strip()
def _git(source: pathlib.Path, *args: str) -> str:
"""Run Git against the materialized upstream checkout."""
return _run("git", "-C", str(source), *args)
def _load_lock() -> dict[str, Any]: def _load_lock() -> dict[str, Any]:
try: try:
lock = json.loads(LOCK_PATH.read_text()) lock = json.loads(LOCK_PATH.read_text())
@@ -113,8 +125,171 @@ def _patches(lock: dict[str, Any]) -> list[pathlib.Path]:
return patches return patches
def _git(source: pathlib.Path, *args: str) -> str: def _parse_patch_files(patch: pathlib.Path) -> dict[str, tuple[str | None, str | None]]:
return _run("git", "-C", str(source), *args) """Parse one patch into {path: (before-short, after-short)}.
Short object IDs come from the patch's ``index`` lines; an all-zero side
means the file is created (``before is None``) or deleted (``after is
None``). Patch order is preserved.
"""
files: dict[str, tuple[str | None, str | None]] = {}
current: str | None = None
for line in patch.read_text().splitlines():
header = re.match(r"^diff --git a/(.+) b/(.+)$", line)
if header:
if header.group(1) != header.group(2):
raise DependencyError(f"{patch.name}: rename/copy diffs are unsupported: {line}")
current = header.group(1)
files[current] = (None, None)
continue
index = re.match(r"^index ([0-9a-f]{7,40})\.\.([0-9a-f]{7,40})(?:\s|$)", line)
if index and current is not None:
before, after = index.group(1), index.group(2)
files[current] = (
None if set(before) == {"0"} else before,
None if set(after) == {"0"} else after,
)
if not files:
raise DependencyError(f"{patch.name}: no file diffs found")
return files
_ASSUMPTIONS_DEFAULT = "patches/UPSTREAM-ASSUMPTIONS.json"
def _assumptions(lock: dict[str, Any], patches: list[pathlib.Path]) -> dict[str, Any]:
"""Load and validate the recorded upstream file/API assumptions.
The record must cover exactly the ordered series, each recorded file must
match the patch's parsed diff headers, and each recorded full object ID
must agree with the patch's abbreviated ``index`` IDs. A stale or edited
record is a fail-closed error, never a warning.
"""
configured = lock.get("patch_assumptions", _ASSUMPTIONS_DEFAULT)
if not isinstance(configured, str) or not configured:
raise DependencyError("patch_assumptions must name a manifest-relative JSON path")
relative = pathlib.Path(configured)
if relative.is_absolute() or ".." in relative.parts:
raise DependencyError("patch_assumptions must stay inside the repository manifest tree")
candidate = (LLAMA_DIR / relative).absolute()
try:
candidate.relative_to(LLAMA_DIR.absolute())
except ValueError as error:
raise DependencyError("patch_assumptions must live under packages/node/native/llama") from error
if not candidate.is_file():
raise DependencyError(f"recorded upstream assumptions are missing: {candidate}")
try:
doc = json.loads(candidate.read_text())
except (OSError, json.JSONDecodeError) as error:
raise DependencyError(f"invalid upstream assumptions: {candidate}: {error}") from error
if not isinstance(doc, dict) or doc.get("schema_version") != 1:
raise DependencyError("upstream assumptions must declare schema_version 1")
if doc.get("upstream_commit") != lock["commit"]:
raise DependencyError("upstream assumptions disagree with the locked commit")
recorded = doc.get("patches")
if not isinstance(recorded, dict):
raise DependencyError("upstream assumptions must record a patches object")
names = [patch.name for patch in patches]
if list(recorded.keys()) != names:
raise DependencyError(
f"upstream assumptions do not cover exactly the ordered series: "
f"expected {names}, got {list(recorded.keys())}"
)
for patch in patches:
entry = recorded[patch.name]
if not isinstance(entry.get("concern"), str) or not entry["concern"]:
raise DependencyError(f"{patch.name}: assumptions are missing the scoped concern")
symbols = entry.get("api_assumptions")
if not isinstance(symbols, list) or not symbols or not all(isinstance(s, str) and s for s in symbols):
raise DependencyError(f"{patch.name}: assumptions must record upstream file/API assumptions")
files = entry.get("files")
if not isinstance(files, dict):
raise DependencyError(f"{patch.name}: assumptions must record a files object")
parsed = _parse_patch_files(patch)
if list(files.keys()) != list(parsed.keys()):
raise DependencyError(
f"{patch.name}: recorded files {list(files.keys())} disagree with the patch bytes {list(parsed.keys())}"
)
for path, (before_short, after_short) in parsed.items():
blobs = files[path]
if not isinstance(blobs, dict):
raise DependencyError(f"{patch.name}: recorded blobs for {path} must be an object")
for side, short in (("before", before_short), ("after", after_short)):
value = blobs.get(side)
if short is None:
if value is not None:
raise DependencyError(f"{patch.name}: {path} {side} must be null for a created/deleted file")
elif not isinstance(value, str) or not re.fullmatch(r"[0-9a-f]{40}", value) or not value.startswith(short):
raise DependencyError(
f"{patch.name}: recorded {side} blob for {path} does not match the patch index ID {short}"
)
return doc
# Control-plane vocabulary that must never enter the upstream patch stack:
# Meshnet routing, billing, relay, authentication, and transport semantics are
# backend-agnostic and live outside the llama.cpp fork boundary (ADR-0024).
_CONTROL_PLANE_TERMS = re.compile(
r"\b(tracker|routing|route session|grpc|billing|wallet|relay|telemetry|"
r"auth|authentication|load balanc\w*)\b",
re.IGNORECASE,
)
_LICENSE_PATH = re.compile(r"(^|/)(license|copying|notice)(\..*)?$", re.IGNORECASE)
_LICENSE_TEXT = re.compile(
r"copyright|permission is hereby granted|mit license|apache license|gnu general public",
re.IGNORECASE,
)
def _verify_patch_stack_boundaries(patches: list[pathlib.Path]) -> None:
"""Refuse license/attribution damage or control-plane code in the stack."""
for patch in patches:
for line in patch.read_text().splitlines():
header = re.match(r"^diff --git a/(.+) b/(.+)$", line)
if header and (_LICENSE_PATH.search(header.group(1)) or _LICENSE_PATH.search(header.group(2))):
raise DependencyError(f"{patch.name}: license/attribution files may not be patched: {line}")
if line.startswith("-") and not line.startswith("---") and _LICENSE_TEXT.search(line[1:]):
raise DependencyError(f"{patch.name}: removing license or attribution text is refused: {line}")
body = patch.read_text()
match = _CONTROL_PLANE_TERMS.search(body)
if match:
raise DependencyError(
f"{patch.name}: Meshnet control-plane term {match.group(0)!r} must not enter the patch stack"
)
def _index_blob(source: pathlib.Path, path: str) -> str | None:
"""Return the staged blob object ID for path, or None when absent."""
records = _git(source, "ls-files", "-s", "-z", "--", path).split("\0")
entries = [record for record in records if record]
if not entries:
return None
if len(entries) != 1:
raise DependencyError(f"unmerged index entry blocks patch verification: {path}")
metadata, _ = entries[0].split("\t", 1)
mode, blob, stage = metadata.split()
if stage != "0":
raise DependencyError(f"unmerged index entry blocks patch verification: {path}")
return blob
def _check_assumption_blobs(
source: pathlib.Path,
patch: pathlib.Path,
files: dict[str, Any],
side: str,
) -> None:
"""Fail on the first recorded pre-/post-image blob the index disagrees with."""
for path, blobs in files.items():
expected = blobs[side]
actual = _index_blob(source, path)
if actual != expected:
raise DependencyError(
f"first incompatible patch: {patch.name}: recorded {side} blob for {path} is "
f"{expected}, found {actual}"
)
def _verify_tracked_content(source: pathlib.Path, lock: dict[str, Any]) -> None: def _verify_tracked_content(source: pathlib.Path, lock: dict[str, Any]) -> None:
@@ -235,16 +410,36 @@ def fetch(workspace: pathlib.Path) -> pathlib.Path:
def apply(source: pathlib.Path) -> None: def apply(source: pathlib.Path) -> None:
lock = _load_lock() lock = _load_lock()
patches = _patches(lock) patches = _patches(lock)
assumptions = _assumptions(lock, patches)
_verify_patch_stack_boundaries(patches)
_verify_source(source, lock, require_clean=True) _verify_source(source, lock, require_clean=True)
for patch in patches: for patch in patches:
files = assumptions["patches"][patch.name]["files"]
_check_assumption_blobs(source, patch, files, "before")
_git(source, "apply", "--check", str(patch)) _git(source, "apply", "--check", str(patch))
_git(source, "apply", "--index", str(patch)) _git(source, "apply", "--index", str(patch))
_check_assumption_blobs(source, patch, files, "after")
_verify_patched_source(source, lock) _verify_patched_source(source, lock)
def reverse(source: pathlib.Path) -> None:
"""Reverse the complete verified stack and recover the exact clean pin."""
lock = _load_lock()
patches = _patches(lock)
assumptions = _assumptions(lock, patches)
_verify_patch_stack_boundaries(patches)
_verify_source(source, lock, require_clean=False)
_verify_patched_source(source, lock)
for patch in reversed(patches):
files = assumptions["patches"][patch.name]["files"]
_check_assumption_blobs(source, patch, files, "after")
_git(source, "apply", "--reverse", "--check", str(patch))
_git(source, "apply", "--reverse", "--index", str(patch))
_check_assumption_blobs(source, patch, files, "before")
_verify_source(source, lock, require_clean=True)
def _verify_patched_source(source: pathlib.Path, lock: dict[str, Any]) -> None: def _verify_patched_source(source: pathlib.Path, lock: dict[str, Any]) -> None:
if _git(source, "diff", "--quiet"):
raise DependencyError("local unstaged edits detected after applying patch stack")
changed_paths = _git(source, "diff", "--cached", "--name-only").splitlines() changed_paths = _git(source, "diff", "--cached", "--name-only").splitlines()
if changed_paths != lock["patched_paths"]: if changed_paths != lock["patched_paths"]:
raise DependencyError(f"patched paths drifted: expected {lock['patched_paths']}, got {changed_paths}") raise DependencyError(f"patched paths drifted: expected {lock['patched_paths']}, got {changed_paths}")
@@ -297,6 +492,13 @@ def smoke(binary: pathlib.Path) -> None:
print(output) print(output)
def verify(workspace: pathlib.Path) -> None:
"""Apply, verify, reverse, and leave the exact cached pin pristine."""
source = fetch(workspace)
apply(source)
reverse(source)
def reproduce(workspace: pathlib.Path) -> None: def reproduce(workspace: pathlib.Path) -> None:
source = fetch(workspace) source = fetch(workspace)
build_dir = workspace.resolve() / "build" build_dir = workspace.resolve() / "build"
@@ -330,6 +532,10 @@ def main() -> int:
fetch_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp") fetch_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
apply_parser = subcommands.add_parser("apply") apply_parser = subcommands.add_parser("apply")
apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True) apply_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
reverse_parser = subcommands.add_parser("reverse")
reverse_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
verify_parser = subcommands.add_parser("verify")
verify_parser.add_argument("--workspace", type=pathlib.Path, default=ROOT / "build/llama.cpp")
build_parser = subcommands.add_parser("build") build_parser = subcommands.add_parser("build")
build_parser.add_argument("--source-dir", type=pathlib.Path, required=True) build_parser.add_argument("--source-dir", type=pathlib.Path, required=True)
build_parser.add_argument("--build-dir", type=pathlib.Path, required=True) build_parser.add_argument("--build-dir", type=pathlib.Path, required=True)
@@ -345,6 +551,10 @@ def main() -> int:
fetch(args.workspace) fetch(args.workspace)
elif args.command == "apply": elif args.command == "apply":
apply(args.source_dir) apply(args.source_dir)
elif args.command == "reverse":
reverse(args.source_dir)
elif args.command == "verify":
verify(args.workspace)
elif args.command == "build": elif args.command == "build":
build(args.source_dir, args.build_dir) build(args.source_dir, args.build_dir)
elif args.command == "smoke": elif args.command == "smoke":

4
tests/test_llama_cpp_dependency.py
View File

@@ -240,7 +240,9 @@ def test_dependency_script_reports_the_locked_boundary_without_network() -> None
report = json.loads(completed.stdout) report = json.loads(completed.stdout)
assert report["commit"] == (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip() assert report["commit"] == (LLAMA_DIR / "UPSTREAM_COMMIT").read_text().strip()
assert report["patch_count"] == 2 assert report["patch_count"] == len(
(LLAMA_DIR / "patches/series").read_text().splitlines()
)
assert report["model_downloads"] is False assert report["model_downloads"] is False
assert report["semantic_certification"] is False assert report["semantic_certification"] is False
assert "dense" in report["glm_stock_limitations"].lower() assert "dense" in report["glm_stock_limitations"].lower()