popov/neuron-tai
1
0
Fork 0
Code Issues 42 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement numbered patch-stack apply/verify enforcement (DGR-028)

Browse Source 
Split the range-loader patch into single-concern patches 0002-0005 (loader,
filtered state report, boundary I/O endpoint guard, worker range-report
hook), add UPSTREAM-ASSUMPTIONS.json describing each patch's assumptions,
and enforce control-plane/license boundary checks plus first-incompatible-
patch reporting in scripts/llama_cpp_dependency.py apply/reverse/verify.

7 passed in tests/test_llama_cpp_dependency.py; SHA256SUMS verified against
all five patches; focused native CTest (test-meshnet-range-ownership 1/1)
recorded in evidence README (build/ dir not present in this environment to
independently reverify).
This commit is contained in:
Dobromir Popov
2026-07-21 13:22:55 +03:00
parent 902ecde363
commit 7da90ef475
14 changed files with 1027 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
packages/node/native/llama/PATCH-STACK.md
View File

@@ -6,14 +6,57 @@ updating the recorded tree/blob assumptions and reviewing every patch anew.
## Ordered series
One numbered patch per concern (ADR-0024 local seams only):
1. `0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch` adds only an
interface-library marker used to prove the patched source was configured.
It has no execution, transport, model-loading, or semantic effect.
2. `0002-dense-llama-owned-range-loading.patch` (range loading) adds the
`meshnet_owned_layer_start/end` model params, validates the half-open range
against the GGUF block count for dense Llama only, filters per-layer tensor
registration and the optional scale pass to the owned range, and keeps
endpoint tensors with their owning endpoints. Zero/zero params preserve
stock whole-model loading.
3. `0003-owned-range-filtered-state-report.patch` (filtered state) adds
`llama_meshnet_range_report` and populates it from registered tensors and
backend buffers — derived, never caller-asserted. Layer-filtered KV and
session-to-sequence mapping remain later scoped stories (DGR-038).
4. `0004-dense-boundary-io-endpoint-guard.patch` (boundary I/O) extends the
report with endpoint ownership flags and fails the dense-Llama graph closed
for any partial owned range until typed head/tail endpoint adapters carry
the architecture boundary I/O (DGR-035).
5. `0005-worker-range-report-hook.patch` (worker hooks) exposes the
`llama_model_meshnet_range_report` C API the project-owned worker binds to
and registers a model-free native fixture test for it.
Future patches may implement only the ADR-0020 local seams: range-aware tensor
loading, endpoint ownership, architecture-defined intermediate boundaries, and
layer-filtered KV/session mapping. Meshnet routing, Tracker, gRPC, relay,
billing, authentication, and telemetry must remain outside this directory.
Meshnet routing, Tracker, gRPC, relay, billing, authentication, and telemetry
remain outside this directory; the stack is checked for such control-plane
code and for license/attribution preservation on every apply.
## Upstream assumptions and fail-closed verification
`patches/UPSTREAM-ASSUMPTIONS.json` records, for every patch, the exact
upstream blob IDs each touched file must have before and after the patch, plus
the upstream file/API assumptions the patch relies on.
`scripts/llama_cpp_dependency.py verify` runs the deterministic cycle against
the exact manifest pin:
1. Verify the materialized checkout identity (commit, tree, blobs, license,
cleanliness) exactly like `fetch`.
2. For each patch in `patches/series` order, check the recorded pre-image
blobs of the files it touches, then `git apply --check`, then
`git apply --index`. The first patch whose assumptions or context fail is
reported as the first incompatible patch and nothing further is attempted.
3. Verify the patched tree equals the manifest `patched_tree` and touches
exactly the manifest `patched_paths`.
4. Reverse the series in reverse order (`git apply -R --index`) and verify the
restored tree equals the pristine locked tree, leaving the checkout
pristine.
`apply` performs steps 1–3 and leaves the stack applied for a native build;
`reverse` performs step 4 on an applied checkout. A pin change that breaks any
patch therefore fails loudly with the first incompatible patch instead of
silently drifting.
`scripts/llama_cpp_dependency.py fetch` reads the in-repo manifest and checks out
only its exact commit as detached HEAD in `build/llama.cpp/source`, an ignored

8
packages/node/native/llama/THIRD_PARTY_NOTICES.md
View File

@@ -6,8 +6,12 @@ The reproducibility harness fetches source from
- Upstream license: MIT. The fetched checkout's `LICENSE` and copyright notices
remain intact and must accompany any redistribution of this source or binary.
- Meshnet's one-patch CMake marker is an additive local change. It does not
replace, relicense, or remove upstream notices.
No patch in the numbered stack touches `LICENSE`/copyright files or removes
any upstream copyright or license text; this is enforced on every apply.
- Meshnet's five-patch stack (CMake marker, range loading, filtered state,
boundary I/O, worker hooks) is an additive local change. It does not
replace, relicense, or remove upstream notices, and it contains no Meshnet
routing, billing, relay, authentication, Tracker, or transport code.
- No donor code is included. In particular, Mesh-LLM remains a research/test
donor only and no part of its scheduler, routing, discovery, package manager,
or patch series is incorporated here.

13
packages/node/native/llama/UPSTREAM_LOCK.json
View File

@@ -10,16 +10,23 @@
"method": "git-clone-detached-commit",
"workspace": "build/llama.cpp"
},
"patched_tree": "322d8b463df74a2226f0b513176643d815f54452",
"patched_tree": "c0045714735ae5ee7b7334a480d8ac04e03e1b18",
"upstream_license": "MIT",
"patch_series": [
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
"0002-dense-llama-owned-range-loader.patch"
"0002-dense-llama-owned-range-loading.patch",
"0003-owned-range-filtered-state-report.patch",
"0004-dense-boundary-io-endpoint-guard.patch",
"0005-worker-range-report-hook.patch"
],
"patch_scope": [
"Reserved CMake ABI marker only; no execution or model semantics.",
"Dense-Llama owned-range registration, mmap reporting, and native fixture tests."
"Range loading: dense-Llama owned-range params, validation, and filtered tensor registration with endpoint ownership.",
"Filtered state: owned-range report populated from registered tensors and backend buffers, derived never asserted.",
"Boundary I/O: endpoint ownership flags and a fail-closed dense graph guard until typed endpoint adapters exist.",
"Worker hooks: public C range-report API and the model-free native fixture test the project-owned worker binds to."
],
"patch_assumptions": "patches/UPSTREAM-ASSUMPTIONS.json",
"build": {
"generator": "Unix Makefiles",
"cmake_minimum": "3.14",

169
packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch
View File

@@ -1,169 +0,0 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense owned-range loading seam
diff --git a/include/llama.h b/include/llama.h
index a311ac20..1f9459cf 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -292,6 +292,19 @@ extern "C" {
ggml_backend_buffer_type_t buft;
};
+ // Immutable report for the project-owned dense-Llama range-loading seam.
+ // The bounds are inclusive/exclusive and are populated only after the
+ // model has registered and allocated its owned tensors.
+ struct llama_meshnet_range_report {
+ int32_t start_layer;
+ int32_t end_layer;
+ uint64_t mapped_bytes;
+ uint64_t resident_bytes;
+ uint64_t registered_bytes;
+ bool has_token_embeddings;
+ bool has_output_head;
+ };
+
struct llama_model_params {
@@ -319,6 +332,12 @@ extern "C" {
const struct llama_model_kv_override * kv_overrides;
+ int32_t meshnet_owned_layer_start;
+ int32_t meshnet_owned_layer_end;
+
// Keep the booleans together to avoid misalignment during copy-by-value.
@@ -616,6 +635,13 @@ extern "C" {
LLAMA_API uint64_t llama_model_size(const struct llama_model * model);
+ LLAMA_API bool llama_model_meshnet_range_report(
+ const struct llama_model * model,
+ struct llama_meshnet_range_report * out);
+
// Get the default chat template. Returns nullptr if not available
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index d8748138..4d2a3ec1 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
std::vector<float> tensor_split_owned;
+ llama_meshnet_range_report meshnet_range_report = {};
+ bool has_meshnet_range_report = false;
};
@@ -1236,6 +1239,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
const bool use_mmap_buffer = true;
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = params.meshnet_owned_layer_start;
+ const int meshnet_end = params.meshnet_owned_layer_end;
+ if (meshnet_range_requested) {
+ if (arch != LLM_ARCH_LLAMA) {
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
+ }
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
+ meshnet_start, meshnet_end, hparams.n_layer()));
+ }
+ }
@@ -1336,7 +1352,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
- for (int i = 0; i < n_layer_all; ++i) {
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
@@ -1487,7 +1505,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
- ml.done_getting_tensors();
+ ml.done_getting_tensors(meshnet_range_requested);
@@ -1613,8 +1631,11 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
+ uint64_t meshnet_mapped_bytes = 0;
+ uint64_t meshnet_resident_bytes = 0;
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
for (auto & buf: bufs) {
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
@@ -1637,6 +1658,35 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
+ if (meshnet_range_requested) {
+ uint64_t registered_bytes = 0;
+ for (const auto & [_, tensor] : tensors_by_name) registered_bytes += ggml_nbytes(tensor);
+ if (ml.use_mmap) for (const auto & [first, last] : ml.mmaps_used) if (last > first) meshnet_mapped_bytes += last - first;
+ const auto registered = [this](const ggml_tensor * tensor) {
+ return tensor != nullptr && std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [tensor](const auto & entry) { return entry.second == tensor; });
+ };
+ const auto registered_name = [this](const char * name) {
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [name](const auto & entry) { return entry.first == name; });
+ };
+ pimpl->meshnet_range_report = { meshnet_start, meshnet_end, meshnet_mapped_bytes, meshnet_resident_bytes,
+ registered_bytes, registered_name("token_embd.weight"), registered(output_norm) && registered(output) };
+ pimpl->has_meshnet_range_report = true;
+ }
return true;
@@ -1711,6 +1761,14 @@ uint64_t llama_model::n_elements() const {
}
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
+ if (out == nullptr || !pimpl->has_meshnet_range_report) return false;
+ *out = pimpl->meshnet_range_report;
+ return true;
+}
@@ -2308,6 +2366,8 @@ llama_model_params llama_model_default_params() {
/*.kv_overrides =*/ nullptr,
+ /*.meshnet_owned_layer_start =*/ 0,
+ /*.meshnet_owned_layer_end =*/ 0,
@@ -2641,6 +2701,10 @@ uint64_t llama_model_size(const llama_model * model) {
}
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
+ return model != nullptr && model->meshnet_range_report(out);
+}
diff --git a/src/llama-model.h b/src/llama-model.h
index 45b054ce..1b3f9bd0 100644
--- a/src/llama-model.h
+++ b/src/llama-model.h
@@ -652,6 +652,8 @@ struct llama_model {
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
+
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index 4bfebc88..b4f25aed 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -34,18 +34,26 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
- // output
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+ if (!meshnet_range_requested || meshnet_start == 0) tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
- // if output is NULL, init from the input tok embed
- if (output == NULL) {
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ if (output == NULL) output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
}
- for (int i = 0; i < n_layer; ++i) {
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
@@ -102,6 +110,25 @@ llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_grap
+ llama_meshnet_range_report meshnet_report = {};
+ if (model.meshnet_range_report(&meshnet_report)) {
+ if (meshnet_report.start_layer != 0) throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
+ if (meshnet_report.end_layer != n_layer) throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
+ if (!meshnet_report.has_token_embeddings) throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
+ if (!meshnet_report.has_output_head) throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
+ }
ggml_tensor * cur;
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 855295c1..9a7be6ee 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
new file mode 100644
index 00000000..7b58ebf8
--- /dev/null
+++ b/tests/test-meshnet-range-ownership.cpp
@@ -0,0 +1,6 @@
+#include "ggml.h"
+#include "gguf.h"
+#include "llama.h"
+#include "../src/llama-model.h"
+#include <cstdio>
+#include <cstring>

122
packages/node/native/llama/patches/0002-dense-llama-owned-range-loading.patch Normal file
View File

@@ -0,0 +1,122 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense owned-range tensor loading
Concern: range loading. Adds meshnet_owned_layer_start/end model params,
validates the half-open range against the GGUF block count for dense Llama
only, filters per-layer tensor registration and the optional per-layer scale
pass to the owned range, and keeps endpoint tensors with their owning
endpoints (head: token embeddings; tail: final norm and output head).
Stock zero/zero params preserve whole-model loading.
---
diff --git a/include/llama.h b/include/llama.h
index a311ac202..229946ede 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -319,6 +319,12 @@ extern "C" {
// override key-value pairs of the model meta data
const struct llama_model_kv_override * kv_overrides;
+ // Project-owned dense-Llama owned range [start, end). A zero/zero
+ // pair preserves stock whole-model loading; any other pair is
+ // validated against the GGUF block count before tensor registration.
+ int32_t meshnet_owned_layer_start;
+ int32_t meshnet_owned_layer_end;
+
// Keep the booleans together to avoid misalignment during copy-by-value.
bool vocab_only; // only load the vocabulary, no weights
bool use_mmap; // use mmap if possible
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index d87481381..05b8b9c91 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1236,6 +1236,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
const bool use_mmap_buffer = true;
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = params.meshnet_owned_layer_start;
+ const int meshnet_end = params.meshnet_owned_layer_end;
+ if (meshnet_range_requested) {
+ if (arch != LLM_ARCH_LLAMA) {
+ throw std::runtime_error("Meshnet owned range currently supports dense Llama only");
+ }
+ if (meshnet_start < 0 || meshnet_end <= meshnet_start || meshnet_end > static_cast<int>(hparams.n_layer())) {
+ throw std::runtime_error(format("invalid Meshnet owned range [%d, %d) for GGUF block count %d",
+ meshnet_start, meshnet_end, hparams.n_layer()));
+ }
+ }
+
this->ml = &ml; // to be used by create_tensor() and load_arch_tensors()
LLAMA_LOG_INFO("%s: loading model tensors, this can take a while... (mmap = %s, direct_io = %s)\n",
@@ -1336,7 +1349,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
// generic pass: load optional per-tensor/per-expert ".scale" tensors (e.g. NVFP4 scale2)
// this avoids having to add scale loading to every architecture
- for (int i = 0; i < n_layer_all; ++i) {
+ const int optional_scale_start = meshnet_range_requested ? meshnet_start : 0;
+ const int optional_scale_end = meshnet_range_requested ? meshnet_end : n_layer_all;
+ for (int i = optional_scale_start; i < optional_scale_end; ++i) {
auto & layer = layers[i];
// attention weight scales (per-tensor, shape {1})
@@ -1487,7 +1502,7 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
}
}
- ml.done_getting_tensors();
+ ml.done_getting_tensors(meshnet_range_requested);
// Tied NVFP4 output is valid when no separate LM-head scale tensors are present.
// If sidecar scales exist, the output weight must be an actual output tensor.
@@ -2308,6 +2323,8 @@ llama_model_params llama_model_default_params() {
/*.progress_callback =*/ nullptr,
/*.progress_callback_user_data =*/ nullptr,
/*.kv_overrides =*/ nullptr,
+ /*.meshnet_owned_layer_start =*/ 0,
+ /*.meshnet_owned_layer_end =*/ 0,
/*.vocab_only =*/ false,
/*.use_mmap =*/ true,
/*.use_direct_io =*/ false,
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index 4bfebc884..c3092763b 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -34,18 +34,29 @@ void llama_model_llama::load_arch_hparams(llama_model_loader & ml) {
void llama_model_llama::load_arch_tensors(llama_model_loader &) {
LLAMA_LOAD_LOCALS;
- tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ const bool meshnet_range_requested = params.meshnet_owned_layer_start != 0 || params.meshnet_owned_layer_end != 0;
+ const int meshnet_start = meshnet_range_requested ? params.meshnet_owned_layer_start : 0;
+ const int meshnet_end = meshnet_range_requested ? params.meshnet_owned_layer_end : n_layer;
+
+ // Endpoint ownership: only the head shard (start == 0) owns the token
+ // embeddings and only the tail shard (end == n_layer) owns the final norm
+ // and output head. Middle ranges register per-layer tensors only.
+ if (!meshnet_range_requested || meshnet_start == 0) {
+ tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+ }
- // output
- output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
- output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+ if (!meshnet_range_requested || meshnet_end == n_layer) {
+ // output
+ output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+ output = create_tensor(tn(LLM_TENSOR_OUTPUT, "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
- // if output is NULL, init from the input tok embed
- if (output == NULL) {
- output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ // if output is NULL, init from the input tok embed
+ if (output == NULL) {
+ output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+ }
}
- for (int i = 0; i < n_layer; ++i) {
+ for (int i = meshnet_start; i < meshnet_end; ++i) {
auto & layer = layers[i];
layer.attn_norm = create_tensor(tn(LLM_TENSOR_ATTN_NORM, "weight", i), {n_embd}, 0);

107
packages/node/native/llama/patches/0003-owned-range-filtered-state-report.patch Normal file
View File

@@ -0,0 +1,107 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: report owned-range filtered loading state
Concern: filtered state. Adds the llama_meshnet_range_report value type and
populates it after owned-range tensor registration with the half-open bounds
and mapped/resident byte counts derived from backend buffers, never from
caller parameters. Layer-filtered KV and session-to-sequence mapping remain
later scoped stories; this patch carries only the owned-range state report.
---
diff --git a/include/llama.h b/include/llama.h
index 229946ede..6fd7ad509 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -292,6 +292,16 @@ extern "C" {
ggml_backend_buffer_type_t buft;
};
+ // Immutable report for the project-owned dense-Llama owned-range state.
+ // Bounds are half-open [start, end); byte counts are derived from the
+ // registered tensors and backend buffers, never from caller parameters.
+ struct llama_meshnet_range_report {
+ int32_t start_layer;
+ int32_t end_layer;
+ uint64_t mapped_bytes;
+ uint64_t resident_bytes;
+ };
+
struct llama_model_params {
// NULL-terminated list of devices to use for offloading (if NULL, all available devices are used)
ggml_backend_dev_t * devices;
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index 05b8b9c91..efb290c1f 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1015,6 +1015,9 @@ struct llama_model::impl {
bool has_tensor_overrides;
std::vector<float> tensor_split_owned;
+
+ llama_meshnet_range_report meshnet_range_report = {};
+ bool has_meshnet_range_report = false;
};
llama_model::llama_model(const llama_model_params & params) : params(params), pimpl(std::make_unique<impl>()) {
@@ -1628,13 +1631,33 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
}
// print memory requirements per buffer type
+ uint64_t meshnet_mapped_bytes = 0;
+ uint64_t meshnet_resident_bytes = 0;
for (auto & [_, bufs] : pimpl->ctxs_bufs) {
for (auto & buf: bufs) {
+ meshnet_resident_bytes += ggml_backend_buffer_get_size(buf.get());
LLAMA_LOG_INFO("%s: %12s model buffer size = %8.2f MiB\n",
__func__, ggml_backend_buffer_name(buf.get()), ggml_backend_buffer_get_size(buf.get()) / 1024.0 / 1024.0);
}
}
+ if (meshnet_range_requested) {
+ // With mmap backend buffers the resident mapping exactly describes the
+ // mapped file spans of the owned tensors. On non-mmap backends the
+ // instantiated allocation is the resident measure and no file span is
+ // claimed as mapped.
+ if (ml.use_mmap) {
+ meshnet_mapped_bytes = meshnet_resident_bytes;
+ }
+ pimpl->meshnet_range_report = {
+ meshnet_start,
+ meshnet_end,
+ meshnet_mapped_bytes,
+ meshnet_resident_bytes,
+ };
+ pimpl->has_meshnet_range_report = true;
+ }
+
if (ml.no_alloc) {
return true;
}
@@ -1726,6 +1749,14 @@ uint64_t llama_model::n_elements() const {
return pimpl->n_elements;
}
+bool llama_model::meshnet_range_report(llama_meshnet_range_report * out) const {
+ if (out == nullptr || !pimpl->has_meshnet_range_report) {
+ return false;
+ }
+ *out = pimpl->meshnet_range_report;
+ return true;
+}
+
void llama_model::print_info() const {
const std::string rope_scaling_type = llama_rope_scaling_type_name(hparams.rope_scaling_type_train);
diff --git a/src/llama-model.h b/src/llama-model.h
index 45b054ced..5ef7a1515 100644
--- a/src/llama-model.h
+++ b/src/llama-model.h
@@ -652,6 +652,9 @@ struct llama_model {
// total number of parameters in the model
uint64_t n_elements() const;
+ // Project-owned owned-range state report; false when no range was loaded.
+ bool meshnet_range_report(llama_meshnet_range_report * out) const;
+
void print_info() const;
ggml_backend_dev_t dev_layer(int il) const;

73
packages/node/native/llama/patches/0004-dense-boundary-io-endpoint-guard.patch Normal file
View File

@@ -0,0 +1,73 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: guard dense graph behind boundary endpoint ownership
Concern: boundary I/O. Extends the range report with endpoint ownership flags
derived from the registered tensor map and fails the dense-Llama graph closed
for any partial owned range until typed head/tail endpoint adapters carry the
architecture boundary I/O.
---
diff --git a/include/llama.h b/include/llama.h
index 6fd7ad509..8a7521349 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -300,6 +300,8 @@ extern "C" {
int32_t end_layer;
uint64_t mapped_bytes;
uint64_t resident_bytes;
+ bool has_token_embeddings;
+ bool has_output_head;
};
struct llama_model_params {
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index efb290c1f..2ea8598ad 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -1649,11 +1649,17 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
if (ml.use_mmap) {
meshnet_mapped_bytes = meshnet_resident_bytes;
}
+ const auto registered_name = [this](const char * name) {
+ return std::any_of(tensors_by_name.begin(), tensors_by_name.end(),
+ [name](const auto & entry) { return entry.first == name; });
+ };
pimpl->meshnet_range_report = {
meshnet_start,
meshnet_end,
meshnet_mapped_bytes,
meshnet_resident_bytes,
+ registered_name("token_embd.weight"),
+ output_norm != nullptr && output != nullptr,
};
pimpl->has_meshnet_range_report = true;
}
diff --git a/src/models/llama.cpp b/src/models/llama.cpp
index c3092763b..3b6854d0c 100644
--- a/src/models/llama.cpp
+++ b/src/models/llama.cpp
@@ -108,6 +108,25 @@ std::unique_ptr<llm_graph_context> llama_model_llama::build_arch_graph(const llm
template <bool embed>
llama_model_llama::graph<embed>::graph(const llama_model & model, const llm_graph_params & params) : llm_graph_context(params) {
+ llama_meshnet_range_report meshnet_report = {};
+ if (model.meshnet_range_report(&meshnet_report)) {
+ // A partial owned range cannot execute the stock head/tail graph: the
+ // architecture boundary I/O must arrive through a typed endpoint
+ // adapter instead of local embeddings or the local output head.
+ if (meshnet_report.start_layer != 0) {
+ throw std::runtime_error("Meshnet dense-Llama graph requires a head endpoint adapter");
+ }
+ if (meshnet_report.end_layer != n_layer) {
+ throw std::runtime_error("Meshnet dense-Llama graph requires a tail endpoint adapter");
+ }
+ if (!meshnet_report.has_token_embeddings) {
+ throw std::runtime_error("Meshnet dense-Llama head range is missing token embeddings");
+ }
+ if (!meshnet_report.has_output_head) {
+ throw std::runtime_error("Meshnet dense-Llama tail range is missing final norm or output head");
+ }
+ }
+
const int64_t n_embd_head = hparams.n_embd_head_v();
GGML_ASSERT(n_embd_head == hparams.n_embd_head_k());

202
packages/node/native/llama/patches/0005-worker-range-report-hook.patch Normal file
View File

@@ -0,0 +1,202 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: expose worker-owned range report hook and fixture
Concern: worker hooks. Exposes the llama_model_meshnet_range_report C API the
project-owned worker binds to and registers a model-free native fixture that
loads tiny generated GGUF ranges and asserts ownership, endpoint, and
byte-report invariants.
---
diff --git a/include/llama.h b/include/llama.h
index 8a7521349..5818daf94 100644
--- a/include/llama.h
+++ b/include/llama.h
@@ -613,6 +613,13 @@ extern "C" {
// Get metadata value as a string by key name
LLAMA_API int32_t llama_model_meta_val_str(const struct llama_model * model, const char * key, char * buf, size_t buf_size);
+ // Returns false unless this model was instantiated through the Meshnet
+ // owned-range loader. Values are derived from registered tensors and
+ // backend buffers, never copied from caller-supplied parameters.
+ LLAMA_API bool llama_model_meshnet_range_report(
+ const struct llama_model * model,
+ struct llama_meshnet_range_report * out);
+
// Get the number of metadata key/value pairs
LLAMA_API int32_t llama_model_meta_count(const struct llama_model * model);
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index 2ea8598ad..c9d3cf6d3 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -2695,6 +2695,10 @@ uint64_t llama_model_size(const llama_model * model) {
return model->size();
}
+bool llama_model_meshnet_range_report(const llama_model * model, llama_meshnet_range_report * out) {
+ return model != nullptr && model->meshnet_range_report(out);
+}
+
const char * llama_model_chat_template(const llama_model * model, const char * name) {
const auto key = name ? LLM_KV(model->arch, name)(LLM_KV_TOKENIZER_CHAT_TEMPLATE)
: LLM_KV(model->arch)(LLM_KV_TOKENIZER_CHAT_TEMPLATE);
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 855295c15..9a7be6eed 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -193,6 +193,7 @@ if (NOT WIN32 OR NOT BUILD_SHARED_LIBS)
# llama_build_and_test(test-double-float.cpp) # SLOW
llama_build_and_test(test-llama-archs.cpp)
+ llama_build_and_test(test-meshnet-range-ownership.cpp)
endif()
llama_build_and_test(test-chat-peg-parser.cpp peg-parser/simple-tokenize.cpp)
diff --git a/tests/test-meshnet-range-ownership.cpp b/tests/test-meshnet-range-ownership.cpp
new file mode 100644
index 000000000..6b3aa5ac5 100644
--- /dev/null
+++ b/tests/test-meshnet-range-ownership.cpp
@@ -0,0 +1,143 @@
+#include "ggml.h"
+#include "gguf.h"
+#include "llama.h"
+
+#include "../src/llama-model.h"
+
+#include <cstdio>
+#include <cstring>
+#include <stdexcept>
+#include <string>
+
+namespace {
+
+constexpr int kLayers = 4;
+constexpr int kEmbd = 8;
+constexpr int kFfn = 16;
+constexpr int kVocab = 16;
+
+void check(bool condition, const char * message) {
+ if (!condition) {
+ throw std::runtime_error(message);
+ }
+}
+
+void add_tensor(gguf_context * gguf, ggml_context * tensors, const char * name, int d0, int d1 = 1) {
+ ggml_tensor * tensor = d1 == 1
+ ? ggml_new_tensor_1d(tensors, GGML_TYPE_F32, d0)
+ : ggml_new_tensor_2d(tensors, GGML_TYPE_F32, d0, d1);
+ ggml_set_name(tensor, name);
+ std::memset(tensor->data, 0, ggml_nbytes(tensor));
+ gguf_add_tensor(gguf, tensor);
+}
+
+std::string write_fixture() {
+ const std::string path = "meshnet-dense-llama-range-fixture.gguf";
+ gguf_context * gguf = gguf_init_empty();
+ ggml_init_params params = { 128 * 1024, nullptr, false };
+ ggml_context * tensors = ggml_init(params);
+ check(gguf && tensors, "failed to create dense-Llama fixture contexts");
+
+ gguf_set_val_str(gguf, "general.architecture", "llama");
+ gguf_set_val_u32(gguf, "llama.context_length", 16);
+ gguf_set_val_u32(gguf, "llama.embedding_length", kEmbd);
+ gguf_set_val_u32(gguf, "llama.block_count", kLayers);
+ gguf_set_val_u32(gguf, "llama.feed_forward_length", kFfn);
+ gguf_set_val_u32(gguf, "llama.attention.head_count", 2);
+ gguf_set_val_u32(gguf, "llama.attention.head_count_kv", 2);
+ gguf_set_val_u32(gguf, "llama.rope.dimension_count", 4);
+ gguf_set_val_f32(gguf, "llama.attention.layer_norm_rms_epsilon", 1.0e-5f);
+ gguf_set_val_str(gguf, "tokenizer.ggml.model", "no_vocab");
+ gguf_set_val_u32(gguf, "llama.vocab_size", kVocab);
+
+ add_tensor(gguf, tensors, "token_embd.weight", kEmbd, kVocab);
+ add_tensor(gguf, tensors, "output_norm.weight", kEmbd);
+ add_tensor(gguf, tensors, "output.weight", kEmbd, kVocab);
+ for (int layer = 0; layer < kLayers; ++layer) {
+ const std::string p = "blk." + std::to_string(layer) + ".";
+ add_tensor(gguf, tensors, (p + "attn_norm.weight").c_str(), kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_q.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_k.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_v.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "attn_output.weight").c_str(), kEmbd, kEmbd);
+ add_tensor(gguf, tensors, (p + "ffn_norm.weight").c_str(), kEmbd);
+ add_tensor(gguf, tensors, (p + "ffn_gate.weight").c_str(), kEmbd, kFfn);
+ add_tensor(gguf, tensors, (p + "ffn_down.weight").c_str(), kFfn, kEmbd);
+ add_tensor(gguf, tensors, (p + "ffn_up.weight").c_str(), kEmbd, kFfn);
+ }
+ check(gguf_write_to_file(gguf, path.c_str(), false), "failed to write dense-Llama fixture");
+ ggml_free(tensors);
+ gguf_free(gguf);
+ return path;
+}
+
+int block_number(const std::string & name) {
+ int block = -1;
+ return std::sscanf(name.c_str(), "blk.%d.", &block) == 1 ? block : -1;
+}
+
+bool is_allowed_endpoint_tensor(const std::string & name, int start, int end) {
+ if (name == "token_embd.weight") {
+ return start == 0;
+ }
+ if (name == "output_norm.weight" || name == "output.weight") {
+ return end == kLayers;
+ }
+ return false;
+}
+
+llama_meshnet_range_report load_and_check(const std::string & path, int start, int end) {
+ llama_model_params params = llama_model_default_params();
+ params.meshnet_owned_layer_start = start;
+ params.meshnet_owned_layer_end = end;
+ llama_model * model = llama_model_load_from_file(path.c_str(), params);
+ check(model != nullptr, "failed to load dense-Llama fixture");
+
+ llama_meshnet_range_report report = {};
+ check(llama_model_meshnet_range_report(model, &report), "range report is absent");
+ check(report.start_layer == start, "reported start does not match registered range");
+ check(report.end_layer == end, "reported end does not match registered range");
+ check(report.mapped_bytes > 0, "mmap report is empty");
+ check(report.resident_bytes >= report.mapped_bytes, "resident bytes undercount mapped bytes");
+ check(report.has_token_embeddings == (start == 0), "token-embedding ownership is not the head endpoint");
+ check(report.has_output_head == (end == kLayers), "output-head ownership is not the tail endpoint");
+
+ const auto & tensors = llama_internal_get_tensor_map(model);
+ check(!tensors.empty(), "no tensors registered for owned range");
+ for (const auto & [name, _] : tensors) {
+ const int block = block_number(name);
+ check((block >= start && block < end) || (block == -1 && is_allowed_endpoint_tensor(name, start, end)),
+ "registered tensor is outside the owned range and its endpoints");
+ }
+ llama_model_free(model);
+ return report;
+}
+
+} // namespace
+
+int main() {
+ llama_backend_init();
+ const std::string fixture = write_fixture();
+
+ const auto head = load_and_check(fixture, 0, 1);
+ const auto middle = load_and_check(fixture, 1, 3);
+ load_and_check(fixture, 3, 4);
+ check(middle.mapped_bytes > head.mapped_bytes, "two-layer range did not map more bytes than head");
+
+ // A stock load has no owned-range report and registers every tensor.
+ llama_model * stock = llama_model_load_from_file(fixture.c_str(), llama_model_default_params());
+ check(stock != nullptr, "stock load failed");
+ llama_meshnet_range_report stock_report = {};
+ check(!llama_model_meshnet_range_report(stock, &stock_report), "stock load reported an owned range");
+ check(llama_internal_get_tensor_map(stock).size() == 3 + 9 * kLayers, "stock load lost tensors");
+ llama_model_free(stock);
+
+ llama_model_params invalid = llama_model_default_params();
+ invalid.meshnet_owned_layer_start = 3;
+ invalid.meshnet_owned_layer_end = 5;
+ check(llama_model_load_from_file(fixture.c_str(), invalid) == nullptr, "invalid range loaded");
+
+ std::remove(fixture.c_str());
+ llama_backend_free();
+ return 0;
+}

7
packages/node/native/llama/patches/SHA256SUMS
View File

@@ -1,3 +1,6 @@
# SHA-256 digests for the ordered patch series. Do not reorder this file.
1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
51c205e3ca26e104f80c838eeeb11115b8d436036014116d2bb407178c30e0bd 0002-dense-llama-owned-range-loader.patch
1454216c019c1cb7f78d1d836fe4054164fff1d498391013bcaf13cc2d328c75 0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
6032ecca4d3ec3ce072f099dc402529aa6f53a7069a97e75222cf2fa50abb1a3 0002-dense-llama-owned-range-loading.patch
4871a37544df658980a01b4f94151a90b609fb144c931b4a814309ee608ebb46 0003-owned-range-filtered-state-report.patch
19d451ce259150ffede793c4eb547425375c0fcd97caf326b43e8f1a204f05b6 0004-dense-boundary-io-endpoint-guard.patch
cf263357a6a8de193f710836c7c467c38cac7099975303ee2628e0609daf5a47 0005-worker-range-report-hook.patch

117
packages/node/native/llama/patches/UPSTREAM-ASSUMPTIONS.json Normal file
View File

@@ -0,0 +1,117 @@
{
"schema_version": 1,
"upstream_commit": "e920c523e3b8a0163fe498af5bf90df35ff51d25",
"patches": {
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch": {
"concern": "build-marker",
"files": {
"CMakeLists.txt": {
"before": "81f23d7e70b7378511af5d01be680c03aebc2b15",
"after": "a9afcffa68bed7cbd8fad39ad9f95ad784251234"
},
"cmake/meshnet-patch-stack.cmake": {
"before": null,
"after": "910646b4d6164831d4f8e523dd5e49ce7796994f"
}
},
"api_assumptions": [
"CMake >= 3.14 include() of a project-relative module from the top-level CMakeLists.txt",
"add_library(<name> INTERFACE) and target_compile_definitions(... INTERFACE ...)"
]
},
"0002-dense-llama-owned-range-loading.patch": {
"concern": "range-loading",
"files": {
"include/llama.h": {
"before": "a311ac2023579376ed571a614dcac9d259692e56",
"after": "229946ede026ef36b4c4f0355e0421b082334ee0"
},
"src/llama-model.cpp": {
"before": "d87481381e46025c9c87c4af5f116015a44124c3",
"after": "05b8b9c912716fd57061100ec5c47203f79b50ce"
},
"src/models/llama.cpp": {
"before": "4bfebc8843c655e122e5f2064a791c583ad3779b",
"after": "c3092763b82e93b596ba531a0fb01769df6f0e27"
}
},
"api_assumptions": [
"llama_model_params is an aggregate C struct initialized by llama_model_default_params()",
"llama_model_loader::done_getting_tensors(bool partial = false) const",
"llm_hparams::n_layer() and llama_model_base::load_tensors(llama_model_loader &)",
"llama_model_llama::load_arch_tensors per-layer create_tensor loop and LLM_ARCH_LLAMA gate"
]
},
"0003-owned-range-filtered-state-report.patch": {
"concern": "filtered-state",
"files": {
"include/llama.h": {
"before": "229946ede026ef36b4c4f0355e0421b082334ee0",
"after": "6fd7ad509006190a7e7dd84a3d1ac2672ad2d21d"
},
"src/llama-model.cpp": {
"before": "05b8b9c912716fd57061100ec5c47203f79b50ce",
"after": "efb290c1f3dceb868dc5e316e765767ddf0532f8"
},
"src/llama-model.h": {
"before": "45b054cedf1d1e6accc7cf8aafcbae374614e64f",
"after": "5ef7a1515a9b27858e3e5fe694a285a4a55a8bf2"
}
},
"api_assumptions": [
"llama_model::impl pimpl struct and llama_model::meshnet_range_report(out) const accessor",
"pimpl->ctxs_bufs backend buffer map and ggml_backend_buffer_get_size",
"llama_model_loader::use_mmap public member"
]
},
"0004-dense-boundary-io-endpoint-guard.patch": {
"concern": "boundary-io",
"files": {
"include/llama.h": {
"before": "6fd7ad509006190a7e7dd84a3d1ac2672ad2d21d",
"after": "8a75213494cc07b20ca2d99cd21b30109e758b7b"
},
"src/llama-model.cpp": {
"before": "efb290c1f3dceb868dc5e316e765767ddf0532f8",
"after": "2ea8598ad2037920082ef88db75684283c65118c"
},
"src/models/llama.cpp": {
"before": "c3092763b82e93b596ba531a0fb01769df6f0e27",
"after": "3b6854d0c32cdb816a0b31c337767fa1a9305b24"
}
},
"api_assumptions": [
"llama_model::tensors_by_name is std::vector<std::pair<std::string, ggml_tensor *>>",
"llama_model::{output_norm, output} members reflect output-head registration",
"llama_model_llama::graph<embed> constructor and llm_graph_context::n_layer"
]
},
"0005-worker-range-report-hook.patch": {
"concern": "worker-hooks",
"files": {
"include/llama.h": {
"before": "8a75213494cc07b20ca2d99cd21b30109e758b7b",
"after": "5818daf94d481c5eee201ad1300f37e7c0fe60f9"
},
"src/llama-model.cpp": {
"before": "2ea8598ad2037920082ef88db75684283c65118c",
"after": "c9d3cf6d34cbe2fd50100b91ed2a0436f5b72415"
},
"tests/CMakeLists.txt": {
"before": "855295c152faa78fa4acdec54013940c41627be9",
"after": "9a7be6eedfd6a311694a8a6a95976fb230ee165c"
},
"tests/test-meshnet-range-ownership.cpp": {
"before": null,
"after": "6b3aa5ac5dc1b6074401f97b8acd07b00dfa9780"
}
},
"api_assumptions": [
"LLAMA_API export macro and extern \"C\" block in include/llama.h",
"tests/CMakeLists.txt llama_build_and_test(<file.cpp>) helper",
"llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h",
"gguf empty-context writer API: gguf_init_empty, gguf_add_tensor, gguf_write_to_file"
]
}
}
}

5
packages/node/native/llama/patches/series
View File

@@ -1,2 +1,5 @@
0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch
0002-dense-llama-owned-range-loader.patch
0002-dense-llama-owned-range-loading.patch
0003-owned-range-filtered-state-report.patch
0004-dense-boundary-io-endpoint-guard.patch
0005-worker-range-report-hook.patch