popov/neuron-tai
1
0
Fork 0
Code Issues 42 Pull Requests Actions Packages Projects Releases Wiki Activity

story: DGR-034 Implement dense-Llama range-aware GGUF ownership

Browse Source
This commit is contained in:
Dobromir Popov
2026-08-01 01:08:28 +03:00
parent 27a0d89678
commit d339cfde25
18 changed files with 1393 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

94
.scratch/distributed-gguf-runtime/evidence/DGR-034/README.md Normal file
View File

@@ -0,0 +1,94 @@
# DGR-034 evidence — dense-Llama range-aware GGUF ownership
**Status:** implemented and live-verified on 2026-08-01. `prd.json` remains
the authority for story state.
## What changed
- The pinned llama.cpp patch stack adds `meshnet_owned_layer_start/end` and
filters dense-Llama GGUF registration to `blk.N.*` for the requested
half-open range. `token_embd.weight` belongs to the head; `output_norm` and
`output.weight` (or the tied embedding) belong to the tail.
- The load state exposes a C range report derived from the registered model
buffers, and a project-owned `meshnet-range-report` tool audits the live
registered tensor map. It rejects empty, inverted, out-of-model, missing,
outside-range, unexpected, and endpoint-inconsistent loads.
- `meshnet_node.range_report` accepts only audited tool output. It makes the
range and endpoint flags authoritative from loaded state rather than caller
assertions, and fails closed on malformed ownership or byte counts.
## Real-model memory evidence
Artifact: `Magistral-Small-2509-Q4_K_M.gguf`, 14,333,911,104 bytes, SHA-256
`a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc`.
It stayed on the configured mounted drive; no artifact was downloaded or put
under `/home`.
The direct non-mmap lane proves resident storage tracks owned tensors:
| Range | Registered tensors | Resident bytes | Process peak RSS |
| --- | ---: | ---: | ---: |
| `[10, 20)` | 90 | 3,304,898,560 | 3,298,800 KiB |
| `[0, 40)` | 363 | 14,326,026,240 | 14,061,632 KiB |
Raw reports and timings are in `runs/default-mid-a.*` and
`runs/default-full-nommap.*`. The middle range is 23.1% of the full
resident allocation and owns 24.8% of the registered tensors.
## Commands and results
```text
python3 scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
python3 scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# apply/check/reverse succeeded against e920c523e3b8a0163fe498af5bf90df35ff51d25;
# the source was then applied for the focused native checks.
(cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS)
# all six patches: OK
/home/popov/.hermes/hermes-agent/venv/bin/ctest \
--test-dir build/llama.cpp/dgr034-check \
-R '^test-meshnet-range-ownership$' --output-on-failure
# 1/1 passed
PYTHONPATH=packages/node MESHNET_RANGE_REPORT_BIN="$PWD/build/llama.cpp/dgr034-check/bin/meshnet-range-report" \
/home/popov/.hermes/hermes-agent/venv/bin/pytest -q \
tests/test_range_report.py tests/test_meshnet_range_report_tool.py \
tests/test_llama_cpp_dependency.py
# 56 passed in 0.87s
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python \
-m compileall -q packages tests
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
git diff --check && git diff --cached --check
# all exit 0; PRD validation: 55 stories validated
```
The model commands used the same `meshnet-range-report` binary with
`--no-mmap --no-extra-bufts`, first for `[10,20)` and then `[0,40)`; both
returned `ok: true` and their exact output is retained above.
## Changed files
- `packages/node/native/llama/PATCH-STACK.md`
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/patches/{series,SHA256SUMS,UPSTREAM-ASSUMPTIONS.json,0006-meshnet-range-report-tool.patch}`
- `packages/node/meshnet_node/range_report.py`
- `tests/test_range_report.py`
- `tests/test_meshnet_range_report_tool.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-034/*`
## Limitations and dependency handoff
- The mmap loader can retain broad contiguous file spans when GGUF tensor
order places a tail endpoint near the beginning of the artifact; the direct
non-mmap lane is the certified resident-memory result. The raw mmap report
is retained in `runs/default-head.json` and must not be presented as a
physical-RSS saving.
- This story proves loading/ownership only. Partial-range graph execution
remains fail-closed until DGR-035 provides typed dense boundary adapters.
- DGR-037 can bind the worker to `llama_model_meshnet_range_report` or the
strict Python consumer; it must use the reported range, not requested range,
for capability publication. DGR-051 must add its V4-specific ownership
rules separately.

1
.scratch/distributed-gguf-runtime/evidence/DGR-034/artifact-sha256.txt Normal file
View File

@@ -0,0 +1 @@
a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc Magistral-Small-2509-Q4_K_M.gguf

24
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-full-nommap.json Normal file
View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 40],
"reported_range": [0, 40],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": true,
"has_output_head": true,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 14326026240,
"registered_tensors": 363,
"registered_bytes": 14326026240,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 14392061952,
"vm_rss_bytes": 14387003392,
"vm_hwm_bytes": 14399111168
}

1
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-full-nommap.stderr Normal file
View File

@@ -0,0 +1 @@
elapsed=0:02.48 maxrss_kib=14061632 exit=0

24
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-head.json Normal file
View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 10],
"reported_range": [0, 10],
"mmap": true,
"touched": false,
"use_extra_bufts": true,
"has_token_embeddings": true,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 6219366400,
"resident_bytes": 6219366400,
"registered_tensors": 91,
"registered_bytes": 3771596800,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 16942260224,
"vm_rss_bytes": 16937005056,
"vm_hwm_bytes": 16947953664
}

0
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-head.stderr Normal file
View File

24
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-mid-a.json Normal file
View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [10, 20],
"reported_range": [10, 20],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": false,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 3304898560,
"registered_tensors": 90,
"registered_bytes": 3304898560,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 3370934272,
"vm_rss_bytes": 3365814272,
"vm_hwm_bytes": 3377971200
}

1
.scratch/distributed-gguf-runtime/evidence/DGR-034/runs/default-mid-a.stderr Normal file
View File

@@ -0,0 +1 @@
elapsed=0:00.82 maxrss_kib=3298800 exit=0

5
.scratch/distributed-gguf-runtime/prd.json
View File

@@ -954,13 +954,14 @@
"Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
],
"passes": false,
"passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md; prd.json is authoritative.",
"blocks": [
"DGR-035",
"DGR-037",
"DGR-051"
]
],
"completionNotes": "Dense-Llama owned-range loading is verified with strict engine-state reports, focused native/Python tests, and a real GGUF resident-memory comparison."
},
{
"id": "DGR-035",