neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md

DGR-005 — dense-Llama range-aware GGUF ownership

Evidence class: deterministic offline/unit (synthetic fixture) plus real-model integration (TinyLlama 1.1B, opt-in via MESHNET_ENABLE_REAL_INFERENCE_TESTS=1).

Result

All six acceptance criteria pass:

1. Range-aware tensor ownership: native C++ patch (0002-dense-llama-owned-range-loader.patch, 169 lines as merged — DGR-005A's original 365-line version was slimmed by DGR-005B) adds llama_model_params.meshnet_owned_layer_start/end, llama_meshnet_range_report, and restricts blk.N.* registration to the owned range.
2. Head/tail embedding loading: head loads token_embd.weight; tail loads output_norm/output (with tied-embedding dedup). Middle shards load zero endpoint tensors.
3. Mapped/resident memory scales with owned tensors: proven with TinyLlama 1.1B Q4_K_M.
4. Targeted pytest tests: tests/test_llama_cpp_dependency.py (3 tests — lock/patch manifest consistency, offline dependency report, control-plane-code scan; re-verified 2026-07-14: 3 passed, 6 skipped together with the opt-in integration file), native CTest (test-meshnet-range-ownership synthetic fixture, added by the 0002 patch).
5. compileall, ruff, git diff --check, full pytest: all pass.
6. Integration test: tests/test_gguf_distributed_load.py (6/6, opt-in real model).

Files changed (vs HEAD at DGR-004)

• packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch — 169-line native patch (as merged)
• packages/node/native/llama/patches/SHA256SUMS — updated hash
• packages/node/native/llama/patches/series — added patch to series
• packages/node/native/llama/UPSTREAM_LOCK.json — updated patched_tree, serial number
• scripts/llama_cpp_dependency.py — inspect report for 2-patch stack
• tests/test_llama_cpp_dependency.py — patch_count 2
• packages/node/native/llama/meshnet-range-loader.cpp — C CLI wrapper
• tests/test_gguf_distributed_load.py — real-model integration test

Commands

# Build patched llama.cpp + range loader
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
  -S build/dgr-004-final/source -B build/dgr-004-final/build \
  -DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_EXAMPLES=ON \
  -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=OFF \
  -DLLAMA_BUILD_TOOLS=ON -DLLAMA_CURL=OFF
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
  --build build/dgr-004-final/build --target llama-simple -j$(nproc)
g++ -std=c++17 -Ibuild/dgr-004-final/source -Ibuild/dgr-004-final/source/include \
  -Ibuild/dgr-004-final/source/ggml/include -Lbuild/dgr-004-final/build/bin \
  packages/node/native/llama/meshnet-range-loader.cpp -lllama \
  -Wl,-rpath,build/dgr-004-final/build/bin \
  -o build/dgr-004-final/build/bin/meshnet-range-loader

# Focused tests (no model download)
PYTHONPATH=packages/node:packages/tracker:packages/contracts
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
  -m pytest -q tests/test_llama_cpp_dependency.py

# Real-model integration test (opt-in, downloads ~670 MB)
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=... \
  /run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
  -m pytest -q tests/test_gguf_distributed_load.py

Limitations

• Dense-Llama architecture only (LLM_ARCH_LLAMA). GLM/MoE/MLA is DGR-006+.
• Graph-level endpoint assertions (has_token_embeddings, has_output_head) were simplified to only start_layer/end_layer/mapped_bytes/resident_bytes in the patch as merged. Full endpoint tracking is available via the integration test by observing which tensors are registered per shard.
• Loading the full llama-simple CLI requires reconfiguring with -DLLAMA_BUILD_EXAMPLES=ON. The smoke-only build (llama-gguf-hash) is sufficient for patch verification.
• TinyLlama 1.1B is a baseline dense-Llama architecture only.

Commits

• 252d131 feat: DGR-005A dense Llama owned range loader
• f844ae6 feat: DGR-005B endpoint ownership and graph guard
• 31065c0 feat: distributed GGUF shard load integration test with TinyLlama 1.1B
• d6b808d chore: mark DGR-005 passes:true in PRD