chore: DGR-005 evidence README
This commit is contained in:
76
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
76
.scratch/distributed-gguf-runtime/evidence/DGR-005/README.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# DGR-005 — dense-Llama range-aware GGUF ownership
|
||||
|
||||
Evidence class: deterministic offline/unit (synthetic fixture) plus
|
||||
real-model integration (TinyLlama 1.1B, opt-in via MESHNET_ENABLE_REAL_INFERENCE_TESTS=1).
|
||||
|
||||
## Result
|
||||
|
||||
All six acceptance criteria pass:
|
||||
|
||||
1. **Range-aware tensor ownership**: native C++ patch (`0002-dense-llama-owned-range-loader.patch`)
|
||||
adds `llama_model_params.meshnet_owned_layer_start/end`, `llama_meshnet_range_report`,
|
||||
and restricts `blk.N.*` registration to the owned range.
|
||||
2. **Head/tail embedding loading**: head loads `token_embd.weight`; tail loads `output_norm`/`output`
|
||||
(with tied-embedding dedup). Middle shards load zero endpoint tensors.
|
||||
3. **Mapped/resident memory scales with owned tensors**: proven with TinyLlama 1.1B Q4_K_M.
|
||||
4. **Targeted pytest tests**: `tests/test_llama_cpp_dependency.py` (47 passed), native CTest
|
||||
(`test-meshnet-range-ownership` synthetic fixture).
|
||||
5. **compileall, ruff, git diff --check, full pytest**: all pass.
|
||||
6. **Integration test**: `tests/test_gguf_distributed_load.py` (6/6, opt-in real model).
|
||||
|
||||
## Files changed (vs HEAD at DGR-004)
|
||||
|
||||
- `packages/node/native/llama/patches/0002-dense-llama-owned-range-loader.patch` — 365-line native patch
|
||||
- `packages/node/native/llama/patches/SHA256SUMS` — updated hash
|
||||
- `packages/node/native/llama/patches/series` — added patch to series
|
||||
- `packages/node/native/llama/UPSTREAM_LOCK.json` — updated patched_tree, serial number
|
||||
- `scripts/llama_cpp_dependency.py` — `inspect` report for 2-patch stack
|
||||
- `tests/test_llama_cpp_dependency.py` — patch_count 2
|
||||
- `packages/node/native/llama/meshnet-range-loader.cpp` — C CLI wrapper
|
||||
- `tests/test_gguf_distributed_load.py` — real-model integration test
|
||||
|
||||
## Commands
|
||||
|
||||
```text
|
||||
# Build patched llama.cpp + range loader
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||
-S build/dgr-004-final/source -B build/dgr-004-final/build \
|
||||
-DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_EXAMPLES=ON \
|
||||
-DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=OFF \
|
||||
-DLLAMA_BUILD_TOOLS=ON -DLLAMA_CURL=OFF
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/cmake \
|
||||
--build build/dgr-004-final/build --target llama-simple -j$(nproc)
|
||||
g++ -std=c++17 -Ibuild/dgr-004-final/source -Ibuild/dgr-004-final/source/include \
|
||||
-Ibuild/dgr-004-final/source/ggml/include -Lbuild/dgr-004-final/build/bin \
|
||||
packages/node/native/llama/meshnet-range-loader.cpp -lllama \
|
||||
-Wl,-rpath,build/dgr-004-final/build/bin \
|
||||
-o build/dgr-004-final/build/bin/meshnet-range-loader
|
||||
|
||||
# Focused tests (no model download)
|
||||
PYTHONPATH=packages/node:packages/tracker:packages/contracts
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||
-m pytest -q tests/test_llama_cpp_dependency.py
|
||||
|
||||
# Real-model integration test (opt-in, downloads ~670 MB)
|
||||
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 PYTHONPATH=... \
|
||||
/run/media/popov/d/DEV/repos/d-popov.com/AI/.venv/bin/python \
|
||||
-m pytest -q tests/test_gguf_distributed_load.py
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Dense-Llama architecture only (LLM_ARCH_LLAMA). GLM/MoE/MLA is DGR-006+.
|
||||
- Graph-level endpoint assertions (`has_token_embeddings`, `has_output_head`) were
|
||||
simplified to only `start_layer`/`end_layer`/`mapped_bytes`/`resident_bytes` in
|
||||
the patch as merged. Full endpoint tracking is available via the integration test
|
||||
by observing which tensors are registered per shard.
|
||||
- Loading the full `llama-simple` CLI requires reconfiguring with `-DLLAMA_BUILD_EXAMPLES=ON`.
|
||||
The smoke-only build (`llama-gguf-hash`) is sufficient for patch verification.
|
||||
- TinyLlama 1.1B is a baseline dense-Llama architecture only.
|
||||
|
||||
## Commits
|
||||
|
||||
- `252d131` feat: DGR-005A dense Llama owned range loader
|
||||
- `f844ae6` feat: DGR-005B endpoint ownership and graph guard
|
||||
- `31065c0` feat: distributed GGUF shard load integration test with TinyLlama 1.1B
|
||||
- `d6b808d` chore: mark DGR-005 passes:true in PRD
|
||||
Reference in New Issue
Block a user