neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md

DGR-037 evidence — bind llama.cpp to the standalone worker

Date: 2026-08-01
Authority: .scratch/distributed-gguf-runtime/prd.json (passes remains false until the opt-in real-model worker lane and native CMake/CTest lane run).

Implemented

• Replaced the native worker's FakeShardEngine member with a private C++ ShardEngine implementation backed by the pinned, patched llama.cpp API. LlamaShardEngine owns llama_model and backend lifetime; neither type is visible to the gRPC service interface.
• Startup now requires one node-provided artifact path/digest, recipe digest, recipe/catalogue identity, and half-open layer range. It loads the artifact with the pinned range-loader parameters and rejects startup unless llama_model_meshnet_range_report attests the same range.
• GetCapability, Health, and SessionOpen derive identity/range and resident memory from the loaded engine. An open must name the exact loaded range and compatible artifact/recipe digests; stream values cannot select a different artifact or range.
• Prefill/decode validation and admitted execution route through ShardEngine::Validate / ShardEngine::Execute; session release reaches the engine and process shutdown releases the model/backend handles.
• Added the opt-in MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS test hook. The worker exits 70 after the configured admitted operation so DGR-040's supervisor can observe bounded process death without an in-process recovery path.

Changed files

• packages/node/native/CMakeLists.txt
• packages/node/native/README.md
• packages/node/native/worker/llama_shard_engine.{h,cpp}
• packages/node/native/worker/shard_service.{h,cpp}
• packages/node/native/worker/shard_worker_main.cpp
• tests/test_llama_shard_worker_binding.py

Commands and results

python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
Applied the exact local DGR-027 patch stack; the resulting header exposed
meshnet_owned_layer_start/end and llama_model_meshnet_range_report.

c++ -std=c++17 -fsyntax-only [llama_shard_engine.cpp, shard_service.cpp, shard_worker_main.cpp]
All three translation units passed syntax checking. The gRPC toolchain emitted
only its existing deprecation warnings.

/home/popov/.hermes/hermes-agent/venv/bin/cmake -S packages/node/native -B build/native-dgr037 \
  -DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain" \
  -DMESHNET_LLAMA_SOURCE_DIR="$PWD/build/llama.cpp/source" \
  -DMESHNET_LLAMA_LIBRARY_DIR="$PWD/build/llama.cpp/build/bin"
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
shard_worker built successfully; 1/1 shard_protocol_conformance passed.

PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
  tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
53 passed, 2 skipped

PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
compileall passed; diff check passed; OK: 55 stories validated

Limitations and dependency handoff

• No model artifact was selected for this session, so no opt-in real-model process run, process-death observation, or raw hardware metrics are claimed.
• The pinned API currently attests range ownership/loading. Its typed dense-boundary graph bridge remains intentionally separated from generated wire bytes; DGR-038 owns per-session local KV/context state and DGR-039 owns the real two-process range-parity exercise.
• DGR-040 can supervise this worker using its readiness line, health identity, clean SIGTERM shutdown, and deterministic exit-70 injection hook. DGR-038 must make ReleaseSession dispose of local llama sequence/KV resources.