78 lines
3.9 KiB
Markdown
78 lines
3.9 KiB
Markdown
# DGR-037 evidence — bind llama.cpp to the standalone worker
|
|
|
|
**Date:** 2026-08-01
|
|
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
|
|
`false` until the opt-in real-model worker lane and native CMake/CTest lane run).
|
|
|
|
## Implemented
|
|
|
|
- Replaced the native worker's `FakeShardEngine` member with a private C++
|
|
`ShardEngine` implementation backed by the pinned, patched llama.cpp API.
|
|
`LlamaShardEngine` owns `llama_model` and backend lifetime; neither type is
|
|
visible to the gRPC service interface.
|
|
- Startup now requires one node-provided artifact path/digest, recipe digest,
|
|
recipe/catalogue identity, and half-open layer range. It loads the artifact
|
|
with the pinned range-loader parameters and rejects startup unless
|
|
`llama_model_meshnet_range_report` attests the same range.
|
|
- `GetCapability`, `Health`, and `SessionOpen` derive identity/range and
|
|
resident memory from the loaded engine. An open must name the exact loaded
|
|
range and compatible artifact/recipe digests; stream values cannot select a
|
|
different artifact or range.
|
|
- Prefill/decode validation and admitted execution route through
|
|
`ShardEngine::Validate` / `ShardEngine::Execute`; session release reaches the
|
|
engine and process shutdown releases the model/backend handles.
|
|
- Added the opt-in `MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS` test hook.
|
|
The worker exits `70` after the configured admitted operation so DGR-040's
|
|
supervisor can observe bounded process death without an in-process recovery
|
|
path.
|
|
|
|
## Changed files
|
|
|
|
- `packages/node/native/CMakeLists.txt`
|
|
- `packages/node/native/README.md`
|
|
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
|
|
- `packages/node/native/worker/shard_service.{h,cpp}`
|
|
- `packages/node/native/worker/shard_worker_main.cpp`
|
|
- `tests/test_llama_shard_worker_binding.py`
|
|
|
|
## Commands and results
|
|
|
|
```text
|
|
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
|
|
Applied the exact local DGR-027 patch stack; the resulting header exposed
|
|
meshnet_owned_layer_start/end and llama_model_meshnet_range_report.
|
|
|
|
c++ -std=c++17 -fsyntax-only [llama_shard_engine.cpp, shard_service.cpp, shard_worker_main.cpp]
|
|
All three translation units passed syntax checking. The gRPC toolchain emitted
|
|
only its existing deprecation warnings.
|
|
|
|
/home/popov/.hermes/hermes-agent/venv/bin/cmake -S packages/node/native -B build/native-dgr037 \
|
|
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain" \
|
|
-DMESHNET_LLAMA_SOURCE_DIR="$PWD/build/llama.cpp/source" \
|
|
-DMESHNET_LLAMA_LIBRARY_DIR="$PWD/build/llama.cpp/build/bin"
|
|
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
|
|
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
|
|
shard_worker built successfully; 1/1 shard_protocol_conformance passed.
|
|
|
|
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
|
|
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
|
|
53 passed, 2 skipped
|
|
|
|
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
|
|
git diff --check
|
|
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
|
compileall passed; diff check passed; OK: 55 stories validated
|
|
```
|
|
|
|
## Limitations and dependency handoff
|
|
|
|
- No model artifact was selected for this session, so no opt-in real-model
|
|
process run, process-death observation, or raw hardware metrics are claimed.
|
|
- The pinned API currently attests range ownership/loading. Its typed
|
|
dense-boundary graph bridge remains intentionally separated from generated
|
|
wire bytes; DGR-038 owns per-session local KV/context state and DGR-039 owns
|
|
the real two-process range-parity exercise.
|
|
- DGR-040 can supervise this worker using its readiness line, health identity,
|
|
clean SIGTERM shutdown, and deterministic exit-70 injection hook. DGR-038
|
|
must make `ReleaseSession` dispose of local llama sequence/KV resources.
|