story: DGR-037 Bind llama.cpp to the standalone worker
This commit is contained in:
77
.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md
Normal file
77
.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# DGR-037 evidence — bind llama.cpp to the standalone worker
|
||||
|
||||
**Date:** 2026-08-01
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
|
||||
`false` until the opt-in real-model worker lane and native CMake/CTest lane run).
|
||||
|
||||
## Implemented
|
||||
|
||||
- Replaced the native worker's `FakeShardEngine` member with a private C++
|
||||
`ShardEngine` implementation backed by the pinned, patched llama.cpp API.
|
||||
`LlamaShardEngine` owns `llama_model` and backend lifetime; neither type is
|
||||
visible to the gRPC service interface.
|
||||
- Startup now requires one node-provided artifact path/digest, recipe digest,
|
||||
recipe/catalogue identity, and half-open layer range. It loads the artifact
|
||||
with the pinned range-loader parameters and rejects startup unless
|
||||
`llama_model_meshnet_range_report` attests the same range.
|
||||
- `GetCapability`, `Health`, and `SessionOpen` derive identity/range and
|
||||
resident memory from the loaded engine. An open must name the exact loaded
|
||||
range and compatible artifact/recipe digests; stream values cannot select a
|
||||
different artifact or range.
|
||||
- Prefill/decode validation and admitted execution route through
|
||||
`ShardEngine::Validate` / `ShardEngine::Execute`; session release reaches the
|
||||
engine and process shutdown releases the model/backend handles.
|
||||
- Added the opt-in `MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS` test hook.
|
||||
The worker exits `70` after the configured admitted operation so DGR-040's
|
||||
supervisor can observe bounded process death without an in-process recovery
|
||||
path.
|
||||
|
||||
## Changed files
|
||||
|
||||
- `packages/node/native/CMakeLists.txt`
|
||||
- `packages/node/native/README.md`
|
||||
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
|
||||
- `packages/node/native/worker/shard_service.{h,cpp}`
|
||||
- `packages/node/native/worker/shard_worker_main.cpp`
|
||||
- `tests/test_llama_shard_worker_binding.py`
|
||||
|
||||
## Commands and results
|
||||
|
||||
```text
|
||||
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
|
||||
Applied the exact local DGR-027 patch stack; the resulting header exposed
|
||||
meshnet_owned_layer_start/end and llama_model_meshnet_range_report.
|
||||
|
||||
c++ -std=c++17 -fsyntax-only [llama_shard_engine.cpp, shard_service.cpp, shard_worker_main.cpp]
|
||||
All three translation units passed syntax checking. The gRPC toolchain emitted
|
||||
only its existing deprecation warnings.
|
||||
|
||||
/home/popov/.hermes/hermes-agent/venv/bin/cmake -S packages/node/native -B build/native-dgr037 \
|
||||
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain" \
|
||||
-DMESHNET_LLAMA_SOURCE_DIR="$PWD/build/llama.cpp/source" \
|
||||
-DMESHNET_LLAMA_LIBRARY_DIR="$PWD/build/llama.cpp/build/bin"
|
||||
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
|
||||
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
|
||||
shard_worker built successfully; 1/1 shard_protocol_conformance passed.
|
||||
|
||||
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
|
||||
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
|
||||
53 passed, 2 skipped
|
||||
|
||||
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
|
||||
git diff --check
|
||||
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||
compileall passed; diff check passed; OK: 55 stories validated
|
||||
```
|
||||
|
||||
## Limitations and dependency handoff
|
||||
|
||||
- No model artifact was selected for this session, so no opt-in real-model
|
||||
process run, process-death observation, or raw hardware metrics are claimed.
|
||||
- The pinned API currently attests range ownership/loading. Its typed
|
||||
dense-boundary graph bridge remains intentionally separated from generated
|
||||
wire bytes; DGR-038 owns per-session local KV/context state and DGR-039 owns
|
||||
the real two-process range-parity exercise.
|
||||
- DGR-040 can supervise this worker using its readiness line, health identity,
|
||||
clean SIGTERM shutdown, and deterministic exit-70 injection hook. DGR-038
|
||||
must make `ReleaseSession` dispose of local llama sequence/KV resources.
|
||||
Reference in New Issue
Block a user