67 lines
3.1 KiB
Markdown
67 lines
3.1 KiB
Markdown
# DGR-038 evidence — isolated shard-local Hot KV State
|
|
|
|
**Date:** 2026-08-01
|
|
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
|
|
`false` until the opt-in real-model concurrency lane runs).
|
|
|
|
## Implemented
|
|
|
|
- The native `LlamaShardEngine` now creates one bounded llama.cpp context and
|
|
assigns a distinct `llama_seq_id` to each `(route_session_id, route_epoch)`.
|
|
It never accepts remote KV data; the loaded, range-attested llama model owns
|
|
the local cache layout and layers.
|
|
- Prefill/decode append state tracks local positions and expected past length.
|
|
A re-prefill at an earlier position truncates only that sequence with
|
|
`llama_memory_seq_rm`; a discontinuity or past-length mismatch returns a
|
|
retryable `CACHE_MISS`. Older route epochs return `EPOCH_STALE`.
|
|
- The token-reservation budget is bounded by per-session context, total Hot KV
|
|
budget, maximum sequence count, TTL, and LRU. Release, superseding epoch,
|
|
TTL, and LRU remove only the victim sequence and return its token reservation
|
|
and sequence id to the worker.
|
|
- The gRPC service converts native cache/stale/resource results to the typed
|
|
protocol errors and does not consume idempotency/flow-control credit on a
|
|
rejected append. Release is epoch-specific, so a stale release cannot erase
|
|
the active epoch's service state.
|
|
- Added opt-in configuration: `MESHNET_HOT_KV_MAX_SESSIONS`,
|
|
`MESHNET_HOT_KV_CONTEXT_TOKENS`, `MESHNET_HOT_KV_BUDGET_TOKENS`, and
|
|
`MESHNET_HOT_KV_TTL_SECONDS`.
|
|
|
|
## Changed files
|
|
|
|
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
|
|
- `packages/node/native/worker/shard_service.cpp`
|
|
- `packages/node/native/worker/shard_worker_main.cpp`
|
|
- `tests/test_llama_shard_worker_binding.py`
|
|
|
|
## Commands and results
|
|
|
|
```text
|
|
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
|
|
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
|
|
54 passed, 2 skipped
|
|
|
|
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
|
|
shard_worker built successfully against the pinned, patched llama.cpp source.
|
|
|
|
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
|
|
1/1 shard_protocol_conformance passed.
|
|
|
|
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
|
|
git diff --check
|
|
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
|
compileall and diff check passed; OK: 55 stories validated.
|
|
```
|
|
|
|
## Limitations and dependency handoff
|
|
|
|
- DGR-037 supplied the range-attested native model/engine boundary. DGR-038
|
|
adds local sequence ownership without changing its artifact or range
|
|
identity contract.
|
|
- No mounted GGUF artifact was selected. Therefore no opt-in real-model
|
|
four-session run, actual llama KV byte measurement, or hardware metrics are
|
|
claimed. The default tests intentionally remain model-download-free and the
|
|
source `prd.json` remains `passes: false`.
|
|
- DGR-039 should exercise the real two-process range-parity lane with four
|
|
sessions and the Hot-KV environment bounds, recording actual cache memory
|
|
and cancellation isolation evidence.
|