neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-031/README.md

# DGR-031 evidence — the project-owned `ShardEngine` interface

**Completed:** 2026-07-23
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-021 (`evidence/DGR-021/README.md` — versioned activation
envelope, `NamedTensor`/`ActivationEnvelope` as the project-owned wire-envelope
layer), DGR-025 (`evidence/DGR-025/README.md` — exact artifact/runtime recipe
identity; both read before changing code).

## Objective

Isolate worker/protocol code from llama.cpp internals behind a stable
project-owned engine contract, so a fake fixture engine (DGR-032) and a real
llama.cpp-backed engine (DGR-037) are interchangeable subclasses of one
interface.

## What was found live before changing code

Per RALPH-CONTEXT, legacy pass states were not trusted; the live surrounding
contracts were read and exercised before designing this one:

- `packages/node/meshnet_node/shard_lifecycle.py` (DGR-022) already defines a
  versioned RPC/session lifecycle contract — `StructuredStatus`, `StatusCode`,
  `CacheExpectation`, `CacheResult`, `LifecycleState`, `SessionLifecycle` — but
  it is explicitly the *wire RPC* contract "consumed by a future generated
  gRPC binding," not an execution-engine boundary.
- `packages/node/meshnet_node/native_backend.py` (DGR-025) is the identity
  boundary for the native GGUF artifact — it derives and attests a
  `ShardIdentity`, but does not define an execution contract either.
- `packages/node/meshnet_node/protocol.py` (DGR-021) defines a project-owned
  `NamedTensor`/`ActivationEnvelope` for activation traffic *between shard
  hops over the network*, distinct from the generated-protobuf wire ABI in
  `native_protocol`.
- `packages/node/meshnet_node/shard_runtime_server.py` (DGR-024) is today a
  real gRPC servicer that proves wire fidelity by checksumming and echoing
  bytes — it has no execution engine behind it yet; that seam is exactly
  where `ShardEngine` plugs in for DGR-037.
- `packages/node/meshnet_node/architecture_boundary.py` established the
  precedent this story follows for tail output: `TailOutput.sampled_token()`
  never exposes raw logits, only a sampled token id.
- No `ShardEngine` (or `shard_engine`) symbol existed anywhere in the
  repository prior to this story (confirmed by
  `grep -rn -i "shardengine\|shard_engine"` across `.py`/`.md`, which returned
  only planning-document prose naming it as future work).

Live verification of the pre-existing dependency contracts before adding new
code: `PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q
tests/test_shard_lifecycle.py tests/test_activation_envelope.py
tests/test_architecture_boundary.py tests/test_native_shard_protocol.py
tests/test_shard_runtime_harness.py` → `95 passed, 3 skipped`.

## What was added (this story's change)

### `packages/node/meshnet_node/shard_engine.py` (new)

The `ShardEngine` boundary: an `abc.ABC` with eight abstract operations —
`load`, `capabilities`, `prefill`, `decode`, `cancel`, `release`, `health`,
`metrics` — matching the acceptance criterion's list exactly (`prefill`/
`decode` share one operation family; their shared result type is what the
criterion calls the "boundary/logits result"). Every request/result type is a
frozen dataclass built from plain `str`/`int`/`bytes`/`Mapping` values:

- `EngineTensor` / `BoundaryBundle` — the project-owned named-tensor
  activation crossing a shard boundary (head/middle/tail-in). Deliberately a
  *new*, minimal type distinct from both `native_protocol.pb.TensorBundle`
  (generated-protobuf ABI) and `protocol.NamedTensor`/`ActivationEnvelope`
  (wire-framing/fragmentation concerns irrelevant to model execution) — a
  fourth, execution-facing layer underneath the three that already existed.
- `TokenOutput` — a tail shard's sampled result: a token id (+ optional
  decoded text), never a raw logits tensor.
- `MtpHook` — reserved multi-token-prediction hook; its own `__post_init__`
  raises if constructed with `enabled=True`, so the type exists (fixing its
  field shape for DGR-051/DGR-066) without any code path being able to turn it
  on before DGR-066, matching RALPH-CONTEXT's "MTP is reserved and off for
  alpha."
- `ArchitectureAuxStateHook` — reserved per-shard architecture auxiliary state
  (V4 CSA/HCA/SWA/indexer/compressor and similar); has no wire encoding and is
  never embedded in a `BoundaryBundle`, matching RALPH-CONTEXT's "remain local
  ... never carried over the WAN seam."
- `LoadRequest`/`LoadResult`, `EngineCapabilities`, `PrefillRequest`/
  `DecodeRequest` (exactly one of `token_ids`/`token_id` (head) or `input`
  (middle/tail) required — enforced in `__post_init__`), `StepResult` (a
  successful result must carry an output; `cache_result` reuses
  `shard_lifecycle.CacheResult`), `HealthResult`, `MetricsResult`.
- Status vocabulary is reused, not reinvented: `StructuredStatus`/
  `StatusCode`/`CacheExpectation`/`CacheResult` are imported from
  `shard_lifecycle` (already project-owned and version-stable) rather than a
  parallel enum living alongside it.
- The module imports nothing from `native_protocol`, `grpc`, or `ctypes` —
  verified structurally, not just by convention (see tests below).

### `tests/shard_engine_contract.py` (new)

A reusable, non-`test_`-prefixed helper: `assert_shard_engine_contract(make_engine)`
takes a zero-arg engine factory and runs nine lifecycle checks — health before
load, load→capabilities range/MTP-off, prefill→decode determinism (byte-identical
output replayed on a fresh session), middle-shard boundary-bundle-in/out vs.
head/tail token-output, deterministic cache-miss on an unopened session,
stale-route-epoch rejection, cancel-then-decode rejection (+ cancel
idempotency), release-then-decode rejection (+ release idempotency), and
metrics reporting cancelled sessions. DGR-032's fixture and DGR-037's
llama.cpp binding are both expected to import this and pass it against their
own engine, proving identical lifecycle semantics without duplicating the
checks.

### `tests/test_shard_engine.py` (new)

- `_ReferenceEngine`: a minimal in-memory `ShardEngine` used only to prove the
  shared contract is non-vacuous. It is explicitly *not* the DGR-032
  deterministic fixture (no delay/memory-pressure/malformed/crash injection —
  that is DGR-032's own, larger scope); the docstring says so to prevent this
  story's evidence from being read as inherited completion credit for DGR-032.
- Dataclass validation tests: abstract-class instantiation refusal, tensor/
  bundle/token-output field validation, MTP-hook enable refusal, exactly-one-
  input-kind enforcement on `PrefillRequest`/`DecodeRequest`, `LoadRequest`
  shard-range-vs-total-layers validation, `StepResult` output-required-on-OK.
- `test_shard_engine_module_imports_no_native_or_grpc_or_wire_abi_types`:
  walks `vars(shard_engine_module)` and asserts no bound name's `__name__` is
  `ctypes`, `grpc`, or `meshnet_node.native_protocol` — a structural check
  (not a docstring-text grep, which produced a false positive on first draft
  because the module's own docstring *names* `ggml_tensor` as an example of
  what must never appear) that the ABI-isolation acceptance criterion holds.

### `.scratch/distributed-gguf-runtime/prd.json` / issue markdown

Marked `DGR-031.passes = true` with `completionNotes`; regenerated
`issues/031-introduce-the-project-owned-shardengine-interface.md` via
`scripts/ralph_prd_schema.py render` so it matches `prd.json` byte-for-byte.

## Acceptance criteria → evidence

1. **load/capabilities/prefill/decode/boundary-logits-result/cancel/release/
   health/metrics** — `ShardEngine`'s eight abstract methods plus
   `StepResult.output: BoundaryBundle | TokenOutput | None`. Verified by
   `test_reference_engine_obeys_the_shared_shard_engine_contract` and the
   middle-shard-vs-tail-shard assertion inside
   `assert_shard_engine_contract`.
2. **No `ggml_tensor`/llama context/scheduler/ABI-owned structure** — every
   type in `shard_engine.py` is a plain dataclass over `str`/`int`/`bytes`/
   `Mapping`; no import of `native_protocol`, `grpc`, or `ctypes`. Verified by
   `test_shard_engine_module_imports_no_native_or_grpc_or_wire_abi_types`.
3. **Reserved typed MTP/architecture-aux-state hooks, not enabled** —
   `MtpHook.__post_init__` raises on `enabled=True`; `ArchitectureAuxStateHook`
   carries opaque shard-local state with no wire path. Verified by
   `test_mtp_hook_is_reserved_and_refuses_to_enable` and
   `test_architecture_aux_state_hook_carries_opaque_shard_local_state`, plus
   `assert_shard_engine_contract`'s `caps.supports_mtp is False` check.
4. **Contract tests proving fake and future llama implementations obey
   identical lifecycle semantics** — `tests/shard_engine_contract.py` is
   written to be imported by DGR-032 and DGR-037 against their own engines;
   `test_shard_engine.py` proves it is real by running it against
   `_ReferenceEngine`.
5. **Gates + this handoff** — below.

## Commands and results

```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q tests/test_shard_engine.py
```
```text
12 passed in 0.13s
```

```bash
PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q \
  tests/test_shard_engine.py tests/test_shard_lifecycle.py \
  tests/test_architecture_boundary.py tests/test_activation_envelope.py \
  tests/test_native_shard_protocol.py tests/test_shard_runtime_harness.py
```
```text
95 passed, 3 skipped in 3.65s
```

```bash
.venv/bin/python3 -m compileall packages/node/meshnet_node/shard_engine.py tests/shard_engine_contract.py tests/test_shard_engine.py
```
```text
Compiling 'packages/node/meshnet_node/shard_engine.py'...
Compiling 'tests/shard_engine_contract.py'...
Compiling 'tests/test_shard_engine.py'...
```

```bash
git diff --check
```
```text
(no output — clean)
```

## Limitations

- `tests/` as a whole does not collect cleanly in this environment: 27
  pre-existing test modules fail to import for missing optional dependencies
  (`cryptography`, etc.) unrelated to this story. Reproduced identically with
  `git stash` before this session's change (`27 errors during collection`),
  so this is pre-existing environment state, not a regression introduced
  here. This story's own gates were run as the targeted, scoped test set
  above per the shared quality gates' own wording ("Targeted deterministic
  tests pass").
- The contract in `shard_engine_contract.py` proves *lifecycle* semantics
  (gating, cache-miss/stale-epoch/cancel/release, boundary-vs-token output
  shape) are identical across implementations. It does not — and cannot yet
  — prove numerical parity between a fake and a real engine; that is
  DGR-036's explicit job once DGR-032 and DGR-037 both exist.
- `_ReferenceEngine` in `test_shard_engine.py` is intentionally minimal
  (no delay/memory-pressure/malformed-output/crash injection). DGR-032's
  acceptance criteria require those independently; nothing here should be
  read as satisfying them.
- No gRPC/CMake/native-build changes were needed or made — this story is
  pure Python interface/type definition (`evidenceClass: model-free`,
  `hardware: none`), so the native CMake/CTest and patch-stack gates in the
  shared quality-gate list do not apply here (consistent with DGR-021/DGR-025,
  which record the same non-applicability for non-native stories).

## Dependency handoff

- **DGR-032** (fake `ShardEngine`): subclass `ShardEngine`, add delay/memory-
  pressure/malformed-output/crash injection, and pass the *same*
  `assert_shard_engine_contract` from `tests/shard_engine_contract.py`
  against it — no new contract vocabulary should be needed.
- **DGR-034/DGR-035** (range-aware GGUF ownership, boundary I/O): `LoadRequest`
  already carries `shard_start`/`shard_end`/`total_layers`/`recipe`; `capabilities()`
  reports the authoritative range via `EngineCapabilities.is_head`/`is_tail`.
  `BoundaryBundle.token_id_sideband` is reserved for the first-three-hash-
  routed-layers V4 requirement RALPH-CONTEXT documents.
- **DGR-037** (bind llama.cpp to the worker): implement `ShardEngine` as a
  thin wrapper around the native artifact from `native_backend.py`/
  `runtime_recipe.py`; `shard_runtime_server.py`'s `Session`/`GetCapability`/
  `Health`/`Cancel`/`Release` handlers become the translation layer between
  `pb.*` wire messages and this module's request/result types — this story
  intentionally does not touch `shard_runtime_server.py` itself, since that
  wiring is DGR-037's scope.
- **DGR-051** (V4 `ShardEngine` adapter): `MtpHook`/`ArchitectureAuxStateHook`
  fix the field shape now so the V4 adapter does not need a breaking change
  to enable MTP after DGR-066 or to carry CSA/HCA/SWA/indexer/compressor
  state.