neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-031/README.md

DGR-031 evidence — the project-owned ShardEngine interface

Completed: 2026-07-23 Branch: ralph/distributed-gguf-runtime Authority: .scratch/distributed-gguf-runtime/prd.json Dependencies: DGR-021 (evidence/DGR-021/README.md — versioned activation envelope, NamedTensor/ActivationEnvelope as the project-owned wire-envelope layer), DGR-025 (evidence/DGR-025/README.md — exact artifact/runtime recipe identity; both read before changing code).

Objective

Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract, so a fake fixture engine (DGR-032) and a real llama.cpp-backed engine (DGR-037) are interchangeable subclasses of one interface.

What was found live before changing code

Per RALPH-CONTEXT, legacy pass states were not trusted; the live surrounding contracts were read and exercised before designing this one:

• packages/node/meshnet_node/shard_lifecycle.py (DGR-022) already defines a versioned RPC/session lifecycle contract — StructuredStatus, StatusCode, CacheExpectation, CacheResult, LifecycleState, SessionLifecycle — but it is explicitly the wire RPC contract "consumed by a future generated gRPC binding," not an execution-engine boundary.
• packages/node/meshnet_node/native_backend.py (DGR-025) is the identity boundary for the native GGUF artifact — it derives and attests a ShardIdentity, but does not define an execution contract either.
• packages/node/meshnet_node/protocol.py (DGR-021) defines a project-owned NamedTensor/ActivationEnvelope for activation traffic between shard hops over the network, distinct from the generated-protobuf wire ABI in native_protocol.
• packages/node/meshnet_node/shard_runtime_server.py (DGR-024) is today a real gRPC servicer that proves wire fidelity by checksumming and echoing bytes — it has no execution engine behind it yet; that seam is exactly where ShardEngine plugs in for DGR-037.
• packages/node/meshnet_node/architecture_boundary.py established the precedent this story follows for tail output: TailOutput.sampled_token() never exposes raw logits, only a sampled token id.
• No ShardEngine (or shard_engine) symbol existed anywhere in the repository prior to this story (confirmed by grep -rn -i "shardengine\|shard_engine" across .py/.md, which returned only planning-document prose naming it as future work).

Live verification of the pre-existing dependency contracts before adding new code: PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q tests/test_shard_lifecycle.py tests/test_activation_envelope.py tests/test_architecture_boundary.py tests/test_native_shard_protocol.py tests/test_shard_runtime_harness.py → 95 passed, 3 skipped.

What was added (this story's change)

packages/node/meshnet_node/shard_engine.py (new)

The ShardEngine boundary: an abc.ABC with eight abstract operations — load, capabilities, prefill, decode, cancel, release, health, metrics — matching the acceptance criterion's list exactly (prefill/ decode share one operation family; their shared result type is what the criterion calls the "boundary/logits result"). Every request/result type is a frozen dataclass built from plain str/int/bytes/Mapping values:

• EngineTensor / BoundaryBundle — the project-owned named-tensor activation crossing a shard boundary (head/middle/tail-in). Deliberately a new, minimal type distinct from both native_protocol.pb.TensorBundle (generated-protobuf ABI) and protocol.NamedTensor/ActivationEnvelope (wire-framing/fragmentation concerns irrelevant to model execution) — a fourth, execution-facing layer underneath the three that already existed.
• TokenOutput — a tail shard's sampled result: a token id (+ optional decoded text), never a raw logits tensor.
• MtpHook — reserved multi-token-prediction hook; its own __post_init__ raises if constructed with enabled=True, so the type exists (fixing its field shape for DGR-051/DGR-066) without any code path being able to turn it on before DGR-066, matching RALPH-CONTEXT's "MTP is reserved and off for alpha."
• ArchitectureAuxStateHook — reserved per-shard architecture auxiliary state (V4 CSA/HCA/SWA/indexer/compressor and similar); has no wire encoding and is never embedded in a BoundaryBundle, matching RALPH-CONTEXT's "remain local ... never carried over the WAN seam."
• LoadRequest/LoadResult, EngineCapabilities, PrefillRequest/ DecodeRequest (exactly one of token_ids/token_id (head) or input (middle/tail) required — enforced in __post_init__), StepResult (a successful result must carry an output; cache_result reuses shard_lifecycle.CacheResult), HealthResult, MetricsResult.
• Status vocabulary is reused, not reinvented: StructuredStatus/ StatusCode/CacheExpectation/CacheResult are imported from shard_lifecycle (already project-owned and version-stable) rather than a parallel enum living alongside it.
• The module imports nothing from native_protocol, grpc, or ctypes — verified structurally, not just by convention (see tests below).

tests/shard_engine_contract.py (new)

A reusable, non-test_-prefixed helper: assert_shard_engine_contract(make_engine) takes a zero-arg engine factory and runs nine lifecycle checks — health before load, load→capabilities range/MTP-off, prefill→decode determinism (byte-identical output replayed on a fresh session), middle-shard boundary-bundle-in/out vs. head/tail token-output, deterministic cache-miss on an unopened session, stale-route-epoch rejection, cancel-then-decode rejection (+ cancel idempotency), release-then-decode rejection (+ release idempotency), and metrics reporting cancelled sessions. DGR-032's fixture and DGR-037's llama.cpp binding are both expected to import this and pass it against their own engine, proving identical lifecycle semantics without duplicating the checks.

tests/test_shard_engine.py (new)

• _ReferenceEngine: a minimal in-memory ShardEngine used only to prove the shared contract is non-vacuous. It is explicitly not the DGR-032 deterministic fixture (no delay/memory-pressure/malformed/crash injection — that is DGR-032's own, larger scope); the docstring says so to prevent this story's evidence from being read as inherited completion credit for DGR-032.
• Dataclass validation tests: abstract-class instantiation refusal, tensor/ bundle/token-output field validation, MTP-hook enable refusal, exactly-one- input-kind enforcement on PrefillRequest/DecodeRequest, LoadRequest shard-range-vs-total-layers validation, StepResult output-required-on-OK.
• test_shard_engine_module_imports_no_native_or_grpc_or_wire_abi_types: walks vars(shard_engine_module) and asserts no bound name's __name__ is ctypes, grpc, or meshnet_node.native_protocol — a structural check (not a docstring-text grep, which produced a false positive on first draft because the module's own docstring names ggml_tensor as an example of what must never appear) that the ABI-isolation acceptance criterion holds.

.scratch/distributed-gguf-runtime/prd.json / issue markdown

Marked DGR-031.passes = true with completionNotes; regenerated issues/031-introduce-the-project-owned-shardengine-interface.md via scripts/ralph_prd_schema.py render so it matches prd.json byte-for-byte.

Acceptance criteria → evidence

1. load/capabilities/prefill/decode/boundary-logits-result/cancel/release/ health/metrics — ShardEngine's eight abstract methods plus StepResult.output: BoundaryBundle | TokenOutput | None. Verified by test_reference_engine_obeys_the_shared_shard_engine_contract and the middle-shard-vs-tail-shard assertion inside assert_shard_engine_contract.
2. No ggml_tensor/llama context/scheduler/ABI-owned structure — every type in shard_engine.py is a plain dataclass over str/int/bytes/ Mapping; no import of native_protocol, grpc, or ctypes. Verified by test_shard_engine_module_imports_no_native_or_grpc_or_wire_abi_types.
3. Reserved typed MTP/architecture-aux-state hooks, not enabled — MtpHook.__post_init__ raises on enabled=True; ArchitectureAuxStateHook carries opaque shard-local state with no wire path. Verified by test_mtp_hook_is_reserved_and_refuses_to_enable and test_architecture_aux_state_hook_carries_opaque_shard_local_state, plus assert_shard_engine_contract's caps.supports_mtp is False check.
4. Contract tests proving fake and future llama implementations obey identical lifecycle semantics — tests/shard_engine_contract.py is written to be imported by DGR-032 and DGR-037 against their own engines; test_shard_engine.py proves it is real by running it against _ReferenceEngine.
5. Gates + this handoff — below.

Commands and results

PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q tests/test_shard_engine.py

12 passed in 0.13s

PYTHONPATH=packages/node:packages/tracker .venv/bin/python3 -m pytest -q \
  tests/test_shard_engine.py tests/test_shard_lifecycle.py \
  tests/test_architecture_boundary.py tests/test_activation_envelope.py \
  tests/test_native_shard_protocol.py tests/test_shard_runtime_harness.py

95 passed, 3 skipped in 3.65s

.venv/bin/python3 -m compileall packages/node/meshnet_node/shard_engine.py tests/shard_engine_contract.py tests/test_shard_engine.py

Compiling 'packages/node/meshnet_node/shard_engine.py'...
Compiling 'tests/shard_engine_contract.py'...
Compiling 'tests/test_shard_engine.py'...

git diff --check

(no output — clean)

Limitations

• tests/ as a whole does not collect cleanly in this environment: 27 pre-existing test modules fail to import for missing optional dependencies (cryptography, etc.) unrelated to this story. Reproduced identically with git stash before this session's change (27 errors during collection), so this is pre-existing environment state, not a regression introduced here. This story's own gates were run as the targeted, scoped test set above per the shared quality gates' own wording ("Targeted deterministic tests pass").
• The contract in shard_engine_contract.py proves lifecycle semantics (gating, cache-miss/stale-epoch/cancel/release, boundary-vs-token output shape) are identical across implementations. It does not — and cannot yet — prove numerical parity between a fake and a real engine; that is DGR-036's explicit job once DGR-032 and DGR-037 both exist.
• _ReferenceEngine in test_shard_engine.py is intentionally minimal (no delay/memory-pressure/malformed-output/crash injection). DGR-032's acceptance criteria require those independently; nothing here should be read as satisfying them.
• No gRPC/CMake/native-build changes were needed or made — this story is pure Python interface/type definition (evidenceClass: model-free, hardware: none), so the native CMake/CTest and patch-stack gates in the shared quality-gate list do not apply here (consistent with DGR-021/DGR-025, which record the same non-applicability for non-native stories).

Dependency handoff

• DGR-032 (fake ShardEngine): subclass ShardEngine, add delay/memory- pressure/malformed-output/crash injection, and pass the same assert_shard_engine_contract from tests/shard_engine_contract.py against it — no new contract vocabulary should be needed.
• DGR-034/DGR-035 (range-aware GGUF ownership, boundary I/O): LoadRequest already carries shard_start/shard_end/total_layers/recipe; capabilities() reports the authoritative range via EngineCapabilities.is_head/is_tail. BoundaryBundle.token_id_sideband is reserved for the first-three-hash- routed-layers V4 requirement RALPH-CONTEXT documents.
• DGR-037 (bind llama.cpp to the worker): implement ShardEngine as a thin wrapper around the native artifact from native_backend.py/ runtime_recipe.py; shard_runtime_server.py's Session/GetCapability/ Health/Cancel/Release handlers become the translation layer between pb.* wire messages and this module's request/result types — this story intentionally does not touch shard_runtime_server.py itself, since that wiring is DGR-037's scope.
• DGR-051 (V4 ShardEngine adapter): MtpHook/ArchitectureAuxStateHook fix the field shape now so the V4 adapter does not need a breaking change to enable MTP after DGR-066 or to carry CSA/HCA/SWA/indexer/compressor state.