neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md

DGR-002 — Versioned gRPC Shard protocol: evidence

Status: done Date: 2026-07-15 Evidence kind: synthetic-unit (schema round-trip + cross-language protobuf compatibility). No model download, no GPU, no network, no API credits.

Summary

Added the versioned Protocol Buffers schema that is the semantic contract between Python and C++ Shards (ADR-0024), plus reproducible Python and C++ code generation/build wiring and generated-schema round-trip + compatibility tests in both languages. The schema defines one long-lived bidirectional gRPC stream per Route Session Activation Seam, bounded prefill chunking, a small decode fast path, and a versioned named-tensor bundle carrying every required identifier.

No existing runtime code was modified — this story is purely additive (a new .proto, a native_protocol loader package, C++ build wiring, and one new test module). Generated stubs are produced on demand into gitignored build/ directories, so nothing generated is committed.

Files changed (all new)

• packages/node/native/proto/shard_runtime.proto — the schema (package meshnet.shard.v1, proto3). Service ShardRuntime with GetCapability, Health, ActivateSession (bidi stream), Release, Cancel.
• packages/node/meshnet_node/native_protocol/__init__.py — reproducible on-demand grpc_tools.protoc codegen + loader (load(), load_grpc()) and shared bundle helpers (compute_checksum, verify_checksum, fragment_tensor, reassemble_tensor).
• packages/node/native/scripts/generate_python.py — standalone reproducible Python generation (self-contained; does not import meshnet_node).
• packages/node/native/scripts/generate_cpp.sh — reproducible C++ generation (message stubs always; gRPC service stubs when grpc_cpp_plugin is present).
• packages/node/native/CMakeLists.txt — C++ build wiring; works with both CONFIG-mode (protobuf::libprotobuf/protobuf::protoc) and CMake's FindProtobuf module.
• packages/node/native/tests/roundtrip_test.cpp — C++ round-trip / compat test (--selftest, --read, --write).
• tests/test_native_shard_protocol.py — Python round-trip + compatibility tests and the Python↔C++ cross-language driver.

Acceptance criteria → evidence

• Capability/health/session-stream/release/cancellation schema — the ShardRuntime service's five RPCs; test_capability_and_health_round_trip, test_session_stream_carries_open_prefill_decode_release_cancel.
• One long-lived bidi stream per Activation Seam with deadlines, cancellation, flow control, structured errors — rpc ActivateSession (stream ...) returns (stream ...). Deadlines: gRPC call deadline on direct transport, plus SessionOpen.deadline_unix_nanos for relay-carried frames. Cancellation: Cancel RPC and in-stream CancelRequest/PHASE_CANCEL. Flow control: FlowControl frames (credits + in-flight byte/message caps). Structured errors: Status (canonical code, message, RetryClass, details). Verified by test_session_response_carries_structured_status_and_results.
• Bounded prefill chunking + small decode fast path — PrefillChunk (chunk_index/chunk_count/final_chunk, SessionOpen.max_prefill_tokens_per_chunk) and DecodeStep (minimal single-bundle path). Bounded fragments via SessionOpen.max_fragment_bytes and fragment_tensor(...).
• Carries schema version, work ID, Route Session ID, route epoch, artifact/recipe fingerprint, shard range/effective start, phase, position, idempotency step, cache expectation, compression, checksum — all on MessageHeader (+ ArtifactFingerprint.runtime_recipe_fingerprint, ShardRange.effective_start_layer). Verified field-by-field by test_message_header_carries_every_required_field.
• Versioned named-tensor bundle (name, shape, dtype, byte order, fragments) — TensorBundle/NamedTensor/TensorFragment; test_named_tensor_bundle_describes_shape_dtype_byteorder_and_fragments, test_fragment_and_reassemble_round_trip_with_checksums.
• Round-trip + compatibility tests in Python and C++ — Python: tests/test_native_shard_protocol.py (11 tests). C++: roundtrip_test.cpp built via CMake; cross-language driver test_cross_language_roundtrip_python_and_cpp exercises Python→C++ and C++→Python in both directions.
• Targeted pytest — 11 passed, 1 skipped (default env); 12 passed with the C++ toolchain on PATH.
• compileall packages tests — exit 0.
• git diff --check — clean.
• Deterministic / download-free / credit-free / GPU-free — all tests are pure protobuf serialization; the C++ path uses only local compilers.
• Full deterministic pytest — 704 passed, 14 skipped, 11 failed. The 11 failures are pre-existing and unrelated (see below).

Commands and real results

See commands.txt for the exact command list. Key results:

• python packages/node/native/scripts/generate_python.py → shard_runtime_pb2.py: ok, shard_runtime_pb2_grpc.py: ok.
• pytest tests/test_native_shard_protocol.py -q → 11 passed, 1 skipped (skip reason: C++ toolchain unavailable: cmake not found on PATH).
• With /tmp/pbsrc/install/bin (protoc 33.1) and .venv/bin (cmake) on PATH and CMAKE_PREFIX_PATH=/tmp/pbsrc/install:
  • generate_cpp.sh → shard_runtime.pb.cc, shard_runtime.pb.h (grpc service stubs skipped: grpc_cpp_plugin absent).
  • cmake -S ... -B ... + cmake --build ... → build OK.
  • shard_protocol_roundtrip_test --selftest → selftest ok (128 bytes), exit 0.
  • ctest → 1/1 Test #1: shard_protocol_roundtrip ... Passed.
  • pytest ...::test_cross_language_roundtrip_python_and_cpp -q → 1 passed (Python serializes → C++ parses & verifies → C++ serializes → Python parses & verifies).
• compileall -q packages tests → exit 0.
• git diff --check → clean.

Pre-existing unrelated failures (full-suite)

pytest -q on the full tree reports 11 failures, all in tracker routing / dynamic routing / manual route benchmark / toploc calibration — none import the Shard protocol. Clean-tree reproduction: with all DGR-002 files moved aside (git status shows only the pre-existing .ralph-tui/config.toml deletion), re-running exactly these tests gives 11 failed, 3 passed — identical failures. They exist on the ralph/distributed-gguf-runtime branch independent of this story. The full list is in results.json.preexisting_unrelated_failures.

Note: the earlier progress.md (RCR-001, on master) recorded a different set of 6 optional-dependency failures (zstandard, langchain_openai). Those did not recur here; this environment has those deps. The 11 above are branch-local routing/benchmark failures, not environmental.

Limitations and deferred work

• C++ toolchain is host-provided, not vendored. The default test env has no protoc/cmake/protobuf C++ headers on PATH, so the C++ cross-language test skips by default (explicit skip reason). It was executed for this evidence using an ephemeral from-source protobuf 33.1 install at /tmp/pbsrc/install plus the .venv cmake. DGR-004/DGR-008 should pin the C++ protobuf/gRPC toolchain (upstream commit + reproducible fetch/build) so this test runs in CI without relying on an ad-hoc /tmp install.
• gRPC C++ service stubs not built here. grpc_cpp_plugin is absent, so generate_cpp.sh produced message stubs only. The round-trip test needs only message serialization; the service stubs are DGR-008's concern.
• No live gRPC transport yet. This story delivers the schema + serialization contract and generation/build wiring only. Channel setup, the bidi stream server/client, deadlines/cancellation propagation over a real HTTP/2 channel, and relay framing are DGR-008/DGR-009.
• Protobuf runtime version skew. Python runtime is pip protobuf 7.35.1; the C++ side used protoc 33.1. Protobuf wire format is stable across these, and the cross-language round-trip confirms interop; version pinning is deferred to the toolchain-pinning stories.

Compatibility / migration notes

• proto3 with a 0-valued *_UNSPECIFIED member on every enum and never-reused field numbers. Forward compatibility (unknown-field preservation) is verified behaviourally by test_unknown_fields_are_preserved_for_forward_compatibility — note protobuf 7.x's upb backend does not implement the UnknownFields() introspection accessor, so the test asserts the observable re-serialization outcome instead. Backward defaults verified by test_defaults_are_stable_for_backward_compatibility.
• Wire schema version is SchemaVersion.SCHEMA_VERSION_1 (int 1), also exposed as meshnet_node.native_protocol.SCHEMA_VERSION.

Handoff for dependent stories

• DGR-003 (recipe/fingerprint): populate ArtifactFingerprint (model_id, revision, artifact_hash, quantization, runtime_recipe_fingerprint). Admission compares these before activation; a mismatch is a fatal Status (RetryClass.RETRY_CLASS_FATAL).
• DGR-004 (llama.cpp pin) / DGR-008 (C++ worker): pin the C++ protobuf + gRPC toolchain and add grpc_cpp_plugin; then generate_cpp.sh emits service stubs and the CMake target can link gRPC. Implement the ShardRuntime servicer; map (route_session_id, route_epoch) to an isolated llama sequence. Use SessionOpen for stream-scoped bounds and FlowControl for backpressure.
• DGR-009 (Meshnet integration/relay): the relay may carry serialized SessionActivation/SessionResponse frames as opaque binary; use the in-message deadline_unix_nanos, CancelRequest, and FlowControl since gRPC call metadata is lost over relay.
• Loader usage: from meshnet_node import native_protocol as proto; pb2 = proto.load(). Stubs regenerate automatically when the .proto changes (mtime check). proto.load_grpc() returns the service stubs (needs the grpc runtime).
• Gotcha: the .venv installs the meshnet packages editable via a PEP 660 meta-path finder pointing at the main checkout. Import the worktree copy by ensuring the worktree packages/node is on sys.path first (conftest already does this for pytest); standalone tooling must derive paths from __file__ and not import meshnet_node (why generate_python.py is self-contained).