feat: checkpoint distributed gguf runtime stories
This commit is contained in:
Dobromir Popov
176
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
176
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# DGR-002 — Versioned gRPC Shard protocol: evidence
|
||||
|
||||
Status: done
|
||||
Date: 2026-07-15
|
||||
Evidence kind: **synthetic-unit** (schema round-trip + cross-language protobuf
|
||||
compatibility). No model download, no GPU, no network, no API credits.
|
||||
|
||||
## Summary
|
||||
|
||||
Added the versioned Protocol Buffers schema that is the semantic contract between
|
||||
Python and C++ Shards (ADR-0024), plus reproducible Python and C++ code
|
||||
generation/build wiring and generated-schema round-trip + compatibility tests in
|
||||
**both** languages. The schema defines one long-lived bidirectional gRPC stream
|
||||
per Route Session Activation Seam, bounded prefill chunking, a small decode fast
|
||||
path, and a versioned named-tensor bundle carrying every required identifier.
|
||||
|
||||
No existing runtime code was modified — this story is purely additive (a new
|
||||
`.proto`, a `native_protocol` loader package, C++ build wiring, and one new test
|
||||
module). Generated stubs are produced on demand into gitignored `build/`
|
||||
directories, so nothing generated is committed.
|
||||
|
||||
## Files changed (all new)
|
||||
|
||||
- `packages/node/native/proto/shard_runtime.proto` — the schema (package
|
||||
`meshnet.shard.v1`, proto3). Service `ShardRuntime` with `GetCapability`,
|
||||
`Health`, `ActivateSession` (bidi stream), `Release`, `Cancel`.
|
||||
- `packages/node/meshnet_node/native_protocol/__init__.py` — reproducible
|
||||
on-demand `grpc_tools.protoc` codegen + loader (`load()`, `load_grpc()`) and
|
||||
shared bundle helpers (`compute_checksum`, `verify_checksum`, `fragment_tensor`,
|
||||
`reassemble_tensor`).
|
||||
- `packages/node/native/scripts/generate_python.py` — standalone reproducible
|
||||
Python generation (self-contained; does not import `meshnet_node`).
|
||||
- `packages/node/native/scripts/generate_cpp.sh` — reproducible C++ generation
|
||||
(message stubs always; gRPC service stubs when `grpc_cpp_plugin` is present).
|
||||
- `packages/node/native/CMakeLists.txt` — C++ build wiring; works with both
|
||||
CONFIG-mode (`protobuf::libprotobuf`/`protobuf::protoc`) and CMake's
|
||||
`FindProtobuf` module.
|
||||
- `packages/node/native/tests/roundtrip_test.cpp` — C++ round-trip / compat test
|
||||
(`--selftest`, `--read`, `--write`).
|
||||
- `tests/test_native_shard_protocol.py` — Python round-trip + compatibility tests
|
||||
and the Python↔C++ cross-language driver.
|
||||
|
||||
## Acceptance criteria → evidence
|
||||
|
||||
- **Capability/health/session-stream/release/cancellation schema** — the
|
||||
`ShardRuntime` service's five RPCs; `test_capability_and_health_round_trip`,
|
||||
`test_session_stream_carries_open_prefill_decode_release_cancel`.
|
||||
- **One long-lived bidi stream per Activation Seam with deadlines, cancellation,
|
||||
flow control, structured errors** — `rpc ActivateSession (stream ...) returns
|
||||
(stream ...)`. Deadlines: gRPC call deadline on direct transport, plus
|
||||
`SessionOpen.deadline_unix_nanos` for relay-carried frames. Cancellation:
|
||||
`Cancel` RPC and in-stream `CancelRequest`/`PHASE_CANCEL`. Flow control:
|
||||
`FlowControl` frames (credits + in-flight byte/message caps). Structured errors:
|
||||
`Status` (canonical code, message, `RetryClass`, details). Verified by
|
||||
`test_session_response_carries_structured_status_and_results`.
|
||||
- **Bounded prefill chunking + small decode fast path** — `PrefillChunk`
|
||||
(`chunk_index`/`chunk_count`/`final_chunk`, `SessionOpen.max_prefill_tokens_per_chunk`)
|
||||
and `DecodeStep` (minimal single-bundle path). Bounded fragments via
|
||||
`SessionOpen.max_fragment_bytes` and `fragment_tensor(...)`.
|
||||
- **Carries schema version, work ID, Route Session ID, route epoch,
|
||||
artifact/recipe fingerprint, shard range/effective start, phase, position,
|
||||
idempotency step, cache expectation, compression, checksum** — all on
|
||||
`MessageHeader` (+ `ArtifactFingerprint.runtime_recipe_fingerprint`,
|
||||
`ShardRange.effective_start_layer`). Verified field-by-field by
|
||||
`test_message_header_carries_every_required_field`.
|
||||
- **Versioned named-tensor bundle (name, shape, dtype, byte order, fragments)** —
|
||||
`TensorBundle`/`NamedTensor`/`TensorFragment`;
|
||||
`test_named_tensor_bundle_describes_shape_dtype_byteorder_and_fragments`,
|
||||
`test_fragment_and_reassemble_round_trip_with_checksums`.
|
||||
- **Round-trip + compatibility tests in Python and C++** — Python:
|
||||
`tests/test_native_shard_protocol.py` (11 tests). C++: `roundtrip_test.cpp`
|
||||
built via CMake; cross-language driver `test_cross_language_roundtrip_python_and_cpp`
|
||||
exercises Python→C++ and C++→Python in both directions.
|
||||
- **Targeted pytest** — `11 passed, 1 skipped` (default env); `12 passed` with the
|
||||
C++ toolchain on PATH.
|
||||
- **compileall packages tests** — exit 0.
|
||||
- **git diff --check** — clean.
|
||||
- **Deterministic / download-free / credit-free / GPU-free** — all tests are pure
|
||||
protobuf serialization; the C++ path uses only local compilers.
|
||||
- **Full deterministic pytest** — `704 passed, 14 skipped, 11 failed`. The 11
|
||||
failures are pre-existing and unrelated (see below).
|
||||
|
||||
## Commands and real results
|
||||
|
||||
See `commands.txt` for the exact command list. Key results:
|
||||
|
||||
- `python packages/node/native/scripts/generate_python.py` →
|
||||
`shard_runtime_pb2.py: ok`, `shard_runtime_pb2_grpc.py: ok`.
|
||||
- `pytest tests/test_native_shard_protocol.py -q` → **11 passed, 1 skipped**
|
||||
(skip reason: `C++ toolchain unavailable: cmake not found on PATH`).
|
||||
- With `/tmp/pbsrc/install/bin` (protoc 33.1) and `.venv/bin` (cmake) on PATH and
|
||||
`CMAKE_PREFIX_PATH=/tmp/pbsrc/install`:
|
||||
- `generate_cpp.sh` → `shard_runtime.pb.cc`, `shard_runtime.pb.h`
|
||||
(grpc service stubs skipped: `grpc_cpp_plugin` absent).
|
||||
- `cmake -S ... -B ...` + `cmake --build ...` → build OK.
|
||||
- `shard_protocol_roundtrip_test --selftest` → `selftest ok (128 bytes)`, exit 0.
|
||||
- `ctest` → `1/1 Test #1: shard_protocol_roundtrip ... Passed`.
|
||||
- `pytest ...::test_cross_language_roundtrip_python_and_cpp -q` → **1 passed**
|
||||
(Python serializes → C++ parses & verifies → C++ serializes → Python parses
|
||||
& verifies).
|
||||
- `compileall -q packages tests` → exit 0.
|
||||
- `git diff --check` → clean.
|
||||
|
||||
## Pre-existing unrelated failures (full-suite)
|
||||
|
||||
`pytest -q` on the full tree reports 11 failures, all in tracker routing /
|
||||
dynamic routing / manual route benchmark / toploc calibration — none import the
|
||||
Shard protocol. Clean-tree reproduction: with **all DGR-002 files moved aside**
|
||||
(`git status` shows only the pre-existing `.ralph-tui/config.toml` deletion),
|
||||
re-running exactly these tests gives `11 failed, 3 passed` — identical failures.
|
||||
They exist on the `ralph/distributed-gguf-runtime` branch independent of this
|
||||
story. The full list is in `results.json.preexisting_unrelated_failures`.
|
||||
|
||||
Note: the earlier `progress.md` (RCR-001, on master) recorded a different set of
|
||||
6 optional-dependency failures (zstandard, langchain_openai). Those did **not**
|
||||
recur here; this environment has those deps. The 11 above are branch-local
|
||||
routing/benchmark failures, not environmental.
|
||||
|
||||
## Limitations and deferred work
|
||||
|
||||
- **C++ toolchain is host-provided, not vendored.** The default test env has no
|
||||
`protoc`/`cmake`/protobuf C++ headers on PATH, so the C++ cross-language test
|
||||
**skips** by default (explicit skip reason). It was executed for this evidence
|
||||
using an ephemeral from-source protobuf 33.1 install at `/tmp/pbsrc/install`
|
||||
plus the `.venv` cmake. DGR-004/DGR-008 should pin the C++ protobuf/gRPC
|
||||
toolchain (upstream commit + reproducible fetch/build) so this test runs in CI
|
||||
without relying on an ad-hoc `/tmp` install.
|
||||
- **gRPC C++ service stubs not built here.** `grpc_cpp_plugin` is absent, so
|
||||
`generate_cpp.sh` produced message stubs only. The round-trip test needs only
|
||||
message serialization; the service stubs are DGR-008's concern.
|
||||
- **No live gRPC transport yet.** This story delivers the schema + serialization
|
||||
contract and generation/build wiring only. Channel setup, the bidi stream
|
||||
server/client, deadlines/cancellation propagation over a real HTTP/2 channel,
|
||||
and relay framing are DGR-008/DGR-009.
|
||||
- **Protobuf runtime version skew.** Python runtime is pip protobuf 7.35.1; the
|
||||
C++ side used protoc 33.1. Protobuf wire format is stable across these, and the
|
||||
cross-language round-trip confirms interop; version pinning is deferred to the
|
||||
toolchain-pinning stories.
|
||||
|
||||
## Compatibility / migration notes
|
||||
|
||||
- proto3 with a 0-valued `*_UNSPECIFIED` member on every enum and never-reused
|
||||
field numbers. Forward compatibility (unknown-field preservation) is verified
|
||||
behaviourally by `test_unknown_fields_are_preserved_for_forward_compatibility`
|
||||
— note protobuf 7.x's upb backend does not implement the `UnknownFields()`
|
||||
introspection accessor, so the test asserts the observable re-serialization
|
||||
outcome instead. Backward defaults verified by
|
||||
`test_defaults_are_stable_for_backward_compatibility`.
|
||||
- Wire schema version is `SchemaVersion.SCHEMA_VERSION_1` (int 1), also exposed as
|
||||
`meshnet_node.native_protocol.SCHEMA_VERSION`.
|
||||
|
||||
## Handoff for dependent stories
|
||||
|
||||
- **DGR-003 (recipe/fingerprint):** populate `ArtifactFingerprint`
|
||||
(`model_id`, `revision`, `artifact_hash`, `quantization`,
|
||||
`runtime_recipe_fingerprint`). Admission compares these before activation; a
|
||||
mismatch is a fatal `Status` (`RetryClass.RETRY_CLASS_FATAL`).
|
||||
- **DGR-004 (llama.cpp pin) / DGR-008 (C++ worker):** pin the C++
|
||||
protobuf + gRPC toolchain and add `grpc_cpp_plugin`; then `generate_cpp.sh`
|
||||
emits service stubs and the CMake target can link gRPC. Implement the
|
||||
`ShardRuntime` servicer; map `(route_session_id, route_epoch)` to an isolated
|
||||
llama sequence. Use `SessionOpen` for stream-scoped bounds and `FlowControl`
|
||||
for backpressure.
|
||||
- **DGR-009 (Meshnet integration/relay):** the relay may carry serialized
|
||||
`SessionActivation`/`SessionResponse` frames as opaque binary; use the in-message
|
||||
`deadline_unix_nanos`, `CancelRequest`, and `FlowControl` since gRPC call
|
||||
metadata is lost over relay.
|
||||
- **Loader usage:** `from meshnet_node import native_protocol as proto;
|
||||
pb2 = proto.load()`. Stubs regenerate automatically when the `.proto` changes
|
||||
(mtime check). `proto.load_grpc()` returns the service stubs (needs the `grpc`
|
||||
runtime).
|
||||
- **Gotcha:** the `.venv` installs the meshnet packages editable via a PEP 660
|
||||
meta-path finder pointing at the **main** checkout. Import the worktree copy by
|
||||
ensuring the worktree `packages/node` is on `sys.path` first (conftest already
|
||||
does this for pytest); standalone tooling must derive paths from `__file__` and
|
||||
not `import meshnet_node` (why `generate_python.py` is self-contained).
|
||||
Reference in New Issue
Block a user