feat: checkpoint distributed gguf runtime stories
This commit is contained in:
Dobromir Popov
176
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
176
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# DGR-002 — Versioned gRPC Shard protocol: evidence
|
||||
|
||||
Status: done
|
||||
Date: 2026-07-15
|
||||
Evidence kind: **synthetic-unit** (schema round-trip + cross-language protobuf
|
||||
compatibility). No model download, no GPU, no network, no API credits.
|
||||
|
||||
## Summary
|
||||
|
||||
Added the versioned Protocol Buffers schema that is the semantic contract between
|
||||
Python and C++ Shards (ADR-0024), plus reproducible Python and C++ code
|
||||
generation/build wiring and generated-schema round-trip + compatibility tests in
|
||||
**both** languages. The schema defines one long-lived bidirectional gRPC stream
|
||||
per Route Session Activation Seam, bounded prefill chunking, a small decode fast
|
||||
path, and a versioned named-tensor bundle carrying every required identifier.
|
||||
|
||||
No existing runtime code was modified — this story is purely additive (a new
|
||||
`.proto`, a `native_protocol` loader package, C++ build wiring, and one new test
|
||||
module). Generated stubs are produced on demand into gitignored `build/`
|
||||
directories, so nothing generated is committed.
|
||||
|
||||
## Files changed (all new)
|
||||
|
||||
- `packages/node/native/proto/shard_runtime.proto` — the schema (package
|
||||
`meshnet.shard.v1`, proto3). Service `ShardRuntime` with `GetCapability`,
|
||||
`Health`, `ActivateSession` (bidi stream), `Release`, `Cancel`.
|
||||
- `packages/node/meshnet_node/native_protocol/__init__.py` — reproducible
|
||||
on-demand `grpc_tools.protoc` codegen + loader (`load()`, `load_grpc()`) and
|
||||
shared bundle helpers (`compute_checksum`, `verify_checksum`, `fragment_tensor`,
|
||||
`reassemble_tensor`).
|
||||
- `packages/node/native/scripts/generate_python.py` — standalone reproducible
|
||||
Python generation (self-contained; does not import `meshnet_node`).
|
||||
- `packages/node/native/scripts/generate_cpp.sh` — reproducible C++ generation
|
||||
(message stubs always; gRPC service stubs when `grpc_cpp_plugin` is present).
|
||||
- `packages/node/native/CMakeLists.txt` — C++ build wiring; works with both
|
||||
CONFIG-mode (`protobuf::libprotobuf`/`protobuf::protoc`) and CMake's
|
||||
`FindProtobuf` module.
|
||||
- `packages/node/native/tests/roundtrip_test.cpp` — C++ round-trip / compat test
|
||||
(`--selftest`, `--read`, `--write`).
|
||||
- `tests/test_native_shard_protocol.py` — Python round-trip + compatibility tests
|
||||
and the Python↔C++ cross-language driver.
|
||||
|
||||
## Acceptance criteria → evidence
|
||||
|
||||
- **Capability/health/session-stream/release/cancellation schema** — the
|
||||
`ShardRuntime` service's five RPCs; `test_capability_and_health_round_trip`,
|
||||
`test_session_stream_carries_open_prefill_decode_release_cancel`.
|
||||
- **One long-lived bidi stream per Activation Seam with deadlines, cancellation,
|
||||
flow control, structured errors** — `rpc ActivateSession (stream ...) returns
|
||||
(stream ...)`. Deadlines: gRPC call deadline on direct transport, plus
|
||||
`SessionOpen.deadline_unix_nanos` for relay-carried frames. Cancellation:
|
||||
`Cancel` RPC and in-stream `CancelRequest`/`PHASE_CANCEL`. Flow control:
|
||||
`FlowControl` frames (credits + in-flight byte/message caps). Structured errors:
|
||||
`Status` (canonical code, message, `RetryClass`, details). Verified by
|
||||
`test_session_response_carries_structured_status_and_results`.
|
||||
- **Bounded prefill chunking + small decode fast path** — `PrefillChunk`
|
||||
(`chunk_index`/`chunk_count`/`final_chunk`, `SessionOpen.max_prefill_tokens_per_chunk`)
|
||||
and `DecodeStep` (minimal single-bundle path). Bounded fragments via
|
||||
`SessionOpen.max_fragment_bytes` and `fragment_tensor(...)`.
|
||||
- **Carries schema version, work ID, Route Session ID, route epoch,
|
||||
artifact/recipe fingerprint, shard range/effective start, phase, position,
|
||||
idempotency step, cache expectation, compression, checksum** — all on
|
||||
`MessageHeader` (+ `ArtifactFingerprint.runtime_recipe_fingerprint`,
|
||||
`ShardRange.effective_start_layer`). Verified field-by-field by
|
||||
`test_message_header_carries_every_required_field`.
|
||||
- **Versioned named-tensor bundle (name, shape, dtype, byte order, fragments)** —
|
||||
`TensorBundle`/`NamedTensor`/`TensorFragment`;
|
||||
`test_named_tensor_bundle_describes_shape_dtype_byteorder_and_fragments`,
|
||||
`test_fragment_and_reassemble_round_trip_with_checksums`.
|
||||
- **Round-trip + compatibility tests in Python and C++** — Python:
|
||||
`tests/test_native_shard_protocol.py` (11 tests). C++: `roundtrip_test.cpp`
|
||||
built via CMake; cross-language driver `test_cross_language_roundtrip_python_and_cpp`
|
||||
exercises Python→C++ and C++→Python in both directions.
|
||||
- **Targeted pytest** — `11 passed, 1 skipped` (default env); `12 passed` with the
|
||||
C++ toolchain on PATH.
|
||||
- **compileall packages tests** — exit 0.
|
||||
- **git diff --check** — clean.
|
||||
- **Deterministic / download-free / credit-free / GPU-free** — all tests are pure
|
||||
protobuf serialization; the C++ path uses only local compilers.
|
||||
- **Full deterministic pytest** — `704 passed, 14 skipped, 11 failed`. The 11
|
||||
failures are pre-existing and unrelated (see below).
|
||||
|
||||
## Commands and real results
|
||||
|
||||
See `commands.txt` for the exact command list. Key results:
|
||||
|
||||
- `python packages/node/native/scripts/generate_python.py` →
|
||||
`shard_runtime_pb2.py: ok`, `shard_runtime_pb2_grpc.py: ok`.
|
||||
- `pytest tests/test_native_shard_protocol.py -q` → **11 passed, 1 skipped**
|
||||
(skip reason: `C++ toolchain unavailable: cmake not found on PATH`).
|
||||
- With `/tmp/pbsrc/install/bin` (protoc 33.1) and `.venv/bin` (cmake) on PATH and
|
||||
`CMAKE_PREFIX_PATH=/tmp/pbsrc/install`:
|
||||
- `generate_cpp.sh` → `shard_runtime.pb.cc`, `shard_runtime.pb.h`
|
||||
(grpc service stubs skipped: `grpc_cpp_plugin` absent).
|
||||
- `cmake -S ... -B ...` + `cmake --build ...` → build OK.
|
||||
- `shard_protocol_roundtrip_test --selftest` → `selftest ok (128 bytes)`, exit 0.
|
||||
- `ctest` → `1/1 Test #1: shard_protocol_roundtrip ... Passed`.
|
||||
- `pytest ...::test_cross_language_roundtrip_python_and_cpp -q` → **1 passed**
|
||||
(Python serializes → C++ parses & verifies → C++ serializes → Python parses
|
||||
& verifies).
|
||||
- `compileall -q packages tests` → exit 0.
|
||||
- `git diff --check` → clean.
|
||||
|
||||
## Pre-existing unrelated failures (full-suite)
|
||||
|
||||
`pytest -q` on the full tree reports 11 failures, all in tracker routing /
|
||||
dynamic routing / manual route benchmark / toploc calibration — none import the
|
||||
Shard protocol. Clean-tree reproduction: with **all DGR-002 files moved aside**
|
||||
(`git status` shows only the pre-existing `.ralph-tui/config.toml` deletion),
|
||||
re-running exactly these tests gives `11 failed, 3 passed` — identical failures.
|
||||
They exist on the `ralph/distributed-gguf-runtime` branch independent of this
|
||||
story. The full list is in `results.json.preexisting_unrelated_failures`.
|
||||
|
||||
Note: the earlier `progress.md` (RCR-001, on master) recorded a different set of
|
||||
6 optional-dependency failures (zstandard, langchain_openai). Those did **not**
|
||||
recur here; this environment has those deps. The 11 above are branch-local
|
||||
routing/benchmark failures, not environmental.
|
||||
|
||||
## Limitations and deferred work
|
||||
|
||||
- **C++ toolchain is host-provided, not vendored.** The default test env has no
|
||||
`protoc`/`cmake`/protobuf C++ headers on PATH, so the C++ cross-language test
|
||||
**skips** by default (explicit skip reason). It was executed for this evidence
|
||||
using an ephemeral from-source protobuf 33.1 install at `/tmp/pbsrc/install`
|
||||
plus the `.venv` cmake. DGR-004/DGR-008 should pin the C++ protobuf/gRPC
|
||||
toolchain (upstream commit + reproducible fetch/build) so this test runs in CI
|
||||
without relying on an ad-hoc `/tmp` install.
|
||||
- **gRPC C++ service stubs not built here.** `grpc_cpp_plugin` is absent, so
|
||||
`generate_cpp.sh` produced message stubs only. The round-trip test needs only
|
||||
message serialization; the service stubs are DGR-008's concern.
|
||||
- **No live gRPC transport yet.** This story delivers the schema + serialization
|
||||
contract and generation/build wiring only. Channel setup, the bidi stream
|
||||
server/client, deadlines/cancellation propagation over a real HTTP/2 channel,
|
||||
and relay framing are DGR-008/DGR-009.
|
||||
- **Protobuf runtime version skew.** Python runtime is pip protobuf 7.35.1; the
|
||||
C++ side used protoc 33.1. Protobuf wire format is stable across these, and the
|
||||
cross-language round-trip confirms interop; version pinning is deferred to the
|
||||
toolchain-pinning stories.
|
||||
|
||||
## Compatibility / migration notes
|
||||
|
||||
- proto3 with a 0-valued `*_UNSPECIFIED` member on every enum and never-reused
|
||||
field numbers. Forward compatibility (unknown-field preservation) is verified
|
||||
behaviourally by `test_unknown_fields_are_preserved_for_forward_compatibility`
|
||||
— note protobuf 7.x's upb backend does not implement the `UnknownFields()`
|
||||
introspection accessor, so the test asserts the observable re-serialization
|
||||
outcome instead. Backward defaults verified by
|
||||
`test_defaults_are_stable_for_backward_compatibility`.
|
||||
- Wire schema version is `SchemaVersion.SCHEMA_VERSION_1` (int 1), also exposed as
|
||||
`meshnet_node.native_protocol.SCHEMA_VERSION`.
|
||||
|
||||
## Handoff for dependent stories
|
||||
|
||||
- **DGR-003 (recipe/fingerprint):** populate `ArtifactFingerprint`
|
||||
(`model_id`, `revision`, `artifact_hash`, `quantization`,
|
||||
`runtime_recipe_fingerprint`). Admission compares these before activation; a
|
||||
mismatch is a fatal `Status` (`RetryClass.RETRY_CLASS_FATAL`).
|
||||
- **DGR-004 (llama.cpp pin) / DGR-008 (C++ worker):** pin the C++
|
||||
protobuf + gRPC toolchain and add `grpc_cpp_plugin`; then `generate_cpp.sh`
|
||||
emits service stubs and the CMake target can link gRPC. Implement the
|
||||
`ShardRuntime` servicer; map `(route_session_id, route_epoch)` to an isolated
|
||||
llama sequence. Use `SessionOpen` for stream-scoped bounds and `FlowControl`
|
||||
for backpressure.
|
||||
- **DGR-009 (Meshnet integration/relay):** the relay may carry serialized
|
||||
`SessionActivation`/`SessionResponse` frames as opaque binary; use the in-message
|
||||
`deadline_unix_nanos`, `CancelRequest`, and `FlowControl` since gRPC call
|
||||
metadata is lost over relay.
|
||||
- **Loader usage:** `from meshnet_node import native_protocol as proto;
|
||||
pb2 = proto.load()`. Stubs regenerate automatically when the `.proto` changes
|
||||
(mtime check). `proto.load_grpc()` returns the service stubs (needs the `grpc`
|
||||
runtime).
|
||||
- **Gotcha:** the `.venv` installs the meshnet packages editable via a PEP 660
|
||||
meta-path finder pointing at the **main** checkout. Import the worktree copy by
|
||||
ensuring the worktree `packages/node` is on `sys.path` first (conftest already
|
||||
does this for pytest); standalone tooling must derive paths from `__file__` and
|
||||
not `import meshnet_node` (why `generate_python.py` is self-contained).
|
||||
@@ -0,0 +1,40 @@
|
||||
# DGR-002 reproduction commands (run from repo root, project .venv = Python 3.14).
|
||||
|
||||
# 1. Generate Python stubs (reproducible; writes to gitignored build/ dir).
|
||||
.venv/bin/python packages/node/native/scripts/generate_python.py
|
||||
|
||||
# 2. Python round-trip + compatibility tests (default env; C++ test skips if
|
||||
# cmake/protoc absent).
|
||||
.venv/bin/python -m pytest tests/test_native_shard_protocol.py -q
|
||||
# => 11 passed, 1 skipped
|
||||
|
||||
# 3. Quality gates.
|
||||
.venv/bin/python -m compileall -q packages tests # exit 0
|
||||
git diff --check # clean
|
||||
|
||||
# 4. Full deterministic suite (records pre-existing unrelated failures).
|
||||
.venv/bin/python -m pytest -q
|
||||
# => 704 passed, 14 skipped, 11 failed (all pre-existing, unrelated; see below)
|
||||
|
||||
# 5. Clean-tree reproduction of the 11 pre-existing failures (DGR-002 files moved
|
||||
# aside): same 11 fail => not caused by this story.
|
||||
|
||||
# --- C++ / cross-language (requires protoc + protobuf C++ dev + cmake) --------
|
||||
# On this host a from-source protobuf 33.1 toolchain lives under /tmp/pbsrc/install
|
||||
# and cmake ships in the .venv. To execute the C++ test instead of skipping it:
|
||||
export PATH="/tmp/pbsrc/install/bin:$PWD/.venv/bin:$PATH"
|
||||
export CMAKE_PREFIX_PATH="/tmp/pbsrc/install:$CMAKE_PREFIX_PATH"
|
||||
|
||||
# 6. Generate C++ stubs (message stubs always; gRPC service stubs if
|
||||
# grpc_cpp_plugin present).
|
||||
packages/node/native/scripts/generate_cpp.sh
|
||||
|
||||
# 7. Standalone C++ build + selftest + ctest.
|
||||
cmake -S packages/node/native -B packages/node/native/build/cpp
|
||||
cmake --build packages/node/native/build/cpp --target shard_protocol_roundtrip_test
|
||||
packages/node/native/build/cpp/shard_protocol_roundtrip_test --selftest # "selftest ok (128 bytes)"
|
||||
(cd packages/node/native/build/cpp && ctest --output-on-failure) # 1/1 passed
|
||||
|
||||
# 8. Cross-language Python<->C++ round-trip via the pytest driver (now runs, not skips).
|
||||
.venv/bin/python -m pytest tests/test_native_shard_protocol.py::test_cross_language_roundtrip_python_and_cpp -q
|
||||
# => 1 passed
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"task": "DGR-002",
|
||||
"title": "Adopt the versioned gRPC Shard protocol",
|
||||
"schema": {
|
||||
"proto": "packages/node/native/proto/shard_runtime.proto",
|
||||
"package": "meshnet.shard.v1",
|
||||
"syntax": "proto3",
|
||||
"schema_version": 1,
|
||||
"service": "ShardRuntime",
|
||||
"rpcs": ["GetCapability", "Health", "ActivateSession", "Release", "Cancel"],
|
||||
"streaming_seam": "ActivateSession (bidirectional stream)"
|
||||
},
|
||||
"toolchain": {
|
||||
"python": "3.14.6",
|
||||
"protobuf_runtime_python": "7.35.1",
|
||||
"grpcio": "1.82.1",
|
||||
"grpcio_tools": "1.82.1",
|
||||
"cpp_protoc": "libprotoc 33.1",
|
||||
"cpp_protobuf_toolchain": "/tmp/pbsrc/install (from-source protobuf 33.1, ephemeral host build)",
|
||||
"cmake": "4.4.0 (.venv)",
|
||||
"cxx": "g++ (system)"
|
||||
},
|
||||
"generation": {
|
||||
"python_cmd": "python packages/node/native/scripts/generate_python.py",
|
||||
"python_out": "packages/node/native/build/python/shard_runtime_pb2{,_grpc}.py (gitignored)",
|
||||
"cpp_cmd": "packages/node/native/scripts/generate_cpp.sh",
|
||||
"cpp_out": "packages/node/native/build/cpp-gen/shard_runtime.pb.{h,cc} (gitignored)",
|
||||
"cpp_build": "cmake -S packages/node/native -B <build> && cmake --build <build>"
|
||||
},
|
||||
"tests": {
|
||||
"python_default_env": {"passed": 11, "skipped": 1, "note": "C++ cross-language test skips when cmake/protoc absent"},
|
||||
"python_with_cpp_toolchain": {"passed": 12, "skipped": 0},
|
||||
"cpp_selftest_bytes": 128,
|
||||
"cpp_ctest": "1/1 passed",
|
||||
"cross_language": "Python->C++ and C++->Python round-trip verified in both directions"
|
||||
},
|
||||
"quality_gates": {
|
||||
"targeted_pytest": "11 passed, 1 skipped (default); 12 passed with C++ toolchain",
|
||||
"compileall_packages_tests": "exit 0",
|
||||
"git_diff_check": "clean",
|
||||
"full_pytest": {
|
||||
"passed": 704,
|
||||
"skipped": 14,
|
||||
"failed": 11,
|
||||
"failed_are_preexisting_unrelated": true,
|
||||
"clean_tree_reproduction": "same 11 fail with all DGR-002 files removed (11 failed, 3 passed)"
|
||||
}
|
||||
},
|
||||
"preexisting_unrelated_failures": [
|
||||
"tests/test_dynamic_routing.py::test_admin_can_replace_a_served_model_and_release_it",
|
||||
"tests/test_manual_route_benchmark.py::test_pinned_route_uses_named_node",
|
||||
"tests/test_manual_route_benchmark.py::test_unknown_route_node_is_400",
|
||||
"tests/test_manual_route_benchmark.py::test_invalid_route_shape_is_400",
|
||||
"tests/test_manual_route_benchmark.py::test_clients_without_route_are_unaffected",
|
||||
"tests/test_manual_route_benchmark.py::test_benchmark_records_one_and_two_node_routes",
|
||||
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_dispatches_only_solo_capable_nodes",
|
||||
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_persists_corpus_and_results_endpoint_reports_it",
|
||||
"tests/test_toploc_calibration_dispatch.py::test_calibration_run_node_without_commitment_endpoint_is_skipped_not_failed",
|
||||
"tests/test_tracker_routing.py::test_torch_node_applies_tracker_load_shard_directive",
|
||||
"tests/test_tracker_routing.py::test_shard_heal_cycle_surviving_node_covers_dead_peers_gap"
|
||||
],
|
||||
"evidence_kind": "synthetic-unit (schema round-trip + cross-language protobuf; no model, no GPU, no network, no API credits)"
|
||||
}
|
||||
Reference in New Issue
Block a user