feat: DGR-002 - Adopt the versioned gRPC Shard protocol
This commit is contained in:
Dobromir Popov
205
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
205
.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# DGR-002 — Adopt the versioned gRPC Shard protocol
|
||||
|
||||
Status: **done**. Every acceptance criterion is met with real command output.
|
||||
Evidence class: **synthetic/unit** — this story defines a schema and proves both
|
||||
languages agree on it. No model, GPU, network peer or benchmark is involved, and
|
||||
none is claimed.
|
||||
|
||||
## 1. Summary
|
||||
|
||||
`packages/node/native/proto/shard_runtime.proto` is now the semantic contract for
|
||||
the native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). Python
|
||||
and C++ both generate from it, and a shared committed conformance vector proves
|
||||
they encode it identically — byte for byte.
|
||||
|
||||
Design decisions worth carrying forward:
|
||||
|
||||
- **Everything gRPC gives you is *also* in the schema.** Deadline, cancellation,
|
||||
identity and flow control are carried as fields, not left to HTTP/2 metadata,
|
||||
because the existing relay carries these frames as **opaque binary**. A relayed
|
||||
frame has no HTTP/2 context to inherit a deadline or a channel identity from.
|
||||
If it is not in the schema, it does not survive the relay.
|
||||
- **Cancellation is both in-band and out-of-band.** `CancelSignal` rides the
|
||||
stream; `Cancel` is also a unary RPC. A cancel that can only travel down a
|
||||
stream that flow control has wedged is not a cancel.
|
||||
- **Checksums cover the uncompressed payload.** Compression is a per-hop
|
||||
transport decision (reusing the existing `activation_compression` policies), so
|
||||
a checksum over the compressed frame would be invalidated by a hop that merely
|
||||
chose differently.
|
||||
- **Application-level flow-control credits, not just HTTP/2 windows.** HTTP/2
|
||||
bounds *bytes in flight*; it does not bound how much *work* a worker has queued,
|
||||
and a relayed frame gets no window at all. Credits bound queue occupancy and KV
|
||||
pressure, and negotiation takes the strictest bound of either peer so a sender
|
||||
cannot talk a worker into unbounded queues.
|
||||
|
||||
## 2. Files changed
|
||||
|
||||
New:
|
||||
|
||||
| Path | What |
|
||||
|---|---|
|
||||
| `packages/node/native/proto/shard_runtime.proto` | The schema (sha256 `077ee349…`, see `protocol.json`) |
|
||||
| `packages/node/native/CMakeLists.txt` | C++ generation + build wiring + ctest |
|
||||
| `packages/node/native/tests/test_shard_protocol_conformance.cpp` | C++ conformance test |
|
||||
| `packages/node/native/testdata/*.binpb` | Committed cross-language vectors |
|
||||
| `packages/node/native/README.md` | How to regenerate and build |
|
||||
| `packages/node/meshnet_node/native_protocol/__init__.py` | Public Python surface |
|
||||
| `packages/node/meshnet_node/native_protocol/codec.py` | Bundle encode/decode, fragmentation, CRC32C, chunking, FC negotiation |
|
||||
| `packages/node/meshnet_node/native_protocol/conformance.py` | Canonical vectors shared by both languages |
|
||||
| `packages/node/meshnet_node/native_protocol/generated/` | Generated Python stubs (committed) |
|
||||
| `scripts/generate_native_protocol.py` | Python generation, with `--check` |
|
||||
| `scripts/generate_protocol_goldens.py` | Vector generation, with `--check` |
|
||||
| `scripts/bootstrap_native_toolchain.sh` | Builds protobuf C++ from source |
|
||||
| `tests/test_native_shard_protocol.py` | 29 Python tests |
|
||||
|
||||
Modified:
|
||||
|
||||
- `packages/node/pyproject.toml` — added `grpcio>=1.60`, `protobuf>=5`; new `proto`
|
||||
extra pinning `grpcio-tools==1.82.1`.
|
||||
|
||||
No other working-tree file was touched. `git status` before this story was clean.
|
||||
|
||||
## 3. Commands and real results
|
||||
|
||||
See `commands.txt` for the exact ordered list. Results:
|
||||
|
||||
```
|
||||
python scripts/generate_native_protocol.py --check -> generated stubs are up to date
|
||||
python scripts/generate_protocol_goldens.py --check -> conformance vectors are up to date
|
||||
|
||||
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install
|
||||
-- gRPC C++ not found: building message types only (sufficient for the conformance test)
|
||||
cmake --build build/native -j -> Built target shard_protocol_conformance
|
||||
ctest --test-dir build/native --output-on-failure -> 1/1 Test #1: shard_protocol_conformance ... Passed
|
||||
100% tests passed out of 1
|
||||
|
||||
cmp build/native/cpp_roundtrip.binpb \
|
||||
packages/node/native/testdata/session_request_golden.binpb -> identical (exit 0)
|
||||
|
||||
pytest -q tests/test_native_shard_protocol.py -> 29 passed
|
||||
pytest -q (full suite) -> 712 passed, 12 skipped
|
||||
compileall -q packages tests -> OK (exit 0)
|
||||
git diff --check -> clean (exit 0)
|
||||
```
|
||||
|
||||
The C++ lane was rebuilt from scratch (`rm -rf build/native`) using only the
|
||||
documented commands, and reproduced the same result.
|
||||
|
||||
### Full-suite note — a pre-existing flaky test
|
||||
|
||||
`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy`
|
||||
is **flaky on a clean tree, independent of this story**. Reproduction, run
|
||||
*before any DGR-002 file existed* (working tree clean, `git status` empty):
|
||||
|
||||
```
|
||||
pytest -q -> 1 failed, 682 passed, 12 skipped
|
||||
FAILED tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
|
||||
# same test, three consecutive isolated runs on the same clean tree:
|
||||
pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy
|
||||
-> 1 passed in 1.76s
|
||||
-> 1 failed in 4.39s
|
||||
-> 1 passed in 1.10s
|
||||
```
|
||||
|
||||
It is a timing race in proxy cancellation (a 3-second in-flight generation raced
|
||||
against the cancel assertion), not a deterministic failure, and it touches no code
|
||||
this story changes. The final full-suite run with DGR-002 applied was green
|
||||
(`712 passed, 12 skipped`) — 683 pre-existing tests plus this story's 29. Flagged
|
||||
for whoever owns the tracker cancel path; **not** fixed here, since silently
|
||||
touching another story's code is out of scope.
|
||||
|
||||
## 4. Acceptance criteria
|
||||
|
||||
| Criterion | Where it is proven |
|
||||
|---|---|
|
||||
| Schema for capability, health, session stream, release, cancellation | `shard_runtime.proto` `service ShardRuntime`; `test_service_exposes_capability_health_session_release_and_cancel` |
|
||||
| One long-lived bidi stream per Activation Seam, with deadlines, cancellation, flow control, structured errors | `rpc Session (stream) returns (stream)`; `test_session_is_one_long_lived_bidirectional_stream`; `Envelope.deadline_unix_nanos`, `CancelSignal` + unary `Cancel`, `FlowControl`, `ShardError` |
|
||||
| Bounded chunking for prefill; small decode fast path | `ChunkInfo` + `plan_prefill_chunks` (128-token bound, ADR-0008); `DecodeStep`; `test_prefill_is_split_into_bounded_token_aligned_chunks`, `test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk` |
|
||||
| Envelope carries schema version, work id, session id, epoch, fingerprint, range/effective start, phase, position, idempotency step, cache expectation, compression, checksum | `Envelope` + `NamedTensor`; `test_envelope_carries_every_field_the_protocol_promises` asserts against the **descriptor**, so deleting a field from the `.proto` fails the test |
|
||||
| Versioned named-tensor bundle: name, shape, dtype, byte order, fragments | `TensorBundle`/`NamedTensor`/`TensorFragment`; `test_named_tensor_bundle_is_versioned_and_fully_described`, `test_bundle_round_trips_multiple_named_tensors` |
|
||||
| Round-trip + compatibility tests in Python and C++ | 29 Python tests; C++ `ctest` 1/1; cross-language byte equality |
|
||||
| Targeted pytest passes | 29 passed |
|
||||
| `compileall packages tests` | exit 0 |
|
||||
| `git diff --check` | exit 0 |
|
||||
| Default tests deterministic, download-free, credit-free, GPU-free | Pure in-memory protobuf; no model, no network, no GPU |
|
||||
| Full deterministic pytest passes, or pre-existing failure recorded | 712 passed, 12 skipped; flaky pre-existing test documented above with clean-tree reproduction |
|
||||
|
||||
## 5. How the cross-language claim is actually earned
|
||||
|
||||
Two codecs that each round-trip their own output prove only that each is
|
||||
self-consistent. Instead:
|
||||
|
||||
1. Python builds the canonical `SessionRequest` and commits its bytes.
|
||||
2. The C++ test parses **those** bytes, asserts every field, recomputes the CRC32C
|
||||
**from the polynomial in independent C++ code**, reassembles the multi-fragment
|
||||
tensor, and re-serializes to `cpp_roundtrip.binpb`.
|
||||
3. `test_cpp_and_python_agree_byte_for_byte` asserts that file equals the golden.
|
||||
|
||||
Compatibility is tested in both languages: an unknown field from a newer peer
|
||||
survives a parse/serialize hop (a Shard forwards activations — silently stripping
|
||||
fields would corrupt a route it is merely a waypoint on), and a sparse message
|
||||
from an older peer parses to proto3 defaults.
|
||||
|
||||
## 6. Limitations and deferred work
|
||||
|
||||
- **gRPC C++ was not built or linked.** The C++ lane verifies the *schema* (message
|
||||
types), not a running gRPC C++ server, because this machine has no gRPC C++ stack
|
||||
and building it is a large dependency the conformance test does not need.
|
||||
`CMakeLists.txt` already generates and exports `shard_runtime_grpc` when
|
||||
`find_package(gRPC)` succeeds. **DGR-008 must install gRPC C++ and extend
|
||||
`scripts/bootstrap_native_toolchain.sh`.**
|
||||
- **No wire is exercised.** No client, server, or stream lifecycle exists yet — no
|
||||
deadline actually fires, no credit is actually consumed. This story defines and
|
||||
proves the contract; DGR-008/DGR-009 implement it.
|
||||
- The protobuf C++ toolchain used here was installed to `/tmp/pbsrc/install` (ephemeral).
|
||||
`scripts/bootstrap_native_toolchain.sh` reproduces it; prefer a durable prefix such
|
||||
as `build/native-toolchain`.
|
||||
- `crc32c` has a pure-Python fallback (used here) and picks up `google_crc32c` when
|
||||
present. The fallback is byte-exact but slow; a worker on the hot path should install
|
||||
the native package. Not a correctness limitation.
|
||||
- Compression on the wire is zstd-or-none only, matching the existing seam.
|
||||
|
||||
## 7. Compatibility and migration notes
|
||||
|
||||
- **This does not change the existing HTTP activation wire.** `X-Meshnet-Wire` stays
|
||||
at `2` and the legacy `/forward` path is untouched. The native protocol is a
|
||||
*separate* contract with its own `SchemaVersion`, starting at 1. Nothing in this
|
||||
story is on any live request path — it is additive.
|
||||
- Semantics are deliberately preserved from the existing ADRs so the two transports
|
||||
mean the same thing: `effective_start_layer` (ADR-0012), `CacheMode`/`expected_past_len`
|
||||
and `ERROR_CODE_CACHE_MISS` mapping to today's HTTP 409 `cache_miss` (ADR-0022),
|
||||
bfloat16 boundary dtype and 128-token prefill chunks (ADR-0008), fingerprint/recipe
|
||||
identity mirroring the capability report (ADR-0023).
|
||||
- `TensorFragment` field 5 (`uncompressed_size`) is **reserved**: it was removed
|
||||
because `NamedTensor.total_bytes` is the single source of truth. Never recycle it —
|
||||
a recycled field number is the one schema change peers cannot detect, because the
|
||||
bytes still parse.
|
||||
- Committed Python stubs are guarded by `--check` in the test suite, so they cannot
|
||||
drift from the schema unnoticed.
|
||||
|
||||
## 8. Handoff to dependent stories
|
||||
|
||||
- **DGR-003 (runtime recipe/fingerprint):** populate `Fingerprint`
|
||||
(`model_artifact_digest`, `runtime_recipe_digest`, `recipe_id`, `recipe_version`,
|
||||
`catalogue_version`). The mismatch outcome is already specified:
|
||||
`ERROR_CODE_FINGERPRINT_MISMATCH`. Do not invent a second identity struct.
|
||||
- **DGR-005/006 (range loading, architecture boundary):** the boundary payload is a
|
||||
**named bundle**, not a bare tensor — a boundary needing more than one tensor is
|
||||
already representable. Execute `[effective_start_layer, end_layer)`, never from
|
||||
`start_layer`.
|
||||
- **DGR-007 (concurrent sessions/KV):** isolate on `(route_session_id, route_epoch)`.
|
||||
`CacheExpectation`/`CacheResult` and `ERROR_CODE_CACHE_MISS` are the contract; a
|
||||
decode step whose `expected_past_len` does not match **must** miss, never fall back
|
||||
to a silent stateless forward. `idempotency_step` means a retried step is
|
||||
acknowledged (`Ack.duplicate`), not re-applied — re-applying advances the KV cache
|
||||
twice and desynchronises the route.
|
||||
- **DGR-008 (C++ worker):** link `shard_runtime_grpc` from `CMakeLists.txt`; you must
|
||||
first install gRPC C++ (see limitations). Honour `FlowControl` credits and the
|
||||
`max_chunk_bytes` bound. Use `packages/node/meshnet_node/native_protocol/codec.py`
|
||||
as the reference for fragment reassembly and checksum validation.
|
||||
- **DGR-009 (Meshnet integration):** the relay may carry these serialized frames as
|
||||
opaque binary — that is exactly why deadline/cancel/identity are in-band. Do not add
|
||||
a second control plane.
|
||||
- **Anyone editing the schema:** run both `--check` scripts; if a vector legitimately
|
||||
changes, regenerate it and say so, because the C++ test asserts those exact bytes.
|
||||
Reference in New Issue
Block a user