diff --git a/.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md b/.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md new file mode 100644 index 0000000..11de0cf --- /dev/null +++ b/.scratch/distributed-gguf-runtime/evidence/DGR-002/README.md @@ -0,0 +1,205 @@ +# DGR-002 — Adopt the versioned gRPC Shard protocol + +Status: **done**. Every acceptance criterion is met with real command output. +Evidence class: **synthetic/unit** — this story defines a schema and proves both +languages agree on it. No model, GPU, network peer or benchmark is involved, and +none is claimed. + +## 1. Summary + +`packages/node/native/proto/shard_runtime.proto` is now the semantic contract for +the native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). Python +and C++ both generate from it, and a shared committed conformance vector proves +they encode it identically — byte for byte. + +Design decisions worth carrying forward: + +- **Everything gRPC gives you is *also* in the schema.** Deadline, cancellation, + identity and flow control are carried as fields, not left to HTTP/2 metadata, + because the existing relay carries these frames as **opaque binary**. A relayed + frame has no HTTP/2 context to inherit a deadline or a channel identity from. + If it is not in the schema, it does not survive the relay. +- **Cancellation is both in-band and out-of-band.** `CancelSignal` rides the + stream; `Cancel` is also a unary RPC. A cancel that can only travel down a + stream that flow control has wedged is not a cancel. +- **Checksums cover the uncompressed payload.** Compression is a per-hop + transport decision (reusing the existing `activation_compression` policies), so + a checksum over the compressed frame would be invalidated by a hop that merely + chose differently. +- **Application-level flow-control credits, not just HTTP/2 windows.** HTTP/2 + bounds *bytes in flight*; it does not bound how much *work* a worker has queued, + and a relayed frame gets no window at all. Credits bound queue occupancy and KV + pressure, and negotiation takes the strictest bound of either peer so a sender + cannot talk a worker into unbounded queues. + +## 2. Files changed + +New: + +| Path | What | +|---|---| +| `packages/node/native/proto/shard_runtime.proto` | The schema (sha256 `077ee349…`, see `protocol.json`) | +| `packages/node/native/CMakeLists.txt` | C++ generation + build wiring + ctest | +| `packages/node/native/tests/test_shard_protocol_conformance.cpp` | C++ conformance test | +| `packages/node/native/testdata/*.binpb` | Committed cross-language vectors | +| `packages/node/native/README.md` | How to regenerate and build | +| `packages/node/meshnet_node/native_protocol/__init__.py` | Public Python surface | +| `packages/node/meshnet_node/native_protocol/codec.py` | Bundle encode/decode, fragmentation, CRC32C, chunking, FC negotiation | +| `packages/node/meshnet_node/native_protocol/conformance.py` | Canonical vectors shared by both languages | +| `packages/node/meshnet_node/native_protocol/generated/` | Generated Python stubs (committed) | +| `scripts/generate_native_protocol.py` | Python generation, with `--check` | +| `scripts/generate_protocol_goldens.py` | Vector generation, with `--check` | +| `scripts/bootstrap_native_toolchain.sh` | Builds protobuf C++ from source | +| `tests/test_native_shard_protocol.py` | 29 Python tests | + +Modified: + +- `packages/node/pyproject.toml` — added `grpcio>=1.60`, `protobuf>=5`; new `proto` + extra pinning `grpcio-tools==1.82.1`. + +No other working-tree file was touched. `git status` before this story was clean. + +## 3. Commands and real results + +See `commands.txt` for the exact ordered list. Results: + +``` +python scripts/generate_native_protocol.py --check -> generated stubs are up to date +python scripts/generate_protocol_goldens.py --check -> conformance vectors are up to date + +cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install + -- gRPC C++ not found: building message types only (sufficient for the conformance test) +cmake --build build/native -j -> Built target shard_protocol_conformance +ctest --test-dir build/native --output-on-failure -> 1/1 Test #1: shard_protocol_conformance ... Passed + 100% tests passed out of 1 + +cmp build/native/cpp_roundtrip.binpb \ + packages/node/native/testdata/session_request_golden.binpb -> identical (exit 0) + +pytest -q tests/test_native_shard_protocol.py -> 29 passed +pytest -q (full suite) -> 712 passed, 12 skipped +compileall -q packages tests -> OK (exit 0) +git diff --check -> clean (exit 0) +``` + +The C++ lane was rebuilt from scratch (`rm -rf build/native`) using only the +documented commands, and reproduced the same result. + +### Full-suite note — a pre-existing flaky test + +`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy` +is **flaky on a clean tree, independent of this story**. Reproduction, run +*before any DGR-002 file existed* (working tree clean, `git status` empty): + +``` +pytest -q -> 1 failed, 682 passed, 12 skipped + FAILED tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy + +# same test, three consecutive isolated runs on the same clean tree: +pytest -q tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy + -> 1 passed in 1.76s + -> 1 failed in 4.39s + -> 1 passed in 1.10s +``` + +It is a timing race in proxy cancellation (a 3-second in-flight generation raced +against the cancel assertion), not a deterministic failure, and it touches no code +this story changes. The final full-suite run with DGR-002 applied was green +(`712 passed, 12 skipped`) — 683 pre-existing tests plus this story's 29. Flagged +for whoever owns the tracker cancel path; **not** fixed here, since silently +touching another story's code is out of scope. + +## 4. Acceptance criteria + +| Criterion | Where it is proven | +|---|---| +| Schema for capability, health, session stream, release, cancellation | `shard_runtime.proto` `service ShardRuntime`; `test_service_exposes_capability_health_session_release_and_cancel` | +| One long-lived bidi stream per Activation Seam, with deadlines, cancellation, flow control, structured errors | `rpc Session (stream) returns (stream)`; `test_session_is_one_long_lived_bidirectional_stream`; `Envelope.deadline_unix_nanos`, `CancelSignal` + unary `Cancel`, `FlowControl`, `ShardError` | +| Bounded chunking for prefill; small decode fast path | `ChunkInfo` + `plan_prefill_chunks` (128-token bound, ADR-0008); `DecodeStep`; `test_prefill_is_split_into_bounded_token_aligned_chunks`, `test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk` | +| Envelope carries schema version, work id, session id, epoch, fingerprint, range/effective start, phase, position, idempotency step, cache expectation, compression, checksum | `Envelope` + `NamedTensor`; `test_envelope_carries_every_field_the_protocol_promises` asserts against the **descriptor**, so deleting a field from the `.proto` fails the test | +| Versioned named-tensor bundle: name, shape, dtype, byte order, fragments | `TensorBundle`/`NamedTensor`/`TensorFragment`; `test_named_tensor_bundle_is_versioned_and_fully_described`, `test_bundle_round_trips_multiple_named_tensors` | +| Round-trip + compatibility tests in Python and C++ | 29 Python tests; C++ `ctest` 1/1; cross-language byte equality | +| Targeted pytest passes | 29 passed | +| `compileall packages tests` | exit 0 | +| `git diff --check` | exit 0 | +| Default tests deterministic, download-free, credit-free, GPU-free | Pure in-memory protobuf; no model, no network, no GPU | +| Full deterministic pytest passes, or pre-existing failure recorded | 712 passed, 12 skipped; flaky pre-existing test documented above with clean-tree reproduction | + +## 5. How the cross-language claim is actually earned + +Two codecs that each round-trip their own output prove only that each is +self-consistent. Instead: + +1. Python builds the canonical `SessionRequest` and commits its bytes. +2. The C++ test parses **those** bytes, asserts every field, recomputes the CRC32C + **from the polynomial in independent C++ code**, reassembles the multi-fragment + tensor, and re-serializes to `cpp_roundtrip.binpb`. +3. `test_cpp_and_python_agree_byte_for_byte` asserts that file equals the golden. + +Compatibility is tested in both languages: an unknown field from a newer peer +survives a parse/serialize hop (a Shard forwards activations — silently stripping +fields would corrupt a route it is merely a waypoint on), and a sparse message +from an older peer parses to proto3 defaults. + +## 6. Limitations and deferred work + +- **gRPC C++ was not built or linked.** The C++ lane verifies the *schema* (message + types), not a running gRPC C++ server, because this machine has no gRPC C++ stack + and building it is a large dependency the conformance test does not need. + `CMakeLists.txt` already generates and exports `shard_runtime_grpc` when + `find_package(gRPC)` succeeds. **DGR-008 must install gRPC C++ and extend + `scripts/bootstrap_native_toolchain.sh`.** +- **No wire is exercised.** No client, server, or stream lifecycle exists yet — no + deadline actually fires, no credit is actually consumed. This story defines and + proves the contract; DGR-008/DGR-009 implement it. +- The protobuf C++ toolchain used here was installed to `/tmp/pbsrc/install` (ephemeral). + `scripts/bootstrap_native_toolchain.sh` reproduces it; prefer a durable prefix such + as `build/native-toolchain`. +- `crc32c` has a pure-Python fallback (used here) and picks up `google_crc32c` when + present. The fallback is byte-exact but slow; a worker on the hot path should install + the native package. Not a correctness limitation. +- Compression on the wire is zstd-or-none only, matching the existing seam. + +## 7. Compatibility and migration notes + +- **This does not change the existing HTTP activation wire.** `X-Meshnet-Wire` stays + at `2` and the legacy `/forward` path is untouched. The native protocol is a + *separate* contract with its own `SchemaVersion`, starting at 1. Nothing in this + story is on any live request path — it is additive. +- Semantics are deliberately preserved from the existing ADRs so the two transports + mean the same thing: `effective_start_layer` (ADR-0012), `CacheMode`/`expected_past_len` + and `ERROR_CODE_CACHE_MISS` mapping to today's HTTP 409 `cache_miss` (ADR-0022), + bfloat16 boundary dtype and 128-token prefill chunks (ADR-0008), fingerprint/recipe + identity mirroring the capability report (ADR-0023). +- `TensorFragment` field 5 (`uncompressed_size`) is **reserved**: it was removed + because `NamedTensor.total_bytes` is the single source of truth. Never recycle it — + a recycled field number is the one schema change peers cannot detect, because the + bytes still parse. +- Committed Python stubs are guarded by `--check` in the test suite, so they cannot + drift from the schema unnoticed. + +## 8. Handoff to dependent stories + +- **DGR-003 (runtime recipe/fingerprint):** populate `Fingerprint` + (`model_artifact_digest`, `runtime_recipe_digest`, `recipe_id`, `recipe_version`, + `catalogue_version`). The mismatch outcome is already specified: + `ERROR_CODE_FINGERPRINT_MISMATCH`. Do not invent a second identity struct. +- **DGR-005/006 (range loading, architecture boundary):** the boundary payload is a + **named bundle**, not a bare tensor — a boundary needing more than one tensor is + already representable. Execute `[effective_start_layer, end_layer)`, never from + `start_layer`. +- **DGR-007 (concurrent sessions/KV):** isolate on `(route_session_id, route_epoch)`. + `CacheExpectation`/`CacheResult` and `ERROR_CODE_CACHE_MISS` are the contract; a + decode step whose `expected_past_len` does not match **must** miss, never fall back + to a silent stateless forward. `idempotency_step` means a retried step is + acknowledged (`Ack.duplicate`), not re-applied — re-applying advances the KV cache + twice and desynchronises the route. +- **DGR-008 (C++ worker):** link `shard_runtime_grpc` from `CMakeLists.txt`; you must + first install gRPC C++ (see limitations). Honour `FlowControl` credits and the + `max_chunk_bytes` bound. Use `packages/node/meshnet_node/native_protocol/codec.py` + as the reference for fragment reassembly and checksum validation. +- **DGR-009 (Meshnet integration):** the relay may carry these serialized frames as + opaque binary — that is exactly why deadline/cancel/identity are in-band. Do not add + a second control plane. +- **Anyone editing the schema:** run both `--check` scripts; if a vector legitimately + changes, regenerate it and say so, because the C++ test asserts those exact bytes. diff --git a/.scratch/distributed-gguf-runtime/evidence/DGR-002/commands.txt b/.scratch/distributed-gguf-runtime/evidence/DGR-002/commands.txt new file mode 100644 index 0000000..2c197b7 --- /dev/null +++ b/.scratch/distributed-gguf-runtime/evidence/DGR-002/commands.txt @@ -0,0 +1,29 @@ +# DGR-002 — exact commands, in order. Run from the repository root. +# Interpreter: /.venv/bin/python (CPython 3.14.6). Deterministic, GPU-free, +# no model download, no API credits. + +# --- toolchain (this machine had no protoc, no cmake, no protobuf C++ headers) +.venv/bin/python -m pip install grpcio-tools==1.82.1 grpcio==1.82.1 cmake==4.4.0 +scripts/bootstrap_native_toolchain.sh /tmp/pbsrc/install # protobuf C++ 33.1 + abseil 20250814.1 + +# --- schema generation (Python stubs; committed) +.venv/bin/python scripts/generate_native_protocol.py +.venv/bin/python scripts/generate_native_protocol.py --check # -> "generated stubs are up to date" + +# --- cross-language conformance vectors (committed) +.venv/bin/python scripts/generate_protocol_goldens.py +.venv/bin/python scripts/generate_protocol_goldens.py --check # -> "conformance vectors are up to date" + +# --- C++ generation, build and conformance test +cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH=/tmp/pbsrc/install +cmake --build build/native -j"$(nproc)" +ctest --test-dir build/native --output-on-failure # -> 1/1 Passed +cmp build/native/cpp_roundtrip.binpb packages/node/native/testdata/session_request_golden.binpb + +# --- Python tests +.venv/bin/python -m pytest -q tests/test_native_shard_protocol.py # -> 29 passed +.venv/bin/python -m pytest -q # full suite + +# --- repository gates +.venv/bin/python -m compileall -q packages tests +git diff --check diff --git a/.scratch/distributed-gguf-runtime/evidence/DGR-002/protocol.json b/.scratch/distributed-gguf-runtime/evidence/DGR-002/protocol.json new file mode 100644 index 0000000..2dda264 --- /dev/null +++ b/.scratch/distributed-gguf-runtime/evidence/DGR-002/protocol.json @@ -0,0 +1,84 @@ +{ + "schema_version": "SCHEMA_VERSION_1", + "bundle_version": 1, + "proto_path": "packages/node/native/proto/shard_runtime.proto", + "proto_sha256": "077ee34961ab787c67457b1a373f73ec3d577f67086d6bc6a32435f37395b7d6", + "protoc": "grpc_tools 1.82.1 (python) / protobuf 33.1 (C++)", + "service": { + "GetCapability": { + "client_streaming": false, + "server_streaming": false + }, + "Health": { + "client_streaming": false, + "server_streaming": false + }, + "Session": { + "client_streaming": true, + "server_streaming": true + }, + "Release": { + "client_streaming": false, + "server_streaming": false + }, + "Cancel": { + "client_streaming": false, + "server_streaming": false + } + }, + "envelope_fields": [ + "cache_expectation", + "chunk", + "deadline_unix_nanos", + "fingerprint", + "idempotency_step", + "phase", + "position", + "route_epoch", + "route_session_id", + "schema_version", + "shard_range", + "work_id" + ], + "named_tensor_fields": [ + "byte_order", + "checksum", + "compression", + "dtype", + "fragments", + "name", + "shape", + "total_bytes" + ], + "phases": [ + "PHASE_UNSPECIFIED", + "PHASE_PREFILL", + "PHASE_DECODE", + "PHASE_RELEASE", + "PHASE_CANCEL" + ], + "error_codes": [ + "ERROR_CODE_UNSPECIFIED", + "ERROR_CODE_SCHEMA_UNSUPPORTED", + "ERROR_CODE_FINGERPRINT_MISMATCH", + "ERROR_CODE_EPOCH_STALE", + "ERROR_CODE_SHARD_RANGE_MISMATCH", + "ERROR_CODE_CACHE_MISS", + "ERROR_CODE_RESOURCE_EXHAUSTED", + "ERROR_CODE_PAYLOAD_CORRUPT", + "ERROR_CODE_CANCELLED", + "ERROR_CODE_DEADLINE_EXCEEDED", + "ERROR_CODE_FLOW_CONTROL_VIOLATION", + "ERROR_CODE_INTERNAL" + ], + "bounds": { + "max_prefill_chunk_tokens": 128, + "max_chunk_bytes": 4194304, + "max_fragment_bytes": 1048576, + "max_inflight_chunks": 8 + }, + "golden_vectors": { + "session_request_golden.binpb": "c2c3df8a717ddeae7bd99624d2c7f34c09a518988de990237fe313b75cff0817", + "capability_report_golden.binpb": "71ac5f150775f398515b43a63596a5cbe8d2ad607e7e4de56bd44fbe7987080c" + } +} diff --git a/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md b/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md index d38c8ce..1b094c6 100644 --- a/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md +++ b/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md @@ -1,6 +1,6 @@ # 02 — Adopt the versioned gRPC Shard protocol -Status: ready-for-agent +Status: done ## Mandatory fresh-session context @@ -22,22 +22,22 @@ As a node developer, I need a battle-proven streaming protocol so that Python an ## Acceptance criteria -- [ ] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations. -- [ ] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors. -- [ ] Define bounded chunking for prefill and a small decode fast path. -- [ ] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum. -- [ ] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments. -- [ ] Add generated-schema round-trip and compatibility tests in Python and C++. -- [ ] Targeted pytest tests pass -- [ ] python -m compileall packages tests passes for Python changes -- [ ] git diff --check passes -- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free -- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction -- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code -- [ ] Read and verify every dependency evidence README before relying on dependency behavior -- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story -- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff -- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes +- [x] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations. +- [x] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors. +- [x] Define bounded chunking for prefill and a small decode fast path. +- [x] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum. +- [x] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments. +- [x] Add generated-schema round-trip and compatibility tests in Python and C++. +- [x] Targeted pytest tests pass +- [x] python -m compileall packages tests passes for Python changes +- [x] git diff --check passes +- [x] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free +- [x] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction +- [x] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code +- [x] Read and verify every dependency evidence README before relying on dependency behavior +- [x] Preserve all pre-existing working-tree changes and stage only files belonging to this story +- [x] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff +- [x] Update only this story issue to Status: done after every acceptance criterion and quality gate passes ## Dependency handoff diff --git a/packages/node/meshnet_node/native_protocol/__init__.py b/packages/node/meshnet_node/native_protocol/__init__.py new file mode 100644 index 0000000..235f9f5 --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/__init__.py @@ -0,0 +1,63 @@ +"""The native Shard protocol: Protobuf over gRPC/HTTP2 (ADR-0020). + +`packages/node/native/proto/shard_runtime.proto` is the contract. This package +is how Python speaks it: the generated stubs plus the validation, framing and +chunking rules that the stubs cannot express. + +Import the message types from here rather than reaching into `.generated`, so +the location of build output stays an implementation detail:: + + from meshnet_node.native_protocol import pb, encode_tensor, decode_bundle +""" + +from __future__ import annotations + +from .generated import shard_runtime_pb2 as pb +from .codec import ( + BUNDLE_VERSION, + DEFAULT_MAX_CHUNK_BYTES, + DEFAULT_MAX_FRAGMENT_BYTES, + DEFAULT_MAX_INFLIGHT_CHUNKS, + DEFAULT_MAX_PREFILL_CHUNK_TOKENS, + HIDDEN_STATES, + SCHEMA_VERSION, + PayloadCorrupt, + PrefillChunk, + ProtocolError, + checksum_of, + crc32c, + decode_bundle, + decode_tensor, + default_flow_control, + encode_bundle, + encode_tensor, + expected_bytes, + itemsize, + negotiate_flow_control, + plan_prefill_chunks, +) + +__all__ = [ + "BUNDLE_VERSION", + "DEFAULT_MAX_CHUNK_BYTES", + "DEFAULT_MAX_FRAGMENT_BYTES", + "DEFAULT_MAX_INFLIGHT_CHUNKS", + "DEFAULT_MAX_PREFILL_CHUNK_TOKENS", + "HIDDEN_STATES", + "SCHEMA_VERSION", + "PayloadCorrupt", + "PrefillChunk", + "ProtocolError", + "checksum_of", + "crc32c", + "decode_bundle", + "decode_tensor", + "default_flow_control", + "encode_bundle", + "encode_tensor", + "expected_bytes", + "itemsize", + "negotiate_flow_control", + "pb", + "plan_prefill_chunks", +] diff --git a/packages/node/meshnet_node/native_protocol/codec.py b/packages/node/meshnet_node/native_protocol/codec.py new file mode 100644 index 0000000..00eb4db --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/codec.py @@ -0,0 +1,383 @@ +"""Encode and decode the native Shard protocol's named-tensor bundles. + +The generated stubs give us message *structure*; they cannot enforce the +invariants that keep a distributed forward correct. A bundle whose declared +shape disagrees with its byte count, whose fragments leave a hole, or whose +checksum does not match is not a slightly-wrong activation — it is silently +wrong tokens for the rest of the generation. So decoding is validating: every +path into a tensor's bytes goes through :func:`decode_tensor`, which refuses a +payload it cannot fully account for. + +Compression is a transport optimisation and is decided by the same policy layer +the existing HTTP seam already uses (``activation_compression``), so a node's +tuned thresholds apply to both transports. +""" + +from __future__ import annotations + +from dataclasses import dataclass +import struct +from typing import Iterable, Sequence + +from ..activation_compression import ( + CompressionPolicy, + compress_activation, + decompress_activation, +) +from .generated import shard_runtime_pb2 as pb + +# The schema generation this build speaks. A peer offering something else is +# rejected at the handshake rather than being half-understood. +SCHEMA_VERSION = pb.SCHEMA_VERSION_1 + +# Generation of the tensor-bundle layout, versioned independently of the +# protocol so a boundary payload can evolve without a protocol bump. +BUNDLE_VERSION = 1 + +# Token-aligned prefill chunk bound. 128 tokens is the size ADR-0008 already +# uses on the HTTP seam; keeping it identical means seam bytes stay comparable +# across transports. +DEFAULT_MAX_PREFILL_CHUNK_TOKENS = 128 + +# gRPC's default maximum receive size. Fragmenting below it keeps us inside the +# default limits of any conformant peer instead of requiring every client to +# raise its window. +DEFAULT_MAX_CHUNK_BYTES = 4 * 1024 * 1024 + +# Leave room for envelope and framing overhead inside one chunk message. +DEFAULT_MAX_FRAGMENT_BYTES = 1024 * 1024 + +DEFAULT_MAX_INFLIGHT_CHUNKS = 8 + +# Canonical boundary tensor name for a dense transformer hidden state. +HIDDEN_STATES = "hidden_states" + +_DTYPE_ITEMSIZE: dict[int, int] = { + pb.DTYPE_BFLOAT16: 2, + pb.DTYPE_FLOAT16: 2, + pb.DTYPE_FLOAT32: 4, + pb.DTYPE_INT32: 4, + pb.DTYPE_INT64: 8, + pb.DTYPE_UINT8: 1, + pb.DTYPE_INT8: 1, + pb.DTYPE_BOOL: 1, +} + + +class ProtocolError(Exception): + """A peer sent something this build cannot safely interpret.""" + + +class PayloadCorrupt(ProtocolError): + """A tensor payload failed validation: size, coverage, or checksum.""" + + +def itemsize(dtype: int) -> int: + try: + return _DTYPE_ITEMSIZE[dtype] + except KeyError: + raise ProtocolError(f"unsupported dtype {dtype}") from None + + +def expected_bytes(shape: Sequence[int], dtype: int) -> int: + """Byte count a tensor of `shape` and `dtype` must occupy.""" + if any(dim < 0 for dim in shape): + raise ProtocolError(f"negative dimension in shape {list(shape)}") + count = 1 + for dim in shape: + count *= dim + return count * itemsize(dtype) + + +# --- CRC32C ---------------------------------------------------------------- +# +# CRC32C (Castagnoli), not zlib's CRC32: it is the checksum gRPC, and the +# storage systems these payloads pass through, already use, and hardware +# implements it. `google_crc32c` is used when present; the table fallback keeps +# the default test suite dependency-free. + +_CRC32C_POLY = 0x82F63B78 +_CRC32C_TABLE: list[int] = [] +for _i in range(256): + _c = _i + for _ in range(8): + _c = (_c >> 1) ^ (_CRC32C_POLY if _c & 1 else 0) + _CRC32C_TABLE.append(_c) + +try: # pragma: no cover - depends on an optional native package + from google_crc32c import value as _fast_crc32c +except ImportError: # pragma: no cover + _fast_crc32c = None + + +def crc32c(data: bytes) -> int: + if _fast_crc32c is not None: # pragma: no cover - optional fast path + return _fast_crc32c(data) + crc = 0xFFFFFFFF + for byte in data: + crc = (crc >> 8) ^ _CRC32C_TABLE[(crc ^ byte) & 0xFF] + return crc ^ 0xFFFFFFFF + + +def checksum_of(data: bytes) -> pb.Checksum: + return pb.Checksum( + algorithm=pb.CHECKSUM_ALGORITHM_CRC32C, + value=struct.pack(">I", crc32c(data)), + ) + + +# --- Tensors --------------------------------------------------------------- + + +def encode_tensor( + name: str, + data: bytes, + shape: Sequence[int], + dtype: int = pb.DTYPE_BFLOAT16, + *, + policy: CompressionPolicy | None = None, + max_fragment_bytes: int = DEFAULT_MAX_FRAGMENT_BYTES, +) -> pb.NamedTensor: + """Build a NamedTensor, compressing and fragmenting as needed. + + `data` is the uncompressed little-endian payload. The checksum is taken over + it *before* compression so it stays valid whichever framing a hop chooses. + """ + if max_fragment_bytes <= 0: + raise ProtocolError("max_fragment_bytes must be positive") + declared = expected_bytes(shape, dtype) + if len(data) != declared: + raise ProtocolError( + f"tensor {name!r} declares shape {list(shape)} ({declared} bytes) " + f"but carries {len(data)} bytes" + ) + + body = data + compression = pb.COMPRESSION_NONE + if policy is not None: + result = compress_activation(data, policy) + if result.compressed: + body = result.body + compression = pb.COMPRESSION_ZSTD + + tensor = pb.NamedTensor( + name=name, + shape=list(shape), + dtype=dtype, + byte_order=pb.BYTE_ORDER_LITTLE_ENDIAN, + total_bytes=len(data), + compression=compression, + checksum=checksum_of(data), + ) + + # Fragment the wire body (compressed if we compressed). Offsets walk the + # wire body so a receiver can verify coverage without assuming arrival + # order; a zstd frame is not decodable per fragment, so reassembly comes + # first and decompression happens once, in decode_tensor. + slices = [body[i : i + max_fragment_bytes] for i in range(0, len(body), max_fragment_bytes)] + if not slices: + # A zero-element tensor is legal (e.g. an empty mask) and still needs a + # fragment, so coverage checks have something to verify. + slices = [b""] + + offset = 0 + for index, piece in enumerate(slices): + tensor.fragments.append( + pb.TensorFragment( + fragment_index=index, + fragment_count=len(slices), + byte_offset=offset, + payload=piece, + ) + ) + offset += len(piece) + return tensor + + +def decode_tensor(tensor: pb.NamedTensor) -> bytes: + """Reassemble, decompress and validate a NamedTensor's payload. + + Raises PayloadCorrupt rather than returning a payload it cannot fully + account for: a hole in the fragments or a bad checksum means the activation + is not what the sender computed, and continuing would corrupt the route. + """ + if tensor.byte_order == pb.BYTE_ORDER_BIG_ENDIAN: + raise ProtocolError(f"tensor {tensor.name!r} is big-endian; wire order is little-endian") + if tensor.byte_order != pb.BYTE_ORDER_LITTLE_ENDIAN: + raise ProtocolError(f"tensor {tensor.name!r} declares no byte order") + + if not tensor.fragments: + raise PayloadCorrupt(f"tensor {tensor.name!r} carries no fragments") + + fragments = sorted(tensor.fragments, key=lambda f: f.byte_offset) + count = fragments[0].fragment_count + if any(f.fragment_count != count for f in fragments): + raise PayloadCorrupt(f"tensor {tensor.name!r} has inconsistent fragment_count") + if len(fragments) != count: + raise PayloadCorrupt( + f"tensor {tensor.name!r} expects {count} fragments but carries {len(fragments)}" + ) + if {f.fragment_index for f in fragments} != set(range(count)): + raise PayloadCorrupt(f"tensor {tensor.name!r} has duplicate or missing fragment indices") + + # Contiguity: offsets must tile the body exactly, with no hole and no overlap. + body = bytearray() + for fragment in fragments: + if fragment.byte_offset != len(body): + raise PayloadCorrupt( + f"tensor {tensor.name!r} fragment {fragment.fragment_index} starts at " + f"{fragment.byte_offset}, expected {len(body)}" + ) + body.extend(fragment.payload) + + if tensor.compression == pb.COMPRESSION_ZSTD: + data = decompress_activation(bytes(body), "zstd").body + elif tensor.compression in (pb.COMPRESSION_NONE, pb.COMPRESSION_UNSPECIFIED): + data = bytes(body) + else: + raise ProtocolError(f"tensor {tensor.name!r} uses unsupported compression") + + if len(data) != tensor.total_bytes: + raise PayloadCorrupt( + f"tensor {tensor.name!r} declares {tensor.total_bytes} bytes but " + f"reassembled {len(data)}" + ) + declared = expected_bytes(tensor.shape, tensor.dtype) + if declared != tensor.total_bytes: + raise PayloadCorrupt( + f"tensor {tensor.name!r} shape {list(tensor.shape)} implies {declared} bytes " + f"but declares {tensor.total_bytes}" + ) + + algorithm = tensor.checksum.algorithm + if algorithm == pb.CHECKSUM_ALGORITHM_CRC32C: + if tensor.checksum.value != struct.pack(">I", crc32c(data)): + raise PayloadCorrupt(f"tensor {tensor.name!r} failed its CRC32C check") + elif algorithm not in (pb.CHECKSUM_ALGORITHM_NONE, pb.CHECKSUM_ALGORITHM_UNSPECIFIED): + raise ProtocolError(f"tensor {tensor.name!r} uses unsupported checksum algorithm") + + return data + + +def encode_bundle(tensors: Iterable[pb.NamedTensor]) -> pb.TensorBundle: + return pb.TensorBundle(bundle_version=BUNDLE_VERSION, tensors=list(tensors)) + + +def decode_bundle(bundle: pb.TensorBundle) -> dict[str, bytes]: + """Validate every tensor in a bundle and return name -> payload.""" + if bundle.bundle_version > BUNDLE_VERSION: + raise ProtocolError( + f"bundle version {bundle.bundle_version} is newer than this build " + f"understands ({BUNDLE_VERSION})" + ) + payloads: dict[str, bytes] = {} + for tensor in bundle.tensors: + if not tensor.name: + raise ProtocolError("bundle carries an unnamed tensor") + if tensor.name in payloads: + raise ProtocolError(f"bundle carries duplicate tensor {tensor.name!r}") + payloads[tensor.name] = decode_tensor(tensor) + return payloads + + +# --- Bounded prefill chunking ---------------------------------------------- + + +@dataclass(frozen=True) +class PrefillChunk: + """One token-aligned slice of a prefill.""" + + chunk_index: int + chunk_count: int + first_position: int + token_count: int + + @property + def final_chunk(self) -> bool: + return self.chunk_index == self.chunk_count - 1 + + def chunk_info(self) -> pb.ChunkInfo: + return pb.ChunkInfo( + chunk_index=self.chunk_index, + chunk_count=self.chunk_count, + final_chunk=self.final_chunk, + ) + + def position(self) -> pb.PositionSpan: + return pb.PositionSpan( + first_position=self.first_position, token_count=self.token_count + ) + + +def plan_prefill_chunks( + total_tokens: int, + *, + first_position: int = 0, + max_tokens: int = DEFAULT_MAX_PREFILL_CHUNK_TOKENS, +) -> list[PrefillChunk]: + """Split a prefill into bounded, token-aligned chunks. + + Splits fall on token boundaries only (ADR-0008): a fragment of a token's + hidden state is not a thing a receiver can execute. + """ + if total_tokens <= 0: + raise ProtocolError("a prefill must carry at least one token") + if max_tokens <= 0: + raise ProtocolError("max_tokens must be positive") + + count = (total_tokens + max_tokens - 1) // max_tokens + chunks = [] + for index in range(count): + offset = index * max_tokens + chunks.append( + PrefillChunk( + chunk_index=index, + chunk_count=count, + first_position=first_position + offset, + token_count=min(max_tokens, total_tokens - offset), + ) + ) + return chunks + + +def default_flow_control() -> pb.FlowControl: + return pb.FlowControl( + credits_granted=DEFAULT_MAX_INFLIGHT_CHUNKS, + max_inflight_chunks=DEFAULT_MAX_INFLIGHT_CHUNKS, + max_chunk_bytes=DEFAULT_MAX_CHUNK_BYTES, + max_prefill_chunk_tokens=DEFAULT_MAX_PREFILL_CHUNK_TOKENS, + ) + + +def negotiate_flow_control( + proposed: pb.FlowControl, limits: pb.FlowControl +) -> pb.FlowControl: + """Settle a stream's limits: the strictest bound of either peer wins. + + Taking the minimum means neither peer can raise the other's ceiling, so a + misconfigured — or hostile — sender cannot talk a worker into unbounded + queues by proposing a large window. + """ + + def _min(a: int, b: int, fallback: int) -> int: + candidates = [v for v in (a, b) if v > 0] + return min(candidates) if candidates else fallback + + return pb.FlowControl( + credits_granted=_min( + proposed.credits_granted, limits.credits_granted, DEFAULT_MAX_INFLIGHT_CHUNKS + ), + max_inflight_chunks=_min( + proposed.max_inflight_chunks, + limits.max_inflight_chunks, + DEFAULT_MAX_INFLIGHT_CHUNKS, + ), + max_chunk_bytes=_min( + proposed.max_chunk_bytes, limits.max_chunk_bytes, DEFAULT_MAX_CHUNK_BYTES + ), + max_prefill_chunk_tokens=_min( + proposed.max_prefill_chunk_tokens, + limits.max_prefill_chunk_tokens, + DEFAULT_MAX_PREFILL_CHUNK_TOKENS, + ), + ) diff --git a/packages/node/meshnet_node/native_protocol/conformance.py b/packages/node/meshnet_node/native_protocol/conformance.py new file mode 100644 index 0000000..a21ea33 --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/conformance.py @@ -0,0 +1,141 @@ +"""Canonical conformance vectors for the native Shard protocol. + +Two independently-written codecs that each round-trip their own output prove +nothing about each other. These vectors are the shared reference: Python builds +the canonical message, the bytes are committed under +`packages/node/native/testdata/`, and the C++ test parses those exact bytes and +asserts the same field values. A change that alters the wire meaning of a field +breaks the vector in both languages instead of drifting silently in one. + +The vector deliberately exercises every field group the protocol promises to +carry — identity, epoch, fingerprint, range, phase, position, idempotency, +cache expectation, deadline, chunking, compression, checksum and a multi- +fragment named tensor — so it doubles as an executable inventory of the +contract. +""" + +from __future__ import annotations + +import pathlib + +from . import codec +from .generated import shard_runtime_pb2 as pb + +# Committed vectors live beside the schema, under `packages/node/native/`. +# parents[2] is `packages/node`: native_protocol -> meshnet_node -> node. +TESTDATA_DIR = pathlib.Path(__file__).resolve().parents[2] / "native/testdata" + +GOLDEN_SESSION_REQUEST = "session_request_golden.binpb" +GOLDEN_CAPABILITY_REPORT = "capability_report_golden.binpb" + +# Written by the C++ conformance test into its build tree; the Python test picks +# it up when present to prove the two languages agree byte-for-byte. +CPP_ROUNDTRIP = "cpp_roundtrip.binpb" + +# Fixed, non-default values. Every one is chosen to be distinguishable from a +# proto3 default so an unset field can never masquerade as a correct one. +WORK_ID = "work-7f3a" +ROUTE_SESSION_ID = "rs-2b91" +ROUTE_EPOCH = 7 +IDEMPOTENCY_STEP = 42 +FIRST_POSITION = 256 +TOKEN_COUNT = 128 +EXPECTED_PAST_LEN = 256 +DEADLINE_UNIX_NANOS = 1_800_000_000_000_000_000 +MODEL_ARTIFACT_DIGEST = "sha256:1f0c9d2e" +RUNTIME_RECIPE_DIGEST = "sha256:ab77e410" +RECIPE_ID = "llama-gguf-q4km-rocm" +RECIPE_VERSION = "3" +CATALOGUE_VERSION = "2026.07.1" +START_LAYER = 12 +END_LAYER = 24 +EFFECTIVE_START_LAYER = 16 +HIDDEN_SIZE = 8 + +# A payload big enough to force more than one fragment at the bound below, so +# the vector actually exercises reassembly rather than the one-fragment path. +FRAGMENT_BYTES = 64 +TENSOR_SHAPE = [1, TOKEN_COUNT, HIDDEN_SIZE] + + +def canonical_payload() -> bytes: + """Deterministic bfloat16-sized payload for the canonical tensor.""" + total = codec.expected_bytes(TENSOR_SHAPE, pb.DTYPE_BFLOAT16) + return bytes((i * 7 + 11) % 256 for i in range(total)) + + +def canonical_session_request() -> pb.SessionRequest: + """The canonical prefill chunk carried on a session stream.""" + tensor = codec.encode_tensor( + codec.HIDDEN_STATES, + canonical_payload(), + TENSOR_SHAPE, + pb.DTYPE_BFLOAT16, + max_fragment_bytes=FRAGMENT_BYTES, + ) + envelope = pb.Envelope( + schema_version=pb.SCHEMA_VERSION_1, + work_id=WORK_ID, + route_session_id=ROUTE_SESSION_ID, + route_epoch=ROUTE_EPOCH, + fingerprint=pb.Fingerprint( + model_artifact_digest=MODEL_ARTIFACT_DIGEST, + runtime_recipe_digest=RUNTIME_RECIPE_DIGEST, + recipe_id=RECIPE_ID, + recipe_version=RECIPE_VERSION, + catalogue_version=CATALOGUE_VERSION, + ), + shard_range=pb.ShardRange( + start_layer=START_LAYER, + end_layer=END_LAYER, + effective_start_layer=EFFECTIVE_START_LAYER, + ), + phase=pb.PHASE_PREFILL, + position=pb.PositionSpan( + first_position=FIRST_POSITION, token_count=TOKEN_COUNT + ), + idempotency_step=IDEMPOTENCY_STEP, + cache_expectation=pb.CacheExpectation( + mode=pb.CACHE_MODE_PREFILL, expected_past_len=EXPECTED_PAST_LEN + ), + deadline_unix_nanos=DEADLINE_UNIX_NANOS, + chunk=pb.ChunkInfo(chunk_index=1, chunk_count=3, final_chunk=False), + ) + return pb.SessionRequest( + chunk=pb.ActivationChunk( + envelope=envelope, bundle=codec.encode_bundle([tensor]) + ) + ) + + +def canonical_capability_report() -> pb.CapabilityReport: + """The canonical capability report a worker answers admission with.""" + return pb.CapabilityReport( + schema_version=pb.SCHEMA_VERSION_1, + fingerprint=pb.Fingerprint( + model_artifact_digest=MODEL_ARTIFACT_DIGEST, + runtime_recipe_digest=RUNTIME_RECIPE_DIGEST, + recipe_id=RECIPE_ID, + recipe_version=RECIPE_VERSION, + catalogue_version=CATALOGUE_VERSION, + ), + shard_range=pb.ShardRange( + start_layer=START_LAYER, + end_layer=END_LAYER, + effective_start_layer=EFFECTIVE_START_LAYER, + ), + backend="rocm", + device="gfx1151", + validated=True, + max_concurrent_sessions=4, + max_context_tokens=8192, + flow_control=codec.default_flow_control(), + accepted_compression=[pb.COMPRESSION_NONE, pb.COMPRESSION_ZSTD], + supported_schema_versions=[pb.SCHEMA_VERSION_1], + validated_at_unix_nanos=DEADLINE_UNIX_NANOS, + ) + + +def serialize(message) -> bytes: + """Serialize deterministically, so committed golden bytes are stable.""" + return message.SerializeToString(deterministic=True) diff --git a/packages/node/meshnet_node/native_protocol/generated/__init__.py b/packages/node/meshnet_node/native_protocol/generated/__init__.py new file mode 100644 index 0000000..e2dff1d --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/generated/__init__.py @@ -0,0 +1,2 @@ +# Generated by scripts/generate_native_protocol.py. Do not edit. +"""Generated protobuf/gRPC stubs for the native Shard protocol.""" diff --git a/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.py b/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.py new file mode 100644 index 0000000..a0ae708 --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.py @@ -0,0 +1,117 @@ +# -*- coding: utf-8 -*- +# Generated by the protocol buffer compiler. DO NOT EDIT! +# NO CHECKED-IN PROTOBUF GENCODE +# source: shard_runtime.proto +# Protobuf Python Version: 7.35.0 +"""Generated protocol buffer code.""" +from google.protobuf import descriptor as _descriptor +from google.protobuf import descriptor_pool as _descriptor_pool +from google.protobuf import runtime_version as _runtime_version +from google.protobuf import symbol_database as _symbol_database +from google.protobuf.internal import builder as _builder +_runtime_version.ValidateProtobufRuntimeVersion( + _runtime_version.Domain.PUBLIC, + 7, + 35, + 0, + '', + 'shard_runtime.proto' +) +# @@protoc_insertion_point(imports) + +_sym_db = _symbol_database.Default() + + + + +DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x13shard_runtime.proto\x12\x10meshnet.shard.v1\"Q\n\x08\x43hecksum\x12\x36\n\talgorithm\x18\x01 \x01(\x0e\x32#.meshnet.shard.v1.ChecksumAlgorithm\x12\r\n\x05value\x18\x02 \x01(\x0c\"\x7f\n\x0eTensorFragment\x12\x16\n\x0e\x66ragment_index\x18\x01 \x01(\r\x12\x16\n\x0e\x66ragment_count\x18\x02 \x01(\r\x12\x13\n\x0b\x62yte_offset\x18\x03 \x01(\x04\x12\x0f\n\x07payload\x18\x04 \x01(\x0cJ\x04\x08\x05\x10\x06R\x11uncompressed_size\"\xaf\x02\n\x0bNamedTensor\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05shape\x18\x02 \x03(\x03\x12&\n\x05\x64type\x18\x03 \x01(\x0e\x32\x17.meshnet.shard.v1.DType\x12/\n\nbyte_order\x18\x04 \x01(\x0e\x32\x1b.meshnet.shard.v1.ByteOrder\x12\x13\n\x0btotal_bytes\x18\x05 \x01(\x04\x12\x32\n\x0b\x63ompression\x18\x06 \x01(\x0e\x32\x1d.meshnet.shard.v1.Compression\x12,\n\x08\x63hecksum\x18\x07 \x01(\x0b\x32\x1a.meshnet.shard.v1.Checksum\x12\x33\n\tfragments\x18\x08 \x03(\x0b\x32 .meshnet.shard.v1.TensorFragment\"V\n\x0cTensorBundle\x12\x16\n\x0e\x62undle_version\x18\x01 \x01(\r\x12.\n\x07tensors\x18\x02 \x03(\x0b\x32\x1d.meshnet.shard.v1.NamedTensor\"\x91\x01\n\x0b\x46ingerprint\x12\x1d\n\x15model_artifact_digest\x18\x01 \x01(\t\x12\x1d\n\x15runtime_recipe_digest\x18\x02 \x01(\t\x12\x11\n\trecipe_id\x18\x03 \x01(\t\x12\x16\n\x0erecipe_version\x18\x04 \x01(\t\x12\x19\n\x11\x63\x61talogue_version\x18\x05 \x01(\t\"S\n\nShardRange\x12\x13\n\x0bstart_layer\x18\x01 \x01(\r\x12\x11\n\tend_layer\x18\x02 \x01(\r\x12\x1d\n\x15\x65\x66\x66\x65\x63tive_start_layer\x18\x03 \x01(\r\";\n\x0cPositionSpan\x12\x16\n\x0e\x66irst_position\x18\x01 \x01(\x04\x12\x13\n\x0btoken_count\x18\x02 \x01(\r\"J\n\tChunkInfo\x12\x13\n\x0b\x63hunk_index\x18\x01 \x01(\r\x12\x13\n\x0b\x63hunk_count\x18\x02 \x01(\r\x12\x13\n\x0b\x66inal_chunk\x18\x03 \x01(\x08\"X\n\x10\x43\x61\x63heExpectation\x12)\n\x04mode\x18\x01 \x01(\x0e\x32\x1b.meshnet.shard.v1.CacheMode\x12\x19\n\x11\x65xpected_past_len\x18\x02 \x01(\x04\"]\n\x0b\x43\x61\x63heResult\x12)\n\x04mode\x18\x01 \x01(\x0e\x32\x1b.meshnet.shard.v1.CacheMode\x12\x10\n\x08past_len\x18\x02 \x01(\x04\x12\x11\n\tcache_hit\x18\x03 \x01(\x08\"\xe6\x03\n\x08\x45nvelope\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x0f\n\x07work_id\x18\x02 \x01(\t\x12\x18\n\x10route_session_id\x18\x03 \x01(\t\x12\x13\n\x0broute_epoch\x18\x04 \x01(\x04\x12\x32\n\x0b\x66ingerprint\x18\x05 \x01(\x0b\x32\x1d.meshnet.shard.v1.Fingerprint\x12\x31\n\x0bshard_range\x18\x06 \x01(\x0b\x32\x1c.meshnet.shard.v1.ShardRange\x12&\n\x05phase\x18\x07 \x01(\x0e\x32\x17.meshnet.shard.v1.Phase\x12\x30\n\x08position\x18\x08 \x01(\x0b\x32\x1e.meshnet.shard.v1.PositionSpan\x12\x18\n\x10idempotency_step\x18\t \x01(\x04\x12=\n\x11\x63\x61\x63he_expectation\x18\n \x01(\x0b\x32\".meshnet.shard.v1.CacheExpectation\x12\x1b\n\x13\x64\x65\x61\x64line_unix_nanos\x18\x0b \x01(\x03\x12*\n\x05\x63hunk\x18\x0c \x01(\x0b\x32\x1b.meshnet.shard.v1.ChunkInfo\"s\n\nShardError\x12)\n\x04\x63ode\x18\x01 \x01(\x0e\x32\x1b.meshnet.shard.v1.ErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x02 \x01(\t\x12\x11\n\tretryable\x18\x03 \x01(\x08\x12\x17\n\x0f\x61\x63tual_past_len\x18\x04 \x01(\x04\"~\n\x0b\x46lowControl\x12\x17\n\x0f\x63redits_granted\x18\x01 \x01(\r\x12\x1b\n\x13max_inflight_chunks\x18\x02 \x01(\r\x12\x17\n\x0fmax_chunk_bytes\x18\x03 \x01(\x04\x12 \n\x18max_prefill_chunk_tokens\x18\x04 \x01(\r\"\xd7\x02\n\x0bSessionOpen\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x18\n\x10route_session_id\x18\x02 \x01(\t\x12\x13\n\x0broute_epoch\x18\x03 \x01(\x04\x12\x32\n\x0b\x66ingerprint\x18\x04 \x01(\x0b\x32\x1d.meshnet.shard.v1.Fingerprint\x12\x31\n\x0bshard_range\x18\x05 \x01(\x0b\x32\x1c.meshnet.shard.v1.ShardRange\x12<\n\x15proposed_flow_control\x18\x06 \x01(\x0b\x32\x1d.meshnet.shard.v1.FlowControl\x12;\n\x14\x61\x63\x63\x65pted_compression\x18\x07 \x03(\x0e\x32\x1d.meshnet.shard.v1.Compression\"\x9f\x02\n\x0fSessionAccepted\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x18\n\x10route_session_id\x18\x02 \x01(\t\x12\x13\n\x0broute_epoch\x18\x03 \x01(\x04\x12\x33\n\x0c\x66low_control\x18\x04 \x01(\x0b\x32\x1d.meshnet.shard.v1.FlowControl\x12;\n\x14\x61\x63\x63\x65pted_compression\x18\x05 \x03(\x0e\x32\x1d.meshnet.shard.v1.Compression\x12\x32\n\x0b\x66ingerprint\x18\x06 \x01(\x0b\x32\x1d.meshnet.shard.v1.Fingerprint\"o\n\x0f\x41\x63tivationChunk\x12,\n\x08\x65nvelope\x18\x01 \x01(\x0b\x32\x1a.meshnet.shard.v1.Envelope\x12.\n\x06\x62undle\x18\x02 \x01(\x0b\x32\x1e.meshnet.shard.v1.TensorBundle\"\xb0\x01\n\nDecodeStep\x12\x18\n\x10idempotency_step\x18\x01 \x01(\x04\x12\x10\n\x08position\x18\x02 \x01(\x04\x12\x19\n\x11\x65xpected_past_len\x18\x03 \x01(\x04\x12-\n\x06tensor\x18\x04 \x01(\x0b\x32\x1d.meshnet.shard.v1.NamedTensor\x12\x0f\n\x07work_id\x18\x05 \x01(\t\x12\x1b\n\x13\x64\x65\x61\x64line_unix_nanos\x18\x06 \x01(\x03\"O\n\rReleaseSignal\x12\x18\n\x10route_session_id\x18\x01 \x01(\t\x12\x13\n\x0broute_epoch\x18\x02 \x01(\x04\x12\x0f\n\x07work_id\x18\x03 \x01(\t\"^\n\x0c\x43\x61ncelSignal\x12\x18\n\x10route_session_id\x18\x01 \x01(\t\x12\x13\n\x0broute_epoch\x18\x02 \x01(\x04\x12\x0f\n\x07work_id\x18\x03 \x01(\t\x12\x0e\n\x06reason\x18\x04 \x01(\t\"\x91\x01\n\x03\x41\x63k\x12\x0f\n\x07work_id\x18\x01 \x01(\t\x12\x18\n\x10idempotency_step\x18\x02 \x01(\x04\x12\x33\n\x0c\x63\x61\x63he_result\x18\x03 \x01(\x0b\x32\x1d.meshnet.shard.v1.CacheResult\x12\x11\n\tduplicate\x18\x04 \x01(\x08\x12\x17\n\x0f\x65xecution_nanos\x18\x05 \x01(\x04\"\x91\x01\n\x0bShardStatus\x12\x0f\n\x07work_id\x18\x01 \x01(\t\x12\x18\n\x10route_session_id\x18\x02 \x01(\t\x12\x18\n\x10idempotency_step\x18\x03 \x01(\x04\x12+\n\x05\x65rror\x18\x04 \x01(\x0b\x32\x1c.meshnet.shard.v1.ShardError\x12\x10\n\x08terminal\x18\x05 \x01(\x08\"\xc8\x02\n\x0eSessionRequest\x12-\n\x04open\x18\x01 \x01(\x0b\x32\x1d.meshnet.shard.v1.SessionOpenH\x00\x12\x32\n\x05\x63hunk\x18\x02 \x01(\x0b\x32!.meshnet.shard.v1.ActivationChunkH\x00\x12.\n\x06\x64\x65\x63ode\x18\x03 \x01(\x0b\x32\x1c.meshnet.shard.v1.DecodeStepH\x00\x12\x35\n\x0c\x66low_control\x18\x04 \x01(\x0b\x32\x1d.meshnet.shard.v1.FlowControlH\x00\x12\x32\n\x07release\x18\x05 \x01(\x0b\x32\x1f.meshnet.shard.v1.ReleaseSignalH\x00\x12\x30\n\x06\x63\x61ncel\x18\x06 \x01(\x0b\x32\x1e.meshnet.shard.v1.CancelSignalH\x00\x42\x06\n\x04kind\"\x92\x02\n\x0fSessionResponse\x12\x35\n\x08\x61\x63\x63\x65pted\x18\x01 \x01(\x0b\x32!.meshnet.shard.v1.SessionAcceptedH\x00\x12\x32\n\x05\x63hunk\x18\x02 \x01(\x0b\x32!.meshnet.shard.v1.ActivationChunkH\x00\x12$\n\x03\x61\x63k\x18\x03 \x01(\x0b\x32\x15.meshnet.shard.v1.AckH\x00\x12\x35\n\x0c\x66low_control\x18\x04 \x01(\x0b\x32\x1d.meshnet.shard.v1.FlowControlH\x00\x12/\n\x06status\x18\x05 \x01(\x0b\x32\x1d.meshnet.shard.v1.ShardStatusH\x00\x42\x06\n\x04kind\"L\n\x11\x43\x61pabilityRequest\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\"\x8a\x04\n\x10\x43\x61pabilityReport\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x32\n\x0b\x66ingerprint\x18\x02 \x01(\x0b\x32\x1d.meshnet.shard.v1.Fingerprint\x12\x31\n\x0bshard_range\x18\x03 \x01(\x0b\x32\x1c.meshnet.shard.v1.ShardRange\x12\x0f\n\x07\x62\x61\x63kend\x18\x04 \x01(\t\x12\x0e\n\x06\x64\x65vice\x18\x05 \x01(\t\x12\x11\n\tvalidated\x18\x06 \x01(\x08\x12\x0e\n\x06\x64\x65tail\x18\x07 \x01(\t\x12\x1f\n\x17max_concurrent_sessions\x18\x08 \x01(\x04\x12\x1a\n\x12max_context_tokens\x18\t \x01(\x04\x12\x33\n\x0c\x66low_control\x18\n \x01(\x0b\x32\x1d.meshnet.shard.v1.FlowControl\x12;\n\x14\x61\x63\x63\x65pted_compression\x18\x0b \x03(\x0e\x32\x1d.meshnet.shard.v1.Compression\x12\x42\n\x19supported_schema_versions\x18\x0c \x03(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x1f\n\x17validated_at_unix_nanos\x18\r \x01(\x03\"H\n\rHealthRequest\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\"\xfc\x01\n\x0cHealthReport\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12-\n\x05state\x18\x02 \x01(\x0e\x32\x1e.meshnet.shard.v1.ServingState\x12\x17\n\x0f\x61\x63tive_sessions\x18\x03 \x01(\r\x12\x15\n\rqueued_chunks\x18\x04 \x01(\r\x12\x17\n\x0f\x62\x61tch_occupancy\x18\x05 \x01(\r\x12\x13\n\x0bkv_pressure\x18\x06 \x01(\x02\x12\x16\n\x0eresident_bytes\x18\x07 \x01(\x04\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\"x\n\x0eReleaseRequest\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x18\n\x10route_session_id\x18\x02 \x01(\t\x12\x13\n\x0broute_epoch\x18\x03 \x01(\x04\"P\n\x0fReleaseResponse\x12\x10\n\x08released\x18\x01 \x01(\x08\x12+\n\x05\x65rror\x18\x02 \x01(\x0b\x32\x1c.meshnet.shard.v1.ShardError\"\x98\x01\n\rCancelRequest\x12\x37\n\x0eschema_version\x18\x01 \x01(\x0e\x32\x1f.meshnet.shard.v1.SchemaVersion\x12\x18\n\x10route_session_id\x18\x02 \x01(\t\x12\x13\n\x0broute_epoch\x18\x03 \x01(\x04\x12\x0f\n\x07work_id\x18\x04 \x01(\t\x12\x0e\n\x06reason\x18\x05 \x01(\t\"[\n\x0e\x43\x61ncelResponse\x12\x1c\n\x14\x63\x61ncelled_work_items\x18\x01 \x01(\r\x12+\n\x05\x65rror\x18\x02 \x01(\x0b\x32\x1c.meshnet.shard.v1.ShardError*E\n\rSchemaVersion\x12\x1e\n\x1aSCHEMA_VERSION_UNSPECIFIED\x10\x00\x12\x14\n\x10SCHEMA_VERSION_1\x10\x01*\xab\x01\n\x05\x44Type\x12\x15\n\x11\x44TYPE_UNSPECIFIED\x10\x00\x12\x12\n\x0e\x44TYPE_BFLOAT16\x10\x01\x12\x11\n\rDTYPE_FLOAT16\x10\x02\x12\x11\n\rDTYPE_FLOAT32\x10\x03\x12\x0f\n\x0b\x44TYPE_INT32\x10\x04\x12\x0f\n\x0b\x44TYPE_INT64\x10\x05\x12\x0f\n\x0b\x44TYPE_UINT8\x10\x06\x12\x0e\n\nDTYPE_INT8\x10\x07\x12\x0e\n\nDTYPE_BOOL\x10\x08*`\n\tByteOrder\x12\x1a\n\x16\x42YTE_ORDER_UNSPECIFIED\x10\x00\x12\x1c\n\x18\x42YTE_ORDER_LITTLE_ENDIAN\x10\x01\x12\x19\n\x15\x42YTE_ORDER_BIG_ENDIAN\x10\x02*V\n\x0b\x43ompression\x12\x1b\n\x17\x43OMPRESSION_UNSPECIFIED\x10\x00\x12\x14\n\x10\x43OMPRESSION_NONE\x10\x01\x12\x14\n\x10\x43OMPRESSION_ZSTD\x10\x02*s\n\x11\x43hecksumAlgorithm\x12\"\n\x1e\x43HECKSUM_ALGORITHM_UNSPECIFIED\x10\x00\x12\x1b\n\x17\x43HECKSUM_ALGORITHM_NONE\x10\x01\x12\x1d\n\x19\x43HECKSUM_ALGORITHM_CRC32C\x10\x02*h\n\x05Phase\x12\x15\n\x11PHASE_UNSPECIFIED\x10\x00\x12\x11\n\rPHASE_PREFILL\x10\x01\x12\x10\n\x0cPHASE_DECODE\x10\x02\x12\x11\n\rPHASE_RELEASE\x10\x03\x12\x10\n\x0cPHASE_CANCEL\x10\x04*p\n\tCacheMode\x12\x1a\n\x16\x43\x41\x43HE_MODE_UNSPECIFIED\x10\x00\x12\x18\n\x14\x43\x41\x43HE_MODE_STATELESS\x10\x01\x12\x16\n\x12\x43\x41\x43HE_MODE_PREFILL\x10\x02\x12\x15\n\x11\x43\x41\x43HE_MODE_DECODE\x10\x03*\x8a\x03\n\tErrorCode\x12\x1a\n\x16\x45RROR_CODE_UNSPECIFIED\x10\x00\x12!\n\x1d\x45RROR_CODE_SCHEMA_UNSUPPORTED\x10\x01\x12#\n\x1f\x45RROR_CODE_FINGERPRINT_MISMATCH\x10\x02\x12\x1a\n\x16\x45RROR_CODE_EPOCH_STALE\x10\x03\x12#\n\x1f\x45RROR_CODE_SHARD_RANGE_MISMATCH\x10\x04\x12\x19\n\x15\x45RROR_CODE_CACHE_MISS\x10\x05\x12!\n\x1d\x45RROR_CODE_RESOURCE_EXHAUSTED\x10\x06\x12\x1e\n\x1a\x45RROR_CODE_PAYLOAD_CORRUPT\x10\x07\x12\x18\n\x14\x45RROR_CODE_CANCELLED\x10\x08\x12 \n\x1c\x45RROR_CODE_DEADLINE_EXCEEDED\x10\t\x12%\n!ERROR_CODE_FLOW_CONTROL_VIOLATION\x10\n\x12\x17\n\x13\x45RROR_CODE_INTERNAL\x10\x0b*\x83\x01\n\x0cServingState\x12\x1d\n\x19SERVING_STATE_UNSPECIFIED\x10\x00\x12\x19\n\x15SERVING_STATE_SERVING\x10\x01\x12\x1a\n\x16SERVING_STATE_DRAINING\x10\x02\x12\x1d\n\x19SERVING_STATE_NOT_SERVING\x10\x03\x32\xa4\x03\n\x0cShardRuntime\x12X\n\rGetCapability\x12#.meshnet.shard.v1.CapabilityRequest\x1a\".meshnet.shard.v1.CapabilityReport\x12I\n\x06Health\x12\x1f.meshnet.shard.v1.HealthRequest\x1a\x1e.meshnet.shard.v1.HealthReport\x12R\n\x07Session\x12 .meshnet.shard.v1.SessionRequest\x1a!.meshnet.shard.v1.SessionResponse(\x01\x30\x01\x12N\n\x07Release\x12 .meshnet.shard.v1.ReleaseRequest\x1a!.meshnet.shard.v1.ReleaseResponse\x12K\n\x06\x43\x61ncel\x12\x1f.meshnet.shard.v1.CancelRequest\x1a .meshnet.shard.v1.CancelResponseB(Z#github.com/meshnet/shard/v1;shardv1\xf8\x01\x01\x62\x06proto3') + +_globals = globals() +_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals) +_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'shard_runtime_pb2', _globals) +if not _descriptor._USE_C_DESCRIPTORS: + _globals['DESCRIPTOR']._loaded_options = None + _globals['DESCRIPTOR']._serialized_options = b'Z#github.com/meshnet/shard/v1;shardv1\370\001\001' + _globals['_SCHEMAVERSION']._serialized_start=5329 + _globals['_SCHEMAVERSION']._serialized_end=5398 + _globals['_DTYPE']._serialized_start=5401 + _globals['_DTYPE']._serialized_end=5572 + _globals['_BYTEORDER']._serialized_start=5574 + _globals['_BYTEORDER']._serialized_end=5670 + _globals['_COMPRESSION']._serialized_start=5672 + _globals['_COMPRESSION']._serialized_end=5758 + _globals['_CHECKSUMALGORITHM']._serialized_start=5760 + _globals['_CHECKSUMALGORITHM']._serialized_end=5875 + _globals['_PHASE']._serialized_start=5877 + _globals['_PHASE']._serialized_end=5981 + _globals['_CACHEMODE']._serialized_start=5983 + _globals['_CACHEMODE']._serialized_end=6095 + _globals['_ERRORCODE']._serialized_start=6098 + _globals['_ERRORCODE']._serialized_end=6492 + _globals['_SERVINGSTATE']._serialized_start=6495 + _globals['_SERVINGSTATE']._serialized_end=6626 + _globals['_CHECKSUM']._serialized_start=41 + _globals['_CHECKSUM']._serialized_end=122 + _globals['_TENSORFRAGMENT']._serialized_start=124 + _globals['_TENSORFRAGMENT']._serialized_end=251 + _globals['_NAMEDTENSOR']._serialized_start=254 + _globals['_NAMEDTENSOR']._serialized_end=557 + _globals['_TENSORBUNDLE']._serialized_start=559 + _globals['_TENSORBUNDLE']._serialized_end=645 + _globals['_FINGERPRINT']._serialized_start=648 + _globals['_FINGERPRINT']._serialized_end=793 + _globals['_SHARDRANGE']._serialized_start=795 + _globals['_SHARDRANGE']._serialized_end=878 + _globals['_POSITIONSPAN']._serialized_start=880 + _globals['_POSITIONSPAN']._serialized_end=939 + _globals['_CHUNKINFO']._serialized_start=941 + _globals['_CHUNKINFO']._serialized_end=1015 + _globals['_CACHEEXPECTATION']._serialized_start=1017 + _globals['_CACHEEXPECTATION']._serialized_end=1105 + _globals['_CACHERESULT']._serialized_start=1107 + _globals['_CACHERESULT']._serialized_end=1200 + _globals['_ENVELOPE']._serialized_start=1203 + _globals['_ENVELOPE']._serialized_end=1689 + _globals['_SHARDERROR']._serialized_start=1691 + _globals['_SHARDERROR']._serialized_end=1806 + _globals['_FLOWCONTROL']._serialized_start=1808 + _globals['_FLOWCONTROL']._serialized_end=1934 + _globals['_SESSIONOPEN']._serialized_start=1937 + _globals['_SESSIONOPEN']._serialized_end=2280 + _globals['_SESSIONACCEPTED']._serialized_start=2283 + _globals['_SESSIONACCEPTED']._serialized_end=2570 + _globals['_ACTIVATIONCHUNK']._serialized_start=2572 + _globals['_ACTIVATIONCHUNK']._serialized_end=2683 + _globals['_DECODESTEP']._serialized_start=2686 + _globals['_DECODESTEP']._serialized_end=2862 + _globals['_RELEASESIGNAL']._serialized_start=2864 + _globals['_RELEASESIGNAL']._serialized_end=2943 + _globals['_CANCELSIGNAL']._serialized_start=2945 + _globals['_CANCELSIGNAL']._serialized_end=3039 + _globals['_ACK']._serialized_start=3042 + _globals['_ACK']._serialized_end=3187 + _globals['_SHARDSTATUS']._serialized_start=3190 + _globals['_SHARDSTATUS']._serialized_end=3335 + _globals['_SESSIONREQUEST']._serialized_start=3338 + _globals['_SESSIONREQUEST']._serialized_end=3666 + _globals['_SESSIONRESPONSE']._serialized_start=3669 + _globals['_SESSIONRESPONSE']._serialized_end=3943 + _globals['_CAPABILITYREQUEST']._serialized_start=3945 + _globals['_CAPABILITYREQUEST']._serialized_end=4021 + _globals['_CAPABILITYREPORT']._serialized_start=4024 + _globals['_CAPABILITYREPORT']._serialized_end=4546 + _globals['_HEALTHREQUEST']._serialized_start=4548 + _globals['_HEALTHREQUEST']._serialized_end=4620 + _globals['_HEALTHREPORT']._serialized_start=4623 + _globals['_HEALTHREPORT']._serialized_end=4875 + _globals['_RELEASEREQUEST']._serialized_start=4877 + _globals['_RELEASEREQUEST']._serialized_end=4997 + _globals['_RELEASERESPONSE']._serialized_start=4999 + _globals['_RELEASERESPONSE']._serialized_end=5079 + _globals['_CANCELREQUEST']._serialized_start=5082 + _globals['_CANCELREQUEST']._serialized_end=5234 + _globals['_CANCELRESPONSE']._serialized_start=5236 + _globals['_CANCELRESPONSE']._serialized_end=5327 + _globals['_SHARDRUNTIME']._serialized_start=6629 + _globals['_SHARDRUNTIME']._serialized_end=7049 +# @@protoc_insertion_point(module_scope) diff --git a/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.pyi b/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.pyi new file mode 100644 index 0000000..4edb765 --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2.pyi @@ -0,0 +1,525 @@ +from google.protobuf.internal import containers as _containers +from google.protobuf.internal import enum_type_wrapper as _enum_type_wrapper +from google.protobuf import descriptor as _descriptor +from google.protobuf import message as _message +from collections.abc import Iterable as _Iterable, Mapping as _Mapping +from typing import ClassVar as _ClassVar, Optional as _Optional, Union as _Union + +DESCRIPTOR: _descriptor.FileDescriptor + +class SchemaVersion(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + SCHEMA_VERSION_UNSPECIFIED: _ClassVar[SchemaVersion] + SCHEMA_VERSION_1: _ClassVar[SchemaVersion] + +class DType(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + DTYPE_UNSPECIFIED: _ClassVar[DType] + DTYPE_BFLOAT16: _ClassVar[DType] + DTYPE_FLOAT16: _ClassVar[DType] + DTYPE_FLOAT32: _ClassVar[DType] + DTYPE_INT32: _ClassVar[DType] + DTYPE_INT64: _ClassVar[DType] + DTYPE_UINT8: _ClassVar[DType] + DTYPE_INT8: _ClassVar[DType] + DTYPE_BOOL: _ClassVar[DType] + +class ByteOrder(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + BYTE_ORDER_UNSPECIFIED: _ClassVar[ByteOrder] + BYTE_ORDER_LITTLE_ENDIAN: _ClassVar[ByteOrder] + BYTE_ORDER_BIG_ENDIAN: _ClassVar[ByteOrder] + +class Compression(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + COMPRESSION_UNSPECIFIED: _ClassVar[Compression] + COMPRESSION_NONE: _ClassVar[Compression] + COMPRESSION_ZSTD: _ClassVar[Compression] + +class ChecksumAlgorithm(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + CHECKSUM_ALGORITHM_UNSPECIFIED: _ClassVar[ChecksumAlgorithm] + CHECKSUM_ALGORITHM_NONE: _ClassVar[ChecksumAlgorithm] + CHECKSUM_ALGORITHM_CRC32C: _ClassVar[ChecksumAlgorithm] + +class Phase(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + PHASE_UNSPECIFIED: _ClassVar[Phase] + PHASE_PREFILL: _ClassVar[Phase] + PHASE_DECODE: _ClassVar[Phase] + PHASE_RELEASE: _ClassVar[Phase] + PHASE_CANCEL: _ClassVar[Phase] + +class CacheMode(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + CACHE_MODE_UNSPECIFIED: _ClassVar[CacheMode] + CACHE_MODE_STATELESS: _ClassVar[CacheMode] + CACHE_MODE_PREFILL: _ClassVar[CacheMode] + CACHE_MODE_DECODE: _ClassVar[CacheMode] + +class ErrorCode(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + ERROR_CODE_UNSPECIFIED: _ClassVar[ErrorCode] + ERROR_CODE_SCHEMA_UNSUPPORTED: _ClassVar[ErrorCode] + ERROR_CODE_FINGERPRINT_MISMATCH: _ClassVar[ErrorCode] + ERROR_CODE_EPOCH_STALE: _ClassVar[ErrorCode] + ERROR_CODE_SHARD_RANGE_MISMATCH: _ClassVar[ErrorCode] + ERROR_CODE_CACHE_MISS: _ClassVar[ErrorCode] + ERROR_CODE_RESOURCE_EXHAUSTED: _ClassVar[ErrorCode] + ERROR_CODE_PAYLOAD_CORRUPT: _ClassVar[ErrorCode] + ERROR_CODE_CANCELLED: _ClassVar[ErrorCode] + ERROR_CODE_DEADLINE_EXCEEDED: _ClassVar[ErrorCode] + ERROR_CODE_FLOW_CONTROL_VIOLATION: _ClassVar[ErrorCode] + ERROR_CODE_INTERNAL: _ClassVar[ErrorCode] + +class ServingState(int, metaclass=_enum_type_wrapper.EnumTypeWrapper): + __slots__ = () + SERVING_STATE_UNSPECIFIED: _ClassVar[ServingState] + SERVING_STATE_SERVING: _ClassVar[ServingState] + SERVING_STATE_DRAINING: _ClassVar[ServingState] + SERVING_STATE_NOT_SERVING: _ClassVar[ServingState] +SCHEMA_VERSION_UNSPECIFIED: SchemaVersion +SCHEMA_VERSION_1: SchemaVersion +DTYPE_UNSPECIFIED: DType +DTYPE_BFLOAT16: DType +DTYPE_FLOAT16: DType +DTYPE_FLOAT32: DType +DTYPE_INT32: DType +DTYPE_INT64: DType +DTYPE_UINT8: DType +DTYPE_INT8: DType +DTYPE_BOOL: DType +BYTE_ORDER_UNSPECIFIED: ByteOrder +BYTE_ORDER_LITTLE_ENDIAN: ByteOrder +BYTE_ORDER_BIG_ENDIAN: ByteOrder +COMPRESSION_UNSPECIFIED: Compression +COMPRESSION_NONE: Compression +COMPRESSION_ZSTD: Compression +CHECKSUM_ALGORITHM_UNSPECIFIED: ChecksumAlgorithm +CHECKSUM_ALGORITHM_NONE: ChecksumAlgorithm +CHECKSUM_ALGORITHM_CRC32C: ChecksumAlgorithm +PHASE_UNSPECIFIED: Phase +PHASE_PREFILL: Phase +PHASE_DECODE: Phase +PHASE_RELEASE: Phase +PHASE_CANCEL: Phase +CACHE_MODE_UNSPECIFIED: CacheMode +CACHE_MODE_STATELESS: CacheMode +CACHE_MODE_PREFILL: CacheMode +CACHE_MODE_DECODE: CacheMode +ERROR_CODE_UNSPECIFIED: ErrorCode +ERROR_CODE_SCHEMA_UNSUPPORTED: ErrorCode +ERROR_CODE_FINGERPRINT_MISMATCH: ErrorCode +ERROR_CODE_EPOCH_STALE: ErrorCode +ERROR_CODE_SHARD_RANGE_MISMATCH: ErrorCode +ERROR_CODE_CACHE_MISS: ErrorCode +ERROR_CODE_RESOURCE_EXHAUSTED: ErrorCode +ERROR_CODE_PAYLOAD_CORRUPT: ErrorCode +ERROR_CODE_CANCELLED: ErrorCode +ERROR_CODE_DEADLINE_EXCEEDED: ErrorCode +ERROR_CODE_FLOW_CONTROL_VIOLATION: ErrorCode +ERROR_CODE_INTERNAL: ErrorCode +SERVING_STATE_UNSPECIFIED: ServingState +SERVING_STATE_SERVING: ServingState +SERVING_STATE_DRAINING: ServingState +SERVING_STATE_NOT_SERVING: ServingState + +class Checksum(_message.Message): + __slots__ = ("algorithm", "value") + ALGORITHM_FIELD_NUMBER: _ClassVar[int] + VALUE_FIELD_NUMBER: _ClassVar[int] + algorithm: ChecksumAlgorithm + value: bytes + def __init__(self, algorithm: _Optional[_Union[ChecksumAlgorithm, str]] = ..., value: _Optional[bytes] = ...) -> None: ... + +class TensorFragment(_message.Message): + __slots__ = ("fragment_index", "fragment_count", "byte_offset", "payload") + FRAGMENT_INDEX_FIELD_NUMBER: _ClassVar[int] + FRAGMENT_COUNT_FIELD_NUMBER: _ClassVar[int] + BYTE_OFFSET_FIELD_NUMBER: _ClassVar[int] + PAYLOAD_FIELD_NUMBER: _ClassVar[int] + fragment_index: int + fragment_count: int + byte_offset: int + payload: bytes + def __init__(self, fragment_index: _Optional[int] = ..., fragment_count: _Optional[int] = ..., byte_offset: _Optional[int] = ..., payload: _Optional[bytes] = ...) -> None: ... + +class NamedTensor(_message.Message): + __slots__ = ("name", "shape", "dtype", "byte_order", "total_bytes", "compression", "checksum", "fragments") + NAME_FIELD_NUMBER: _ClassVar[int] + SHAPE_FIELD_NUMBER: _ClassVar[int] + DTYPE_FIELD_NUMBER: _ClassVar[int] + BYTE_ORDER_FIELD_NUMBER: _ClassVar[int] + TOTAL_BYTES_FIELD_NUMBER: _ClassVar[int] + COMPRESSION_FIELD_NUMBER: _ClassVar[int] + CHECKSUM_FIELD_NUMBER: _ClassVar[int] + FRAGMENTS_FIELD_NUMBER: _ClassVar[int] + name: str + shape: _containers.RepeatedScalarFieldContainer[int] + dtype: DType + byte_order: ByteOrder + total_bytes: int + compression: Compression + checksum: Checksum + fragments: _containers.RepeatedCompositeFieldContainer[TensorFragment] + def __init__(self, name: _Optional[str] = ..., shape: _Optional[_Iterable[int]] = ..., dtype: _Optional[_Union[DType, str]] = ..., byte_order: _Optional[_Union[ByteOrder, str]] = ..., total_bytes: _Optional[int] = ..., compression: _Optional[_Union[Compression, str]] = ..., checksum: _Optional[_Union[Checksum, _Mapping]] = ..., fragments: _Optional[_Iterable[_Union[TensorFragment, _Mapping]]] = ...) -> None: ... + +class TensorBundle(_message.Message): + __slots__ = ("bundle_version", "tensors") + BUNDLE_VERSION_FIELD_NUMBER: _ClassVar[int] + TENSORS_FIELD_NUMBER: _ClassVar[int] + bundle_version: int + tensors: _containers.RepeatedCompositeFieldContainer[NamedTensor] + def __init__(self, bundle_version: _Optional[int] = ..., tensors: _Optional[_Iterable[_Union[NamedTensor, _Mapping]]] = ...) -> None: ... + +class Fingerprint(_message.Message): + __slots__ = ("model_artifact_digest", "runtime_recipe_digest", "recipe_id", "recipe_version", "catalogue_version") + MODEL_ARTIFACT_DIGEST_FIELD_NUMBER: _ClassVar[int] + RUNTIME_RECIPE_DIGEST_FIELD_NUMBER: _ClassVar[int] + RECIPE_ID_FIELD_NUMBER: _ClassVar[int] + RECIPE_VERSION_FIELD_NUMBER: _ClassVar[int] + CATALOGUE_VERSION_FIELD_NUMBER: _ClassVar[int] + model_artifact_digest: str + runtime_recipe_digest: str + recipe_id: str + recipe_version: str + catalogue_version: str + def __init__(self, model_artifact_digest: _Optional[str] = ..., runtime_recipe_digest: _Optional[str] = ..., recipe_id: _Optional[str] = ..., recipe_version: _Optional[str] = ..., catalogue_version: _Optional[str] = ...) -> None: ... + +class ShardRange(_message.Message): + __slots__ = ("start_layer", "end_layer", "effective_start_layer") + START_LAYER_FIELD_NUMBER: _ClassVar[int] + END_LAYER_FIELD_NUMBER: _ClassVar[int] + EFFECTIVE_START_LAYER_FIELD_NUMBER: _ClassVar[int] + start_layer: int + end_layer: int + effective_start_layer: int + def __init__(self, start_layer: _Optional[int] = ..., end_layer: _Optional[int] = ..., effective_start_layer: _Optional[int] = ...) -> None: ... + +class PositionSpan(_message.Message): + __slots__ = ("first_position", "token_count") + FIRST_POSITION_FIELD_NUMBER: _ClassVar[int] + TOKEN_COUNT_FIELD_NUMBER: _ClassVar[int] + first_position: int + token_count: int + def __init__(self, first_position: _Optional[int] = ..., token_count: _Optional[int] = ...) -> None: ... + +class ChunkInfo(_message.Message): + __slots__ = ("chunk_index", "chunk_count", "final_chunk") + CHUNK_INDEX_FIELD_NUMBER: _ClassVar[int] + CHUNK_COUNT_FIELD_NUMBER: _ClassVar[int] + FINAL_CHUNK_FIELD_NUMBER: _ClassVar[int] + chunk_index: int + chunk_count: int + final_chunk: bool + def __init__(self, chunk_index: _Optional[int] = ..., chunk_count: _Optional[int] = ..., final_chunk: _Optional[bool] = ...) -> None: ... + +class CacheExpectation(_message.Message): + __slots__ = ("mode", "expected_past_len") + MODE_FIELD_NUMBER: _ClassVar[int] + EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int] + mode: CacheMode + expected_past_len: int + def __init__(self, mode: _Optional[_Union[CacheMode, str]] = ..., expected_past_len: _Optional[int] = ...) -> None: ... + +class CacheResult(_message.Message): + __slots__ = ("mode", "past_len", "cache_hit") + MODE_FIELD_NUMBER: _ClassVar[int] + PAST_LEN_FIELD_NUMBER: _ClassVar[int] + CACHE_HIT_FIELD_NUMBER: _ClassVar[int] + mode: CacheMode + past_len: int + cache_hit: bool + def __init__(self, mode: _Optional[_Union[CacheMode, str]] = ..., past_len: _Optional[int] = ..., cache_hit: _Optional[bool] = ...) -> None: ... + +class Envelope(_message.Message): + __slots__ = ("schema_version", "work_id", "route_session_id", "route_epoch", "fingerprint", "shard_range", "phase", "position", "idempotency_step", "cache_expectation", "deadline_unix_nanos", "chunk") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + WORK_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + FINGERPRINT_FIELD_NUMBER: _ClassVar[int] + SHARD_RANGE_FIELD_NUMBER: _ClassVar[int] + PHASE_FIELD_NUMBER: _ClassVar[int] + POSITION_FIELD_NUMBER: _ClassVar[int] + IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int] + CACHE_EXPECTATION_FIELD_NUMBER: _ClassVar[int] + DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int] + CHUNK_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + work_id: str + route_session_id: str + route_epoch: int + fingerprint: Fingerprint + shard_range: ShardRange + phase: Phase + position: PositionSpan + idempotency_step: int + cache_expectation: CacheExpectation + deadline_unix_nanos: int + chunk: ChunkInfo + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., work_id: _Optional[str] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ..., shard_range: _Optional[_Union[ShardRange, _Mapping]] = ..., phase: _Optional[_Union[Phase, str]] = ..., position: _Optional[_Union[PositionSpan, _Mapping]] = ..., idempotency_step: _Optional[int] = ..., cache_expectation: _Optional[_Union[CacheExpectation, _Mapping]] = ..., deadline_unix_nanos: _Optional[int] = ..., chunk: _Optional[_Union[ChunkInfo, _Mapping]] = ...) -> None: ... + +class ShardError(_message.Message): + __slots__ = ("code", "detail", "retryable", "actual_past_len") + CODE_FIELD_NUMBER: _ClassVar[int] + DETAIL_FIELD_NUMBER: _ClassVar[int] + RETRYABLE_FIELD_NUMBER: _ClassVar[int] + ACTUAL_PAST_LEN_FIELD_NUMBER: _ClassVar[int] + code: ErrorCode + detail: str + retryable: bool + actual_past_len: int + def __init__(self, code: _Optional[_Union[ErrorCode, str]] = ..., detail: _Optional[str] = ..., retryable: _Optional[bool] = ..., actual_past_len: _Optional[int] = ...) -> None: ... + +class FlowControl(_message.Message): + __slots__ = ("credits_granted", "max_inflight_chunks", "max_chunk_bytes", "max_prefill_chunk_tokens") + CREDITS_GRANTED_FIELD_NUMBER: _ClassVar[int] + MAX_INFLIGHT_CHUNKS_FIELD_NUMBER: _ClassVar[int] + MAX_CHUNK_BYTES_FIELD_NUMBER: _ClassVar[int] + MAX_PREFILL_CHUNK_TOKENS_FIELD_NUMBER: _ClassVar[int] + credits_granted: int + max_inflight_chunks: int + max_chunk_bytes: int + max_prefill_chunk_tokens: int + def __init__(self, credits_granted: _Optional[int] = ..., max_inflight_chunks: _Optional[int] = ..., max_chunk_bytes: _Optional[int] = ..., max_prefill_chunk_tokens: _Optional[int] = ...) -> None: ... + +class SessionOpen(_message.Message): + __slots__ = ("schema_version", "route_session_id", "route_epoch", "fingerprint", "shard_range", "proposed_flow_control", "accepted_compression") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + FINGERPRINT_FIELD_NUMBER: _ClassVar[int] + SHARD_RANGE_FIELD_NUMBER: _ClassVar[int] + PROPOSED_FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int] + ACCEPTED_COMPRESSION_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + route_session_id: str + route_epoch: int + fingerprint: Fingerprint + shard_range: ShardRange + proposed_flow_control: FlowControl + accepted_compression: _containers.RepeatedScalarFieldContainer[Compression] + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ..., shard_range: _Optional[_Union[ShardRange, _Mapping]] = ..., proposed_flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., accepted_compression: _Optional[_Iterable[_Union[Compression, str]]] = ...) -> None: ... + +class SessionAccepted(_message.Message): + __slots__ = ("schema_version", "route_session_id", "route_epoch", "flow_control", "accepted_compression", "fingerprint") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int] + ACCEPTED_COMPRESSION_FIELD_NUMBER: _ClassVar[int] + FINGERPRINT_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + route_session_id: str + route_epoch: int + flow_control: FlowControl + accepted_compression: _containers.RepeatedScalarFieldContainer[Compression] + fingerprint: Fingerprint + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., accepted_compression: _Optional[_Iterable[_Union[Compression, str]]] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ...) -> None: ... + +class ActivationChunk(_message.Message): + __slots__ = ("envelope", "bundle") + ENVELOPE_FIELD_NUMBER: _ClassVar[int] + BUNDLE_FIELD_NUMBER: _ClassVar[int] + envelope: Envelope + bundle: TensorBundle + def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., bundle: _Optional[_Union[TensorBundle, _Mapping]] = ...) -> None: ... + +class DecodeStep(_message.Message): + __slots__ = ("idempotency_step", "position", "expected_past_len", "tensor", "work_id", "deadline_unix_nanos") + IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int] + POSITION_FIELD_NUMBER: _ClassVar[int] + EXPECTED_PAST_LEN_FIELD_NUMBER: _ClassVar[int] + TENSOR_FIELD_NUMBER: _ClassVar[int] + WORK_ID_FIELD_NUMBER: _ClassVar[int] + DEADLINE_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int] + idempotency_step: int + position: int + expected_past_len: int + tensor: NamedTensor + work_id: str + deadline_unix_nanos: int + def __init__(self, idempotency_step: _Optional[int] = ..., position: _Optional[int] = ..., expected_past_len: _Optional[int] = ..., tensor: _Optional[_Union[NamedTensor, _Mapping]] = ..., work_id: _Optional[str] = ..., deadline_unix_nanos: _Optional[int] = ...) -> None: ... + +class ReleaseSignal(_message.Message): + __slots__ = ("route_session_id", "route_epoch", "work_id") + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + WORK_ID_FIELD_NUMBER: _ClassVar[int] + route_session_id: str + route_epoch: int + work_id: str + def __init__(self, route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., work_id: _Optional[str] = ...) -> None: ... + +class CancelSignal(_message.Message): + __slots__ = ("route_session_id", "route_epoch", "work_id", "reason") + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + WORK_ID_FIELD_NUMBER: _ClassVar[int] + REASON_FIELD_NUMBER: _ClassVar[int] + route_session_id: str + route_epoch: int + work_id: str + reason: str + def __init__(self, route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., work_id: _Optional[str] = ..., reason: _Optional[str] = ...) -> None: ... + +class Ack(_message.Message): + __slots__ = ("work_id", "idempotency_step", "cache_result", "duplicate", "execution_nanos") + WORK_ID_FIELD_NUMBER: _ClassVar[int] + IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int] + CACHE_RESULT_FIELD_NUMBER: _ClassVar[int] + DUPLICATE_FIELD_NUMBER: _ClassVar[int] + EXECUTION_NANOS_FIELD_NUMBER: _ClassVar[int] + work_id: str + idempotency_step: int + cache_result: CacheResult + duplicate: bool + execution_nanos: int + def __init__(self, work_id: _Optional[str] = ..., idempotency_step: _Optional[int] = ..., cache_result: _Optional[_Union[CacheResult, _Mapping]] = ..., duplicate: _Optional[bool] = ..., execution_nanos: _Optional[int] = ...) -> None: ... + +class ShardStatus(_message.Message): + __slots__ = ("work_id", "route_session_id", "idempotency_step", "error", "terminal") + WORK_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + IDEMPOTENCY_STEP_FIELD_NUMBER: _ClassVar[int] + ERROR_FIELD_NUMBER: _ClassVar[int] + TERMINAL_FIELD_NUMBER: _ClassVar[int] + work_id: str + route_session_id: str + idempotency_step: int + error: ShardError + terminal: bool + def __init__(self, work_id: _Optional[str] = ..., route_session_id: _Optional[str] = ..., idempotency_step: _Optional[int] = ..., error: _Optional[_Union[ShardError, _Mapping]] = ..., terminal: _Optional[bool] = ...) -> None: ... + +class SessionRequest(_message.Message): + __slots__ = ("open", "chunk", "decode", "flow_control", "release", "cancel") + OPEN_FIELD_NUMBER: _ClassVar[int] + CHUNK_FIELD_NUMBER: _ClassVar[int] + DECODE_FIELD_NUMBER: _ClassVar[int] + FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int] + RELEASE_FIELD_NUMBER: _ClassVar[int] + CANCEL_FIELD_NUMBER: _ClassVar[int] + open: SessionOpen + chunk: ActivationChunk + decode: DecodeStep + flow_control: FlowControl + release: ReleaseSignal + cancel: CancelSignal + def __init__(self, open: _Optional[_Union[SessionOpen, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., decode: _Optional[_Union[DecodeStep, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., release: _Optional[_Union[ReleaseSignal, _Mapping]] = ..., cancel: _Optional[_Union[CancelSignal, _Mapping]] = ...) -> None: ... + +class SessionResponse(_message.Message): + __slots__ = ("accepted", "chunk", "ack", "flow_control", "status") + ACCEPTED_FIELD_NUMBER: _ClassVar[int] + CHUNK_FIELD_NUMBER: _ClassVar[int] + ACK_FIELD_NUMBER: _ClassVar[int] + FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int] + STATUS_FIELD_NUMBER: _ClassVar[int] + accepted: SessionAccepted + chunk: ActivationChunk + ack: Ack + flow_control: FlowControl + status: ShardStatus + def __init__(self, accepted: _Optional[_Union[SessionAccepted, _Mapping]] = ..., chunk: _Optional[_Union[ActivationChunk, _Mapping]] = ..., ack: _Optional[_Union[Ack, _Mapping]] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., status: _Optional[_Union[ShardStatus, _Mapping]] = ...) -> None: ... + +class CapabilityRequest(_message.Message): + __slots__ = ("schema_version",) + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ...) -> None: ... + +class CapabilityReport(_message.Message): + __slots__ = ("schema_version", "fingerprint", "shard_range", "backend", "device", "validated", "detail", "max_concurrent_sessions", "max_context_tokens", "flow_control", "accepted_compression", "supported_schema_versions", "validated_at_unix_nanos") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + FINGERPRINT_FIELD_NUMBER: _ClassVar[int] + SHARD_RANGE_FIELD_NUMBER: _ClassVar[int] + BACKEND_FIELD_NUMBER: _ClassVar[int] + DEVICE_FIELD_NUMBER: _ClassVar[int] + VALIDATED_FIELD_NUMBER: _ClassVar[int] + DETAIL_FIELD_NUMBER: _ClassVar[int] + MAX_CONCURRENT_SESSIONS_FIELD_NUMBER: _ClassVar[int] + MAX_CONTEXT_TOKENS_FIELD_NUMBER: _ClassVar[int] + FLOW_CONTROL_FIELD_NUMBER: _ClassVar[int] + ACCEPTED_COMPRESSION_FIELD_NUMBER: _ClassVar[int] + SUPPORTED_SCHEMA_VERSIONS_FIELD_NUMBER: _ClassVar[int] + VALIDATED_AT_UNIX_NANOS_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + fingerprint: Fingerprint + shard_range: ShardRange + backend: str + device: str + validated: bool + detail: str + max_concurrent_sessions: int + max_context_tokens: int + flow_control: FlowControl + accepted_compression: _containers.RepeatedScalarFieldContainer[Compression] + supported_schema_versions: _containers.RepeatedScalarFieldContainer[SchemaVersion] + validated_at_unix_nanos: int + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., fingerprint: _Optional[_Union[Fingerprint, _Mapping]] = ..., shard_range: _Optional[_Union[ShardRange, _Mapping]] = ..., backend: _Optional[str] = ..., device: _Optional[str] = ..., validated: _Optional[bool] = ..., detail: _Optional[str] = ..., max_concurrent_sessions: _Optional[int] = ..., max_context_tokens: _Optional[int] = ..., flow_control: _Optional[_Union[FlowControl, _Mapping]] = ..., accepted_compression: _Optional[_Iterable[_Union[Compression, str]]] = ..., supported_schema_versions: _Optional[_Iterable[_Union[SchemaVersion, str]]] = ..., validated_at_unix_nanos: _Optional[int] = ...) -> None: ... + +class HealthRequest(_message.Message): + __slots__ = ("schema_version",) + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ...) -> None: ... + +class HealthReport(_message.Message): + __slots__ = ("schema_version", "state", "active_sessions", "queued_chunks", "batch_occupancy", "kv_pressure", "resident_bytes", "detail") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + STATE_FIELD_NUMBER: _ClassVar[int] + ACTIVE_SESSIONS_FIELD_NUMBER: _ClassVar[int] + QUEUED_CHUNKS_FIELD_NUMBER: _ClassVar[int] + BATCH_OCCUPANCY_FIELD_NUMBER: _ClassVar[int] + KV_PRESSURE_FIELD_NUMBER: _ClassVar[int] + RESIDENT_BYTES_FIELD_NUMBER: _ClassVar[int] + DETAIL_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + state: ServingState + active_sessions: int + queued_chunks: int + batch_occupancy: int + kv_pressure: float + resident_bytes: int + detail: str + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., state: _Optional[_Union[ServingState, str]] = ..., active_sessions: _Optional[int] = ..., queued_chunks: _Optional[int] = ..., batch_occupancy: _Optional[int] = ..., kv_pressure: _Optional[float] = ..., resident_bytes: _Optional[int] = ..., detail: _Optional[str] = ...) -> None: ... + +class ReleaseRequest(_message.Message): + __slots__ = ("schema_version", "route_session_id", "route_epoch") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + route_session_id: str + route_epoch: int + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ...) -> None: ... + +class ReleaseResponse(_message.Message): + __slots__ = ("released", "error") + RELEASED_FIELD_NUMBER: _ClassVar[int] + ERROR_FIELD_NUMBER: _ClassVar[int] + released: bool + error: ShardError + def __init__(self, released: _Optional[bool] = ..., error: _Optional[_Union[ShardError, _Mapping]] = ...) -> None: ... + +class CancelRequest(_message.Message): + __slots__ = ("schema_version", "route_session_id", "route_epoch", "work_id", "reason") + SCHEMA_VERSION_FIELD_NUMBER: _ClassVar[int] + ROUTE_SESSION_ID_FIELD_NUMBER: _ClassVar[int] + ROUTE_EPOCH_FIELD_NUMBER: _ClassVar[int] + WORK_ID_FIELD_NUMBER: _ClassVar[int] + REASON_FIELD_NUMBER: _ClassVar[int] + schema_version: SchemaVersion + route_session_id: str + route_epoch: int + work_id: str + reason: str + def __init__(self, schema_version: _Optional[_Union[SchemaVersion, str]] = ..., route_session_id: _Optional[str] = ..., route_epoch: _Optional[int] = ..., work_id: _Optional[str] = ..., reason: _Optional[str] = ...) -> None: ... + +class CancelResponse(_message.Message): + __slots__ = ("cancelled_work_items", "error") + CANCELLED_WORK_ITEMS_FIELD_NUMBER: _ClassVar[int] + ERROR_FIELD_NUMBER: _ClassVar[int] + cancelled_work_items: int + error: ShardError + def __init__(self, cancelled_work_items: _Optional[int] = ..., error: _Optional[_Union[ShardError, _Mapping]] = ...) -> None: ... diff --git a/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2_grpc.py b/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2_grpc.py new file mode 100644 index 0000000..eec4711 --- /dev/null +++ b/packages/node/meshnet_node/native_protocol/generated/shard_runtime_pb2_grpc.py @@ -0,0 +1,295 @@ +# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT! +"""Client and server classes corresponding to protobuf-defined services.""" +import grpc +import warnings + +from . import shard_runtime_pb2 as shard__runtime__pb2 + +GRPC_GENERATED_VERSION = '1.82.1' +GRPC_VERSION = grpc.__version__ +_version_not_supported = False + +try: + from grpc._utilities import first_version_is_lower + _version_not_supported = first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION) +except ImportError: + _version_not_supported = True + +if _version_not_supported: + raise RuntimeError( + f'The grpc package installed is at version {GRPC_VERSION},' + + ' but the generated code in shard_runtime_pb2_grpc.py depends on' + + f' grpcio>={GRPC_GENERATED_VERSION}.' + + f' Please upgrade your grpc module to grpcio>={GRPC_GENERATED_VERSION}' + + f' or downgrade your generated code using grpcio-tools<={GRPC_VERSION}.' + ) + + +class ShardRuntimeStub: + """--------------------------------------------------------------------------- + Service + --------------------------------------------------------------------------- + + """ + + def __init__(self, channel): + """Constructor. + + Args: + channel: A grpc.Channel. + """ + self.GetCapability = channel.unary_unary( + '/meshnet.shard.v1.ShardRuntime/GetCapability', + request_serializer=shard__runtime__pb2.CapabilityRequest.SerializeToString, + response_deserializer=shard__runtime__pb2.CapabilityReport.FromString, + _registered_method=True) + self.Health = channel.unary_unary( + '/meshnet.shard.v1.ShardRuntime/Health', + request_serializer=shard__runtime__pb2.HealthRequest.SerializeToString, + response_deserializer=shard__runtime__pb2.HealthReport.FromString, + _registered_method=True) + self.Session = channel.stream_stream( + '/meshnet.shard.v1.ShardRuntime/Session', + request_serializer=shard__runtime__pb2.SessionRequest.SerializeToString, + response_deserializer=shard__runtime__pb2.SessionResponse.FromString, + _registered_method=True) + self.Release = channel.unary_unary( + '/meshnet.shard.v1.ShardRuntime/Release', + request_serializer=shard__runtime__pb2.ReleaseRequest.SerializeToString, + response_deserializer=shard__runtime__pb2.ReleaseResponse.FromString, + _registered_method=True) + self.Cancel = channel.unary_unary( + '/meshnet.shard.v1.ShardRuntime/Cancel', + request_serializer=shard__runtime__pb2.CancelRequest.SerializeToString, + response_deserializer=shard__runtime__pb2.CancelResponse.FromString, + _registered_method=True) + + +class ShardRuntimeServicer: + """--------------------------------------------------------------------------- + Service + --------------------------------------------------------------------------- + + """ + + def GetCapability(self, request, context): + """What this worker can execute. Read before a route is built. + """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') + + def Health(self, request, context): + """Live load and serving state. + """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') + + def Session(self, request_iterator, context): + """One long-lived bidirectional stream per Route Session Activation Seam. + + The stream opens with SessionOpen/SessionAccepted, then carries bounded + prefill chunks and decode steps in both directions for the life of the + session. Per-token channel creation is a non-goal: the handshake cost is + paid once and the hot path carries only what changes. + """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') + + def Release(self, request, context): + """Drop session state out of band. Idempotent. + """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') + + def Cancel(self, request, context): + """Cancel out of band, on a fresh call. + + In-band CancelSignal is preferred, but a sender that is blocked on flow + control cannot write one — a cancel that can only travel down a wedged + stream is not a cancel. This RPC always has a path to the worker. + """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') + + +def add_ShardRuntimeServicer_to_server(servicer, server): + rpc_method_handlers = { + 'GetCapability': grpc.unary_unary_rpc_method_handler( + servicer.GetCapability, + request_deserializer=shard__runtime__pb2.CapabilityRequest.FromString, + response_serializer=shard__runtime__pb2.CapabilityReport.SerializeToString, + ), + 'Health': grpc.unary_unary_rpc_method_handler( + servicer.Health, + request_deserializer=shard__runtime__pb2.HealthRequest.FromString, + response_serializer=shard__runtime__pb2.HealthReport.SerializeToString, + ), + 'Session': grpc.stream_stream_rpc_method_handler( + servicer.Session, + request_deserializer=shard__runtime__pb2.SessionRequest.FromString, + response_serializer=shard__runtime__pb2.SessionResponse.SerializeToString, + ), + 'Release': grpc.unary_unary_rpc_method_handler( + servicer.Release, + request_deserializer=shard__runtime__pb2.ReleaseRequest.FromString, + response_serializer=shard__runtime__pb2.ReleaseResponse.SerializeToString, + ), + 'Cancel': grpc.unary_unary_rpc_method_handler( + servicer.Cancel, + request_deserializer=shard__runtime__pb2.CancelRequest.FromString, + response_serializer=shard__runtime__pb2.CancelResponse.SerializeToString, + ), + } + generic_handler = grpc.method_handlers_generic_handler( + 'meshnet.shard.v1.ShardRuntime', rpc_method_handlers) + server.add_generic_rpc_handlers((generic_handler,)) + server.add_registered_method_handlers('meshnet.shard.v1.ShardRuntime', rpc_method_handlers) + + + # This class is part of an EXPERIMENTAL API. +class ShardRuntime: + """--------------------------------------------------------------------------- + Service + --------------------------------------------------------------------------- + + """ + + @staticmethod + def GetCapability(request, + target, + options=(), + channel_credentials=None, + call_credentials=None, + insecure=False, + compression=None, + wait_for_ready=None, + timeout=None, + metadata=None): + return grpc.experimental.unary_unary( + request, + target, + '/meshnet.shard.v1.ShardRuntime/GetCapability', + shard__runtime__pb2.CapabilityRequest.SerializeToString, + shard__runtime__pb2.CapabilityReport.FromString, + options, + channel_credentials, + insecure, + call_credentials, + compression, + wait_for_ready, + timeout, + metadata, + _registered_method=True) + + @staticmethod + def Health(request, + target, + options=(), + channel_credentials=None, + call_credentials=None, + insecure=False, + compression=None, + wait_for_ready=None, + timeout=None, + metadata=None): + return grpc.experimental.unary_unary( + request, + target, + '/meshnet.shard.v1.ShardRuntime/Health', + shard__runtime__pb2.HealthRequest.SerializeToString, + shard__runtime__pb2.HealthReport.FromString, + options, + channel_credentials, + insecure, + call_credentials, + compression, + wait_for_ready, + timeout, + metadata, + _registered_method=True) + + @staticmethod + def Session(request_iterator, + target, + options=(), + channel_credentials=None, + call_credentials=None, + insecure=False, + compression=None, + wait_for_ready=None, + timeout=None, + metadata=None): + return grpc.experimental.stream_stream( + request_iterator, + target, + '/meshnet.shard.v1.ShardRuntime/Session', + shard__runtime__pb2.SessionRequest.SerializeToString, + shard__runtime__pb2.SessionResponse.FromString, + options, + channel_credentials, + insecure, + call_credentials, + compression, + wait_for_ready, + timeout, + metadata, + _registered_method=True) + + @staticmethod + def Release(request, + target, + options=(), + channel_credentials=None, + call_credentials=None, + insecure=False, + compression=None, + wait_for_ready=None, + timeout=None, + metadata=None): + return grpc.experimental.unary_unary( + request, + target, + '/meshnet.shard.v1.ShardRuntime/Release', + shard__runtime__pb2.ReleaseRequest.SerializeToString, + shard__runtime__pb2.ReleaseResponse.FromString, + options, + channel_credentials, + insecure, + call_credentials, + compression, + wait_for_ready, + timeout, + metadata, + _registered_method=True) + + @staticmethod + def Cancel(request, + target, + options=(), + channel_credentials=None, + call_credentials=None, + insecure=False, + compression=None, + wait_for_ready=None, + timeout=None, + metadata=None): + return grpc.experimental.unary_unary( + request, + target, + '/meshnet.shard.v1.ShardRuntime/Cancel', + shard__runtime__pb2.CancelRequest.SerializeToString, + shard__runtime__pb2.CancelResponse.FromString, + options, + channel_credentials, + insecure, + call_credentials, + compression, + wait_for_ready, + timeout, + metadata, + _registered_method=True) diff --git a/packages/node/native/CMakeLists.txt b/packages/node/native/CMakeLists.txt new file mode 100644 index 0000000..71aed7a --- /dev/null +++ b/packages/node/native/CMakeLists.txt @@ -0,0 +1,74 @@ +# Native Shard protocol: C++ schema generation, build wiring and conformance test. +# +# The C++ stubs are generated into the build tree at configure time — they are +# never committed. A C++ consumer already needs a toolchain, so committing +# generated C++ would only create a second copy of the schema that can rot. +# +# gRPC C++ is optional here on purpose. The conformance test only needs message +# types, so the schema can be verified on a machine that has protobuf but not +# the gRPC C++ stack. When gRPC *is* found, the service stubs are generated too +# and exported as `shard_runtime_grpc` for the worker (DGR-008) to link. +# +# Build: +# cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH= +# cmake --build build/native -j +# ctest --test-dir build/native --output-on-failure +# +# `scripts/bootstrap_native_toolchain.sh` builds a protobuf install from source +# when the system has none. + +cmake_minimum_required(VERSION 3.24) +project(meshnet_shard_protocol CXX) + +set(CMAKE_CXX_STANDARD 17) +set(CMAKE_CXX_STANDARD_REQUIRED ON) + +find_package(protobuf CONFIG REQUIRED) +find_package(gRPC CONFIG QUIET) + +set(SHARD_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/proto/shard_runtime.proto") + +# Message types: always available. +add_library(shard_runtime_proto STATIC "${SHARD_PROTO}") +target_link_libraries(shard_runtime_proto PUBLIC protobuf::libprotobuf) +target_include_directories(shard_runtime_proto PUBLIC "${CMAKE_CURRENT_BINARY_DIR}") +protobuf_generate( + TARGET shard_runtime_proto + LANGUAGE cpp + IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto" + PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}" +) + +# Service stubs: only when the gRPC C++ stack is present. +if(gRPC_FOUND) + add_library(shard_runtime_grpc STATIC "${SHARD_PROTO}") + target_link_libraries(shard_runtime_grpc PUBLIC shard_runtime_proto gRPC::grpc++) + target_include_directories(shard_runtime_grpc PUBLIC "${CMAKE_CURRENT_BINARY_DIR}") + protobuf_generate( + TARGET shard_runtime_grpc + LANGUAGE grpc + GENERATE_EXTENSIONS .grpc.pb.h .grpc.pb.cc + PLUGIN "protoc-gen-grpc=$" + IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto" + PROTOC_OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}" + ) + message(STATUS "gRPC C++ found: building ShardRuntime service stubs") +else() + message(STATUS "gRPC C++ not found: building message types only " + "(sufficient for the conformance test)") +endif() + +enable_testing() + +add_executable(shard_protocol_conformance tests/test_shard_protocol_conformance.cpp) +target_link_libraries(shard_protocol_conformance PRIVATE shard_runtime_proto) + +# The test asserts against the same committed vectors the Python test uses, and +# writes its own re-serialization back out so Python can prove the two languages +# agree byte-for-byte. +add_test( + NAME shard_protocol_conformance + COMMAND shard_protocol_conformance + "${CMAKE_CURRENT_SOURCE_DIR}/testdata" + "${CMAKE_CURRENT_BINARY_DIR}" +) diff --git a/packages/node/native/README.md b/packages/node/native/README.md new file mode 100644 index 0000000..b5f6ed4 --- /dev/null +++ b/packages/node/native/README.md @@ -0,0 +1,69 @@ +# Native Shard protocol + +`proto/shard_runtime.proto` is the semantic contract between a Meshnet node and +a Shard worker: Protocol Buffers over gRPC/HTTP2 (ADR-0020). It is the source of +truth. The Python and C++ types are generated from it; neither is the contract. + +## What lives here + +| Path | Purpose | +|---|---| +| `proto/shard_runtime.proto` | The schema: capability, health, session stream, release, cancel | +| `testdata/*.binpb` | Committed conformance vectors both languages assert against | +| `tests/test_shard_protocol_conformance.cpp` | C++ conformance test | +| `CMakeLists.txt` | C++ generation, build wiring, and `ctest` registration | + +The Python stubs are generated into +`packages/node/meshnet_node/native_protocol/generated/` and are committed, so +installing a node needs no protoc. The C++ stubs are generated into the build +tree and are never committed — a C++ consumer already has a toolchain, and a +committed copy could only rot. + +## Regenerating + +```bash +pip install grpcio-tools==1.82.1 # bundles protoc; no system protoc needed +python scripts/generate_native_protocol.py # rewrite the Python stubs +python scripts/generate_native_protocol.py --check # fail if they drifted +python scripts/generate_protocol_goldens.py --check # fail if the vectors drifted +``` + +Both `--check` modes run in CI via `tests/test_native_shard_protocol.py`, so a +schema edit that is not accompanied by regenerated output fails the suite rather +than shipping stubs that disagree with the schema they claim to implement. + +## Building and running the C++ conformance test + +If the machine has no protobuf C++ toolchain: + +```bash +scripts/bootstrap_native_toolchain.sh build/native-toolchain +``` + +Then: + +```bash +cmake -S packages/node/native -B build/native \ + -DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain" +cmake --build build/native -j +ctest --test-dir build/native --output-on-failure +``` + +gRPC C++ is optional: without it, CMake builds the message types only, which is +all the conformance test needs. When gRPC C++ *is* found, the `ShardRuntime` +service stubs are built too and exported as `shard_runtime_grpc` for the worker +(DGR-008) to link. + +## How the cross-language check actually proves something + +Two codecs that each round-trip their own output prove only that each is +self-consistent. Instead: + +1. Python builds the canonical message and commits its bytes to `testdata/`. +2. The C++ test parses *those* bytes, asserts every field, independently + recomputes the CRC32C from the polynomial, and re-serializes to + `cpp_roundtrip.binpb` in the build tree. +3. `test_cpp_and_python_agree_byte_for_byte` compares that file to the golden. + +Byte equality across the two implementations is the claim; anything less is two +parallel test suites that can drift apart. diff --git a/packages/node/native/proto/shard_runtime.proto b/packages/node/native/proto/shard_runtime.proto new file mode 100644 index 0000000..71a00c9 --- /dev/null +++ b/packages/node/native/proto/shard_runtime.proto @@ -0,0 +1,591 @@ +// The native Shard data plane: Protocol Buffers over gRPC/HTTP2 (ADR-0020). +// +// This schema — not any Python or C++ type — is the semantic contract between a +// Meshnet node and a Shard worker. Direct hops speak gRPC. When direct +// connectivity is unavailable the existing relay carries these exact serialized +// frames as opaque binary, which is why every deadline, cancellation and +// identity field is carried *in the schema* rather than left to gRPC metadata: +// a relayed frame has no HTTP/2 context to inherit them from. +// +// Meshnet remains the only control plane. Nothing here selects routes, prices +// work, or authenticates peers; the worker executes a layer range it has been +// admitted for and reports what it did. + +syntax = "proto3"; + +package meshnet.shard.v1; + +option cc_enable_arenas = true; +option go_package = "github.com/meshnet/shard/v1;shardv1"; + +// --------------------------------------------------------------------------- +// Versioning +// --------------------------------------------------------------------------- + +// Wire-compatibility generation of this schema. A worker rejects a peer whose +// SCHEMA_VERSION it cannot satisfy instead of guessing at field meaning. +// +// This is deliberately NOT the `X-Meshnet-Wire` version of the legacy HTTP +// activation format (currently "2", see ADR-0008). The native protocol is a +// separate contract with its own generation counter and starts at 1. +enum SchemaVersion { + SCHEMA_VERSION_UNSPECIFIED = 0; + SCHEMA_VERSION_1 = 1; +} + +// --------------------------------------------------------------------------- +// Tensor bundle — the public activation boundary +// --------------------------------------------------------------------------- + +// Element type of a tensor payload. +// +// bfloat16 is the canonical activation dtype at every Shard boundary regardless +// of the weight quantization a node uses internally (ADR-0008): weights may be +// NF4 or INT8, compute upcasts, and the boundary stays bfloat16. The other +// values exist for auxiliary tensors (position ids, attention masks) and for +// architectures whose boundary is not a plain hidden state. +enum DType { + DTYPE_UNSPECIFIED = 0; + DTYPE_BFLOAT16 = 1; + DTYPE_FLOAT16 = 2; + DTYPE_FLOAT32 = 3; + DTYPE_INT32 = 4; + DTYPE_INT64 = 5; + DTYPE_UINT8 = 6; + DTYPE_INT8 = 7; + DTYPE_BOOL = 8; +} + +// Byte order of a tensor payload. Little-endian is canonical on the wire; the +// field is explicit so a big-endian peer is rejected loudly rather than +// silently reading byte-swapped activations. +enum ByteOrder { + BYTE_ORDER_UNSPECIFIED = 0; + BYTE_ORDER_LITTLE_ENDIAN = 1; + BYTE_ORDER_BIG_ENDIAN = 2; +} + +// Payload compression. Mirrors the policy-driven zstd decision the existing +// activation seam already makes (`activation_compression.py`): compression is a +// transport optimisation and NONE is always a legal choice for any payload. +enum Compression { + COMPRESSION_UNSPECIFIED = 0; + COMPRESSION_NONE = 1; + COMPRESSION_ZSTD = 2; +} + +// Integrity algorithm covering a payload. +enum ChecksumAlgorithm { + CHECKSUM_ALGORITHM_UNSPECIFIED = 0; + CHECKSUM_ALGORITHM_NONE = 1; + CHECKSUM_ALGORITHM_CRC32C = 2; +} + +// An integrity check over the *uncompressed* canonical payload bytes. +// +// Checksumming the uncompressed bytes (not the compressed frame) means the +// value is stable whether a hop compressed the payload or not, so a relay may +// re-frame or a peer may decline compression without invalidating it. +message Checksum { + ChecksumAlgorithm algorithm = 1; + // Big-endian encoding of the checksum value. + bytes value = 2; +} + +// One slice of a tensor's wire payload. +// +// Fragments bound the size of a single gRPC message. `byte_offset` lets a +// receiver verify the fragments tile the body exactly — no hole, no overlap — +// instead of trusting arrival order. +// +// Offsets and payloads describe the *wire* body, which is the compressed frame +// when the parent tensor declares a compression. A zstd frame is not decodable +// per fragment, so a receiver reassembles first and decompresses once. The +// uncompressed size is not repeated here: `NamedTensor.total_bytes` is the +// single source of truth, and a second copy could only ever disagree with it. +message TensorFragment { + uint32 fragment_index = 1; + uint32 fragment_count = 2; + // Offset of this fragment within the tensor's wire body. + uint64 byte_offset = 3; + // Fragment bytes, compressed iff the parent tensor declares a compression. + bytes payload = 4; + + reserved 5; + reserved "uncompressed_size"; +} + +// One named tensor at an architecture boundary. +message NamedTensor { + // Architecture-defined boundary name, e.g. "hidden_states", "position_ids". + // A boundary may need more than one tensor, which is why the payload is a + // named bundle rather than a bare buffer (ADR-0020). + string name = 1; + repeated int64 shape = 2; + DType dtype = 3; + ByteOrder byte_order = 4; + // Total uncompressed payload size. A receiver sizes its buffer from this + // before reading fragments and refuses a bundle that exceeds its budget. + uint64 total_bytes = 5; + Compression compression = 6; + // Over the uncompressed payload; see Checksum. + Checksum checksum = 7; + // Ordered fragments. A tensor small enough for one message carries exactly + // one fragment with fragment_count == 1. + repeated TensorFragment fragments = 8; +} + +// A versioned named-tensor bundle: the public activation boundary payload. +message TensorBundle { + // Generation of the bundle layout itself, independent of SchemaVersion so a + // boundary payload can evolve without a whole-protocol version bump. + uint32 bundle_version = 1; + repeated NamedTensor tensors = 2; +} + +// --------------------------------------------------------------------------- +// Identity: what work this is, for which route, against which artifact +// --------------------------------------------------------------------------- + +// Exact identity of the executable thing a Shard runs. +// +// Both fingerprints must match end to end across a route. Two nodes agreeing on +// a model name but not on quantization, kernel or tail-norm placement would +// produce silently wrong tokens, so identity is a digest, not a label. +message Fingerprint { + // Digest of the exact Model Artifact (weights + config), e.g. the GGUF hash. + string model_artifact_digest = 1; + // Digest of the exact runtime recipe (backend, quantization, kernels, dtype). + string runtime_recipe_digest = 2; + // Human-readable recipe identity, mirroring the capability report an admitted + // node already registers with (ADR-0023). Labels are for diagnosis; the + // digests above are what a peer actually compares. + string recipe_id = 3; + string recipe_version = 4; + string catalogue_version = 5; +} + +// The contiguous transformer layer range a Shard owns. +message ShardRange { + // Registered range, inclusive start, exclusive end. + uint32 start_layer = 1; + uint32 end_layer = 2; + // The layer this Shard must actually begin at for this route (ADR-0012). + // + // Shard ranges may overlap; the Tracker resolves the overlap when it builds + // the route and every hop is told where the previous hop stopped. Executing + // from `start_layer` when `effective_start_layer` is higher would re-apply + // layers already applied to the incoming activation and silently corrupt the + // result. A worker executes [effective_start_layer, end_layer). + uint32 effective_start_layer = 3; +} + +// Which part of generation a message belongs to. +enum Phase { + PHASE_UNSPECIFIED = 0; + PHASE_PREFILL = 1; + PHASE_DECODE = 2; + PHASE_RELEASE = 3; + PHASE_CANCEL = 4; +} + +// Token span carried by one chunk. +message PositionSpan { + // Absolute position of the first token in this chunk within the sequence. + uint64 first_position = 1; + // Number of token positions in this chunk. A decode step carries 1. + uint32 token_count = 2; +} + +// Bounded chunking for a prefill. +// +// A long prompt is split into token-aligned chunks before the first forward +// pass (ADR-0008) so peak transfer per boundary stays bounded regardless of +// prompt length. Splits never fall mid-token. +message ChunkInfo { + uint32 chunk_index = 1; + uint32 chunk_count = 2; + // True on the final chunk of a prefill. A receiver that has not seen a final + // chunk knows the prefill is still incomplete. + bool final_chunk = 3; +} + +// How the sender expects the receiver's Hot KV State to be positioned. +enum CacheMode { + CACHE_MODE_UNSPECIFIED = 0; + // No session state; the payload carries everything needed. Always a legal + // fallback and the recovery path after a miss. + CACHE_MODE_STATELESS = 1; + // Establish fresh session state for this Shard's layer range. + CACHE_MODE_PREFILL = 2; + // Continue existing session state. `expected_past_len` must match exactly. + CACHE_MODE_DECODE = 3; +} + +// The sender's expectation about receiver-side cache state (ADR-0022). +// +// A mismatch is always a declared cache miss, never a silent stateless forward: +// running a single-token decode payload without the matching cache would emit +// plausible garbage. The receiver answers a miss with ShardError.CACHE_MISS and +// the head re-prefills the whole sequence under the same Route Session ID. +message CacheExpectation { + CacheMode mode = 1; + // Decode only: the number of tokens the receiver's session cache must already + // hold for this Shard's layer range. + uint64 expected_past_len = 2; +} + +// What the receiver's cache actually did. +message CacheResult { + CacheMode mode = 1; + // Tokens held for this Shard's range after the step completed. + uint64 past_len = 2; + // True when session state was reused rather than rebuilt. + bool cache_hit = 3; +} + +// Everything required to interpret one unit of work, independent of transport. +// +// Carried on every request and response because a relayed frame arrives as +// opaque binary with no gRPC metadata, no deadline and no channel identity. +message Envelope { + SchemaVersion schema_version = 1; + // Unique id for this unit of work; also the unit of billing attribution. + string work_id = 2; + // The Route Session this work belongs to. Together with `route_epoch` it maps + // to exactly one isolated llama sequence / bounded context (ADR-0020). + string route_session_id = 3; + // Bumped by the control plane whenever the route changes. A worker refuses + // work from a stale epoch rather than mixing it into live session state. + uint64 route_epoch = 4; + Fingerprint fingerprint = 5; + ShardRange shard_range = 6; + Phase phase = 7; + PositionSpan position = 8; + // Monotonic step within (route_session_id, route_epoch). + // + // Idempotency key: a retried or duplicated step carries the same value and + // must be acknowledged, not re-applied. Re-applying a decode step would + // advance the KV cache twice and desynchronise the route. + uint64 idempotency_step = 9; + CacheExpectation cache_expectation = 10; + // Absolute deadline. Carried in-band so a relayed frame keeps its deadline; + // on a direct hop it is set consistently with the gRPC deadline. + int64 deadline_unix_nanos = 11; + // Chunking, when this message is part of a bounded prefill. + ChunkInfo chunk = 12; +} + +// --------------------------------------------------------------------------- +// Structured errors +// --------------------------------------------------------------------------- + +// Why a unit of work could not be executed as asked. +// +// These are semantic outcomes, distinct from transport failure. They travel as +// a ShardStatus on the stream (and as google.rpc.Status details on unary RPCs) +// so a relayed frame carries the same diagnosis a direct gRPC hop would. +enum ErrorCode { + ERROR_CODE_UNSPECIFIED = 0; + // Peer cannot satisfy the requested SchemaVersion. + ERROR_CODE_SCHEMA_UNSUPPORTED = 1; + // Model artifact or runtime recipe digest differs from this worker's. + ERROR_CODE_FINGERPRINT_MISMATCH = 2; + // Work arrived for a route epoch the worker has already moved past. + ERROR_CODE_EPOCH_STALE = 3; + // Requested layer range is not the range this worker serves. + ERROR_CODE_SHARD_RANGE_MISMATCH = 4; + // Session state absent or positioned differently than expected (ADR-0022). + // The head recovers with one full re-prefill; this is not a fatal error. + ERROR_CODE_CACHE_MISS = 5; + // Admission budget (weights, KV, scratch, queue depth) would be exceeded. + ERROR_CODE_RESOURCE_EXHAUSTED = 6; + // Payload failed its checksum, or fragments did not cover the tensor. + ERROR_CODE_PAYLOAD_CORRUPT = 7; + // Work was cancelled by the control plane or the peer. + ERROR_CODE_CANCELLED = 8; + // Deadline in the envelope had already passed when work was dequeued. + ERROR_CODE_DEADLINE_EXCEEDED = 9; + // Sender exceeded its granted flow-control credit. + ERROR_CODE_FLOW_CONTROL_VIOLATION = 10; + // The worker failed while executing; detail is sanitized. + ERROR_CODE_INTERNAL = 11; +} + +message ShardError { + ErrorCode code = 1; + // Sanitized, operator-facing explanation. Never a raw exception, file path or + // credential (ADR-0023 sanitization rule). + string detail = 2; + // True when the same work may be retried unchanged. A CACHE_MISS is not + // retryable unchanged — the head must re-prefill first. + bool retryable = 3; + // Set on CACHE_MISS so the head knows where the receiver actually is. + uint64 actual_past_len = 4; +} + +// --------------------------------------------------------------------------- +// Flow control +// --------------------------------------------------------------------------- + +// Application-level credit, layered on top of HTTP/2 flow control. +// +// HTTP/2 bounds bytes in flight; it does not bound how much *work* a worker has +// queued, and a relayed frame gets no HTTP/2 window at all. Credits bound the +// number of un-acked chunks a sender may have outstanding, which is what keeps +// worker queues and KV pressure bounded and lets prefill avoid starving decode. +message FlowControl { + // Additional chunks the receiver is willing to accept beyond those already + // granted. Purely additive: a grant never reduces outstanding credit. + uint32 credits_granted = 1; + // Hard ceiling on un-acked chunks, independent of granted credit. + uint32 max_inflight_chunks = 2; + // Hard ceiling on the serialized size of one chunk message. + uint64 max_chunk_bytes = 3; + // Hard ceiling on token positions per prefill chunk. + uint32 max_prefill_chunk_tokens = 4; +} + +// --------------------------------------------------------------------------- +// Session stream messages +// --------------------------------------------------------------------------- + +// Opens one long-lived bidirectional stream for one Route Session Activation +// Seam. The handshake settles version, identity and the initial flow-control +// window before any activation is sent, so an incompatible peer fails at open +// rather than mid-generation. +message SessionOpen { + // Highest schema version the initiator supports; the acceptor replies with + // the version actually negotiated. + SchemaVersion schema_version = 1; + string route_session_id = 2; + uint64 route_epoch = 3; + Fingerprint fingerprint = 4; + ShardRange shard_range = 5; + // Flow-control limits the initiator can honour. The acceptor replies with the + // limits that actually apply. + FlowControl proposed_flow_control = 6; + // Compressions the initiator can decode. NONE is implicitly always supported. + repeated Compression accepted_compression = 7; +} + +message SessionAccepted { + // The version both peers will use for the life of this stream. + SchemaVersion schema_version = 1; + string route_session_id = 2; + uint64 route_epoch = 3; + // Authoritative limits. The initiator must not exceed these. + FlowControl flow_control = 4; + repeated Compression accepted_compression = 5; + // Fingerprint the worker actually serves, so a mismatch is visible at open. + Fingerprint fingerprint = 6; +} + +// One unit of activation work: a bounded prefill chunk or a decode step. +message ActivationChunk { + Envelope envelope = 1; + TensorBundle bundle = 2; +} + +// The small decode fast path. +// +// A decode step is one token: the envelope's identity fields are already fixed +// for the life of the stream, so repeating them per token is pure overhead on +// the hottest path. A DecodeStep carries only what changes — the step, the +// position, and one tensor — and inherits the rest from the SessionOpen +// handshake. A peer may always fall back to ActivationChunk with PHASE_DECODE. +message DecodeStep { + // Idempotency step within the session; also orders the stream. + uint64 idempotency_step = 1; + // Absolute position of this token. + uint64 position = 2; + // Tokens the receiver's cache must already hold. A mismatch is a CACHE_MISS. + uint64 expected_past_len = 3; + // The single boundary tensor for this token, typically [1, 1, hidden]. + NamedTensor tensor = 4; + // Work id for attribution; the session/epoch/fingerprint/range are inherited. + string work_id = 5; + int64 deadline_unix_nanos = 6; +} + +// Drop session state for a Route Session. Bounded memory does not depend on +// this arriving — workers also evict by TTL and LRU (ADR-0022) — but an +// explicit release returns KV immediately instead of holding it for the TTL. +message ReleaseSignal { + string route_session_id = 1; + uint64 route_epoch = 2; + string work_id = 3; +} + +// Abandon in-flight work. +// +// Cancellation must remain possible when the stream is wedged behind flow +// control, which is why Cancel also exists as a unary RPC on a fresh call. +message CancelSignal { + string route_session_id = 1; + uint64 route_epoch = 2; + // Cancel one unit of work; empty cancels every in-flight unit for the session. + string work_id = 3; + string reason = 4; +} + +// Acknowledges a chunk or decode step, releasing its flow-control credit and +// recording where the receiver's cache now stands. +message Ack { + string work_id = 1; + uint64 idempotency_step = 2; + CacheResult cache_result = 3; + // True when the step was recognised as an already-applied duplicate and was + // therefore acknowledged without being re-applied. + bool duplicate = 4; + // Observed execution time, for Generation Telemetry. + uint64 execution_nanos = 5; +} + +// A terminal outcome for one unit of work, or for the session. +message ShardStatus { + string work_id = 1; + string route_session_id = 2; + uint64 idempotency_step = 3; + ShardError error = 4; + // True when the stream itself is finished and no further work will be served. + bool terminal = 5; +} + +message SessionRequest { + oneof kind { + SessionOpen open = 1; + ActivationChunk chunk = 2; + DecodeStep decode = 3; + FlowControl flow_control = 4; + ReleaseSignal release = 5; + CancelSignal cancel = 6; + } +} + +message SessionResponse { + oneof kind { + SessionAccepted accepted = 1; + // Result activations forwarded toward the next Shard, or to the head. + ActivationChunk chunk = 2; + Ack ack = 3; + FlowControl flow_control = 4; + ShardStatus status = 5; + } +} + +// --------------------------------------------------------------------------- +// Capability and health +// --------------------------------------------------------------------------- + +message CapabilityRequest { + SchemaVersion schema_version = 1; +} + +// What this worker can actually execute, proven by a bounded real forward +// (ADR-0023). Detected hardware is not a capability; only a validated recipe is. +message CapabilityReport { + SchemaVersion schema_version = 1; + Fingerprint fingerprint = 2; + ShardRange shard_range = 3; + // Backend/device identity, e.g. "rocm:gfx1151", "cpu:avx512". + string backend = 4; + string device = 5; + // True only when a bounded real forward has passed for exactly this + // (artifact, range, recipe, device). Never set from hardware detection alone. + bool validated = 6; + // Sanitized reason when `validated` is false. + string detail = 7; + // Admission budgets this worker will enforce. + uint64 max_concurrent_sessions = 8; + uint64 max_context_tokens = 9; + FlowControl flow_control = 10; + repeated Compression accepted_compression = 11; + repeated SchemaVersion supported_schema_versions = 12; + int64 validated_at_unix_nanos = 13; +} + +message HealthRequest { + SchemaVersion schema_version = 1; +} + +enum ServingState { + SERVING_STATE_UNSPECIFIED = 0; + // Accepting new Route Sessions. + SERVING_STATE_SERVING = 1; + // Alive but refusing new sessions; existing sessions continue draining. + SERVING_STATE_DRAINING = 2; + // Alive but not routable — e.g. registered-but-dark pending certification. + SERVING_STATE_NOT_SERVING = 3; +} + +// Live load, for backpressure and Generation Telemetry. +message HealthReport { + SchemaVersion schema_version = 1; + ServingState state = 2; + uint32 active_sessions = 3; + uint32 queued_chunks = 4; + // Occupancy of the continuous decode batch. + uint32 batch_occupancy = 5; + // Fraction of the KV budget in use, 0..1. + float kv_pressure = 6; + uint64 resident_bytes = 7; + string detail = 8; +} + +message ReleaseRequest { + SchemaVersion schema_version = 1; + string route_session_id = 2; + uint64 route_epoch = 3; +} + +message ReleaseResponse { + // True when session state existed and was dropped; false when there was + // nothing to drop, which is a success, not an error — release is idempotent. + bool released = 1; + ShardError error = 2; +} + +message CancelRequest { + SchemaVersion schema_version = 1; + string route_session_id = 2; + uint64 route_epoch = 3; + // Empty cancels every in-flight unit for the session. + string work_id = 4; + string reason = 5; +} + +message CancelResponse { + uint32 cancelled_work_items = 1; + ShardError error = 2; +} + +// --------------------------------------------------------------------------- +// Service +// --------------------------------------------------------------------------- + +service ShardRuntime { + // What this worker can execute. Read before a route is built. + rpc GetCapability(CapabilityRequest) returns (CapabilityReport); + + // Live load and serving state. + rpc Health(HealthRequest) returns (HealthReport); + + // One long-lived bidirectional stream per Route Session Activation Seam. + // + // The stream opens with SessionOpen/SessionAccepted, then carries bounded + // prefill chunks and decode steps in both directions for the life of the + // session. Per-token channel creation is a non-goal: the handshake cost is + // paid once and the hot path carries only what changes. + rpc Session(stream SessionRequest) returns (stream SessionResponse); + + // Drop session state out of band. Idempotent. + rpc Release(ReleaseRequest) returns (ReleaseResponse); + + // Cancel out of band, on a fresh call. + // + // In-band CancelSignal is preferred, but a sender that is blocked on flow + // control cannot write one — a cancel that can only travel down a wedged + // stream is not a cancel. This RPC always has a path to the worker. + rpc Cancel(CancelRequest) returns (CancelResponse); +} diff --git a/packages/node/native/testdata/capability_report_golden.binpb b/packages/node/native/testdata/capability_report_golden.binpb new file mode 100644 index 0000000..9b08867 --- /dev/null +++ b/packages/node/native/testdata/capability_report_golden.binpb @@ -0,0 +1,2 @@ +F +sha256:1f0c9d2esha256:ab77e410llama-gguf-q4km-rocm"3* 2026.07.1 "rocm*gfx11510@H@R  ZbhНθ \ No newline at end of file diff --git a/packages/node/native/testdata/session_request_golden.binpb b/packages/node/native/testdata/session_request_golden.binpb new file mode 100644 index 0000000..9233d5a Binary files /dev/null and b/packages/node/native/testdata/session_request_golden.binpb differ diff --git a/packages/node/native/tests/test_shard_protocol_conformance.cpp b/packages/node/native/tests/test_shard_protocol_conformance.cpp new file mode 100644 index 0000000..966198f --- /dev/null +++ b/packages/node/native/tests/test_shard_protocol_conformance.cpp @@ -0,0 +1,337 @@ +// C++ conformance test for the native Shard protocol. +// +// This test does not check that C++ can round-trip its own output — that would +// only prove C++ is self-consistent. It parses the *Python-produced* committed +// vectors, asserts every field the protocol promises to carry, independently +// recomputes the CRC32C over the reassembled tensor, and re-serializes the +// message back out for the Python test to compare byte-for-byte. Only that +// closes the loop: both languages agree on the same bytes and the same meaning. +// +// Run via ctest; see packages/node/native/CMakeLists.txt. + +#include "shard_runtime.pb.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +// `pb` is already taken by protobuf's own generated headers. +namespace sp = meshnet::shard::v1; + +namespace { + +int g_failures = 0; + +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + std::cerr << "FAIL " << __FILE__ << ":" << __LINE__ << ": " << #cond \ + << "\n"; \ + ++g_failures; \ + } \ + } while (0) + +#define CHECK_EQ(actual, expected) \ + do { \ + auto &&a_ = (actual); \ + auto &&e_ = (expected); \ + if (!(a_ == e_)) { \ + std::cerr << "FAIL " << __FILE__ << ":" << __LINE__ << ": " << #actual \ + << " == " << #expected << "\n actual: " << a_ \ + << "\n expected: " << e_ << "\n"; \ + ++g_failures; \ + } \ + } while (0) + +// Independent CRC32C (Castagnoli). Written from the polynomial rather than +// shared with the Python side on purpose: a checksum that both languages +// compute with the *same* code proves nothing about interoperability. +uint32_t Crc32c(const std::string &data) { + static uint32_t table[256]; + static bool built = false; + if (!built) { + for (uint32_t i = 0; i < 256; ++i) { + uint32_t c = i; + for (int k = 0; k < 8; ++k) { + c = (c & 1) ? (c >> 1) ^ 0x82F63B78u : (c >> 1); + } + table[i] = c; + } + built = true; + } + uint32_t crc = 0xFFFFFFFFu; + for (unsigned char byte : data) { + crc = (crc >> 8) ^ table[(crc ^ byte) & 0xFF]; + } + return crc ^ 0xFFFFFFFFu; +} + +std::string ReadFile(const std::filesystem::path &path) { + std::ifstream in(path, std::ios::binary); + if (!in) { + std::cerr << "FAIL cannot read " << path << "\n"; + ++g_failures; + return {}; + } + std::ostringstream buffer; + buffer << in.rdbuf(); + return buffer.str(); +} + +// Reassemble a tensor's fragments and validate coverage, exactly as a worker +// must before it feeds the bytes to a forward pass. +std::string ReassembleUncompressed(const sp::NamedTensor &tensor) { + std::string body; + std::vector fragments; + for (const auto &fragment : tensor.fragments()) { + fragments.push_back(&fragment); + } + CHECK(!fragments.empty()); + for (uint32_t index = 0; index < fragments.size(); ++index) { + const sp::TensorFragment *found = nullptr; + for (const auto *fragment : fragments) { + if (fragment->fragment_index() == index) { + found = fragment; + break; + } + } + CHECK(found != nullptr); + if (found == nullptr) { + return {}; + } + CHECK_EQ(found->fragment_count(), + static_cast(fragments.size())); + // Offsets must tile the wire body exactly: no hole, no overlap. + CHECK_EQ(found->byte_offset(), static_cast(body.size())); + body.append(found->payload()); + } + return body; +} + +void CheckFingerprint(const sp::Fingerprint &fingerprint) { + CHECK_EQ(fingerprint.model_artifact_digest(), std::string("sha256:1f0c9d2e")); + CHECK_EQ(fingerprint.runtime_recipe_digest(), std::string("sha256:ab77e410")); + CHECK_EQ(fingerprint.recipe_id(), std::string("llama-gguf-q4km-rocm")); + CHECK_EQ(fingerprint.recipe_version(), std::string("3")); + CHECK_EQ(fingerprint.catalogue_version(), std::string("2026.07.1")); +} + +// The canonical session request, as produced by Python. +void TestSessionRequestVector(const std::string &bytes) { + sp::SessionRequest request; + CHECK(request.ParseFromString(bytes)); + CHECK(request.kind_case() == sp::SessionRequest::kChunk); + + const sp::ActivationChunk &chunk = request.chunk(); + const sp::Envelope &envelope = chunk.envelope(); + + // Every field the protocol promises to carry (acceptance criterion 4). + CHECK_EQ(envelope.schema_version(), sp::SCHEMA_VERSION_1); + CHECK_EQ(envelope.work_id(), std::string("work-7f3a")); + CHECK_EQ(envelope.route_session_id(), std::string("rs-2b91")); + CHECK_EQ(envelope.route_epoch(), 7u); + CheckFingerprint(envelope.fingerprint()); + CHECK_EQ(envelope.shard_range().start_layer(), 12u); + CHECK_EQ(envelope.shard_range().end_layer(), 24u); + // Overlap-safe effective start (ADR-0012): a worker that begins at + // start_layer instead would re-apply layers 12..15. + CHECK_EQ(envelope.shard_range().effective_start_layer(), 16u); + CHECK_EQ(envelope.phase(), sp::PHASE_PREFILL); + CHECK_EQ(envelope.position().first_position(), 256u); + CHECK_EQ(envelope.position().token_count(), 128u); + CHECK_EQ(envelope.idempotency_step(), 42u); + CHECK_EQ(envelope.cache_expectation().mode(), sp::CACHE_MODE_PREFILL); + CHECK_EQ(envelope.cache_expectation().expected_past_len(), 256u); + CHECK_EQ(envelope.deadline_unix_nanos(), 1800000000000000000LL); + CHECK_EQ(envelope.chunk().chunk_index(), 1u); + CHECK_EQ(envelope.chunk().chunk_count(), 3u); + CHECK_EQ(envelope.chunk().final_chunk(), false); + + // The versioned named-tensor bundle (acceptance criterion 5). + const sp::TensorBundle &bundle = chunk.bundle(); + CHECK_EQ(bundle.bundle_version(), 1u); + CHECK_EQ(bundle.tensors_size(), 1); + + const sp::NamedTensor &tensor = bundle.tensors(0); + CHECK_EQ(tensor.name(), std::string("hidden_states")); + CHECK_EQ(tensor.shape_size(), 3); + CHECK_EQ(tensor.shape(0), 1); + CHECK_EQ(tensor.shape(1), 128); + CHECK_EQ(tensor.shape(2), 8); + CHECK_EQ(tensor.dtype(), sp::DTYPE_BFLOAT16); + CHECK_EQ(tensor.byte_order(), sp::BYTE_ORDER_LITTLE_ENDIAN); + CHECK_EQ(tensor.total_bytes(), 1u * 128u * 8u * 2u); + CHECK_EQ(tensor.compression(), sp::COMPRESSION_NONE); + + // Multi-fragment on purpose: reassembly is what a worker actually does. + CHECK(tensor.fragments_size() > 1); + const std::string payload = ReassembleUncompressed(tensor); + CHECK_EQ(payload.size(), static_cast(tensor.total_bytes())); + + // The payload Python generated, recomputed here from its rule. + std::string expected; + expected.reserve(payload.size()); + for (size_t i = 0; i < payload.size(); ++i) { + expected.push_back(static_cast((i * 7 + 11) % 256)); + } + CHECK(payload == expected); + + // Checksum: computed by Python, verified by an independent C++ CRC32C. + CHECK_EQ(tensor.checksum().algorithm(), sp::CHECKSUM_ALGORITHM_CRC32C); + const std::string &checksum = tensor.checksum().value(); + CHECK_EQ(checksum.size(), 4u); + if (checksum.size() == 4) { + const uint32_t actual = Crc32c(payload); + const uint32_t declared = + (static_cast(static_cast(checksum[0])) << 24) | + (static_cast(static_cast(checksum[1])) << 16) | + (static_cast(static_cast(checksum[2])) << 8) | + static_cast(static_cast(checksum[3])); + CHECK_EQ(actual, declared); + } +} + +void TestCapabilityReportVector(const std::string &bytes) { + sp::CapabilityReport report; + CHECK(report.ParseFromString(bytes)); + CHECK_EQ(report.schema_version(), sp::SCHEMA_VERSION_1); + CheckFingerprint(report.fingerprint()); + CHECK_EQ(report.backend(), std::string("rocm")); + CHECK_EQ(report.device(), std::string("gfx1151")); + CHECK_EQ(report.validated(), true); + CHECK_EQ(report.max_concurrent_sessions(), 4u); + CHECK_EQ(report.max_context_tokens(), 8192u); + CHECK_EQ(report.flow_control().max_prefill_chunk_tokens(), 128u); + CHECK_EQ(report.flow_control().max_chunk_bytes(), 4u * 1024u * 1024u); + CHECK_EQ(report.accepted_compression_size(), 2); + CHECK_EQ(report.supported_schema_versions_size(), 1); +} + +// Forward compatibility: a field this build has never heard of must survive a +// parse/serialize cycle untouched. Without this, an old Shard silently strips +// fields a newer peer depends on as it forwards activations down the route. +void TestUnknownFieldsArePreserved(const std::string &bytes) { + // Field 9999, wire type 0 (varint), value 12345 — not in this schema. + std::string forward = bytes; + forward.push_back(static_cast(0xB8)); // tag: (9999 << 3) | 0 + forward.push_back(static_cast(0xE0)); + forward.push_back(static_cast(0x04)); + forward.push_back(static_cast(0xB9)); // value: 12345 + forward.push_back(static_cast(0x60)); + + sp::SessionRequest request; + CHECK(request.ParseFromString(forward)); + // Known fields still parse. + CHECK_EQ(request.chunk().envelope().work_id(), std::string("work-7f3a")); + // And the unknown field is retained rather than dropped. + CHECK(!request.GetReflection()->GetUnknownFields(request).empty()); + + std::string reserialized; + CHECK(request.SerializeToString(&reserialized)); + CHECK_EQ(reserialized.size(), forward.size()); + + sp::SessionRequest reparsed; + CHECK(reparsed.ParseFromString(reserialized)); + CHECK(!reparsed.GetReflection()->GetUnknownFields(reparsed).empty()); +} + +// Backward compatibility: a message from a peer that sets none of the optional +// groups must still parse, yielding proto3 defaults rather than an error. +void TestSparseMessageParses() { + sp::SessionRequest minimal; + minimal.mutable_chunk()->mutable_envelope()->set_work_id("w"); + std::string bytes; + CHECK(minimal.SerializeToString(&bytes)); + + sp::SessionRequest parsed; + CHECK(parsed.ParseFromString(bytes)); + CHECK_EQ(parsed.chunk().envelope().work_id(), std::string("w")); + CHECK_EQ(parsed.chunk().envelope().route_epoch(), 0u); + CHECK_EQ(parsed.chunk().envelope().phase(), sp::PHASE_UNSPECIFIED); + CHECK_EQ(parsed.chunk().bundle().tensors_size(), 0); +} + +// The decode fast path must stay small: repeating the full envelope per token +// is pure overhead on the hottest path in the system. +void TestDecodeFastPathIsSmall() { + sp::SessionRequest decode; + sp::DecodeStep *step = decode.mutable_decode(); + step->set_idempotency_step(9); + step->set_position(1024); + step->set_expected_past_len(1024); + step->set_work_id("work-7f3a"); + sp::NamedTensor *tensor = step->mutable_tensor(); + tensor->set_name("hidden_states"); + tensor->add_shape(1); + tensor->add_shape(1); + tensor->add_shape(8); + tensor->set_dtype(sp::DTYPE_BFLOAT16); + tensor->set_byte_order(sp::BYTE_ORDER_LITTLE_ENDIAN); + tensor->set_total_bytes(16); + tensor->set_compression(sp::COMPRESSION_NONE); + sp::TensorFragment *fragment = tensor->add_fragments(); + fragment->set_fragment_index(0); + fragment->set_fragment_count(1); + fragment->set_byte_offset(0); + fragment->set_payload(std::string(16, '\x01')); + + std::string bytes; + CHECK(decode.SerializeToString(&bytes)); + // Payload is 16 bytes; the framing around it must stay well under the + // envelope-carrying prefill path. + CHECK(bytes.size() < 96); + + sp::SessionRequest parsed; + CHECK(parsed.ParseFromString(bytes)); + CHECK(parsed.kind_case() == sp::SessionRequest::kDecode); + CHECK_EQ(parsed.decode().position(), 1024u); + CHECK_EQ(parsed.decode().tensor().total_bytes(), 16u); +} + +} // namespace + +int main(int argc, char **argv) { + GOOGLE_PROTOBUF_VERIFY_VERSION; + if (argc < 3) { + std::cerr << "usage: " << argv[0] << " \n"; + return 2; + } + const std::filesystem::path testdata(argv[1]); + const std::filesystem::path output(argv[2]); + + const std::string session_bytes = + ReadFile(testdata / "session_request_golden.binpb"); + const std::string capability_bytes = + ReadFile(testdata / "capability_report_golden.binpb"); + + TestSessionRequestVector(session_bytes); + TestCapabilityReportVector(capability_bytes); + TestUnknownFieldsArePreserved(session_bytes); + TestSparseMessageParses(); + TestDecodeFastPathIsSmall(); + + // Re-serialize the canonical message from the C++ object model and hand it + // back for Python to compare against the golden bytes. If the two languages + // disagreed about any field's encoding, this file would differ. + sp::SessionRequest request; + if (request.ParseFromString(session_bytes)) { + std::string reserialized; + if (request.SerializeToString(&reserialized)) { + std::ofstream out(output / "cpp_roundtrip.binpb", std::ios::binary); + out.write(reserialized.data(), + static_cast(reserialized.size())); + } + } + + if (g_failures != 0) { + std::cerr << g_failures << " check(s) failed\n"; + return 1; + } + std::cout << "all C++ conformance checks passed\n"; + return 0; +} diff --git a/packages/node/pyproject.toml b/packages/node/pyproject.toml index 3ba2d56..a11d2cb 100644 --- a/packages/node/pyproject.toml +++ b/packages/node/pyproject.toml @@ -13,6 +13,8 @@ dependencies = [ "huggingface-hub>=0.20", "accelerate>=0.28", "bitsandbytes>=0.43", + "grpcio>=1.60", + "protobuf>=5", "rich>=13", "safetensors>=0.4", "torch>=2.1", @@ -23,6 +25,11 @@ dependencies = [ "kernels>=0.11.1,<0.16", ] +[project.optional-dependencies] +# Regenerating the native Shard protocol stubs. Not needed to run a node: the +# generated modules are committed, and `grpcio-tools` bundles its own protoc. +proto = ["grpcio-tools==1.82.1"] + [project.scripts] meshnet-node = "meshnet_node.cli:main" diff --git a/scripts/bootstrap_native_toolchain.sh b/scripts/bootstrap_native_toolchain.sh new file mode 100644 index 0000000..f6dc846 --- /dev/null +++ b/scripts/bootstrap_native_toolchain.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# Build a protobuf C++ toolchain for the native Shard protocol. +# +# The Python side needs nothing beyond `pip install grpcio-tools` — it bundles +# protoc. The C++ side needs libprotobuf headers and a protoc binary, and a +# machine that has neither (no protobuf-devel, no cmake, no system protoc) can +# still get a working one from source with this script. It is the exact recipe +# DGR-002 used to build and run the C++ conformance test. +# +# gRPC C++ is deliberately NOT built here. The conformance test only needs +# message types, so verifying the schema does not require the whole gRPC stack. +# The worker (DGR-008) will need gRPC C++ and should extend this script then. +# +# Usage: +# scripts/bootstrap_native_toolchain.sh [install-prefix] +# +# Then: +# cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH= +# cmake --build build/native -j +# ctest --test-dir build/native --output-on-failure + +set -euo pipefail + +PREFIX="${1:-${PWD}/build/native-toolchain}" +WORK="$(mktemp -d)" +trap 'rm -rf "${WORK}"' EXIT + +# Pinned: the C++ runtime a generated stub is compiled against must be a version +# that stub is allowed to use, so these are exact, not floating. +PROTOBUF_VERSION="33.1" +ABSEIL_VERSION="20250814.1" + +command -v cmake >/dev/null || { + echo "cmake is required (pip install cmake==4.4.0)" >&2 + exit 1 +} + +echo "--- fetching protobuf ${PROTOBUF_VERSION} and abseil ${ABSEIL_VERSION}" +cd "${WORK}" +curl -sfL -o protobuf.tar.gz \ + "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protobuf-${PROTOBUF_VERSION}.tar.gz" +tar xzf protobuf.tar.gz + +# The protobuf release tarball ships utf8_range but not abseil, and its default +# CMake provider expects abseil as a submodule, so vendor it into place. +curl -sfL -o abseil.tar.gz \ + "https://github.com/abseil/abseil-cpp/releases/download/${ABSEIL_VERSION}/abseil-cpp-${ABSEIL_VERSION}.tar.gz" +tar xzf abseil.tar.gz +rm -rf "protobuf-${PROTOBUF_VERSION}/third_party/abseil-cpp" +mv "abseil-cpp-${ABSEIL_VERSION}" "protobuf-${PROTOBUF_VERSION}/third_party/abseil-cpp" + +echo "--- building protobuf into ${PREFIX}" +cmake -S "protobuf-${PROTOBUF_VERSION}" -B build \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="${PREFIX}" \ + -DCMAKE_POSITION_INDEPENDENT_CODE=ON \ + -Dprotobuf_ABSL_PROVIDER=module \ + -Dprotobuf_BUILD_TESTS=OFF \ + -Dprotobuf_BUILD_SHARED_LIBS=OFF \ + -DABSL_PROPAGATE_CXX_STD=ON +cmake --build build -j"$(nproc)" +cmake --install build + +echo "--- done" +"${PREFIX}/bin/protoc" --version +echo "configure the protocol build with: -DCMAKE_PREFIX_PATH=${PREFIX}" diff --git a/scripts/generate_native_protocol.py b/scripts/generate_native_protocol.py new file mode 100644 index 0000000..6b3154f --- /dev/null +++ b/scripts/generate_native_protocol.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +"""Generate the Python Shard-protocol stubs from `shard_runtime.proto`. + +The `.proto` file is the contract; the Python modules under +`meshnet_node/native_protocol/generated/` are build output that happens to be +committed. They are committed so that installing the node does not require a +protoc toolchain, and `--check` exists so a committed stub can never silently +drift from the schema it claims to implement. + +Usage:: + + python scripts/generate_native_protocol.py # regenerate in place + python scripts/generate_native_protocol.py --check # fail if out of date + +C++ stubs are *not* generated here. They are build artifacts produced by CMake +(`packages/node/native/CMakeLists.txt`) into the build tree, because a C++ build +already requires a toolchain and nothing is gained by committing them. +""" + +from __future__ import annotations + +import argparse +import pathlib +import shutil +import subprocess +import sys +import tempfile + +REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent +PROTO_DIR = REPO_ROOT / "packages/node/native/proto" +PROTO_FILE = PROTO_DIR / "shard_runtime.proto" +OUT_DIR = REPO_ROOT / "packages/node/meshnet_node/native_protocol/generated" + +# Regenerating with a different protoc emits different gencode headers, so the +# generator version is part of the contract and `--check` would catch a drift. +REQUIRED_GRPCIO_TOOLS = "1.82.1" + +_HEADER = "# Generated by scripts/generate_native_protocol.py. Do not edit.\n" + + +def _generate(into: pathlib.Path) -> None: + """Run protoc, writing generated modules into `into`.""" + try: + from grpc_tools import protoc + except ImportError: # pragma: no cover - exercised only without the toolchain + sys.exit( + "grpc_tools is required to generate stubs:\n" + f" pip install grpcio-tools=={REQUIRED_GRPCIO_TOOLS}" + ) + + into.mkdir(parents=True, exist_ok=True) + # grpc_tools bundles protoc and the well-known types, so generation needs no + # system protoc and produces identical output on any machine. + well_known = pathlib.Path(protoc.__file__).parent / "_proto" + args = [ + "protoc", + f"--proto_path={PROTO_DIR}", + f"--proto_path={well_known}", + f"--python_out={into}", + f"--pyi_out={into}", + f"--grpc_python_out={into}", + str(PROTO_FILE), + ] + if protoc.main(args) != 0: + sys.exit("protoc failed") + + # protoc emits `import shard_runtime_pb2` — a bare top-level import that only + # resolves if the generated directory happens to be on sys.path. Rewrite it + # to a relative import so the package works as an installed package. + grpc_module = into / "shard_runtime_pb2_grpc.py" + text = grpc_module.read_text() + text = text.replace( + "import shard_runtime_pb2 as shard__runtime__pb2", + "from . import shard_runtime_pb2 as shard__runtime__pb2", + ) + grpc_module.write_text(text) + + (into / "__init__.py").write_text( + _HEADER + '"""Generated protobuf/gRPC stubs for the native Shard protocol."""\n' + ) + + +def _tracked_files(directory: pathlib.Path) -> dict[str, bytes]: + return { + path.name: path.read_bytes() + for path in sorted(directory.iterdir()) + if path.is_file() and path.suffix in {".py", ".pyi"} + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--check", + action="store_true", + help="verify the committed stubs match the .proto instead of rewriting them", + ) + args = parser.parse_args() + + if args.check: + with tempfile.TemporaryDirectory() as tmp: + fresh = pathlib.Path(tmp) / "generated" + _generate(fresh) + if not OUT_DIR.is_dir(): + print(f"generated stubs are missing: {OUT_DIR}", file=sys.stderr) + return 1 + if _tracked_files(fresh) != _tracked_files(OUT_DIR): + print( + "generated stubs are out of date with shard_runtime.proto.\n" + "Run: python scripts/generate_native_protocol.py", + file=sys.stderr, + ) + return 1 + print("generated stubs are up to date") + return 0 + + if OUT_DIR.exists(): + shutil.rmtree(OUT_DIR) + _generate(OUT_DIR) + print(f"wrote {OUT_DIR.relative_to(REPO_ROOT)}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/generate_protocol_goldens.py b/scripts/generate_protocol_goldens.py new file mode 100644 index 0000000..1936d16 --- /dev/null +++ b/scripts/generate_protocol_goldens.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +"""Write the committed cross-language conformance vectors. + +The bytes under `packages/node/native/testdata/` are the reference both the +Python and the C++ conformance tests assert against. They are committed so the +C++ test can run without a Python step, and `--check` exists so they can never +drift from the schema unnoticed: if a schema edit changes the canonical +message's encoding, `--check` fails and the change has to be acknowledged. + +Usage:: + + python scripts/generate_protocol_goldens.py # rewrite vectors + python scripts/generate_protocol_goldens.py --check # fail if stale +""" + +from __future__ import annotations + +import argparse +import pathlib +import sys + +REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent +sys.path.insert(0, str(REPO_ROOT / "packages/node")) + +from meshnet_node.native_protocol import conformance # noqa: E402 + + +def _vectors() -> dict[str, bytes]: + return { + conformance.GOLDEN_SESSION_REQUEST: conformance.serialize( + conformance.canonical_session_request() + ), + conformance.GOLDEN_CAPABILITY_REPORT: conformance.serialize( + conformance.canonical_capability_report() + ), + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--check", action="store_true") + args = parser.parse_args() + + out_dir = conformance.TESTDATA_DIR + out_dir.mkdir(parents=True, exist_ok=True) + stale = [] + + for name, payload in _vectors().items(): + path = out_dir / name + if args.check: + if not path.is_file() or path.read_bytes() != payload: + stale.append(name) + continue + path.write_bytes(payload) + print(f"wrote {path.relative_to(REPO_ROOT)} ({len(payload)} bytes)") + + if stale: + print( + "conformance vectors are stale: " + ", ".join(stale) + "\n" + "The canonical message no longer encodes to the committed bytes. If " + "that is intended, run: python scripts/generate_protocol_goldens.py", + file=sys.stderr, + ) + return 1 + if args.check: + print("conformance vectors are up to date") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_native_shard_protocol.py b/tests/test_native_shard_protocol.py new file mode 100644 index 0000000..ab36e3e --- /dev/null +++ b/tests/test_native_shard_protocol.py @@ -0,0 +1,412 @@ +"""Conformance tests for the native Shard protocol (ADR-0020, DGR-002). + +Three layers are tested, and they are not the same thing: + +1. The *schema* — asserted against the descriptor, not against the Python + helpers. If a field the protocol promises to carry were dropped from the + `.proto`, a test that only exercised the codec would still pass. +2. The *codec* — that a payload which is corrupt, short, holed, or byte-swapped + is rejected rather than fed to a forward pass. +3. *Compatibility* — that an old build preserves fields a newer peer added, and + that the committed cross-language vectors still encode as promised. +""" + +from __future__ import annotations + +import pathlib +import subprocess +import sys + +import pytest + +pytest.importorskip("google.protobuf", reason="protobuf runtime is required") + +from google.protobuf import descriptor_pb2 + +from meshnet_node.activation_compression import CompressionPolicy +from meshnet_node.native_protocol import ( + DEFAULT_MAX_CHUNK_BYTES, + DEFAULT_MAX_PREFILL_CHUNK_TOKENS, + HIDDEN_STATES, + PayloadCorrupt, + ProtocolError, + checksum_of, + decode_bundle, + decode_tensor, + default_flow_control, + encode_bundle, + encode_tensor, + negotiate_flow_control, + pb, + plan_prefill_chunks, +) +from meshnet_node.native_protocol import conformance + +REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent +CPP_ROUNDTRIP = REPO_ROOT / "build/native" / conformance.CPP_ROUNDTRIP + + +# --- The schema itself ------------------------------------------------------ + + +def test_service_exposes_capability_health_session_release_and_cancel(): + service = pb.DESCRIPTOR.services_by_name["ShardRuntime"] + assert set(service.methods_by_name) == { + "GetCapability", + "Health", + "Session", + "Release", + "Cancel", + } + + +def test_session_is_one_long_lived_bidirectional_stream(): + session = pb.DESCRIPTOR.services_by_name["ShardRuntime"].methods_by_name["Session"] + assert session.client_streaming, "the seam must stream requests" + assert session.server_streaming, "the seam must stream responses" + + # Cancellation must not have to travel down a stream that flow control has + # wedged, so it also exists as its own unary call. + cancel = pb.DESCRIPTOR.services_by_name["ShardRuntime"].methods_by_name["Cancel"] + assert not cancel.client_streaming and not cancel.server_streaming + + +def test_envelope_carries_every_field_the_protocol_promises(): + # Asserted against the descriptor: this is the acceptance criterion, and it + # must fail if the .proto drops a field, not merely if the codec stops + # setting one. + fields = set(pb.Envelope.DESCRIPTOR.fields_by_name) + assert { + "schema_version", + "work_id", + "route_session_id", + "route_epoch", + "fingerprint", + "shard_range", + "phase", + "position", + "idempotency_step", + "cache_expectation", + "deadline_unix_nanos", + "chunk", + } <= fields + + assert {"model_artifact_digest", "runtime_recipe_digest"} <= set( + pb.Fingerprint.DESCRIPTOR.fields_by_name + ) + # Overlap-safe start (ADR-0012) is a distinct field from the registered one. + assert {"start_layer", "end_layer", "effective_start_layer"} <= set( + pb.ShardRange.DESCRIPTOR.fields_by_name + ) + assert {"mode", "expected_past_len"} <= set( + pb.CacheExpectation.DESCRIPTOR.fields_by_name + ) + + +def test_named_tensor_bundle_is_versioned_and_fully_described(): + assert "bundle_version" in pb.TensorBundle.DESCRIPTOR.fields_by_name + assert {"name", "shape", "dtype", "byte_order", "compression", "checksum", "fragments"} <= set( + pb.NamedTensor.DESCRIPTOR.fields_by_name + ) + + +def test_phases_and_error_codes_cover_the_lifecycle(): + assert {"PHASE_PREFILL", "PHASE_DECODE", "PHASE_RELEASE", "PHASE_CANCEL"} <= set( + pb.Phase.keys() + ) + # A cache miss is a first-class, recoverable outcome (ADR-0022), not a crash. + assert { + "ERROR_CODE_CACHE_MISS", + "ERROR_CODE_FINGERPRINT_MISMATCH", + "ERROR_CODE_EPOCH_STALE", + "ERROR_CODE_SCHEMA_UNSUPPORTED", + "ERROR_CODE_DEADLINE_EXCEEDED", + "ERROR_CODE_FLOW_CONTROL_VIOLATION", + } <= set(pb.ErrorCode.keys()) + + +# --- Tensor bundle round-trip ---------------------------------------------- + + +def test_tensor_round_trips_through_fragments(): + payload = bytes(range(256)) * 4 + tensor = encode_tensor( + HIDDEN_STATES, payload, [1, 64, 8], pb.DTYPE_BFLOAT16, max_fragment_bytes=100 + ) + + assert len(tensor.fragments) > 1, "the bound must actually split the payload" + assert tensor.total_bytes == len(payload) + assert decode_tensor(tensor) == payload + + +def test_bundle_round_trips_multiple_named_tensors(): + # An architecture boundary may need more than one tensor; that is why the + # payload is a named bundle rather than a bare buffer. + hidden = encode_tensor(HIDDEN_STATES, b"\x01\x02" * 8, [1, 8, 1], pb.DTYPE_BFLOAT16) + positions = encode_tensor( + "position_ids", (7).to_bytes(4, "little") * 8, [8], pb.DTYPE_INT32 + ) + bundle = encode_bundle([hidden, positions]) + + restored = decode_bundle(pb.TensorBundle.FromString(bundle.SerializeToString())) + assert restored == { + HIDDEN_STATES: b"\x01\x02" * 8, + "position_ids": (7).to_bytes(4, "little") * 8, + } + + +def test_compressed_tensor_round_trips_and_keeps_its_uncompressed_checksum(): + pytest.importorskip("zstandard") + # Highly compressible, and over the policy's minimum input size. + payload = b"\x00" * 65536 + always = CompressionPolicy(min_input_bytes=0, min_savings_bytes=0, min_savings_ratio=0.0) + tensor = encode_tensor( + HIDDEN_STATES, payload, [1, 4096, 8], pb.DTYPE_BFLOAT16, policy=always + ) + + assert tensor.compression == pb.COMPRESSION_ZSTD + assert sum(len(f.payload) for f in tensor.fragments) < len(payload) + # The checksum covers the uncompressed bytes, so it stays valid whether or + # not a hop chose to compress. + assert tensor.checksum == checksum_of(payload) + assert decode_tensor(tensor) == payload + + +# --- The codec refuses what it cannot account for -------------------------- + + +def test_corrupt_payload_is_rejected_by_checksum(): + tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16) + # Flip one byte, as a lossy relay or a bad NIC would. + tensor.fragments[0].payload = b"\xab" + tensor.fragments[0].payload[1:] + + with pytest.raises(PayloadCorrupt, match="CRC32C"): + decode_tensor(tensor) + + +def test_missing_fragment_is_rejected_rather_than_silently_truncated(): + tensor = encode_tensor( + HIDDEN_STATES, b"\xaa" * 64, [1, 32, 1], pb.DTYPE_BFLOAT16, max_fragment_bytes=16 + ) + del tensor.fragments[1] + + with pytest.raises(PayloadCorrupt): + decode_tensor(tensor) + + +def test_fragment_hole_is_rejected(): + tensor = encode_tensor( + HIDDEN_STATES, b"\xaa" * 64, [1, 32, 1], pb.DTYPE_BFLOAT16, max_fragment_bytes=16 + ) + # A gap in coverage: offsets no longer tile the body. + tensor.fragments[2].byte_offset += 4 + + with pytest.raises(PayloadCorrupt, match="expected"): + decode_tensor(tensor) + + +def test_shape_that_disagrees_with_payload_is_rejected_at_encode(): + with pytest.raises(ProtocolError, match="carries"): + encode_tensor(HIDDEN_STATES, b"\x01\x02", [1, 8, 8], pb.DTYPE_BFLOAT16) + + +def test_shape_that_disagrees_with_declared_bytes_is_rejected_at_decode(): + tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16) + # A peer claiming a larger tensor than its bytes describe. + tensor.shape[1] = 32 + + with pytest.raises(PayloadCorrupt, match="implies"): + decode_tensor(tensor) + + +def test_big_endian_tensor_is_rejected_loudly(): + tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16) + tensor.byte_order = pb.BYTE_ORDER_BIG_ENDIAN + + # Byte-swapped activations would be plausible-looking garbage, so this is an + # error rather than a best-effort read. + with pytest.raises(ProtocolError, match="big-endian"): + decode_tensor(tensor) + + +def test_bundle_from_a_newer_layout_is_refused(): + bundle = encode_bundle([]) + bundle.bundle_version = 99 + + with pytest.raises(ProtocolError, match="newer"): + decode_bundle(bundle) + + +# --- Bounded prefill chunking and the decode fast path ---------------------- + + +def test_prefill_is_split_into_bounded_token_aligned_chunks(): + chunks = plan_prefill_chunks(2048) + + assert all(c.token_count <= DEFAULT_MAX_PREFILL_CHUNK_TOKENS for c in chunks) + assert sum(c.token_count for c in chunks) == 2048 + # Contiguous, token-aligned, and no split falls mid-token. + assert [c.first_position for c in chunks] == [ + i * DEFAULT_MAX_PREFILL_CHUNK_TOKENS for i in range(len(chunks)) + ] + assert [c.final_chunk for c in chunks] == [False] * (len(chunks) - 1) + [True] + + +def test_final_prefill_chunk_carries_the_remainder(): + chunks = plan_prefill_chunks(300, max_tokens=128) + + assert [c.token_count for c in chunks] == [128, 128, 44] + assert chunks[-1].chunk_info().final_chunk + assert chunks[0].position().first_position == 0 + assert chunks[-1].position().first_position == 256 + + +def test_empty_prefill_is_refused(): + with pytest.raises(ProtocolError): + plan_prefill_chunks(0) + + +def test_decode_fast_path_is_much_smaller_than_a_full_envelope_chunk(): + hidden = b"\x01\x02" * 8 # one token, hidden=8, bfloat16 + tensor = encode_tensor(HIDDEN_STATES, hidden, [1, 1, 8], pb.DTYPE_BFLOAT16) + + fast = pb.SessionRequest( + decode=pb.DecodeStep( + idempotency_step=9, + position=1024, + expected_past_len=1024, + tensor=tensor, + work_id="work-7f3a", + ) + ) + # The same single token carried the long way, repeating identity that the + # handshake already fixed for the life of the stream. + full = pb.SessionRequest( + chunk=pb.ActivationChunk( + envelope=conformance.canonical_session_request().chunk.envelope, + bundle=encode_bundle([tensor]), + ) + ) + + assert len(fast.SerializeToString()) * 2 < len(full.SerializeToString()) + assert decode_tensor(fast.decode.tensor) == hidden + + +def test_flow_control_defaults_bound_the_queue_and_the_message(): + limits = default_flow_control() + + assert limits.max_prefill_chunk_tokens == DEFAULT_MAX_PREFILL_CHUNK_TOKENS + assert limits.max_chunk_bytes == DEFAULT_MAX_CHUNK_BYTES + assert limits.max_inflight_chunks > 0 + + +def test_flow_control_negotiation_takes_the_strictest_bound(): + proposed = pb.FlowControl( + credits_granted=64, max_inflight_chunks=64, max_chunk_bytes=64 << 20, + max_prefill_chunk_tokens=1024, + ) + settled = negotiate_flow_control(proposed, default_flow_control()) + + # A sender cannot talk a worker into unbounded queues by proposing a large + # window: neither peer can raise the other's ceiling. + assert settled.max_inflight_chunks == default_flow_control().max_inflight_chunks + assert settled.max_chunk_bytes == DEFAULT_MAX_CHUNK_BYTES + assert settled.max_prefill_chunk_tokens == DEFAULT_MAX_PREFILL_CHUNK_TOKENS + + +# --- Compatibility ---------------------------------------------------------- + + +def test_committed_vectors_still_encode_as_promised(): + # The C++ test asserts against these exact bytes. If a schema change alters + # the canonical encoding, it must be acknowledged by regenerating them. + golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes() + assert conformance.serialize(conformance.canonical_session_request()) == golden + + report = (conformance.TESTDATA_DIR / conformance.GOLDEN_CAPABILITY_REPORT).read_bytes() + assert conformance.serialize(conformance.canonical_capability_report()) == report + + +def test_golden_session_request_round_trips_with_every_field_intact(): + golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes() + request = pb.SessionRequest.FromString(golden) + envelope = request.chunk.envelope + + assert envelope.work_id == conformance.WORK_ID + assert envelope.route_session_id == conformance.ROUTE_SESSION_ID + assert envelope.route_epoch == conformance.ROUTE_EPOCH + assert envelope.idempotency_step == conformance.IDEMPOTENCY_STEP + assert envelope.shard_range.effective_start_layer == conformance.EFFECTIVE_START_LAYER + assert envelope.phase == pb.PHASE_PREFILL + assert envelope.cache_expectation.mode == pb.CACHE_MODE_PREFILL + assert envelope.deadline_unix_nanos == conformance.DEADLINE_UNIX_NANOS + assert decode_bundle(request.chunk.bundle) == { + HIDDEN_STATES: conformance.canonical_payload() + } + assert request.SerializeToString(deterministic=True) == golden + + +def test_unknown_fields_from_a_newer_peer_survive_a_forwarding_hop(): + # A Shard forwards activations onward. If it silently dropped fields a newer + # peer added, it would corrupt a route it is merely a waypoint on. + golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes() + future_field = b"\xb8\xe0\x04\xb9\x60" # field 9999, varint 12345 + request = pb.SessionRequest.FromString(golden + future_field) + + assert request.chunk.envelope.work_id == conformance.WORK_ID + assert request.SerializeToString() == golden + future_field + + +def test_a_message_missing_newer_field_groups_still_parses(): + sparse = pb.SessionRequest(chunk=pb.ActivationChunk(envelope=pb.Envelope(work_id="w"))) + parsed = pb.SessionRequest.FromString(sparse.SerializeToString()) + + assert parsed.chunk.envelope.work_id == "w" + assert parsed.chunk.envelope.route_epoch == 0 + assert parsed.chunk.envelope.phase == pb.PHASE_UNSPECIFIED + + +def test_retired_fragment_field_stays_reserved(): + # `uncompressed_size` (field 5) was removed because NamedTensor.total_bytes + # is the single source of truth. The number stays reserved so it can never + # be recycled for a different meaning — a recycled number is the one schema + # change that old and new peers cannot detect, because the bytes still parse. + descriptor = descriptor_pb2.DescriptorProto() + pb.TensorFragment.DESCRIPTOR.CopyToProto(descriptor) + + assert 5 not in {field.number for field in descriptor.field} + assert any(r.start <= 5 < r.end for r in descriptor.reserved_range) + assert "uncompressed_size" in descriptor.reserved_name + + +def test_a_peer_still_sending_the_retired_field_does_not_corrupt_the_tensor(): + # An older peer that still sets field 5 must be parsed, not rejected: the + # value lands in unknown fields and the payload is unaffected. + tensor = encode_tensor(HIDDEN_STATES, b"\xaa" * 32, [1, 16, 1], pb.DTYPE_BFLOAT16) + wire = tensor.SerializeToString() + fragment = pb.TensorFragment.FromString(tensor.fragments[0].SerializeToString() + b"\x28\x20") + + assert fragment.payload == tensor.fragments[0].payload + assert decode_tensor(pb.NamedTensor.FromString(wire)) == b"\xaa" * 32 + + +def test_generated_python_stubs_match_the_proto(): + pytest.importorskip("grpc_tools", reason="protoc toolchain is required to verify") + result = subprocess.run( + [sys.executable, str(REPO_ROOT / "scripts/generate_native_protocol.py"), "--check"], + capture_output=True, + text=True, + ) + assert result.returncode == 0, result.stdout + result.stderr + + +@pytest.mark.skipif( + not CPP_ROUNDTRIP.is_file(), + reason="build the C++ conformance test to check cross-language agreement", +) +def test_cpp_and_python_agree_byte_for_byte(): + # Written by the C++ conformance test: it parsed the golden bytes into its + # own object model and serialized them back. Byte equality means both + # languages encode every field of this schema identically. + golden = (conformance.TESTDATA_DIR / conformance.GOLDEN_SESSION_REQUEST).read_bytes() + assert CPP_ROUNDTRIP.read_bytes() == golden