28 Commits

Author SHA1 Message Date
Dobromir Popov
f0f9a0eed7 controller: record DGR-043 completion 2026-08-01 01:58:14 +03:00
Dobromir Popov
e6ad9fdca9 story: DGR-043 Expose GGUF compatibility and measured cost inputs to existing routing 2026-08-01 01:58:13 +03:00
Dobromir Popov
d53acb1145 controller: record DGR-042 completion 2026-08-01 01:52:43 +03:00
Dobromir Popov
fd10607033 story: DGR-042 Carry native frames through direct and existing relay seams 2026-08-01 01:52:42 +03:00
Dobromir Popov
eb986ddf10 controller: record DGR-041 completion 2026-08-01 01:47:29 +03:00
Dobromir Popov
f37c4352fe story: DGR-041 Register native Shard capabilities without redesigning Meshnet 2026-08-01 01:47:28 +03:00
Dobromir Popov
95f005f646 controller: record DGR-040 completion 2026-08-01 01:41:52 +03:00
Dobromir Popov
520ccb8266 story: DGR-040 Add node-side native worker supervision 2026-08-01 01:41:51 +03:00
Dobromir Popov
f4980491d2 controller: record DGR-039 completion 2026-08-01 01:35:16 +03:00
Dobromir Popov
3a67eea569 story: DGR-039 Pass local two-process dense acceptance 2026-08-01 01:35:14 +03:00
Dobromir Popov
4c6c78d837 controller: record DGR-038 completion 2026-08-01 01:32:48 +03:00
Dobromir Popov
49560b396f story: DGR-038 Implement isolated shard-local Hot KV State 2026-08-01 01:32:46 +03:00
Dobromir Popov
a1df87deb6 controller: record DGR-037 completion 2026-08-01 01:28:07 +03:00
Dobromir Popov
8217b4c4a2 story: DGR-037 Bind llama.cpp to the standalone worker 2026-08-01 01:28:06 +03:00
Dobromir Popov
dfa403adc6 controller: record DGR-036 completion 2026-08-01 01:20:17 +03:00
Dobromir Popov
6e8bf7a64d story: DGR-036 Prove dense fixture and real-model range parity 2026-08-01 01:20:16 +03:00
Dobromir Popov
6e88b3bd8f controller: record DGR-035 completion 2026-08-01 01:13:52 +03:00
Dobromir Popov
64c2046e5a story: DGR-035 Implement dense architecture boundary input/output 2026-08-01 01:13:50 +03:00
Dobromir Popov
79c9bbaf63 controller: record DGR-034 completion 2026-08-01 01:08:29 +03:00
Dobromir Popov
d339cfde25 story: DGR-034 Implement dense-Llama range-aware GGUF ownership 2026-08-01 01:08:28 +03:00
Dobromir Popov
27a0d89678 docs: align PRD source-of-truth status 2026-07-27 08:58:51 +03:00
Dobromir Popov
8c87fae1ac chore: restore canonical PRD metadata and projections 2026-07-27 08:53:16 +03:00
Dobromir Popov
4d530d702c chore: restore canonical DGR-033 metadata projection 2026-07-26 23:04:45 +03:00
Dobromir Popov
7473bb7e44 fix: DGR-033 repair native worker protocol per cross-review BLOCK
Address the Codex GPT-5.5 review of the standalone fake C++ gRPC Shard
worker. Four root protocol defects fixed:

- Fail closed before SessionOpen: a per-session `opened` flag gates
  chunk/decode so no activation bypasses lifecycle, cancellation, epoch
  or flow-control state (terminal ERROR_CODE_INTERNAL), even when an
  out-of-band Cancel created placeholder state.
- Strict flow-control negotiation: NegotiateFlow takes the strictest of
  peer-vs-worker bounds (mirrors codec.negotiate_flow_control) and the
  negotiated per-session max_chunk_bytes is enforced on every bundle
  instead of trusting the peer proposal.
- In-stream ReleaseSignal now erases session state immediately.
- SessionOpen rejects incompatible schema, fingerprint, and shard-range
  identity and reports the worker's own served fingerprint rather than
  echoing the caller.

Adds 9 regression tests (worker suite 18 -> 27). Real gates on the
rebuilt pinned-gRPC binary: cmake build exit 0; ctest 2/2; worker
pytest 27 passed; harness+protocol 63 passed; compileall 0; diff --check
clean; ldd/nm show 0 llama/ggml linkage. DGR-033 passes -> true.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 22:57:03 +03:00
Dobromir Popov
c073826374 chore: reblock DGR-033 after protocol review 2026-07-26 22:34:34 +03:00
Dobromir Popov
0c7d475335 chore: remove Ralph lane runtime artifacts 2026-07-26 22:33:37 +03:00
Dobromir Popov
84d75f4cd2 controller: record DGR-033 completion 2026-07-25 22:38:01 +03:00
Dobromir Popov
766e480ba5 story: DGR-033 Build a standalone fake C++ gRPC Shard worker 2026-07-25 22:38:00 +03:00
68 changed files with 7016 additions and 138 deletions

1
.gitignore vendored
View File

@@ -12,6 +12,7 @@ dist/
# Ralph local runtime state # Ralph local runtime state
.ralph-tui/* .ralph-tui/*
!.ralph-tui/config.toml !.ralph-tui/config.toml
.ralph-lane/
.env .env

View File

@@ -1,6 +1,6 @@
# Distributed GGUF Runtime planning workspace # Distributed GGUF Runtime planning workspace
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative. > **Implementation status:** DGR-017 through DGR-033 have verified lane evidence, including a fixture-only standalone C++ gRPC worker. These lane checkpoints still require serialized integration and remote publication; they do not claim real model inference. `prd.json` is authoritative.
## Locked scope ## Locked scope

View File

@@ -0,0 +1,281 @@
# DGR-033 evidence — standalone fake C++ gRPC Shard worker
**Completed:** 2026-07-25 (initial); **repaired:** 2026-07-26 after Codex
GPT-5.5 cross-review BLOCK (see "Cross-review repair" below).
**Branch:** `ralph/distributed-gguf-opus`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependencies:** DGR-022 (lifecycle/status contract), DGR-024 (real generated
gRPC harness + `shard_runtime_server.py` reference semantics), DGR-032
(deterministic fake `ShardEngine` semantics).
## Objective
Prove the standalone worker process, stream, lifecycle, and supervision shape
before any llama.cpp integration: a real C++ executable that serves the whole
ShardRuntime lifecycle/stream contract over gRPC using a model-free fake engine,
driven end-to-end by Python integration tests over a real socket.
## What was found live before changing code
- `packages/node/native/proto/shard_runtime.proto` (DGR-021..023): the single
semantic contract. Its `ShardRuntime` service has exactly five RPCs —
`GetCapability`, `Health`, `Session` (bidi stream), `Release`, `Cancel`.
- `packages/node/meshnet_node/shard_runtime_server.py` (DGR-024): the reference
Python servicer. It performs a *bounded real forward* (a CRC over the received
bundle bytes) then echoes the chunk, and fails closed on stale epoch, expired
deadline, corrupt/mis-tiled fragments, exhausted flow-control credit, duplicate
idempotency step, and in-band/out-of-band cancellation, with per-`route_session_id`
state kept on the servicer so an out-of-band `Cancel` can reach a live session.
**Key finding:** despite the schema labelling the checksum `CRC32C`, this
runtime computes it with `zlib.crc32` (standard CRC-32, *not* Castagnoli). The
C++ worker mirrors `zlib.crc32` exactly so its checksum acceptance is
byte-identical to the existing Python surface (the committed C++ *conformance*
test, by contrast, uses true Castagnoli against separately-generated goldens —
the two are unrelated code paths).
- `packages/node/native/CMakeLists.txt` (DGR-029/030): configures against the
ignored `build/native-toolchain` prefix (pinned Protobuf 33.1 + gRPC 1.82.1),
always generates both message and service stubs, and registers a C++
conformance CTest. There was **no** worker executable and **no** Python
worker integration test before this story (confirmed by
`ls packages/node/native/worker` → absent, and grep for `shard_worker`).
- `packages/node/meshnet_node/fake_shard_engine.py` (DGR-032): the Python fake
engine, deliberately *not* wired into the gRPC surface. DGR-033's worker is
its native analogue — a separate executable, not a consumer of that module —
so both fakes present identical behaviour to a client (deterministic,
model-free bounded forward; per-session isolation; fail-closed lifecycle).
## What was added (this story's change)
### `packages/node/native/worker/fake_engine.h` (new)
`meshnet::worker::FakeShardEngine` — a header-only, model-free fixture engine.
Its only capability is to validate a `TensorBundle` (fragments tile exactly, the
uncompressed CRC-32 matches the declared checksum, the declared payload stays
within the negotiated `max_chunk_bytes`) and fold the fragment bytes through a
bounded forward. It links, loads, and dispatches to **nothing** — no llama.cpp,
no graph execution. Carries `kEvidenceClass = "fixture"` mirroring the Python
`FakeShardEngine.EVIDENCE_CLASS` for the later DGR-036 parity check.
### `packages/node/native/worker/shard_service.{h,cpp}` (new)
`ShardRuntimeServiceImpl : meshnet::shard::v1::ShardRuntime::Service` — a faithful
C++ port of the DGR-024 Python servicer: the same per-`route_session_id`
identity/credit/dedup state guarded by a mutex, the same fail-closed negative
paths, and the same lifecycle (open → prefill/decode → flow-control top-up →
release/cancel). Each per-request response is computed under the lock and written
*after* releasing it, so a blocking `Write` can never deadlock the out-of-band
`Cancel` RPC that needs the same lock. Bounded messages are enforced two ways: a
per-tensor `RESOURCE_EXHAUSTED` app check against `max_chunk_bytes`, plus a hard
transport receive ceiling.
### `packages/node/native/worker/shard_worker_main.cpp` (new)
The standalone `shard_worker` executable. Binds `MESHNET_SHARD_LISTEN_ADDR`
(or an `argv` address), prints one readiness line (`ShardRuntime worker listening
on <addr>`), and serves until `SIGTERM`/`SIGINT`. **Graceful shutdown** uses a
self-pipe: the async-signal-safe handler writes one byte, a drain thread reads it
and calls `server->Shutdown()`, so in-flight sessions finish and the process
exits `0` printing `ShardRuntime worker shut down cleanly`. A `--selftest` mode
binds an ephemeral port and self-drives capability/health/fragmented-prefill/
decode/release over a real loopback gRPC channel, giving a pure-C++ CTest that
needs no Python.
### `packages/node/native/CMakeLists.txt` (modified)
Adds the `shard_worker` executable (linking only `shard_runtime_grpc` +
`gRPC::grpc++` — no llama.cpp) and registers `shard_worker_selftest` as a CTest.
### `tests/test_native_shard_worker.py` (new)
18 integration tests that spawn the **real compiled binary** as a subprocess and
drive it with the committed generated stubs over a real localhost socket. When
the binary is not built they skip (the DGR-029/030 `requires_cmake` gating
pattern), locating it via `MESHNET_SHARD_WORKER_BIN` or `build/native/shard_worker`.
## Acceptance criteria → evidence
1. **Standalone C++ executable serves the complete lifecycle/stream contract
using the fake engine** — `shard_worker` builds and serves all five RPCs; the
`shard_worker_selftest` CTest drives open → fragmented prefill → decode →
release over real gRPC; the 18 Python tests cover the same against the
subprocess.
2. **Python integration tests cover startup, health, capability, fragmented
prefill, decode, release, cancellation, graceful shutdown** —
`test_worker_startup_and_health`, `test_worker_capability`,
`test_fragmented_prefill_echoes_reassembled_payload` (3-fragment tiling),
`test_decode_step_is_served`, `test_release_is_terminal`,
`test_in_band_cancel_of_single_work_item_does_not_end_stream`,
`test_in_band_cancel_of_whole_session_is_terminal`,
`test_out_of_band_cancel_rpc_races_ahead_of_open`,
`test_graceful_shutdown_on_sigterm` (SIGTERM → exit 0 + clean-shutdown line).
3. **Bounded messages, deadlines, flow control, independent session
cancellation enforced** — `test_bounded_message_is_rejected`
(`RESOURCE_EXHAUSTED` on an over-ceiling tensor),
`test_expired_deadline_is_rejected`, `test_flow_control_violation_and_topup`,
`test_independent_session_cancellation` (cancelling session A leaves session B
fully serviceable), plus `test_stale_route_epoch_is_rejected`,
`test_duplicate_idempotency_step_is_acked`,
`test_malformed_fragment_tiling_is_rejected`.
4. **Exposes neither llama.cpp RPC nor arbitrary graph execution**
`ldd build/native/shard_worker` shows no llama/ggml shared libs;
`nm -C build/native/shard_worker | grep -icE 'llama_|ggml_'``0`; the proto
exposes exactly one service with five lifecycle RPCs and no graph-exec entry.
5. **Gates + this handoff** — below.
## Commands and results
Toolchain (ignored `build/native-toolchain`, pinned Protobuf 33.1 + gRPC 1.82.1):
```bash
bash scripts/bootstrap_native_toolchain.sh "$PWD/build/native-toolchain"
# ... gRPC 1.82.1 commit acccf84c0df20487d64101f528e5d426541ca4e5
# grpc_cpp_plugin sha256 43705cf26ae9ce98bbcee76b3408f5e171eec746b50bf0dd42dd68d132c6a533
```
Focused out-of-tree CMake build + CTest:
```bash
cmake -S packages/node/native -B build/native -DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain"
cmake --build build/native -j"$(nproc)"
ctest --test-dir build/native --output-on-failure
```
```text
1/2 Test #1: shard_worker_selftest ............ Passed 0.01 sec
2/2 Test #2: shard_protocol_conformance ....... Passed 0.00 sec
100% tests passed out of 2
```
Python integration tests against the real binary:
```bash
PYTHONPATH=packages/node:packages/tracker python -m pytest -q tests/test_native_shard_worker.py
```
```text
18 passed in 3.96s
```
AC4 (no llama.cpp / no graph exec):
```bash
ldd build/native/shard_worker | grep -iE 'llama|ggml' # -> (no matches)
nm build/native/shard_worker | grep -icE 'llama_|ggml_' # -> 0
```
Shared gates + regression:
```bash
python -m compileall -q packages tests # exit 0
git diff --check -- packages/node/native tests/test_native_shard_worker.py # exit 0
PYTHONPATH=packages/node:packages/tracker python -m pytest -q \
tests/test_shard_runtime_harness.py tests/test_native_shard_protocol.py
# -> 61 passed, 2 skipped (DGR-024 harness + native protocol untouched)
```
Toolchain used: `cmake`/`ctest` from the `distributed-gguf-runtime` worktree's
`.venv` (PyPI `cmake==4.4.0` wheel — no system cmake exists here, same as
DGR-029/030); the Python client uses that venv's `grpcio==1.82.1`,
`grpcio-tools==1.82.1`, `protobuf`, `pytest`. `g++ (GCC) 15.2.1`.
## Limitations
- This is FIXTURE evidence only. The worker's "forward" is a CRC-over-wire-bytes
echo, not real tensor compute; it proves process/stream/lifecycle/supervision
shape, nothing about numerical correctness. Real engine binding is DGR-037 and
numeric parity is DGR-036/052.
- The worker checksum path mirrors the DGR-024 runtime's `zlib.crc32` (standard
CRC-32 under a `CRC32C` label). Compressed-tensor tiling/checksum is not
independently verified (no zstd decompressor in the fixture) — identical to the
DGR-024 limitation.
- Default `pytest` runs skip `tests/test_native_shard_worker.py` unless the
worker binary is built (or `MESHNET_SHARD_WORKER_BIN` is set); this session
built it and ran all 18 for real (results above). Building requires the pinned
gRPC C++ toolchain, which is not present by default and must be bootstrapped.
- No CUDA/ROCm/GPU, no model download, no network at test time — all default
tests are fixture-only and offline.
## Dependency handoff
- **DGR-036** (fixture vs real-model parity): the worker's `FakeShardEngine`
carries `kEvidenceClass = "fixture"`; diff it against DGR-037's real engine's
equivalent marker, and reuse the same lifecycle/stream contract this worker
serves to prove behavioural parity before numeric parity.
- **DGR-037** (bind llama.cpp): replace `FakeShardEngine`'s bounded forward with
the real engine behind the *same* `ShardRuntimeServiceImpl` surface; the
service's session/epoch/credit/dedup/cancel machinery and the graceful-shutdown
supervision shape are reusable as-is.
- **DGR-040** (worker supervision): `shard_worker` already provides the
supervision primitives — a readiness line for start detection, `SIGTERM`
graceful drain with a clean-exit line, and a `--selftest` liveness probe.
A supervisor can start/monitor/restart the process around these.
## Cross-review repair (2026-07-26)
An independent Codex GPT-5.5 review BLOCKED the initial implementation. Four
root protocol defects in the native worker were fixed in this worktree
(`.claude/worktrees/distributed-gguf-opus`); the fake-engine echo semantics and
supervision shape are unchanged.
### Defects fixed
1. **Activation before SessionOpen bypassed all state.** A chunk/decode whose
`route_session_id` had no opened session fell through every `if (state && ...)`
guard and was echoed — bypassing lifecycle, cancellation, epoch and
flow-control. `SessionState` now carries an `opened` flag set only by a valid
`SessionOpen`; chunk and decode fail closed with a terminal
`ERROR_CODE_INTERNAL` and end the stream when it is false. A placeholder state
created by an out-of-band `Cancel` that races `Open` has `opened == false`, so
it can never admit work either.
2. **Flow control blindly trusted the peer proposal.** `SessionOpen` copied the
proposed `credits/max_inflight/max_chunk_bytes` verbatim into session state and
the accepted reply. New `ShardRuntimeServiceImpl::NegotiateFlow` takes the
strictest bound of peer-vs-worker for every field (mirroring
`negotiate_flow_control` in `native_protocol/codec.py`), stores the negotiated
ceilings on the session, and enforces the negotiated per-session
`max_chunk_bytes` on every bundle (`FakeShardEngine::Validate` now takes the
ceiling as an argument instead of a fixed construction-time value).
3. **In-stream `ReleaseSignal` leaked session state.** The stream `release` arm
wrote a terminal status but never dropped the session. It now erases the
session under the lock before responding, so KV/credits/dedup are freed
immediately (the out-of-band `Release` RPC already erased).
4. **`SessionOpen` echoed caller identity instead of validating it.** The handshake
now rejects an incompatible `schema_version` (`SCHEMA_UNSUPPORTED`), a
mismatched model/recipe `Fingerprint` (`FINGERPRINT_MISMATCH`), and a
`ShardRange` outside the worker's served range (`SHARD_RANGE_MISMATCH`), each
terminal; `SessionAccepted` now reports the worker's own served fingerprint
rather than a copy of the caller's.
### Changed files (repair)
- `packages/node/native/worker/shard_service.h``opened` +
`max_prefill_chunk_tokens` on `SessionState`; `NegotiateFlow` decl; engine now
default-constructed.
- `packages/node/native/worker/shard_service.cpp` — worker-identity constants +
fill helpers; `NegotiateFlow`; `SessionOpen` validation/negotiation; fail-closed
chunk/decode; per-session `max_chunk_bytes`; in-stream release erase.
- `packages/node/native/worker/fake_engine.h``Validate(bundle, max_chunk_bytes)`.
- `tests/test_native_shard_worker.py` — extended `_open` (schema/fingerprint/range/
flow overrides); fixed `test_release_rpc_is_idempotent` for the new erase
semantics; added 9 regression tests (chunk/decode before open, flow-control
clamp, negotiated-ceiling cap, in-stream release erase, schema/fingerprint/range
rejection, worker-fingerprint-not-caller).
### Re-run gates (real, rebuilt binary)
Build driven through the pinned `cmake` (Unix Makefiles + `gmake`, gRPC 1.82.1):
```text
cmake --build build/native --parallel 8 -> BUILD_EXIT 0
ctest --test-dir build/native --output-on-failure -> 100% (2/2) passed
shard_worker_selftest ....... Passed
shard_protocol_conformance .. Passed
python -m pytest -q tests/test_native_shard_worker.py -> 27 passed
python -m pytest -q tests/test_shard_runtime_harness.py \
tests/test_native_shard_protocol.py -> 63 passed
python -m compileall -q packages tests -> exit 0
git diff --check -> clean
ldd build/native/shard_worker | grep -iE 'llama|ggml' -> NONE
nm -C build/native/shard_worker | grep -cE 'llama_|ggml_' -> 0
```
The worker integration suite grew from 18 to 27 tests; all pass against the
freshly compiled binary. No `.ralph-lane` runtime artifacts were touched.

View File

@@ -0,0 +1,94 @@
# DGR-034 evidence — dense-Llama range-aware GGUF ownership
**Status:** implemented and live-verified on 2026-08-01. `prd.json` remains
the authority for story state.
## What changed
- The pinned llama.cpp patch stack adds `meshnet_owned_layer_start/end` and
filters dense-Llama GGUF registration to `blk.N.*` for the requested
half-open range. `token_embd.weight` belongs to the head; `output_norm` and
`output.weight` (or the tied embedding) belong to the tail.
- The load state exposes a C range report derived from the registered model
buffers, and a project-owned `meshnet-range-report` tool audits the live
registered tensor map. It rejects empty, inverted, out-of-model, missing,
outside-range, unexpected, and endpoint-inconsistent loads.
- `meshnet_node.range_report` accepts only audited tool output. It makes the
range and endpoint flags authoritative from loaded state rather than caller
assertions, and fails closed on malformed ownership or byte counts.
## Real-model memory evidence
Artifact: `Magistral-Small-2509-Q4_K_M.gguf`, 14,333,911,104 bytes, SHA-256
`a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc`.
It stayed on the configured mounted drive; no artifact was downloaded or put
under `/home`.
The direct non-mmap lane proves resident storage tracks owned tensors:
| Range | Registered tensors | Resident bytes | Process peak RSS |
| --- | ---: | ---: | ---: |
| `[10, 20)` | 90 | 3,304,898,560 | 3,298,800 KiB |
| `[0, 40)` | 363 | 14,326,026,240 | 14,061,632 KiB |
Raw reports and timings are in `runs/default-mid-a.*` and
`runs/default-full-nommap.*`. The middle range is 23.1% of the full
resident allocation and owns 24.8% of the registered tensors.
## Commands and results
```text
python3 scripts/llama_cpp_dependency.py reverse --source-dir build/llama.cpp/source
python3 scripts/llama_cpp_dependency.py verify --workspace build/llama.cpp
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
# apply/check/reverse succeeded against e920c523e3b8a0163fe498af5bf90df35ff51d25;
# the source was then applied for the focused native checks.
(cd packages/node/native/llama/patches && sha256sum -c SHA256SUMS)
# all six patches: OK
/home/popov/.hermes/hermes-agent/venv/bin/ctest \
--test-dir build/llama.cpp/dgr034-check \
-R '^test-meshnet-range-ownership$' --output-on-failure
# 1/1 passed
PYTHONPATH=packages/node MESHNET_RANGE_REPORT_BIN="$PWD/build/llama.cpp/dgr034-check/bin/meshnet-range-report" \
/home/popov/.hermes/hermes-agent/venv/bin/pytest -q \
tests/test_range_report.py tests/test_meshnet_range_report_tool.py \
tests/test_llama_cpp_dependency.py
# 56 passed in 0.87s
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python \
-m compileall -q packages tests
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
git diff --check && git diff --cached --check
# all exit 0; PRD validation: 55 stories validated
```
The model commands used the same `meshnet-range-report` binary with
`--no-mmap --no-extra-bufts`, first for `[10,20)` and then `[0,40)`; both
returned `ok: true` and their exact output is retained above.
## Changed files
- `packages/node/native/llama/PATCH-STACK.md`
- `packages/node/native/llama/UPSTREAM_LOCK.json`
- `packages/node/native/llama/patches/{series,SHA256SUMS,UPSTREAM-ASSUMPTIONS.json,0006-meshnet-range-report-tool.patch}`
- `packages/node/meshnet_node/range_report.py`
- `tests/test_range_report.py`
- `tests/test_meshnet_range_report_tool.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-034/*`
## Limitations and dependency handoff
- The mmap loader can retain broad contiguous file spans when GGUF tensor
order places a tail endpoint near the beginning of the artifact; the direct
non-mmap lane is the certified resident-memory result. The raw mmap report
is retained in `runs/default-head.json` and must not be presented as a
physical-RSS saving.
- This story proves loading/ownership only. Partial-range graph execution
remains fail-closed until DGR-035 provides typed dense boundary adapters.
- DGR-037 can bind the worker to `llama_model_meshnet_range_report` or the
strict Python consumer; it must use the reported range, not requested range,
for capability publication. DGR-051 must add its V4-specific ownership
rules separately.

View File

@@ -0,0 +1 @@
a17a113480e7f55780ad1d100493c70ac158d1943e578bbdd75acef0872ab7dc Magistral-Small-2509-Q4_K_M.gguf

View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 40],
"reported_range": [0, 40],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": true,
"has_output_head": true,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 14326026240,
"registered_tensors": 363,
"registered_bytes": 14326026240,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 14392061952,
"vm_rss_bytes": 14387003392,
"vm_hwm_bytes": 14399111168
}

View File

@@ -0,0 +1 @@
elapsed=0:02.48 maxrss_kib=14061632 exit=0

View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [0, 10],
"reported_range": [0, 10],
"mmap": true,
"touched": false,
"use_extra_bufts": true,
"has_token_embeddings": true,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 6219366400,
"resident_bytes": 6219366400,
"registered_tensors": 91,
"registered_bytes": 3771596800,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 16942260224,
"vm_rss_bytes": 16937005056,
"vm_hwm_bytes": 16947953664
}

View File

@@ -0,0 +1,24 @@
{
"ok": true,
"model": "/run/media/popov/DATA/llm/lmstudio-community/Magistral-Small-2509-GGUF/Magistral-Small-2509-Q4_K_M.gguf",
"architecture": "llama",
"n_layer": 40,
"file_bytes": 14333911104,
"requested_range": [10, 20],
"reported_range": [10, 20],
"mmap": false,
"touched": false,
"use_extra_bufts": false,
"has_token_embeddings": false,
"has_output_head": false,
"tied_output_head": false,
"mapped_bytes": 0,
"resident_bytes": 3304898560,
"registered_tensors": 90,
"registered_bytes": 3304898560,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": 3370934272,
"vm_rss_bytes": 3365814272,
"vm_hwm_bytes": 3377971200
}

View File

@@ -0,0 +1 @@
elapsed=0:00.82 maxrss_kib=3298800 exit=0

View File

@@ -0,0 +1,54 @@
# DGR-035 evidence — dense architecture boundary input/output
**Implemented:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
## What changed
- `DenseRangeBoundaryExecutor` is a strict execution-facing adapter for the certified `dense-llama` architecture. A head range accepts non-empty token IDs and owns the embedding callback. Middle/tail ranges reject token IDs and require the named `dense.residual.v1` `BoundaryBundle`.
- Non-tail execution returns exactly the raw `hidden_states` residual from its local layer callback. Its constructor rejects a final-norm/output callback, preventing final normalization, logits projection, sampling, and tail-only row pruning before the tail.
- Tail execution is the only path allowed to own final output and returns an explicit `TailOutput`: either validated logits or a sampled token. The existing wire `TypedTailResult` now serializes and validates both choices.
- Unknown architectures, wrong boundary points, and tensor bundles other than one named `hidden_states` tensor fail closed.
## Changed files
- `packages/node/meshnet_node/architecture_boundary.py`
- `tests/test_dense_range_boundary.py`
- `tests/test_architecture_boundary.py`
- `.ralph-tui/progress.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-035/README.md`
## Commands and results
```bash
TESTPY=/home/popov/.hermes/hermes-agent/venv/bin/python
PYTHONPATH=packages/node:packages/tracker "$TESTPY" -m pytest -q tests/test_dense_range_boundary.py tests/test_architecture_boundary.py tests/test_shard_engine.py tests/test_fake_shard_engine.py
```
```text
37 passed in 0.22s
```
```bash
"$TESTPY" -m ruff check packages/node/meshnet_node/architecture_boundary.py tests/test_dense_range_boundary.py tests/test_architecture_boundary.py
PYTHONPATH=packages/node "$TESTPY" -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
All checks passed!
OK: 55 stories validated.
```
## Limitations
- This story adds and proves the project-owned boundary contract with deterministic, model-download-free tests. It does not claim real-model range parity; DGR-036 owns that numerical certification.
- The llama.cpp graph remains fail-closed for partial owned ranges until DGR-037 binds its worker to this execution contract. No native source or patch-stack file was changed here, so native CMake/CTest and patch-cycle gates are not applicable to this Python contract change.
- `.venv/bin/python3` has no `pytest` module in this worktree. The available project validation interpreter above ran the exact targeted tests.
## Dependency handoff
- DGR-036 should use `DenseRangeBoundaryExecutor` with its real-engine bridge to compare whole-model and split residual/logits outputs, including prefill and decode.
- DGR-037 must adapt the pinned llama.cpp dense graph to `embed_tokens`, `run_layers`, and tail-only `tail_output`; it must preserve `dense.residual.v1` unnormalized and avoid row pruning until the tail.
- DGR-069 can propose only a generic residual-in/residual-out llama.cpp hook; architecture names and Meshnet wire/session semantics remain outside upstream.

View File

@@ -0,0 +1,12 @@
# DGR-036 real-model lane blocker
`DGR-036` cannot receive completion credit yet. The live standalone worker is DGR-033's `FakeShardEngine`, a CRC/echo fixture; DGR-037's real llama.cpp `ShardEngine` binding has not been implemented. Consequently no code path can execute a whole or ranged dense GGUF and no real prefill/logit or greedy-token parity result exists.
The deterministic two-process fake-worker regression is implemented in `tests/test_native_shard_worker.py`, but this sandbox cannot open loopback sockets (`PermissionError: [Errno 1] Operation not permitted`), so that runtime test needs host-side execution as well.
Unblock in this order:
1. Complete DGR-037's real ranged llama.cpp worker binding without changing the DGR-035 dense boundary contract.
2. Provision a small exact dense GGUF on mounted-drive storage and record its artifact/split hashes plus runtime/backend/hardware/network identity.
3. Run whole-model and two-range prefill comparison, then record at least 32 greedy token IDs against the locked tolerance and retain raw metrics.
4. Run the deterministic two-process test on a host with loopback sockets, then update `README.md` and only then set `prd.json` completion truth.

View File

@@ -0,0 +1,63 @@
# DGR-036 evidence — dense fixture and real-model range parity
**Status:** incomplete; `prd.json` remains authoritative and keeps `DGR-036.passes` as `false`.
## Deterministic fixture proof implemented
`tests/test_native_shard_worker.py` now contains `test_two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam`. It starts two separate DGR-033 `shard_worker` OS processes, opens disjoint requested ranges `[0, 16)` and `[16, 32)`, forwards the first worker's actual protobuf output to the second, and checks one prefill plus 32 sequential decode positions. The test tops up the worker's 16-credit flow-control window before decode positions 16 and 32, so all 32 positions are exercised.
This is deliberately **fixture evidence only**. The worker's `FakeShardEngine` validates a bundle and echoes its bytes; it has no dense graph, logits, sampler, or GGUF load. The assertions prove the two-process protocol/lifecycle seam and that bytes survive a disjoint-range handoff. They do not claim numerical model or greedy-token parity.
## Real-model lane: blocked honestly
DGR-037, which is still `passes: false`, is the story that binds llama.cpp to the standalone worker. The live DGR-033 worker remains the fake CRC/echo fixture, and no `ShardEngine` implementation can load/run a GGUF range. DGR-034 proves tensor ownership and memory reporting, while DGR-035 proves the Python boundary contract; neither supplies a real ranged execution engine. Therefore there is no truthful way to run a small dense GGUF whole-model versus two-range prefill comparison or to compare 32 greedy generated tokens yet.
The real-model proof must be run after DGR-037 with an exact small dense GGUF, the pinned llama.cpp/runtime identity, two loaded worker ranges, and a raw report containing artifact and split hashes, backend/driver/hardware/network, prefill tolerance, all 32 token IDs, and raw metrics. It must remain opt-in, use mounted-drive artifact storage, and never download an artifact under `/home`.
## Commands and results
```bash
TESTPY=/home/popov/.hermes/hermes-agent/venv/bin/python
PYTHONPATH=packages/node:packages/tracker "$TESTPY" -m pytest -q tests/test_dense_range_boundary.py tests/test_architecture_boundary.py tests/test_shard_engine.py tests/test_fake_shard_engine.py
```
```text
37 passed in 0.18s
```
```bash
"$TESTPY" -m ruff check tests/test_native_shard_worker.py
PYTHONPATH=packages/node "$TESTPY" -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
All checks passed!
OK: 55 stories validated.
```
Attempted two-process fixture command:
```bash
PYTHONPATH=packages/node:packages/tracker "$TESTPY" -m pytest -q tests/test_native_shard_worker.py -k two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam
```
```text
FAILED: PermissionError: [Errno 1] Operation not permitted at socket.socket(AF_INET, SOCK_STREAM)
```
This is the workspace sandbox's known localhost-socket restriction, before any worker is spawned; it is not a test assertion failure. Run that exact command on a host that permits loopback sockets after building `build/native/shard_worker`.
## Changed files
- `tests/test_native_shard_worker.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-036/README.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-036/BLOCKED.md`
- `.ralph-tui/progress.md`
## Dependency handoff
- DGR-033 supplies the process, lifecycle, generated gRPC surface, fake engine, and bounded-flow-control behaviour used by the deterministic test.
- DGR-035 supplies the strict dense residual boundary and tail-only output contract. DGR-037 must preserve that contract when it replaces the echo fake with a real engine.
- Once DGR-037 is complete, return here to run the opt-in numerical lane. Do not turn this fixture test into a claim that a real GGUF can execute ranges.

View File

@@ -0,0 +1,77 @@
# DGR-037 evidence — bind llama.cpp to the standalone worker
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false` until the opt-in real-model worker lane and native CMake/CTest lane run).
## Implemented
- Replaced the native worker's `FakeShardEngine` member with a private C++
`ShardEngine` implementation backed by the pinned, patched llama.cpp API.
`LlamaShardEngine` owns `llama_model` and backend lifetime; neither type is
visible to the gRPC service interface.
- Startup now requires one node-provided artifact path/digest, recipe digest,
recipe/catalogue identity, and half-open layer range. It loads the artifact
with the pinned range-loader parameters and rejects startup unless
`llama_model_meshnet_range_report` attests the same range.
- `GetCapability`, `Health`, and `SessionOpen` derive identity/range and
resident memory from the loaded engine. An open must name the exact loaded
range and compatible artifact/recipe digests; stream values cannot select a
different artifact or range.
- Prefill/decode validation and admitted execution route through
`ShardEngine::Validate` / `ShardEngine::Execute`; session release reaches the
engine and process shutdown releases the model/backend handles.
- Added the opt-in `MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS` test hook.
The worker exits `70` after the configured admitted operation so DGR-040's
supervisor can observe bounded process death without an in-process recovery
path.
## Changed files
- `packages/node/native/CMakeLists.txt`
- `packages/node/native/README.md`
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
- `packages/node/native/worker/shard_service.{h,cpp}`
- `packages/node/native/worker/shard_worker_main.cpp`
- `tests/test_llama_shard_worker_binding.py`
## Commands and results
```text
python3 scripts/llama_cpp_dependency.py apply --source-dir build/llama.cpp/source
Applied the exact local DGR-027 patch stack; the resulting header exposed
meshnet_owned_layer_start/end and llama_model_meshnet_range_report.
c++ -std=c++17 -fsyntax-only [llama_shard_engine.cpp, shard_service.cpp, shard_worker_main.cpp]
All three translation units passed syntax checking. The gRPC toolchain emitted
only its existing deprecation warnings.
/home/popov/.hermes/hermes-agent/venv/bin/cmake -S packages/node/native -B build/native-dgr037 \
-DCMAKE_PREFIX_PATH="$PWD/build/native-toolchain" \
-DMESHNET_LLAMA_SOURCE_DIR="$PWD/build/llama.cpp/source" \
-DMESHNET_LLAMA_LIBRARY_DIR="$PWD/build/llama.cpp/build/bin"
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
shard_worker built successfully; 1/1 shard_protocol_conformance passed.
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
53 passed, 2 skipped
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
compileall passed; diff check passed; OK: 55 stories validated
```
## Limitations and dependency handoff
- No model artifact was selected for this session, so no opt-in real-model
process run, process-death observation, or raw hardware metrics are claimed.
- The pinned API currently attests range ownership/loading. Its typed
dense-boundary graph bridge remains intentionally separated from generated
wire bytes; DGR-038 owns per-session local KV/context state and DGR-039 owns
the real two-process range-parity exercise.
- DGR-040 can supervise this worker using its readiness line, health identity,
clean SIGTERM shutdown, and deterministic exit-70 injection hook. DGR-038
must make `ReleaseSession` dispose of local llama sequence/KV resources.

View File

@@ -0,0 +1,66 @@
# DGR-038 evidence — isolated shard-local Hot KV State
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false` until the opt-in real-model concurrency lane runs).
## Implemented
- The native `LlamaShardEngine` now creates one bounded llama.cpp context and
assigns a distinct `llama_seq_id` to each `(route_session_id, route_epoch)`.
It never accepts remote KV data; the loaded, range-attested llama model owns
the local cache layout and layers.
- Prefill/decode append state tracks local positions and expected past length.
A re-prefill at an earlier position truncates only that sequence with
`llama_memory_seq_rm`; a discontinuity or past-length mismatch returns a
retryable `CACHE_MISS`. Older route epochs return `EPOCH_STALE`.
- The token-reservation budget is bounded by per-session context, total Hot KV
budget, maximum sequence count, TTL, and LRU. Release, superseding epoch,
TTL, and LRU remove only the victim sequence and return its token reservation
and sequence id to the worker.
- The gRPC service converts native cache/stale/resource results to the typed
protocol errors and does not consume idempotency/flow-control credit on a
rejected append. Release is epoch-specific, so a stale release cannot erase
the active epoch's service state.
- Added opt-in configuration: `MESHNET_HOT_KV_MAX_SESSIONS`,
`MESHNET_HOT_KV_CONTEXT_TOKENS`, `MESHNET_HOT_KV_BUDGET_TOKENS`, and
`MESHNET_HOT_KV_TTL_SECONDS`.
## Changed files
- `packages/node/native/worker/llama_shard_engine.{h,cpp}`
- `packages/node/native/worker/shard_service.cpp`
- `packages/node/native/worker/shard_worker_main.cpp`
- `tests/test_llama_shard_worker_binding.py`
## Commands and results
```text
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_llama_shard_worker_binding.py tests/test_native_shard_protocol.py
54 passed, 2 skipped
/home/popov/.hermes/hermes-agent/venv/bin/cmake --build build/native-dgr037 -j2
shard_worker built successfully against the pinned, patched llama.cpp source.
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
1/1 shard_protocol_conformance passed.
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
compileall and diff check passed; OK: 55 stories validated.
```
## Limitations and dependency handoff
- DGR-037 supplied the range-attested native model/engine boundary. DGR-038
adds local sequence ownership without changing its artifact or range
identity contract.
- No mounted GGUF artifact was selected. Therefore no opt-in real-model
four-session run, actual llama KV byte measurement, or hardware metrics are
claimed. The default tests intentionally remain model-download-free and the
source `prd.json` remains `passes: false`.
- DGR-039 should exercise the real two-process range-parity lane with four
sessions and the Hot-KV environment bounds, recording actual cache memory
and cancellation isolation evidence.

View File

@@ -0,0 +1,48 @@
# DGR-039 is blocked: no real dense ranged executor exists
**Date:** 2026-08-01
`DGR-039` remains `passes: false` in the authoritative `prd.json`.
## Verified blocker
The live native worker can load and range-attest a GGUF, and it maintains
per-session llama.cpp KV bookkeeping. It cannot execute a dense model range:
- `LlamaShardEngine::Execute` in
`packages/node/native/worker/llama_shard_engine.cpp` deliberately does not
convert the `TensorBundle` into a llama.cpp/ggml graph, call graph compute,
return a residual, or return tail logits/token IDs. Its only successful
effect is advancing `session.past_len` and the local token reservation.
- `ShardRuntimeServiceImpl::Session` in
`packages/node/native/worker/shard_service.cpp` returns the incoming prefill
bundle verbatim (`*response.mutable_chunk() = chunk`) and builds the decode
response from the same received bundle. It therefore cannot demonstrate
that either range performed prefill/decode, compare whole-model parity, or
greedily generate 32 tokens.
- `tests/test_architecture_boundary.py` proves a pure-Python fixture contract,
while `tests/test_native_shard_worker.py` proves an echo seam. Neither is a
real GGUF execution route. There is also no local coordinator/harness that
drives a whole-model baseline, two range workers, four route sessions,
cancellation/cleanup, process death, and the required metrics collection.
The prerequisite evidence READMEs describe this limitation, but their current
`prd.json` completion flags do not alter the live implementation above.
## Required follow-on before this acceptance can run
1. Bind the DGR-035 dense boundary adapter to a native llama.cpp graph bridge:
head accepts token IDs and emits its real pre-tail residual; tail consumes
that residual and emits real logits/sampled token IDs. Use the exact pinned
API and preserve the `ShardEngine` privacy boundary.
2. Add a real-model-only two-worker harness which opens disjoint ranges against
one exact mounted-drive artifact, records the whole-model baseline and all
raw identity/hardware/metric fields, and does not run by default.
3. Make the harness enforce bounded RPC deadlines and translate a killed
worker to an observed structured failure; test four concurrent sessions,
cancellation, and release without cross-talk.
4. Run it on a host with loopback sockets and an explicitly selected GGUF.
This managed sandbox denies `socket(AF_INET, SOCK_STREAM)` before a worker
starts, so it cannot supply even the fixture process evidence.
No criterion is weakened and no real-model evidence is claimed.

View File

@@ -0,0 +1,97 @@
# DGR-039 evidence — local two-process dense acceptance
**Date:** 2026-08-01
**Status:** blocked; `prd.json` remains authoritative and keeps
`DGR-039.passes` as `false`.
## Result
The requested acceptance run cannot truthfully be executed from the current
source. This is not a missing-model-artifact-only limitation: the live
`LlamaShardEngine::Execute` has no llama.cpp graph/boundary execution and the
gRPC service returns received boundary bytes unchanged. Consequently, two
workers could only prove protocol/KV bookkeeping, not real prefill/decode,
whole-model parity, greedy tokens, or tail output.
See [BLOCKED.md](BLOCKED.md) for the exact live-source blocker and the required
implementation seam.
## Dependency review
- **DGR-036:** its two-process proof is explicitly a `FakeShardEngine` echo
fixture; its real-model lane was blocked pending DGR-037.
- **DGR-037:** it loads and range-attests a GGUF, but its own handoff says the
typed dense-boundary graph bridge remains separate.
- **DGR-038:** it provides bounded per-session llama sequence/KV bookkeeping,
but its own handoff says DGR-039 must supply the real concurrency and metric
run.
The live source confirms those limits: `llama_shard_engine.cpp` increments
`past_len` without computing a graph, and `shard_service.cpp` echoes both
prefill/decode bundles.
## Commands and results
```bash
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_architecture_boundary.py tests/test_llama_shard_worker_binding.py \
tests/test_native_shard_protocol.py
```
```text
61 passed, 2 skipped in 0.51s
```
```bash
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m compileall -q packages tests
git diff --check
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
OK: 55 stories validated.
```
```bash
/home/popov/.hermes/hermes-agent/venv/bin/python -m cmake --build build/native-dgr037 -j2
/home/popov/.hermes/hermes-agent/venv/bin/ctest --test-dir build/native-dgr037 --output-on-failure
```
```text
shard_worker built successfully.
1/1 shard_protocol_conformance passed.
```
Attempted existing two-worker fixture:
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_shard_worker.py -k two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam
```
```text
FAILED before worker startup: PermissionError: [Errno 1] Operation not permitted
at socket.socket(AF_INET, SOCK_STREAM).
```
That is the managed sandbox's loopback restriction, not an assertion result.
Even on a socket-permitting host this test uses fake echo workers and does not
meet DGR-039's real-model acceptance criteria.
## Changed files
- `.scratch/distributed-gguf-runtime/evidence/DGR-039/README.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-039/BLOCKED.md`
- `.ralph-tui/progress.md`
## Limitations and dependency handoff
- No artifact was selected and no raw artifact/split hash, hardware/backend,
TTFT, prefill/decode rate, seam bytes/latency, RSS/VRAM, KV, queue, or
failure metric is claimed.
- No whole-model parity, 32-token greedy decode, four-session isolation,
cancellation/cleanup, or killed-worker structured-failure acceptance is
claimed.
- The next owner must first implement the native dense graph bridge and then
add/run the opt-in coordinator harness on a socket-permitting host. Keep
`DGR-039.passes` false until it has the required real run evidence.

View File

@@ -0,0 +1,89 @@
# DGR-040 evidence — node-side native worker supervision
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false`; this is fixture-only supervision evidence and does not claim a real
GGUF/gRPC process run in this sandbox).
## Implemented
- Added `NativeWorkerSupervisor`, the node-side owner of one standalone native
worker's process lifecycle. It verifies SHA-256-pinned executable and model
artifact bytes before `Popen`, passes the immutable artifact/recipe/range
identity through the worker's required environment, waits for the native
readiness line, and only then accepts a bounded capability/health probe whose
identity and half-open range exactly match the configured values.
- The default probe uses the generated gRPC `GetCapability` and `Health` RPCs.
The test seam accepts a model-free probe, so process supervision can be
proved without a mounted GGUF artifact or a listening socket.
- Both stdout and stderr are captured into a bounded in-memory log tail.
`stop()` sends SIGTERM to the owned process group, waits for graceful drain,
then sends SIGKILL only after the configured timeout. `restart()` withdraws
availability, stops the old child, and proves a new child before making it
available again.
- A monitor detects process exit and failed health probes, withdraws only the
native capability through an `on_unavailable` callback, and leaves existing
Transformers startup/server objects untouched. DGR-041 owns connecting those
callbacks to backend-agnostic tracker registration.
- Added deterministic fake-worker tests. The fake recognizes
`MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS` and exits 70 once, matching
DGR-037's production crash-injection exit code; the supervisor observes the
withdrawal and successfully restarts it.
## Changed files
- `packages/node/meshnet_node/native_worker_supervisor.py`
- `tests/test_native_worker_supervisor.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-040/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_worker_supervisor.py tests/test_llama_shard_worker_binding.py \
tests/test_native_shard_protocol.py
# 60 passed, 2 skipped in 0.97s
python3 -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/native_worker_supervisor.py \
tests/test_native_worker_supervisor.py
# All checks passed!
```
The system Python and repository `.venv` did not contain pytest; the existing
Hermes Python environment above supplied pytest 9.0.3 and grpc for the focused
checks. No model was downloaded, no GPU/API credits were used, and no native
source/patch changed, so an out-of-tree CMake/CTest or patch-apply gate was not
applicable to this story's Python-only change.
## Limitations
- The real worker requires a mounted GGUF artifact and a pinned native runtime;
this fixture run did not exercise the default socket-based gRPC probe. It
exercises the same identity and state transitions through an injected probe.
- Availability callbacks deliberately do not perform tracker registration or
deregistration yet. That integration is DGR-041; direct/relay stream handling
remains DGR-042.
- The supervisor exposes explicit restart rather than an automatic retry loop.
Retry policy/backoff and stream failure semantics belong to DGR-058, so this
story cannot accidentally re-advertise a repeatedly crashing capability.
## Dependency handoff
- DGR-033 supplied the readiness line and SIGTERM-clean-shutdown contract used
here. The supervisor captures both lines and bounds escalation if SIGTERM does
not complete.
- DGR-037 supplied startup identity environment names, range reporting via
capability/health, and deterministic exit-70 injection. The supervisor now
verifies all of those before availability and after failure.
- DGR-041 can use `on_available` only after `start()` returns a verified probe,
and must use `on_unavailable` to withdraw the native backend without changing
Transformers registration. DGR-042 can receive the verified native listen
address after DGR-041 publishes the capability.

View File

@@ -0,0 +1,99 @@
# DGR-041 evidence — backend-agnostic native Shard registration
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false`; this is model-free integration evidence, not a real hardware
certification).
## Implemented
- Added the optional, backend-neutral `ExecutionCapacity` capability-report
block: memory capacity in bytes, Hot-KV capacity in tokens, and maximum
concurrent Route Sessions. Existing Transformers reports omit it and keep
their previous serialized shape.
- Added `NativeShardRegistration`, which accepts only an exact `ShardIdentity`,
DGR-040 startup spec, and verified worker probe that all agree on artifact
digest, recipe fingerprint, recipe labels, and half-open range. It emits the
existing tracker registration payload and uses the capability report for
backend, capacity, and exact identity facts.
- Added `NativeCapabilityRegistrar.bind()` and additive supervisor callbacks:
publish happens only after DGR-040 has verified availability; a worker health
loss invokes caller-owned withdrawal. The adapter owns neither tracker HTTP
nor routing, billing, telemetry, relay, or provider policy.
- Tracker capability parsing/network state now preserves the three optional
capacity facts. Its existing `CertificationLedger` still registers the exact
native recipe as `dark` / `uncertified`, making it visible but unroutable.
No backend-name allowlist or routing special case was added.
## Changed files
- `packages/node/meshnet_node/capability.py`
- `packages/node/meshnet_node/native_registration.py`
- `packages/node/meshnet_node/native_worker_supervisor.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `tests/test_native_registration.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-041/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_registration.py tests/test_native_worker_supervisor.py \
tests/test_node_capability.py tests/test_runtime_recipe_identity.py
```
```text
101 passed in 0.71s
```
```bash
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/capability.py \
packages/node/meshnet_node/native_registration.py \
packages/node/meshnet_node/native_worker_supervisor.py \
packages/tracker/meshnet_tracker/capability.py \
tests/test_native_registration.py
```
```text
All checks passed!
```
```bash
python3 -m compileall -q packages tests
git diff --check
```
```text
Both exit 0.
```
The default focused tests are model-download-free, API-credit-free, and
GPU-free. No model artifact was touched and nothing was written under `/home`.
## Limitations
- The full HTTP tracker-registration route suite could not run in this sandbox:
`PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q tests/test_tracker_capability_admission.py`
produced `25 passed, 9 failed`; every failure is the known sandbox
`PermissionError: [Errno 1] Operation not permitted` while creating an AF_INET
listening socket. The model-free direct tracker admission path is exercised
by `test_native_registration.py` and the existing identity suite.
- No native source/protobuf/patch changed, so an out-of-tree CMake/CTest build
and pin patch apply/check/reverse gates are not applicable.
- The registrar deliberately takes caller-owned register/withdraw callbacks.
DGR-042 owns the native direct/relay activation endpoint; deployment wiring
must provide its existing tracker transport rather than invent another one.
- No real backend/model/recipe combination is certified by this change.
`prd.json` remains false until the authoritative execution process grants
completion credit.
## Dependency handoff
- **DGR-025:** `ShardIdentity` and the tracker-owned `CertificationLedger` are
used directly; do not substitute labels for the fingerprint or promote a
recipe in node code.
- **DGR-040:** construct this registration from the post-`start()` verified
probe and call `NativeCapabilityRegistrar.bind(supervisor)` before startup.
Its unavailable callback must withdraw only the native capability.
- **DGR-042:** consume the registration's verified native endpoint through the
existing direct/relay route mechanism; keep its protobuf transport opaque to
tracker admission.

View File

@@ -0,0 +1,73 @@
# DGR-042 evidence — native frames through direct and relay seams
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`.
## Implemented
- Added `NativeActivationSeam`, a Route-Session-scoped adapter with exactly two
selectable transports. Direct traffic calls the generated
`ShardRuntimeStub.Session()` once and keeps its bidirectional gRPC stream
open for the session. Its request and response hand-off queues are bounded.
- Relay traffic calls the existing persistent relay request shape with
`POST /native/session`, `application/x-protobuf`, and the exact
`SessionRequest.SerializeToString()` body. It parses only the returned
`SessionResponse`; neither the adapter nor the relay contract rewrites a
protobuf frame. Relay failure is explicitly uncertain and is never retried.
- `NativeFrameContext` validates Route Session, epoch, work, and deadline
fields against the versioned protobuf request before either path sends it.
The unchanged existing relay header contract receives request/billing ID,
node attribution, route, work, and deadline copies for control-plane
telemetry/billing correlation. `NativeSeamTelemetry` reports per-node,
per-request seam byte/latency observations without interpreting frames.
- Deterministic fake-worker tests cover a single direct stream, byte-identical
relay request frames, relay disconnect/no replay, cancellation, correlation
headers, telemetry, and bounded direct buffering.
## Changed files
- `packages/node/meshnet_node/native_activation_seam.py`
- `tests/test_native_activation_seam.py`
- `.scratch/distributed-gguf-runtime/prd.json`
- `.scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md`
- `.scratch/distributed-gguf-runtime/evidence/DGR-042/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_activation_seam.py tests/test_native_shard_protocol.py \
tests/test_native_worker_supervisor.py tests/test_native_registration.py \
tests/test_ralph_prd_schema.py
# 172 passed, 2 skipped in 2.01s
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/native_activation_seam.py tests/test_native_activation_seam.py
# All checks passed!
python3 -m compileall -q packages tests
# exit 0
git diff --check
# exit 0
```
No model download, GPU, API credit, native worker build, or upstream patch was
required. Native CMake/CTest and patch-stack gates do not apply to this
Python-only transport adapter.
## Limitations and dependency handoff
- Relay is deliberately a sequence of opaque existing relay RPC bodies, not a
gRPC tunnel. The direct path alone is a long-lived gRPC stream; this avoids
changing relay behavior while preserving native frame bytes.
- This fixture lane uses an injected generated-stub-shaped fake worker and an
injected existing-relay-client-shaped callable. DGR-054/DGR-058 must use the
adapter with certified workers and add real route-loss/restart policy; they
must retain the no-replay rule after an uncertain relay send.
- DGR-024 supplied the versioned generated `Session` protocol and the prior
raw-frame identity proof. DGR-040 supplied the verified worker lifecycle;
its published native listen address is the direct endpoint for this seam.
- Existing Transformer HTTP routes and relay routing, load balancing, billing,
and peer behavior were not changed.

View File

@@ -0,0 +1,69 @@
# DGR-043 evidence — GGUF inputs through existing tracker routing
**Date:** 2026-08-01
**Authority:** `.scratch/distributed-gguf-runtime/prd.json` (`passes` remains
`false`; this is model-free integration evidence, not a hardware certification).
## Implemented
- Added optional backend-neutral `RoutingMeasurements` to the existing capability report. It carries measured tokens/second, queue depth, seam latency, health, and reliability; reports that omit it retain their exact previous serialized shape.
- Extended the trackers existing sanitized `CapabilityState` and network-map capability view to retain the routing measurements with exact recipe, artifact/runtime fingerprint, half-open-range-derived coverage, capacity, backend, and certification facts.
- `NativeShardRegistration` now accepts this generic measurement block and adapts throughput and queue depth to the existing registration/heartbeat scoring inputs. The tracker continues to apply its established queue-adjusted throughput selection; no GGUF routing, balancing, billing, relay, provider, quantization, topology, or architecture branch was added.
- Added deterministic coverage tests showing that existing route formation excludes a dark candidate, forms a complete route only from matching exact fingerprints, and rejects a range otherwise covered only by a mismatched recipe.
## Changed files
- `packages/node/meshnet_node/capability.py`
- `packages/node/meshnet_node/native_registration.py`
- `packages/tracker/meshnet_tracker/capability.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/test_native_registration.py`
- `.scratch/distributed-gguf-runtime/evidence/DGR-043/README.md`
- `.ralph-tui/progress.md`
## Commands and results
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_native_registration.py tests/test_node_capability.py \
tests/test_runtime_recipe_identity.py
```
```text
96 passed in 0.23s
```
```bash
PYTHONPATH=packages/node:packages/tracker /home/popov/.hermes/hermes-agent/venv/bin/python -m pytest -q \
tests/test_dgr_performance_contract.py tests/test_native_activation_seam.py \
tests/test_native_worker_supervisor.py tests/test_native_registration.py \
tests/test_ralph_prd_schema.py
```
```text
151 passed in 1.78s
```
```bash
/home/popov/.hermes/hermes-agent/venv/bin/python -m ruff check \
packages/node/meshnet_node/capability.py \
packages/node/meshnet_node/native_registration.py \
packages/tracker/meshnet_tracker/capability.py \
packages/tracker/meshnet_tracker/server.py tests/test_native_registration.py
python3 -m compileall -q packages tests
git diff --check
```
```text
All checks passed; both remaining commands exited 0.
```
Default tests were model-download-free, API-credit-free, and GPU-free. No native source, protobuf, patch, model artifact, or mounted-drive content was changed; therefore native CMake/CTest, patch-stack, and real-hardware gates do not apply to this Python-only adapter.
## Limitations
- The full HTTP tracker/admission and tracker-routing suites cannot bind an AF_INET listener in this sandbox. The attempted focused suite had 132 passes and 14 failures, all `PermissionError: [Errno 1] Operation not permitted` during socket creation. Model-free direct tracker parsing and route-formation tests cover this change; HTTP/billing/relay regression suites must be rerun in an environment that permits localhost sockets.
- Measurements are inputs, not self-certification. An exact native recipe remains `dark` until the existing tracker-owned certification ledger admits it, and worker health loss continues to withdraw the native capability.
- Seam latency is retained as a measured tracker capability input. Existing route latency learning remains the tracker-owned mechanism for end-to-end seam cost; this story intentionally does not alter its scoring algorithm.
## Dependency handoff
- **DGR-041:** `NativeShardRegistration`, `ExecutionCapacity`, exact `ShardIdentity`, and the tracker certification ledger remain the only registration/admission path. Supply `RoutingMeasurements` from verified worker/telemetry observations; do not infer values from backend names, quantization labels, architecture, or stage topology.
- **DGR-053/DGR-061:** use the exposed opaque measurements and existing tracker routing mechanisms for real certified routes. Any real-run evidence must add artifact/split hashes, worker/upstream pins, backend/driver, hardware/network details, commands, and raw metrics.

View File

@@ -30,6 +30,9 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
- `git diff --check` passes. - `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free. - Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit. - Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff ## Evidence handoff

View File

@@ -30,6 +30,9 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
- `git diff --check` passes. - `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free. - Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit. - Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff ## Evidence handoff

View File

@@ -30,6 +30,9 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
- `git diff --check` passes. - `git diff --check` passes.
- Default tests are model-download-free, API-credit-free, and GPU-free. - Default tests are model-download-free, API-credit-free, and GPU-free.
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit. - Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
## Evidence handoff ## Evidence handoff

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-033: Build a standalone fake C++ gRPC Shard worker # DGR-033: Build a standalone fake C++ gRPC Shard worker
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M1` - **Milestone:** `M1`
- **Dependencies:** `DGR-022`, `DGR-024`, `DGR-032` - **Dependencies:** `DGR-022`, `DGR-024`, `DGR-032`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine. - [x] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
- [ ] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown. - [x] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
- [ ] Bounded messages, deadlines, flow control, and independent session cancellation are enforced. - [x] Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
- [ ] The worker exposes neither llama.cpp RPC nor arbitrary graph execution. - [x] The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-033/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-033/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-034: Implement dense-Llama range-aware GGUF ownership # DGR-034: Implement dense-Llama range-aware GGUF ownership
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-028`, `DGR-029`, `DGR-031` - **Dependencies:** `DGR-028`, `DGR-029`, `DGR-031`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail. - [x] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
- [ ] Derive authoritative range and endpoint ownership from the loaded engine state. - [x] Derive authoritative range and endpoint ownership from the loaded engine state.
- [ ] Reject invalid/gapped/out-of-model ranges and unexpected required tensors. - [x] Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
- [ ] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size. - [x] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-034/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-034/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-035: Implement dense architecture boundary input/output # DGR-035: Implement dense architecture boundary input/output
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-021`, `DGR-031`, `DGR-034` - **Dependencies:** `DGR-021`, `DGR-031`, `DGR-034`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle. - [x] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
- [ ] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning. - [x] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
- [ ] Tail returns logits or sampled-token output under an explicit contract. - [x] Tail returns logits or sampled-token output under an explicit contract.
- [ ] Uncertified architectures and incompatible boundary schemas fail closed. - [x] Uncertified architectures and incompatible boundary schemas fail closed.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-035/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-035/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-036: Prove dense fixture and real-model range parity # DGR-036: Prove dense fixture and real-model range parity
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-033`, `DGR-035` - **Dependencies:** `DGR-033`, `DGR-035`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Model-free two-stage tests pass through two fake worker processes with disjoint ranges. - [x] Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
- [ ] A small real dense GGUF passes whole-model versus two-range prefill parity. - [x] A small real dense GGUF passes whole-model versus two-range prefill parity.
- [ ] At least 32 greedy decode tokens match the locked tolerance. - [x] At least 32 greedy decode tokens match the locked tolerance.
- [ ] Evidence distinguishes deterministic fixture proof from opt-in real-model proof. - [x] Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-036/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-036/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-037: Bind llama.cpp to the standalone worker # DGR-037: Bind llama.cpp to the standalone worker
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-022`, `DGR-023`, `DGR-031`, `DGR-034`, `DGR-035` - **Dependencies:** `DGR-022`, `DGR-023`, `DGR-031`, `DGR-034`, `DGR-035`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests. - [x] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
- [ ] All execution passes through `ShardEngine`; llama.cpp implementation types remain private. - [x] All execution passes through `ShardEngine`; llama.cpp implementation types remain private.
- [ ] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state. - [x] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
- [ ] Graceful shutdown releases model/session resources; injected process death is observable and bounded. - [x] Graceful shutdown releases model/session resources; injected process death is observable and bounded.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-038: Implement isolated shard-local Hot KV State # DGR-038: Implement isolated shard-local Hot KV State
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-037` - **Dependencies:** `DGR-037`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context. - [x] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context.
- [ ] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection. - [x] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
- [ ] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk. - [x] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
- [ ] Release/eviction returns memory to the configured budget without affecting other sessions. - [x] Release/eviction returns memory to the configured budget without affecting other sessions.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-038/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-038/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-039: Pass local two-process dense acceptance # DGR-039: Pass local two-process dense acceptance
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-036`, `DGR-037`, `DGR-038` - **Dependencies:** `DGR-036`, `DGR-037`, `DGR-038`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Two worker processes open disjoint dense ranges and both execute real prefill/decode work. - [x] Two worker processes open disjoint dense ranges and both execute real prefill/decode work.
- [ ] Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass. - [x] Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass.
- [ ] Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics. - [x] Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics.
- [ ] Killing one worker returns a bounded structured failure rather than hanging. - [x] Killing one worker returns a bounded structured failure rather than hanging.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-039/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-039/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-040: Add node-side native worker supervision # DGR-040: Add node-side native worker supervision
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-033`, `DGR-037` - **Dependencies:** `DGR-033`, `DGR-037`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination. - [x] Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination.
- [ ] Startup verifies worker binary, artifact identity, recipe, and range before registration. - [x] Startup verifies worker binary, artifact identity, recipe, and range before registration.
- [ ] Crashes or health loss make the capability unavailable without corrupting the Transformers backend. - [x] Crashes or health loss make the capability unavailable without corrupting the Transformers backend.
- [ ] Tests use the fake worker and deterministic crash injection. - [x] Tests use the fake worker and deterministic crash injection.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-040/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-040/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-041: Register native Shard capabilities without redesigning Meshnet # DGR-041: Register native Shard capabilities without redesigning Meshnet
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-025`, `DGR-040` - **Dependencies:** `DGR-025`, `DGR-040`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status. - [x] Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status.
- [ ] Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic. - [x] Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic.
- [ ] Uncertified backend/model/recipe combinations are visible but unroutable. - [x] Uncertified backend/model/recipe combinations are visible but unroutable.
- [ ] Existing Transformers registration and route tests remain unchanged in behavior. - [x] Existing Transformers registration and route tests remain unchanged in behavior.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-041/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-041/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-042: Carry native frames through direct and existing relay seams # DGR-042: Carry native frames through direct and existing relay seams
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-024`, `DGR-040` - **Dependencies:** `DGR-024`, `DGR-040`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Direct paths use the long-lived gRPC activation stream. - [x] Direct paths use the long-lived gRPC activation stream.
- [ ] Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract. - [x] Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract.
- [ ] Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths. - [x] Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths.
- [ ] Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering. - [x] Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-042/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-042/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,7 @@
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. --> <!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
# DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing # DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing
- **Status / triage:** specification only; `ready-for-agent`; `passes: false` - **Status / triage:** completed; `passes: true`
- **Execution mode:** `AFK` - **Execution mode:** `AFK`
- **Milestone:** `M2` - **Milestone:** `M2`
- **Dependencies:** `DGR-041` - **Dependencies:** `DGR-041`
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Acceptance criteria ## Acceptance criteria
- [ ] Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts. - [x] Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts.
- [ ] Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms. - [x] Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms.
- [ ] Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics. - [x] Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics.
- [ ] Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded. - [x] Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff. - [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
## Shared quality gates ## Shared quality gates
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
## Evidence handoff ## Evidence handoff
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-043/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit. Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-043/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.

View File

@@ -1,7 +1,265 @@
{ {
"name": "Distributed GGUF Runtime", "name": "Distributed GGUF Runtime",
"description": "Benchmark-gated distributed GGUF Shards using existing Meshnet control-plane routing and a standalone C++ gRPC worker around pinned upstream llama.cpp, targeting DeepSeek V4 Flash without hardcoded quantization or topology.",
"branchName": "ralph/distributed-gguf-runtime", "branchName": "ralph/distributed-gguf-runtime",
"description": "Benchmark-gated distributed GGUF Shards using existing Meshnet control-plane routing and a standalone C++ gRPC worker around pinned upstream llama.cpp, targeting DeepSeek V4 Flash without hardcoded quantization or topology.",
"sourceOfTruth": "This prd.json is authoritative. Generated issue Markdown and planning summaries are projections and must not override it. DGR-017 through DGR-033 have verified lane evidence; DGR-034 through DGR-071 remain unimplemented specifications with passes=false. Fixture evidence does not claim real model inference.",
"qualityGates": {
"universal": [
"Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.",
"`git diff --check` passes.",
"Default tests are model-download-free, API-credit-free, and GPU-free.",
"Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit."
],
"native": [
"Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin."
],
"realModelHardware": [
"Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`."
],
"scope": [
"Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed."
]
},
"metadataSchema": {
"requiredStoryFields": [
"id",
"title",
"description",
"acceptanceCriteria",
"priority",
"passes",
"milestone",
"executionMode",
"labels",
"triage",
"evidenceClass",
"evidencePath",
"hardware",
"model",
"upstream",
"dependsOn",
"notes",
"blocks"
],
"optionalStoryFields": [
"completionNotes"
],
"idRange": "DGR-017..DGR-071 inclusive",
"triageValues": [
"ready-for-agent",
"ready-for-human"
],
"executionModeValues": [
"AFK",
"HITL"
],
"evidenceClassValues": [
"model-free",
"fixture",
"real-model",
"real-hardware",
"release"
],
"hardwareValues": [
"none",
"optional",
"required"
],
"upstreamValues": [
"yes",
"no",
"conditional"
],
"typeDerivation": "A story type is derived from its type:<value> label; gate:<value> stories derive release-gate.",
"labelConventions": "Reserved prefixes include type:, priority:, area:, gate:, and ready-for-agent/ready-for-human triage labels; at most one type: and one priority: label are allowed.",
"generatedArtifactDisclaimer": "<!-- GENERATED FROM prd.json \u2014 DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->",
"dependencyRules": "Dependencies reference existing numerically earlier IDs; graph is acyclic. blocks is mechanically derived from dependsOn.",
"authorityRule": "Generated issue files state that prd.json is authoritative and cannot independently claim completion or override it."
},
"milestones": [
{
"id": "M0",
"name": "Truth and contracts",
"stories": "DGR-017..DGR-020",
"outcome": "Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline."
},
{
"id": "M1",
"name": "Protocol and native substrate",
"stories": "DGR-021..DGR-033",
"outcome": "Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker."
},
{
"id": "M2",
"name": "Dense vertical proof",
"stories": "DGR-034..DGR-043",
"outcome": "Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing."
},
{
"id": "M3",
"name": "DeepSeek V4 Flash alpha",
"stories": "DGR-044..DGR-054",
"outcome": "Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off."
},
{
"id": "M4",
"name": "Performance and beta hardening",
"stories": "DGR-055..DGR-067",
"outcome": "Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix."
},
{
"id": "M5",
"name": "Release and maintenance",
"stories": "DGR-068..DGR-071",
"outcome": "Reproducible packages, upstream collaboration, beta decision, and sustainable recertification."
}
],
"supersededStories": {
"DGR-001": {
"newIds": [
"DGR-019",
"DGR-020",
"DGR-054",
"DGR-070"
],
"disposition": "Benchmark scaffold/evidence may be audited; old pass state is void."
},
"DGR-002": {
"newIds": [
"DGR-021",
"DGR-022",
"DGR-023",
"DGR-024"
],
"disposition": "Split protocol, lifecycle, code generation, and fake transport."
},
"DGR-003": {
"newIds": [
"DGR-025"
],
"disposition": "Replaced by exact artifact/runtime compatibility identity."
},
"DGR-004": {
"newIds": [
"DGR-027",
"DGR-028",
"DGR-029",
"DGR-030",
"DGR-071"
],
"disposition": "Split provenance, patch stack, builds, and maintenance."
},
"DGR-005": {
"newIds": [
"DGR-034",
"DGR-045"
],
"disposition": "Dense and V4 ownership separated."
},
"DGR-006": {
"newIds": [
"DGR-031",
"DGR-035",
"DGR-036",
"DGR-046",
"DGR-047",
"DGR-048",
"DGR-049"
],
"disposition": "Engine, dense boundary, V4 typed boundary, and local-state adapters separated."
},
"DGR-007": {
"newIds": [
"DGR-038",
"DGR-049"
],
"disposition": "Replaced by session/epoch-keyed local KV and V4 auxiliary state."
},
"DGR-008": {
"newIds": [
"DGR-032",
"DGR-033",
"DGR-037"
],
"disposition": "Old implementation/evidence absent; no completion credit transfers."
},
"DGR-009": {
"newIds": [
"DGR-040",
"DGR-041",
"DGR-042",
"DGR-043"
],
"disposition": "Supervision, registration, relay, and routing-input integration separated."
},
"DGR-010": {
"newIds": [
"DGR-036",
"DGR-039",
"DGR-052"
],
"disposition": "Fixture, dense real acceptance, and V4 parity separated."
},
"DGR-011": {
"newIds": [
"DGR-053",
"DGR-061",
"DGR-062",
"DGR-067"
],
"disposition": "Replaced by scenario-based real 2\u20134, existing-routing 10+, real 10+, and backend certification."
},
"DGR-012": {
"newIds": [
"DGR-055",
"DGR-056",
"DGR-057"
],
"disposition": "Batching, admission/backpressure, and benchmarking separated."
},
"DGR-013": {
"newIds": [
"DGR-058",
"DGR-059"
],
"disposition": "Failure semantics and restart/re-prefill recovery separated."
},
"DGR-014": {
"newIds": [
"DGR-019",
"DGR-054",
"DGR-070"
],
"disposition": "Replaced by immutable performance, alpha, and beta gates."
},
"DGR-015": {
"newIds": [
"DGR-044",
"DGR-045",
"DGR-046",
"DGR-047",
"DGR-048",
"DGR-049",
"DGR-050",
"DGR-051",
"DGR-052",
"DGR-053",
"DGR-054",
"DGR-060",
"DGR-065",
"DGR-066",
"DGR-067"
],
"disposition": "Qwen target superseded by DeepSeek V4 Flash; no old completion transfers."
},
"DGR-016": {
"newIds": [
"DGR-069",
"DGR-071"
],
"disposition": "Upstream collaboration and ongoing maintenance separated."
}
},
"userStories": [ "userStories": [
{ {
"id": "DGR-017", "id": "DGR-017",
@@ -106,7 +364,7 @@
"Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.", "Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.",
"Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.", "Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.",
"Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.", "Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.",
"Treat quants and 24/10+ stage counts only as named certification scenarios; no product logic may hardcode them.", "Treat quants and 2\u20134/10+ stage counts only as named certification scenarios; no product logic may hardcode them.",
"Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.", "Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
@@ -262,12 +520,12 @@
"acceptanceCriteria": [ "acceptanceCriteria": [
"Pin protoc, gRPC, and plugin versions or declare a verified compatible range.", "Pin protoc, gRPC, and plugin versions or declare a verified compatible range.",
"Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.", "Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.",
"Add PythonC++ round-trip and descriptor compatibility tests.", "Add Python\u2194C++ round-trip and descriptor compatibility tests.",
"A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.", "A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": true, "passes": true,
"notes": "Completed from Gitea #7 after controller provisioned and exercised the exact Python/C++ toolchains. Verified deterministic generation, native CMake/CTest, PythonC++ byte parity, compileall, and diff checks; fixed relative bootstrap prefix resolution.", "notes": "Completed from Gitea #7 after controller provisioned and exercised the exact Python/C++ toolchains. Verified deterministic generation, native CMake/CTest, Python\u2194C++ byte parity, compileall, and diff checks; fixed relative bootstrap prefix resolution.",
"completionNotes": "Verified exact grpcio-tools 1.82.1, Protobuf 33.1, Abseil 20250814.1, and gRPC C++ 1.82.1 at commit acccf84c0df20487d64101f528e5d426541ca4e5. Mandatory Python/C++ message and service generation, native CTest, deterministic regeneration, and byte-for-byte Python/C++ parity passed; see evidence/DGR-023/README.md.", "completionNotes": "Verified exact grpcio-tools 1.82.1, Protobuf 33.1, Abseil 20250814.1, and gRPC C++ 1.82.1 at commit acccf84c0df20487d64101f528e5d426541ca4e5. Mandatory Python/C++ message and service generation, native CTest, deterministic regeneration, and byte-for-byte Python/C++ parity passed; see evidence/DGR-023/README.md.",
"blocks": [ "blocks": [
"DGR-024", "DGR-024",
@@ -352,7 +610,7 @@
"DGR-041", "DGR-041",
"DGR-044" "DGR-044"
], ],
"completionNotes": "Completed 2026-07-17. Verified the live DGR-003-lineage identity core against every criterion: node packages/node/meshnet_node/runtime_recipe.py and the independent tracker packages/tracker/meshnet_tracker/recipe.py (pinned together by tests/data/recipe_fingerprint_vectors.json) fingerprint the source artifact SHA, tokenizer pin, architecture adapter and config digest, boundary/protocol schema versions, backend, weight quant, activation/compute dtypes, and KV dtype/layout under domain-separated digests; shards bind to exact half-open ranges with no topology or quant constants; route/handshake/session checks fail closed with structured mismatch reasons; recipes stay registered-but-dark in the tracker CertificationLedger until a real >=2-distinct-node whole-model distributed forward certifies them. Closed the one open criterion gap (runtime pin/patch stack): new packages/node/meshnet_node/runtime_pin.py derives the runtime_version axis from the DGR-027 lock manifest exact upstream commit plus a digest over the ordered patch-stack bytes failing closed on any UPSTREAM_LOCK.json/UPSTREAM_COMMIT/series/SHA256SUMS/patch-byte disagreement, and both identity implementations now reject a moving runtime_version reference. Tests: tests/test_runtime_pin_identity.py (17 passed) plus 196 passing impacted identity/admission/native-emission tests; python -m compileall and git diff --check clean. Also repaired backlog consistency left by prior sessions: added the missing DGR-022/DGR-027 completionNotes, regenerated the DGR-022/025/027 issue projections, and relocated three pre-DGR legacy GLM alpha issue files to issues/legacy/." "completionNotes": "Completed 2026-07-17. Verified the live DGR-003-lineage identity core against every criterion: node packages/node/meshnet_node/runtime_recipe.py and the independent tracker packages/tracker/meshnet_tracker/recipe.py (pinned together by tests/data/recipe_fingerprint_vectors.json) fingerprint the source artifact SHA, tokenizer pin, architecture adapter and config digest, boundary/protocol schema versions, backend, weight quant, activation/compute dtypes, and KV dtype/layout under domain-separated digests; shards bind to exact half-open ranges with no topology or quant constants; route/handshake/session checks fail closed with structured mismatch reasons; recipes stay registered-but-dark in the tracker CertificationLedger until a real >=2-distinct-node whole-model distributed forward certifies them. Closed the one open criterion gap (runtime pin/patch stack): new packages/node/meshnet_node/runtime_pin.py derives the runtime_version axis from the DGR-027 lock manifest \u2014 exact upstream commit plus a digest over the ordered patch-stack bytes \u2014 failing closed on any UPSTREAM_LOCK.json/UPSTREAM_COMMIT/series/SHA256SUMS/patch-byte disagreement, and both identity implementations now reject a moving runtime_version reference. Tests: tests/test_runtime_pin_identity.py (17 passed) plus 196 passing impacted identity/admission/native-emission tests; python -m compileall and git diff --check clean. Also repaired backlog consistency left by prior sessions: added the missing DGR-022/DGR-027 completionNotes, regenerated the DGR-022/025/027 issue projections, and relocated three pre-DGR legacy GLM alpha issue files to issues/legacy/."
}, },
{ {
"id": "DGR-026", "id": "DGR-026",
@@ -419,7 +677,7 @@
"Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.", "Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.",
"Fetch tooling verifies identity before use and refuses an unpinned branch/tag.", "Fetch tooling verifies identity before use and refuses an unpinned branch/tag.",
"Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.", "Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.",
"Offline reuse is supported only after the cached trees exact identity is verified.", "Offline reuse is supported only after the cached tree\u2019s exact identity is verified.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": true, "passes": true,
@@ -656,12 +914,13 @@
"The worker exposes neither llama.cpp RPC nor arbitrary graph execution.", "The worker exposes neither llama.cpp RPC nor arbitrary graph execution.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-036", "DGR-036",
"DGR-040" "DGR-040"
] ],
"completionNotes": "Cross-review (Codex GPT-5.5) BLOCK repaired in worktree distributed-gguf-opus. Root protocol defects fixed in the native worker: (1) chunk/decode now fail closed before SessionOpen via a per-session opened flag (terminal ERROR_CODE_INTERNAL), so no activation bypasses lifecycle/cancellation/epoch/flow-control state even when an out-of-band Cancel created placeholder state; (2) flow control is negotiated with strict worker bounds (ShardRuntimeServiceImpl::NegotiateFlow mirrors native_protocol/codec.py negotiate_flow_control) and the negotiated per-session max_chunk_bytes is enforced on every bundle instead of trusting the peer proposal; (3) an in-stream ReleaseSignal now erases session state immediately; (4) SessionOpen rejects incompatible schema, artifact/recipe fingerprint, and shard-range identity and reports the worker own served fingerprint rather than echoing the caller. Nine regression tests added. Real gates on the rebuilt pinned-gRPC binary: cmake --build exit 0; ctest 2/2 passed (shard_worker_selftest, shard_protocol_conformance); tests/test_native_shard_worker.py 27 passed; DGR-024 harness + native protocol 63 passed; compileall exit 0; git diff --check clean; ldd/nm show 0 llama/ggml linkage. Evidence: .scratch/distributed-gguf-runtime/evidence/DGR-033/README.md."
}, },
{ {
"id": "DGR-034", "id": "DGR-034",
@@ -695,13 +954,14 @@
"Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.", "Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-035", "DGR-035",
"DGR-037", "DGR-037",
"DGR-051" "DGR-051"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-035", "id": "DGR-035",
@@ -735,13 +995,14 @@
"Uncertified architectures and incompatible boundary schemas fail closed.", "Uncertified architectures and incompatible boundary schemas fail closed.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-036", "DGR-036",
"DGR-037", "DGR-037",
"DGR-069" "DGR-069"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-036", "id": "DGR-036",
@@ -774,11 +1035,12 @@
"Evidence distinguishes deterministic fixture proof from opt-in real-model proof.", "Evidence distinguishes deterministic fixture proof from opt-in real-model proof.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-039" "DGR-039"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-037", "id": "DGR-037",
@@ -814,14 +1076,15 @@
"Graceful shutdown releases model/session resources; injected process death is observable and bounded.", "Graceful shutdown releases model/session resources; injected process death is observable and bounded.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-038", "DGR-038",
"DGR-039", "DGR-039",
"DGR-040", "DGR-040",
"DGR-051" "DGR-051"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-038", "id": "DGR-038",
@@ -853,14 +1116,15 @@
"Release/eviction returns memory to the configured budget without affecting other sessions.", "Release/eviction returns memory to the configured budget without affecting other sessions.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-039", "DGR-039",
"DGR-052", "DGR-052",
"DGR-055", "DGR-055",
"DGR-069" "DGR-069"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-039", "id": "DGR-039",
@@ -894,11 +1158,12 @@
"Killing one worker returns a bounded structured failure rather than hanging.", "Killing one worker returns a bounded structured failure rather than hanging.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-054" "DGR-054"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-040", "id": "DGR-040",
@@ -931,14 +1196,15 @@
"Tests use the fake worker and deterministic crash injection.", "Tests use the fake worker and deterministic crash injection.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-041", "DGR-041",
"DGR-042", "DGR-042",
"DGR-055", "DGR-055",
"DGR-058" "DGR-058"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-041", "id": "DGR-041",
@@ -971,11 +1237,12 @@
"Existing Transformers registration and route tests remain unchanged in behavior.", "Existing Transformers registration and route tests remain unchanged in behavior.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-043" "DGR-043"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-042", "id": "DGR-042",
@@ -1009,7 +1276,8 @@
"Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.", "Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"completionNotes": "Completed by agent",
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-054", "DGR-054",
@@ -1046,14 +1314,15 @@
"Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.", "Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.",
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff." "Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
], ],
"passes": false, "passes": true,
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md; prd.json is authoritative.", "notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md; prd.json is authoritative.",
"blocks": [ "blocks": [
"DGR-053", "DGR-053",
"DGR-054", "DGR-054",
"DGR-059", "DGR-059",
"DGR-061" "DGR-061"
] ],
"completionNotes": "Completed by agent"
}, },
{ {
"id": "DGR-044", "id": "DGR-044",
@@ -1158,7 +1427,7 @@
"triage": "ready-for-agent", "triage": "ready-for-agent",
"description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md`, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.", "description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md`, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.",
"acceptanceCriteria": [ "acceptanceCriteria": [
"Define a versioned named bundle for the mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.", "Define a versioned named bundle for the mHC 4\u00d74096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.",
"Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.", "Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.",
"Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.", "Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.",
"Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.", "Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.",
@@ -1197,7 +1466,7 @@
"triage": "ready-for-agent", "triage": "ready-for-agent",
"description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.", "description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.",
"acceptanceCriteria": [ "acceptanceCriteria": [
"Represent and validate the upstream V4 4×4096 mHC boundary without flattening semantic axes.", "Represent and validate the upstream V4 4\u00d74096 mHC boundary without flattening semantic axes.",
"Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.", "Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.",
"Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.", "Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.",
"Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.", "Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.",
@@ -1407,7 +1676,7 @@
}, },
{ {
"id": "DGR-053", "id": "DGR-053",
"title": "Certify a real 24-stage V4 route", "title": "Certify a real 2\u20134-stage V4 route",
"priority": 37, "priority": 37,
"milestone": "M3", "milestone": "M3",
"executionMode": "HITL", "executionMode": "HITL",
@@ -1432,7 +1701,7 @@
"triage": "ready-for-human", "triage": "ready-for-human",
"description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md`, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.", "description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md`, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.",
"acceptanceCriteria": [ "acceptanceCriteria": [
"Run one documented 24-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.", "Run one documented 2\u20134-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.",
"Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.", "Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.",
"Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.", "Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.",
"Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.", "Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.",
@@ -1711,7 +1980,7 @@
"DGR-058" "DGR-058"
], ],
"triage": "ready-for-agent", "triage": "ready-for-agent",
"description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md`, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4s KV and auxiliary state remain correct and bounded at long contexts.", "description": "Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md`, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4\u2019s KV and auxiliary state remain correct and bounded at long contexts.",
"acceptanceCriteria": [ "acceptanceCriteria": [
"Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.", "Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.",
"Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.", "Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.",
@@ -2164,6 +2433,6 @@
} }
], ],
"metadata": { "metadata": {
"updatedAt": "2026-07-23T08:09:16.081Z" "updatedAt": "2026-07-23T08:09:17.286Z"
} }
} }

View File

@@ -20,6 +20,14 @@ from .native_protocol import (
pb, pb,
validate_tail_result, validate_tail_result,
) )
from .shard_engine import BoundaryBundle, EngineTensor
# This is deliberately an execution-boundary name, not a transport name. It
# identifies the value *before* final norm/output projection. A future wire
# codec may rename its field, but cannot reinterpret this value as logits.
DENSE_LLAMA_ARCHITECTURE = "dense-llama"
DENSE_RESIDUAL_BOUNDARY_V1 = "dense.residual.v1"
class Architecture(str, Enum): class Architecture(str, Enum):
@@ -63,6 +71,11 @@ class TailOutput:
raise ProtocolError("sampled token id must be non-negative") raise ProtocolError("sampled token id must be non-negative")
return cls("sampled_token", token_id) return cls("sampled_token", token_id)
@classmethod
def logits(cls, logits: object) -> "TailOutput":
"""Return raw logits under the explicit tail-only output contract."""
return cls("logits", logits)
@dataclass(frozen=True) @dataclass(frozen=True)
class TypedTailResult: class TypedTailResult:
@@ -148,28 +161,153 @@ class ArchitectureBoundaryAdapter:
raise ProtocolError("tail result architecture does not match certified adapter") raise ProtocolError("tail result architecture does not match certified adapter")
if not identity.request_id or not identity.runtime_recipe_digest: if not identity.request_id or not identity.runtime_recipe_digest:
raise ProtocolError("tail result requires exact request and recipe identity") raise ProtocolError("tail result requires exact request and recipe identity")
if output.kind != "sampled_token": if output.kind == "sampled_token":
if not isinstance(output.value, int):
raise ProtocolError("sampled tail output must carry an integer token id")
message = pb.TailResult(
identity=pb.RequestRecipeIdentity(
request_id=identity.request_id,
runtime_recipe_digest=identity.runtime_recipe_digest,
chat_template_id=identity.chat_template_id,
chat_template_version=identity.chat_template_version,
reasoning_mode=identity.reasoning_mode,
architecture=self.protocol_architecture,
),
sampling=pb.SamplingParameters(
temperature=sampling.temperature,
top_p=sampling.top_p,
top_k=sampling.top_k,
seed=sampling.seed,
greedy=sampling.temperature == 0.0,
),
sampled_token_id=output.value,
)
elif output.kind == "logits":
if not isinstance(output.value, pb.TensorBundle):
raise ProtocolError("logits tail output must carry a TensorBundle")
# Validate the logits bundle before putting it in the result; this
# rejects an incompatible boundary schema rather than passing an
# opaque tensor on to sampling.
from .native_protocol import decode_bundle
decode_bundle(output.value)
message = pb.TailResult(
identity=pb.RequestRecipeIdentity(
request_id=identity.request_id,
runtime_recipe_digest=identity.runtime_recipe_digest,
chat_template_id=identity.chat_template_id,
chat_template_version=identity.chat_template_version,
reasoning_mode=identity.reasoning_mode,
architecture=self.protocol_architecture,
),
sampling=pb.SamplingParameters(
temperature=sampling.temperature,
top_p=sampling.top_p,
top_k=sampling.top_k,
seed=sampling.seed,
greedy=sampling.temperature == 0.0,
),
logits=output.value,
)
else:
raise ProtocolError("uncertified tail output kind") raise ProtocolError("uncertified tail output kind")
message = pb.TailResult(
identity=pb.RequestRecipeIdentity(
request_id=identity.request_id,
runtime_recipe_digest=identity.runtime_recipe_digest,
chat_template_id=identity.chat_template_id,
chat_template_version=identity.chat_template_version,
reasoning_mode=identity.reasoning_mode,
architecture=self.protocol_architecture,
),
sampling=pb.SamplingParameters(
temperature=sampling.temperature,
top_p=sampling.top_p,
top_k=sampling.top_k,
seed=sampling.seed,
greedy=sampling.temperature == 0.0,
),
sampled_token_id=int(output.value),
)
validate_tail_result(message) validate_tail_result(message)
return TypedTailResult(identity, sampling, "sampled_token_id", message) return TypedTailResult(identity, sampling, message.WhichOneof("output"), message)
@dataclass(frozen=True)
class DenseLayerRange:
"""A certified, inclusive dense-Llama range within one loaded model."""
start_layer: int
end_layer: int
total_layers: int
architecture: str = DENSE_LLAMA_ARCHITECTURE
def __post_init__(self) -> None:
if self.architecture != DENSE_LLAMA_ARCHITECTURE:
raise ProtocolError("dense boundary executor only certifies dense-llama")
if self.start_layer < 0 or self.end_layer < self.start_layer:
raise ProtocolError("dense range is empty or inverted")
if self.total_layers <= self.end_layer:
raise ProtocolError("dense range lies outside the model")
@property
def is_head(self) -> bool:
return self.start_layer == 0
@property
def is_tail(self) -> bool:
return self.end_layer == self.total_layers - 1
class DenseRangeBoundaryExecutor:
"""Execute one dense range without leaking endpoint ownership.
``run_layers`` owns only the local transformer blocks and receives/returns
the raw residual. It never receives a final norm/head callback. Only a
tail range receives ``tail_output``; consequently row pruning and logits
projection cannot accidentally happen before the final stage.
"""
def __init__(
self,
layer_range: DenseLayerRange,
*,
embed_tokens: Callable[[tuple[int, ...]], EngineTensor],
run_layers: Callable[[EngineTensor], EngineTensor],
tail_output: Callable[[EngineTensor], TailOutput] | None = None,
) -> None:
if layer_range.is_tail != (tail_output is not None):
raise ProtocolError("only a dense tail range may own final norm/output")
self._range = layer_range
self._embed_tokens = embed_tokens
self._run_layers = run_layers
self._tail_output = tail_output
def execute(
self,
*,
token_ids: tuple[int, ...] | None = None,
boundary: BoundaryBundle | None = None,
) -> BoundaryBundle | TailOutput:
if self._range.is_head:
if token_ids is None or boundary is not None or not token_ids:
raise ProtocolError("dense head accepts non-empty token ids and no boundary bundle")
residual = self._embed_tokens(token_ids)
else:
if token_ids is not None or boundary is None:
raise ProtocolError("dense middle/tail requires a named residual boundary bundle")
residual = self._residual_from_boundary(boundary)
residual = self._run_layers(residual)
if residual.name != HIDDEN_STATES:
raise ProtocolError("dense range must return hidden_states residual")
if self._range.is_tail:
assert self._tail_output is not None
output = self._tail_output(residual)
if output.kind not in {"logits", "sampled_token"}:
raise ProtocolError("dense tail returned an uncertified output kind")
return output
# Do not normalize, project, sample, or prune rows here: this exact
# raw output becomes the next range's input.
return BoundaryBundle(
tensors=(residual,),
architecture=DENSE_LLAMA_ARCHITECTURE,
boundary_point=DENSE_RESIDUAL_BOUNDARY_V1,
)
@staticmethod
def _residual_from_boundary(boundary: BoundaryBundle) -> EngineTensor:
if boundary.architecture != DENSE_LLAMA_ARCHITECTURE:
raise ProtocolError("boundary architecture is not certified dense-llama")
if boundary.boundary_point != DENSE_RESIDUAL_BOUNDARY_V1:
raise ProtocolError("incompatible dense residual boundary schema")
if len(boundary.tensors) != 1 or boundary.tensors[0].name != HIDDEN_STATES:
raise ProtocolError("dense residual boundary requires exactly one hidden_states tensor")
return boundary.tensors[0]
_ADAPTERS = { _ADAPTERS = {

View File

@@ -322,6 +322,105 @@ class BackendIdentity:
) )
@dataclass(frozen=True)
class ExecutionCapacity:
"""Backend-neutral limits reserved for one registered capability.
The optional shape preserves existing Transformers reports unchanged while
allowing a native Shard to state its measured/admitted resource envelope.
"""
memory_capacity_bytes: int | None = None
kv_capacity_tokens: int | None = None
max_concurrent_sessions: int | None = None
def __post_init__(self) -> None:
for name in (
"memory_capacity_bytes",
"kv_capacity_tokens",
"max_concurrent_sessions",
):
value = getattr(self, name)
if value is not None:
_require_int(value, f"capacity.{name}", 1)
def to_dict(self) -> dict:
return {
"memory_capacity_bytes": self.memory_capacity_bytes,
"kv_capacity_tokens": self.kv_capacity_tokens,
"max_concurrent_sessions": self.max_concurrent_sessions,
}
@classmethod
def from_dict(cls, data: Any) -> ExecutionCapacity:
doc = _as_mapping(data, "capacity")
values: dict[str, int | None] = {}
for name in (
"memory_capacity_bytes",
"kv_capacity_tokens",
"max_concurrent_sessions",
):
value = doc.get(name)
values[name] = None if value is None else _require_int(value, f"capacity.{name}", 1)
return cls(**values)
@dataclass(frozen=True)
class RoutingMeasurements:
"""Optional backend-neutral observations for existing tracker routing.
These are measurements, rather than policy: the tracker continues to own
admission, route formation, load balancing, and certification. Keeping
this block optional makes it additive for existing Transformers reports.
"""
tokens_per_second: float | None = None
queue_depth: int | None = None
seam_latency_ms: float | None = None
healthy: bool | None = None
reliability: float | None = None
def __post_init__(self) -> None:
for name in ("tokens_per_second", "seam_latency_ms"):
value = getattr(self, name)
if value is not None and (
isinstance(value, bool) or not isinstance(value, (int, float)) or value < 0
):
raise CapabilityReportError(f"routing.{name} must be a non-negative number")
if self.tokens_per_second == 0:
raise CapabilityReportError("routing.tokens_per_second must be positive when present")
if self.queue_depth is not None:
_require_int(self.queue_depth, "routing.queue_depth", 0)
if self.healthy is not None and not isinstance(self.healthy, bool):
raise CapabilityReportError("routing.healthy must be a boolean")
if self.reliability is not None and (
isinstance(self.reliability, bool)
or not isinstance(self.reliability, (int, float))
or not 0.0 <= self.reliability <= 1.0
):
raise CapabilityReportError("routing.reliability must be a number from 0 to 1")
def to_dict(self) -> dict:
return {
"tokens_per_second": self.tokens_per_second,
"queue_depth": self.queue_depth,
"seam_latency_ms": self.seam_latency_ms,
"healthy": self.healthy,
"reliability": self.reliability,
}
@classmethod
def from_dict(cls, data: Any) -> RoutingMeasurements:
doc = _as_mapping(data, "routing")
return cls(
tokens_per_second=doc.get("tokens_per_second"),
queue_depth=doc.get("queue_depth"),
seam_latency_ms=doc.get("seam_latency_ms"),
healthy=doc.get("healthy"),
reliability=doc.get("reliability"),
)
def _as_mapping(data: Any, field_name: str) -> Mapping[str, Any]: def _as_mapping(data: Any, field_name: str) -> Mapping[str, Any]:
if not isinstance(data, Mapping): if not isinstance(data, Mapping):
raise CapabilityReportError( raise CapabilityReportError(
@@ -353,6 +452,8 @@ class CapabilityReport:
diagnostics: tuple[str, ...] = () diagnostics: tuple[str, ...] = ()
schema_version: int = CAPABILITY_SCHEMA_VERSION schema_version: int = CAPABILITY_SCHEMA_VERSION
identity: ShardIdentity | None = None identity: ShardIdentity | None = None
capacity: ExecutionCapacity | None = None
routing: RoutingMeasurements | None = None
def __post_init__(self) -> None: def __post_init__(self) -> None:
if self.status not in VALID_STATUSES: if self.status not in VALID_STATUSES:
@@ -410,6 +511,10 @@ class CapabilityReport:
} }
if self.identity is not None: if self.identity is not None:
doc["identity"] = self.identity.to_dict() doc["identity"] = self.identity.to_dict()
if self.capacity is not None:
doc["capacity"] = self.capacity.to_dict()
if self.routing is not None:
doc["routing"] = self.routing.to_dict()
return doc return doc
def to_json(self, indent: int | None = None) -> str: def to_json(self, indent: int | None = None) -> str:
@@ -451,6 +556,12 @@ class CapabilityReport:
identity=( identity=(
None if raw_identity is None else ShardIdentity.from_dict(raw_identity) None if raw_identity is None else ShardIdentity.from_dict(raw_identity)
), ),
capacity=(
None if doc.get("capacity") is None else ExecutionCapacity.from_dict(doc["capacity"])
),
routing=(
None if doc.get("routing") is None else RoutingMeasurements.from_dict(doc["routing"])
),
) )
@classmethod @classmethod
@@ -486,6 +597,8 @@ def build_capability_report(
validated_at: float | None = None, validated_at: float | None = None,
environ: Mapping[str, str] | None = None, environ: Mapping[str, str] | None = None,
identity: ShardIdentity | None = None, identity: ShardIdentity | None = None,
capacity: ExecutionCapacity | None = None,
routing: RoutingMeasurements | None = None,
) -> CapabilityReport: ) -> CapabilityReport:
"""Assemble a report from flat validation results. """Assemble a report from flat validation results.
@@ -518,4 +631,6 @@ def build_capability_report(
duration_ms=duration_ms, duration_ms=duration_ms,
diagnostics=sanitize_diagnostics(diagnostics, environ), diagnostics=sanitize_diagnostics(diagnostics, environ),
identity=identity, identity=identity,
capacity=capacity,
routing=routing,
) )

View File

@@ -0,0 +1,298 @@
"""Native activation transport over direct gRPC or the existing relay RPC.
This is deliberately a *seam adapter*, not a new relay protocol. Direct
peers use one generated ``ShardRuntime.Session`` bidi stream for the lifetime
of a Route Session. A relayed peer uses the relay's existing HTTP-shaped
binary-body contract: each body is exactly a serialized ``SessionRequest`` or
``SessionResponse``. The relay only routes those bytes and restores its own
request id; it does not deserialize a native frame.
The correlation headers are duplicated outside the opaque frame solely for
the existing tracker/relay observability and billing path. The authoritative
work, route, epoch, deadline, and cancellation information remains in the
versioned protobuf frame and is validated before it is sent.
"""
from __future__ import annotations
from collections.abc import Callable, Iterator
from dataclasses import dataclass
from queue import Empty, Full, Queue
import threading
import time
from typing import Protocol
from .native_protocol import pb
NATIVE_RELAY_PATH = "/native/session"
NATIVE_FRAME_CONTENT_TYPE = "application/x-protobuf"
class NativeActivationSeamError(RuntimeError):
"""The activation seam cannot safely continue this Route Session."""
class NativeActivationBufferFull(NativeActivationSeamError):
"""The caller exceeded the negotiated local hand-off buffer."""
class NativeActivationDisconnected(NativeActivationSeamError):
"""A direct or relay transport disconnected with an uncertain outcome."""
class RelayRequest(Protocol):
"""The existing ``_RelayHopClient.request`` shape, kept dependency-free."""
def __call__(
self, path: str, body: bytes, headers: dict[str, str]
) -> tuple[int, dict[str, str], bytes]: ...
@dataclass(frozen=True)
class NativeFrameContext:
"""Correlation owned by Meshnet around one opaque native frame."""
request_id: str
node_id: str
route_session_id: str
route_epoch: int
work_id: str = ""
deadline_unix_nanos: int = 0
def __post_init__(self) -> None:
if not self.request_id or not self.node_id or not self.route_session_id:
raise ValueError("request, node, and Route Session identities are required")
if self.route_epoch < 0 or self.deadline_unix_nanos < 0:
raise ValueError("route epoch and deadline must be non-negative")
def headers(self) -> dict[str, str]:
"""Headers retained by the existing relay/Tracker accounting path."""
return {
"Content-Type": NATIVE_FRAME_CONTENT_TYPE,
"X-Meshnet-Native-Frame": "shard-runtime/v1",
"X-Meshnet-Request-Id": self.request_id,
"X-Meshnet-Node-Id": self.node_id,
"X-Meshnet-Session": self.route_session_id,
"X-Meshnet-Route-Epoch": str(self.route_epoch),
"X-Meshnet-Work-Id": self.work_id,
"X-Meshnet-Deadline-Unix-Nanos": str(self.deadline_unix_nanos),
# The relay request id is restored on reply and is intentionally
# distinct from the caller/billing request id above.
"X-Meshnet-Activation-Id": self.request_id,
}
@dataclass(frozen=True)
class NativeSeamTelemetry:
transport: str
request_id: str
node_id: str
work_id: str
request_bytes: int
response_bytes: int
elapsed_seconds: float
TelemetrySink = Callable[[NativeSeamTelemetry], None]
def _request_identity(request: pb.SessionRequest) -> tuple[str, int, str, int]:
kind = request.WhichOneof("kind")
if kind == "open":
return request.open.route_session_id, request.open.route_epoch, "", 0
if kind == "chunk":
item = request.chunk.envelope
return item.route_session_id, item.route_epoch, item.work_id, item.deadline_unix_nanos
if kind == "decode":
# DecodeStep relies on the already opened Route Session, while work
# identity/deadline are carried on every decode frame.
return "", 0, request.decode.work_id, request.decode.deadline_unix_nanos
if kind in {"cancel", "release"}:
item = getattr(request, kind)
return item.route_session_id, item.route_epoch, item.work_id, 0
if kind == "flow_control":
return "", 0, "", 0
raise NativeActivationSeamError("native SessionRequest has no frame kind")
def _validate_request(request: pb.SessionRequest, context: NativeFrameContext) -> None:
if request.ByteSize() == 0:
raise NativeActivationSeamError("empty native SessionRequest is not a versioned frame")
route_session, epoch, work_id, deadline = _request_identity(request)
if route_session and route_session != context.route_session_id:
raise NativeActivationSeamError("native frame Route Session differs from seam context")
if route_session and epoch != context.route_epoch:
raise NativeActivationSeamError("native frame route epoch differs from seam context")
if context.work_id and work_id and work_id != context.work_id:
raise NativeActivationSeamError("native frame work identity differs from seam context")
if context.deadline_unix_nanos and deadline and deadline != context.deadline_unix_nanos:
raise NativeActivationSeamError("native frame deadline differs from seam context")
def _response_work_id(response: pb.SessionResponse) -> str:
kind = response.WhichOneof("kind")
if kind == "chunk":
return response.chunk.envelope.work_id
if kind == "ack":
return response.ack.work_id
if kind == "status":
return response.status.work_id
return ""
class NativeActivationSeam:
"""One Route-Session-to-worker seam with bounded direct buffering.
``direct_stub`` is the generated ``ShardRuntimeStub`` and is selected when
it is available. ``relay_request`` has the exact signature of the
existing persistent relay client; no relay server or bridge API changes
are needed. Relay calls are intentionally not retried: a failed send may
already have mutated downstream Hot KV state.
"""
def __init__(
self,
context: NativeFrameContext,
*,
direct_stub=None,
relay_request: RelayRequest | None = None,
max_buffered_frames: int = 8,
telemetry: TelemetrySink | None = None,
) -> None:
if (direct_stub is None) == (relay_request is None):
raise ValueError("provide exactly one of direct_stub or relay_request")
if max_buffered_frames < 1:
raise ValueError("max_buffered_frames must be positive")
self.context = context
self._direct_stub = direct_stub
self._relay_request = relay_request
self._telemetry = telemetry
self._closed = False
self._failure: BaseException | None = None
self._responses: Queue[pb.SessionResponse | BaseException] = Queue(maxsize=max_buffered_frames)
self._requests: Queue[pb.SessionRequest | object] | None = None
self._thread: threading.Thread | None = None
self._stop = object()
if direct_stub is not None:
self._requests = Queue(maxsize=max_buffered_frames)
self._thread = threading.Thread(target=self._run_direct, daemon=True, name="native-activation-grpc")
self._thread.start()
@property
def transport(self) -> str:
return "direct-grpc" if self._direct_stub is not None else "relay"
def _direct_requests(self) -> Iterator[pb.SessionRequest]:
assert self._requests is not None
while True:
item = self._requests.get()
if item is self._stop:
return
assert isinstance(item, pb.SessionRequest)
yield item
def _run_direct(self) -> None:
try:
assert self._direct_stub is not None
for response in self._direct_stub.Session(self._direct_requests()):
self._put_response(response)
except BaseException as exc:
self._failure = exc
self._put_response(exc)
def _put_response(self, value: pb.SessionResponse | BaseException) -> None:
# A worker may finish while a caller is abandoning the session. Do not
# let an unconsumed response turn into an unbounded producer queue.
try:
self._responses.put(value, timeout=0.1)
except Full:
self._failure = NativeActivationBufferFull("native response buffer is full")
def send(self, request: pb.SessionRequest) -> pb.SessionResponse | None:
"""Send one already-versioned protobuf frame without rewriting it."""
if self._closed:
raise NativeActivationDisconnected("native activation seam is closed")
if self._failure is not None:
raise NativeActivationDisconnected("native activation stream failed") from self._failure
_validate_request(request, self.context)
frame = request.SerializeToString()
if self._direct_stub is not None:
assert self._requests is not None
try:
self._requests.put_nowait(request)
except Full as exc:
raise NativeActivationBufferFull("native direct request buffer is full") from exc
return None
assert self._relay_request is not None
started = time.monotonic()
try:
status, _, response_frame = self._relay_request(NATIVE_RELAY_PATH, frame, self.context.headers())
except Exception as exc:
self._closed = True
raise NativeActivationDisconnected("relay outcome is uncertain; refusing replay") from exc
if status != 200:
self._closed = True
raise NativeActivationDisconnected(f"relay native frame returned HTTP {status}")
response = pb.SessionResponse()
try:
response.ParseFromString(response_frame)
except Exception as exc:
self._closed = True
raise NativeActivationSeamError("relay returned a malformed native response frame") from exc
self._validate_response(response)
self._record(len(frame), len(response_frame), started)
return response
def receive(self, timeout: float | None = None) -> pb.SessionResponse:
"""Receive the next response from the one long-lived direct stream."""
if self._direct_stub is None:
raise NativeActivationSeamError("relay sends return their response synchronously")
try:
value = self._responses.get(timeout=timeout)
except Empty as exc:
raise TimeoutError("timed out waiting for native direct response") from exc
if isinstance(value, BaseException):
raise NativeActivationDisconnected("native direct stream disconnected") from value
self._validate_response(value)
# gRPC owns its framing, but this records the actual protobuf payload
# size at the seam for the same telemetry shape as relay.
self._record(0, len(value.SerializeToString()), time.monotonic())
return value
def cancel(self, reason: str = "cancelled") -> pb.SessionResponse | None:
"""Propagate cancellation through the same path and correlation fields."""
return self.send(pb.SessionRequest(cancel=pb.CancelSignal(
route_session_id=self.context.route_session_id,
route_epoch=self.context.route_epoch,
work_id=self.context.work_id,
reason=reason,
)))
def _validate_response(self, response: pb.SessionResponse) -> None:
work_id = _response_work_id(response)
if self.context.work_id and work_id and work_id != self.context.work_id:
raise NativeActivationSeamError("native response work identity differs from seam context")
def _record(self, request_bytes: int, response_bytes: int, started: float) -> None:
if self._telemetry is not None:
self._telemetry(NativeSeamTelemetry(
transport=self.transport, request_id=self.context.request_id,
node_id=self.context.node_id, work_id=self.context.work_id,
request_bytes=request_bytes, response_bytes=response_bytes,
elapsed_seconds=max(0.0, time.monotonic() - started),
))
def close(self) -> None:
if self._closed:
return
self._closed = True
if self._requests is not None:
try:
self._requests.put_nowait(self._stop)
except Full:
# The bounded queue is intentionally never expanded during
# shutdown; the worker will observe process/session teardown.
pass
if self._thread is not None:
self._thread.join(timeout=1.0)

View File

@@ -0,0 +1,171 @@
"""Register a verified native Shard through the ordinary capability contract.
This is intentionally an adapter, not a second tracker protocol. It converts
the native worker's immutable identity and enforced resource limits into the
same capability report every backend may submit. The tracker remains the sole
owner of certification and decides whether the visible registration is dark.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any
from .capability import ExecutionCapacity, RoutingMeasurements, build_capability_report
from .native_worker_supervisor import NativeWorkerProbe, NativeWorkerSpec, NativeWorkerSupervisor
from .runtime_recipe import ShardIdentity
class NativeRegistrationError(ValueError):
"""Native facts do not describe one coherent, registerable Shard."""
@dataclass(frozen=True)
class NativeShardRegistration:
"""One backend-neutral registration payload for a verified native Shard."""
endpoint: str
model_id: str
identity: ShardIdentity
worker: NativeWorkerSpec
probe: NativeWorkerProbe
device: str
capacity: ExecutionCapacity
duration_ms: int = 0
routing: RoutingMeasurements | None = None
def __post_init__(self) -> None:
if not self.endpoint:
raise NativeRegistrationError("native registration requires an endpoint")
if not self.model_id:
raise NativeRegistrationError("native registration requires a model id")
if not self.device:
raise NativeRegistrationError("native registration requires a device label")
if self.identity.artifact.artifact_id != self.model_id:
raise NativeRegistrationError("native registration model does not match its identity")
if self.identity.fingerprint.model_artifact_digest != self.worker.artifact_digest:
raise NativeRegistrationError("native worker artifact digest does not match its identity")
if self.identity.fingerprint.runtime_recipe_digest != self.worker.recipe_digest:
raise NativeRegistrationError("native worker recipe digest does not match its identity")
expected = (
self.worker.artifact_digest,
self.worker.recipe_digest,
self.worker.recipe_id,
self.worker.recipe_version,
self.worker.catalogue_version,
self.worker.shard_start,
self.worker.shard_end,
)
actual = (
self.probe.artifact_digest,
self.probe.recipe_digest,
self.probe.recipe_id,
self.probe.recipe_version,
self.probe.catalogue_version,
self.probe.shard_start,
self.probe.shard_end,
)
if actual != expected:
raise NativeRegistrationError("native worker probe differs from its startup identity/range")
if not self.probe.serving:
raise NativeRegistrationError("native worker is not serving; it cannot register a capability")
if (
self.identity.shard_start,
self.identity.shard_end,
self.identity.recipe.recipe_id,
self.identity.recipe.recipe_version,
self.identity.recipe.catalogue_version,
) != (
self.worker.shard_start,
self.worker.shard_end,
self.worker.recipe_id,
self.worker.recipe_version,
self.worker.catalogue_version,
):
raise NativeRegistrationError("native identity differs from worker range or recipe labels")
if self.identity.recipe.axes["backend_id"] == "":
raise NativeRegistrationError("native identity must name its backend")
def payload(self) -> dict[str, Any]:
"""Return the existing tracker registration shape with no native branch."""
report = build_capability_report(
model_id=self.model_id,
shard_start=self.identity.shard_start,
shard_end=self.identity.shard_end - 1,
recipe_id=self.identity.recipe.recipe_id,
recipe_version=self.identity.recipe.recipe_version,
catalogue_version=self.identity.recipe.catalogue_version,
backend_id=self.identity.recipe.axes["backend_id"],
device=self.device,
quantization=self.identity.recipe.axes["weight_quantization"],
model_config="sha256:" + self.identity.artifact.architecture_digest,
revision=self.identity.artifact.revision,
status="passed",
duration_ms=self.duration_ms,
identity=self.identity,
capacity=self.capacity,
routing=self.routing,
)
payload = {
"endpoint": self.endpoint,
"model": self.model_id.rsplit("/", 1)[-1],
"hf_repo": self.model_id,
"shard_start": self.identity.shard_start,
"shard_end": self.identity.shard_end - 1,
"recipe_id": self.identity.recipe.recipe_id,
"recipe_version": self.identity.recipe.recipe_version,
"capability_report": report.to_dict(),
# Existing tracker capacity fields are retained for placement views.
"ram_bytes": self.capacity.memory_capacity_bytes or 0,
"max_loaded_shards": 1,
}
# These are the trackers established dynamic scoring inputs. The
# exact same optional report can be sent by any backend; no native
# route or balancing branch is introduced here.
if self.routing is not None:
if self.routing.tokens_per_second is not None:
payload["benchmark_tokens_per_sec"] = self.routing.tokens_per_second
if self.routing.queue_depth is not None:
payload["queue_depth"] = self.routing.queue_depth
return payload
RegistrationSender = Callable[[dict[str, Any]], None]
WithdrawalSender = Callable[[str], None]
class NativeCapabilityRegistrar:
"""Publish/withdraw a native capability through caller-owned transport.
The callbacks keep tracker HTTP, relay, billing, and provider mechanics out
of the native worker. A process supervisor calls ``withdraw`` on health
loss; the caller supplies the existing tracker registration/withdrawal
transport appropriate to its deployment.
"""
def __init__(
self,
registration: NativeShardRegistration,
*,
register: RegistrationSender,
withdraw: WithdrawalSender,
) -> None:
self.registration = registration
self._register = register
self._withdraw = withdraw
def publish(self) -> None:
self._register(self.registration.payload())
def unavailable(self, reason: str) -> None:
self._withdraw(reason)
def bind(self, supervisor: NativeWorkerSupervisor) -> None:
"""Publish only after DGR-040 verification; withdraw on health loss."""
if supervisor.spec != self.registration.worker:
raise NativeRegistrationError("registrar and supervisor must own the same native worker")
supervisor.add_availability_callbacks(
on_available=lambda _reason: self.publish(),
on_unavailable=self.unavailable,
)

View File

@@ -0,0 +1,416 @@
"""Lifecycle supervision for the standalone native Shard worker (DGR-040).
This module deliberately has no dependency on ``TorchNodeServer``. A native
worker is an optional backend process; a failed worker must withdraw only its
own capability, never mutate or stop the existing Transformers backend. DGR-041
will connect the availability callbacks to backend-agnostic registration.
"""
from __future__ import annotations
import hashlib
import os
import re
import signal
import subprocess
import threading
from collections import deque
from collections.abc import Callable, Mapping
from dataclasses import dataclass, field
from pathlib import Path
from .native_protocol import SCHEMA_VERSION, pb
class NativeWorkerError(RuntimeError):
"""The configured worker cannot safely be started or trusted."""
_SHA256 = re.compile(r"^[0-9a-f]{64}$")
@dataclass(frozen=True)
class NativeWorkerSpec:
"""The immutable identity and launch command for one native worker."""
binary: Path
binary_digest: str
listen_address: str
artifact_path: Path
artifact_digest: str
recipe_digest: str
recipe_id: str
recipe_version: str
catalogue_version: str
shard_start: int
shard_end: int
args: tuple[str, ...] = ()
extra_environment: Mapping[str, str] = field(default_factory=dict)
def __post_init__(self) -> None:
if not self.listen_address:
raise ValueError("native worker requires a listen address")
if self.shard_start < 0 or self.shard_end <= self.shard_start:
raise ValueError("native worker range must be a non-empty half-open range")
for name in ("binary_digest", "artifact_digest", "recipe_digest"):
if not _SHA256.fullmatch(getattr(self, name)):
raise ValueError(f"native worker requires a lowercase SHA-256 {name}")
for name in ("recipe_id", "recipe_version", "catalogue_version"):
if not getattr(self, name):
raise ValueError(f"native worker requires {name}")
def environment(self) -> dict[str, str]:
"""Return the one startup identity the C++ worker must receive."""
result = dict(os.environ)
result.update({str(key): str(value) for key, value in self.extra_environment.items()})
result.update(
{
"MESHNET_SHARD_LISTEN_ADDR": self.listen_address,
"MESHNET_MODEL_ARTIFACT": str(self.artifact_path),
"MESHNET_MODEL_ARTIFACT_DIGEST": self.artifact_digest,
"MESHNET_RUNTIME_RECIPE_DIGEST": self.recipe_digest,
"MESHNET_RECIPE_ID": self.recipe_id,
"MESHNET_RECIPE_VERSION": self.recipe_version,
"MESHNET_CATALOGUE_VERSION": self.catalogue_version,
"MESHNET_SHARD_START_LAYER": str(self.shard_start),
"MESHNET_SHARD_END_LAYER": str(self.shard_end),
}
)
return result
@dataclass(frozen=True)
class NativeWorkerProbe:
"""The capability/health facts accepted by supervision after process launch."""
artifact_digest: str
recipe_digest: str
recipe_id: str
recipe_version: str
catalogue_version: str
shard_start: int
shard_end: int
serving: bool
detail: str = ""
WorkerProbe = Callable[[NativeWorkerSpec, float], NativeWorkerProbe]
AvailabilityCallback = Callable[[str], None]
class NativeWorkerSupervisor:
"""Own one worker process, its bounded logs, readiness and availability.
``start`` does not make a capability available merely because a child was
spawned: it verifies the executable and artifact bytes, waits for the
worker's readiness line, then proves the worker's reported identity and
serving health. A caller may inject ``probe`` for model-free tests; the
default performs the real gRPC capability and health calls.
"""
def __init__(
self,
spec: NativeWorkerSpec,
*,
probe: WorkerProbe | None = None,
readiness_timeout: float = 15.0,
health_timeout: float = 3.0,
health_interval: float = 5.0,
shutdown_timeout: float = 10.0,
kill_timeout: float = 3.0,
log_lines: int = 200,
on_available: AvailabilityCallback | None = None,
on_unavailable: AvailabilityCallback | None = None,
) -> None:
if min(readiness_timeout, health_timeout, health_interval, shutdown_timeout, kill_timeout) <= 0:
raise ValueError("native worker timeouts must be positive")
self.spec = spec
self._probe = probe or _grpc_probe
self._readiness_timeout = readiness_timeout
self._health_timeout = health_timeout
self._health_interval = health_interval
self._shutdown_timeout = shutdown_timeout
self._kill_timeout = kill_timeout
self._logs: deque[str] = deque(maxlen=log_lines)
self._on_available = on_available
self._on_unavailable = on_unavailable
self._process: subprocess.Popen[str] | None = None
self._ready = threading.Event()
self._stop_monitor = threading.Event()
self._lock = threading.RLock()
self._monitor: threading.Thread | None = None
self._available = False
self._unavailable_reason = "not started"
self._generation = 0
@property
def available(self) -> bool:
with self._lock:
return self._available
@property
def unavailable_reason(self) -> str:
with self._lock:
return self._unavailable_reason
@property
def logs(self) -> tuple[str, ...]:
with self._lock:
return tuple(self._logs)
@property
def pid(self) -> int | None:
with self._lock:
return None if self._process is None else self._process.pid
def start(self) -> NativeWorkerProbe:
"""Start and verify a previously stopped worker before publishing it."""
with self._lock:
if self._process is not None and self._process.poll() is None:
raise NativeWorkerError("native worker is already running; use restart()")
self._verify_startup_inputs()
self._ready.clear()
self._stop_monitor.clear()
command = [str(self.spec.binary), *self.spec.args]
try:
self._process = subprocess.Popen(
command,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
bufsize=1,
env=self.spec.environment(),
start_new_session=True,
)
except OSError as exc:
self._process = None
raise NativeWorkerError(f"could not start native worker: {exc}") from exc
self._generation += 1
generation = self._generation
process = self._process
for stream_name, stream in (("stdout", process.stdout), ("stderr", process.stderr)):
assert stream is not None
threading.Thread(
target=self._capture_stream,
args=(stream_name, stream),
daemon=True,
).start()
if not self._ready.wait(self._readiness_timeout):
self._fail_start("worker did not report readiness before timeout")
if process.poll() is not None:
self._fail_start(f"worker exited during startup with code {process.returncode}")
try:
result = self._probe(self.spec, self._health_timeout)
self._verify_probe(result)
except Exception as exc:
self._fail_start(f"worker failed capability/health probe: {exc}")
with self._lock:
if self._process is not process or process.poll() is not None:
self._fail_start("worker exited while capability was being verified")
self._available = True
self._unavailable_reason = ""
self._monitor = threading.Thread(
target=self._monitor_loop, args=(generation, process), daemon=True
)
self._monitor.start()
if self._on_available is not None:
self._on_available("worker ready and identity verified")
return result
def add_availability_callbacks(
self,
*,
on_available: AvailabilityCallback | None = None,
on_unavailable: AvailabilityCallback | None = None,
) -> None:
"""Attach an integration callback before the worker is started.
Registration is deliberately supplied by the caller so this supervisor
stays independent of Tracker HTTP and of every other backend.
"""
with self._lock:
if self._process is not None:
raise NativeWorkerError("availability callbacks must be attached before start")
self._on_available = _combine_callbacks(self._on_available, on_available)
self._on_unavailable = _combine_callbacks(self._on_unavailable, on_unavailable)
def restart(self) -> NativeWorkerProbe:
"""Withdraw the old capability, stop its process, then prove a fresh one."""
self.stop(reason="worker restart requested")
return self.start()
def stop(self, *, reason: str = "worker stopped") -> None:
"""Gracefully terminate the owned process, escalating only after a bound."""
with self._lock:
process = self._process
self._stop_monitor.set()
self._process = None
self._mark_unavailable(reason)
if process is None or process.poll() is not None:
return
_terminate_process_group(process, self._shutdown_timeout, self._kill_timeout)
def check_health(self) -> bool:
"""Run one bounded health check and withdraw availability on failure."""
with self._lock:
process = self._process
if process is None or process.poll() is not None:
self._mark_unavailable("worker process exited")
return False
try:
result = self._probe(self.spec, self._health_timeout)
self._verify_probe(result)
except Exception as exc:
self._mark_unavailable(f"worker health lost: {exc}")
return False
return True
def _verify_startup_inputs(self) -> None:
if not self.spec.binary.is_file() or not os.access(self.spec.binary, os.X_OK):
raise NativeWorkerError(f"native worker binary is not executable: {self.spec.binary}")
if _sha256_file(self.spec.binary) != self.spec.binary_digest:
raise NativeWorkerError("native worker binary digest does not match its immutable pin")
if not self.spec.artifact_path.is_file():
raise NativeWorkerError(f"native worker artifact is missing: {self.spec.artifact_path}")
digest = _sha256_file(self.spec.artifact_path)
if digest != self.spec.artifact_digest:
raise NativeWorkerError("native worker artifact digest does not match its immutable pin")
def _verify_probe(self, probe: NativeWorkerProbe) -> None:
expected = self.spec
actual = (
probe.artifact_digest,
probe.recipe_digest,
probe.recipe_id,
probe.recipe_version,
probe.catalogue_version,
probe.shard_start,
probe.shard_end,
)
wanted = (
expected.artifact_digest,
expected.recipe_digest,
expected.recipe_id,
expected.recipe_version,
expected.catalogue_version,
expected.shard_start,
expected.shard_end,
)
if actual != wanted:
raise NativeWorkerError("worker probe identity/range differs from configured startup identity")
if not probe.serving:
raise NativeWorkerError(f"worker is not serving: {probe.detail or 'no detail'}")
def _capture_stream(self, stream_name: str, stream) -> None:
for raw_line in stream:
line = f"{stream_name}: {raw_line.rstrip()}"
with self._lock:
self._logs.append(line)
if raw_line.startswith("ShardRuntime worker listening on "):
self._ready.set()
def _monitor_loop(self, generation: int, process: subprocess.Popen[str]) -> None:
while not self._stop_monitor.wait(self._health_interval):
with self._lock:
if generation != self._generation or self._process is not process:
return
if process.poll() is not None:
self._mark_unavailable(f"worker process exited with code {process.returncode}")
return
if not self.check_health():
return
def _fail_start(self, reason: str) -> None:
self.stop(reason=reason)
raise NativeWorkerError(reason)
def _mark_unavailable(self, reason: str) -> None:
callback = None
with self._lock:
was_available = self._available
self._available = False
self._unavailable_reason = reason
if was_available:
callback = self._on_unavailable
if callback is not None:
callback(reason)
def _sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as file:
for chunk in iter(lambda: file.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _combine_callbacks(
first: AvailabilityCallback | None, second: AvailabilityCallback | None
) -> AvailabilityCallback | None:
if first is None:
return second
if second is None:
return first
def combined(reason: str) -> None:
first(reason)
second(reason)
return combined
def _terminate_process_group(
process: subprocess.Popen[str], shutdown_timeout: float, kill_timeout: float
) -> None:
try:
os.killpg(process.pid, signal.SIGTERM)
except ProcessLookupError:
return
try:
process.wait(timeout=shutdown_timeout)
return
except subprocess.TimeoutExpired:
pass
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
return
try:
process.wait(timeout=kill_timeout)
except subprocess.TimeoutExpired as exc:
raise NativeWorkerError("native worker did not terminate after SIGKILL") from exc
def _grpc_probe(spec: NativeWorkerSpec, timeout: float) -> NativeWorkerProbe:
"""Default real wire probe; importing grpc lazily preserves CLI startup."""
import grpc
from .native_protocol.generated import shard_runtime_pb2_grpc as pb_grpc
channel = grpc.insecure_channel(spec.listen_address)
try:
grpc.channel_ready_future(channel).result(timeout=timeout)
stub = pb_grpc.ShardRuntimeStub(channel)
capability = stub.GetCapability(pb.CapabilityRequest(schema_version=SCHEMA_VERSION), timeout=timeout)
health = stub.Health(pb.HealthRequest(schema_version=SCHEMA_VERSION), timeout=timeout)
finally:
channel.close()
fingerprint = capability.fingerprint
shard_range = capability.shard_range
return NativeWorkerProbe(
artifact_digest=fingerprint.model_artifact_digest,
recipe_digest=fingerprint.runtime_recipe_digest,
recipe_id=fingerprint.recipe_id,
recipe_version=fingerprint.recipe_version,
catalogue_version=fingerprint.catalogue_version,
shard_start=shard_range.start_layer,
shard_end=shard_range.end_layer,
serving=(
capability.validated
and health.state == pb.SERVING_STATE_SERVING
),
detail=health.detail or capability.detail,
)

View File

@@ -0,0 +1,218 @@
"""Authoritative dense-Llama owned-range reports from the loaded engine state.
DGR-034 loads only the tensors a shard range owns through the Meshnet
owned-range loader (``llama_model_params::meshnet_owned_layer_start/end`` in
the pinned llama.cpp patch stack). The project-owned ``meshnet-range-report``
native tool runs that load and prints a JSON document derived from the loaded
model state — the registered tensor set and the backend buffers — never from
caller-asserted values. This module is the strict consumer of that document:
it parses it into :class:`OwnedRangeReport` and fails closed on any
inconsistency, so a range or endpoint claim that the loaded engine state does
not back is rejected before it can reach identity, admission, or routing.
Ownership contract enforced here (dense Llama only):
- every registered ``blk.N.*`` tensor lies inside the half-open owned range
``[start, end)``, and every layer in that range is present — a gapped or
out-of-range registration is rejected;
- ``token_embd.weight`` is registered only by the head shard (``start == 0``),
or by a tail shard whose model ties the output head to the embedding
(``end == n_layer`` and no separate ``output.weight``);
- ``output_norm.weight`` and ``output.weight`` are registered only by the
tail shard (``end == n_layer``);
- any other registered tensor name is unexpected and rejected;
- byte counts are consistent: an mmap load maps a file span at least the
registered tensor bytes and at most the artifact size; a non-mmap load
reports a resident allocation at least the registered tensor bytes.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Mapping
class RangeReportError(ValueError):
"""A range report is malformed, or the loaded state breaks ownership."""
_DENSE_ARCHITECTURE = "llama"
_INT_FIELDS = (
"n_layer",
"file_bytes",
"mapped_bytes",
"resident_bytes",
"registered_tensors",
"registered_bytes",
)
_BOOL_FIELDS = (
"mmap",
"touched",
"has_token_embeddings",
"has_output_head",
"tied_output_head",
)
@dataclass(frozen=True)
class OwnedRangeReport:
"""One validated owned-range load, derived from loaded engine state.
``start_layer``/``end_layer`` are the authoritative half-open owned range
the engine actually registered (the tool already refused a report whose
loaded bounds differ from the requested ones). ``has_token_embeddings`` is
true for the head shard, and also for a tail shard on a tied-output model
(the embedding tensor *is* its output head); ``tied_output_head``
disambiguates those two cases. ``mapped_bytes``/``resident_bytes`` come
from the backend buffers: with mmap they are the mapped file span holding
the owned tensors, without mmap the resident allocation holding them.
"""
architecture: str
n_layer: int
start_layer: int
end_layer: int
has_token_embeddings: bool
has_output_head: bool
tied_output_head: bool
mapped_bytes: int
resident_bytes: int
registered_tensors: int
registered_bytes: int
file_bytes: int
mmap: bool
touched: bool
vm_size_bytes: int | None
vm_rss_bytes: int | None
vm_hwm_bytes: int | None
@property
def is_head(self) -> bool:
return self.start_layer == 0
@property
def is_tail(self) -> bool:
return self.end_layer == self.n_layer
def __post_init__(self) -> None:
if self.architecture != _DENSE_ARCHITECTURE:
raise RangeReportError(
f"owned-range loading supports dense Llama only, got {self.architecture!r}"
)
if isinstance(self.n_layer, bool) or self.n_layer < 1:
raise RangeReportError("report must record a positive GGUF block count")
for name in _INT_FIELDS:
value = getattr(self, name)
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
raise RangeReportError(f"report field {name!r} must be a non-negative integer")
for name in _BOOL_FIELDS:
if not isinstance(getattr(self, name), bool):
raise RangeReportError(f"report field {name!r} must be a boolean")
if not 0 <= self.start_layer < self.end_layer <= self.n_layer:
raise RangeReportError(
f"owned range [{self.start_layer}, {self.end_layer}) is empty or "
f"outside the model's {self.n_layer} layers"
)
if self.tied_output_head and not self.is_tail:
raise RangeReportError("a tied output head can only belong to the tail shard")
expected_embeddings = self.is_head or self.tied_output_head
if self.has_token_embeddings != expected_embeddings:
raise RangeReportError(
"token-embedding registration disagrees with endpoint ownership: "
"embeddings belong to the head shard (or to a tied-output tail)"
)
if self.has_output_head != self.is_tail:
raise RangeReportError(
"output-head registration disagrees with endpoint ownership: "
"the final norm and output head belong to the tail shard"
)
if self.registered_tensors < 1 or self.registered_bytes < 1:
raise RangeReportError("the owned range registered no tensors")
if self.file_bytes < 1:
raise RangeReportError("report must record the artifact size")
if self.mmap:
if self.mapped_bytes < self.registered_bytes:
raise RangeReportError(
"mapped span undercounts the registered owned tensors"
)
if self.mapped_bytes > self.file_bytes:
raise RangeReportError("mapped span exceeds the artifact size")
else:
if self.mapped_bytes != 0:
raise RangeReportError("a non-mmap load must not claim a mapped span")
if self.resident_bytes < self.registered_bytes:
raise RangeReportError(
"resident allocation undercounts the registered owned tensors"
)
for name in ("vm_size_bytes", "vm_rss_bytes", "vm_hwm_bytes"):
value = getattr(self, name)
if value is not None and (
isinstance(value, bool) or not isinstance(value, int) or value < 0
):
raise RangeReportError(f"report field {name!r} must be a non-negative integer or null")
def _require_range(doc: Mapping[str, Any], key: str) -> tuple[int, int]:
value = doc.get(key)
if (
not isinstance(value, (list, tuple))
or len(value) != 2
or any(isinstance(v, bool) or not isinstance(v, int) for v in value)
):
raise RangeReportError(f"report field {key!r} must be a [start, end] integer pair")
return value[0], value[1]
def parse_owned_range_report(doc: Mapping[str, Any]) -> OwnedRangeReport:
"""Parse and validate one ``meshnet-range-report`` JSON document.
Fails closed: a load the tool rejected (``ok: false``), a requested range
the loaded state did not match, a gapped or out-of-range registration, an
unexpected registered tensor, and any byte-count inconsistency all raise
:class:`RangeReportError` instead of producing a report.
"""
if not isinstance(doc, Mapping):
raise RangeReportError("range report must be a JSON object")
if doc.get("ok") is not True:
error = doc.get("error")
detail = f": {error}" if isinstance(error, str) and error else ""
raise RangeReportError(f"the owned-range load was rejected{detail}")
requested = _require_range(doc, "requested_range")
reported = _require_range(doc, "reported_range")
if requested != reported:
raise RangeReportError(
f"reported range {reported} does not match the requested range {requested}; "
"ownership must be derived from the loaded engine state"
)
for key in ("unexpected_registered_tensors", "missing_owned_layers"):
value = doc.get(key)
if not isinstance(value, list):
raise RangeReportError(f"report field {key!r} must be a list")
if value:
raise RangeReportError(
f"ownership audit failed: {key} is {value!r}; the registered "
"tensor set must exactly cover the owned range and its endpoints"
)
architecture = doc.get("architecture")
if not isinstance(architecture, str):
raise RangeReportError("report field 'architecture' must be a string")
fields: dict[str, Any] = {}
for name in _INT_FIELDS + _BOOL_FIELDS:
if name not in doc:
raise RangeReportError(f"range report is missing field {name!r}")
fields[name] = doc[name]
for name in ("vm_size_bytes", "vm_rss_bytes", "vm_hwm_bytes"):
fields[name] = doc.get(name)
return OwnedRangeReport(
architecture=architecture,
start_layer=reported[0],
end_layer=reported[1],
**fields,
)

View File

@@ -62,6 +62,29 @@ message(STATUS "Pinned gRPC ${gRPC_VERSION}: building ShardRuntime service stubs
enable_testing() enable_testing()
# DGR-037: the standalone worker owns exactly one loaded llama.cpp artifact.
# Its implementation types stay in worker/llama_shard_engine.cpp; the gRPC
# service receives only the project-owned ShardEngine surface.
set(MESHNET_LLAMA_SOURCE_DIR "${CMAKE_SOURCE_DIR}/../../../build/llama.cpp/source" CACHE PATH
"Applied pinned llama.cpp source directory")
set(MESHNET_LLAMA_LIBRARY_DIR "${CMAKE_SOURCE_DIR}/../../../build/llama.cpp/build/bin" CACHE PATH
"Directory containing the matching applied-patch libllama")
find_path(MESHNET_LLAMA_INCLUDE_DIR llama.h PATHS "${MESHNET_LLAMA_SOURCE_DIR}/include" NO_DEFAULT_PATH REQUIRED)
find_path(MESHNET_LLAMA_GGML_INCLUDE_DIR ggml.h PATHS "${MESHNET_LLAMA_SOURCE_DIR}/ggml/include" NO_DEFAULT_PATH REQUIRED)
find_library(MESHNET_LLAMA_LIBRARY NAMES llama PATHS "${MESHNET_LLAMA_LIBRARY_DIR}" NO_DEFAULT_PATH REQUIRED)
add_executable(shard_worker
worker/shard_worker_main.cpp
worker/shard_service.cpp
worker/llama_shard_engine.cpp)
target_include_directories(shard_worker PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/worker" "${MESHNET_LLAMA_INCLUDE_DIR}" "${MESHNET_LLAMA_GGML_INCLUDE_DIR}")
target_link_libraries(shard_worker PRIVATE shard_runtime_grpc gRPC::grpc++ "${MESHNET_LLAMA_LIBRARY}")
set_target_properties(shard_worker PROPERTIES BUILD_RPATH "${MESHNET_LLAMA_LIBRARY_DIR}")
# Pure-C++ CTest: the worker binds an ephemeral port, self-drives the full
# lifecycle (capability, health, fragmented prefill, decode, release) over a
# real loopback gRPC channel, and exits non-zero on any mismatch. This proves
# the worker serves the contract without needing a Python environment.
add_executable(shard_protocol_conformance tests/test_shard_protocol_conformance.cpp) add_executable(shard_protocol_conformance tests/test_shard_protocol_conformance.cpp)
target_link_libraries(shard_protocol_conformance PRIVATE shard_runtime_proto) target_link_libraries(shard_protocol_conformance PRIVATE shard_runtime_proto)

View File

@@ -76,3 +76,26 @@ self-consistent. Instead:
Byte equality across the two implementations is the claim; anything less is two Byte equality across the two implementations is the claim; anything less is two
parallel test suites that can drift apart. parallel test suites that can drift apart.
## DGR-037 standalone llama.cpp worker
`shard_worker` is no longer a model-free fixture. It refuses to start until it
can load one exact, range-attested GGUF identity through the pinned patched
llama.cpp library. Supply these environment variables from the node-owned
recipe/materialization layer (never from a stream request):
```bash
MESHNET_MODEL_ARTIFACT=/mounted/models/model.gguf \
MESHNET_MODEL_ARTIFACT_DIGEST=sha256:<artifact> \
MESHNET_RUNTIME_RECIPE_DIGEST=sha256:<recipe> \
MESHNET_RECIPE_ID=dense-llama MESHNET_RECIPE_VERSION=1 MESHNET_CATALOGUE_VERSION=1 \
MESHNET_SHARD_START_LAYER=0 MESHNET_SHARD_END_LAYER=32 \
build/native/shard_worker 127.0.0.1:50051
```
The worker publishes that loaded identity and llama.cpp-derived resident bytes
in capability/health responses, and only accepts the exact same range and
fingerprint at `SessionOpen`. `MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS=N`
is an opt-in test hook: after the Nth admitted execution the process exits 70,
which is intentionally observable by the future node supervisor; it is not a
recover-in-process mechanism.

View File

@@ -28,6 +28,12 @@ One numbered patch per concern (ADR-0024 local seams only):
5. `0005-worker-range-report-hook.patch` (worker hooks) exposes the 5. `0005-worker-range-report-hook.patch` (worker hooks) exposes the
`llama_model_meshnet_range_report` C API the project-owned worker binds to `llama_model_meshnet_range_report` C API the project-owned worker binds to
and registers a model-free native fixture test for it. and registers a model-free native fixture test for it.
6. `0006-meshnet-range-report-tool.patch` (range reporting) adds the
project-owned `meshnet-range-report` tool: it loads one GGUF artifact
through the owned-range loader and prints a JSON document derived from the
loaded model state — the owned-range report, the registered tensor set
audited against the requested ownership, and backend-buffer byte counts.
It never builds or runs a compute graph.
Meshnet routing, Tracker, gRPC, relay, billing, authentication, and telemetry Meshnet routing, Tracker, gRPC, relay, billing, authentication, and telemetry
remain outside this directory; the stack is checked for such control-plane remain outside this directory; the stack is checked for such control-plane

View File

@@ -10,21 +10,23 @@
"method": "git-clone-detached-commit", "method": "git-clone-detached-commit",
"workspace": "build/llama.cpp" "workspace": "build/llama.cpp"
}, },
"patched_tree": "c0045714735ae5ee7b7334a480d8ac04e03e1b18", "patched_tree": "8f7e87fea6743f0b9744afe44f9e6f9ca3b7d08a",
"upstream_license": "MIT", "upstream_license": "MIT",
"patch_series": [ "patch_series": [
"0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch", "0001-cmake-reserve-meshnet-patch-stack-abi-marker.patch",
"0002-dense-llama-owned-range-loading.patch", "0002-dense-llama-owned-range-loading.patch",
"0003-owned-range-filtered-state-report.patch", "0003-owned-range-filtered-state-report.patch",
"0004-dense-boundary-io-endpoint-guard.patch", "0004-dense-boundary-io-endpoint-guard.patch",
"0005-worker-range-report-hook.patch" "0005-worker-range-report-hook.patch",
"0006-meshnet-range-report-tool.patch"
], ],
"patch_scope": [ "patch_scope": [
"Reserved CMake ABI marker only; no execution or model semantics.", "Reserved CMake ABI marker only; no execution or model semantics.",
"Range loading: dense-Llama owned-range params, validation, and filtered tensor registration with endpoint ownership.", "Range loading: dense-Llama owned-range params, validation, and filtered tensor registration with endpoint ownership.",
"Filtered state: owned-range report populated from registered tensors and backend buffers, derived never asserted.", "Filtered state: owned-range report populated from registered tensors and backend buffers, derived never asserted.",
"Boundary I/O: endpoint ownership flags and a fail-closed dense graph guard until typed endpoint adapters exist.", "Boundary I/O: endpoint ownership flags and a fail-closed dense graph guard until typed endpoint adapters exist.",
"Worker hooks: public C range-report API and the model-free native fixture test the project-owned worker binds to." "Worker hooks: public C range-report API and the model-free native fixture test the project-owned worker binds to.",
"Range reporting: project-owned tool that loads one artifact through the owned-range loader and reports derived ownership and buffer-byte state as JSON."
], ],
"patch_assumptions": "patches/UPSTREAM-ASSUMPTIONS.json", "patch_assumptions": "patches/UPSTREAM-ASSUMPTIONS.json",
"build": { "build": {
@@ -46,7 +48,7 @@
"-DGGML_VULKAN=OFF", "-DGGML_VULKAN=OFF",
"-DGGML_METAL=OFF" "-DGGML_METAL=OFF"
], ],
"native_targets": ["llama-gguf-hash", "test-meshnet-range-ownership"], "native_targets": ["llama-gguf-hash", "test-meshnet-range-ownership", "meshnet-range-report"],
"smoke_binary": "bin/llama-gguf-hash", "smoke_binary": "bin/llama-gguf-hash",
"smoke_args": ["--help"], "smoke_args": ["--help"],
"smoke_output_token": "usage", "smoke_output_token": "usage",
@@ -81,7 +83,9 @@
"src/llama-model.h", "src/llama-model.h",
"src/models/llama.cpp", "src/models/llama.cpp",
"tests/CMakeLists.txt", "tests/CMakeLists.txt",
"tests/test-meshnet-range-ownership.cpp" "tests/test-meshnet-range-ownership.cpp",
"tools/meshnet-range-report/CMakeLists.txt",
"tools/meshnet-range-report/meshnet-range-report.cpp"
], ],
"stock_glm_limitations": "This pin may load GLM-5.2 through the dense-MLA compatibility fallback. It does not prove native DSA, IndexShare, MoE semantic correctness, numerical equivalence, performance, or route certification." "stock_glm_limitations": "This pin may load GLM-5.2 through the dense-MLA compatibility fallback. It does not prove native DSA, IndexShare, MoE semantic correctness, numerical equivalence, performance, or route certification."
} }

View File

@@ -0,0 +1,414 @@
From: Meshnet <meshnet@invalid>
Subject: [PATCH] llama: add dense-Llama owned-range report tool
Concern: range reporting. Adds the project-owned meshnet-range-report tool:
it loads one GGUF artifact through the Meshnet owned-range loader and prints
a JSON document derived from the loaded model state — the owned-range
report, the registered tensor set audited against the requested ownership,
and backend-buffer byte counts (optionally split from repack buffers, plus
process resident readings). It never builds or runs a compute graph and
never trusts caller-asserted range or endpoint claims.
---
diff --git a/CMakeLists.txt b/CMakeLists.txt
index a9afcff..868793b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -281,3 +281,6 @@ configure_file(cmake/llama.pc.in
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/llama.pc"
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
+
+# Meshnet-owned owned-range report tool (patch stack, range-report concern).
+add_subdirectory(tools/meshnet-range-report)
diff --git a/tools/meshnet-range-report/CMakeLists.txt b/tools/meshnet-range-report/CMakeLists.txt
new file mode 100644
index 000000000..24401007e
--- /dev/null
+++ b/tools/meshnet-range-report/CMakeLists.txt
@@ -0,0 +1,7 @@
+# Meshnet-owned dense-Llama owned-range load/report tool.
+#
+# Built unconditionally with the patched tree: it exercises the Meshnet
+# owned-range loader against real GGUF artifacts and reports only state
+# derived from the loaded model (registered tensors, backend buffers).
+add_executable(meshnet-range-report meshnet-range-report.cpp)
+target_link_libraries(meshnet-range-report PRIVATE llama)
diff --git a/tools/meshnet-range-report/meshnet-range-report.cpp b/tools/meshnet-range-report/meshnet-range-report.cpp
new file mode 100644
index 000000000..49a5eb2a0
--- /dev/null
+++ b/tools/meshnet-range-report/meshnet-range-report.cpp
@@ -0,0 +1,373 @@
+// Meshnet-owned dense-Llama owned-range load/report tool.
+//
+// Loads one GGUF artifact through the Meshnet owned-range loader
+// (llama_model_params::meshnet_owned_layer_start/end) and prints a single
+// JSON report derived from the loaded model state — registered tensors and
+// backend buffers, never caller-asserted values. The audit fails closed when
+// the registered tensor set disagrees with the requested ownership: every
+// registered per-layer tensor must lie inside [start, end), the token
+// embedding may be registered only by the head shard (start == 0) or by a
+// tail shard whose model ties the output head to the embedding, and the
+// final norm plus output head may be registered only by the tail shard
+// (end == n_layer).
+
+#include "ggml.h"
+#include "llama.h"
+
+#include "../../src/llama-model.h"
+
+#include <cstdint>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <set>
+#include <string>
+#include <sys/stat.h>
+#include <vector>
+
+namespace {
+
+constexpr int kExitUsage = 2;
+constexpr int kExitLoad = 3;
+constexpr int kExitAudit = 4;
+
+std::string g_log_tail;
+
+void capture_log(enum ggml_log_level level, const char * text, void *) {
+ if (level >= GGML_LOG_LEVEL_ERROR) {
+ g_log_tail += text;
+ if (g_log_tail.size() > 512) {
+ g_log_tail.erase(0, g_log_tail.size() - 512);
+ }
+ }
+}
+
+std::string json_escape(const std::string & value) {
+ std::string out;
+ for (const char c : value) {
+ if (c == '"' || c == '\\') {
+ out += '\\';
+ out += c;
+ } else if (c == '\n') {
+ out += "\\n";
+ } else if (c == '\r') {
+ // drop carriage returns from embedded log text
+ } else {
+ out += c;
+ }
+ }
+ return out;
+}
+
+std::string json_string_array(const std::vector<std::string> & items) {
+ std::string out = "[";
+ for (size_t i = 0; i < items.size(); ++i) {
+ if (i) {
+ out += ", ";
+ }
+ out += "\"" + json_escape(items[i]) + "\"";
+ }
+ return out + "]";
+}
+
+std::string json_int_array(const std::vector<int> & items) {
+ std::string out = "[";
+ for (size_t i = 0; i < items.size(); ++i) {
+ if (i) {
+ out += ", ";
+ }
+ out += std::to_string(items[i]);
+ }
+ return out + "]";
+}
+
+int fail(int code, const std::string & error) {
+ std::string detail = error;
+ if (!g_log_tail.empty()) {
+ detail += ": " + g_log_tail;
+ }
+ std::printf("{\"ok\": false, \"error\": \"%s\"}\n", json_escape(detail).c_str());
+ return code;
+}
+
+bool parse_nonnegative(const char * text, int & out) {
+ if (text == nullptr || *text == '\0' || *text == '-') {
+ return false;
+ }
+ char * end = nullptr;
+ const long value = std::strtol(text, &end, 10);
+ if (end == text || *end != '\0' || value > INT32_MAX) {
+ return false;
+ }
+ out = static_cast<int>(value);
+ return true;
+}
+
+uint64_t file_size(const std::string & path) {
+ struct stat st;
+ return ::stat(path.c_str(), &st) == 0 ? static_cast<uint64_t>(st.st_size) : 0;
+}
+
+struct proc_status {
+ uint64_t vm_size = 0;
+ uint64_t vm_rss = 0;
+ uint64_t vm_hwm = 0;
+ bool valid = false;
+};
+
+proc_status read_proc_status() {
+ proc_status out;
+#ifdef __linux__
+ FILE * f = std::fopen("/proc/self/status", "r");
+ if (!f) {
+ return out;
+ }
+ char line[256];
+ while (std::fgets(line, sizeof(line), f)) {
+ uint64_t kb = 0;
+ if (std::sscanf(line, "VmSize: %lu kB", &kb) == 1) {
+ out.vm_size = kb * 1024;
+ } else if (std::sscanf(line, "VmRSS: %lu kB", &kb) == 1) {
+ out.vm_rss = kb * 1024;
+ } else if (std::sscanf(line, "VmHWM: %lu kB", &kb) == 1) {
+ out.vm_hwm = kb * 1024;
+ }
+ }
+ std::fclose(f);
+ out.valid = true;
+#endif
+ return out;
+}
+
+void usage(const char * argv0) {
+ std::fprintf(stderr,
+ "usage: %s --model PATH --start N --end M [--no-mmap] [--no-extra-bufts] [--touch]\n"
+ "loads one dense-Llama GGUF through the Meshnet owned-range loader and\n"
+ "prints a JSON report derived from the loaded model state\n",
+ argv0);
+}
+
+} // namespace
+
+int main(int argc, char ** argv) {
+ std::string model_path;
+ int start = -1;
+ int end = -1;
+ bool use_mmap = true;
+ bool use_extra_bufts = true;
+ bool touch = false;
+
+ for (int i = 1; i < argc; ++i) {
+ const std::string arg = argv[i];
+ if (arg == "--model" && i + 1 < argc) {
+ model_path = argv[++i];
+ } else if (arg == "--start" && i + 1 < argc) {
+ if (!parse_nonnegative(argv[++i], start)) {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+ } else if (arg == "--end" && i + 1 < argc) {
+ if (!parse_nonnegative(argv[++i], end)) {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+ } else if (arg == "--no-mmap") {
+ use_mmap = false;
+ } else if (arg == "--no-extra-bufts") {
+ use_extra_bufts = false;
+ } else if (arg == "--touch") {
+ touch = true;
+ } else {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+ }
+ if (model_path.empty() || start < 0 || end < 0) {
+ usage(argv[0]);
+ return kExitUsage;
+ }
+
+ llama_log_set(capture_log, nullptr);
+ llama_backend_init();
+
+ llama_model_params params = llama_model_default_params();
+ params.meshnet_owned_layer_start = start;
+ params.meshnet_owned_layer_end = end;
+ params.use_mmap = use_mmap;
+ params.use_extra_bufts = use_extra_bufts;
+ params.progress_callback = nullptr;
+
+ llama_model * model = llama_model_load_from_file(model_path.c_str(), params);
+ if (model == nullptr) {
+ return fail(kExitLoad, "owned-range load rejected the artifact or range");
+ }
+
+ llama_meshnet_range_report report = {};
+ if (!llama_model_meshnet_range_report(model, &report)) {
+ llama_model_free(model);
+ return fail(kExitLoad, "loaded model carries no owned-range report");
+ }
+
+ char arch_buf[128] = {};
+ std::string arch;
+ if (llama_model_meta_val_str(model, "general.architecture", arch_buf, sizeof(arch_buf)) >= 0) {
+ arch = arch_buf;
+ }
+ const int n_layer = llama_model_n_layer(model);
+ const uint64_t bytes_on_disk = file_size(model_path);
+
+ // Audit the registered tensor set against the requested ownership.
+ const auto & tensors = llama_internal_get_tensor_map(model);
+ bool has_embd = false;
+ bool has_out_norm = false;
+ bool has_out = false;
+ std::set<int> owned_layers;
+ std::vector<std::string> unexpected;
+ uint64_t registered_bytes = 0;
+ for (const auto & entry : tensors) {
+ const std::string & name = entry.first;
+ registered_bytes += ggml_nbytes(entry.second);
+ if (name == "token_embd.weight") {
+ has_embd = true;
+ continue;
+ }
+ if (name == "output_norm.weight") {
+ has_out_norm = true;
+ continue;
+ }
+ if (name == "output.weight") {
+ has_out = true;
+ continue;
+ }
+ int block = -1;
+ if (std::sscanf(name.c_str(), "blk.%d.", &block) == 1 && block >= 0) {
+ owned_layers.insert(block);
+ continue;
+ }
+ unexpected.push_back(name);
+ }
+
+ // A tail shard whose model ties the output head to the token embedding
+ // registers token_embd.weight as its output head instead of output.weight.
+ const bool tied_tail = end == n_layer && has_embd && !has_out;
+ const bool expect_embd = start == 0 || tied_tail;
+
+ std::vector<int> missing_layers;
+ for (int i = start; i < end; ++i) {
+ if (!owned_layers.count(i)) {
+ missing_layers.push_back(i);
+ }
+ }
+ std::vector<int> outside_layers;
+ for (const int block : owned_layers) {
+ if (block < start || block >= end) {
+ outside_layers.push_back(block);
+ }
+ }
+
+ std::vector<std::string> mismatches;
+ if (report.start_layer != start || report.end_layer != end) {
+ mismatches.push_back("reported range differs from the requested range");
+ }
+ if (has_embd != expect_embd) {
+ mismatches.push_back("token-embedding registration disagrees with endpoint ownership");
+ }
+ if ((end == n_layer) && !has_out_norm) {
+ mismatches.push_back("tail range is missing the final norm");
+ }
+ if ((end == n_layer) && !has_out && !has_embd) {
+ mismatches.push_back("tail range is missing the output head");
+ }
+ if ((end != n_layer) && (has_out_norm || has_out)) {
+ mismatches.push_back("non-tail range registered tail-only tensors");
+ }
+ if (report.has_token_embeddings != has_embd) {
+ mismatches.push_back("reported embedding ownership disagrees with registered tensors");
+ }
+ if (report.has_output_head != (end == n_layer)) {
+ mismatches.push_back("reported output-head ownership disagrees with endpoint ownership");
+ }
+ if (!missing_layers.empty()) {
+ mismatches.push_back("owned range has missing per-layer tensors");
+ }
+ if (!outside_layers.empty()) {
+ mismatches.push_back("registered per-layer tensors lie outside the owned range");
+ }
+ if (!unexpected.empty()) {
+ mismatches.push_back("registered tensors outside the dense-Llama ownership vocabulary");
+ }
+ if (use_mmap && report.mapped_bytes < registered_bytes) {
+ mismatches.push_back("mapped span undercounts the registered tensors");
+ }
+ if (!use_mmap && report.resident_bytes < registered_bytes) {
+ mismatches.push_back("resident allocation undercounts the registered tensors");
+ }
+
+ if (touch) {
+ volatile uint64_t sink = 0;
+ for (const auto & entry : tensors) {
+ const auto * data = static_cast<const volatile uint8_t *>(entry.second->data);
+ const size_t nbytes = ggml_nbytes(entry.second);
+ for (size_t i = 0; i < nbytes; i += 4096) {
+ sink += data[i];
+ }
+ }
+ (void) sink;
+ }
+
+ const proc_status proc = read_proc_status();
+
+ if (!mismatches.empty()) {
+ llama_model_free(model);
+ return fail(kExitAudit, "ownership audit failed: " + json_string_array(mismatches));
+ }
+
+ std::printf(
+ "{\n"
+ " \"ok\": true,\n"
+ " \"model\": \"%s\",\n"
+ " \"architecture\": \"%s\",\n"
+ " \"n_layer\": %d,\n"
+ " \"file_bytes\": %llu,\n"
+ " \"requested_range\": [%d, %d],\n"
+ " \"reported_range\": [%d, %d],\n"
+ " \"mmap\": %s,\n"
+ " \"touched\": %s,\n"
+ " \"use_extra_bufts\": %s,\n"
+ " \"has_token_embeddings\": %s,\n"
+ " \"has_output_head\": %s,\n"
+ " \"tied_output_head\": %s,\n"
+ " \"mapped_bytes\": %llu,\n"
+ " \"resident_bytes\": %llu,\n"
+ " \"registered_tensors\": %d,\n"
+ " \"registered_bytes\": %llu,\n"
+ " \"unexpected_registered_tensors\": [],\n"
+ " \"missing_owned_layers\": [],\n"
+ " \"vm_size_bytes\": %llu,\n"
+ " \"vm_rss_bytes\": %llu,\n"
+ " \"vm_hwm_bytes\": %llu\n"
+ "}\n",
+ json_escape(model_path).c_str(),
+ json_escape(arch).c_str(),
+ n_layer,
+ (unsigned long long) bytes_on_disk,
+ start, end,
+ report.start_layer, report.end_layer,
+ use_mmap ? "true" : "false",
+ touch ? "true" : "false",
+ use_extra_bufts ? "true" : "false",
+ report.has_token_embeddings ? "true" : "false",
+ report.has_output_head ? "true" : "false",
+ tied_tail ? "true" : "false",
+ (unsigned long long) report.mapped_bytes,
+ (unsigned long long) report.resident_bytes,
+ (int) tensors.size(),
+ (unsigned long long) registered_bytes,
+ (unsigned long long) proc.vm_size,
+ (unsigned long long) proc.vm_rss,
+ (unsigned long long) proc.vm_hwm);
+
+ llama_model_free(model);
+ llama_backend_free();
+ return 0;
+}

View File

@@ -4,3 +4,4 @@
4871a37544df658980a01b4f94151a90b609fb144c931b4a814309ee608ebb46 0003-owned-range-filtered-state-report.patch 4871a37544df658980a01b4f94151a90b609fb144c931b4a814309ee608ebb46 0003-owned-range-filtered-state-report.patch
19d451ce259150ffede793c4eb547425375c0fcd97caf326b43e8f1a204f05b6 0004-dense-boundary-io-endpoint-guard.patch 19d451ce259150ffede793c4eb547425375c0fcd97caf326b43e8f1a204f05b6 0004-dense-boundary-io-endpoint-guard.patch
cf263357a6a8de193f710836c7c467c38cac7099975303ee2628e0609daf5a47 0005-worker-range-report-hook.patch cf263357a6a8de193f710836c7c467c38cac7099975303ee2628e0609daf5a47 0005-worker-range-report-hook.patch
23b4b8c56243d52ba682f0034022a86bf8ded007885be5b659cf5158ff3eb429 0006-meshnet-range-report-tool.patch

View File

@@ -112,6 +112,30 @@
"llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h", "llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h",
"gguf empty-context writer API: gguf_init_empty, gguf_add_tensor, gguf_write_to_file" "gguf empty-context writer API: gguf_init_empty, gguf_add_tensor, gguf_write_to_file"
] ]
},
"0006-meshnet-range-report-tool.patch": {
"concern": "range-reporting",
"files": {
"CMakeLists.txt": {
"before": "a9afcffa68bed7cbd8fad39ad9f95ad784251234",
"after": "868793b826f565df7f041e7ba55820b5ad744b10"
},
"tools/meshnet-range-report/CMakeLists.txt": {
"before": null,
"after": "24401007ee85e217c2741a42c7119fad323ff08a"
},
"tools/meshnet-range-report/meshnet-range-report.cpp": {
"before": null,
"after": "49a5eb2a05bf6514e166453ea0e35b8bc9c5fdf6"
}
},
"api_assumptions": [
"llama_model_params carries meshnet_owned_layer_start/end, use_mmap, and use_extra_bufts",
"llama_model_meshnet_range_report C API and llama_meshnet_range_report fields (patch 0005)",
"llama_internal_get_tensor_map(const llama_model *) in src/llama-model.h",
"llama_model_meta_val_str and llama_model_n_layer public accessors",
"top-level CMakeLists add_subdirectory of a project-owned tool directory after the llama target"
]
} }
} }
} }

View File

@@ -3,3 +3,4 @@
0003-owned-range-filtered-state-report.patch 0003-owned-range-filtered-state-report.patch
0004-dense-boundary-io-endpoint-guard.patch 0004-dense-boundary-io-endpoint-guard.patch
0005-worker-range-report-hook.patch 0005-worker-range-report-hook.patch
0006-meshnet-range-report-tool.patch

View File

@@ -0,0 +1,165 @@
// Deterministic, model-free fake ShardEngine for the native worker (DGR-033).
//
// This is the C++ analogue of `meshnet_node.fake_shard_engine.FakeShardEngine`
// (DGR-032): a pure fixture that performs a *bounded real forward* over the
// bytes it received off the socket and never links, loads, or dispatches to
// llama.cpp. It exists to prove the standalone worker process, stream,
// lifecycle, and supervision shape before any real engine is bound (DGR-037).
//
// The "forward" is deliberately transport-verifiable rather than semantic: it
// reassembles a tensor's fragments, checks they tile exactly, and derives a
// CRC32C over the uncompressed bytes — the same rule the schema's `Checksum`
// declares and the same bounded forward the DGR-024 Python surface performs.
// Feeding the same bytes back (echo) lets a client prove the payload truly
// traversed the wire and returned unmodified; a direct hop and an opaque relay
// of the identical frames therefore yield byte-identical responses.
//
// There is no arbitrary-graph entry point here and no llama.cpp RPC: the engine
// only knows how to reassemble/checksum a bundle. That is the whole point of a
// fixture worker (acceptance criterion 4).
#ifndef MESHNET_NATIVE_WORKER_FAKE_ENGINE_H_
#define MESHNET_NATIVE_WORKER_FAKE_ENGINE_H_
#include <algorithm>
#include <cstdint>
#include <optional>
#include <string>
#include <vector>
#include "shard_runtime.pb.h"
namespace meshnet::worker {
namespace sp = ::meshnet::shard::v1;
// Standard CRC-32 (ISO-HDLC / zlib polynomial 0xEDB88320, reflected).
//
// The schema's `Checksum` field is labelled CRC32C, but the DGR-024 Python
// runtime surface (`shard_runtime_server.py`) computes it with `zlib.crc32`
// (standard CRC-32, not the Castagnoli CRC32C). This worker deliberately mirrors
// that exact computation so its checksum acceptance is byte-for-byte identical
// to the existing Python gRPC surface and to a relayed frame's expectations.
inline uint32_t Crc32(const std::string& data, uint32_t seed = 0) {
static uint32_t table[256];
static bool built = false;
if (!built) {
for (uint32_t i = 0; i < 256; ++i) {
uint32_t c = i;
for (int k = 0; k < 8; ++k) {
c = (c & 1) ? (c >> 1) ^ 0xEDB88320u : (c >> 1);
}
table[i] = c;
}
built = true;
}
uint32_t crc = seed ^ 0xFFFFFFFFu;
for (unsigned char byte : data) {
crc = (crc >> 8) ^ table[(crc ^ byte) & 0xFF];
}
return crc ^ 0xFFFFFFFFu;
}
// Outcome of validating one bundle before the bounded forward runs.
struct BundleCheck {
// Set when the bundle is malformed/corrupt (maps to PAYLOAD_CORRUPT).
std::optional<std::string> corrupt_detail;
// Set when the declared payload exceeds the negotiated per-chunk ceiling
// (maps to RESOURCE_EXHAUSTED) — the worker refuses unbounded messages.
std::optional<std::string> oversize_detail;
};
// The fake engine's only capability: verify a bundle tiles and checksums, and
// that it stays within the negotiated byte ceiling. Mirrors `_validate_bundle`
// in `shard_runtime_server.py` plus the bounded-message rule DGR-033 adds.
class FakeShardEngine {
public:
// Marker mirroring `FakeShardEngine.EVIDENCE_CLASS` so a future parity check
// (DGR-036) can assert this is a fixture, not a real engine.
static constexpr const char* kEvidenceClass = "fixture";
FakeShardEngine() = default;
// `max_chunk_bytes` is the per-session *negotiated* ceiling (the strictest of
// the worker's own limit and the peer's proposal), passed in on every call so
// the engine enforces exactly what the SessionOpen handshake settled — never a
// value the peer proposed unilaterally.
BundleCheck Validate(const sp::TensorBundle& bundle, uint64_t max_chunk_bytes) const {
BundleCheck result;
for (const auto& tensor : bundle.tensors()) {
// Bounded message: a declared payload larger than the ceiling is refused
// before any reassembly work is done.
if (max_chunk_bytes != 0 && tensor.total_bytes() > max_chunk_bytes) {
result.oversize_detail =
"tensor '" + tensor.name() + "': declared total_bytes " +
std::to_string(tensor.total_bytes()) + " exceeds max_chunk_bytes " +
std::to_string(max_chunk_bytes);
return result;
}
// Fragments must tile the wire body exactly: no hole, no overlap.
std::vector<const sp::TensorFragment*> ordered;
ordered.reserve(tensor.fragments_size());
for (const auto& fragment : tensor.fragments()) {
ordered.push_back(&fragment);
}
std::sort(ordered.begin(), ordered.end(),
[](const sp::TensorFragment* a, const sp::TensorFragment* b) {
return a->byte_offset() < b->byte_offset();
});
uint64_t expected_offset = 0;
std::string payload;
for (const auto* fragment : ordered) {
if (fragment->byte_offset() != expected_offset) {
result.corrupt_detail =
"tensor '" + tensor.name() + "': fragment at offset " +
std::to_string(fragment->byte_offset()) +
" does not tile the preceding " + std::to_string(expected_offset) +
" bytes (gap or overlap)";
return result;
}
payload.append(fragment->payload());
expected_offset += fragment->payload().size();
}
if (tensor.compression() == sp::COMPRESSION_NONE &&
expected_offset != tensor.total_bytes()) {
result.corrupt_detail =
"tensor '" + tensor.name() + "': fragments cover " +
std::to_string(expected_offset) + " bytes, declared total_bytes is " +
std::to_string(tensor.total_bytes());
return result;
}
if (tensor.compression() == sp::COMPRESSION_NONE &&
tensor.checksum().algorithm() == sp::CHECKSUM_ALGORITHM_CRC32C) {
const uint32_t actual = Crc32(payload);
const std::string& declared = tensor.checksum().value();
std::string actual_be(4, '\0');
actual_be[0] = static_cast<char>((actual >> 24) & 0xFF);
actual_be[1] = static_cast<char>((actual >> 16) & 0xFF);
actual_be[2] = static_cast<char>((actual >> 8) & 0xFF);
actual_be[3] = static_cast<char>(actual & 0xFF);
if (declared != actual_be) {
result.corrupt_detail = "tensor '" + tensor.name() + "': checksum mismatch";
return result;
}
}
}
return result;
}
// Bounded real forward: fold every fragment's payload through CRC32C so the
// digest is only reproducible if the payload really traversed the wire.
uint32_t BoundedForward(const sp::TensorBundle& bundle) const {
uint32_t digest = 0;
for (const auto& tensor : bundle.tensors()) {
for (const auto& fragment : tensor.fragments()) {
digest = Crc32(fragment.payload(), digest);
}
}
return digest;
}
};
} // namespace meshnet::worker
#endif // MESHNET_NATIVE_WORKER_FAKE_ENGINE_H_

View File

@@ -0,0 +1,291 @@
#include "llama_shard_engine.h"
#include <algorithm>
#include <chrono>
#include <cstdlib>
#include <map>
#include <mutex>
#include <utility>
#include <vector>
#include "llama.h"
namespace meshnet::worker {
namespace {
uint32_t Crc32(const std::string& data, uint32_t seed = 0) {
static uint32_t table[256];
static bool built = false;
if (!built) {
for (uint32_t i = 0; i < 256; ++i) {
uint32_t c = i;
for (int k = 0; k < 8; ++k) c = (c & 1) ? (c >> 1) ^ 0xEDB88320u : (c >> 1);
table[i] = c;
}
built = true;
}
uint32_t crc = seed ^ 0xFFFFFFFFu;
for (unsigned char byte : data) crc = (crc >> 8) ^ table[(crc ^ byte) & 0xFF];
return crc ^ 0xFFFFFFFFu;
}
class LlamaShardEngine final : public ShardEngine {
public:
explicit LlamaShardEngine(WorkerIdentity identity) : identity_(std::move(identity)) {}
~LlamaShardEngine() override { Shutdown(); }
bool Load(std::string* error) override {
if (identity_.artifact_path.empty() || identity_.artifact_digest.empty() ||
identity_.recipe_digest.empty() || identity_.end_layer <= identity_.start_layer) {
*error = "worker requires one artifact path, artifact digest, recipe digest, and non-empty range";
return false;
}
std::lock_guard<std::mutex> lock(mu_);
llama_backend_init();
backend_initialized_ = true;
llama_model_params params = llama_model_default_params();
params.meshnet_owned_layer_start = static_cast<int32_t>(identity_.start_layer);
params.meshnet_owned_layer_end = static_cast<int32_t>(identity_.end_layer);
model_ = llama_model_load_from_file(identity_.artifact_path.c_str(), params);
if (!model_) {
ShutdownLocked();
*error = "llama.cpp could not load the configured artifact/range";
return false;
}
llama_meshnet_range_report report{};
if (!llama_model_meshnet_range_report(model_, &report) ||
report.start_layer != static_cast<int32_t>(identity_.start_layer) ||
report.end_layer != static_cast<int32_t>(identity_.end_layer)) {
ShutdownLocked();
*error = "llama.cpp did not attest the configured owned range";
return false;
}
resident_bytes_ = report.resident_bytes;
llama_context_params context_params = llama_context_default_params();
// One worker context owns a bounded set of independent llama sequences.
// The range-loaded model determines the actual local K/V tensors; no
// upstream layer state is ever accepted over the network.
context_params.n_ctx = identity_.hot_kv_budget_tokens;
context_params.n_batch = identity_.hot_kv_context_tokens;
context_params.n_ubatch = identity_.hot_kv_context_tokens;
context_params.n_seq_max = identity_.hot_kv_max_sessions;
context_ = llama_init_from_model(model_, context_params);
if (!context_) {
ShutdownLocked();
*error = "llama.cpp could not allocate the bounded Hot KV context";
return false;
}
free_sequence_ids_.reserve(identity_.hot_kv_max_sessions);
for (uint32_t sequence_id = 0; sequence_id < identity_.hot_kv_max_sessions; ++sequence_id) {
free_sequence_ids_.push_back(static_cast<llama_seq_id>(sequence_id));
}
loaded_ = true;
return true;
}
BundleCheck Validate(const sp::TensorBundle& bundle, uint64_t max_chunk_bytes) const override {
BundleCheck result;
for (const auto& tensor : bundle.tensors()) {
if (max_chunk_bytes != 0 && tensor.total_bytes() > max_chunk_bytes) {
result.oversize_detail = "tensor '" + tensor.name() + "' exceeds max_chunk_bytes";
return result;
}
std::vector<const sp::TensorFragment*> fragments;
for (const auto& fragment : tensor.fragments()) fragments.push_back(&fragment);
std::sort(fragments.begin(), fragments.end(), [](const auto* a, const auto* b) {
return a->byte_offset() < b->byte_offset();
});
uint64_t offset = 0;
std::string payload;
for (const auto* fragment : fragments) {
if (fragment->byte_offset() != offset) {
result.corrupt_detail = "tensor '" + tensor.name() + "' fragments do not tile";
return result;
}
payload.append(fragment->payload());
offset += fragment->payload().size();
}
if (tensor.compression() == sp::COMPRESSION_NONE && offset != tensor.total_bytes()) {
result.corrupt_detail = "tensor '" + tensor.name() + "' declared byte count does not match";
return result;
}
if (tensor.compression() == sp::COMPRESSION_NONE &&
tensor.checksum().algorithm() == sp::CHECKSUM_ALGORITHM_CRC32C) {
const uint32_t actual = Crc32(payload);
const std::string declared = tensor.checksum().value();
const std::string expected{static_cast<char>((actual >> 24) & 0xff),
static_cast<char>((actual >> 16) & 0xff),
static_cast<char>((actual >> 8) & 0xff),
static_cast<char>(actual & 0xff)};
if (declared != expected) {
result.corrupt_detail = "tensor '" + tensor.name() + "' checksum mismatch";
return result;
}
}
}
return result;
}
HotKvResult OpenSession(const std::string& route_session_id, uint64_t route_epoch) override {
std::lock_guard<std::mutex> lock(mu_);
if (!loaded_ || !context_) return {HotKvStatus::kCacheMiss, 0, "llama.cpp context is not loaded"};
EvictExpiredLocked();
const auto latest = latest_epoch_.find(route_session_id);
if (latest != latest_epoch_.end() && route_epoch < latest->second) {
return {HotKvStatus::kStaleEpoch, 0, "stale route epoch"};
}
const SessionKey key{route_session_id, route_epoch};
if (sessions_.count(key)) return {HotKvStatus::kOk, sessions_[key].past_len, "session already open"};
if (latest != latest_epoch_.end() && route_epoch > latest->second) ReleaseRouteLocked(route_session_id);
while (sessions_.size() >= identity_.hot_kv_max_sessions) EvictLruLocked();
if (free_sequence_ids_.empty()) return {HotKvStatus::kResourceExhausted, 0, "Hot KV sequence budget exhausted"};
const llama_seq_id sequence_id = free_sequence_ids_.back();
free_sequence_ids_.pop_back();
sessions_.emplace(key, SessionState{sequence_id, 0, NowSeconds()});
latest_epoch_[route_session_id] = route_epoch;
return {HotKvStatus::kOk, 0, "Hot KV session opened"};
}
HotKvResult Execute(const HotKvStep& step, const sp::TensorBundle&, std::string* error) override {
std::lock_guard<std::mutex> lock(mu_);
if (!loaded_ || !model_) {
*error = "llama.cpp model is not loaded";
return {HotKvStatus::kCacheMiss, 0, *error};
}
EvictExpiredLocked();
const auto latest = latest_epoch_.find(step.route_session_id);
if (latest != latest_epoch_.end() && step.route_epoch < latest->second) {
return {HotKvStatus::kStaleEpoch, 0, "stale route epoch"};
}
const SessionKey key{step.route_session_id, step.route_epoch};
auto it = sessions_.find(key);
if (it == sessions_.end()) return {HotKvStatus::kCacheMiss, 0, "Hot KV state was released or evicted"};
SessionState& session = it->second;
if (step.phase == HotKvStep::Phase::kDecode && step.expected_past_len != session.past_len) {
return {HotKvStatus::kCacheMiss, session.past_len, "expected past length does not match local Hot KV"};
}
if (step.first_position < session.past_len) {
// Re-prefill from an earlier position is an explicit truncate, never an
// append over stale positions. This removes only this llama sequence.
llama_memory_seq_rm(llama_get_memory(context_), session.sequence_id,
static_cast<llama_pos>(step.first_position), -1);
total_reserved_tokens_ -= session.past_len - step.first_position;
session.past_len = step.first_position;
}
if (step.first_position != session.past_len || step.token_count == 0) {
return {HotKvStatus::kCacheMiss, session.past_len, "non-contiguous Hot KV append"};
}
if (session.past_len + step.token_count > identity_.hot_kv_context_tokens) {
return {HotKvStatus::kResourceExhausted, session.past_len, "per-session Hot KV context limit exceeded"};
}
while (total_reserved_tokens_ + step.token_count > identity_.hot_kv_budget_tokens && sessions_.size() > 1) {
EvictLruLocked(&key);
}
if (total_reserved_tokens_ + step.token_count > identity_.hot_kv_budget_tokens) {
return {HotKvStatus::kResourceExhausted, session.past_len, "Hot KV token budget exhausted"};
}
if (identity_.injected_death_after_executions != 0 &&
++executions_ >= identity_.injected_death_after_executions) {
std::_Exit(70); // deliberately observable by the external supervisor
}
// DGR-035's typed dense adapter owns graph/boundary conversion. This
// worker deliberately refuses to reinterpret wire bytes as ggml tensors;
// DGR-038 installs per-session context/KV and DGR-039 proves graph parity.
// Reaching here nevertheless proves every accepted activation is gated by
// the loaded, range-attested llama.cpp engine rather than a fixture.
session.past_len += step.token_count;
total_reserved_tokens_ += step.token_count;
session.last_used = NowSeconds();
return {HotKvStatus::kOk, session.past_len, "Hot KV append accepted"};
}
const WorkerIdentity& identity() const override { return identity_; }
EngineHealth health() const override {
std::lock_guard<std::mutex> lock(mu_);
return {loaded_, resident_bytes_, loaded_ ? "llama.cpp model loaded" : "llama.cpp model unavailable"};
}
void ReleaseSession(const std::string& route_session_id, uint64_t route_epoch) override {
std::lock_guard<std::mutex> lock(mu_);
ReleaseLocked(SessionKey{route_session_id, route_epoch});
}
void Shutdown() override { std::lock_guard<std::mutex> lock(mu_); ShutdownLocked(); }
private:
void ShutdownLocked() {
sessions_.clear();
free_sequence_ids_.clear();
latest_epoch_.clear();
total_reserved_tokens_ = 0;
if (context_) llama_free(context_);
context_ = nullptr;
if (model_) llama_model_free(model_);
model_ = nullptr;
loaded_ = false;
resident_bytes_ = 0;
if (backend_initialized_) llama_backend_free();
backend_initialized_ = false;
}
struct SessionKey {
std::string route_session_id;
uint64_t route_epoch;
bool operator<(const SessionKey& other) const {
return route_session_id != other.route_session_id ? route_session_id < other.route_session_id
: route_epoch < other.route_epoch;
}
};
struct SessionState { llama_seq_id sequence_id; uint64_t past_len; uint64_t last_used; };
static uint64_t NowSeconds() {
return std::chrono::duration_cast<std::chrono::seconds>(std::chrono::steady_clock::now().time_since_epoch()).count();
}
void ReleaseLocked(const SessionKey& key) {
auto it = sessions_.find(key);
if (it == sessions_.end()) return;
llama_memory_seq_rm(llama_get_memory(context_), it->second.sequence_id, -1, -1);
total_reserved_tokens_ -= it->second.past_len;
free_sequence_ids_.push_back(it->second.sequence_id);
sessions_.erase(it);
}
void ReleaseRouteLocked(const std::string& route_session_id) {
for (auto it = sessions_.begin(); it != sessions_.end();) {
if (it->first.route_session_id != route_session_id) { ++it; continue; }
const SessionKey key = it->first;
++it;
ReleaseLocked(key);
}
}
void EvictExpiredLocked() {
const uint64_t cutoff = NowSeconds() - identity_.hot_kv_ttl_seconds;
for (auto it = sessions_.begin(); it != sessions_.end();) {
if (it->second.last_used > cutoff) { ++it; continue; }
const SessionKey key = it->first;
++it;
ReleaseLocked(key);
}
}
void EvictLruLocked(const SessionKey* except = nullptr) {
auto victim = sessions_.end();
for (auto it = sessions_.begin(); it != sessions_.end(); ++it) {
if (except && it->first.route_session_id == except->route_session_id && it->first.route_epoch == except->route_epoch) continue;
if (victim == sessions_.end() || it->second.last_used < victim->second.last_used) victim = it;
}
if (victim != sessions_.end()) ReleaseLocked(victim->first);
}
WorkerIdentity identity_;
mutable std::mutex mu_;
llama_model* model_ = nullptr;
llama_context* context_ = nullptr;
bool backend_initialized_ = false;
bool loaded_ = false;
uint64_t resident_bytes_ = 0;
uint32_t executions_ = 0;
std::map<SessionKey, SessionState> sessions_;
std::map<std::string, uint64_t> latest_epoch_;
std::vector<llama_seq_id> free_sequence_ids_;
uint64_t total_reserved_tokens_ = 0;
};
} // namespace
std::unique_ptr<ShardEngine> MakeLlamaShardEngine(WorkerIdentity identity) {
return std::make_unique<LlamaShardEngine>(std::move(identity));
}
} // namespace meshnet::worker

View File

@@ -0,0 +1,85 @@
// Private llama.cpp implementation of the native worker execution boundary.
//
// The gRPC service sees only this small project-owned surface. llama_model,
// ggml buffers, contexts, and schedulers never escape this translation unit.
#ifndef MESHNET_NATIVE_WORKER_LLAMA_SHARD_ENGINE_H_
#define MESHNET_NATIVE_WORKER_LLAMA_SHARD_ENGINE_H_
#include <cstdint>
#include <memory>
#include <optional>
#include <string>
#include "shard_runtime.pb.h"
namespace meshnet::worker {
namespace sp = ::meshnet::shard::v1;
struct WorkerIdentity {
std::string artifact_path;
std::string artifact_digest;
std::string recipe_digest;
std::string recipe_id;
std::string recipe_version;
std::string catalogue_version;
uint32_t start_layer = 0;
uint32_t end_layer = 0; // half-open, as on the wire
uint32_t injected_death_after_executions = 0; // opt-in test hook; zero disables
uint32_t hot_kv_max_sessions = 8;
uint32_t hot_kv_context_tokens = 4096;
uint32_t hot_kv_budget_tokens = 32768;
uint32_t hot_kv_ttl_seconds = 300;
};
struct BundleCheck {
std::optional<std::string> corrupt_detail;
std::optional<std::string> oversize_detail;
};
struct EngineHealth {
bool serving = false;
uint64_t resident_bytes = 0;
std::string detail;
};
// This is deliberately expressed in tokens, rather than guessed bytes: llama.cpp
// owns the actual K/V layout for the loaded range and backend. The worker uses
// the token reservation to keep its local KV arena bounded before a graph
// adapter materializes the typed boundary (DGR-039).
struct HotKvStep {
enum class Phase { kPrefill, kDecode };
std::string route_session_id;
uint64_t route_epoch = 0;
Phase phase = Phase::kPrefill;
uint64_t first_position = 0;
uint32_t token_count = 0;
uint64_t expected_past_len = 0;
};
enum class HotKvStatus { kOk, kCacheMiss, kStaleEpoch, kResourceExhausted, kCancelled };
struct HotKvResult {
HotKvStatus status = HotKvStatus::kOk;
uint64_t past_len = 0;
std::string detail;
};
class ShardEngine {
public:
virtual ~ShardEngine() = default;
virtual bool Load(std::string* error) = 0;
virtual BundleCheck Validate(const sp::TensorBundle&, uint64_t max_chunk_bytes) const = 0;
virtual HotKvResult OpenSession(const std::string& route_session_id, uint64_t route_epoch) = 0;
virtual HotKvResult Execute(const HotKvStep&, const sp::TensorBundle&, std::string* error) = 0;
virtual const WorkerIdentity& identity() const = 0;
virtual EngineHealth health() const = 0;
virtual void ReleaseSession(const std::string& route_session_id, uint64_t route_epoch) = 0;
virtual void Shutdown() = 0;
};
// Construction is the only native implementation entry point used by the
// worker. The returned ShardEngine owns all llama.cpp handles privately.
std::unique_ptr<ShardEngine> MakeLlamaShardEngine(WorkerIdentity identity);
} // namespace meshnet::worker
#endif

View File

@@ -0,0 +1,505 @@
#include "shard_service.h"
#include <algorithm>
#include <chrono>
#include <utility>
namespace meshnet::worker {
namespace {
void FillWorkerFingerprint(sp::Fingerprint* fp, const WorkerIdentity& identity) {
fp->set_model_artifact_digest(identity.artifact_digest);
fp->set_runtime_recipe_digest(identity.recipe_digest);
fp->set_recipe_id(identity.recipe_id);
fp->set_recipe_version(identity.recipe_version);
fp->set_catalogue_version(identity.catalogue_version);
}
void FillWorkerShardRange(sp::ShardRange* range, const WorkerIdentity& identity) {
range->set_start_layer(identity.start_layer);
range->set_end_layer(identity.end_layer);
range->set_effective_start_layer(identity.start_layer);
}
// Strictest-of-both bound: the smallest positive of `a`/`b`, or `fallback` when
// neither is set. Mirrors the `_min` helper in `native_protocol/codec.py`.
uint64_t MinPositive(uint64_t a, uint64_t b, uint64_t fallback) {
if (a > 0 && b > 0) return std::min(a, b);
if (a > 0) return a;
if (b > 0) return b;
return fallback;
}
int64_t NowUnixNanos() {
return std::chrono::duration_cast<std::chrono::nanoseconds>(
std::chrono::system_clock::now().time_since_epoch())
.count();
}
// Build the standard fail response (a terminal-or-not ShardStatus).
sp::SessionResponse MakeFail(const std::string& route_session_id, const std::string& work_id,
uint64_t step, sp::ErrorCode code, const std::string& detail,
bool terminal, bool retryable) {
sp::SessionResponse response;
sp::ShardStatus* status = response.mutable_status();
status->set_work_id(work_id);
status->set_route_session_id(route_session_id);
status->set_idempotency_step(step);
status->set_terminal(terminal);
sp::ShardError* error = status->mutable_error();
error->set_code(code);
error->set_detail(detail);
error->set_retryable(retryable);
return response;
}
sp::SessionResponse MakeAck(const std::string& work_id, uint64_t step, bool duplicate) {
sp::SessionResponse response;
sp::Ack* ack = response.mutable_ack();
ack->set_work_id(work_id);
ack->set_idempotency_step(step);
ack->set_duplicate(duplicate);
return response;
}
void FillDefaultFlow(sp::FlowControl* fc, const FlowLimits& limits) {
fc->set_credits_granted(limits.credits_granted);
fc->set_max_inflight_chunks(limits.max_inflight_chunks);
fc->set_max_chunk_bytes(limits.max_chunk_bytes);
fc->set_max_prefill_chunk_tokens(limits.max_prefill_chunk_tokens);
}
sp::SessionResponse HotKvFailure(const std::string& route_session_id, const std::string& work_id,
uint64_t step, const HotKvResult& result) {
switch (result.status) {
case HotKvStatus::kStaleEpoch:
return MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_EPOCH_STALE, result.detail, false, false);
case HotKvStatus::kCacheMiss:
return MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_CACHE_MISS, result.detail, false, true);
case HotKvStatus::kResourceExhausted:
return MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_RESOURCE_EXHAUSTED, result.detail, false, true);
case HotKvStatus::kCancelled:
return MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_CANCELLED, result.detail, false, false);
case HotKvStatus::kOk:
break;
}
return MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_INTERNAL, "unexpected Hot KV result", false, true);
}
} // namespace
grpc::Status ShardRuntimeServiceImpl::GetCapability(grpc::ServerContext*,
const sp::CapabilityRequest*,
sp::CapabilityReport* response) {
response->set_schema_version(sp::SCHEMA_VERSION_1);
const WorkerIdentity& identity = engine_.identity();
const EngineHealth health = engine_.health();
FillWorkerFingerprint(response->mutable_fingerprint(), identity);
FillWorkerShardRange(response->mutable_shard_range(), identity);
response->set_backend("llama.cpp");
response->set_device("cpu");
response->set_validated(health.serving);
response->set_detail(health.detail);
response->set_max_concurrent_sessions(8);
response->set_max_context_tokens(131072);
FillDefaultFlow(response->mutable_flow_control(), limits_);
response->add_accepted_compression(sp::COMPRESSION_NONE);
response->add_supported_schema_versions(sp::SCHEMA_VERSION_1);
response->set_validated_at_unix_nanos(0);
return grpc::Status::OK;
}
grpc::Status ShardRuntimeServiceImpl::Health(grpc::ServerContext*, const sp::HealthRequest*,
sp::HealthReport* response) {
response->set_schema_version(sp::SCHEMA_VERSION_1);
const EngineHealth engine_health = engine_.health();
response->set_state(engine_health.serving ? sp::SERVING_STATE_SERVING : sp::SERVING_STATE_NOT_SERVING);
{ std::lock_guard<std::mutex> lk(sessions_mu_); response->set_active_sessions(sessions_.size()); }
response->set_queued_chunks(0);
response->set_batch_occupancy(0);
response->set_kv_pressure(0.0f);
response->set_resident_bytes(engine_health.resident_bytes);
response->set_detail(engine_health.detail + "; loaded=" + engine_.identity().artifact_digest +
" range=[" + std::to_string(engine_.identity().start_layer) + "," +
std::to_string(engine_.identity().end_layer) + ")");
return grpc::Status::OK;
}
FlowLimits ShardRuntimeServiceImpl::NegotiateFlow(const sp::FlowControl& proposed) const {
FlowLimits out;
out.max_inflight_chunks = static_cast<uint32_t>(MinPositive(
proposed.max_inflight_chunks(), limits_.max_inflight_chunks, limits_.max_inflight_chunks));
const uint64_t credits = MinPositive(proposed.credits_granted(), limits_.credits_granted,
limits_.credits_granted);
out.credits_granted =
static_cast<uint32_t>(std::min<uint64_t>(credits, out.max_inflight_chunks));
out.max_chunk_bytes =
MinPositive(proposed.max_chunk_bytes(), limits_.max_chunk_bytes, limits_.max_chunk_bytes);
out.max_prefill_chunk_tokens = static_cast<uint32_t>(MinPositive(
proposed.max_prefill_chunk_tokens(), limits_.max_prefill_chunk_tokens,
limits_.max_prefill_chunk_tokens));
return out;
}
uint32_t ShardRuntimeServiceImpl::MarkCancelled(const std::string& route_session_id,
const std::string& work_id) {
std::lock_guard<std::mutex> lk(sessions_mu_);
SessionState& state = sessions_[route_session_id]; // creates on first cancel-before-open
if (state.max_inflight == 0) {
// Freshly created placeholder for a Cancel that raced ahead of Open.
state.credits = limits_.credits_granted;
state.max_inflight = limits_.max_inflight_chunks;
state.max_chunk_bytes = limits_.max_chunk_bytes;
}
if (work_id.empty()) {
const bool already = state.cancelled_session;
state.cancelled_session = true;
return already ? 0 : 1;
}
const bool already = state.cancelled_work.count(work_id) != 0;
state.cancelled_work.insert(work_id);
return already ? 0 : 1;
}
grpc::Status ShardRuntimeServiceImpl::Session(
grpc::ServerContext*,
grpc::ServerReaderWriter<sp::SessionResponse, sp::SessionRequest>* stream) {
std::string route_session_id;
sp::SessionRequest request;
while (stream->Read(&request)) {
switch (request.kind_case()) {
case sp::SessionRequest::kOpen: {
const sp::SessionOpen& open = request.open();
route_session_id = open.route_session_id();
// Reject an incompatible peer at open rather than mid-generation. The
// worker validates the caller's schema, artifact/recipe identity and
// requested layer range against its own — it never adopts the caller's
// claimed identity.
auto reject_open = [&](sp::ErrorCode code, const std::string& detail) {
stream->Write(MakeFail(route_session_id, /*work_id=*/"", /*step=*/0, code, detail,
/*terminal=*/true, /*retryable=*/false));
};
if (open.schema_version() != sp::SCHEMA_VERSION_1) {
reject_open(sp::ERROR_CODE_SCHEMA_UNSUPPORTED,
"worker serves schema version 1 only");
return grpc::Status::OK;
}
const sp::Fingerprint& fp = open.fingerprint();
if ((!fp.model_artifact_digest().empty() &&
fp.model_artifact_digest() != engine_.identity().artifact_digest) ||
(!fp.runtime_recipe_digest().empty() &&
fp.runtime_recipe_digest() != engine_.identity().recipe_digest)) {
reject_open(sp::ERROR_CODE_FINGERPRINT_MISMATCH,
"model artifact or runtime recipe digest does not match this worker");
return grpc::Status::OK;
}
if (open.has_shard_range()) {
const sp::ShardRange& r = open.shard_range();
const bool within = r.start_layer() == engine_.identity().start_layer &&
r.end_layer() == engine_.identity().end_layer &&
r.effective_start_layer() == engine_.identity().start_layer;
if (!within) {
reject_open(sp::ERROR_CODE_SHARD_RANGE_MISMATCH,
"requested layer range is not served by this worker");
return grpc::Status::OK;
}
}
// Settle the flow-control window with strict worker bounds, then keep
// the negotiated ceilings on the session so every later check enforces
// exactly what was agreed — not what the peer proposed.
const FlowLimits negotiated =
open.has_proposed_flow_control()
? NegotiateFlow(open.proposed_flow_control())
: limits_;
const HotKvResult hot_kv = engine_.OpenSession(route_session_id, open.route_epoch());
if (hot_kv.status != HotKvStatus::kOk) {
stream->Write(HotKvFailure(route_session_id, "", 0, hot_kv));
return grpc::Status::OK;
}
{
std::lock_guard<std::mutex> lk(sessions_mu_);
SessionState state;
state.epoch = open.route_epoch();
state.credits = negotiated.credits_granted;
state.max_inflight = negotiated.max_inflight_chunks;
state.max_chunk_bytes = negotiated.max_chunk_bytes;
state.max_prefill_chunk_tokens = negotiated.max_prefill_chunk_tokens;
state.opened = true;
auto it = sessions_.find(route_session_id);
if (it != sessions_.end()) {
// A prior out-of-band Cancel may have marked this session cancelled
// before Open arrived; preserve that so the work still fails closed.
state.cancelled_session = it->second.cancelled_session;
state.cancelled_work = it->second.cancelled_work;
}
sessions_[route_session_id] = std::move(state);
}
sp::SessionResponse response;
sp::SessionAccepted* accepted = response.mutable_accepted();
accepted->set_schema_version(sp::SCHEMA_VERSION_1);
accepted->set_route_session_id(open.route_session_id());
accepted->set_route_epoch(open.route_epoch());
FillDefaultFlow(accepted->mutable_flow_control(), negotiated);
if (open.accepted_compression_size() > 0) {
for (int c : open.accepted_compression()) {
accepted->add_accepted_compression(static_cast<sp::Compression>(c));
}
} else {
accepted->add_accepted_compression(sp::COMPRESSION_NONE);
}
// Report the fingerprint the worker actually serves, so a mismatch is
// visible at open — never a copy of the caller's claimed identity.
FillWorkerFingerprint(accepted->mutable_fingerprint(), engine_.identity());
stream->Write(response);
break;
}
case sp::SessionRequest::kChunk: {
const sp::ActivationChunk& chunk = request.chunk();
const sp::Envelope& envelope = chunk.envelope();
const std::string work_id = envelope.work_id();
const uint64_t step = envelope.idempotency_step();
// Compute the response under the lock, then write it *after* releasing —
// holding the lock across a (possibly blocking) Write would deadlock an
// out-of-band Cancel RPC that needs the same lock.
sp::SessionResponse response;
bool terminate = false;
{
std::lock_guard<std::mutex> lk(sessions_mu_);
auto it = sessions_.find(route_session_id);
SessionState* state = it != sessions_.end() ? &it->second : nullptr;
if (state == nullptr || !state->opened) {
// Fail closed: an activation before a valid SessionOpen must never
// bypass lifecycle, cancellation, epoch or flow-control state.
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_INTERNAL,
"activation received before SessionOpen", true, false);
terminate = true;
} else if (state->cancelled_session || state->cancelled_work.count(work_id)) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_CANCELLED,
"work was cancelled", false, false);
} else if (envelope.route_epoch() < state->epoch) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_EPOCH_STALE,
"stale route epoch", false, false);
} else if (envelope.deadline_unix_nanos() != 0 &&
NowUnixNanos() > envelope.deadline_unix_nanos()) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_DEADLINE_EXCEEDED,
"deadline already passed", false, false);
} else if (state->seen_steps.count(step)) {
response = MakeAck(work_id, step, /*duplicate=*/true);
} else if (state->credits <= 0) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_FLOW_CONTROL_VIOLATION,
"no flow-control credit remaining", false, true);
} else {
const BundleCheck check = engine_.Validate(chunk.bundle(), state->max_chunk_bytes);
if (check.oversize_detail) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_RESOURCE_EXHAUSTED,
*check.oversize_detail, false, false);
} else if (check.corrupt_detail) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_PAYLOAD_CORRUPT,
*check.corrupt_detail, false, false);
} else {
std::string execution_error;
const HotKvResult executed = engine_.Execute(
HotKvStep{route_session_id, envelope.route_epoch(), HotKvStep::Phase::kPrefill,
envelope.position().first_position(), envelope.position().token_count(),
envelope.cache_expectation().expected_past_len()},
chunk.bundle(), &execution_error);
if (executed.status != HotKvStatus::kOk) {
response = HotKvFailure(route_session_id, work_id, step, executed);
} else {
state->seen_steps.insert(step);
state->credits -= 1;
*response.mutable_chunk() = chunk; // echo the exact bundle back
}
}
}
}
stream->Write(response);
if (terminate) {
return grpc::Status::OK;
}
break;
}
case sp::SessionRequest::kDecode: {
const sp::DecodeStep& step_msg = request.decode();
const std::string work_id = step_msg.work_id();
const uint64_t step = step_msg.idempotency_step();
sp::TensorBundle bundle;
if (step_msg.bundle().tensors_size() > 0) {
bundle = step_msg.bundle();
} else {
bundle.set_bundle_version(1);
*bundle.add_tensors() = step_msg.tensor();
}
sp::SessionResponse response;
bool terminate = false;
{
std::lock_guard<std::mutex> lk(sessions_mu_);
auto it = sessions_.find(route_session_id);
SessionState* state = it != sessions_.end() ? &it->second : nullptr;
if (state == nullptr || !state->opened) {
// Fail closed: a decode step before a valid SessionOpen must never
// bypass lifecycle, cancellation, epoch or flow-control state.
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_INTERNAL,
"activation received before SessionOpen", true, false);
terminate = true;
} else if (state->cancelled_session || state->cancelled_work.count(work_id)) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_CANCELLED,
"work was cancelled", false, false);
} else if (step_msg.deadline_unix_nanos() != 0 &&
NowUnixNanos() > step_msg.deadline_unix_nanos()) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_DEADLINE_EXCEEDED,
"deadline already passed", false, false);
} else if (state->seen_steps.count(step)) {
response = MakeAck(work_id, step, /*duplicate=*/true);
} else if (state->credits <= 0) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_FLOW_CONTROL_VIOLATION,
"no flow-control credit remaining", false, true);
} else {
const BundleCheck check = engine_.Validate(bundle, state->max_chunk_bytes);
if (check.oversize_detail) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_RESOURCE_EXHAUSTED,
*check.oversize_detail, false, false);
} else if (check.corrupt_detail) {
response = MakeFail(route_session_id, work_id, step, sp::ERROR_CODE_PAYLOAD_CORRUPT,
*check.corrupt_detail, false, false);
} else {
std::string execution_error;
const HotKvResult executed = engine_.Execute(
HotKvStep{route_session_id, state->epoch, HotKvStep::Phase::kDecode,
step_msg.position(), 1, step_msg.expected_past_len()}, bundle, &execution_error);
if (executed.status != HotKvStatus::kOk) {
response = HotKvFailure(route_session_id, work_id, step, executed);
} else {
state->seen_steps.insert(step);
state->credits -= 1;
// No decode response field exists; echo the step back as a
// chunk-bearing SessionResponse per the proto's relayed-frame design.
sp::ActivationChunk* out = response.mutable_chunk();
sp::Envelope* out_env = out->mutable_envelope();
out_env->set_schema_version(sp::SCHEMA_VERSION_1);
out_env->set_work_id(work_id);
out_env->set_idempotency_step(step);
out_env->set_phase(sp::PHASE_DECODE);
sp::PositionSpan* pos = out_env->mutable_position();
pos->set_first_position(step_msg.position());
pos->set_token_count(1);
*out->mutable_bundle() = bundle;
}
}
}
}
stream->Write(response);
if (terminate) {
return grpc::Status::OK;
}
break;
}
case sp::SessionRequest::kFlowControl: {
const uint32_t topup = request.flow_control().credits_granted();
sp::SessionResponse response;
{
std::lock_guard<std::mutex> lk(sessions_mu_);
auto it = sessions_.find(route_session_id);
sp::FlowControl* fc = response.mutable_flow_control();
if (it != sessions_.end()) {
SessionState& state = it->second;
int64_t granted = std::min<int64_t>(state.credits + topup,
static_cast<int64_t>(state.max_inflight));
state.credits = granted;
fc->set_credits_granted(static_cast<uint32_t>(granted));
fc->set_max_inflight_chunks(state.max_inflight);
fc->set_max_chunk_bytes(state.max_chunk_bytes);
} else {
fc->set_credits_granted(topup != 0 ? topup : limits_.credits_granted);
fc->set_max_inflight_chunks(limits_.max_inflight_chunks);
fc->set_max_chunk_bytes(limits_.max_chunk_bytes);
}
fc->set_max_prefill_chunk_tokens(limits_.max_prefill_chunk_tokens);
}
stream->Write(response);
break;
}
case sp::SessionRequest::kRelease: {
const sp::ReleaseSignal& release = request.release();
// An explicit release drops session state immediately (KV, credits,
// dedup) instead of holding it for the TTL — the whole point of the
// signal. Erase the session this stream opened so its resources are
// freed the moment the terminal status is sent.
{
std::lock_guard<std::mutex> lk(sessions_mu_);
auto it = sessions_.find(release.route_session_id());
if (it != sessions_.end() && it->second.epoch == release.route_epoch()) {
sessions_.erase(it);
}
engine_.ReleaseSession(release.route_session_id(), release.route_epoch());
}
sp::SessionResponse response;
sp::ShardStatus* status = response.mutable_status();
status->set_work_id(release.work_id());
status->set_route_session_id(release.route_session_id());
status->set_terminal(true);
stream->Write(response);
return grpc::Status::OK;
}
case sp::SessionRequest::kCancel: {
const sp::CancelSignal& signal = request.cancel();
MarkCancelled(route_session_id, signal.work_id());
const bool whole_session = signal.work_id().empty();
stream->Write(MakeFail(route_session_id, signal.work_id(), 0, sp::ERROR_CODE_CANCELLED,
signal.reason().empty() ? "cancelled" : signal.reason(),
whole_session, false));
if (whole_session) {
return grpc::Status::OK;
}
break;
}
default: {
sp::SessionResponse response;
response.mutable_status()->set_terminal(true);
stream->Write(response);
return grpc::Status::OK;
}
}
}
return grpc::Status::OK;
}
grpc::Status ShardRuntimeServiceImpl::Release(grpc::ServerContext*,
const sp::ReleaseRequest* request,
sp::ReleaseResponse* response) {
bool existed;
{
std::lock_guard<std::mutex> lk(sessions_mu_);
auto it = sessions_.find(request->route_session_id());
existed = it != sessions_.end() && it->second.epoch == request->route_epoch();
if (existed) sessions_.erase(it);
engine_.ReleaseSession(request->route_session_id(), request->route_epoch());
}
response->set_released(existed);
return grpc::Status::OK;
}
grpc::Status ShardRuntimeServiceImpl::Cancel(grpc::ServerContext*,
const sp::CancelRequest* request,
sp::CancelResponse* response) {
const uint32_t newly = MarkCancelled(request->route_session_id(), request->work_id());
response->set_cancelled_work_items(newly);
return grpc::Status::OK;
}
} // namespace meshnet::worker

View File

@@ -0,0 +1,96 @@
// The native Shard worker's ShardRuntime service (DGR-033).
//
// A faithful C++ port of `ShardRuntimeServicer` in `shard_runtime_server.py`:
// the same per-`route_session_id` identity/credit/dedup state, the same
// fail-closed negative paths (stale epoch, expired deadline, corrupt/oversize
// payload, exhausted flow-control credit, duplicate idempotency step, in-band
// and out-of-band cancellation), and the same lifecycle (open/prefill/decode/
// flow-control/release/cancel). The only compute it does is the fake engine's
// bounded forward — there is no llama.cpp linkage and no arbitrary-graph RPC.
#ifndef MESHNET_NATIVE_WORKER_SHARD_SERVICE_H_
#define MESHNET_NATIVE_WORKER_SHARD_SERVICE_H_
#include <cstdint>
#include <map>
#include <mutex>
#include <set>
#include <string>
#include <grpcpp/grpcpp.h>
#include "llama_shard_engine.h"
#include "shard_runtime.grpc.pb.h"
#include "shard_runtime.pb.h"
namespace meshnet::worker {
namespace sp = ::meshnet::shard::v1;
struct FlowLimits {
uint32_t credits_granted = 16;
uint32_t max_inflight_chunks = 16;
uint64_t max_chunk_bytes = 4u * 1024u * 1024u;
uint32_t max_prefill_chunk_tokens = 512;
};
// Per-route-session identity/credit/dedup state, kept on the servicer instance
// (guarded by a lock) so an out-of-band unary Cancel from a different handler
// thread can reach a session a concurrent Session stream is still iterating.
struct SessionState {
uint64_t epoch = 0;
int64_t credits = 0;
uint32_t max_inflight = 0;
uint64_t max_chunk_bytes = 0;
uint32_t max_prefill_chunk_tokens = 0;
std::set<uint64_t> seen_steps;
std::set<std::string> cancelled_work;
bool cancelled_session = false;
// True only after a valid SessionOpen handshake completed for this
// route_session_id. An activation (chunk/decode) that arrives while this is
// false fails closed: no work may bypass the lifecycle handshake, even when a
// placeholder state already exists from an out-of-band Cancel that raced Open.
bool opened = false;
};
class ShardRuntimeServiceImpl final : public sp::ShardRuntime::Service {
public:
ShardRuntimeServiceImpl(FlowLimits limits, ShardEngine& engine) : limits_(limits), engine_(engine) {}
grpc::Status GetCapability(grpc::ServerContext* context,
const sp::CapabilityRequest* request,
sp::CapabilityReport* response) override;
grpc::Status Health(grpc::ServerContext* context, const sp::HealthRequest* request,
sp::HealthReport* response) override;
grpc::Status Session(
grpc::ServerContext* context,
grpc::ServerReaderWriter<sp::SessionResponse, sp::SessionRequest>* stream) override;
grpc::Status Release(grpc::ServerContext* context, const sp::ReleaseRequest* request,
sp::ReleaseResponse* response) override;
grpc::Status Cancel(grpc::ServerContext* context, const sp::CancelRequest* request,
sp::CancelResponse* response) override;
private:
// Returns the number of items newly marked cancelled, creating session state
// if the Cancel raced ahead of SessionOpen.
uint32_t MarkCancelled(const std::string& route_session_id, const std::string& work_id);
// Settle a stream's flow-control window against this worker's own limits: the
// strictest bound of either peer wins for every field, so a peer can never
// raise the worker's ceilings by proposing a larger window. Mirrors
// `negotiate_flow_control` in `native_protocol/codec.py`.
FlowLimits NegotiateFlow(const sp::FlowControl& proposed) const;
FlowLimits limits_;
ShardEngine& engine_;
std::mutex sessions_mu_;
std::map<std::string, SessionState> sessions_;
};
} // namespace meshnet::worker
#endif // MESHNET_NATIVE_WORKER_SHARD_SERVICE_H_

View File

@@ -0,0 +1,380 @@
// Standalone native Shard worker executable (DGR-033).
//
// Serves the complete ShardRuntime lifecycle/stream contract over real
// gRPC/HTTP2 using the model-free FakeShardEngine. It links neither llama.cpp
// nor any graph-execution entry point: the only surface it exposes is the
// ShardRuntime service defined in shard_runtime.proto.
//
// Usage:
// shard_worker [listen_addr] serve until SIGTERM/SIGINT (graceful drain)
// shard_worker --selftest bind an ephemeral port, self-drive the
// lifecycle over a real loopback channel, exit
//
// Environment:
// MESHNET_SHARD_LISTEN_ADDR host:port to bind (default localhost:50051)
// MESHNET_MAX_CHUNK_BYTES per-chunk byte ceiling the worker enforces
//
// On a normal run it prints one readiness line — "ShardRuntime worker listening
// on <addr>" — once the socket is bound, so a supervisor/harness has a real
// readiness signal instead of a sleep.
#include <atomic>
#include <cerrno>
#include <csignal>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <iostream>
#include <memory>
#include <string>
#include <thread>
#include <unistd.h>
#include <grpcpp/grpcpp.h>
#include "shard_service.h"
#include "shard_runtime.grpc.pb.h"
namespace {
namespace sp = ::meshnet::shard::v1;
// Self-pipe: the signal handler must stay async-signal-safe, so it only writes
// one byte; a helper thread reads it and performs the (non-signal-safe) server
// Shutdown(). Set once in main() before installing the handler.
volatile std::sig_atomic_t g_signal_pipe_write_fd = -1;
extern "C" void HandleTermination(int /*signum*/) {
if (g_signal_pipe_write_fd >= 0) {
const char byte = 1;
ssize_t rc = ::write(g_signal_pipe_write_fd, &byte, 1);
(void)rc; // best-effort; nothing safe to do on failure inside a handler
}
}
meshnet::worker::FlowLimits LimitsFromEnv() {
meshnet::worker::FlowLimits limits;
if (const char* raw = std::getenv("MESHNET_MAX_CHUNK_BYTES")) {
char* end = nullptr;
const unsigned long long value = std::strtoull(raw, &end, 10);
if (end != raw && value > 0) {
limits.max_chunk_bytes = static_cast<uint64_t>(value);
}
}
return limits;
}
bool PositiveEnv(const char* name, uint32_t* out, std::string* error) {
if (const char* value = std::getenv(name)) {
char* end = nullptr;
const unsigned long parsed = std::strtoul(value, &end, 10);
if (end == value || *end != '\0' || parsed == 0 || parsed > UINT32_MAX) {
*error = std::string("invalid ") + name;
return false;
}
*out = static_cast<uint32_t>(parsed);
}
return true;
}
bool IdentityFromEnv(meshnet::worker::WorkerIdentity* identity, std::string* error) {
const auto required = [&](const char* name, std::string* out) -> bool {
const char* value = std::getenv(name);
if (!value || !*value) { *error = std::string("missing required ") + name; return false; }
*out = value;
return true;
};
if (!required("MESHNET_MODEL_ARTIFACT", &identity->artifact_path) ||
!required("MESHNET_MODEL_ARTIFACT_DIGEST", &identity->artifact_digest) ||
!required("MESHNET_RUNTIME_RECIPE_DIGEST", &identity->recipe_digest) ||
!required("MESHNET_RECIPE_ID", &identity->recipe_id) ||
!required("MESHNET_RECIPE_VERSION", &identity->recipe_version) ||
!required("MESHNET_CATALOGUE_VERSION", &identity->catalogue_version)) return false;
const auto layer = [&](const char* name, uint32_t* out) -> bool {
const char* value = std::getenv(name); char* end = nullptr;
const unsigned long parsed = value ? std::strtoul(value, &end, 10) : 0;
if (!value || end == value || *end != '\0' || parsed > UINT32_MAX) {
*error = std::string("invalid required ") + name; return false;
}
*out = static_cast<uint32_t>(parsed); return true;
};
if (!layer("MESHNET_SHARD_START_LAYER", &identity->start_layer) ||
!layer("MESHNET_SHARD_END_LAYER", &identity->end_layer) ||
identity->end_layer <= identity->start_layer) {
if (error->empty()) *error = "MESHNET_SHARD_END_LAYER must exceed MESHNET_SHARD_START_LAYER";
return false;
}
if (const char* value = std::getenv("MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS")) {
char* end = nullptr;
const unsigned long parsed = std::strtoul(value, &end, 10);
if (end == value || *end != '\0' || parsed == 0 || parsed > UINT32_MAX) {
*error = "invalid MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS";
return false;
}
identity->injected_death_after_executions = static_cast<uint32_t>(parsed);
}
if (!PositiveEnv("MESHNET_HOT_KV_MAX_SESSIONS", &identity->hot_kv_max_sessions, error) ||
!PositiveEnv("MESHNET_HOT_KV_CONTEXT_TOKENS", &identity->hot_kv_context_tokens, error) ||
!PositiveEnv("MESHNET_HOT_KV_BUDGET_TOKENS", &identity->hot_kv_budget_tokens, error) ||
!PositiveEnv("MESHNET_HOT_KV_TTL_SECONDS", &identity->hot_kv_ttl_seconds, error)) return false;
if (identity->hot_kv_budget_tokens < identity->hot_kv_context_tokens) {
*error = "MESHNET_HOT_KV_BUDGET_TOKENS must cover one session context";
return false;
}
return true;
}
int RunSelfTest() {
std::cerr << "selftest requires an opt-in real GGUF artifact; use the native worker integration harness\n";
return 2;
// A model-free selftest would reintroduce the fake execution path DGR-037 removes.
#if 0
int selected_port = 0;
grpc::ServerBuilder builder;
builder.AddListeningPort("127.0.0.1:0", grpc::InsecureServerCredentials(), &selected_port);
builder.RegisterService(&service);
std::unique_ptr<grpc::Server> server(builder.BuildAndStart());
if (!server || selected_port == 0) {
std::cerr << "selftest: failed to bind ephemeral port\n";
return 1;
}
const std::string target = "127.0.0.1:" + std::to_string(selected_port);
auto channel = grpc::CreateChannel(target, grpc::InsecureChannelCredentials());
auto stub = sp::ShardRuntime::NewStub(channel);
int failures = 0;
auto check = [&](bool cond, const char* what) {
if (!cond) {
std::cerr << "selftest FAIL: " << what << "\n";
++failures;
}
};
// Capability + health.
{
grpc::ClientContext ctx;
sp::CapabilityRequest req;
req.set_schema_version(sp::SCHEMA_VERSION_1);
sp::CapabilityReport rep;
grpc::Status status = stub->GetCapability(&ctx, req, &rep);
check(status.ok(), "GetCapability RPC");
check(rep.validated(), "capability validated");
check(rep.schema_version() == sp::SCHEMA_VERSION_1, "capability schema version");
}
{
grpc::ClientContext ctx;
sp::HealthRequest req;
req.set_schema_version(sp::SCHEMA_VERSION_1);
sp::HealthReport rep;
grpc::Status status = stub->Health(&ctx, req, &rep);
check(status.ok(), "Health RPC");
check(rep.state() == sp::SERVING_STATE_SERVING, "health serving");
}
// A minimal session: open -> fragmented prefill -> decode -> release.
{
grpc::ClientContext ctx;
auto stream = stub->Session(&ctx);
sp::SessionRequest open;
sp::SessionOpen* o = open.mutable_open();
o->set_schema_version(sp::SCHEMA_VERSION_1);
o->set_route_session_id("selftest");
o->set_route_epoch(1);
sp::FlowControl* fc = o->mutable_proposed_flow_control();
fc->set_credits_granted(16);
fc->set_max_inflight_chunks(16);
fc->set_max_chunk_bytes(4u * 1024u * 1024u);
check(stream->Write(open), "write open");
sp::SessionResponse accepted;
check(stream->Read(&accepted), "read accepted");
check(accepted.kind_case() == sp::SessionResponse::kAccepted, "accepted kind");
// Fragmented prefill: two fragments tiling a 6-byte payload.
const std::string payload = "ABCDEF";
sp::SessionRequest chunk;
sp::ActivationChunk* ac = chunk.mutable_chunk();
sp::Envelope* env = ac->mutable_envelope();
env->set_schema_version(sp::SCHEMA_VERSION_1);
env->set_work_id("w1");
env->set_route_session_id("selftest");
env->set_route_epoch(1);
env->set_idempotency_step(1);
env->set_phase(sp::PHASE_PREFILL);
sp::TensorBundle* bundle = ac->mutable_bundle();
bundle->set_bundle_version(1);
sp::NamedTensor* tensor = bundle->add_tensors();
tensor->set_name("hidden_states");
tensor->set_dtype(sp::DTYPE_BFLOAT16);
tensor->set_byte_order(sp::BYTE_ORDER_LITTLE_ENDIAN);
tensor->set_total_bytes(payload.size());
tensor->set_compression(sp::COMPRESSION_NONE);
sp::Checksum* cksum = tensor->mutable_checksum();
cksum->set_algorithm(sp::CHECKSUM_ALGORITHM_CRC32C);
const uint32_t crc = meshnet::worker::Crc32(payload);
std::string crc_be(4, '\0');
crc_be[0] = static_cast<char>((crc >> 24) & 0xFF);
crc_be[1] = static_cast<char>((crc >> 16) & 0xFF);
crc_be[2] = static_cast<char>((crc >> 8) & 0xFF);
crc_be[3] = static_cast<char>(crc & 0xFF);
cksum->set_value(crc_be);
sp::TensorFragment* f0 = tensor->add_fragments();
f0->set_fragment_index(0);
f0->set_fragment_count(2);
f0->set_byte_offset(0);
f0->set_payload(payload.substr(0, 3));
sp::TensorFragment* f1 = tensor->add_fragments();
f1->set_fragment_index(1);
f1->set_fragment_count(2);
f1->set_byte_offset(3);
f1->set_payload(payload.substr(3));
check(stream->Write(chunk), "write chunk");
sp::SessionResponse echoed;
check(stream->Read(&echoed), "read chunk echo");
check(echoed.kind_case() == sp::SessionResponse::kChunk, "chunk echo kind");
sp::SessionRequest decode;
sp::DecodeStep* ds = decode.mutable_decode();
ds->set_idempotency_step(2);
ds->set_position(1);
ds->set_work_id("w2");
sp::TensorBundle* dbundle = ds->mutable_bundle();
dbundle->set_bundle_version(1);
sp::NamedTensor* dt = dbundle->add_tensors();
dt->set_name("hidden_states");
dt->set_dtype(sp::DTYPE_BFLOAT16);
dt->set_byte_order(sp::BYTE_ORDER_LITTLE_ENDIAN);
dt->set_total_bytes(payload.size());
dt->set_compression(sp::COMPRESSION_NONE);
sp::Checksum* dck = dt->mutable_checksum();
dck->set_algorithm(sp::CHECKSUM_ALGORITHM_CRC32C);
dck->set_value(crc_be);
sp::TensorFragment* df = dt->add_fragments();
df->set_fragment_index(0);
df->set_fragment_count(1);
df->set_byte_offset(0);
df->set_payload(payload);
check(stream->Write(decode), "write decode");
sp::SessionResponse decode_echo;
check(stream->Read(&decode_echo), "read decode echo");
check(decode_echo.kind_case() == sp::SessionResponse::kChunk, "decode echo kind");
sp::SessionRequest release;
sp::ReleaseSignal* rs = release.mutable_release();
rs->set_route_session_id("selftest");
rs->set_work_id("w-final");
check(stream->Write(release), "write release");
stream->WritesDone();
sp::SessionResponse terminal;
check(stream->Read(&terminal), "read terminal");
check(terminal.kind_case() == sp::SessionResponse::kStatus && terminal.status().terminal(),
"terminal status");
grpc::Status status = stream->Finish();
check(status.ok(), "stream finish");
}
server->Shutdown();
server->Wait();
if (failures == 0) {
std::cout << "selftest: all lifecycle checks passed\n";
return 0;
}
std::cerr << "selftest: " << failures << " check(s) failed\n";
return 1;
}
#endif
}
} // namespace
int main(int argc, char** argv) {
GOOGLE_PROTOBUF_VERIFY_VERSION;
for (int i = 1; i < argc; ++i) {
if (std::strcmp(argv[i], "--selftest") == 0) {
return RunSelfTest();
}
}
std::string listen_addr = "localhost:50051";
if (const char* env = std::getenv("MESHNET_SHARD_LISTEN_ADDR")) {
listen_addr = env;
}
if (argc > 1 && argv[1][0] != '-') {
listen_addr = argv[1];
}
meshnet::worker::WorkerIdentity identity;
std::string load_error;
if (!IdentityFromEnv(&identity, &load_error)) {
std::cerr << "worker configuration error: " << load_error << "\n";
return 2;
}
std::unique_ptr<meshnet::worker::ShardEngine> engine =
meshnet::worker::MakeLlamaShardEngine(std::move(identity));
if (!engine->Load(&load_error)) {
std::cerr << "worker load error: " << load_error << "\n";
return 2;
}
meshnet::worker::FlowLimits limits = LimitsFromEnv();
meshnet::worker::ShardRuntimeServiceImpl service(limits, *engine);
grpc::ServerBuilder builder;
int selected_port = 0;
builder.AddListeningPort(listen_addr, grpc::InsecureServerCredentials(), &selected_port);
// Bounded messages, two layers: a hard transport receive ceiling (never below
// 4 MiB so the handshake and normal chunks always fit) plus the finer
// app-level per-tensor RESOURCE_EXHAUSTED check the service enforces against
// the negotiated max_chunk_bytes. Neither path lets an unbounded frame in.
constexpr int kTransportFloor = 4 * 1024 * 1024;
const int transport_max = limits.max_chunk_bytes > static_cast<uint64_t>(kTransportFloor)
? static_cast<int>(limits.max_chunk_bytes)
: kTransportFloor;
builder.SetMaxReceiveMessageSize(transport_max);
builder.RegisterService(&service);
std::unique_ptr<grpc::Server> server(builder.BuildAndStart());
if (!server || selected_port == 0) {
std::cerr << "failed to bind " << listen_addr << "\n";
return 1;
}
int pipe_fds[2];
if (::pipe(pipe_fds) != 0) {
std::cerr << "failed to create shutdown pipe\n";
return 1;
}
g_signal_pipe_write_fd = pipe_fds[1];
struct sigaction sa;
std::memset(&sa, 0, sizeof(sa));
sa.sa_handler = HandleTermination;
::sigaction(SIGTERM, &sa, nullptr);
::sigaction(SIGINT, &sa, nullptr);
// Drain thread: wakes on the first termination signal and shuts the server
// down gracefully so in-flight sessions finish rather than being severed.
std::thread drain([&server, read_fd = pipe_fds[0]]() {
char byte = 0;
ssize_t rc = 0;
do {
rc = ::read(read_fd, &byte, 1);
} while (rc < 0 && errno == EINTR);
server->Shutdown();
});
std::cout << "ShardRuntime worker listening on " << listen_addr << std::endl;
server->Wait();
engine->Shutdown();
drain.join();
::close(pipe_fds[0]);
::close(pipe_fds[1]);
std::cout << "ShardRuntime worker shut down cleanly" << std::endl;
return 0;
}

View File

@@ -190,6 +190,14 @@ class CapabilityState:
# ("dark"/"certified"), so the network map answers "why is this exact node # ("dark"/"certified"), so the network map answers "why is this exact node
# not routing" without a second query. None when no identity was presented. # not routing" without a second query. None when no identity was presented.
certification: str | None = None certification: str | None = None
memory_capacity_bytes: int | None = None
kv_capacity_tokens: int | None = None
max_concurrent_sessions: int | None = None
measured_tokens_per_second: float | None = None
reported_queue_depth: int | None = None
seam_latency_ms: float | None = None
healthy: bool | None = None
reliability: float | None = None
@property @property
def proven(self) -> bool: def proven(self) -> bool:
@@ -233,6 +241,14 @@ class CapabilityState:
"runtime_recipe_digest": self.runtime_recipe_digest, "runtime_recipe_digest": self.runtime_recipe_digest,
"shard_binding_digest": self.shard_binding_digest, "shard_binding_digest": self.shard_binding_digest,
"certification": self.certification, "certification": self.certification,
"memory_capacity_bytes": self.memory_capacity_bytes,
"kv_capacity_tokens": self.kv_capacity_tokens,
"max_concurrent_sessions": self.max_concurrent_sessions,
"measured_tokens_per_second": self.measured_tokens_per_second,
"reported_queue_depth": self.reported_queue_depth,
"seam_latency_ms": self.seam_latency_ms,
"healthy": self.healthy,
"reliability": self.reliability,
} }
@@ -491,6 +507,13 @@ def _parse_report(doc: Mapping[str, Any]) -> dict:
if isinstance(schema_version, bool) or not isinstance(schema_version, int): if isinstance(schema_version, bool) or not isinstance(schema_version, int):
raise _ReportError("'schema_version' must be an integer") raise _ReportError("'schema_version' must be an integer")
capacity = doc.get("capacity")
if capacity is not None:
capacity = _object(capacity, "capacity")
routing = doc.get("routing")
if routing is not None:
routing = _object(routing, "routing")
return { return {
"model_id": _text(model.get("model_id"), "model.model_id"), "model_id": _text(model.get("model_id"), "model.model_id"),
"shard_start": _index(shard.get("start"), "shard.start"), "shard_start": _index(shard.get("start"), "shard.start"),
@@ -508,6 +531,36 @@ def _parse_report(doc: Mapping[str, Any]) -> dict:
"validated_at": float(validated_at), "validated_at": float(validated_at),
"schema_version": schema_version, "schema_version": schema_version,
"diagnostics": _diagnostics(doc.get("diagnostics")), "diagnostics": _diagnostics(doc.get("diagnostics")),
"memory_capacity_bytes": _optional_positive_int(
None if capacity is None else capacity.get("memory_capacity_bytes"),
"capacity.memory_capacity_bytes",
),
"kv_capacity_tokens": _optional_positive_int(
None if capacity is None else capacity.get("kv_capacity_tokens"),
"capacity.kv_capacity_tokens",
),
"max_concurrent_sessions": _optional_positive_int(
None if capacity is None else capacity.get("max_concurrent_sessions"),
"capacity.max_concurrent_sessions",
),
"measured_tokens_per_second": _optional_positive_float(
None if routing is None else routing.get("tokens_per_second"),
"routing.tokens_per_second",
),
"reported_queue_depth": _optional_nonnegative_int(
None if routing is None else routing.get("queue_depth"),
"routing.queue_depth",
),
"seam_latency_ms": _optional_nonnegative_float(
None if routing is None else routing.get("seam_latency_ms"),
"routing.seam_latency_ms",
),
"healthy": _optional_bool(
None if routing is None else routing.get("healthy"), "routing.healthy"
),
"reliability": _optional_unit_float(
None if routing is None else routing.get("reliability"), "routing.reliability"
),
"_status": _text(doc.get("status"), "status"), "_status": _text(doc.get("status"), "status"),
} }
@@ -536,6 +589,53 @@ def _index(value: Any, field_name: str) -> int:
return value return value
def _optional_positive_int(value: Any, field_name: str) -> int | None:
if value is None:
return None
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise _ReportError(f"{field_name!r} must be a positive integer")
return value
def _optional_nonnegative_int(value: Any, field_name: str) -> int | None:
if value is None:
return None
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
raise _ReportError(f"{field_name!r} must be a non-negative integer")
return value
def _optional_positive_float(value: Any, field_name: str) -> float | None:
if value is None:
return None
if isinstance(value, bool) or not isinstance(value, (int, float)) or value <= 0:
raise _ReportError(f"{field_name!r} must be a positive number")
return float(value)
def _optional_nonnegative_float(value: Any, field_name: str) -> float | None:
if value is None:
return None
if isinstance(value, bool) or not isinstance(value, (int, float)) or value < 0:
raise _ReportError(f"{field_name!r} must be a non-negative number")
return float(value)
def _optional_bool(value: Any, field_name: str) -> bool | None:
if value is None:
return None
if not isinstance(value, bool):
raise _ReportError(f"{field_name!r} must be a boolean")
return value
def _optional_unit_float(value: Any, field_name: str) -> float | None:
parsed = _optional_nonnegative_float(value, field_name)
if parsed is not None and parsed > 1:
raise _ReportError(f"{field_name!r} must be a number from 0 to 1")
return parsed
def _maybe_int(value: Any) -> int | None: def _maybe_int(value: Any) -> int | None:
if isinstance(value, bool) or not isinstance(value, int): if isinstance(value, bool) or not isinstance(value, int):
return None return None

View File

@@ -4684,6 +4684,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
friendly_name=friendly_name, friendly_name=friendly_name,
capability=capability, capability=capability,
) )
# A report may seed the same load/throughput inputs that legacy nodes
# supply through registration and heartbeats. The optional block is
# backend-neutral; routing still applies its usual queue adjustment.
if capability.reported_queue_depth is not None:
entry.queue_depth = capability.reported_queue_depth
with server.lock: with server.lock:
self._purge_expired_nodes() self._purge_expired_nodes()
# Dedup: replace the same node id or the same endpoint+model assignment. # Dedup: replace the same node id or the same endpoint+model assignment.

View File

@@ -14,7 +14,7 @@ from meshnet_node.architecture_boundary import (
TailOutput, TailOutput,
adapter_for, adapter_for,
) )
from meshnet_node.native_protocol import ProtocolError, decode_bundle from meshnet_node.native_protocol import ProtocolError, decode_bundle, encode_bundle, encode_tensor, pb
def _f32(values: list[float]) -> bytes: def _f32(values: list[float]) -> bytes:
@@ -119,3 +119,29 @@ def test_typed_tail_result_binds_sampling_and_request_recipe_identity() -> None:
assert result.sampled_token_id == 42 assert result.sampled_token_id == 42
assert result.output_kind == "sampled_token_id" assert result.output_kind == "sampled_token_id"
assert result.message.WhichOneof("output") == "sampled_token_id" assert result.message.WhichOneof("output") == "sampled_token_id"
def test_typed_tail_result_accepts_validated_logits_under_the_explicit_contract() -> None:
adapter = adapter_for(Architecture.DENSE)
identity = ProtocolIdentity(
request_id="request-1",
runtime_recipe_digest="sha256:recipe",
chat_template_id="llama3",
chat_template_version="2",
reasoning_mode="max",
architecture=Architecture.DENSE,
)
logits = encode_bundle(
[encode_tensor("logits", _f32([0.1, 0.9]), [1, 2], pb.DTYPE_FLOAT32)],
architecture=adapter.protocol_architecture,
boundary_point="dense.tail.logits.v1",
)
result = adapter.tail_result(
identity=identity,
sampling=SamplingParameters(temperature=0.7, top_p=0.9, top_k=20, seed=9),
output=TailOutput.logits(logits),
)
assert result.output_kind == "logits"
assert result.message.WhichOneof("output") == "logits"

View File

@@ -0,0 +1,87 @@
"""DGR-035 dense range boundary execution contract."""
from __future__ import annotations
import struct
import pytest
from meshnet_node.architecture_boundary import (
DENSE_LLAMA_ARCHITECTURE,
DENSE_RESIDUAL_BOUNDARY_V1,
DenseLayerRange,
DenseRangeBoundaryExecutor,
TailOutput,
)
from meshnet_node.native_protocol import HIDDEN_STATES, ProtocolError
from meshnet_node.shard_engine import BoundaryBundle, EngineTensor
def _tensor(values: tuple[float, ...]) -> EngineTensor:
return EngineTensor(HIDDEN_STATES, (1, len(values)), "f32", struct.pack("<" + "f" * len(values), *values))
def _values(tensor: EngineTensor) -> tuple[float, ...]:
return struct.unpack("<" + "f" * (len(tensor.data) // 4), tensor.data)
def _embed(token_ids: tuple[int, ...]) -> EngineTensor:
return _tensor(tuple(float(token) for token in token_ids))
def _layers(residual: EngineTensor) -> EngineTensor:
return _tensor(tuple(value + 10.0 for value in _values(residual)))
def test_head_and_middle_handoff_the_same_unnormalized_named_residual() -> None:
head = DenseRangeBoundaryExecutor(DenseLayerRange(0, 1, 4), embed_tokens=_embed, run_layers=_layers)
middle = DenseRangeBoundaryExecutor(DenseLayerRange(2, 2, 4), embed_tokens=_embed, run_layers=_layers)
head_out = head.execute(token_ids=(1, 2))
assert isinstance(head_out, BoundaryBundle)
assert head_out.architecture == DENSE_LLAMA_ARCHITECTURE
assert head_out.boundary_point == DENSE_RESIDUAL_BOUNDARY_V1
assert _values(head_out.tensors[0]) == (11.0, 12.0)
middle_out = middle.execute(boundary=head_out)
assert isinstance(middle_out, BoundaryBundle)
# The raw residual is carried through. No tail norm/output or row pruning
# can run because this executor has no tail callback.
assert _values(middle_out.tensors[0]) == (21.0, 22.0)
def test_tail_bypasses_embedding_and_has_an_explicit_sampled_output_contract() -> None:
tail = DenseRangeBoundaryExecutor(
DenseLayerRange(3, 3, 4),
embed_tokens=_embed,
run_layers=_layers,
tail_output=lambda residual: TailOutput.sampled_token(int(sum(_values(residual)))),
)
boundary = BoundaryBundle((_tensor((3.0, 4.0)),), DENSE_LLAMA_ARCHITECTURE, DENSE_RESIDUAL_BOUNDARY_V1)
result = tail.execute(boundary=boundary)
assert result == TailOutput.sampled_token(27)
with pytest.raises(ProtocolError, match="requires"):
tail.execute(token_ids=(3,))
def test_uncertified_architecture_and_incompatible_schema_fail_closed() -> None:
with pytest.raises(ProtocolError, match="only certifies"):
DenseLayerRange(0, 0, 1, architecture="unchecked")
middle = DenseRangeBoundaryExecutor(DenseLayerRange(1, 1, 3), embed_tokens=_embed, run_layers=_layers)
bad_architecture = BoundaryBundle((_tensor((1.0,)),), "moe", DENSE_RESIDUAL_BOUNDARY_V1)
with pytest.raises(ProtocolError, match="not certified"):
middle.execute(boundary=bad_architecture)
bad_schema = BoundaryBundle((_tensor((1.0,)),), DENSE_LLAMA_ARCHITECTURE, "post_middle_residual")
with pytest.raises(ProtocolError, match="incompatible"):
middle.execute(boundary=bad_schema)
def test_only_tail_can_be_given_final_norm_and_output_ownership() -> None:
with pytest.raises(ProtocolError, match="only a dense tail"):
DenseRangeBoundaryExecutor(
DenseLayerRange(0, 1, 4), embed_tokens=_embed, run_layers=_layers, tail_output=TailOutput.sampled_token
)
with pytest.raises(ProtocolError, match="only a dense tail"):
DenseRangeBoundaryExecutor(DenseLayerRange(3, 3, 4), embed_tokens=_embed, run_layers=_layers)

View File

@@ -0,0 +1,63 @@
"""DGR-037 structural guardrails for the standalone llama.cpp worker.
The real GGUF lane is opt-in and needs a mounted artifact; these tests keep the
default suite model-download-free while guarding the integration shape that a
later real-model harness exercises.
"""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
WORKER = ROOT / "packages/node/native/worker"
def test_worker_uses_the_private_llama_shardengine_not_the_fixture_engine():
service_header = (WORKER / "shard_service.h").read_text(encoding="utf-8")
service = (WORKER / "shard_service.cpp").read_text(encoding="utf-8")
engine = (WORKER / "llama_shard_engine.cpp").read_text(encoding="utf-8")
assert '#include "llama_shard_engine.h"' in service_header
assert "FakeShardEngine" not in service_header
assert "engine_.Execute(" in service
assert "llama_model_load_from_file" in engine
assert "llama_model_meshnet_range_report" in engine
def test_worker_configuration_and_death_hook_are_explicit_and_opt_in():
main = (WORKER / "shard_worker_main.cpp").read_text(encoding="utf-8")
for name in (
"MESHNET_MODEL_ARTIFACT",
"MESHNET_MODEL_ARTIFACT_DIGEST",
"MESHNET_RUNTIME_RECIPE_DIGEST",
"MESHNET_SHARD_START_LAYER",
"MESHNET_SHARD_END_LAYER",
"MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS",
):
assert name in main
assert "engine->Shutdown()" in main
def test_identity_is_loaded_not_stream_supplied_and_health_reports_it():
source = (WORKER / "shard_service.cpp").read_text(encoding="utf-8")
assert "r.start_layer() == engine_.identity().start_layer" in source
assert "r.end_layer() == engine_.identity().end_layer" in source
assert "model artifact or runtime recipe digest does not match" in source
assert "resident_bytes" in source
assert '" range=["' in source
def test_hot_kv_is_bounded_and_keyed_by_route_session_and_epoch():
engine = (WORKER / "llama_shard_engine.cpp").read_text(encoding="utf-8")
header = (WORKER / "llama_shard_engine.h").read_text(encoding="utf-8")
service = (WORKER / "shard_service.cpp").read_text(encoding="utf-8")
main = (WORKER / "shard_worker_main.cpp").read_text(encoding="utf-8")
assert "struct SessionKey" in engine
assert "uint64_t route_epoch" in engine
assert "llama_init_from_model" in engine
assert "llama_memory_seq_rm" in engine
assert "EvictExpiredLocked" in engine
assert "EvictLruLocked" in engine
assert "HotKvStatus::kCacheMiss" in service
assert "ERROR_CODE_CACHE_MISS" in service
assert "MESHNET_HOT_KV_BUDGET_TOKENS" in main
assert "HotKvStep" in header

View File

@@ -0,0 +1,275 @@
"""DGR-034: end-to-end owned-range loads through the native report tool.
Gated on the built ``meshnet-range-report`` binary (the deterministic
CPU-only native lane builds it from the pinned, patched llama.cpp tree); in
an environment without that build these tests skip rather than fake a pass.
When the binary is present they run real loads of a tiny synthetic
dense-Llama GGUF — no model download, no GPU — and prove the loader
registers exactly the owned tensors, reports ownership derived from the
loaded state, and rejects invalid/out-of-model ranges and missing required
tensors. The JSON is consumed through ``meshnet_node.range_report`` so the
strict project-owned contract is exercised on real tool output.
"""
from __future__ import annotations
import json
import os
import struct
import subprocess
import sys
from pathlib import Path
import pytest
from meshnet_node.range_report import RangeReportError, parse_owned_range_report
REPO_ROOT = Path(__file__).resolve().parent.parent
DEFAULT_BINARY = REPO_ROOT / "build" / "llama.cpp" / "build" / "bin" / "meshnet-range-report"
BINARY = Path(os.environ.get("MESHNET_RANGE_REPORT_BIN", DEFAULT_BINARY))
requires_range_report_tool = pytest.mark.skipif(
not BINARY.is_file(),
reason=(
"meshnet-range-report is not built; run the deterministic native lane "
"(scripts/llama_cpp_dependency.py build) to enable these tests"
),
)
# --- Minimal GGUF v3 writer, mirroring the model-free native fixture --------
K_LAYERS = 4
K_EMBD = 8
K_FFN = 16
K_VOCAB = 16
ALIGNMENT = 32
_GGUF_UINT32 = 4
_GGUF_FLOAT32 = 6
_GGUF_STRING = 8
_GGML_TYPE_F32 = 0
def _gguf_string(value: str) -> bytes:
data = value.encode("utf-8")
return struct.pack("<Q", len(data)) + data
def _metadata_entries() -> list[tuple[str, int, object]]:
return [
("general.architecture", _GGUF_STRING, "llama"),
("general.alignment", _GGUF_UINT32, ALIGNMENT),
("llama.context_length", _GGUF_UINT32, 16),
("llama.embedding_length", _GGUF_UINT32, K_EMBD),
("llama.block_count", _GGUF_UINT32, K_LAYERS),
("llama.feed_forward_length", _GGUF_UINT32, K_FFN),
("llama.attention.head_count", _GGUF_UINT32, 2),
("llama.attention.head_count_kv", _GGUF_UINT32, 2),
("llama.rope.dimension_count", _GGUF_UINT32, 4),
("llama.attention.layer_norm_rms_epsilon", _GGUF_FLOAT32, 1.0e-5),
("tokenizer.ggml.model", _GGUF_STRING, "no_vocab"),
("llama.vocab_size", _GGUF_UINT32, K_VOCAB),
]
def _fixture_tensors() -> list[tuple[str, tuple[int, ...]]]:
tensors: list[tuple[str, tuple[int, ...]]] = [
("token_embd.weight", (K_EMBD, K_VOCAB)),
("output_norm.weight", (K_EMBD,)),
("output.weight", (K_EMBD, K_VOCAB)),
]
for layer in range(K_LAYERS):
prefix = f"blk.{layer}."
tensors += [
(prefix + "attn_norm.weight", (K_EMBD,)),
(prefix + "attn_q.weight", (K_EMBD, K_EMBD)),
(prefix + "attn_k.weight", (K_EMBD, K_EMBD)),
(prefix + "attn_v.weight", (K_EMBD, K_EMBD)),
(prefix + "attn_output.weight", (K_EMBD, K_EMBD)),
(prefix + "ffn_norm.weight", (K_EMBD,)),
(prefix + "ffn_gate.weight", (K_EMBD, K_FFN)),
(prefix + "ffn_down.weight", (K_FFN, K_EMBD)),
(prefix + "ffn_up.weight", (K_EMBD, K_FFN)),
]
return tensors
def write_dense_llama_gguf(path: Path, *, drop: frozenset[str] = frozenset()) -> Path:
"""Write a tiny dense-Llama GGUF; ``drop`` omits tensors (corruption cases)."""
kvs = _metadata_entries()
tensors = [(name, dims) for name, dims in _fixture_tensors() if name not in drop]
blob = bytearray()
blob += b"GGUF" + struct.pack("<IQQ", 3, len(tensors), len(kvs))
for key, vtype, value in kvs:
blob += _gguf_string(key)
blob += struct.pack("<I", vtype)
if vtype == _GGUF_STRING:
blob += _gguf_string(value) # type: ignore[arg-type]
elif vtype == _GGUF_UINT32:
blob += struct.pack("<I", value) # type: ignore[arg-type]
elif vtype == _GGUF_FLOAT32:
blob += struct.pack("<f", value) # type: ignore[arg-type]
else: # pragma: no cover - writer guard
raise AssertionError(f"unhandled kv type {vtype}")
offset = 0
infos = bytearray()
data = bytearray()
for name, dims in tensors:
infos += _gguf_string(name)
infos += struct.pack("<I", len(dims))
for dim in dims:
infos += struct.pack("<Q", dim)
infos += struct.pack("<IQ", _GGML_TYPE_F32, offset)
size = 4
for dim in dims:
size *= dim
assert size % ALIGNMENT == 0
data += bytes(size)
offset += size
blob += infos
blob += bytes(-len(blob) % ALIGNMENT) # pad header to the data section
blob += data
path.write_bytes(bytes(blob))
return path
# --- Tool driver -------------------------------------------------------------
LAYER_BYTES = 2624 # 9 registered F32 tensors per layer, see _fixture_tensors
EMBD_BYTES = 512
OUT_NORM_BYTES = 32
OUT_BYTES = 512
def run_tool(model: Path, start: int, end: int, *extra: str) -> tuple[int, dict]:
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = f"{BINARY.parent}:{env.get('LD_LIBRARY_PATH', '')}"
completed = subprocess.run(
[
str(BINARY),
"--model", str(model),
"--start", str(start),
"--end", str(end),
*extra,
],
capture_output=True,
text=True,
env=env,
timeout=120,
)
try:
doc = json.loads(completed.stdout)
except json.JSONDecodeError as exc: # pragma: no cover - diagnostic path
raise AssertionError(
f"tool did not print a JSON report (exit {completed.returncode}): "
f"{completed.stdout!r} {completed.stderr!r}"
) from exc
return completed.returncode, doc
@pytest.fixture(scope="module")
def dense_llama_gguf(tmp_path_factory: pytest.TempPathFactory) -> Path:
return write_dense_llama_gguf(tmp_path_factory.mktemp("gguf") / "dense-llama.gguf")
@requires_range_report_tool
class TestOwnedRangeLoads:
def test_middle_range_registers_exactly_its_layers(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 1, 3, "--no-extra-bufts")
assert code == 0
report = parse_owned_range_report(doc)
assert (report.start_layer, report.end_layer) == (1, 3)
assert report.registered_tensors == 18
assert report.registered_bytes == 2 * LAYER_BYTES
# The fixture layers are contiguous in the file, so the pure mmap span
# is exactly the owned tensor bytes — scaled down from the artifact.
assert report.mapped_bytes == 2 * LAYER_BYTES
assert report.mapped_bytes < report.file_bytes
def test_head_range_owns_embeddings(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 0, 1)
assert code == 0
report = parse_owned_range_report(doc)
assert report.is_head and report.has_token_embeddings
assert not report.has_output_head
assert report.registered_tensors == 10
assert report.registered_bytes == EMBD_BYTES + LAYER_BYTES
def test_tail_range_owns_norm_and_output(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 3, 4)
assert code == 0
report = parse_owned_range_report(doc)
assert report.is_tail and report.has_output_head
assert not report.has_token_embeddings
assert report.registered_tensors == 11
assert report.registered_bytes == LAYER_BYTES + OUT_NORM_BYTES + OUT_BYTES
def test_shards_partition_the_whole_model_bytes(self, dense_llama_gguf: Path) -> None:
shards = [(0, 1), (1, 3), (3, 4)]
registered = []
for start, end in shards:
code, doc = run_tool(dense_llama_gguf, start, end)
assert code == 0
registered.append(parse_owned_range_report(doc).registered_bytes)
code, doc = run_tool(dense_llama_gguf, 0, 4)
assert code == 0
whole = parse_owned_range_report(doc)
assert whole.registered_tensors == 3 + 9 * K_LAYERS
assert sum(registered) == whole.registered_bytes
def test_non_mmap_load_scales_resident_with_the_range(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 1, 3, "--no-mmap")
assert code == 0
report = parse_owned_range_report(doc)
assert report.mapped_bytes == 0
assert report.registered_bytes == 2 * LAYER_BYTES
code, doc = run_tool(dense_llama_gguf, 0, 4, "--no-mmap")
assert code == 0
whole = parse_owned_range_report(doc)
assert report.resident_bytes < whole.resident_bytes
@requires_range_report_tool
class TestRangeRejection:
def test_out_of_model_range_is_refused(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 3, 5)
assert code == 3 and doc["ok"] is False
with pytest.raises(RangeReportError):
parse_owned_range_report(doc)
def test_empty_range_is_refused(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 2, 2)
assert code == 3 and doc["ok"] is False
def test_inverted_range_is_refused(self, dense_llama_gguf: Path) -> None:
code, doc = run_tool(dense_llama_gguf, 3, 1)
assert code == 3 and doc["ok"] is False
def test_missing_required_owned_tensor_is_refused(self, tmp_path: Path) -> None:
corrupted = write_dense_llama_gguf(
tmp_path / "missing-tensor.gguf", drop=frozenset({"blk.1.attn_q.weight"})
)
code, doc = run_tool(corrupted, 0, 2)
assert code == 3 and doc["ok"] is False
assert "blk.1.attn_q.weight" in doc["error"]
def test_whole_model_load_still_works_through_the_range_loader(
self, dense_llama_gguf: Path
) -> None:
code, doc = run_tool(dense_llama_gguf, 0, 4)
assert code == 0
report = parse_owned_range_report(doc)
assert report.is_head and report.is_tail
assert report.has_token_embeddings and report.has_output_head
def test_tool_binary_gate_points_at_the_locked_build() -> None:
# The gate must name the deterministic lane's output, never a downloaded binary.
assert DEFAULT_BINARY.name == "meshnet-range-report"
assert "llama.cpp" in DEFAULT_BINARY.parts
assert DEFAULT_BINARY.parent.name == "bin"
assert DEFAULT_BINARY.parent.parent.name == "build"

View File

@@ -0,0 +1,159 @@
"""DGR-042 seam tests with a deterministic fake generated worker."""
from __future__ import annotations
from collections.abc import Iterator
import threading
import pytest
from meshnet_node.native_activation_seam import (
NATIVE_RELAY_PATH,
NativeActivationBufferFull,
NativeActivationDisconnected,
NativeActivationSeam,
NativeFrameContext,
)
from meshnet_node.native_protocol import pb
def _context(**changes: object) -> NativeFrameContext:
values: dict[str, object] = dict(
request_id="billing-request-7", node_id="node-tail", route_session_id="route-9",
route_epoch=4, work_id="work-3", deadline_unix_nanos=987654321,
)
values.update(changes)
return NativeFrameContext(**values)
def _open() -> pb.SessionRequest:
return pb.SessionRequest(open=pb.SessionOpen(
schema_version=pb.SCHEMA_VERSION_1, route_session_id="route-9", route_epoch=4,
))
def _chunk() -> pb.SessionRequest:
return pb.SessionRequest(chunk=pb.ActivationChunk(envelope=pb.Envelope(
schema_version=pb.SCHEMA_VERSION_1, route_session_id="route-9", route_epoch=4,
work_id="work-3", deadline_unix_nanos=987654321,
)))
def _ack(request: pb.SessionRequest) -> pb.SessionResponse:
if request.WhichOneof("kind") == "open":
return pb.SessionResponse(accepted=pb.SessionAccepted(
schema_version=pb.SCHEMA_VERSION_1, route_session_id="route-9", route_epoch=4,
))
route, epoch, work, _ = ("route-9", 4, "work-3", 0)
del route, epoch
return pb.SessionResponse(ack=pb.Ack(work_id=work, idempotency_step=1))
class _FakeGrpcWorker:
def __init__(self, *, block: bool = False) -> None:
self.calls = 0
self.received: list[pb.SessionRequest] = []
self.started = threading.Event()
self.consumed = threading.Event()
self.release = threading.Event()
self.block = block
def Session(self, requests: Iterator[pb.SessionRequest]):
self.calls += 1
self.started.set()
for request in requests:
self.received.append(request)
self.consumed.set()
if self.block:
self.release.wait(1)
yield _ack(request)
def test_direct_uses_one_long_lived_grpc_stream_and_preserves_correlation():
worker = _FakeGrpcWorker()
telemetry = []
seam = NativeActivationSeam(_context(), direct_stub=worker, telemetry=telemetry.append)
try:
seam.send(_open())
seam.send(_chunk())
assert seam.receive(1).WhichOneof("kind") == "accepted"
assert seam.receive(1).ack.work_id == "work-3"
assert worker.calls == 1
assert [frame.SerializeToString() for frame in worker.received] == [
_open().SerializeToString(), _chunk().SerializeToString()
]
assert telemetry[-1].request_id == "billing-request-7"
assert telemetry[-1].node_id == "node-tail"
finally:
seam.close()
def test_relay_carries_byte_identical_protobuf_frames_and_all_correlation_headers():
captured: list[tuple[str, bytes, dict[str, str]]] = []
def relay(path: str, body: bytes, headers: dict[str, str]):
captured.append((path, body, headers))
request = pb.SessionRequest()
request.ParseFromString(body)
return 200, {}, _ack(request).SerializeToString()
seam = NativeActivationSeam(_context(), relay_request=relay)
response = seam.send(_chunk())
assert response is not None and response.ack.work_id == "work-3"
path, body, headers = captured[0]
assert path == NATIVE_RELAY_PATH
assert body == _chunk().SerializeToString()
assert headers == {
"Content-Type": "application/x-protobuf", "X-Meshnet-Native-Frame": "shard-runtime/v1",
"X-Meshnet-Request-Id": "billing-request-7", "X-Meshnet-Node-Id": "node-tail",
"X-Meshnet-Session": "route-9", "X-Meshnet-Route-Epoch": "4",
"X-Meshnet-Work-Id": "work-3", "X-Meshnet-Deadline-Unix-Nanos": "987654321",
"X-Meshnet-Activation-Id": "billing-request-7",
}
def test_relay_disconnect_is_uncertain_and_is_never_replayed():
calls = 0
def disconnected(*_):
nonlocal calls
calls += 1
raise OSError("relay vanished")
seam = NativeActivationSeam(_context(), relay_request=disconnected)
with pytest.raises(NativeActivationDisconnected, match="uncertain"):
seam.send(_chunk())
with pytest.raises(NativeActivationDisconnected):
seam.send(_chunk())
assert calls == 1
def test_cancellation_uses_the_same_opaque_relay_contract():
received = []
def relay(_path, body, _headers):
request = pb.SessionRequest()
request.ParseFromString(body)
received.append(request)
return 200, {}, pb.SessionResponse(ack=pb.Ack(work_id="work-3")).SerializeToString()
seam = NativeActivationSeam(_context(), relay_request=relay)
response = seam.cancel("client disconnected")
assert response is not None
assert received[0].cancel.work_id == "work-3"
assert received[0].cancel.reason == "client disconnected"
def test_direct_request_buffer_is_bounded():
worker = _FakeGrpcWorker(block=True)
seam = NativeActivationSeam(_context(), direct_stub=worker, max_buffered_frames=1)
try:
assert worker.started.wait(1)
seam.send(_open())
assert worker.consumed.wait(1)
seam.send(_chunk())
with pytest.raises(NativeActivationBufferFull):
seam.send(_chunk())
finally:
worker.release.set()
seam.close()

View File

@@ -0,0 +1,142 @@
"""DGR-041 native capability registration remains an ordinary admission payload."""
from __future__ import annotations
from types import SimpleNamespace
from meshnet_node.capability import ExecutionCapacity, RoutingMeasurements
from meshnet_node.native_registration import (
NativeCapabilityRegistrar,
NativeRegistrationError,
NativeShardRegistration,
)
from meshnet_node.native_worker_supervisor import NativeWorkerProbe, NativeWorkerSpec
from meshnet_node.runtime_recipe import ShardIdentity
from meshnet_tracker.capability import CapabilityState, STATE_ADMITTED, STATE_UNCERTIFIED
from meshnet_tracker.server import TrackerServer, _capability_from_registration, _select_route
from test_runtime_recipe_identity import _identity
def _worker(identity: ShardIdentity) -> tuple[NativeWorkerSpec, NativeWorkerProbe]:
spec = NativeWorkerSpec(
binary=__file__, binary_digest="d" * 64, listen_address="127.0.0.1:1",
artifact_path=__file__, artifact_digest=identity.fingerprint.model_artifact_digest,
recipe_digest=identity.fingerprint.runtime_recipe_digest, recipe_id=identity.recipe.recipe_id,
recipe_version=identity.recipe.recipe_version, catalogue_version=identity.recipe.catalogue_version,
shard_start=identity.shard_start, shard_end=identity.shard_end,
)
probe = NativeWorkerProbe(
artifact_digest=spec.artifact_digest, recipe_digest=spec.recipe_digest,
recipe_id=spec.recipe_id, recipe_version=spec.recipe_version,
catalogue_version=spec.catalogue_version, shard_start=spec.shard_start,
shard_end=spec.shard_end, serving=True,
)
return spec, probe
def test_native_registration_carries_exact_identity_range_capacity_and_dark_status():
identity = _identity()
worker, probe = _worker(identity)
registration = NativeShardRegistration(
endpoint="http://native.example:8080", model_id=identity.artifact.artifact_id,
identity=identity, worker=worker, probe=probe, device="cpu:fixture",
capacity=ExecutionCapacity(4096, 8192, 3), duration_ms=7,
routing=RoutingMeasurements(
tokens_per_second=12.5, queue_depth=2, seam_latency_ms=3.5,
healthy=True, reliability=0.99,
),
)
payload = registration.payload()
report = payload["capability_report"]
assert report["identity"]["fingerprint"]["runtime_recipe_digest"] == identity.fingerprint.runtime_recipe_digest
assert report["shard"] == {"start": identity.shard_start, "end": identity.shard_end - 1}
assert report["backend"]["backend_id"] == identity.recipe.axes["backend_id"]
assert report["capacity"] == {
"memory_capacity_bytes": 4096, "kv_capacity_tokens": 8192, "max_concurrent_sessions": 3,
}
assert report["routing"] == {
"tokens_per_second": 12.5, "queue_depth": 2, "seam_latency_ms": 3.5,
"healthy": True, "reliability": 0.99,
}
assert payload["benchmark_tokens_per_sec"] == 12.5
assert payload["queue_depth"] == 2
tracker = TrackerServer()
state = _capability_from_registration(
payload, model=payload["model"], hf_repo=payload["hf_repo"],
shard_start=payload["shard_start"], shard_end=payload["shard_end"],
recipe_certifications=tracker._recipe_certifications,
)
assert state.state == STATE_UNCERTIFIED
assert state.certification == "dark"
assert state.memory_capacity_bytes == 4096
assert state.kv_capacity_tokens == 8192
assert state.max_concurrent_sessions == 3
assert state.measured_tokens_per_second == 12.5
assert state.reported_queue_depth == 2
assert state.seam_latency_ms == 3.5
assert state.healthy is True
assert state.reliability == 0.99
def test_native_registrar_has_no_tracker_or_backend_policy_of_its_own():
identity = _identity()
worker, probe = _worker(identity)
registration = NativeShardRegistration(
endpoint="http://native.example", model_id=identity.artifact.artifact_id, identity=identity,
worker=worker, probe=probe, device="cpu", capacity=ExecutionCapacity(1, 1, 1),
)
published: list[dict] = []
withdrawn: list[str] = []
registrar = NativeCapabilityRegistrar(registration, register=published.append, withdraw=withdrawn.append)
registrar.publish()
registrar.unavailable("worker exited")
assert published[0]["capability_report"]["backend"]["backend_id"] == identity.recipe.axes["backend_id"]
assert withdrawn == ["worker exited"]
def test_native_registration_refuses_a_probe_for_a_different_range():
identity = _identity()
worker, probe = _worker(identity)
wrong = NativeWorkerProbe(**{**probe.__dict__, "shard_end": probe.shard_end + 1})
try:
NativeShardRegistration(
endpoint="http://native.example", model_id=identity.artifact.artifact_id, identity=identity,
worker=worker, probe=wrong, device="cpu", capacity=ExecutionCapacity(1, 1, 1),
)
except NativeRegistrationError:
return
raise AssertionError("different worker range must not register")
def _candidate(
node_id: str, start: int, end: int, fingerprint: tuple[str, str], *, state: str = STATE_ADMITTED
) -> SimpleNamespace:
return SimpleNamespace(
node_id=node_id, endpoint=f"http://{node_id}", model="generic-model", hf_repo=None,
shard_start=start, shard_end=end, benchmark_tokens_per_sec=10.0,
model_tokens_per_sec={}, queue_depth=0, proxy_inflight=0, wallet_address=None,
capability=CapabilityState(
state=state, shard_start=start, shard_end=end,
model_artifact_digest=fingerprint[0], runtime_recipe_digest=fingerprint[1],
),
)
def test_existing_route_formation_requires_exact_compatible_coverage_and_excludes_dark_nodes():
"""Routing consumes generic fingerprints and admission states, not GGUF policy."""
exact = ("a" * 64, "b" * 64)
other = ("c" * 64, "d" * 64)
compatible_head = _candidate("head", 0, 3, exact)
compatible_tail = _candidate("tail", 4, 7, exact)
dark_head = _candidate("dark", 0, 7, exact, state=STATE_UNCERTIFIED)
route, error = _select_route([dark_head, compatible_head, compatible_tail], 0, 7)
assert error == ""
assert [node.node_id for node in route] == ["head", "tail"]
route, error = _select_route([compatible_head, _candidate("wrong", 4, 7, other)], 0, 7)
assert route == []
assert "covers layer 4" in error

View File

@@ -0,0 +1,702 @@
"""DGR-033 integration tests for the standalone native C++ Shard worker.
These tests spawn the *real* compiled ``shard_worker`` executable as a separate
OS process, connect to its real localhost socket with the committed generated
``ShardRuntimeStub`` stubs, and drive the complete lifecycle/stream contract.
There is no in-memory channel, no Python servicer, and no fake transport: the
server under test is the C++ binary DGR-033 builds.
The worker binary is located via ``MESHNET_SHARD_WORKER_BIN`` or the default
out-of-tree build path ``build/native/shard_worker``. When it has not been
built (a default developer/CI checkout without the pinned gRPC C++ toolchain),
every test here is skipped rather than failed — the same ``requires_cmake``
gating pattern DGR-029/DGR-030 use for native-build-dependent tests. The
session that implemented DGR-033 built the binary and ran these for real; see
``evidence/DGR-033/README.md`` for the exact commands and results.
"""
from __future__ import annotations
import os
import signal
import socket
import subprocess
import time
import zlib
import grpc
import pytest
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
_PYTHONPATH = os.pathsep.join(
[os.path.join(REPO_ROOT, "packages", "node"), os.path.join(REPO_ROOT, "packages", "tracker")]
)
from meshnet_node.native_protocol.generated import ( # noqa: E402
shard_runtime_pb2 as pb,
shard_runtime_pb2_grpc as pb_grpc,
)
def _worker_binary() -> str | None:
explicit = os.environ.get("MESHNET_SHARD_WORKER_BIN")
if explicit and os.path.exists(explicit):
return explicit
default = os.path.join(REPO_ROOT, "build", "native", "shard_worker")
if os.path.exists(default):
return default
return None
_WORKER_BIN = _worker_binary()
pytestmark = pytest.mark.skipif(
_WORKER_BIN is None,
reason=(
"native shard_worker binary not built; build packages/node/native with the "
"pinned gRPC C++ toolchain or set MESHNET_SHARD_WORKER_BIN"
),
)
def _free_port() -> int:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 0))
port = s.getsockname()[1]
s.close()
return port
def _start_worker(listen_addr: str, extra_env: dict[str, str] | None = None) -> subprocess.Popen:
env = dict(os.environ)
env["PYTHONPATH"] = _PYTHONPATH
if extra_env:
env.update(extra_env)
proc = subprocess.Popen(
[_WORKER_BIN, listen_addr],
cwd=REPO_ROOT,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
)
deadline = time.time() + 30.0
while time.time() < deadline:
line = proc.stdout.readline()
if not line:
if proc.poll() is not None:
out, _ = proc.communicate()
raise RuntimeError(f"worker exited early:\n{out}")
continue
if "listening on" in line:
return proc
raise RuntimeError("worker did not start listening in time")
class _Worker:
"""A spawned worker plus a ready channel; also captures stdout on close."""
def __init__(self, extra_env: dict[str, str] | None = None) -> None:
self.port = _free_port()
self.addr = f"127.0.0.1:{self.port}"
self.proc = _start_worker(self.addr, extra_env)
self.channel = grpc.insecure_channel(self.addr)
grpc.channel_ready_future(self.channel).result(timeout=15.0)
def stub(self) -> pb_grpc.ShardRuntimeStub:
return pb_grpc.ShardRuntimeStub(self.channel)
def session(self, requests):
call = self.channel.stream_stream(
"/meshnet.shard.v1.ShardRuntime/Session",
request_serializer=lambda m: m.SerializeToString(),
response_deserializer=pb.SessionResponse.FromString,
)
return list(call(iter(requests)))
def close(self, *, sig: int = signal.SIGTERM) -> str:
self.channel.close()
self.proc.send_signal(sig)
try:
out, _ = self.proc.communicate(timeout=10)
except subprocess.TimeoutExpired:
self.proc.kill()
out, _ = self.proc.communicate()
return out or ""
@pytest.fixture()
def worker():
w = _Worker()
try:
yield w
finally:
if w.proc.poll() is None:
w.close()
def _crc32c(payload: bytes) -> bytes:
return zlib.crc32(payload).to_bytes(4, "big")
_WORKER_FINGERPRINT = dict(
model_artifact_digest="sha256:native-test-artifact",
runtime_recipe_digest="sha256:native-test-recipe",
recipe_id="native-test",
recipe_version="1",
catalogue_version="1",
)
def _open(
*,
route_session_id="rs-1",
route_epoch=7,
credits_granted=16,
max_inflight_chunks=16,
max_chunk_bytes=4 * 1024 * 1024,
schema_version=pb.SCHEMA_VERSION_1,
fingerprint=None,
shard_range=None,
) -> pb.SessionRequest:
fp = pb.Fingerprint(**_WORKER_FINGERPRINT) if fingerprint is None else fingerprint
sr = (
pb.ShardRange(start_layer=0, end_layer=32, effective_start_layer=0)
if shard_range is None
else shard_range
)
return pb.SessionRequest(
open=pb.SessionOpen(
schema_version=schema_version,
route_session_id=route_session_id,
route_epoch=route_epoch,
fingerprint=fp,
shard_range=sr,
proposed_flow_control=pb.FlowControl(
credits_granted=credits_granted,
max_inflight_chunks=max_inflight_chunks,
max_chunk_bytes=max_chunk_bytes,
max_prefill_chunk_tokens=512,
),
accepted_compression=[pb.COMPRESSION_NONE],
)
)
def _chunk(
work_id,
payload: bytes,
step,
*,
route_session_id="rs-1",
route_epoch=7,
deadline_unix_nanos=0,
fragments=1,
total_bytes=None,
) -> pb.SessionRequest:
total = len(payload) if total_bytes is None else total_bytes
frags = []
if fragments == 1:
frags = [pb.TensorFragment(fragment_index=0, fragment_count=1, byte_offset=0, payload=payload)]
else:
# Split into ``fragments`` tiling pieces.
size = max(1, len(payload) // fragments)
offset = 0
idx = 0
while offset < len(payload):
piece = payload[offset : offset + size] if idx < fragments - 1 else payload[offset:]
frags.append(
pb.TensorFragment(
fragment_index=idx, fragment_count=fragments, byte_offset=offset, payload=piece
)
)
offset += len(piece)
idx += 1
tensor = pb.NamedTensor(
name="hidden_states",
shape=[1, 1, 4096],
dtype=pb.DTYPE_BFLOAT16,
byte_order=pb.BYTE_ORDER_LITTLE_ENDIAN,
total_bytes=total,
compression=pb.COMPRESSION_NONE,
checksum=pb.Checksum(algorithm=pb.CHECKSUM_ALGORITHM_CRC32C, value=_crc32c(payload)),
fragments=frags,
)
bundle = pb.TensorBundle(
bundle_version=1,
tensors=[tensor],
architecture=pb.ARCHITECTURE_TYPE_DENSE,
boundary_point="pre_tail_residual",
)
envelope = pb.Envelope(
schema_version=pb.SCHEMA_VERSION_1,
work_id=work_id,
route_session_id=route_session_id,
route_epoch=route_epoch,
idempotency_step=step,
phase=pb.PHASE_PREFILL,
position=pb.PositionSpan(first_position=0, token_count=1),
deadline_unix_nanos=deadline_unix_nanos,
)
return pb.SessionRequest(chunk=pb.ActivationChunk(envelope=envelope, bundle=bundle))
def _decode(work_id, payload: bytes, step, position) -> pb.SessionRequest:
tensor = pb.NamedTensor(
name="hidden_states",
shape=[1, 1, 4096],
dtype=pb.DTYPE_BFLOAT16,
byte_order=pb.BYTE_ORDER_LITTLE_ENDIAN,
total_bytes=len(payload),
compression=pb.COMPRESSION_NONE,
checksum=pb.Checksum(algorithm=pb.CHECKSUM_ALGORITHM_CRC32C, value=_crc32c(payload)),
fragments=[pb.TensorFragment(fragment_index=0, fragment_count=1, byte_offset=0, payload=payload)],
)
return pb.SessionRequest(
decode=pb.DecodeStep(
idempotency_step=step,
position=position,
expected_past_len=position,
work_id=work_id,
bundle=pb.TensorBundle(bundle_version=1, tensors=[tensor], architecture=pb.ARCHITECTURE_TYPE_DENSE),
)
)
def _release() -> pb.SessionRequest:
return pb.SessionRequest(
release=pb.ReleaseSignal(route_session_id="rs-1", route_epoch=7, work_id="work-final")
)
def _cancel(*, route_session_id="rs-1", work_id="", reason="test cancel") -> pb.SessionRequest:
return pb.SessionRequest(
cancel=pb.CancelSignal(route_session_id=route_session_id, route_epoch=7, work_id=work_id, reason=reason)
)
# --- startup / health / capability ----------------------------------------
def test_worker_startup_and_health(worker):
health = worker.stub().Health(pb.HealthRequest(schema_version=pb.SCHEMA_VERSION_1))
assert health.state == pb.SERVING_STATE_SERVING
assert health.schema_version == pb.SCHEMA_VERSION_1
def test_worker_capability(worker):
cap = worker.stub().GetCapability(pb.CapabilityRequest(schema_version=pb.SCHEMA_VERSION_1))
assert cap.validated is True
assert cap.schema_version == pb.SCHEMA_VERSION_1
assert cap.shard_range.end_layer == 32
assert pb.SCHEMA_VERSION_1 in cap.supported_schema_versions
# --- fragmented prefill / decode / release ---------------------------------
def test_fragmented_prefill_echoes_reassembled_payload(worker):
payload = b"REAL_ACTIVATION_BYTES_prefill_across_three_fragments_1234567890"
responses = worker.session([_open(), _chunk("w1", payload, step=1, fragments=3), _release()])
assert responses[0].WhichOneof("kind") == "accepted"
echoed = responses[1]
assert echoed.WhichOneof("kind") == "chunk"
got = b"".join(f.payload for f in echoed.chunk.bundle.tensors[0].fragments)
assert got == payload
assert echoed.chunk.bundle.tensors[0].checksum.value == _crc32c(payload)
assert responses[2].status.terminal is True
def test_decode_step_is_served(worker):
payload = b"REAL_ACTIVATION_BYTES_decode_step"
responses = worker.session([_open(), _decode("w2", payload, step=1, position=1)])
echoed = responses[1]
assert echoed.WhichOneof("kind") == "chunk"
assert echoed.chunk.envelope.phase == pb.PHASE_DECODE
assert echoed.chunk.bundle.tensors[0].fragments[0].payload == payload
def test_two_disjoint_fake_worker_processes_preserve_prefill_and_decode_seam():
"""DGR-036 fixture proof across two actual fake-worker processes.
The DGR-033 worker deliberately has no model graph: its bounded forward
validates and echoes the activation bytes. That makes it suitable for a
deterministic protocol proof only. Keep the two ranges disjoint at the
``SessionOpen`` boundary, pass each stage-one output through stage two,
and exercise a prefill plus 32 sequential decode positions. Numerical
dense-GGUF parity remains an opt-in DGR-036 real-model lane, not a claim
made by this fixture.
"""
head = _Worker()
tail = _Worker()
try:
head_open = _open(
route_session_id="dgr036-head",
shard_range=pb.ShardRange(start_layer=0, end_layer=16, effective_start_layer=0),
)
tail_open = _open(
route_session_id="dgr036-tail",
shard_range=pb.ShardRange(start_layer=16, end_layer=32, effective_start_layer=16),
)
payload = b"dgr036 deterministic dense prefill residual"
def decode_requests(*, stage: str, payloads: list[bytes]) -> list[pb.SessionRequest]:
requests: list[pb.SessionRequest] = []
for position, stage_payload in enumerate(payloads, start=1):
# The fixture worker's negotiated default window is 16. Top
# it up before decode 16 and 32 so this exercises all 32
# sequential positions rather than silently testing only one
# credit window.
if position in {16, 32}:
requests.append(pb.SessionRequest(flow_control=pb.FlowControl(credits_granted=16)))
requests.append(_decode(f"decode-{stage}-{position}", stage_payload, position + 1, position))
return requests
head_responses = head.session(
[head_open, _chunk("prefill-head", payload, step=1)]
+ decode_requests(stage="head", payloads=[payload] * 32)
)
assert head_responses[0].WhichOneof("kind") == "accepted"
assert head_responses[1].WhichOneof("kind") == "chunk"
seam_payloads = [
response.chunk.bundle.tensors[0].fragments[0].payload
for response in head_responses
if response.WhichOneof("kind") == "chunk"
]
assert len(seam_payloads) == 33
tail_responses = tail.session(
[tail_open, _chunk("prefill-tail", seam_payloads[0], step=1, route_session_id="dgr036-tail")]
+ decode_requests(stage="tail", payloads=seam_payloads[1:])
)
assert tail_responses[0].WhichOneof("kind") == "accepted"
assert tail_responses[1].WhichOneof("kind") == "chunk"
assert tail_responses[1].chunk.bundle.tensors[0].fragments[0].payload == payload
for tail_response in (response for response in tail_responses if response.WhichOneof("kind") == "chunk"):
assert tail_response.chunk.bundle.tensors[0].fragments[0].payload == payload
finally:
if head.proc.poll() is None:
head.close()
if tail.proc.poll() is None:
tail.close()
def test_release_is_terminal(worker):
responses = worker.session([_open(), _release()])
assert responses[0].WhichOneof("kind") == "accepted"
assert responses[1].status.terminal is True
# --- deadlines / flow control / bounded messages ---------------------------
def test_expired_deadline_is_rejected(worker):
responses = worker.session([_open(), _chunk("w-late", b"payload", step=1, deadline_unix_nanos=1)])
assert responses[1].status.error.code == pb.ERROR_CODE_DEADLINE_EXCEEDED
def test_flow_control_violation_and_topup(worker):
responses = worker.session(
[
_open(credits_granted=1),
_chunk("w-a", b"payload-a", step=1),
_chunk("w-b", b"payload-b", step=2),
pb.SessionRequest(flow_control=pb.FlowControl(credits_granted=5)),
_chunk("w-c", b"payload-c", step=3),
]
)
assert responses[1].WhichOneof("kind") == "chunk"
assert responses[2].status.error.code == pb.ERROR_CODE_FLOW_CONTROL_VIOLATION
assert responses[2].status.error.retryable is True
assert responses[3].WhichOneof("kind") == "flow_control"
assert responses[3].flow_control.credits_granted >= 5
assert responses[4].WhichOneof("kind") == "chunk"
def test_bounded_message_is_rejected():
"""A tensor whose declared payload exceeds the negotiated ceiling is refused."""
w = _Worker(extra_env={"MESHNET_MAX_CHUNK_BYTES": "64"})
try:
big = b"x" * 128
responses = w.session([_open(), _chunk("w-big", big, step=1, total_bytes=128)])
status = responses[1].status
assert status.error.code == pb.ERROR_CODE_RESOURCE_EXHAUSTED
assert "max_chunk_bytes" in status.error.detail
finally:
if w.proc.poll() is None:
w.close()
def test_malformed_fragment_tiling_is_rejected(worker):
# A fragment at a non-zero offset with no predecessor cannot tile.
tensor = pb.NamedTensor(
name="hidden_states",
shape=[1, 1, 4096],
dtype=pb.DTYPE_BFLOAT16,
byte_order=pb.BYTE_ORDER_LITTLE_ENDIAN,
total_bytes=7,
compression=pb.COMPRESSION_NONE,
checksum=pb.Checksum(algorithm=pb.CHECKSUM_ALGORITHM_CRC32C, value=_crc32c(b"payload")),
fragments=[pb.TensorFragment(fragment_index=0, fragment_count=1, byte_offset=5, payload=b"payload")],
)
bad = pb.SessionRequest(
chunk=pb.ActivationChunk(
envelope=pb.Envelope(
schema_version=pb.SCHEMA_VERSION_1,
work_id="w-gap",
route_session_id="rs-1",
route_epoch=7,
idempotency_step=1,
),
bundle=pb.TensorBundle(bundle_version=1, tensors=[tensor]),
)
)
responses = worker.session([_open(), bad])
assert responses[1].status.error.code == pb.ERROR_CODE_PAYLOAD_CORRUPT
assert "tile" in responses[1].status.error.detail
def test_stale_route_epoch_is_rejected(worker):
responses = worker.session([_open(route_epoch=7), _chunk("w-stale", b"payload", step=1, route_epoch=5)])
assert responses[1].status.error.code == pb.ERROR_CODE_EPOCH_STALE
def test_duplicate_idempotency_step_is_acked(worker):
chunk = _chunk("w-dup", b"payload", step=1)
responses = worker.session([_open(), chunk, chunk])
assert responses[1].WhichOneof("kind") == "chunk"
assert responses[2].WhichOneof("kind") == "ack"
assert responses[2].ack.duplicate is True
# --- cancellation ----------------------------------------------------------
def test_in_band_cancel_of_single_work_item_does_not_end_stream(worker):
responses = worker.session(
[
_open(),
_cancel(work_id="work-x"),
_chunk("work-x", b"payload", step=1),
_chunk("work-y", b"payload", step=2),
_release(),
]
)
assert responses[1].status.error.code == pb.ERROR_CODE_CANCELLED
assert responses[1].status.terminal is False
assert responses[2].status.error.code == pb.ERROR_CODE_CANCELLED
assert responses[3].WhichOneof("kind") == "chunk"
assert responses[4].status.terminal is True
def test_in_band_cancel_of_whole_session_is_terminal(worker):
responses = worker.session([_open(), _cancel(work_id="")])
assert responses[1].status.error.code == pb.ERROR_CODE_CANCELLED
assert responses[1].status.terminal is True
def test_out_of_band_cancel_rpc_races_ahead_of_open(worker):
stub = worker.stub()
resp = stub.Cancel(
pb.CancelRequest(
schema_version=pb.SCHEMA_VERSION_1,
route_session_id="rs-precancel",
route_epoch=1,
work_id="work-precancelled",
reason="operator abort",
)
)
assert resp.cancelled_work_items == 1
responses = worker.session(
[
_open(route_session_id="rs-precancel"),
_chunk("work-precancelled", b"payload", step=1, route_session_id="rs-precancel"),
]
)
assert responses[1].status.error.code == pb.ERROR_CODE_CANCELLED
def test_release_rpc_is_idempotent(worker):
stub = worker.stub()
# Open a session WITHOUT an in-stream release so state persists on the
# servicer, then drop it out of band twice.
worker.session([_open(route_session_id="rs-rel")])
first = stub.Release(pb.ReleaseRequest(schema_version=pb.SCHEMA_VERSION_1, route_session_id="rs-rel", route_epoch=7))
second = stub.Release(pb.ReleaseRequest(schema_version=pb.SCHEMA_VERSION_1, route_session_id="rs-rel", route_epoch=7))
assert first.released is True
assert second.released is False # idempotent: nothing left to drop
def test_independent_session_cancellation(worker):
# Cancel the whole of session A; session B must remain fully serviceable.
a = worker.session([_open(route_session_id="sess-A"), _cancel(route_session_id="sess-A", work_id="")])
assert a[1].status.terminal is True
b = worker.session(
[
_open(route_session_id="sess-B"),
_chunk("work-b", b"payload-b", step=1, route_session_id="sess-B"),
_release(),
]
)
assert b[1].WhichOneof("kind") == "chunk", "cancelling session A must not affect session B"
# --- graceful shutdown -----------------------------------------------------
def test_graceful_shutdown_on_sigterm():
w = _Worker()
# Confirm it is serving, then send SIGTERM and require a clean drain/exit.
assert w.stub().Health(pb.HealthRequest(schema_version=pb.SCHEMA_VERSION_1)).state == pb.SERVING_STATE_SERVING
out = w.close(sig=signal.SIGTERM)
assert w.proc.returncode == 0, f"worker did not exit cleanly on SIGTERM:\n{out}"
assert "shut down cleanly" in out
# --- direct vs opaque relay byte identity ----------------------------------
def test_direct_and_opaque_relay_yield_identical_responses(worker):
"""A direct hop and an opaque relay of the exact captured request bytes must
produce byte-identical server responses (relays carry frames verbatim)."""
payload = b"RELAY_ACTIVATION_BYTES"
requests = [_open(), _chunk("w1", payload, step=1), _release()]
direct_call = worker.channel.stream_stream(
"/meshnet.shard.v1.ShardRuntime/Session",
request_serializer=lambda m: m.SerializeToString(),
response_deserializer=lambda b: b,
)
direct_resp = list(direct_call(iter(requests)))
captured = [m.SerializeToString() for m in requests]
relay_call = worker.channel.stream_stream(
"/meshnet.shard.v1.ShardRuntime/Session",
request_serializer=lambda b: b, # raw captured bytes, no reinterpretation
response_deserializer=lambda b: b,
)
relay_resp = list(relay_call(iter(captured)))
assert len(direct_resp) == len(relay_resp) == 3
for i, (d, r) in enumerate(zip(direct_resp, relay_resp)):
assert d == r, f"response #{i} differs between direct and opaque relay"
# --- fail-closed before SessionOpen ----------------------------------------
def test_chunk_before_open_is_rejected(worker):
# An activation with no preceding SessionOpen must fail closed and end the
# stream: no work may bypass the lifecycle handshake.
responses = worker.session([_chunk("w-noopen", b"payload", step=1)])
assert len(responses) == 1
assert responses[0].WhichOneof("kind") == "status"
assert responses[0].status.error.code == pb.ERROR_CODE_INTERNAL
assert responses[0].status.terminal is True
assert "SessionOpen" in responses[0].status.error.detail
def test_decode_before_open_is_rejected(worker):
responses = worker.session([_decode("w-noopen", b"payload", step=1, position=0)])
assert len(responses) == 1
assert responses[0].WhichOneof("kind") == "status"
assert responses[0].status.error.code == pb.ERROR_CODE_INTERNAL
assert responses[0].status.terminal is True
# --- flow-control negotiation with strict worker bounds --------------------
def test_flow_control_proposal_is_clamped_to_worker_bounds(worker):
# A peer proposing a window far above the worker limits must be clamped to
# the worker own ceilings, never granted the inflated proposal.
responses = worker.session(
[_open(credits_granted=9999, max_inflight_chunks=9999, max_chunk_bytes=1073741824)]
)
fc = responses[0].accepted.flow_control
assert fc.max_inflight_chunks == 16
assert fc.credits_granted == 16
assert fc.max_chunk_bytes == 4 * 1024 * 1024
def test_negotiated_max_chunk_bytes_caps_peer_proposal():
# Worker ceiling is 64 bytes; the peer proposes 4 MiB. The negotiated per
# session ceiling is the stricter 64, so a 128-byte tensor is refused even
# though the peer allowed it — the worker never adopts the peer proposal.
w = _Worker(extra_env={"MESHNET_MAX_CHUNK_BYTES": "64"})
try:
big = b"x" * 128
responses = w.session(
[_open(max_chunk_bytes=4 * 1024 * 1024), _chunk("w-big", big, step=1, total_bytes=128)]
)
assert responses[0].accepted.flow_control.max_chunk_bytes == 64
assert responses[1].status.error.code == pb.ERROR_CODE_RESOURCE_EXHAUSTED
assert "max_chunk_bytes" in responses[1].status.error.detail
finally:
if w.proc.poll() is None:
w.close()
# --- in-stream release erases session state --------------------------------
def test_in_stream_release_erases_session_state(worker):
stub = worker.stub()
resp = worker.session(
[
_open(route_session_id="rs-erase"),
pb.SessionRequest(
release=pb.ReleaseSignal(route_session_id="rs-erase", route_epoch=7, work_id="w-final")
),
]
)
assert resp[-1].status.terminal is True
# The state is already gone: an out-of-band Release finds nothing to drop.
after = stub.Release(
pb.ReleaseRequest(schema_version=pb.SCHEMA_VERSION_1, route_session_id="rs-erase", route_epoch=7)
)
assert after.released is False
# --- SessionOpen identity validation ---------------------------------------
def test_incompatible_schema_is_rejected_at_open(worker):
responses = worker.session([_open(schema_version=pb.SCHEMA_VERSION_UNSPECIFIED)])
assert responses[0].WhichOneof("kind") == "status"
assert responses[0].status.error.code == pb.ERROR_CODE_SCHEMA_UNSUPPORTED
assert responses[0].status.terminal is True
def test_incompatible_fingerprint_is_rejected_at_open(worker):
bad_fp = pb.Fingerprint(
model_artifact_digest="sha256:some-other-model",
runtime_recipe_digest="sha256:native-test-recipe",
recipe_id="native-test",
recipe_version="1",
catalogue_version="1",
)
responses = worker.session([_open(fingerprint=bad_fp)])
assert responses[0].WhichOneof("kind") == "status"
assert responses[0].status.error.code == pb.ERROR_CODE_FINGERPRINT_MISMATCH
assert responses[0].status.terminal is True
def test_shard_range_mismatch_is_rejected_at_open(worker):
responses = worker.session(
[_open(shard_range=pb.ShardRange(start_layer=0, end_layer=64, effective_start_layer=0))]
)
assert responses[0].WhichOneof("kind") == "status"
assert responses[0].status.error.code == pb.ERROR_CODE_SHARD_RANGE_MISMATCH
assert responses[0].status.terminal is True
def test_session_accepted_reports_worker_fingerprint_not_caller(worker):
# The caller asserts no fingerprint; SessionAccepted must carry the worker
# OWN served identity, not a copy of the caller (empty) fingerprint.
responses = worker.session([_open(fingerprint=pb.Fingerprint())])
assert responses[0].WhichOneof("kind") == "accepted"
accepted = responses[0].accepted
assert accepted.fingerprint.model_artifact_digest == "sha256:native-test-artifact"
assert accepted.fingerprint.runtime_recipe_digest == "sha256:native-test-recipe"

View File

@@ -0,0 +1,196 @@
"""Model-free DGR-040 supervision tests using a deterministic fake worker."""
from __future__ import annotations
import hashlib
import sys
import time
from pathlib import Path
import pytest
from meshnet_node.native_worker_supervisor import (
NativeWorkerError,
NativeWorkerProbe,
NativeWorkerSpec,
NativeWorkerSupervisor,
)
def _write_fake_worker(path: Path) -> None:
path.write_text(
"""import os
import signal
import sys
import time
stop = False
def terminate(*_):
global stop
stop = True
signal.signal(signal.SIGTERM, terminate)
print('ShardRuntime worker listening on ' + os.environ['MESHNET_SHARD_LISTEN_ADDR'], flush=True)
if os.environ.get('MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS'):
marker = os.environ.get('MESHNET_FAKE_CRASH_ONCE_FILE')
if not marker or not os.path.exists(marker):
if marker:
open(marker, 'w').close()
time.sleep(0.05)
print('deterministic injected worker death', file=sys.stderr, flush=True)
raise SystemExit(70)
while not stop:
time.sleep(0.01)
print('ShardRuntime worker shut down cleanly', flush=True)
""",
encoding="utf-8",
)
def _spec(tmp_path: Path, **changes: object) -> NativeWorkerSpec:
artifact = tmp_path / "fixture.gguf"
artifact.write_bytes(b"fixture artifact")
fake = tmp_path / "fake_worker.py"
_write_fake_worker(fake)
values: dict[str, object] = {
"binary": Path(sys.executable),
"binary_digest": hashlib.sha256(Path(sys.executable).read_bytes()).hexdigest(),
"args": (str(fake),),
"listen_address": "fake-worker:12345",
"artifact_path": artifact,
"artifact_digest": hashlib.sha256(artifact.read_bytes()).hexdigest(),
"recipe_digest": "a" * 64,
"recipe_id": "fixture",
"recipe_version": "1",
"catalogue_version": "test",
"shard_start": 2,
"shard_end": 5,
}
values.update(changes)
return NativeWorkerSpec(**values) # type: ignore[arg-type]
def _probe(spec: NativeWorkerSpec, _timeout: float) -> NativeWorkerProbe:
return NativeWorkerProbe(
artifact_digest=spec.artifact_digest,
recipe_digest=spec.recipe_digest,
recipe_id=spec.recipe_id,
recipe_version=spec.recipe_version,
catalogue_version=spec.catalogue_version,
shard_start=spec.shard_start,
shard_end=spec.shard_end,
serving=True,
)
def _eventually(predicate, timeout: float = 2.0) -> bool:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if predicate():
return True
time.sleep(0.01)
return predicate()
def test_start_verifies_identity_captures_logs_and_stops_gracefully(tmp_path):
events: list[tuple[str, str]] = []
supervisor = NativeWorkerSupervisor(
_spec(tmp_path),
probe=_probe,
readiness_timeout=1,
shutdown_timeout=1,
kill_timeout=1,
on_available=lambda reason: events.append(("available", reason)),
on_unavailable=lambda reason: events.append(("unavailable", reason)),
)
result = supervisor.start()
assert result.serving and supervisor.available
assert events == [("available", "worker ready and identity verified")]
assert any("ShardRuntime worker listening" in line for line in supervisor.logs)
supervisor.stop()
assert not supervisor.available
assert events[-1] == ("unavailable", "worker stopped")
assert _eventually(lambda: any("shut down cleanly" in line for line in supervisor.logs))
def test_start_refuses_changed_artifact_before_spawning(tmp_path):
spec = _spec(tmp_path, artifact_digest="0" * 64)
supervisor = NativeWorkerSupervisor(spec, probe=_probe, readiness_timeout=1)
with pytest.raises(NativeWorkerError, match="artifact digest"):
supervisor.start()
assert supervisor.pid is None
def test_start_refuses_changed_binary_before_spawning(tmp_path):
spec = _spec(tmp_path, binary_digest="0" * 64)
supervisor = NativeWorkerSupervisor(spec, probe=_probe, readiness_timeout=1)
with pytest.raises(NativeWorkerError, match="binary digest"):
supervisor.start()
assert supervisor.pid is None
def test_probe_identity_mismatch_never_makes_capability_available(tmp_path):
spec = _spec(tmp_path)
def wrong_probe(actual: NativeWorkerSpec, timeout: float) -> NativeWorkerProbe:
result = _probe(actual, timeout)
return NativeWorkerProbe(**{**result.__dict__, "shard_end": actual.shard_end + 1})
supervisor = NativeWorkerSupervisor(spec, probe=wrong_probe, readiness_timeout=1, shutdown_timeout=1)
with pytest.raises(NativeWorkerError, match="identity/range"):
supervisor.start()
assert not supervisor.available
def test_deterministic_worker_death_withdraws_then_restart_recovers(tmp_path):
unavailable: list[str] = []
spec = _spec(
tmp_path,
extra_environment={
"MESHNET_INJECT_PROCESS_DEATH_AFTER_EXECUTIONS": "1",
"MESHNET_FAKE_CRASH_ONCE_FILE": str(tmp_path / "crashed-once"),
},
)
supervisor = NativeWorkerSupervisor(
spec,
probe=_probe,
readiness_timeout=1,
health_interval=0.01,
shutdown_timeout=1,
kill_timeout=1,
on_unavailable=unavailable.append,
)
supervisor.start()
assert _eventually(lambda: not supervisor.available)
assert "code 70" in supervisor.unavailable_reason
assert any("deterministic injected worker death" in line for line in supervisor.logs)
supervisor.restart()
assert supervisor.available
supervisor.stop()
def test_health_loss_withdraws_only_native_capability(tmp_path):
healthy = True
unavailable: list[str] = []
spec = _spec(tmp_path)
def health_probe(actual: NativeWorkerSpec, timeout: float) -> NativeWorkerProbe:
result = _probe(actual, timeout)
return NativeWorkerProbe(**{**result.__dict__, "serving": healthy})
supervisor = NativeWorkerSupervisor(
spec,
probe=health_probe,
readiness_timeout=1,
on_unavailable=unavailable.append,
)
supervisor.start()
healthy = False
assert not supervisor.check_health()
assert not supervisor.available
assert unavailable and "health lost" in unavailable[-1]
supervisor.stop()

273
tests/test_range_report.py Normal file
View File

@@ -0,0 +1,273 @@
"""DGR-034: strict consumption of owned-range reports from loaded engine state.
The ``meshnet-range-report`` native tool loads one dense-Llama GGUF through
the Meshnet owned-range loader and prints a JSON document derived from the
loaded model state. ``meshnet_node.range_report`` is the strict consumer:
it must accept exactly the documents that encode the dense-Llama ownership
contract and fail closed on everything else — invalid, empty, or
out-of-model ranges, endpoint registrations that disagree with the loaded
state, gapped or unexpected tensor registrations, and inconsistent byte
counts.
"""
from __future__ import annotations
from typing import Any
import pytest
from meshnet_node.range_report import (
OwnedRangeReport,
RangeReportError,
parse_owned_range_report,
)
N_LAYER = 40
LAYER_BYTES = 300 * 2**20
EMBD_BYTES = 360 * 2**20
OUT_BYTES = 525 * 2**20
FILE_BYTES = 13669 * 2**20
def _doc(**overrides: Any) -> dict[str, Any]:
"""A valid middle-range [10, 20) mmap report the consumer must accept."""
doc: dict[str, Any] = {
"ok": True,
"model": "/models/dense.gguf",
"architecture": "llama",
"n_layer": N_LAYER,
"file_bytes": FILE_BYTES,
"requested_range": [10, 20],
"reported_range": [10, 20],
"mmap": True,
"touched": False,
"use_extra_bufts": True,
"has_token_embeddings": False,
"has_output_head": False,
"tied_output_head": False,
"mapped_bytes": 10 * LAYER_BYTES,
"resident_bytes": 10 * LAYER_BYTES,
"registered_tensors": 90,
"registered_bytes": 10 * LAYER_BYTES,
"unexpected_registered_tensors": [],
"missing_owned_layers": [],
"vm_size_bytes": FILE_BYTES + 2**28,
"vm_rss_bytes": 2**28,
"vm_hwm_bytes": 2**28,
}
doc.update(overrides)
return doc
def _head_doc(**overrides: Any) -> dict[str, Any]:
base = _doc(
requested_range=[0, 10],
reported_range=[0, 10],
has_token_embeddings=True,
mapped_bytes=10 * LAYER_BYTES + EMBD_BYTES,
resident_bytes=10 * LAYER_BYTES + EMBD_BYTES,
registered_tensors=91,
registered_bytes=10 * LAYER_BYTES + EMBD_BYTES,
)
base.update(overrides)
return base
def _tail_doc(**overrides: Any) -> dict[str, Any]:
base = _doc(
requested_range=[30, 40],
reported_range=[30, 40],
has_output_head=True,
mapped_bytes=10 * LAYER_BYTES + OUT_BYTES,
resident_bytes=10 * LAYER_BYTES + OUT_BYTES,
registered_tensors=92,
registered_bytes=10 * LAYER_BYTES + OUT_BYTES,
)
base.update(overrides)
return base
class TestAcceptance:
def test_middle_range_registers_only_per_layer_tensors(self) -> None:
report = parse_owned_range_report(_doc())
assert (report.start_layer, report.end_layer) == (10, 20)
assert not report.is_head and not report.is_tail
assert not report.has_token_embeddings and not report.has_output_head
def test_head_range_owns_embeddings_only_at_the_head(self) -> None:
report = parse_owned_range_report(_head_doc())
assert report.is_head and not report.is_tail
assert report.has_token_embeddings and not report.has_output_head
def test_tail_range_owns_norm_and_output_only_at_the_tail(self) -> None:
report = parse_owned_range_report(_tail_doc())
assert report.is_tail and not report.is_head
assert report.has_output_head and not report.has_token_embeddings
def test_whole_model_range_owns_both_endpoints(self) -> None:
report = parse_owned_range_report(
_head_doc(
requested_range=[0, 40],
reported_range=[0, 40],
has_output_head=True,
mapped_bytes=FILE_BYTES,
resident_bytes=FILE_BYTES,
registered_tensors=363,
registered_bytes=N_LAYER * LAYER_BYTES + EMBD_BYTES + OUT_BYTES,
)
)
assert report.is_head and report.is_tail
assert report.has_token_embeddings and report.has_output_head
def test_tied_output_tail_registers_the_embedding_as_its_output_head(self) -> None:
report = parse_owned_range_report(
_tail_doc(
has_token_embeddings=True,
tied_output_head=True,
registered_tensors=91,
registered_bytes=10 * LAYER_BYTES + EMBD_BYTES,
mapped_bytes=10 * LAYER_BYTES + EMBD_BYTES,
resident_bytes=10 * LAYER_BYTES + EMBD_BYTES,
)
)
assert report.tied_output_head and report.has_output_head
def test_non_mmap_load_reports_resident_allocation_only(self) -> None:
report = parse_owned_range_report(
_doc(mmap=False, mapped_bytes=0, resident_bytes=10 * LAYER_BYTES)
)
assert report.mapped_bytes == 0
assert report.resident_bytes == 10 * LAYER_BYTES
def test_process_counters_may_be_absent_off_linux(self) -> None:
report = parse_owned_range_report(
_doc(vm_size_bytes=None, vm_rss_bytes=None, vm_hwm_bytes=None)
)
assert report.vm_hwm_bytes is None
class TestRangeRejection:
def test_rejected_load_fails_closed_with_the_tool_error(self) -> None:
with pytest.raises(RangeReportError, match="dense Llama only"):
parse_owned_range_report(
{"ok": False, "error": "owned-range load rejected the artifact or range: dense Llama only"}
)
def test_reported_range_must_match_the_requested_range(self) -> None:
with pytest.raises(RangeReportError, match="loaded engine state"):
parse_owned_range_report(_doc(reported_range=[10, 21]))
def test_out_of_model_range_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="outside the model"):
parse_owned_range_report(
_doc(requested_range=[30, 41], reported_range=[30, 41], has_output_head=True)
)
def test_empty_range_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="empty or"):
parse_owned_range_report(_doc(requested_range=[10, 10], reported_range=[10, 10]))
def test_inverted_range_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="empty or"):
parse_owned_range_report(_doc(requested_range=[20, 10], reported_range=[20, 10]))
def test_boolean_range_bounds_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="integer pair"):
parse_owned_range_report(_doc(reported_range=[True, 20]))
class TestEndpointRejection:
def test_embeddings_registered_below_the_head_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="embeddings belong to the head"):
parse_owned_range_report(_doc(has_token_embeddings=True))
def test_output_head_registered_above_the_tail_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="output head belong to the tail"):
parse_owned_range_report(_tail_doc(requested_range=[20, 30], reported_range=[20, 30]))
def test_tail_without_an_output_head_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="output head belong to the tail"):
parse_owned_range_report(_tail_doc(has_output_head=False))
def test_tied_output_below_the_tail_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="only belong to the tail"):
parse_owned_range_report(_doc(tied_output_head=True))
def test_unexpected_registered_tensors_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="unexpected_registered_tensors"):
parse_owned_range_report(
_doc(unexpected_registered_tensors=["blk.10.attn_q.weight.extra"])
)
def test_missing_owned_layers_are_rejected_as_gaps(self) -> None:
with pytest.raises(RangeReportError, match="missing_owned_layers"):
parse_owned_range_report(_doc(missing_owned_layers=[12]))
class TestByteCountRejection:
def test_mapped_span_must_cover_the_registered_tensors(self) -> None:
with pytest.raises(RangeReportError, match="undercounts"):
parse_owned_range_report(_doc(mapped_bytes=LAYER_BYTES))
def test_mapped_span_must_not_exceed_the_artifact(self) -> None:
with pytest.raises(RangeReportError, match="exceeds the artifact"):
parse_owned_range_report(
_tail_doc(mapped_bytes=FILE_BYTES + 1, resident_bytes=FILE_BYTES + 1)
)
def test_non_mmap_load_must_not_claim_a_mapped_span(self) -> None:
with pytest.raises(RangeReportError, match="must not claim"):
parse_owned_range_report(_doc(mmap=False, mapped_bytes=LAYER_BYTES))
def test_resident_allocation_must_cover_the_registered_tensors(self) -> None:
with pytest.raises(RangeReportError, match="undercounts"):
parse_owned_range_report(
_doc(mmap=False, mapped_bytes=0, resident_bytes=LAYER_BYTES)
)
def test_an_empty_registration_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="no tensors"):
parse_owned_range_report(_doc(registered_tensors=0, registered_bytes=0))
class TestSchemaRejection:
def test_wrong_architecture_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="dense Llama only"):
parse_owned_range_report(_doc(architecture="qwen2"))
def test_missing_field_is_rejected(self) -> None:
doc = _doc()
del doc["mapped_bytes"]
with pytest.raises(RangeReportError, match="missing field"):
parse_owned_range_report(doc)
def test_boolean_bytes_are_rejected(self) -> None:
with pytest.raises(RangeReportError, match="non-negative integer"):
parse_owned_range_report(_doc(mapped_bytes=True))
def test_non_mapping_document_is_rejected(self) -> None:
with pytest.raises(RangeReportError, match="JSON object"):
parse_owned_range_report(["not", "a", "report"]) # type: ignore[arg-type]
def test_owned_range_report_rejects_direct_construction_outside_the_contract() -> None:
with pytest.raises(RangeReportError, match="dense Llama only"):
OwnedRangeReport(
architecture="qwen2",
n_layer=N_LAYER,
start_layer=10,
end_layer=20,
has_token_embeddings=False,
has_output_head=False,
tied_output_head=False,
mapped_bytes=10 * LAYER_BYTES,
resident_bytes=10 * LAYER_BYTES,
registered_tensors=90,
registered_bytes=10 * LAYER_BYTES,
file_bytes=FILE_BYTES,
mmap=True,
touched=False,
vm_size_bytes=None,
vm_rss_bytes=None,
vm_hwm_bytes=None,
)