fix: DGR-033 repair native worker protocol per cross-review BLOCK
Address the Codex GPT-5.5 review of the standalone fake C++ gRPC Shard worker. Four root protocol defects fixed: - Fail closed before SessionOpen: a per-session `opened` flag gates chunk/decode so no activation bypasses lifecycle, cancellation, epoch or flow-control state (terminal ERROR_CODE_INTERNAL), even when an out-of-band Cancel created placeholder state. - Strict flow-control negotiation: NegotiateFlow takes the strictest of peer-vs-worker bounds (mirrors codec.negotiate_flow_control) and the negotiated per-session max_chunk_bytes is enforced on every bundle instead of trusting the peer proposal. - In-stream ReleaseSignal now erases session state immediately. - SessionOpen rejects incompatible schema, fingerprint, and shard-range identity and reports the worker's own served fingerprint rather than echoing the caller. Adds 9 regression tests (worker suite 18 -> 27). Real gates on the rebuilt pinned-gRPC binary: cmake build exit 0; ctest 2/2; worker pytest 27 passed; harness+protocol 63 passed; compileall 0; diff --check clean; ldd/nm show 0 llama/ggml linkage. DGR-033 passes -> true. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dobromir Popov
@@ -1,6 +1,7 @@
|
||||
# DGR-033 evidence — standalone fake C++ gRPC Shard worker
|
||||
|
||||
**Completed:** 2026-07-25
|
||||
**Completed:** 2026-07-25 (initial); **repaired:** 2026-07-26 after Codex
|
||||
GPT-5.5 cross-review BLOCK (see "Cross-review repair" below).
|
||||
**Branch:** `ralph/distributed-gguf-opus`
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
**Dependencies:** DGR-022 (lifecycle/status contract), DGR-024 (real generated
|
||||
@@ -206,3 +207,75 @@ DGR-029/030); the Python client uses that venv's `grpcio==1.82.1`,
|
||||
supervision primitives — a readiness line for start detection, `SIGTERM`
|
||||
graceful drain with a clean-exit line, and a `--selftest` liveness probe.
|
||||
A supervisor can start/monitor/restart the process around these.
|
||||
|
||||
## Cross-review repair (2026-07-26)
|
||||
|
||||
An independent Codex GPT-5.5 review BLOCKED the initial implementation. Four
|
||||
root protocol defects in the native worker were fixed in this worktree
|
||||
(`.claude/worktrees/distributed-gguf-opus`); the fake-engine echo semantics and
|
||||
supervision shape are unchanged.
|
||||
|
||||
### Defects fixed
|
||||
|
||||
1. **Activation before SessionOpen bypassed all state.** A chunk/decode whose
|
||||
`route_session_id` had no opened session fell through every `if (state && ...)`
|
||||
guard and was echoed — bypassing lifecycle, cancellation, epoch and
|
||||
flow-control. `SessionState` now carries an `opened` flag set only by a valid
|
||||
`SessionOpen`; chunk and decode fail closed with a terminal
|
||||
`ERROR_CODE_INTERNAL` and end the stream when it is false. A placeholder state
|
||||
created by an out-of-band `Cancel` that races `Open` has `opened == false`, so
|
||||
it can never admit work either.
|
||||
2. **Flow control blindly trusted the peer proposal.** `SessionOpen` copied the
|
||||
proposed `credits/max_inflight/max_chunk_bytes` verbatim into session state and
|
||||
the accepted reply. New `ShardRuntimeServiceImpl::NegotiateFlow` takes the
|
||||
strictest bound of peer-vs-worker for every field (mirroring
|
||||
`negotiate_flow_control` in `native_protocol/codec.py`), stores the negotiated
|
||||
ceilings on the session, and enforces the negotiated per-session
|
||||
`max_chunk_bytes` on every bundle (`FakeShardEngine::Validate` now takes the
|
||||
ceiling as an argument instead of a fixed construction-time value).
|
||||
3. **In-stream `ReleaseSignal` leaked session state.** The stream `release` arm
|
||||
wrote a terminal status but never dropped the session. It now erases the
|
||||
session under the lock before responding, so KV/credits/dedup are freed
|
||||
immediately (the out-of-band `Release` RPC already erased).
|
||||
4. **`SessionOpen` echoed caller identity instead of validating it.** The handshake
|
||||
now rejects an incompatible `schema_version` (`SCHEMA_UNSUPPORTED`), a
|
||||
mismatched model/recipe `Fingerprint` (`FINGERPRINT_MISMATCH`), and a
|
||||
`ShardRange` outside the worker's served range (`SHARD_RANGE_MISMATCH`), each
|
||||
terminal; `SessionAccepted` now reports the worker's own served fingerprint
|
||||
rather than a copy of the caller's.
|
||||
|
||||
### Changed files (repair)
|
||||
|
||||
- `packages/node/native/worker/shard_service.h` — `opened` +
|
||||
`max_prefill_chunk_tokens` on `SessionState`; `NegotiateFlow` decl; engine now
|
||||
default-constructed.
|
||||
- `packages/node/native/worker/shard_service.cpp` — worker-identity constants +
|
||||
fill helpers; `NegotiateFlow`; `SessionOpen` validation/negotiation; fail-closed
|
||||
chunk/decode; per-session `max_chunk_bytes`; in-stream release erase.
|
||||
- `packages/node/native/worker/fake_engine.h` — `Validate(bundle, max_chunk_bytes)`.
|
||||
- `tests/test_native_shard_worker.py` — extended `_open` (schema/fingerprint/range/
|
||||
flow overrides); fixed `test_release_rpc_is_idempotent` for the new erase
|
||||
semantics; added 9 regression tests (chunk/decode before open, flow-control
|
||||
clamp, negotiated-ceiling cap, in-stream release erase, schema/fingerprint/range
|
||||
rejection, worker-fingerprint-not-caller).
|
||||
|
||||
### Re-run gates (real, rebuilt binary)
|
||||
|
||||
Build driven through the pinned `cmake` (Unix Makefiles + `gmake`, gRPC 1.82.1):
|
||||
|
||||
```text
|
||||
cmake --build build/native --parallel 8 -> BUILD_EXIT 0
|
||||
ctest --test-dir build/native --output-on-failure -> 100% (2/2) passed
|
||||
shard_worker_selftest ....... Passed
|
||||
shard_protocol_conformance .. Passed
|
||||
python -m pytest -q tests/test_native_shard_worker.py -> 27 passed
|
||||
python -m pytest -q tests/test_shard_runtime_harness.py \
|
||||
tests/test_native_shard_protocol.py -> 63 passed
|
||||
python -m compileall -q packages tests -> exit 0
|
||||
git diff --check -> clean
|
||||
ldd build/native/shard_worker | grep -iE 'llama|ggml' -> NONE
|
||||
nm -C build/native/shard_worker | grep -cE 'llama_|ggml_' -> 0
|
||||
```
|
||||
|
||||
The worker integration suite grew from 18 to 27 tests; all pass against the
|
||||
freshly compiled binary. No `.ralph-lane` runtime artifacts were touched.
|
||||
|
||||
@@ -656,13 +656,13 @@
|
||||
"The worker exposes neither llama.cpp RPC nor arbitrary graph execution.",
|
||||
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
|
||||
],
|
||||
"passes": false,
|
||||
"passes": true,
|
||||
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md; prd.json is authoritative.",
|
||||
"blocks": [
|
||||
"DGR-036",
|
||||
"DGR-040"
|
||||
],
|
||||
"completionNotes": "Cross-review BLOCKED the initial implementation on pre-open activation acceptance, non-strict flow-control negotiation, in-stream release state retention, and fail-closed identity/version/range validation. Repair required before completion credit."
|
||||
"completionNotes": "Cross-review (Codex GPT-5.5) BLOCK repaired in worktree distributed-gguf-opus. Root protocol defects fixed in the native worker: (1) chunk/decode now fail closed before SessionOpen via a per-session opened flag (terminal ERROR_CODE_INTERNAL), so no activation bypasses lifecycle/cancellation/epoch/flow-control state even when an out-of-band Cancel created placeholder state; (2) flow control is negotiated with strict worker bounds (ShardRuntimeServiceImpl::NegotiateFlow mirrors native_protocol/codec.py negotiate_flow_control) and the negotiated per-session max_chunk_bytes is enforced on every bundle instead of trusting the peer proposal; (3) an in-stream ReleaseSignal now erases session state immediately; (4) SessionOpen rejects incompatible schema, artifact/recipe fingerprint, and shard-range identity and reports the worker own served fingerprint rather than echoing the caller. Nine regression tests added. Real gates on the rebuilt pinned-gRPC binary: cmake --build exit 0; ctest 2/2 passed (shard_worker_selftest, shard_protocol_conformance); tests/test_native_shard_worker.py 27 passed; DGR-024 harness + native protocol 63 passed; compileall exit 0; git diff --check clean; ldd/nm show 0 llama/ggml linkage. Evidence: .scratch/distributed-gguf-runtime/evidence/DGR-033/README.md."
|
||||
},
|
||||
{
|
||||
"id": "DGR-034",
|
||||
|
||||
Reference in New Issue
Block a user