Files

Dobromir Popov 369b2072cc chore: clean superseded GGUF scaffolding

2026-07-16 22:32:37 +03:00

73 KiB

Raw Blame History

PRD: Distributed GGUF Runtime

Specification status: planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. prd.json is authoritative.

Goal

Deliver benchmark-gated, concurrent, distributed GGUF inference across consumer machines through existing Meshnet control-plane behavior and a standalone native worker around upstream llama.cpp. The accepted target is DeepSeek V4 Flash, not the superseded legacy target.

Locked scope

Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are not redesigned. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned ShardEngine boundary.
llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.

Target identities

DeepSeek V4 official target SHA: 60d8d70770c6776ff598c94bb586a859a38244f1.
llama.cpp V4 support lineage began at PR 24162 / merge 8c146a8366304c871efc26057cc90370ccf58dad; DGR-027 later pins one exact validated current commit.
V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under /home.

Quality gates

The canonical gate groups live in prd.json. Every story explicitly requires its applicable shared gates: deterministic targeted tests, Python compile checks where applicable, native CMake/CTest and exact patch verification where applicable, real artifact/hardware provenance for opt-in lanes, git diff --check, and a durable evidence handoff.

Milestones

M0 — Truth and contracts (DGR-017..DGR-020): Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline.
M1 — Protocol and native substrate (DGR-021..DGR-033): Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker.
M2 — Dense vertical proof (DGR-034..DGR-043): Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing.
M3 — DeepSeek V4 Flash alpha (DGR-044..DGR-054): Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off.
M4 — Performance and beta hardening (DGR-055..DGR-067): Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix.
M5 — Release and maintenance (DGR-068..DGR-071): Reproducible packages, upstream collaboration, beta decision, and sustainable recertification.

User stories

DGR-017: Reconcile and clean the superseded DGR backlog

Milestone: M0 · Mode: AFK · State: complete · Depends on: none

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.

Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-018: Define canonical Ralph and Gitea metadata schema

Milestone: M0 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-017

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make prd.json the validated source from which Markdown and Gitea issues can later be generated losslessly.

Define fields for stable ID/title, labels, milestone, type, dependsOn, derived blocks, triage, evidence class, and hardware/model/upstream flags.
Validate that all stories start passes: false, use known dependencies, and have unique stable IDs.
Reject cycles, missing dependencies, mismatched generated blocks, duplicate titles/IDs, and generated artifacts claiming authority over prd.json.
Add deterministic model-free tests for parse, validation, and generation round trips.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-019: Lock alpha and beta performance contracts

Milestone: M0 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-017

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.

Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-020: Run the controlled whole-model GGUF baseline

Milestone: M0 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-019

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.

Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
Publish a threshold-based go, optimize baseline, or stop decision without changing the locked contract.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-021: Define the versioned named-tensor stream envelope

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-018

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/021-define-the-versioned-named-tensor-stream-envelope.md, and evidence READMEs for dependencies (DGR-018) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the backend-neutral protobuf envelope used by direct and relayed Shard activation traffic.

Define schema version, request/work ID, route session/epoch, shard range/effective start, phase, position, and idempotency step.
Define named tensors with shape, dtype, byte order, bounded fragments, compression identity, and checksum.
Reserve extensible fields for token-ID sidebands, architecture state, recurrent state, and MTP without claiming implementations.
Add deterministic serialization, fragmentation, checksum, unknown-field, and size-limit tests.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-022: Define Shard lifecycle and structured status RPCs

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-021

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/022-define-shard-lifecycle-and-structured-status-rpcs.md, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Complete the gRPC contract for worker capability, health, sessions, cancellation, release, and metrics.

Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-023: Make Python and C++ protobuf generation reproducible

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-021

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Generate identical Python/C++ protocol bindings without manual copying or checked-in build debris.

Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
Add Python↔C++ round-trip and descriptor compatibility tests.
A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-024: Implement in-memory fake gRPC seam transport

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-022, DGR-023

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.

Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.
Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.
Verify direct and opaque-relay framing preserve identical protobuf bytes.
Tests require no sockets outside localhost, model downloads, or native accelerator.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-025: Define exact artifact and runtime recipe identity

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-018, DGR-021

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md, and evidence READMEs for dependencies (DGR-018, DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure the tracker and worker only combine numerically and operationally compatible shards.

Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
Bind each shard to an exact half-open range without hardcoding a topology or quant.
Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-026: Provision exact split-GGUF artifacts outside /home

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-025

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md, and evidence READMEs for dependencies (DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make exact split-GGUF inputs reproducibly available from mounted-drive storage without embedding a quantization or topology assumption in product code.

Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under /home and incomplete or mismatched splits.
Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and /home rejection.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-027: Add exact llama.cpp provenance manifest and fetch workspace

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-017

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Pin llama.cpp exactly through an in-repo manifest while fetching source only into an ignored build workspace.

Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
Offline reuse is supported only after the cached tree’s exact identity is verified.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-028: Implement numbered patch-stack apply and verification

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-027

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md, and evidence READMEs for dependencies (DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Maintain a minimal auditable llama.cpp delta with one numbered patch per concern.

Add deterministic apply/check/reverse verification against the exact manifest pin.
Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-029: Create the native CMake skeleton and deterministic CPU lane

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-027, DGR-028

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md, and evidence READMEs for dependencies (DGR-027, DGR-028) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish an out-of-tree standalone native build with a deterministic CPU lane before accelerator matrix work.

Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
Ensure build success alone does not advertise any backend/model/recipe capability.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-030: Add accelerator build presets and native CI matrix

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-029

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/030-add-accelerator-build-presets-and-native-ci-matrix.md, and evidence READMEs for dependencies (DGR-029) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Compile all required accelerator lanes reproducibly while keeping untested real-hardware capabilities registered-dark.

Add isolated out-of-tree presets for CUDA, ROCm, Vulkan, and Metal without changing the deterministic CPU default.
Add a native CI/build matrix that reports unavailable SDKs as explicit unavailable/skipped lanes rather than false success.
Compile each available lane and preserve exact compiler, SDK, upstream pin, patch-stack, and build-option evidence.
Keep every backend/model/recipe lane registered-dark until a separate real-hardware certification record exists.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-031: Introduce the project-owned `ShardEngine` interface

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-021, DGR-025

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/031-introduce-the-project-owned-shardengine-interface.md, and evidence READMEs for dependencies (DGR-021, DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract.

Define load, capabilities, prefill/decode, boundary/logits result, cancel, release, health, and metrics operations.
Use project-owned request/result/state types; expose no ggml_tensor, llama context, scheduler, or ABI-owned structure.
Reserve typed MTP and architecture auxiliary-state hooks without enabling them.
Add contract tests proving fake and future llama implementations obey identical lifecycle semantics.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-032: Implement deterministic fake `ShardEngine`

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-031

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/032-implement-deterministic-fake-shardengine.md, and evidence READMEs for dependencies (DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Provide an engine fixture that deterministically transforms typed boundary bundles and session state.

Support head, middle, tail, prefill, decode, cancellation, and release with deterministic outputs.
Model isolated session/epoch state and deterministic cache-miss/stale-epoch failures.
Support configurable delay, memory pressure, malformed output, and crash injection.
Contract tests distinguish fixture evidence from real-model certification.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-033: Build a standalone fake C++ gRPC Shard worker

Milestone: M1 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-022, DGR-024, DGR-032

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md, and evidence READMEs for dependencies (DGR-022, DGR-024, DGR-032) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the standalone worker process, stream, lifecycle, and supervision shape before llama.cpp integration.

A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-034: Implement dense-Llama range-aware GGUF ownership

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-028, DGR-029, DGR-031

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md, and evidence READMEs for dependencies (DGR-028, DGR-029, DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Load only the dense-Llama tensors owned by a shard range.

Load only blk.N.* tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
Derive authoritative range and endpoint ownership from the loaded engine state.
Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-035: Implement dense architecture boundary input/output

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-021, DGR-031, DGR-034

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md, and evidence READMEs for dependencies (DGR-021, DGR-031, DGR-034) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute dense head/middle/tail ranges through an architecture-defined residual boundary.

Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
Tail returns logits or sampled-token output under an explicit contract.
Uncertified architectures and incompatible boundary schemas fail closed.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-036: Prove dense fixture and real-model range parity

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-033, DGR-035

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md, and evidence READMEs for dependencies (DGR-033, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the engine/protocol seam before adding session concurrency.

Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
A small real dense GGUF passes whole-model versus two-range prefill parity.
At least 32 greedy decode tokens match the locked tolerance.
Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-037: Bind llama.cpp to the standalone worker

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-022, DGR-023, DGR-031, DGR-034, DGR-035

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md, and evidence READMEs for dependencies (DGR-022, DGR-023, DGR-031, DGR-034, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Replace the fake engine with the pinned llama.cpp ShardEngine implementation without changing the public worker contract.

Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
All execution passes through ShardEngine; llama.cpp implementation types remain private.
Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
Graceful shutdown releases model/session resources; injected process death is observable and bounded.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-038: Implement isolated shard-local Hot KV State

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-037

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md, and evidence READMEs for dependencies (DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Map each route session and epoch to isolated state containing only locally owned layers.

Map (route_session_id, route_epoch) to an isolated llama sequence or bounded context.
Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
Release/eviction returns memory to the configured budget without affecting other sessions.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-039: Pass local two-process dense acceptance

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-036, DGR-037, DGR-038

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md, and evidence READMEs for dependencies (DGR-036, DGR-037, DGR-038) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the first real, independently executing distributed GGUF route on one host.

Two worker processes open disjoint dense ranges and both execute real prefill/decode work.
Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass.
Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics.
Killing one worker returns a bounded structured failure rather than hanging.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-040: Add node-side native worker supervision

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-033, DGR-037

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md, and evidence READMEs for dependencies (DGR-033, DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Let the existing node service safely start, probe, stop, and restart the standalone worker.

Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination.
Startup verifies worker binary, artifact identity, recipe, and range before registration.
Crashes or health loss make the capability unavailable without corrupting the Transformers backend.
Tests use the fake worker and deterministic crash injection.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-041: Register native Shard capabilities without redesigning Meshnet

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-025, DGR-040

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md, and evidence READMEs for dependencies (DGR-025, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt native worker identity and capacity to existing backend-agnostic capability admission.

Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status.
Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic.
Uncertified backend/model/recipe combinations are visible but unroutable.
Existing Transformers registration and route tests remain unchanged in behavior.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-042: Carry native frames through direct and existing relay seams

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-024, DGR-040

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md, and evidence READMEs for dependencies (DGR-024, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Use direct gRPC where available and the existing relay as opaque protobuf transport without redesigning relay behavior.

Direct paths use the long-lived gRPC activation stream.
Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract.
Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths.
Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing

Milestone: M2 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-041

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md, and evidence READMEs for dependencies (DGR-041) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt exact GGUF recipe/range compatibility and measured capacity, queue, seam-cost, health, and reliability data into the existing backend-agnostic routing inputs without redesigning routing or load balancing.

Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts.
Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms.
Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics.
Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-044: Pin the DeepSeek V4 Flash target contract

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-019, DGR-025, DGR-026, DGR-027

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/044-pin-the-deepseek-v4-flash-target-contract.md, and evidence READMEs for dependencies (DGR-019, DGR-025, DGR-026, DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Fix the alpha/beta model identity and upstream maturity assumptions before implementing the adapter.

Pin Hugging Face target SHA 60d8d70770c6776ff598c94bb586a859a38244f1.
Record the selected GGUF provenance and exact source binding without assuming one quant.
Record that llama.cpp support began with PR 24162/commit 8c146a8 and remains maturing; the build manifest still pins one exact validated upstream commit.
Record 43 main layers plus MTP and all known architecture-specific state obligations.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-045: Inventory V4 GGUF tensors and layer ownership

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-026, DGR-044

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/045-inventory-v4-gguf-tensors-and-layer-ownership.md, and evidence READMEs for dependencies (DGR-026, DGR-044) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Derive exact head, repeated-layer, tail, expert, auxiliary-state, and MTP ownership from the pinned artifact.

Produce a machine-readable inventory tied to the target SHA and exact GGUF artifact.
Classify embeddings, 43 main layers, final head, MTP, mHC, attention variants, indexer/compressor, and MoE tensors.
Validate half-open range ownership and fail on unclassified required tensors.
Inventory tooling works across quants without embedding quant names in architecture logic.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-046: Define the V4 typed architecture boundary schema

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-021, DGR-045

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.

Define a versioned named bundle for the mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.
Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.
Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.
Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-047: Adapt the upstream V4 mHC boundary for ranged ownership

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-045, DGR-046

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.

Represent and validate the upstream V4 4×4096 mHC boundary without flattening semantic axes.
Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.
Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.
Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-048: Carry token-ID sideband through the first three hash-routed layers

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-046, DGR-047

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/048-carry-token-id-sideband-through-the-first-three-hash-routed-layers.md, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure any stage containing the first three hash-routed layers receives exact token identities.

Head emits token IDs in the typed sideband for boundaries that require them.
Intermediate stages preserve or consume the sideband according to their layer range.
Routes splitting before, within, and after the first three layers pass deterministic partition tests.
Missing or mismatched token IDs fail closed rather than silently selecting incorrect experts.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-049: Keep V4 attention and auxiliary state shard-local

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-046, DGR-047

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/049-keep-v4-attention-and-auxiliary-state-shard-local.md, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Bind V4 CSA/HCA/SWA/indexer/compressor and KV state to the owning shard session/epoch rather than transferring caches over the WAN.

Map CSA, HCA, SWA, indexer, compressor, and KV ownership from the pinned upstream graph to each owned layer range.
Key all such caches/state by route session and epoch with bounded lifecycle, memory accounting, stale-epoch rejection, release, and eviction.
Prove cross-stage messages carry only the typed architecture boundary and cache expectations, never per-layer cache contents.
On route change or state loss, use explicit cache miss and re-prefill/restart; do not migrate V4 caches across workers.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-050: Validate upstream V4 MoE and hash-routing execution under ranged ownership

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-045, DGR-047

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/050-validate-upstream-v4-moe-and-hash-routing-execution-under-ranged-ownership.md, and evidence READMEs for dependencies (DGR-045, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Validate the pinned upstream llama.cpp V4 MoE and hash-routing graph when layers are range-owned, without reimplementing routing or expert kernels in Meshnet.

Load range-owned tensors for 256 routed plus one shared expert and validate upstream execution activates six routed experts where specified.
Validate the first three upstream hash-routed layers receive exact token-ID sideband across splits before, within, and after them.
Compare upstream whole-model and ranged router/expert outputs, shared-expert contribution, and malformed sideband failures.
Document that llama.cpp owns V4 MoE, attention, hash routing, graph, and kernels; no expert-parallel WAN collective or Meshnet reimplementation is introduced.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-051: Assemble the DeepSeek V4 Flash `ShardEngine` adapter

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/051-assemble-the-deepseek-v4-flash-shardengine-adapter.md, and evidence READMEs for dependencies (DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Assemble range ownership hooks and boundary/local-state adapters around upstream llama.cpp V4 execution behind the stable ShardEngine interface.

Support valid head, middle, tail, and whole-model ranges over all 43 main layers using the pinned upstream llama.cpp V4 graph/kernels.
Integrate exact range ownership, mHC boundary, positions, required token-ID sideband, and shard-local session/epoch state adapters.
Report MTP reserved-but-not-executable for alpha and reject unsupported split points or cache expectations.
Include exact adapter/schema/runtime/artifact identity and valid split restrictions without reimplementing V4 MoE, attention, mHC, or hash routing.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-052: Pass local V4 whole-model versus ranged parity

Milestone: M3 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-038, DGR-051

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/052-pass-local-v4-whole-model-versus-ranged-parity.md, and evidence READMEs for dependencies (DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4 prefill and decode correctness locally before network certification.

Compare pinned whole-model llama.cpp with at least two valid ranged partitions.
Prefill intermediates and locked-length greedy decode pass architecture-specific tolerances.
Test splits around the first three hash-routed layers and at least one CSA/HCA/SWA/indexer/compressor transition.
Four concurrent sessions pass KV/state isolation and cleanup.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-053: Certify a real 2–4-stage V4 route

Milestone: M3 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-030, DGR-043, DGR-052

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.

Run one documented 2–4-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.
Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.
Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.
Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.
Only the exact real-hardware lanes exercised become eligible for advertisement.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-054: Enforce the DeepSeek V4 Flash alpha gate

Milestone: M3 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/054-enforce-the-deepseek-v4-flash-alpha-gate.md, and evidence READMEs for dependencies (DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Release alpha only when V4 is correct and clears the pre-locked useful-speed requirement.

Evaluate correctness, model fit, TTFT, prefill/decode speed, seam cost, memory, and failure rate against DGF-004 without changing thresholds.
Separate quant/model-fit gains from runtime/transport gains.
Decision is alpha, optimize measured bottleneck, or stop; failures cannot be relabeled as certification.
Alpha documentation states MTP is reserved but not implemented and identifies every dark hardware lane.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-055: Implement compatible continuous decode batching

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-038, DGR-040, DGR-054

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/055-implement-compatible-continuous-decode-batching.md, and evidence READMEs for dependencies (DGR-038, DGR-040, DGR-054) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Batch compatible active sessions without violating route, recipe, position, or architecture state.

Batch only sessions with compatible engine, recipe, phase, boundary schema, and state layout.
Preserve per-session positions, sequence IDs, outputs, cancellation, and accounting.
Decode receives bounded service while prefill remains schedulable.
Deterministic 1/2/4/8 concurrency tests show no cross-session corruption.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-056: Enforce bounded admission, queues, and backpressure

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-055

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/056-enforce-bounded-admission-queues-and-backpressure.md, and evidence READMEs for dependencies (DGR-055) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prevent weight, KV, state, scratch, and stream buffers from exceeding admitted capacity.

Admission accounts separately for weights, KV/state, scratch, in-flight fragments, and queue slots.
Queue and stream limits produce structured retryable or terminal responses.
Prefill cannot starve decode and one route session cannot monopolize queue capacity.
Telemetry reports active sessions, queue depth/time, batch occupancy, memory pressure, and rejects.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-057: Benchmark batching saturation and fairness

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-055, DGR-056

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/057-benchmark-batching-saturation-and-fairness.md, and evidence READMEs for dependencies (DGR-055, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Identify useful concurrency and saturation points without assuming one global batch size.

Run controlled concurrency 1/2/4/8 and record aggregate throughput, per-request p50/p95, TTFT, queue time, occupancy, and memory.
Measure dense and V4 lanes separately where hardware permits.
Verify bounded fairness and no prefill/decode starvation.
Produce machine-readable saturation recommendations consumed by capability reporting, not hardcoded constants.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-058: Harden worker and stream failure semantics

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-040, DGR-042, DGR-056

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/058-harden-worker-and-stream-failure-semantics.md, and evidence READMEs for dependencies (DGR-040, DGR-042, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure malformed input, process death, transport reset, and cancellation terminate cleanly.

Test worker death, stream reset, malformed bundle, checksum failure, deadline, cache miss, stale epoch, and partial fragment loss.
Cancellation propagates across all stages and releases KV/state, queue entries, and fragment buffers.
Duplicate steps are idempotent; uncertain mutations are never silently replayed.
Billing/work outcomes distinguish completed, cancelled, failed, and unverified work.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-059: Implement alpha-safe route recovery

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-043, DGR-058

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/059-implement-alpha-safe-route-recovery.md, and evidence READMEs for dependencies (DGR-043, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Recover from route loss without importing or continuing unverified state.

On route loss or local-state cache miss, restart from token zero or re-prefill on a newly compatible route according to the locked alpha contract.
Never import KV, CSA, HCA, SWA, indexer, compressor, recurrent, or MTP cache/state across workers.
Keep exact same-route duplicate delivery idempotent while cross-route continuation fails closed.
Test direct/relay loss, cache miss, replacement selection, client cancellation, and billing reconciliation.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-060: Certify V4 long-context state correctness

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-051, DGR-056, DGR-058

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4’s KV and auxiliary state remain correct and bounded at long contexts.

Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.
Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.
Measure memory growth, fragment sizes, TTFT, decode degradation, cancellation latency, and cleanup.
Cache mismatch, truncation, and context-limit behavior fail deterministically.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-061: Certify existing routing with 10+ GGUF stage candidates

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-043, DGR-057, DGR-058

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/061-certify-existing-routing-with-10-gguf-stage-candidates.md, and evidence READMEs for dependencies (DGR-043, DGR-057, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Certify that existing backend-agnostic routing consumes GGUF compatibility and measured-cost inputs for a 10+ candidate scenario without algorithm or policy redesign.

Generate deterministic pools with 10+ compatible GGUF stage candidates plus slower, overloaded, unhealthy, and incompatible alternatives.
Run the existing routing implementation unchanged and verify complete coverage, compatibility, acyclicity, and use of measured inputs.
Regression-test that no stage count, quant, fixed split, architecture, or backend sequence is encoded in production routing policy.
Verify existing route recomputation reacts to measured load/failure while tracker, load-balancing, billing, relay, and provider semantics remain unchanged.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-062: Certify a dynamic real 10+ stage V4 scenario

Milestone: M4 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-053, DGR-060, DGR-061

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/062-certify-a-dynamic-real-10-stage-v4-scenario.md, and evidence READMEs for dependencies (DGR-053, DGR-060, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Demonstrate that a large real route can be dynamically selected and usefully execute V4.

Run one real 10+-stage V4 certification scenario; stage count, quant, and split remain scenario metadata only.
Every selected stage proves real work, exact compatible identity, local state ownership, and bounded resources.
Record end-to-end and per-stage TTFT, prefill/decode, seam cost, queueing, memory, failures, and cleanup.
Inject at least one incompatible or degraded candidate and verify dynamic exclusion/reselection.
The scenario must meet its pre-locked usefulness threshold or remain failed evidence.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-063: Profile and optimize the critical seam

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-057, DGR-061

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/063-profile-and-optimize-the-critical-seam.md, and evidence READMEs for dependencies (DGR-057, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Optimize only the measured dominant compute, serialization, copy, or transport bottleneck.

Profile worker compute, tensor conversion, copies, protobuf framing, compression, relay/direct transport, queueing, and synchronization.
Select one dominant bottleneck from evidence and lock its before/after benchmark.
Implement one bounded optimization without weakening correctness or protocol identity.
Report whether the optimization changes TTFT, prefill, decode, throughput, or memory and preserve raw data.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-064: Add negotiated activation compression

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-060, DGR-063

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/064-add-negotiated-activation-compression.md, and evidence READMEs for dependencies (DGR-060, DGR-063) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce V4 seam cost only when a certified compression recipe improves end-to-end performance acceptably.

Negotiate compression through recipe identity; unnegotiated or mismatched compression fails closed.
Preserve names, shapes, semantic axes, token-ID sideband, and checksums.
Measure quality drift, encode/decode cost, bytes, TTFT, and throughput against uncompressed boundaries.
Compression remains dark unless it meets pre-locked quality and speed thresholds.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-065: Define V4 MTP ownership and distributed state contract

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-051, DGR-054, DGR-060

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/065-define-v4-mtp-ownership-and-distributed-state-contract.md, and evidence READMEs for dependencies (DGR-051, DGR-054, DGR-060) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Lock MTP tensor, execution, boundary, acceptance/rejection, and session-state ownership before implementation while keeping MTP disabled for alpha.

Derive exact MTP tensor and execution ownership from the pinned V4 target and upstream llama.cpp graph, distinguishing main-layer, tail, and MTP responsibilities.
Define typed MTP inputs/outputs plus accepted-token, rejection, rollback/truncation, position, cache, and session/epoch lifecycle semantics.
Specify which MTP state remains shard-local and which typed boundary values cross a seam; prohibit WAN migration of KV or V4 auxiliary caches.
Keep every MTP recipe disabled and unroutable for alpha, with fail-closed capability negotiation and deterministic contract fixtures.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-066: Implement and benchmark V4 MTP

Milestone: M4 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-054, DGR-060, DGR-064, DGR-065

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/066-implement-and-benchmark-v4-mtp.md, and evidence READMEs for dependencies (DGR-054, DGR-060, DGR-064, DGR-065) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Implement the locked MTP contract around upstream llama.cpp before beta, without making MTP an alpha prerequisite.

Implement the exact DGR-065 MTP ownership and typed state contract against the pinned upstream V4 graph while preserving shard-local caches.
Pass whole-model versus ranged MTP correctness, accepted-token, rejection, rollback/truncation, and session-isolation tests.
Benchmark MTP-off versus MTP-on quality, accepted-token rate, TTFT, decode speed, seam bytes, and memory on the locked lane.
Distinguish implemented, certified, disabled, and unsupported MTP recipes in fail-closed capability advertisement.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-067: Certify the backend capability matrix

Milestone: M4 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-030, DGR-053, DGR-062, DGR-066

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/067-certify-the-backend-capability-matrix.md, and evidence READMEs for dependencies (DGR-030, DGR-053, DGR-062, DGR-066) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Publish only backend/model/recipe lanes proven on actual CPU, CUDA, ROCm, Vulkan, or Metal hardware.

Maintain independent build and certification status for CPU, CUDA, ROCm, Vulkan, and Metal.
Each advertised lane includes real hardware/driver/runtime/model evidence for parity, concurrency, cancellation, cleanup, and useful speed.
Build success alone never makes a lane routable; unavailable hardware remains explicitly dark.
Tracker admission consumes signed/versioned certification records rather than backend-name allowlists.
Mixed-backend routes require exact compatibility evidence and fail closed otherwise.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-068: Package reproducible native worker releases

Milestone: M5 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-030, DGR-054, DGR-067

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/068-package-reproducible-native-worker-releases.md, and evidence READMEs for dependencies (DGR-030, DGR-054, DGR-067) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Produce installable worker packages that preserve exact source, patch, protocol, and certification identity.

Package worker binaries/configuration for supported host/backend combinations without bundling an unverified llama.cpp tree.
Embed protocol, engine, upstream pin, patch-stack, build-toolchain, and certification fingerprints.
Installation/startup refuses mismatched artifacts, runtime recipes, or unsupported hardware lanes.
Produce checksums, SBOM/license attribution, reproducible build instructions, and smoke tests.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-069: Prepare narrow upstream llama.cpp collaboration patches

Milestone: M5 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-028, DGR-035, DGR-038, DGR-051

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/069-prepare-narrow-upstream-llama-cpp-collaboration-patches.md, and evidence READMEs for dependencies (DGR-028, DGR-035, DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce long-term patch burden through generic upstreamable hooks without asking llama.cpp to own Meshnet.

Separate generic range loading, boundary I/O, and filtered state hooks from project worker/protocol code.
Provide one scoped patch, focused test, and minimal reproducer per concern against the exact upstream pin.
Compare the proposal with prior-art evidence and current upstream V4 support/maturity.
Prepare human-ready design/outreach text; actual issue/PR submission remains a human action.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-070: Enforce the DeepSeek V4 Flash beta gate

Milestone: M5 · Mode: HITL · Triage: ready-for-human · Depends on: DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/070-enforce-the-deepseek-v4-flash-beta-gate.md, and evidence READMEs for dependencies (DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Promote beta only after concurrency, resilience, long-context, MTP, scale, packaging, and speed gates pass.

Evaluate all beta metrics against DGF-004 without changing thresholds after observing results.
Require V4 parity, batching/fairness, bounded backpressure, failure recovery, long-context correctness, MTP evidence, and dynamic 10+ stage evidence.
Publish the exact certified hardware/backend/recipe matrix; all other lanes remain dark.
Decision is beta, targeted optimization, or stop/rollback, with unresolved failures listed explicitly.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

DGR-071: Establish upstream pin, patch, and certification maintenance

Milestone: M5 · Mode: AFK · Triage: ready-for-agent · Depends on: DGR-069, DGR-070

Fresh Ralph session: read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md, source issue .scratch/distributed-gguf-runtime/issues/071-establish-upstream-pin-patch-and-certification-maintenance.md, and evidence READMEs for dependencies (DGR-069, DGR-070) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Keep beta reproducible and detect upstream/model/hardware drift before it reaches routable nodes.

Add a documented update procedure for llama.cpp pin, numbered patches, protocol/engine versions, V4 target revision, and certification records.
A candidate pin update must apply/build/test in isolation and cannot replace the production pin automatically.
Any changed graph, tensor inventory, boundary, cache/state layout, backend behavior, or performance invalidates affected certification.
Add scheduled model-free checks plus opt-in real-hardware recertification instructions and rollback procedure.
Applicable shared quality gates in prd.json pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.

Non-goals

Replacing or redesigning Tracker routing/load-balancing, billing, relay, or provider policy.
Reimplementing DeepSeek V4 architecture graphs or kernels in Meshnet.
Hardcoding a quant, stage count, node count, split, architecture, or backend sequence.
WAN migration of KV or V4 auxiliary caches.
Advertising a compile-only or fixture-only hardware lane.
Making MTP an alpha dependency.

73 KiB Raw Blame History Unescape Escape