Compare commits
2 Commits
ralph/dist
...
ralph/prox
| Author | SHA1 | Date | |
|---|---|---|---|
|
994546f78e | ||
|
|
6fd9d93e4b |
13
.ralph-tui/config.toml
Normal file
13
.ralph-tui/config.toml
Normal file
@@ -0,0 +1,13 @@
|
||||
# Ralph TUI Configuration
|
||||
# Generated by setup wizard
|
||||
# See: ralph-tui config help
|
||||
|
||||
configVersion = "2.1"
|
||||
tracker = "json"
|
||||
agent = "codex"
|
||||
model = "gpt-5.6-terra"
|
||||
maxIterations = 0
|
||||
autoCommit = true
|
||||
|
||||
[trackerOptions]
|
||||
[agentOptions]
|
||||
@@ -1,714 +1,252 @@
|
||||
# PRD: Distributed GGUF Runtime
|
||||
# PRD: Performant Concurrent Distributed GGUF Runtime
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
## Overview
|
||||
|
||||
## Goal
|
||||
Build one lean native GGUF execution path that lets an Inference Route combine consumer machines to serve models larger than any one node can hold. Reuse the existing Meshnet control plane and llama.cpp/GGML execution engine. Adopt gRPC/HTTP2 and Protocol Buffers for the native Shard worker data plane rather than inventing a transport.
|
||||
|
||||
Deliver benchmark-gated, concurrent, distributed GGUF inference across consumer machines through existing Meshnet control-plane behavior and a standalone native worker around upstream llama.cpp. The accepted target is DeepSeek V4 Flash, not the superseded legacy target.
|
||||
The program is benchmark-gated. GGUF is not assumed faster merely because it is quantized or uses a different file format. The first story compares the current Transformers/safetensors backend against whole-model llama.cpp on controlled model/hardware/quality lanes and locks a performance contract. Native distributed work proceeds only when GGUF provides a meaningful speed or fit benefit.
|
||||
|
||||
## Locked scope
|
||||
## Goals
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
- Execute one GGUF model across independently addressable contiguous Shards.
|
||||
- Retain Hot KV State locally for each Shard and isolate concurrent Route Sessions.
|
||||
- Batch compatible decode steps across active sessions for aggregate throughput.
|
||||
- Use consumer CPU, AMD, NVIDIA, Vulkan, Metal, and mixed routes only where a real certified forward passes.
|
||||
- Beat the current distributed safetensors route under a controlled performance contract or enable a larger otherwise-unroutable model at useful measured speed.
|
||||
- Keep the critical path to Meshnet plus a small pinned llama.cpp fork and standalone C++ worker.
|
||||
- Produce narrow upstream collaboration material for llama.cpp without placing Meshnet networking or economics inside upstream.
|
||||
|
||||
## Quality Gates
|
||||
|
||||
Every story must:
|
||||
|
||||
## Target identities
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
|
||||
## Quality gates
|
||||
|
||||
The canonical gate groups live in `prd.json`. Every story explicitly requires its applicable shared gates: deterministic targeted tests, Python compile checks where applicable, native CMake/CTest and exact patch verification where applicable, real artifact/hardware provenance for opt-in lanes, `git diff --check`, and a durable evidence handoff.
|
||||
|
||||
## Milestones
|
||||
|
||||
- **M0 — Truth and contracts (DGR-017..DGR-020):** Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline.
|
||||
- **M1 — Protocol and native substrate (DGR-021..DGR-033):** Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker.
|
||||
- **M2 — Dense vertical proof (DGR-034..DGR-043):** Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing.
|
||||
- **M3 — DeepSeek V4 Flash alpha (DGR-044..DGR-054):** Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off.
|
||||
- **M4 — Performance and beta hardening (DGR-055..DGR-067):** Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix.
|
||||
- **M5 — Release and maintenance (DGR-068..DGR-071):** Reproducible packages, upstream collaboration, beta decision, and sustainable recertification.
|
||||
|
||||
## User stories
|
||||
|
||||
### DGR-017: Reconcile and clean the superseded DGR backlog
|
||||
|
||||
**Milestone:** M0 · **Mode:** AFK · **State:** complete · **Depends on:** none
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md`, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.
|
||||
|
||||
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
|
||||
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
|
||||
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
|
||||
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-018: Define canonical Ralph and Gitea metadata schema
|
||||
|
||||
**Milestone:** M0 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-017
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make `prd.json` the validated source from which Markdown and Gitea issues can later be generated losslessly.
|
||||
|
||||
- [ ] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
|
||||
- [ ] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
|
||||
- [ ] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
|
||||
- [ ] Add deterministic model-free tests for parse, validation, and generation round trips.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-019: Lock alpha and beta performance contracts
|
||||
|
||||
**Milestone:** M0 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-017
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.
|
||||
|
||||
- [ ] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
|
||||
- [ ] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
|
||||
- [ ] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
|
||||
- [ ] Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
|
||||
- [ ] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-020: Run the controlled whole-model GGUF baseline
|
||||
|
||||
**Milestone:** M0 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-019
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.
|
||||
|
||||
- [ ] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
|
||||
- [ ] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
|
||||
- [ ] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
|
||||
- [ ] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-021: Define the versioned named-tensor stream envelope
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-018
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/021-define-the-versioned-named-tensor-stream-envelope.md`, and evidence READMEs for dependencies (DGR-018) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the backend-neutral protobuf envelope used by direct and relayed Shard activation traffic.
|
||||
|
||||
- [ ] Define schema version, request/work ID, route session/epoch, shard range/effective start, phase, position, and idempotency step.
|
||||
- [ ] Define named tensors with shape, dtype, byte order, bounded fragments, compression identity, and checksum.
|
||||
- [ ] Reserve extensible fields for token-ID sidebands, architecture state, recurrent state, and MTP without claiming implementations.
|
||||
- [ ] Add deterministic serialization, fragmentation, checksum, unknown-field, and size-limit tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-022: Define Shard lifecycle and structured status RPCs
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/022-define-shard-lifecycle-and-structured-status-rpcs.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Complete the gRPC contract for worker capability, health, sessions, cancellation, release, and metrics.
|
||||
|
||||
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
|
||||
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
|
||||
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
|
||||
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-023: Make Python and C++ protobuf generation reproducible
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Generate identical Python/C++ protocol bindings without manual copying or checked-in build debris.
|
||||
|
||||
- [ ] Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
|
||||
- [ ] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
|
||||
- [ ] Add Python↔C++ round-trip and descriptor compatibility tests.
|
||||
- [ ] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-024: Implement in-memory fake gRPC seam transport
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-023
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md`, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.
|
||||
|
||||
- [ ] Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.
|
||||
- [ ] Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.
|
||||
- [ ] Verify direct and opaque-relay framing preserve identical protobuf bytes.
|
||||
- [ ] Tests require no sockets outside localhost, model downloads, or native accelerator.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-025: Define exact artifact and runtime recipe identity
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-018, DGR-021
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md`, and evidence READMEs for dependencies (DGR-018, DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure the tracker and worker only combine numerically and operationally compatible shards.
|
||||
|
||||
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
|
||||
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant.
|
||||
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
|
||||
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-026: Provision exact split-GGUF artifacts outside /home
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-025
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md`, and evidence READMEs for dependencies (DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make exact split-GGUF inputs reproducibly available from mounted-drive storage without embedding a quantization or topology assumption in product code.
|
||||
|
||||
- [ ] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
|
||||
- [ ] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
|
||||
- [ ] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
|
||||
- [ ] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-017
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Pin llama.cpp exactly through an in-repo manifest while fetching source only into an ignored build workspace.
|
||||
|
||||
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
|
||||
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
|
||||
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
|
||||
- [ ] Offline reuse is supported only after the cached tree’s exact identity is verified.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-028: Implement numbered patch-stack apply and verification
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-027
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md`, and evidence READMEs for dependencies (DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Maintain a minimal auditable llama.cpp delta with one numbered patch per concern.
|
||||
|
||||
- [ ] Add deterministic apply/check/reverse verification against the exact manifest pin.
|
||||
- [ ] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
|
||||
- [ ] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
|
||||
- [ ] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-029: Create the native CMake skeleton and deterministic CPU lane
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-027, DGR-028
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md`, and evidence READMEs for dependencies (DGR-027, DGR-028) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish an out-of-tree standalone native build with a deterministic CPU lane before accelerator matrix work.
|
||||
|
||||
- [ ] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
|
||||
- [ ] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
|
||||
- [ ] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
|
||||
- [ ] Ensure build success alone does not advertise any backend/model/recipe capability.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-030: Add accelerator build presets and native CI matrix
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-029
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/030-add-accelerator-build-presets-and-native-ci-matrix.md`, and evidence READMEs for dependencies (DGR-029) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Compile all required accelerator lanes reproducibly while keeping untested real-hardware capabilities registered-dark.
|
||||
|
||||
- [ ] Add isolated out-of-tree presets for CUDA, ROCm, Vulkan, and Metal without changing the deterministic CPU default.
|
||||
- [ ] Add a native CI/build matrix that reports unavailable SDKs as explicit unavailable/skipped lanes rather than false success.
|
||||
- [ ] Compile each available lane and preserve exact compiler, SDK, upstream pin, patch-stack, and build-option evidence.
|
||||
- [ ] Keep every backend/model/recipe lane registered-dark until a separate real-hardware certification record exists.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-031: Introduce the project-owned `ShardEngine` interface
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-025
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/031-introduce-the-project-owned-shardengine-interface.md`, and evidence READMEs for dependencies (DGR-021, DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract.
|
||||
|
||||
- [ ] Define load, capabilities, prefill/decode, boundary/logits result, cancel, release, health, and metrics operations.
|
||||
- [ ] Use project-owned request/result/state types; expose no `ggml_tensor`, llama context, scheduler, or ABI-owned structure.
|
||||
- [ ] Reserve typed MTP and architecture auxiliary-state hooks without enabling them.
|
||||
- [ ] Add contract tests proving fake and future llama implementations obey identical lifecycle semantics.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-032: Implement deterministic fake `ShardEngine`
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-031
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/032-implement-deterministic-fake-shardengine.md`, and evidence READMEs for dependencies (DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Provide an engine fixture that deterministically transforms typed boundary bundles and session state.
|
||||
|
||||
- [ ] Support head, middle, tail, prefill, decode, cancellation, and release with deterministic outputs.
|
||||
- [ ] Model isolated session/epoch state and deterministic cache-miss/stale-epoch failures.
|
||||
- [ ] Support configurable delay, memory pressure, malformed output, and crash injection.
|
||||
- [ ] Contract tests distinguish fixture evidence from real-model certification.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-033: Build a standalone fake C++ gRPC Shard worker
|
||||
|
||||
**Milestone:** M1 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-024, DGR-032
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-024, DGR-032) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the standalone worker process, stream, lifecycle, and supervision shape before llama.cpp integration.
|
||||
|
||||
- [ ] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
|
||||
- [ ] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
|
||||
- [ ] Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
|
||||
- [ ] The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-034: Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-028, DGR-029, DGR-031
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md`, and evidence READMEs for dependencies (DGR-028, DGR-029, DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Load only the dense-Llama tensors owned by a shard range.
|
||||
|
||||
- [ ] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
|
||||
- [ ] Derive authoritative range and endpoint ownership from the loaded engine state.
|
||||
- [ ] Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
|
||||
- [ ] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-035: Implement dense architecture boundary input/output
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-031, DGR-034
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md`, and evidence READMEs for dependencies (DGR-021, DGR-031, DGR-034) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute dense head/middle/tail ranges through an architecture-defined residual boundary.
|
||||
|
||||
- [ ] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
|
||||
- [ ] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail returns logits or sampled-token output under an explicit contract.
|
||||
- [ ] Uncertified architectures and incompatible boundary schemas fail closed.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-036: Prove dense fixture and real-model range parity
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-033, DGR-035
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md`, and evidence READMEs for dependencies (DGR-033, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the engine/protocol seam before adding session concurrency.
|
||||
|
||||
- [ ] Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
|
||||
- [ ] A small real dense GGUF passes whole-model versus two-range prefill parity.
|
||||
- [ ] At least 32 greedy decode tokens match the locked tolerance.
|
||||
- [ ] Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-037: Bind llama.cpp to the standalone worker
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-022, DGR-023, DGR-031, DGR-034, DGR-035
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-023, DGR-031, DGR-034, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Replace the fake engine with the pinned llama.cpp `ShardEngine` implementation without changing the public worker contract.
|
||||
|
||||
- [ ] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
|
||||
- [ ] All execution passes through `ShardEngine`; llama.cpp implementation types remain private.
|
||||
- [ ] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
|
||||
- [ ] Graceful shutdown releases model/session resources; injected process death is observable and bounded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-038: Implement isolated shard-local Hot KV State
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-037
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md`, and evidence READMEs for dependencies (DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Map each route session and epoch to isolated state containing only locally owned layers.
|
||||
|
||||
- [ ] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
|
||||
- [ ] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
|
||||
- [ ] Release/eviction returns memory to the configured budget without affecting other sessions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-039: Pass local two-process dense acceptance
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-036, DGR-037, DGR-038
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md`, and evidence READMEs for dependencies (DGR-036, DGR-037, DGR-038) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the first real, independently executing distributed GGUF route on one host.
|
||||
|
||||
- [ ] Two worker processes open disjoint dense ranges and both execute real prefill/decode work.
|
||||
- [ ] Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass.
|
||||
- [ ] Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics.
|
||||
- [ ] Killing one worker returns a bounded structured failure rather than hanging.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-040: Add node-side native worker supervision
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-033, DGR-037
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md`, and evidence READMEs for dependencies (DGR-033, DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Let the existing node service safely start, probe, stop, and restart the standalone worker.
|
||||
|
||||
- [ ] Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination.
|
||||
- [ ] Startup verifies worker binary, artifact identity, recipe, and range before registration.
|
||||
- [ ] Crashes or health loss make the capability unavailable without corrupting the Transformers backend.
|
||||
- [ ] Tests use the fake worker and deterministic crash injection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-041: Register native Shard capabilities without redesigning Meshnet
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-025, DGR-040
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md`, and evidence READMEs for dependencies (DGR-025, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt native worker identity and capacity to existing backend-agnostic capability admission.
|
||||
|
||||
- [ ] Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status.
|
||||
- [ ] Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic.
|
||||
- [ ] Uncertified backend/model/recipe combinations are visible but unroutable.
|
||||
- [ ] Existing Transformers registration and route tests remain unchanged in behavior.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-042: Carry native frames through direct and existing relay seams
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-024, DGR-040
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md`, and evidence READMEs for dependencies (DGR-024, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Use direct gRPC where available and the existing relay as opaque protobuf transport without redesigning relay behavior.
|
||||
|
||||
- [ ] Direct paths use the long-lived gRPC activation stream.
|
||||
- [ ] Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract.
|
||||
- [ ] Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths.
|
||||
- [ ] Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing
|
||||
|
||||
**Milestone:** M2 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-041
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md`, and evidence READMEs for dependencies (DGR-041) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt exact GGUF recipe/range compatibility and measured capacity, queue, seam-cost, health, and reliability data into the existing backend-agnostic routing inputs without redesigning routing or load balancing.
|
||||
|
||||
- [ ] Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts.
|
||||
- [ ] Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms.
|
||||
- [ ] Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics.
|
||||
- [ ] Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-044: Pin the DeepSeek V4 Flash target contract
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-019, DGR-025, DGR-026, DGR-027
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/044-pin-the-deepseek-v4-flash-target-contract.md`, and evidence READMEs for dependencies (DGR-019, DGR-025, DGR-026, DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Fix the alpha/beta model identity and upstream maturity assumptions before implementing the adapter.
|
||||
|
||||
- [ ] Pin Hugging Face target SHA `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- [ ] Record the selected GGUF provenance and exact source binding without assuming one quant.
|
||||
- [ ] Record that llama.cpp support began with PR 24162/commit `8c146a8` and remains maturing; the build manifest still pins one exact validated upstream commit.
|
||||
- [ ] Record 43 main layers plus MTP and all known architecture-specific state obligations.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-045: Inventory V4 GGUF tensors and layer ownership
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-026, DGR-044
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/045-inventory-v4-gguf-tensors-and-layer-ownership.md`, and evidence READMEs for dependencies (DGR-026, DGR-044) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Derive exact head, repeated-layer, tail, expert, auxiliary-state, and MTP ownership from the pinned artifact.
|
||||
|
||||
- [ ] Produce a machine-readable inventory tied to the target SHA and exact GGUF artifact.
|
||||
- [ ] Classify embeddings, 43 main layers, final head, MTP, mHC, attention variants, indexer/compressor, and MoE tensors.
|
||||
- [ ] Validate half-open range ownership and fail on unclassified required tensors.
|
||||
- [ ] Inventory tooling works across quants without embedding quant names in architecture logic.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-046: Define the V4 typed architecture boundary schema
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-021, DGR-045
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md`, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.
|
||||
|
||||
- [ ] Define a versioned named bundle for the mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.
|
||||
- [ ] Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.
|
||||
- [ ] Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.
|
||||
- [ ] Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-047: Adapt the upstream V4 mHC boundary for ranged ownership
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-045, DGR-046
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.
|
||||
|
||||
- [ ] Represent and validate the upstream V4 4×4096 mHC boundary without flattening semantic axes.
|
||||
- [ ] Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.
|
||||
- [ ] Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.
|
||||
- [ ] Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-048: Carry token-ID sideband through the first three hash-routed layers
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-046, DGR-047
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/048-carry-token-id-sideband-through-the-first-three-hash-routed-layers.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure any stage containing the first three hash-routed layers receives exact token identities.
|
||||
|
||||
- [ ] Head emits token IDs in the typed sideband for boundaries that require them.
|
||||
- [ ] Intermediate stages preserve or consume the sideband according to their layer range.
|
||||
- [ ] Routes splitting before, within, and after the first three layers pass deterministic partition tests.
|
||||
- [ ] Missing or mismatched token IDs fail closed rather than silently selecting incorrect experts.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-049: Keep V4 attention and auxiliary state shard-local
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-046, DGR-047
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/049-keep-v4-attention-and-auxiliary-state-shard-local.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Bind V4 CSA/HCA/SWA/indexer/compressor and KV state to the owning shard session/epoch rather than transferring caches over the WAN.
|
||||
|
||||
- [ ] Map CSA, HCA, SWA, indexer, compressor, and KV ownership from the pinned upstream graph to each owned layer range.
|
||||
- [ ] Key all such caches/state by route session and epoch with bounded lifecycle, memory accounting, stale-epoch rejection, release, and eviction.
|
||||
- [ ] Prove cross-stage messages carry only the typed architecture boundary and cache expectations, never per-layer cache contents.
|
||||
- [ ] On route change or state loss, use explicit cache miss and re-prefill/restart; do not migrate V4 caches across workers.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-050: Validate upstream V4 MoE and hash-routing execution under ranged ownership
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-045, DGR-047
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/050-validate-upstream-v4-moe-and-hash-routing-execution-under-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Validate the pinned upstream llama.cpp V4 MoE and hash-routing graph when layers are range-owned, without reimplementing routing or expert kernels in Meshnet.
|
||||
|
||||
- [ ] Load range-owned tensors for 256 routed plus one shared expert and validate upstream execution activates six routed experts where specified.
|
||||
- [ ] Validate the first three upstream hash-routed layers receive exact token-ID sideband across splits before, within, and after them.
|
||||
- [ ] Compare upstream whole-model and ranged router/expert outputs, shared-expert contribution, and malformed sideband failures.
|
||||
- [ ] Document that llama.cpp owns V4 MoE, attention, hash routing, graph, and kernels; no expert-parallel WAN collective or Meshnet reimplementation is introduced.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-051: Assemble the DeepSeek V4 Flash `ShardEngine` adapter
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/051-assemble-the-deepseek-v4-flash-shardengine-adapter.md`, and evidence READMEs for dependencies (DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Assemble range ownership hooks and boundary/local-state adapters around upstream llama.cpp V4 execution behind the stable ShardEngine interface.
|
||||
|
||||
- [ ] Support valid head, middle, tail, and whole-model ranges over all 43 main layers using the pinned upstream llama.cpp V4 graph/kernels.
|
||||
- [ ] Integrate exact range ownership, mHC boundary, positions, required token-ID sideband, and shard-local session/epoch state adapters.
|
||||
- [ ] Report MTP reserved-but-not-executable for alpha and reject unsupported split points or cache expectations.
|
||||
- [ ] Include exact adapter/schema/runtime/artifact identity and valid split restrictions without reimplementing V4 MoE, attention, mHC, or hash routing.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-052: Pass local V4 whole-model versus ranged parity
|
||||
|
||||
**Milestone:** M3 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-038, DGR-051
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/052-pass-local-v4-whole-model-versus-ranged-parity.md`, and evidence READMEs for dependencies (DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4 prefill and decode correctness locally before network certification.
|
||||
|
||||
- [ ] Compare pinned whole-model llama.cpp with at least two valid ranged partitions.
|
||||
- [ ] Prefill intermediates and locked-length greedy decode pass architecture-specific tolerances.
|
||||
- [ ] Test splits around the first three hash-routed layers and at least one CSA/HCA/SWA/indexer/compressor transition.
|
||||
- [ ] Four concurrent sessions pass KV/state isolation and cleanup.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-053: Certify a real 2–4-stage V4 route
|
||||
|
||||
**Milestone:** M3 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-030, DGR-043, DGR-052
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md`, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.
|
||||
|
||||
- [ ] Run one documented 2–4-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.
|
||||
- [ ] Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.
|
||||
- [ ] Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.
|
||||
- [ ] Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.
|
||||
- [ ] Only the exact real-hardware lanes exercised become eligible for advertisement.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-054: Enforce the DeepSeek V4 Flash alpha gate
|
||||
|
||||
**Milestone:** M3 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/054-enforce-the-deepseek-v4-flash-alpha-gate.md`, and evidence READMEs for dependencies (DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Release alpha only when V4 is correct and clears the pre-locked useful-speed requirement.
|
||||
|
||||
- [ ] Evaluate correctness, model fit, TTFT, prefill/decode speed, seam cost, memory, and failure rate against DGF-004 without changing thresholds.
|
||||
- [ ] Separate quant/model-fit gains from runtime/transport gains.
|
||||
- [ ] Decision is `alpha`, `optimize measured bottleneck`, or `stop`; failures cannot be relabeled as certification.
|
||||
- [ ] Alpha documentation states MTP is reserved but not implemented and identifies every dark hardware lane.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-055: Implement compatible continuous decode batching
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-038, DGR-040, DGR-054
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/055-implement-compatible-continuous-decode-batching.md`, and evidence READMEs for dependencies (DGR-038, DGR-040, DGR-054) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Batch compatible active sessions without violating route, recipe, position, or architecture state.
|
||||
|
||||
- [ ] Batch only sessions with compatible engine, recipe, phase, boundary schema, and state layout.
|
||||
- [ ] Preserve per-session positions, sequence IDs, outputs, cancellation, and accounting.
|
||||
- [ ] Decode receives bounded service while prefill remains schedulable.
|
||||
- [ ] Deterministic 1/2/4/8 concurrency tests show no cross-session corruption.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-056: Enforce bounded admission, queues, and backpressure
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-055
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/056-enforce-bounded-admission-queues-and-backpressure.md`, and evidence READMEs for dependencies (DGR-055) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prevent weight, KV, state, scratch, and stream buffers from exceeding admitted capacity.
|
||||
|
||||
- [ ] Admission accounts separately for weights, KV/state, scratch, in-flight fragments, and queue slots.
|
||||
- [ ] Queue and stream limits produce structured retryable or terminal responses.
|
||||
- [ ] Prefill cannot starve decode and one route session cannot monopolize queue capacity.
|
||||
- [ ] Telemetry reports active sessions, queue depth/time, batch occupancy, memory pressure, and rejects.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-057: Benchmark batching saturation and fairness
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-055, DGR-056
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/057-benchmark-batching-saturation-and-fairness.md`, and evidence READMEs for dependencies (DGR-055, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Identify useful concurrency and saturation points without assuming one global batch size.
|
||||
|
||||
- [ ] Run controlled concurrency 1/2/4/8 and record aggregate throughput, per-request p50/p95, TTFT, queue time, occupancy, and memory.
|
||||
- [ ] Measure dense and V4 lanes separately where hardware permits.
|
||||
- [ ] Verify bounded fairness and no prefill/decode starvation.
|
||||
- [ ] Produce machine-readable saturation recommendations consumed by capability reporting, not hardcoded constants.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-058: Harden worker and stream failure semantics
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-040, DGR-042, DGR-056
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/058-harden-worker-and-stream-failure-semantics.md`, and evidence READMEs for dependencies (DGR-040, DGR-042, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure malformed input, process death, transport reset, and cancellation terminate cleanly.
|
||||
|
||||
- [ ] Test worker death, stream reset, malformed bundle, checksum failure, deadline, cache miss, stale epoch, and partial fragment loss.
|
||||
- [ ] Cancellation propagates across all stages and releases KV/state, queue entries, and fragment buffers.
|
||||
- [ ] Duplicate steps are idempotent; uncertain mutations are never silently replayed.
|
||||
- [ ] Billing/work outcomes distinguish completed, cancelled, failed, and unverified work.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-059: Implement alpha-safe route recovery
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-043, DGR-058
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/059-implement-alpha-safe-route-recovery.md`, and evidence READMEs for dependencies (DGR-043, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Recover from route loss without importing or continuing unverified state.
|
||||
|
||||
- [ ] On route loss or local-state cache miss, restart from token zero or re-prefill on a newly compatible route according to the locked alpha contract.
|
||||
- [ ] Never import KV, CSA, HCA, SWA, indexer, compressor, recurrent, or MTP cache/state across workers.
|
||||
- [ ] Keep exact same-route duplicate delivery idempotent while cross-route continuation fails closed.
|
||||
- [ ] Test direct/relay loss, cache miss, replacement selection, client cancellation, and billing reconciliation.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-060: Certify V4 long-context state correctness
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-051, DGR-056, DGR-058
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md`, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4’s KV and auxiliary state remain correct and bounded at long contexts.
|
||||
|
||||
- [ ] Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.
|
||||
- [ ] Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.
|
||||
- [ ] Measure memory growth, fragment sizes, TTFT, decode degradation, cancellation latency, and cleanup.
|
||||
- [ ] Cache mismatch, truncation, and context-limit behavior fail deterministically.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-061: Certify existing routing with 10+ GGUF stage candidates
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-043, DGR-057, DGR-058
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/061-certify-existing-routing-with-10-gguf-stage-candidates.md`, and evidence READMEs for dependencies (DGR-043, DGR-057, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Certify that existing backend-agnostic routing consumes GGUF compatibility and measured-cost inputs for a 10+ candidate scenario without algorithm or policy redesign.
|
||||
|
||||
- [ ] Generate deterministic pools with 10+ compatible GGUF stage candidates plus slower, overloaded, unhealthy, and incompatible alternatives.
|
||||
- [ ] Run the existing routing implementation unchanged and verify complete coverage, compatibility, acyclicity, and use of measured inputs.
|
||||
- [ ] Regression-test that no stage count, quant, fixed split, architecture, or backend sequence is encoded in production routing policy.
|
||||
- [ ] Verify existing route recomputation reacts to measured load/failure while tracker, load-balancing, billing, relay, and provider semantics remain unchanged.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-062: Certify a dynamic real 10+ stage V4 scenario
|
||||
|
||||
**Milestone:** M4 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-053, DGR-060, DGR-061
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/062-certify-a-dynamic-real-10-stage-v4-scenario.md`, and evidence READMEs for dependencies (DGR-053, DGR-060, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Demonstrate that a large real route can be dynamically selected and usefully execute V4.
|
||||
|
||||
- [ ] Run one real 10+-stage V4 certification scenario; stage count, quant, and split remain scenario metadata only.
|
||||
- [ ] Every selected stage proves real work, exact compatible identity, local state ownership, and bounded resources.
|
||||
- [ ] Record end-to-end and per-stage TTFT, prefill/decode, seam cost, queueing, memory, failures, and cleanup.
|
||||
- [ ] Inject at least one incompatible or degraded candidate and verify dynamic exclusion/reselection.
|
||||
- [ ] The scenario must meet its pre-locked usefulness threshold or remain failed evidence.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-063: Profile and optimize the critical seam
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-057, DGR-061
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/063-profile-and-optimize-the-critical-seam.md`, and evidence READMEs for dependencies (DGR-057, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Optimize only the measured dominant compute, serialization, copy, or transport bottleneck.
|
||||
|
||||
- [ ] Profile worker compute, tensor conversion, copies, protobuf framing, compression, relay/direct transport, queueing, and synchronization.
|
||||
- [ ] Select one dominant bottleneck from evidence and lock its before/after benchmark.
|
||||
- [ ] Implement one bounded optimization without weakening correctness or protocol identity.
|
||||
- [ ] Report whether the optimization changes TTFT, prefill, decode, throughput, or memory and preserve raw data.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-064: Add negotiated activation compression
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-060, DGR-063
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/064-add-negotiated-activation-compression.md`, and evidence READMEs for dependencies (DGR-060, DGR-063) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce V4 seam cost only when a certified compression recipe improves end-to-end performance acceptably.
|
||||
|
||||
- [ ] Negotiate compression through recipe identity; unnegotiated or mismatched compression fails closed.
|
||||
- [ ] Preserve names, shapes, semantic axes, token-ID sideband, and checksums.
|
||||
- [ ] Measure quality drift, encode/decode cost, bytes, TTFT, and throughput against uncompressed boundaries.
|
||||
- [ ] Compression remains dark unless it meets pre-locked quality and speed thresholds.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-065: Define V4 MTP ownership and distributed state contract
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-051, DGR-054, DGR-060
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/065-define-v4-mtp-ownership-and-distributed-state-contract.md`, and evidence READMEs for dependencies (DGR-051, DGR-054, DGR-060) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Lock MTP tensor, execution, boundary, acceptance/rejection, and session-state ownership before implementation while keeping MTP disabled for alpha.
|
||||
|
||||
- [ ] Derive exact MTP tensor and execution ownership from the pinned V4 target and upstream llama.cpp graph, distinguishing main-layer, tail, and MTP responsibilities.
|
||||
- [ ] Define typed MTP inputs/outputs plus accepted-token, rejection, rollback/truncation, position, cache, and session/epoch lifecycle semantics.
|
||||
- [ ] Specify which MTP state remains shard-local and which typed boundary values cross a seam; prohibit WAN migration of KV or V4 auxiliary caches.
|
||||
- [ ] Keep every MTP recipe disabled and unroutable for alpha, with fail-closed capability negotiation and deterministic contract fixtures.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-066: Implement and benchmark V4 MTP
|
||||
|
||||
**Milestone:** M4 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-054, DGR-060, DGR-064, DGR-065
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/066-implement-and-benchmark-v4-mtp.md`, and evidence READMEs for dependencies (DGR-054, DGR-060, DGR-064, DGR-065) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Implement the locked MTP contract around upstream llama.cpp before beta, without making MTP an alpha prerequisite.
|
||||
|
||||
- [ ] Implement the exact DGR-065 MTP ownership and typed state contract against the pinned upstream V4 graph while preserving shard-local caches.
|
||||
- [ ] Pass whole-model versus ranged MTP correctness, accepted-token, rejection, rollback/truncation, and session-isolation tests.
|
||||
- [ ] Benchmark MTP-off versus MTP-on quality, accepted-token rate, TTFT, decode speed, seam bytes, and memory on the locked lane.
|
||||
- [ ] Distinguish implemented, certified, disabled, and unsupported MTP recipes in fail-closed capability advertisement.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-067: Certify the backend capability matrix
|
||||
|
||||
**Milestone:** M4 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-030, DGR-053, DGR-062, DGR-066
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/067-certify-the-backend-capability-matrix.md`, and evidence READMEs for dependencies (DGR-030, DGR-053, DGR-062, DGR-066) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Publish only backend/model/recipe lanes proven on actual CPU, CUDA, ROCm, Vulkan, or Metal hardware.
|
||||
|
||||
- [ ] Maintain independent build and certification status for CPU, CUDA, ROCm, Vulkan, and Metal.
|
||||
- [ ] Each advertised lane includes real hardware/driver/runtime/model evidence for parity, concurrency, cancellation, cleanup, and useful speed.
|
||||
- [ ] Build success alone never makes a lane routable; unavailable hardware remains explicitly dark.
|
||||
- [ ] Tracker admission consumes signed/versioned certification records rather than backend-name allowlists.
|
||||
- [ ] Mixed-backend routes require exact compatibility evidence and fail closed otherwise.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-068: Package reproducible native worker releases
|
||||
|
||||
**Milestone:** M5 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-030, DGR-054, DGR-067
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/068-package-reproducible-native-worker-releases.md`, and evidence READMEs for dependencies (DGR-030, DGR-054, DGR-067) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Produce installable worker packages that preserve exact source, patch, protocol, and certification identity.
|
||||
|
||||
- [ ] Package worker binaries/configuration for supported host/backend combinations without bundling an unverified llama.cpp tree.
|
||||
- [ ] Embed protocol, engine, upstream pin, patch-stack, build-toolchain, and certification fingerprints.
|
||||
- [ ] Installation/startup refuses mismatched artifacts, runtime recipes, or unsupported hardware lanes.
|
||||
- [ ] Produce checksums, SBOM/license attribution, reproducible build instructions, and smoke tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-069: Prepare narrow upstream llama.cpp collaboration patches
|
||||
|
||||
**Milestone:** M5 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-028, DGR-035, DGR-038, DGR-051
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/069-prepare-narrow-upstream-llama-cpp-collaboration-patches.md`, and evidence READMEs for dependencies (DGR-028, DGR-035, DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce long-term patch burden through generic upstreamable hooks without asking llama.cpp to own Meshnet.
|
||||
|
||||
- [ ] Separate generic range loading, boundary I/O, and filtered state hooks from project worker/protocol code.
|
||||
- [ ] Provide one scoped patch, focused test, and minimal reproducer per concern against the exact upstream pin.
|
||||
- [ ] Compare the proposal with prior-art evidence and current upstream V4 support/maturity.
|
||||
- [ ] Prepare human-ready design/outreach text; actual issue/PR submission remains a human action.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-070: Enforce the DeepSeek V4 Flash beta gate
|
||||
|
||||
**Milestone:** M5 · **Mode:** HITL · **Triage:** `ready-for-human` · **Depends on:** DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/070-enforce-the-deepseek-v4-flash-beta-gate.md`, and evidence READMEs for dependencies (DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Promote beta only after concurrency, resilience, long-context, MTP, scale, packaging, and speed gates pass.
|
||||
|
||||
- [ ] Evaluate all beta metrics against DGF-004 without changing thresholds after observing results.
|
||||
- [ ] Require V4 parity, batching/fairness, bounded backpressure, failure recovery, long-context correctness, MTP evidence, and dynamic 10+ stage evidence.
|
||||
- [ ] Publish the exact certified hardware/backend/recipe matrix; all other lanes remain dark.
|
||||
- [ ] Decision is `beta`, `targeted optimization`, or `stop/rollback`, with unresolved failures listed explicitly.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
### DGR-071: Establish upstream pin, patch, and certification maintenance
|
||||
|
||||
**Milestone:** M5 · **Mode:** AFK · **Triage:** `ready-for-agent` · **Depends on:** DGR-069, DGR-070
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/071-establish-upstream-pin-patch-and-certification-maintenance.md`, and evidence READMEs for dependencies (DGR-069, DGR-070) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Keep beta reproducible and detect upstream/model/hardware drift before it reaches routable nodes.
|
||||
|
||||
- [ ] Add a documented update procedure for llama.cpp pin, numbered patches, protocol/engine versions, V4 target revision, and certification records.
|
||||
- [ ] A candidate pin update must apply/build/test in isolation and cannot replace the production pin automatically.
|
||||
- [ ] Any changed graph, tensor inventory, boundary, cache/state layout, backend behavior, or performance invalidates affected certification.
|
||||
- [ ] Add scheduled model-free checks plus opt-in real-hardware recertification instructions and rollback procedure.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Replacing or redesigning Tracker routing/load-balancing, billing, relay, or provider policy.
|
||||
- Reimplementing DeepSeek V4 architecture graphs or kernels in Meshnet.
|
||||
- Hardcoding a quant, stage count, node count, split, architecture, or backend sequence.
|
||||
- WAN migration of KV or V4 auxiliary caches.
|
||||
- Advertising a compile-only or fixture-only hardware lane.
|
||||
- Making MTP an alpha dependency.
|
||||
- Run its targeted `pytest` tests.
|
||||
- Run `python -m compileall packages tests` for Python changes.
|
||||
- Run `git diff --check`.
|
||||
- Keep default tests deterministic, model-download-free, API-credit-free, and GPU-free.
|
||||
- Preserve existing Transformers/safetensors behavior unless the story explicitly changes a versioned compatibility contract.
|
||||
|
||||
Stories touching the native worker must also:
|
||||
|
||||
- Build the pinned C++ target with CMake.
|
||||
- Run focused C++/protocol tests through CTest or the documented equivalent.
|
||||
- Verify the llama.cpp patch stack applies cleanly to the exact pinned commit.
|
||||
|
||||
Real-model/hardware stories must:
|
||||
|
||||
- Require `MESHNET_ENABLE_REAL_INFERENCE_TESTS=1`.
|
||||
- Use the machine-specific mounted-drive model path and the certified runtime environment; never place model artifacts under `/home`.
|
||||
- Record exact model revision, artifact hash, runtime recipe, hardware, driver/backend, commands, raw JSON metrics, and output-quality result.
|
||||
- Label synthetic tests as unit coverage rather than distributed acceptance.
|
||||
|
||||
Before a story is marked complete, run the full deterministic `pytest -q` suite or record the exact pre-existing unrelated failure with a clean-tree reproduction.
|
||||
|
||||
## User Stories
|
||||
|
||||
### DGR-001: Lock the safetensors-versus-GGUF performance contract
|
||||
**Description:** As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.
|
||||
- [ ] Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.
|
||||
- [ ] Report TTFT, prefill tok/s, decode tok/s, p50/p95 latency, aggregate throughput, RSS, VRAM, artifact size, failures, and output drift in machine-readable JSON.
|
||||
- [ ] Add concurrency levels 1 and 4 where memory permits.
|
||||
- [ ] Write a versioned performance contract consumed by later release gates, including an explicit stop condition when llama.cpp/GGUF has no meaningful speed or fit benefit.
|
||||
|
||||
### DGR-002: Adopt the versioned gRPC Shard protocol
|
||||
**Description:** As a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
|
||||
- [ ] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
|
||||
- [ ] Define bounded chunking for prefill and a small decode fast path.
|
||||
- [ ] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
|
||||
- [ ] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
|
||||
- [ ] Add generated-schema round-trip and compatibility tests in Python and C++.
|
||||
|
||||
### DGR-003: Define exact Artifact and runtime recipe identity
|
||||
**Description:** As the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
|
||||
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||
|
||||
### DGR-004: Create the reproducible pinned llama.cpp patch stack
|
||||
**Description:** As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
|
||||
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
|
||||
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
|
||||
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
|
||||
- [ ] Preserve upstream license and attribution notices.
|
||||
- [ ] Add a clean rebuild smoke test that does not download a model.
|
||||
|
||||
### DGR-005: Implement dense-Llama range-aware GGUF ownership
|
||||
**Description:** As a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Register and allocate only `blk.N.*` tensors in the assigned range.
|
||||
- [ ] Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.
|
||||
- [ ] Prefer range-aware mapping from one exact source GGUF; if derivative sub-GGUFs are used temporarily, verify source/slice hashes and avoid claiming final artifact semantics.
|
||||
- [ ] Report authoritative loaded range and endpoint ownership from the model, not operator CLI claims.
|
||||
- [ ] Demonstrate mapped/resident memory scales with owned tensors rather than full model size.
|
||||
|
||||
### DGR-006: Implement architecture-defined boundary input/output
|
||||
**Description:** As a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Head accepts token IDs and owns token embedding.
|
||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||
- [ ] The adapter interface fails closed for uncertified architectures.
|
||||
|
||||
### DGR-007: Add isolated concurrent local Hot KV State
|
||||
**Description:** As a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Map `(Route Session ID, route epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Allocate KV only for owned layers.
|
||||
- [ ] Support prefill append, decode append, truncate, release, TTL/LRU eviction, and explicit cache-miss response.
|
||||
- [ ] Reject stale epochs and incompatible cache recipes.
|
||||
- [ ] At least four concurrent sessions on a small model complete without token or KV cross-talk.
|
||||
- [ ] Cancellation/release of one session leaves other sessions intact and memory returns to the configured budget.
|
||||
|
||||
### DGR-008: Build the standalone C++ gRPC Shard worker
|
||||
**Description:** As a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Worker exposes capability, health, session stream, release, cancellation, and metrics services from DGR-002.
|
||||
- [ ] Worker loads one exact Artifact/recipe/Shard identity and refuses mismatched requests.
|
||||
- [ ] Streaming path enforces bounded messages, flow control, deadlines, idempotency, and independent session cancellation.
|
||||
- [ ] Worker does not expose raw llama.cpp RPC or arbitrary GGML graph execution.
|
||||
- [ ] Graceful shutdown releases sessions; crash behavior is bounded and observable.
|
||||
- [ ] Python integration tests run against a fake model mode without model downloads.
|
||||
|
||||
### DGR-009: Integrate the native worker with Meshnet
|
||||
**Description:** As the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Implement the existing model-backend surface without changing Transformers behavior.
|
||||
- [ ] Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.
|
||||
- [ ] Tracker forms only complete compatible routes and keeps uncertified recipes dark.
|
||||
- [ ] Direct routes use gRPC streams; relayed routes carry the same versioned protobuf frames as opaque binary through the existing relay seam.
|
||||
- [ ] Existing request/work IDs, cancellation, Generation Telemetry, billing, and per-node attribution remain correlated.
|
||||
- [ ] No vLLM, Nakshatra, prima.cpp, or custom-engine control plane becomes a core dependency.
|
||||
|
||||
### DGR-010: Pass local real-model two-process acceptance
|
||||
**Description:** As a release engineer, I need real local distributed parity before involving network variability.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Two local worker processes open disjoint dense-Llama ranges from the certified Artifact.
|
||||
- [ ] Prefill and at least 32 greedy decode tokens match whole-model llama.cpp within the certified tolerance.
|
||||
- [ ] Each worker retains only its own tensors and Hot KV State.
|
||||
- [ ] Four concurrent Route Sessions pass isolation and cleanup checks.
|
||||
- [ ] Report TTFT, prefill/decode throughput, seam bytes/latency, worker RSS/VRAM, KV memory, batch size, and queue time.
|
||||
- [ ] Killing one worker produces a bounded structured failure rather than a deadlock.
|
||||
|
||||
### DGR-011: Pass a real heterogeneous two-machine route
|
||||
**Description:** As a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Tracker selects two physical nodes with disjoint Shards and one exact certified recipe/compatibility class.
|
||||
- [ ] Actual CPU/GPU execution occurs on both nodes; synthetic workers do not satisfy acceptance.
|
||||
- [ ] Prefill/decode, concurrent-session isolation, telemetry, cancellation, and cleanup pass over the real transport/relay path.
|
||||
- [ ] Exact hardware, network, backend, model hash, route, commands, and raw metrics are recorded.
|
||||
- [ ] A model or recipe larger than one participating node's admitted memory is exercised when available.
|
||||
- [ ] Output drift is measured and incompatible mixed backends fail closed.
|
||||
|
||||
### DGR-012: Implement continuous batching and bounded admission
|
||||
**Description:** As a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Node scheduler admits sessions against weight, KV, scratch, and queue budgets.
|
||||
- [ ] Compatible decode steps from multiple sessions form llama.cpp batches while preserving per-session positions and outputs.
|
||||
- [ ] Prefill does not starve decode; scheduling policy and bounds are explicit.
|
||||
- [ ] Backpressure prevents unbounded queued activations or KV growth.
|
||||
- [ ] Capability telemetry reports active sessions, queue depth, batch occupancy, KV pressure, prefill/decode rates, and rejected admissions.
|
||||
- [ ] Concurrency 1/2/4/8 benchmark identifies saturation and shows no cross-session corruption.
|
||||
|
||||
### DGR-013: Harden failure, cancellation, and restart semantics
|
||||
**Description:** As a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Deadlines and heartbeat/health loss terminate blocked stream operations.
|
||||
- [ ] Cancellation propagates across every Shard and releases local KV and queued buffers.
|
||||
- [ ] Duplicate steps are idempotent; uncertain mutations are never replayed silently.
|
||||
- [ ] Alpha failover restarts from token zero on a newly compatible route rather than importing unverified KV.
|
||||
- [ ] Worker death, stream reset, malformed bundle, stale epoch, and cache miss tests pass.
|
||||
- [ ] Billing/work records distinguish completed, cancelled, failed, and unverified work.
|
||||
|
||||
### DGR-014: Enforce the GGUF-versus-safetensors release gate
|
||||
**Description:** As the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.
|
||||
- [ ] Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.
|
||||
- [ ] Evaluate against the DGR-001 performance contract without changing thresholds after seeing results.
|
||||
- [ ] Ship recommendation is one of: promote GGUF, optimize a measured bottleneck with a new bounded task, or stop the native track.
|
||||
- [ ] Results clearly separate quantization gains from transport/runtime gains.
|
||||
|
||||
### DGR-015: Add and certify a Qwen3/Qwen3-MoE adapter
|
||||
**Description:** As a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Implement explicit tensor ownership, router/top-k, expert/shared-expert, Q/K normalization, boundary bundle, and cache semantics for the selected Qwen3 family recipe.
|
||||
- [ ] Do not reuse the dense-Llama adapter through unchecked name substitutions.
|
||||
- [ ] Whole-model versus distributed prefill/decode parity passes the architecture-specific tolerance.
|
||||
- [ ] Expert memory ownership and communication are measured.
|
||||
- [ ] Real consumer-hardware acceptance and capability admission pass before the recipe becomes routable.
|
||||
|
||||
### DGR-016: Produce the upstream llama.cpp collaboration package
|
||||
**Description:** As a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] Separate generic llama.cpp hooks from Meshnet protocol/control-plane code.
|
||||
- [ ] Prepare minimal reproducible examples and tests for range-aware loading, boundary input/output, and layer-filtered KV.
|
||||
- [ ] Compare the proposal with Nakshatra and prima.cpp evidence and explain why the API is generally useful.
|
||||
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
||||
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
1. The public distributed primitive is an ordered Inference Route of contiguous Shards.
|
||||
2. The native runtime uses llama.cpp/GGML; vLLM remains optional as a complete managed provider.
|
||||
3. Native worker communication uses gRPC/HTTP2 and Protocol Buffers with one stable stream per Route Session Activation Seam.
|
||||
4. Artifact identity, runtime recipe, boundary schema, activation dtype and cache layout must match exactly before routing.
|
||||
5. Hot KV State remains local to the node serving the Shard.
|
||||
6. Multiple Route Sessions must execute concurrently without shared-cache corruption.
|
||||
7. Nodes batch compatible active decode steps and enforce bounded admission/backpressure.
|
||||
8. Unsupported architectures and hardware recipes remain non-routable until real certification passes.
|
||||
9. Default tests never download models or require GPUs; real tests are explicit and preserve artifacts off `/home`.
|
||||
10. The release decision is based on measured performance, fit, quality, concurrency, and reliability relative to the safetensors baseline.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Forking vLLM or importing its PagedAttention/Torch distributed runtime.
|
||||
- Adopting Nakshatra, prima.cpp, llama-gguf, LiGGUF, or GPUStack as the control plane.
|
||||
- Public WAN tensor/expert parallel collectives.
|
||||
- QUIC, WebRTC, or a custom socket protocol.
|
||||
- Automatic KV migration or mid-generation route repair in the first release.
|
||||
- Speculative decoding or disaggregated prefill before the core release gate.
|
||||
- Supporting every GGUF architecture before dense Llama and Qwen3-family certification.
|
||||
- A marketing-scale model demo that bypasses parity, concurrency, admission, or performance gates.
|
||||
|
||||
## Success Metrics
|
||||
|
||||
- A real model larger than one admitted node can execute across consumer machines when suitable hardware/artifacts are available.
|
||||
- Four or more concurrent sessions complete without cross-talk; hardware-specific saturation is measured.
|
||||
- Distributed GGUF passes the locked performance/fit contract against the existing safetensors route.
|
||||
- Worker and Tracker recover all resources after completion, cancellation, malformed input, and node failure.
|
||||
- The critical runtime remains Meshnet plus one standalone worker and a small auditable llama.cpp patch stack.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Exact benchmark model and quantization lanes are selected by DGR-001 from currently supported, legally redistributable artifacts.
|
||||
- Final hardware-specific concurrency and useful-speed thresholds are locked by measured baselines rather than guessed globally.
|
||||
- Upstream llama.cpp acceptance is desirable but not a prerequisite for the first narrow pinned fork.
|
||||
|
||||
@@ -1,57 +1,309 @@
|
||||
# Ralph context: Distributed GGUF Runtime
|
||||
# Ralph execution context: Performant Concurrent Distributed GGUF Runtime
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
Status: authoritative context for every fresh Ralph iteration
|
||||
Last updated: 2026-07-13
|
||||
|
||||
## Mandatory startup for every fresh story
|
||||
## Mandatory startup sequence
|
||||
|
||||
1. Read this file and authoritative `prd.json` completely.
|
||||
2. Read the generated source issue named in the selected story description.
|
||||
3. Read every dependency evidence README; legacy DGR-001..016 evidence is provenance only.
|
||||
4. Read `docs/adr/0024-distributed-gguf-runtime.md`, root `CONTEXT.md`, `.claude/memory/MEMORY.md`, and relevant live source/tests.
|
||||
5. Inspect `git status`; preserve unrelated work. Never infer implementation from planning text or old pass states.
|
||||
6. If blocked or oversized, keep `passes: false` and write an honest `BLOCKED.md`/`DECOMPOSITION.md`; never weaken criteria or fabricate evidence.
|
||||
Before changing code, every Ralph agent must:
|
||||
|
||||
## Locked scope
|
||||
1. Read this file completely.
|
||||
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
|
||||
3. Read `docs/adr/0024-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
|
||||
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
|
||||
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
|
||||
6. Read the evidence/handoff directories for every declared dependency.
|
||||
7. Inspect `git status` and preserve all pre-existing working-tree changes.
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
A fresh Ralph iteration has no conversational memory. These files are the context contract.
|
||||
|
||||
## Target identities
|
||||
## Story sizing and interruption rule
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
Each story is intended to fit one focused Ralph context. Before implementation, estimate whether every acceptance criterion can be completed and verified in the current iteration.
|
||||
|
||||
## Control/data-plane contract
|
||||
If the story is too large, an external dependency is unavailable, or the context/provider limit prevents completion:
|
||||
|
||||
Meshnet continues to own registration, coverage, existing route selection/load balancing, route epochs/sessions, direct/relay behavior, capability admission, cancellation, telemetry, billing, validation, and attribution. The GGUF adapter exposes measured inputs to those existing mechanisms. Direct seams use long-lived gRPC streams; relay seams carry byte-identical protobuf frames opaquely.
|
||||
- Do not weaken criteria.
|
||||
- Do not mark the issue done or set `passes: true`.
|
||||
- Avoid leaving an unverified cross-cutting partial implementation when a smaller safe spike is possible.
|
||||
- Write `evidence/<TASK-ID>/DECOMPOSITION.md` or `BLOCKED.md` with the exact blocker, current verified state, proposed child stories, dependency graph and rollback/continuation instructions.
|
||||
- Stop for supervised review.
|
||||
|
||||
The project-owned `ShardEngine` hides llama.cpp internals. A worker loads one exact artifact/recipe/range identity. Default tests use fake/tiny fixtures. Real runs are opt-in, preserve raw metrics, and never download models under `/home`.
|
||||
If interrupted after code changes, record every changed file, command result and unresolved invariant so the next fresh loop can verify rather than guess.
|
||||
|
||||
## Gitea issue synchronization
|
||||
## Product objective
|
||||
|
||||
Gitea is a projection of `prd.json`, never a competing source of truth. Before and after every supervised Ralph run, invoke:
|
||||
Build performant, concurrent distributed inference that combines consumer machines to serve top open models that exceed one node's RAM/VRAM.
|
||||
|
||||
```bash
|
||||
python3 scripts/ralph_gitea_sync.py sync
|
||||
A distributed demo is not success. The product must provide:
|
||||
|
||||
- Useful measured prefill and decode speed.
|
||||
- Multiple concurrent Route Sessions.
|
||||
- No KV/token cross-talk.
|
||||
- Bounded memory, queues, cancellation and failures.
|
||||
- Real execution on every participating node.
|
||||
- A model-fit or performance advantage over the current Transformers/safetensors route.
|
||||
|
||||
## Critical-path architecture
|
||||
|
||||
```text
|
||||
Existing Meshnet control plane
|
||||
|
|
||||
Versioned Protobuf over gRPC/HTTP2
|
||||
|
|
||||
Project-owned standalone C++ Shard worker
|
||||
|
|
||||
Small exact-commit llama.cpp patch stack
|
||||
```
|
||||
|
||||
For a complete Ralph invocation with automatic state reconciliation, use:
|
||||
Meshnet remains the only control plane and owns:
|
||||
|
||||
```bash
|
||||
scripts/ralph-gitea-run.sh ralph-tui run --prd .scratch/distributed-gguf-runtime/prd.json --agent claude --model sonnet --iterations 1 --no-tui --no-setup --direct-merge --no-sandbox
|
||||
- Tracker registration, Coverage Map, route selection and route epochs.
|
||||
- Route Sessions and Activation Seams.
|
||||
- Direct/relay routing.
|
||||
- Capability admission.
|
||||
- Cancellation, Generation Telemetry and backpressure.
|
||||
- Billing, validation and per-node work attribution.
|
||||
|
||||
Do not introduce another scheduler/control plane from vLLM, Nakshatra, prima.cpp, llama-gguf, GPUStack or another project.
|
||||
|
||||
## Runtime decisions that are not open
|
||||
|
||||
1. Public-network Shards are contiguous transformer layer ranges.
|
||||
2. llama.cpp/GGML is the native GGUF execution substrate.
|
||||
3. The project owns a small standalone worker and a narrow pinned llama.cpp patch stack.
|
||||
4. The native Shard protocol is Protocol Buffers over gRPC/HTTP2.
|
||||
5. One long-lived bidirectional stream serves one Route Session Activation Seam.
|
||||
6. The public activation boundary is a versioned named-tensor bundle.
|
||||
7. Hot KV State remains local to the node serving the Shard.
|
||||
8. `(Route Session ID, route epoch)` maps to an isolated llama sequence or bounded context.
|
||||
9. Concurrency uses continuous batching of compatible active sessions inside each node.
|
||||
10. Transformers/safetensors remains the correctness and performance baseline.
|
||||
11. vLLM may be an optional complete managed provider and concept donor; it is not forked into public Shards.
|
||||
12. Tensor/expert collectives are deferred to a trusted composite provider, not public WAN routes.
|
||||
13. Unsupported architectures/backends remain registered-but-dark until real certification passes.
|
||||
14. Alpha failure retries from token zero; unverified KV is never migrated silently.
|
||||
15. Model artifacts must remain on mounted-drive storage and never under `/home`.
|
||||
|
||||
Changing one of these requires an explicit ADR update and human review, not an incidental story implementation.
|
||||
|
||||
## Performance discipline
|
||||
|
||||
GGUF performance is a hypothesis. Never write “GGUF is faster” without measurements.
|
||||
|
||||
DGR-001 locks controlled benchmark lanes and thresholds. DGR-014 enforces the final distributed comparison.
|
||||
|
||||
Always distinguish:
|
||||
|
||||
- Weight quantization from activation/compute/KV dtype.
|
||||
- Runtime/kernel gains from quantization/model-fit gains.
|
||||
- Single-request latency from aggregate concurrency throughput.
|
||||
- Synthetic unit coverage from real distributed acceptance.
|
||||
|
||||
Required metrics where applicable:
|
||||
|
||||
```text
|
||||
TTFT
|
||||
prefill tokens/sec
|
||||
decode tokens/sec
|
||||
aggregate throughput
|
||||
p50/p95 latency
|
||||
seam bytes and latency
|
||||
queue and batch occupancy
|
||||
RSS and VRAM
|
||||
KV pressure
|
||||
output-quality drift
|
||||
failures and cleanup
|
||||
```
|
||||
|
||||
The sync creates/reconciles one Gitea issue per `DGR-*` story, creates missing labels/milestones, closes issues whose `passes` is true, marks the selected next eligible story `status:in-progress`, and marks blocked stories `status:blocked`. `gitea-issues.json` is a derived mapping only.
|
||||
Do not weaken or move performance thresholds after seeing implementation results.
|
||||
|
||||
## Evidence and completion
|
||||
## Transport discipline
|
||||
|
||||
Each story writes `/run/media/popov/d/DEV/repos/d-popov.com/AI/.claude/worktrees/distributed-gguf-runtime/.scratch/distributed-gguf-runtime/evidence/<DGR-ID>/README.md` with exact files, commands/results, limitations, identities, and dependent-story handoff. Only `prd.json` may record `passes`; DGR-017 and DGR-018 are complete and DGR-019 onward remain false. Generated Markdown and Gitea issues cannot override it. One scoped commit per story is expected during future execution.
|
||||
Do not invent a raw TCP protocol, new WebSocket protocol, QUIC layer or bespoke binary control format.
|
||||
|
||||
The `.proto` schema is the semantic contract. Direct transport uses gRPC. Existing relay infrastructure may carry the same serialized protobuf frames as opaque binary.
|
||||
|
||||
Protocol requirements:
|
||||
|
||||
- Schema/version negotiation.
|
||||
- Request/work ID.
|
||||
- Route Session ID and route epoch.
|
||||
- Exact Model Artifact/runtime recipe fingerprint.
|
||||
- Shard range and effective overlap-safe start.
|
||||
- Prefill/decode/release/cancel phases.
|
||||
- Position/token range and idempotency step.
|
||||
- Named tensors with shape, dtype, byte order and bounded fragments.
|
||||
- Compression/checksum.
|
||||
- Cache expectation/result.
|
||||
- Deadlines, cancellation, flow control and structured status.
|
||||
|
||||
Avoid per-token channel creation and unbounded unary payloads. Generated code and build tooling must be reproducible; do not require manual copying.
|
||||
|
||||
## Native runtime discipline
|
||||
|
||||
Reuse llama.cpp for GGUF, mmap, kernels, architecture graphs, tokenizer, KV, sequences and heterogeneous backends.
|
||||
|
||||
The project patch stack is limited to:
|
||||
|
||||
- Range-aware tensor registration/loading.
|
||||
- Endpoint-specific embedding/final head ownership.
|
||||
- Architecture-defined intermediate input/output.
|
||||
- Intermediate output before final norm/head.
|
||||
- Layer-filtered KV and session mapping.
|
||||
|
||||
Do not place Meshnet routing, transport, billing or authentication inside llama.cpp. Keep patches numbered, scoped, pinned and upstreamable.
|
||||
|
||||
Dense Llama-family is first. Qwen3/Qwen3-MoE is a separate adapter after the dense release gate. Do not generalize through unchecked tensor-name substitutions.
|
||||
|
||||
## Existing code seams to inspect first
|
||||
|
||||
- `packages/node/meshnet_node/model_backend.py` — backend abstraction.
|
||||
- `packages/node/meshnet_node/torch_server.py` — reference ranged execution and session behavior.
|
||||
- `packages/node/meshnet_node/activation_compression.py` — current activation framing/compression.
|
||||
- `packages/node/meshnet_node/route_session_benchmark.py` — existing benchmark infrastructure.
|
||||
- `packages/tracker/meshnet_tracker/server.py` — registration, route and proxy behavior.
|
||||
- `packages/tracker/meshnet_tracker/capability.py` — fail-closed capability admission.
|
||||
- `tests/test_real_model_backend.py` — real backend coverage.
|
||||
- `tests/test_tracker_routing.py` — route/session behavior.
|
||||
- `tests/test_tracker_capability_admission.py` — recipe admission.
|
||||
- `tests/test_route_session_benchmark.py` and `tests/test_manual_route_benchmark.py` — benchmark patterns.
|
||||
- `docs/adr/0008-binary-activation-wire-format.md` — existing wire compatibility.
|
||||
- `docs/adr/0012-start-layer-overlapping-shards.md` — effective start semantics.
|
||||
- `docs/adr/0022-sharded-per-node-kv-cache.md` — Hot KV State contract.
|
||||
- `docs/adr/0023-model-agnostic-node-capability-admission.md` — certification/admission.
|
||||
|
||||
Do not edit generated `build/`, `__pycache__`, egg-info, Ralph logs or unrelated scratch features.
|
||||
|
||||
## Planned source layout
|
||||
|
||||
Use these paths unless current code inspection proves a better project-consistent location. If changed, document the reason in task evidence.
|
||||
|
||||
```text
|
||||
packages/node/native/
|
||||
proto/shard_runtime.proto
|
||||
cmake/
|
||||
llama/
|
||||
UPSTREAM_COMMIT
|
||||
patches/
|
||||
gguf_worker/
|
||||
tests/
|
||||
|
||||
packages/node/meshnet_node/
|
||||
native_protocol/
|
||||
gguf_backend.py
|
||||
runtime_recipe.py
|
||||
|
||||
.scratch/distributed-gguf-runtime/evidence/<TASK-ID>/
|
||||
README.md
|
||||
commands.txt
|
||||
results.json or other machine-readable evidence
|
||||
```
|
||||
|
||||
Generated protobuf/C++ build outputs belong in build directories unless packaging explicitly requires checked-in generated Python modules. The story must document the generation command and version.
|
||||
|
||||
## Story output map
|
||||
|
||||
| Story | Required durable outputs |
|
||||
|---|---|
|
||||
| DGR-001 | benchmark harness/tests; `evidence/DGR-001/performance-contract.json`; raw/summary benchmark evidence |
|
||||
| DGR-002 | `packages/node/native/proto/shard_runtime.proto`; reproducible Python/C++ generation/build wiring; protocol round-trip/compatibility tests; `evidence/DGR-002/` |
|
||||
| DGR-003 | exact runtime-recipe/fingerprint implementation and admission tests; `evidence/DGR-003/` |
|
||||
| DGR-004 | exact upstream pin, numbered patch series, reproducible fetch/apply/build smoke; `evidence/DGR-004/` |
|
||||
| DGR-005 | dense-Llama range ownership loader and memory evidence; `evidence/DGR-005/` |
|
||||
| DGR-006 | architecture boundary adapter/parity tests and results; `evidence/DGR-006/` |
|
||||
| DGR-007 | concurrent session/KV manager, isolation/cleanup tests; `evidence/DGR-007/` |
|
||||
| DGR-008 | standalone C++ gRPC worker, fake-model integration tests, lifecycle evidence; `evidence/DGR-008/` |
|
||||
| DGR-009 | Meshnet backend/registration/relay integration and tests; `evidence/DGR-009/` |
|
||||
| DGR-010 | real local two-process commands, raw metrics and parity report; `evidence/DGR-010/` |
|
||||
| DGR-011 | two-machine configuration, commands, hardware/network manifest and raw results; `evidence/DGR-011/` |
|
||||
| DGR-012 | continuous scheduler/admission implementation and 1/2/4/8 concurrency report; `evidence/DGR-012/` |
|
||||
| DGR-013 | failure/cancel/restart test matrix and resource-cleanup evidence; `evidence/DGR-013/` |
|
||||
| DGR-014 | immutable final comparison against DGR-001 thresholds and ship/stop recommendation; `evidence/DGR-014/` |
|
||||
| DGR-015 | Qwen3-family adapter, architecture-specific parity/admission/performance evidence; `evidence/DGR-015/` |
|
||||
| DGR-016 | narrow upstream patches/tests, design note and human-ready outreach package; `evidence/DGR-016/` |
|
||||
|
||||
## Dependency handoff rule
|
||||
|
||||
For every dependency listed by Ralph:
|
||||
|
||||
1. Confirm its `passes` state in `prd.json`.
|
||||
2. Read `.scratch/distributed-gguf-runtime/evidence/<DEPENDENCY-ID>/README.md`.
|
||||
3. Verify referenced source paths and commands still exist.
|
||||
4. Do not repeat completed work unless verification exposes a concrete defect.
|
||||
5. If dependency evidence is missing or contradictory, stop and repair the dependency instead of guessing.
|
||||
|
||||
## Testing and hardware rules
|
||||
|
||||
Default tests must be deterministic, GPU-free, model-download-free and API-credit-free.
|
||||
|
||||
Real model tests require:
|
||||
|
||||
```text
|
||||
MESHNET_ENABLE_REAL_INFERENCE_TESTS=1
|
||||
```
|
||||
|
||||
On this machine:
|
||||
|
||||
- Use `.venv-rocm` for real Radeon 8060S ROCm execution.
|
||||
- The default Python 3.14 `.venv` is unsuitable for real ROCm inference.
|
||||
- Resolve model storage through the machine-specific `.env.<hostname>` configuration.
|
||||
- Never download model artifacts under `/home`.
|
||||
- Real acceptance must exercise actual Tracker-routed CPU/GPU computation; synthetic workers are only unit tests.
|
||||
|
||||
Record exact:
|
||||
|
||||
- Model/revision and Artifact hash.
|
||||
- Quantization and runtime recipe.
|
||||
- Host/hardware/backend/driver.
|
||||
- Commands and environment names without secrets.
|
||||
- Raw output and metrics.
|
||||
- Whether the evidence is synthetic, local-real, or multi-machine-real.
|
||||
|
||||
## Worktree and commit discipline
|
||||
|
||||
This repository may contain pre-existing changes from research or another feature.
|
||||
|
||||
- Inspect `git status` before editing.
|
||||
- Never reset, checkout over, stash, delete or reformat unrelated changes.
|
||||
- Stage only files belonging to the selected story.
|
||||
- Exclude `.ralph-tui`, iteration logs, caches, generated builds, FUSE artifacts and unrelated scratch work.
|
||||
- Keep one scoped commit per completed story when the supervising loop requests commits.
|
||||
- Do not modify `passes` for another story.
|
||||
|
||||
## Mandatory finish/handoff sequence
|
||||
|
||||
Before emitting `<promise>COMPLETE</promise>`:
|
||||
|
||||
1. Verify every acceptance criterion with real command output or file evidence.
|
||||
2. Run story-specific gates and repository quality gates.
|
||||
3. Write `.scratch/distributed-gguf-runtime/evidence/<TASK-ID>/README.md` containing:
|
||||
- Summary of changes.
|
||||
- Exact files changed.
|
||||
- Commands run and their real results.
|
||||
- Performance/correctness evidence.
|
||||
- Known limitations and deferred work.
|
||||
- Compatibility or migration notes.
|
||||
- Clear handoff for dependent stories.
|
||||
4. Save machine-readable evidence beside it when the story produces metrics or schemas.
|
||||
5. Update the source issue status to `done` only after all gates pass.
|
||||
6. Preserve failures honestly. Never fabricate model, benchmark, test or hardware output.
|
||||
|
||||
## Authoritative references
|
||||
|
||||
Active decisions:
|
||||
|
||||
- `.scratch/distributed-gguf-runtime/README.md`
|
||||
- `.scratch/distributed-gguf-runtime/implementation-strategy.md`
|
||||
- `.scratch/distributed-gguf-runtime/architecture.md`
|
||||
- `docs/adr/0024-distributed-gguf-runtime.md`
|
||||
- `.scratch/distributed-gguf-runtime/PRD.md`
|
||||
- `.scratch/distributed-gguf-runtime/prd.json`
|
||||
|
||||
Source research:
|
||||
|
||||
- `docs/research/distributed-gguf-landscape.md`
|
||||
- `docs/research/distributed-gguf-github-followup.md`
|
||||
- `docs/research/vllm-distributed-gguf-assessment.md`
|
||||
|
||||
If historical notes conflict with these files, the active decisions above win.
|
||||
|
||||
@@ -1,32 +1,46 @@
|
||||
# Distributed GGUF Runtime planning workspace
|
||||
# Performant concurrent distributed GGUF runtime
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
Status: active benchmark-gated implementation program.
|
||||
|
||||
## Objective
|
||||
|
||||
## Locked scope
|
||||
Serve top open models across consumer machines with useful performance and concurrent Route Sessions while keeping the runtime lean.
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
## Critical path
|
||||
|
||||
## Target identities
|
||||
```text
|
||||
Meshnet control plane
|
||||
-> versioned gRPC/Protobuf Shard protocol
|
||||
-> project-owned standalone C++ worker
|
||||
-> small pinned llama.cpp patch stack
|
||||
```
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
Transformers/safetensors remains the correctness baseline. vLLM remains an optional complete managed provider and a design donor; it is not forked into the public mesh.
|
||||
|
||||
## Navigation
|
||||
## Planning artifacts
|
||||
|
||||
- [`prd.json`](prd.json) — sole authoritative 55-story backlog, DGR-017..071.
|
||||
- [`PRD.md`](PRD.md) — human-readable projection of goals, gates, and all stories.
|
||||
- [`RALPH-CONTEXT.md`](RALPH-CONTEXT.md) — mandatory fresh-session context.
|
||||
- [`architecture.md`](architecture.md), [`implementation-strategy.md`](implementation-strategy.md), [`milestones.md`](milestones.md) — design and execution sequence.
|
||||
- [`issues/`](issues/) — generated story specs; files 01..16 are retained legacy artifacts pending DGR-017.
|
||||
- [`evidence/`](evidence/) — provenance and future per-story handoffs.
|
||||
- **[Mandatory Ralph context](RALPH-CONTEXT.md)** — read first in every fresh iteration
|
||||
- [Task evidence contract](evidence/README.md)
|
||||
- [Implementation strategy](implementation-strategy.md)
|
||||
- [Current architecture](architecture.md)
|
||||
- [PRD](PRD.md)
|
||||
- [Ralph backlog](prd.json)
|
||||
- [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
- [Milestones](milestones.md)
|
||||
- [Issues](issues/)
|
||||
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)
|
||||
- [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md)
|
||||
- [vLLM assessment](../../docs/research/vllm-distributed-gguf-assessment.md)
|
||||
|
||||
## Ralph execution
|
||||
|
||||
Use supervised one-story iterations for this high-risk runtime:
|
||||
|
||||
```bash
|
||||
ralph-tui run \
|
||||
--prd .scratch/distributed-gguf-runtime/prd.json \
|
||||
--agent claude --model opus \
|
||||
--iterations 1 --no-tui --no-setup --verify
|
||||
```
|
||||
|
||||
Inspect the diff, run the story gates, and commit one verified story before the next iteration. Real-model stories require the explicit environment gate and mounted-drive model storage.
|
||||
|
||||
@@ -1,45 +1,259 @@
|
||||
# Distributed GGUF Runtime architecture
|
||||
# Performant Concurrent Distributed GGUF Architecture
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
Status: current target architecture
|
||||
Last updated: 2026-07-13
|
||||
|
||||
## Product invariant
|
||||
|
||||
## Locked scope
|
||||
The system exists to serve high-quality models that exceed one consumer node's memory while retaining useful interactive speed and aggregate concurrency. A feature that only produces a distributed demo but is slower, globally serialized, or impossible to operate on consumer hardware is not complete.
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
## Existing control plane
|
||||
|
||||
## Target identities
|
||||
Meshnet remains the only public control plane:
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
- Tracker registration, Coverage Map, route scoring and assignment.
|
||||
- Contiguous Shards and overlap-safe effective starts.
|
||||
- Stable Route Sessions and route epochs.
|
||||
- Local per-Shard Hot KV State in the reference backend.
|
||||
- Direct/relay transport, cancellation and backpressure.
|
||||
- Generation Telemetry, billing, validation and per-node attribution.
|
||||
- Model-agnostic capability admission.
|
||||
|
||||
## Topology
|
||||
No external engine replaces these responsibilities.
|
||||
|
||||
## Runtime topology
|
||||
|
||||
```text
|
||||
existing Meshnet Tracker/control plane
|
||||
-> existing backend-agnostic route/load-balancing decision
|
||||
-> direct gRPC or existing opaque relay
|
||||
-> project-owned standalone C++ Shard worker
|
||||
-> project-owned ShardEngine
|
||||
-> pinned upstream llama.cpp + numbered range/boundary/state hook patches
|
||||
-> GGUF mmap, upstream V4 graph/kernels, local per-shard state
|
||||
OpenAI-compatible client
|
||||
|
|
||||
Gateway / Tracker Node
|
||||
|
|
||||
ordered Inference Route
|
||||
|
|
||||
+-- head Shard: tokenizer/embedding + early layers
|
||||
| local weights and Hot KV State
|
||||
|
|
||||
+-- middle Shard(s): architecture boundary + owned layers
|
||||
| local weights and Hot KV State
|
||||
|
|
||||
+-- tail Shard: final layers + norm/head/sampling
|
||||
local weights and Hot KV State
|
||||
```
|
||||
|
||||
A route is ordered contiguous half-open ranges. Head owns token embedding; tail owns final norm/head/sampling. Compatibility fingerprints bind source/split hashes, tokenizer, architecture adapter, typed boundary, runtime pin/patches, backend, quant, activation/compute/KV layout, range, and certification.
|
||||
Weights never move in the per-request hot path. Every node opens and verifies its local Model Artifact before becoming routable.
|
||||
|
||||
## V4 boundary and state
|
||||
## Primary execution substrate
|
||||
|
||||
The inter-stage boundary is semantic and versioned: mHC 4×4096 residual, positions, token IDs only where the first three hash-routed layers require them, and cache/schema expectations. CSA/HCA/SWA/indexer/compressor/KV state belongs to upstream layer execution on the owning worker and is isolated by `(route_session_id, route_epoch)`. On loss, return cache miss and re-prefill/restart. Never serialize those caches into the WAN bundle.
|
||||
```text
|
||||
project-owned C++ Shard worker
|
||||
|
|
||||
small exact-commit llama.cpp patch stack
|
||||
|
|
||||
GGUF mmap, quantized kernels, architecture graphs,
|
||||
KV/sequence operations, CPU/CUDA/HIP/Vulkan/Metal backends
|
||||
```
|
||||
|
||||
## Concurrency, failure, and admission
|
||||
The patch stack adds only the missing local execution seam:
|
||||
|
||||
Compatible sessions may be continuously batched within a worker while retaining isolated positions/state. Admission bounds weights, local state/KV, scratch, fragments, and queues. Uncertain cross-route mutation is not replayed. Registration can show an uncertified lane, but existing admission keeps it unroutable until signed/versioned real-hardware evidence exists.
|
||||
1. Range-aware tensor registration/loading.
|
||||
2. Endpoint-specific embedding and final head ownership.
|
||||
3. Architecture-defined intermediate input.
|
||||
4. Architecture-defined pre-tail boundary output.
|
||||
5. Layer-filtered KV and external session mapping.
|
||||
|
||||
The worker owns protocol translation and process lifecycle. llama.cpp never receives Tracker, relay, billing or volunteer-network code.
|
||||
|
||||
## Shard data plane
|
||||
|
||||
Use Protocol Buffers and gRPC over HTTP/2.
|
||||
|
||||
### Service shape
|
||||
|
||||
- Unary capability and health.
|
||||
- Bidirectional Route Session stream.
|
||||
- Explicit release and cancellation.
|
||||
- Metrics suitable for capability admission and route scoring.
|
||||
|
||||
### Session stream
|
||||
|
||||
One long-lived stream represents one Route Session Activation Seam. It amortizes connection setup and inherits HTTP/2 flow control. Every message carries enough identity to reject stale or incompatible work.
|
||||
|
||||
```text
|
||||
schema version
|
||||
request/work id
|
||||
Route Session id
|
||||
route epoch
|
||||
Model Artifact hash
|
||||
runtime recipe fingerprint
|
||||
Shard begin/end and effective start
|
||||
prefill/decode/release/cancel phase
|
||||
position and token range
|
||||
idempotency step id
|
||||
cache expectation/result
|
||||
named tensor bundle
|
||||
compression/checksum
|
||||
```
|
||||
|
||||
Prefill tensors are split into bounded ordered frames. Decode messages carry one-step architecture boundary bundles and remain small.
|
||||
|
||||
Direct nodes use gRPC. Nodes requiring the existing relay carry the same protobuf frames as opaque binary through the relay session. This preserves one semantic protocol instead of maintaining separate direct and relay payload contracts.
|
||||
|
||||
## Architecture boundary
|
||||
|
||||
The public boundary is a versioned named-tensor bundle:
|
||||
|
||||
```text
|
||||
bundle schema/version
|
||||
architecture adapter and boundary point
|
||||
named tensors
|
||||
per-tensor shape, dtype and byte order
|
||||
payload fragments
|
||||
compression/checksum
|
||||
```
|
||||
|
||||
Dense Llama may use one residual tensor. Other adapters may require more. vLLM's Llama and Qwen3-MoE PP paths demonstrate a boundary with both `hidden_states` and `residual`; therefore the generic protocol must not assume one anonymous tensor.
|
||||
|
||||
Only the head owns token embedding. Only the tail owns final normalization, LM head and sampling. Middle Shards exchange the architecture-defined pre-tail boundary, not final normalized embeddings.
|
||||
|
||||
## Hot KV State and concurrency
|
||||
|
||||
```text
|
||||
(Route Session id, route epoch)
|
||||
-> local llama sequence or bounded context
|
||||
-> KV for owned layers only
|
||||
-> lease, memory accounting and lifecycle
|
||||
```
|
||||
|
||||
Required operations:
|
||||
|
||||
- Prefill append.
|
||||
- Decode append.
|
||||
- Truncate after rejected speculative positions if later enabled.
|
||||
- Explicit release.
|
||||
- TTL/LRU eviction.
|
||||
- Cache-miss response.
|
||||
- Stale-epoch rejection.
|
||||
|
||||
A node must not clear global KV on a new stream or serialize all requests behind one logical serving sequence.
|
||||
|
||||
## Continuous batching
|
||||
|
||||
Autoregressive dependencies remain sequential inside one Route Session. Aggregate throughput comes from batching compatible decode steps across active sessions:
|
||||
|
||||
```text
|
||||
time 0: session A token 1 + session B token 8 + session C token 3
|
||||
-> one llama batch for this Shard
|
||||
|
||||
time 1: next ready positions from active sessions
|
||||
-> next llama batch
|
||||
```
|
||||
|
||||
The node scheduler:
|
||||
|
||||
- Admits work against weight, KV, scratch and queue budgets.
|
||||
- Keeps per-session token positions and outputs separate.
|
||||
- Prevents long prefill from starving decode.
|
||||
- Applies bounded backpressure.
|
||||
- Reports active sessions, queue depth, batch occupancy, KV pressure and throughput.
|
||||
|
||||
The initial deterministic gate is four concurrent sessions on a small model without cross-talk. Hardware-specific limits are measured and advertised through capability admission.
|
||||
|
||||
## Parallelism boundaries
|
||||
|
||||
| Mechanism | First-runtime use |
|
||||
|---|---|
|
||||
| Layer/pipeline parallelism | Public Inference Route across contiguous Shards |
|
||||
| Continuous batching | Inside every node across active Route Sessions |
|
||||
| Data parallelism | Multiple complete routes for independent requests |
|
||||
| Tensor parallelism | Deferred to a trusted composite node/managed cluster |
|
||||
| Expert parallelism | Deferred to a trusted composite node/managed cluster |
|
||||
| Disaggregated prefill | Deferred until core route performance passes |
|
||||
| Speculative decoding | Deferred optimization |
|
||||
|
||||
Public WAN tensor/expert collectives are rejected for the first runtime because their per-layer communication and static rank assumptions conflict with heterogeneous volunteer nodes.
|
||||
|
||||
## Optional providers
|
||||
|
||||
### Transformers/safetensors
|
||||
|
||||
Remains:
|
||||
|
||||
- Correctness/reference backend.
|
||||
- Fallback for unsupported architectures.
|
||||
- Baseline for performance and output quality.
|
||||
|
||||
### vLLM
|
||||
|
||||
May run unmodified as a complete model or managed TP/PP/EP cluster represented as one logical provider. Its internal ranks are not independently routed or rewarded.
|
||||
|
||||
Borrow only concepts such as named bundles, continuous batching, typed compatibility fingerprints, explicit transfer lifecycle and load telemetry.
|
||||
|
||||
### Whole-model llama.cpp
|
||||
|
||||
Provides a local proxy backend, correctness oracle and performance baseline. It is not the native distributed milestone.
|
||||
|
||||
## Artifact and recipe compatibility
|
||||
|
||||
A routable recipe identifies separately:
|
||||
|
||||
- Source Model Artifact hash and optional derivative/slice hash.
|
||||
- Architecture and adapter version.
|
||||
- Tokenizer revision and vocabulary.
|
||||
- Weight quantization.
|
||||
- Activation interchange dtype/schema.
|
||||
- Backend compute dtype and backend implementation.
|
||||
- KV dtype/layout.
|
||||
- RoPE/context parameters.
|
||||
- llama.cpp commit and project patch version.
|
||||
- Shard range and endpoint ownership.
|
||||
|
||||
Compatibility fails closed. Similar quantization labels or model names are not enough.
|
||||
|
||||
## Admission and failure
|
||||
|
||||
A recipe becomes routable only after a real local and distributed forward passes. Synthetic tests remain unit coverage.
|
||||
|
||||
Alpha failure behavior:
|
||||
|
||||
- Deadline or node loss cancels the Route Session.
|
||||
- Every node releases KV and queued buffers.
|
||||
- Uncertain mutations are not replayed silently.
|
||||
- Retry starts from token zero on a newly compatible route.
|
||||
- No cross-node KV import is trusted until a later signed/compatible snapshot protocol exists.
|
||||
|
||||
## Performance release contract
|
||||
|
||||
Before native development proceeds, compare the current Transformers/safetensors backend with whole-model llama.cpp under controlled model/hardware/quality lanes.
|
||||
|
||||
Final release compares distributed GGUF with distributed safetensors using thresholds locked before seeing final results.
|
||||
|
||||
Required measurements:
|
||||
|
||||
- TTFT.
|
||||
- Prefill and decode tokens/sec.
|
||||
- Aggregate concurrency throughput.
|
||||
- p50/p95 latency.
|
||||
- Seam bytes and latency.
|
||||
- Queue/batch occupancy.
|
||||
- RSS, VRAM and KV pressure.
|
||||
- Output-quality drift.
|
||||
- Cancellation/failure cleanup.
|
||||
|
||||
The GGUF path ships only if it is faster at acceptable quality or enables a larger otherwise-unroutable model at useful measured speed.
|
||||
|
||||
## Implementation sequence
|
||||
|
||||
1. Lock benchmark/performance contract.
|
||||
2. Define gRPC/protobuf and exact recipe identity.
|
||||
3. Pin llama.cpp and create the minimal patch stack.
|
||||
4. Implement dense-Llama range loading and boundary parity.
|
||||
5. Implement concurrent local KV.
|
||||
6. Build and integrate the standalone worker.
|
||||
7. Pass local two-process real-model acceptance.
|
||||
8. Pass real heterogeneous two-machine acceptance.
|
||||
9. Add continuous batching and failure hardening.
|
||||
10. Enforce the GGUF-versus-safetensors release gate.
|
||||
11. Add Qwen3/Qwen3-MoE as a separately certified adapter.
|
||||
12. Prepare narrow upstream collaboration patches/tests.
|
||||
|
||||
See [the Ralph backlog](prd.json) and [implementation strategy](implementation-strategy.md).
|
||||
|
||||
@@ -1,40 +1,270 @@
|
||||
# Distributed GGUF Runtime decision framework
|
||||
# Distributed GGUF Decision Framework
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
|
||||
|
||||
## Decision order
|
||||
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.
|
||||
|
||||
1. DGR-019 locks comparable lanes and thresholds before results.
|
||||
2. DGR-020 runs safetensors and whole-model llama.cpp only, then returns `go`, `optimize baseline`, or `stop`.
|
||||
3. Dense and V4 work must prove parity, independent per-stage execution, local-state isolation, bounded failure, and measured resources.
|
||||
4. DGR-054 returns `alpha`, `optimize measured bottleneck`, or `stop`; MTP is explicitly off.
|
||||
5. Post-alpha optimizations must be selected from profiles, not assumptions.
|
||||
6. DGR-070 returns `beta`, `targeted optimization`, or `stop/rollback`, and requires MTP and the exact certified hardware/recipe matrix.
|
||||
## Core Vocabulary
|
||||
|
||||
## Interpretation rules
|
||||
Use the existing domain terms this way:
|
||||
|
||||
- Quant/model-fit gains are separate from runtime/kernel/transport gains.
|
||||
- Fixture, real-model, real-hardware, and release evidence are never interchangeable.
|
||||
- 2–4 and 10+ stages are certification scenarios only.
|
||||
- Existing routing policy is certified, not redesigned.
|
||||
- Build success is not hardware certification; dark lanes remain unroutable.
|
||||
- Route loss uses cache miss and re-prefill/restart, never WAN cache migration.
|
||||
- **Shard**: contiguous transformer layer range. This is the compute, routing, cache, and reward unit.
|
||||
- **Shard Swarm**: storage/download group for artifacts needed by a shard.
|
||||
- **Inference Route**: ordered node sequence that covers all layers for one request.
|
||||
- **Route Session**: one active request bound to one inference route and stable session id.
|
||||
- **Hot KV State**: live per-shard cache held by the route node during a route session.
|
||||
- **Prefix Snapshot**: persisted route-session state used for reuse or failover, not the hot decode path.
|
||||
- **Artifact Manifest**: canonical mapping from model artifacts to semantic model parts and runtime support.
|
||||
- **Generation Telemetry**: realtime progress for a route session, including phase and tokens/sec, independent of whether token deltas are streamed.
|
||||
|
||||
## Locked scope
|
||||
## The Five Planes
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
### 1. Control Plane
|
||||
|
||||
## Target identities
|
||||
Owner: Tracker.
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
Responsibilities:
|
||||
|
||||
- node registry
|
||||
- coverage map
|
||||
- route selection
|
||||
- rebalance directives
|
||||
- route-session creation
|
||||
- health and telemetry
|
||||
- client-visible Generation Telemetry
|
||||
- billing/audit records
|
||||
|
||||
Must not do:
|
||||
|
||||
- serve hot KV during every token
|
||||
- become the only place model artifacts can be fetched
|
||||
|
||||
### 2. Artifact Plane
|
||||
|
||||
Owner: Shard Swarms, local node storage, optional CDN/bootstrap mirrors.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- GGUF/safetensors/tokenizer download
|
||||
- content-addressed verification
|
||||
- local artifact inventory
|
||||
- artifact-to-layer mapping
|
||||
- cache eviction
|
||||
|
||||
Must not do:
|
||||
|
||||
- define execution order by file split alone
|
||||
- imply that a downloaded file chunk equals a Shard
|
||||
|
||||
### 3. Execution Plane
|
||||
|
||||
Owner: active Inference Route.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- chunked prefill
|
||||
- one-step decode
|
||||
- hidden-state transfer across activation seams
|
||||
- start-layer handling for overlapping shards
|
||||
- backpressure
|
||||
|
||||
Must not do:
|
||||
|
||||
- resend full context activations during decode
|
||||
- require cross-node tensor parallel all-reduce for public v1
|
||||
|
||||
### 4. Session State Plane
|
||||
|
||||
Owner: route nodes for hot KV; cache servers only for snapshots.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- per-shard local KV ownership
|
||||
- cache allocation and eviction
|
||||
- cache ABI compatibility
|
||||
- session close/release
|
||||
- optional prefix snapshots
|
||||
|
||||
Must not do:
|
||||
|
||||
- centralize hot KV in a remote service
|
||||
- let a replacement node continue from incompatible state
|
||||
|
||||
### 5. Economics And Trust Plane
|
||||
|
||||
Owner: tracker plus settlement/validation components.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- distinguish storage/seeding work from inference work
|
||||
- account for prefill and decode separately
|
||||
- record route participation
|
||||
- sample validation events
|
||||
- slash proven fraud
|
||||
|
||||
Must not do:
|
||||
|
||||
- pay a node for merely holding files as if it generated tokens
|
||||
- hide public-swarm privacy limits from clients
|
||||
|
||||
## Hard Invariants
|
||||
|
||||
These are the framework rules unless we deliberately write a new ADR:
|
||||
|
||||
1. Public-network Shards are contiguous layer ranges.
|
||||
2. Hot KV State is local to the node serving that Shard in that Route Session.
|
||||
3. Artifact distribution and route execution are separate systems.
|
||||
4. Decode seam payload must be `O(hidden_size)`.
|
||||
5. Prefill may be `O(sequence_length * hidden_size)`, but only in bounded chunks.
|
||||
6. The tracker chooses routes; nodes do not negotiate route topology peer-to-peer.
|
||||
7. Model/backend-specific cache internals stay behind backend capability reports.
|
||||
8. PyTorch remains the correctness/reference backend while llama.cpp/GGUF becomes the performance backend.
|
||||
9. Streaming responses are preferred when feasible; Generation Telemetry is always required.
|
||||
|
||||
## Resolved Gates
|
||||
|
||||
### Gate 1: Public Shard Semantics
|
||||
|
||||
Decision: public-network Shards are contiguous transformer layer ranges. Tensor-parallel or ring-style execution is allowed only inside one trusted node, one colocated pod, or a future composite node abstraction.
|
||||
|
||||
Rationale:
|
||||
|
||||
- Layer ranges match the existing `Shard`, `Coverage Map`, `Inference Route`, billing, and fraud vocabulary.
|
||||
- Public volunteer nodes should not require cross-node all-reduce or tight per-layer synchronization in v1.
|
||||
- Existing projects such as prima.cpp and Distributed Llama can still inform local-cluster/backend execution without becoming the public routing primitive.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Artifact Manifests must map files/tensors to semantic layer ranges.
|
||||
- Route selection remains ordered layer coverage.
|
||||
- Rewards can be attributed to layer-range work.
|
||||
- Hot KV State is naturally owned by the node serving that layer range for the Route Session.
|
||||
|
||||
### Gate 2: Hot KV Strategy
|
||||
|
||||
Decision: v1 rejects centralized hot KV. Hot KV State is local to the node serving the relevant Shard in the active Route Session. Cache servers may store Prefix Snapshots for reuse, retry, or failover, but they are not in the per-token decode path.
|
||||
|
||||
Rationale:
|
||||
|
||||
- Decode is the tight loop; adding remote cache I/O there makes latency and bandwidth worse at the worst point.
|
||||
- Local KV naturally follows layer-range Shard ownership.
|
||||
- Centralized hot KV increases privacy exposure and creates consistency problems.
|
||||
- Prefix Snapshots preserve the useful part of central storage without making it mandatory for every generated token.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Route Session must be sticky.
|
||||
- Failover is limited in alpha unless a compatible Prefix Snapshot exists.
|
||||
- Cache servers are optimization infrastructure, not required runtime infrastructure.
|
||||
- Route repair requires compatible model revision, layer range, backend cache ABI, and snapshot position.
|
||||
|
||||
### Gate 3: First Runtime Proof
|
||||
|
||||
Decision: prove distributed Route Session and Hot KV State semantics in the existing PyTorch route before modifying llama.cpp/GGUF.
|
||||
|
||||
Rationale:
|
||||
|
||||
- PyTorch exposes model internals and cache objects more directly, so it is the fastest way to validate the distributed protocol.
|
||||
- The current distributed PyTorch route already has the right high-level shape but disables cache and recomputes full prompts.
|
||||
- Fixing that path gives us a reference implementation for correctness tests, telemetry, session lifecycle, and wire protocol behavior.
|
||||
- llama.cpp/GGUF should receive a clear target ABI rather than becoming both the protocol experiment and the performance backend at once.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Issue 02 precedes issue 05.
|
||||
- llama.cpp collaboration has a concrete target ABI.
|
||||
- The PyTorch route remains the architecture-coverage/reference backend even after GGUF becomes the preferred performance path.
|
||||
- The first success metric is eliminating full-prompt recompute in distributed decode.
|
||||
|
||||
### Gate 3A: Client Feedback During Latency
|
||||
|
||||
Decision: streaming responses are preferred when feasible, and realtime Generation Telemetry is required regardless of streaming support.
|
||||
|
||||
Rationale:
|
||||
|
||||
- The product optimizes for access to large capable models, so some latency is acceptable.
|
||||
- Users still need confidence that the route is alive and roughly how fast it is generating.
|
||||
- Streaming token deltas give the best user experience when the backend exposes them cleanly.
|
||||
- Tokens/sec remains useful during prefill, queueing, and any backend that cannot stream token deltas.
|
||||
|
||||
Consequences:
|
||||
|
||||
- The gateway should stream token deltas through an OpenAI-compatible response when possible.
|
||||
- The gateway must expose progress through SSE, WebSocket, or polling.
|
||||
- The final answer can be delivered after completion only as a fallback.
|
||||
- Telemetry must include route phase, generated token count, and rolling tokens/sec.
|
||||
- Non-streaming clients still need realtime telemetry.
|
||||
|
||||
### Gate 4: llama.cpp Collaboration Shape
|
||||
|
||||
Decision: target upstreamable `libllama`/ggml hooks instead of planning around a permanent fork.
|
||||
|
||||
Rationale:
|
||||
|
||||
- llama.cpp changes quickly across model support, quantization, kernels, and hardware backends.
|
||||
- A permanent fork would become expensive to maintain and would lag upstream improvements.
|
||||
- A short-lived prototype branch is acceptable if it proves the API and makes upstream collaboration concrete.
|
||||
- Keeping tracker/routing logic outside llama.cpp makes the upstream ask smaller and cleaner.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Need a minimal reproducible localhost demo before asking upstream to carry the design.
|
||||
- Need to separate "what llama.cpp should expose" from "what our tracker does".
|
||||
- Desired upstream surface is layer-range execution, hidden-state boundary I/O, partial loading/introspection, and per-session KV ownership.
|
||||
- If upstream rejects the shape, we revisit whether to carry a narrow adapter fork or keep GGUF distributed execution as experimental.
|
||||
|
||||
### Gate 5: First Model Target
|
||||
|
||||
Decision: use a two-tier model target. Use a small, boring, llama.cpp-supported GGUF model for the first protocol smoke test. Use `deepseek-ai/DeepSeek-V4-Flash` as the first serious large-model target. Keep GLM-5.2 and Ornith as later support audits.
|
||||
|
||||
Rationale:
|
||||
|
||||
- The first protocol proof should isolate route/session/KV bugs from model-architecture bugs.
|
||||
- DeepSeek-V4-Flash is a strong first serious target because it is much smaller than 1.6T-class models while still being large enough to validate the product thesis.
|
||||
- DeepSeek-V4-Flash still has architecture-specific risks, so it should not be the first smoke test.
|
||||
- GLM-5.2 and Ornith remain valuable targets, but they add DSA/MLA/hybrid attention uncertainty.
|
||||
|
||||
Consequences:
|
||||
|
||||
- 128K cache accounting can be modeled now.
|
||||
- The first "real" target-model audit is DeepSeek-V4-Flash support in PyTorch, vLLM/SGLang, and any available GGUF/llama.cpp quantization path.
|
||||
- Production support waits for backend capability reports and exact cache ABI support.
|
||||
|
||||
### Gate 6: Failure Semantics
|
||||
|
||||
Decision: alpha fails Route Sessions on route-node loss instead of attempting automatic route repair.
|
||||
|
||||
Rationale:
|
||||
|
||||
- Route repair requires compatible Prefix Snapshots, cache ABI checks, replacement-node selection, billing correction, and client stream/error recovery.
|
||||
- Local Hot KV State means a replacement node cannot continue unless it has compatible state at the same position.
|
||||
- Fail-fast keeps the first implementation correct while the session/KV protocol is still being proven.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Better observability and explicit errors are required.
|
||||
- Snapshotting becomes a later feature, not a blocker for first inference.
|
||||
- Generation Telemetry must report the last known phase and failure reason.
|
||||
- Client or gateway retry starts a new Route Session from scratch.
|
||||
|
||||
### Gate 7: Transport
|
||||
|
||||
Decision: keep binary HTTP for v1 activation transfer instead of jumping immediately to QUIC, WebRTC, or a custom transport.
|
||||
|
||||
Rationale:
|
||||
|
||||
- ADR-0008 already defines binary activation bodies with HTTP headers.
|
||||
- HTTP keeps the first implementation debuggable with the existing server stack and tooling.
|
||||
- The core risk is route/session/KV correctness, not transport optimization.
|
||||
- QUIC/WebRTC can be introduced later behind the same activation protocol once semantics are proven.
|
||||
|
||||
Consequences:
|
||||
|
||||
- Focus benchmark work on payload shape, chunking, and cache behavior first.
|
||||
- QUIC/WebRTC can be introduced as an optimization behind the same activation protocol.
|
||||
- v1 implementation can reuse the current HTTP routing, relay, and observability infrastructure.
|
||||
- Transport abstraction should be kept narrow enough that HTTP can be replaced later without changing backend cache semantics.
|
||||
|
||||
## Grilling Progress
|
||||
|
||||
Gates 1, 2, 3, 3A, 4, 5, 6, and 7 are resolved. The remaining work is to convert the resolved framework into implementation-ready issue briefs and prototype milestones.
|
||||
|
||||
@@ -1,114 +0,0 @@
|
||||
# DGR-017 evidence — superseded backlog cleanup
|
||||
|
||||
**Completed:** 2026-07-16
|
||||
**Branch:** `ralph/distributed-gguf-runtime`
|
||||
**Planning checkpoint before cleanup:** `81b1fa6`
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
|
||||
## Outcome
|
||||
|
||||
The old DGR-001…016 completion claims and active artifacts were reconciled against the live branch. No old pass state transferred to the new implementation roadmap.
|
||||
|
||||
The active `packages/` and `tests/` trees were restored exactly to `origin/master`. The branch therefore no longer exposes a nominal GGUF startup path backed by unimplemented transport methods, a protobuf-only native scaffold, or isolated synthetic scheduler/cache/failure modules as if they were a working distributed GGUF runtime.
|
||||
|
||||
## Classification and disposition
|
||||
|
||||
### Retained
|
||||
|
||||
- Accepted ADRs and repository research, including `docs/research/colibri-implementation-audit.md`.
|
||||
- The authoritative 55-story roadmap `DGR-017…071` and its generated issue specifications.
|
||||
- The real public-relay smoke benchmark, moved with provenance to `legacy-public-relay-smoke-benchmark.json`.
|
||||
- Git history containing the complete superseded implementation/reference work.
|
||||
|
||||
### Removed from the active tree
|
||||
|
||||
- Legacy issue specifications DGR-001…016 and their stale/blocked/synthetic evidence directories.
|
||||
- The nonfunctional `gguf_backend` startup path whose gRPC execution methods raised not-implemented errors.
|
||||
- Synthetic/reference-only boundary, Hot KV, scheduler, failure, recipe, ownership, and native-protocol modules that were not a real llama.cpp Shard runtime.
|
||||
- The protobuf round-trip-only native scaffold, placeholder llama.cpp patch, generated bindings/build workspace, and associated tests.
|
||||
- Tracker/admission/source modifications coupled to that superseded scaffold.
|
||||
|
||||
### Confirmed absent and still required
|
||||
|
||||
- Real standalone C++ gRPC Shard worker.
|
||||
- Exact pinned llama.cpp manifest and verified patch stack.
|
||||
- Range-aware GGUF tensor ownership and real ranged execution.
|
||||
- Real Shard-local llama.cpp KV/V4 auxiliary state.
|
||||
- DeepSeek V4 boundary adapter and ranged parity.
|
||||
- Real multi-machine DeepSeek V4 alpha or beta acceptance.
|
||||
|
||||
These remain `passes: false` in DGR-018…071.
|
||||
|
||||
## Before-cleanup baseline
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
.venv-rocm/bin/python -m pytest -q \
|
||||
tests/test_performance_contract.py tests/test_native_shard_protocol.py \
|
||||
tests/test_gguf_ownership.py tests/test_boundary_adapter.py \
|
||||
tests/test_hot_kv_state.py tests/test_gguf_backend.py \
|
||||
tests/test_batch_scheduler.py tests/test_failure_semantics.py \
|
||||
tests/test_llama_worker_build.py tests/test_node_admission.py \
|
||||
tests/test_node_capability.py tests/test_tracker_capability_admission.py
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
216 passed, 2 skipped, 1 failed, 1 warning
|
||||
```
|
||||
|
||||
The failure was a synthetic capability-test helper `KeyError: 'compatibility_fingerprint'`. The warning was a pre-existing heartbeat-thread `SystemExit` warning.
|
||||
|
||||
## Cleanup verification
|
||||
|
||||
### Source equality
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
git diff --quiet origin/master -- packages tests
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
packages_tests_match_origin_master=yes
|
||||
```
|
||||
|
||||
The staged cleanup removes approximately 15.3k obsolete source/test/evidence lines from the active branch.
|
||||
|
||||
### Cleanup-relevant regression suite
|
||||
|
||||
Command:
|
||||
|
||||
```bash
|
||||
.venv-rocm/bin/python -m pytest -q \
|
||||
tests/test_node_admission.py tests/test_node_capability.py \
|
||||
tests/test_tracker_capability_admission.py \
|
||||
tests/test_kv_cache_distributed.py tests/test_real_distributed_inference.py
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
119 passed, 2 skipped, 1 warning in 15.90s
|
||||
```
|
||||
|
||||
The warning is the same pre-existing heartbeat-thread `SystemExit` warning.
|
||||
|
||||
### Known `origin/master` limitations
|
||||
|
||||
The wider routing run produced `210 passed, 2 skipped, 4 failed, 1 warning`. Each failure reproduced individually while `packages/` and `tests/` matched `origin/master` exactly:
|
||||
|
||||
- `test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`
|
||||
- `test_torch_node_applies_tracker_load_shard_directive`
|
||||
- `test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`
|
||||
- `test_a_node_with_an_unusable_precision_covers_no_layers`
|
||||
|
||||
They are recorded as pre-existing baseline defects and were not repaired or hidden by this cleanup story.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
DGR-018 and later stories must start from the cleaned upstream-equivalent runtime tree. Reuse concepts from superseded commits only by explicitly porting the smallest verified slice under the new story’s contracts, tests, and evidence gates. Git history is provenance, not completion evidence.
|
||||
@@ -1,83 +0,0 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"executed_at_utc": "2026-07-15T10:41:14Z",
|
||||
"test_kind": "public-relay-single-node-streaming-smoke-benchmark",
|
||||
"target": {
|
||||
"public_chat_endpoint": "https://meshnet.2.d-popov.com/v1/chat/completions",
|
||||
"relay_url": "wss://meshnet.2.d-popov.com/ws",
|
||||
"model": "qwen2.5-0.5b-instruct",
|
||||
"quantization": "bfloat16"
|
||||
},
|
||||
"recovery": {
|
||||
"problem": "The local node's capability proof had expired and its port-7000 HTTP server had wedged with CLOSE-WAIT sockets.",
|
||||
"action": "Gracefully restarted the local public-tracker meshnet-node process on port 7000.",
|
||||
"startup_validation": {
|
||||
"device": "cuda",
|
||||
"capability_proof_ms": 336,
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"relay_addr": "wss://meshnet.2.d-popov.com/rpc/7j77FsPY1evV8tuf-7000"
|
||||
}
|
||||
},
|
||||
"tracker_admission_after_recovery": {
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"alive": true,
|
||||
"status": "ready",
|
||||
"capability_state": "admitted",
|
||||
"routable": true,
|
||||
"route_hops": 1
|
||||
},
|
||||
"client_measurements": {
|
||||
"warmup": {
|
||||
"http_status": 200,
|
||||
"ttft_ms": 420.8,
|
||||
"elapsed_ms": 610.23,
|
||||
"response_text": "MeshNet Relay Benchmark Passed"
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"run": 1,
|
||||
"ttft_ms": 376.04,
|
||||
"elapsed_ms": 458.65,
|
||||
"response_text": "relay benchmark pass"
|
||||
},
|
||||
{
|
||||
"run": 2,
|
||||
"ttft_ms": 258.33,
|
||||
"elapsed_ms": 336.71,
|
||||
"response_text": "relay benchmark pass"
|
||||
},
|
||||
{
|
||||
"run": 3,
|
||||
"ttft_ms": 288.26,
|
||||
"elapsed_ms": 363.2,
|
||||
"response_text": "relay benchmark pass"
|
||||
}
|
||||
],
|
||||
"p50_ttft_ms": 288.26,
|
||||
"p50_elapsed_ms": 363.2
|
||||
},
|
||||
"tracker_relay_evidence": [
|
||||
{
|
||||
"status": 200,
|
||||
"relay": true,
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"tokens": 11,
|
||||
"elapsed_seconds": 0.1686,
|
||||
"tokens_per_sec": 65.2541
|
||||
},
|
||||
{
|
||||
"status": 200,
|
||||
"relay": true,
|
||||
"node_id": "7j77FsPY-b32476219492",
|
||||
"tokens": 11,
|
||||
"elapsed_seconds": 0.1891,
|
||||
"tokens_per_sec": 58.1799
|
||||
}
|
||||
],
|
||||
"scope_and_remaining_work": {
|
||||
"validated": "Public HTTPS chat endpoint routed a streaming request through the tracker relay to the local CUDA node and completed with HTTP 200.",
|
||||
"not_validated": "Two-node shard routing was not run because the remote node 5gMLrmyB-88f5cba044d0 still had an expired capability proof and was not routable.",
|
||||
"next_gate": "Refresh the remote node capability proof, then load a multi-node-compatible assignment and repeat the benchmark through the public tracker relay."
|
||||
},
|
||||
"reproduction": "Use a valid bearer API key with the public /v1/chat/completions endpoint and stream a short qwen2.5-0.5b-instruct request. Do not connect directly to private node HTTP endpoints; the tracker relay is the required path."
|
||||
}
|
||||
@@ -1,204 +0,0 @@
|
||||
# DGR-018 evidence — canonical Ralph and Gitea metadata schema
|
||||
|
||||
**Completed:** 2026-07-16
|
||||
**Branch:** `ralph/distributed-gguf-runtime`
|
||||
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||
**Dependency:** DGR-017 (`evidence/DGR-017/README.md`) — cleaned backlog reconciled to `origin/master`; no old pass state transferred.
|
||||
|
||||
## Objective
|
||||
|
||||
Make `prd.json` the validated source from which Markdown (and, later, Gitea) issues
|
||||
can be generated losslessly, per
|
||||
`.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`.
|
||||
|
||||
## Pre-existing state found (not caused by this story)
|
||||
|
||||
Before any change in this session, `git status` showed `.scratch/distributed-gguf-runtime/prd.json`
|
||||
already modified in the working tree relative to `HEAD` (commit `369b207`), with no corresponding
|
||||
progress-log entry. Diffing against `HEAD` showed the working copy had **dropped** prd.json's
|
||||
top-level `sourceOfTruth`, `qualityGates`, `metadataSchema`, `milestones`, and `supersededStories`
|
||||
objects, while `userStories` itself was byte-identical to `HEAD`. This looked like an abandoned,
|
||||
uncommitted partial edit from a prior session, not intentional current work — those fields are
|
||||
exactly the schema/quality-gate/audit-provenance content this story depends on, and their loss
|
||||
wasn't explained by any acceptance criterion. They were restored (see "Changes" below) rather than
|
||||
silently accepted or discarded, per the instruction to investigate unexplained working-tree state
|
||||
before building on top of it.
|
||||
|
||||
## Changes
|
||||
|
||||
### `scripts/ralph_prd_schema.py` (new)
|
||||
|
||||
Single module providing:
|
||||
|
||||
- **Parse:** `load_prd(path)` — JSON load with clear `PrdValidationError`s for missing file /
|
||||
invalid JSON / non-object document.
|
||||
- **Canonical schema registry:** `STORY_FIELDS` (name → required/type), `EXECUTION_MODES`,
|
||||
`EVIDENCE_CLASSES`, `HARDWARE_FLAGS`, `UPSTREAM_FLAGS`, `TRIAGE_VALUES`. Covers every field named
|
||||
in the acceptance criteria: stable `id`/`title`, `labels`, `milestone`, derived `type`
|
||||
(`derive_type`), `dependsOn`, derived `blocks`, `triage`, `evidenceClass`, and
|
||||
`hardware`/`model`/`upstream` flags.
|
||||
- **Structural validation:** `validate_schema(data)` — required fields, types, enum membership,
|
||||
ID convention, `type:`/`priority:` label cardinality, non-empty `acceptanceCriteria`.
|
||||
- **Semantic validation:** `validate_semantics(data)` — unique IDs, unique titles, `dependsOn`
|
||||
resolves to known stories (no self-dependency), dependency graph is acyclic (with a reported
|
||||
cycle path on failure), `blocks` matches the dependency graph exactly (sorted set equality, not
|
||||
superset), and `evidencePath` matches the per-story convention.
|
||||
- **Fresh vs. in-progress backlog:** `validate_fresh_backlog(data)` additionally requires every
|
||||
story to start `passes: false` (for a backlog that hasn't started execution yet);
|
||||
`validate_backlog(data)` is the composed check for a real, in-flight backlog where some stories
|
||||
have legitimately completed.
|
||||
- **Self-consistency check:** `validate_metadata_schema_consistency(data)` — when prd.json declares
|
||||
its own `metadataSchema`/`qualityGates` (as this one now does), verifies that self-documentation
|
||||
hasn't drifted from what the validator actually enforces (enum sets, required/optional field
|
||||
lists, presence of `qualityGates` and `generatedArtifactDisclaimer`). This is a no-op for minimal
|
||||
fixture PRDs that don't carry that documentation.
|
||||
- **Generation (one-directional, prd.json → artifact):** `render_issue_markdown(story, data)`
|
||||
renders the exact Markdown convention already used by
|
||||
`.scratch/distributed-gguf-runtime/issues/*.md`, sourcing the "Shared quality gates" bullets from
|
||||
`data["qualityGates"]` and the leading disclaimer from
|
||||
`data["metadataSchema"]["generatedArtifactDisclaimer"]` (falling back to a module default only
|
||||
when `data` omits them) — not from a duplicated Python string literal.
|
||||
`to_gitea_issue_payload(story, data)` wraps the same body into a Gitea create-issue-shaped payload
|
||||
(`title`, `body`, `labels`, `milestone`).
|
||||
- **Authority guard:** `check_generated_markdown_authority(text, disclaimer=...)` rejects generated
|
||||
Markdown that's missing the disclaimer or that contains a conflicting authority claim (e.g. "this
|
||||
file is authoritative"). There is deliberately no Markdown → prd.json parser, so a generated
|
||||
artifact structurally cannot feed `passes` (or anything else) back into the authoritative source.
|
||||
- CLI: `python scripts/ralph_prd_schema.py validate <prd.json> [--fresh]` and
|
||||
`... render <prd.json> <STORY-ID>`.
|
||||
|
||||
### `.scratch/distributed-gguf-runtime/prd.json`
|
||||
|
||||
- Restored the top-level `sourceOfTruth`, `qualityGates`, `milestones`, and `supersededStories`
|
||||
objects to their `HEAD` content (see "Pre-existing state" above); `userStories` was already
|
||||
identical to `HEAD` and is unchanged in content.
|
||||
- Extended `metadataSchema` (previously incomplete for this story's own acceptance criteria) with:
|
||||
`requiredStoryFields` now also lists `notes` and `blocks` (present on all 55 stories); new
|
||||
`optionalStoryFields: ["completionNotes"]`; new `hardwareValues`/`upstreamValues` enums (`model`
|
||||
is documented as an open convention, not a closed enum, since quantization/model targets are
|
||||
dynamic recipe inputs per `RALPH-CONTEXT.md`); new `typeDerivation` and `labelConventions`
|
||||
(reserved prefixes, cardinality); new `generatedArtifactDisclaimer` (the exact string generated
|
||||
artifacts must start with); extended `dependencyRules`/`authorityRule` prose to match what the
|
||||
validator enforces.
|
||||
- Reworded `sourceOfTruth`'s stale "All stories are unimplemented ... passes=false" clause, which
|
||||
was no longer accurate once DGR-017 completed.
|
||||
- Marked `DGR-018.passes = true` with `completionNotes` recording this story's outcome.
|
||||
|
||||
### `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`
|
||||
|
||||
Regenerated via `render_issue_markdown` to reflect `passes: true` (checked acceptance criteria,
|
||||
"completed" status line, "Verified evidence" handoff line) — matching the same convention DGR-017's
|
||||
issue file already used for a completed story.
|
||||
|
||||
### `tests/test_ralph_prd_schema.py` (new)
|
||||
|
||||
108 deterministic, model-download-free, GPU-free tests:
|
||||
|
||||
- **Parse** (4 tests): real backlog parses to 55 stories; missing file, invalid JSON, and
|
||||
non-object documents raise `PrdValidationError`.
|
||||
- **Structural/semantic validation against the real backlog** (7 tests): passes `validate_schema`,
|
||||
`validate_semantics`, and the composed `validate_backlog`; unique IDs/titles; all `dependsOn`
|
||||
resolve; `blocks` matches the derived dependency graph for all 55 stories; no cycle; every
|
||||
`passes: true` story carries `completionNotes` and an existing evidence README (a durable
|
||||
invariant, not a hardcoded list of which stories have completed — that list will keep growing).
|
||||
- **Structural/semantic failure-mode fixtures** (13 tests): missing required field, bad enum, wrong
|
||||
type, empty `acceptanceCriteria`, multiple `type:` labels, duplicate ID, duplicate title, unknown
|
||||
dependency, self-dependency, dependency cycle, mismatched `blocks`, bad `evidencePath`.
|
||||
- **Fresh-backlog invariant** (3 tests): accepts all-`false`, rejects a premature `passes: true`,
|
||||
and confirms `validate_backlog` (the in-progress variant) permits completed stories.
|
||||
- **prd.json-as-source-of-truth for boilerplate** (9 tests): `qualityGates`/`metadataSchema`
|
||||
self-consistency checks (no-op without them, catches a drifted enum, catches a missing
|
||||
`qualityGates`), `quality_gate_bullets` flattening order, `authority_disclaimer` precedence and
|
||||
fallback, and 3 tests asserting the real backlog's declared schema matches the code, its 7
|
||||
quality-gate bullets are intact, and its disclaimer matches the module default.
|
||||
- **`derive_type`** (4 tests): label-derived type, release-gate synthetic type for HITL gate
|
||||
stories, `None` when absent, and confirmation that the real backlog's two release-gate stories
|
||||
(`DGR-054`, `DGR-070`) derive `release-gate`.
|
||||
- **Markdown generation round trips** (55 parametrized + 6 tests): `render_issue_markdown` for
|
||||
every story `DGR-017`..`DGR-071` is byte-for-byte identical to the corresponding file already in
|
||||
`.scratch/distributed-gguf-runtime/issues/`; determinism; leading disclaimer; `Blocks (derived)`
|
||||
rendering (`None` vs. listed); checkbox reflects `passes`; filename convention.
|
||||
- **Authority-claim rejection** (4 tests): accepts real generated text, rejects a missing
|
||||
disclaimer, rejects an overriding claim, and confirms every committed issue file in
|
||||
`.scratch/distributed-gguf-runtime/issues/` passes the check.
|
||||
- **Gitea payload generation** (3 tests): payload shape, body carries no information beyond what's
|
||||
in prd.json, and every real story's payload is well-formed and authority-clean.
|
||||
|
||||
## Commands and results
|
||||
|
||||
```bash
|
||||
python3 -m pytest -q tests/test_ralph_prd_schema.py
|
||||
```
|
||||
```text
|
||||
108 passed in 0.16s
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 -m compileall -q packages tests
|
||||
```
|
||||
Exit code 0, no output (all files compile).
|
||||
|
||||
```bash
|
||||
git diff --check
|
||||
```
|
||||
Exit code 0 (no whitespace errors).
|
||||
|
||||
```bash
|
||||
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||
```
|
||||
```text
|
||||
OK: 55 stories validated.
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json --fresh
|
||||
```
|
||||
```text
|
||||
ERROR: DGR-017: fresh backlog requires passes=false, got True
|
||||
ERROR: DGR-018: fresh backlog requires passes=false, got True
|
||||
2 validation error(s).
|
||||
```
|
||||
Expected: `--fresh` is the invariant for a backlog that hasn't started execution; this backlog has
|
||||
legitimately completed two stories, so it correctly fails that stricter check while passing the
|
||||
plain (in-progress) `validate` command above.
|
||||
|
||||
### Baseline: full repository suite (ad hoc `python3`, not a project venv)
|
||||
|
||||
```bash
|
||||
python3 -m pytest -q
|
||||
```
|
||||
```text
|
||||
20 failed, 776 passed, 13 skipped, 2 warnings in 244.17s (0:04:04)
|
||||
```
|
||||
None of the failures touch `scripts/ralph_prd_schema.py` or `tests/test_ralph_prd_schema.py`
|
||||
(neither file existed before this story; this story adds no changes to `packages/`). Four of the
|
||||
20 failures reproduce exactly the pre-existing baseline defects DGR-017's evidence already recorded
|
||||
(`test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`,
|
||||
`test_torch_node_applies_tracker_load_shard_directive`,
|
||||
`test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`,
|
||||
`test_a_node_with_an_unusable_precision_covers_no_layers`). The remaining 16 (activation
|
||||
compression, dynamic routing, gossip/relay, manual route benchmark, openai gateway, TOPLoC
|
||||
calibration dispatch, tracker control plane) include a `ModuleNotFoundError: langchain` failure,
|
||||
indicating this ad hoc `python3` lacks the project's `dev` extras (`langchain-openai`, etc.) rather
|
||||
than a real regression; this environment has no project virtualenv (e.g. no `.venv-rocm`) to run
|
||||
against instead. Not investigated further as out of scope for this story.
|
||||
|
||||
## Limitations
|
||||
|
||||
- No real Gitea instance or API integration exists; `to_gitea_issue_payload` defines the payload
|
||||
shape (title/body/labels/milestone) only. Creating issues against a live Gitea server is future
|
||||
work, not claimed here.
|
||||
- `model` is intentionally validated as an open string, not a closed enum, per
|
||||
`RALPH-CONTEXT.md`'s "Quantization and placement are dynamic recipe inputs" constraint; the schema
|
||||
documents (`metadataSchema.modelConvention`) but does not restrict its value set.
|
||||
- Validation and generation were exercised only against this feature's `prd.json`
|
||||
(`.scratch/distributed-gguf-runtime/prd.json`); `docs/prd.json` and other `.scratch/*/prd.json`
|
||||
files in this repo use a materially different (simpler) shape and are out of scope.
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
DGR-021 and DGR-025 (this story's derived `blocks`) may treat `prd.json`'s `metadataSchema`,
|
||||
`qualityGates`, and this validator/generator as stable. Any future field addition to a story shape
|
||||
must extend `STORY_FIELDS` in `scripts/ralph_prd_schema.py` and the corresponding
|
||||
`metadataSchema.requiredStoryFields`/`optionalStoryFields` in `prd.json` together —
|
||||
`validate_metadata_schema_consistency` fails closed if they drift apart.
|
||||
@@ -1,15 +1,15 @@
|
||||
# Distributed GGUF Runtime evidence
|
||||
# Ralph task evidence
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
Each completed story creates `evidence/<TASK-ID>/README.md`. Fresh dependent iterations must read it before coding.
|
||||
|
||||
## Authority and classes
|
||||
Required README sections:
|
||||
|
||||
Evidence supports but never overrides `prd.json`. Valid classes are `model-free`, `fixture`, `real-model`, `real-hardware`, and `release`; lower classes cannot satisfy higher-class acceptance. Legacy DGR-001..016 directories remain unchanged for DGR-017 provenance audit and confer no completion credit.
|
||||
1. Summary and acceptance decision.
|
||||
2. Exact files changed.
|
||||
3. Commands run and real exit/results.
|
||||
4. Correctness, performance and hardware evidence classification.
|
||||
5. Known limitations and deferred work.
|
||||
6. Compatibility/migration notes.
|
||||
7. Explicit handoff for each dependent story.
|
||||
|
||||
## Future story layout
|
||||
|
||||
Each DGR-017..071 story writes `evidence/<ID>/README.md` with summary, exact changed files, exact commands and real outputs, limitations, compatibility/migration notes, and dependent-story handoff. Machine-readable contracts, manifests, metrics, and raw logs live beside it. Never fabricate output.
|
||||
|
||||
Real runs record exact model SHA and all split hashes, tokenizer, quant/recipe, llama.cpp pin+patch identity, backend/driver/toolchain, host/hardware/network, commands/environment (without secrets), raw parity/performance/resource results, and evidence class. Models live on configured mounted-drive storage, never `/home`.
|
||||
|
||||
Routing certification records prove only the exact exercised backend/model/recipe lane. Compile-only, fixture, failed, or unavailable lanes remain registered-dark. V4 cache/state evidence must show KV and CSA/HCA/SWA/indexer/compressor data remain shard-local/session-keyed; route recovery evidence must show cache miss plus re-prefill/restart, not migration.
|
||||
Store raw machine-readable metrics, manifests and protocol artifacts beside the README. Never store secrets, model weights, build outputs or Ralph iteration logs here.
|
||||
|
||||
@@ -1,335 +0,0 @@
|
||||
{
|
||||
"repository": "https://git.d-popov.com/popov/neuron-tai",
|
||||
"stories": {
|
||||
"DGR-017": {
|
||||
"number": 1,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/1",
|
||||
"state": "closed",
|
||||
"status": "completed"
|
||||
},
|
||||
"DGR-018": {
|
||||
"number": 2,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/2",
|
||||
"state": "closed",
|
||||
"status": "completed"
|
||||
},
|
||||
"DGR-019": {
|
||||
"number": 3,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/3",
|
||||
"state": "open",
|
||||
"status": "ready"
|
||||
},
|
||||
"DGR-020": {
|
||||
"number": 4,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/4",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-021": {
|
||||
"number": 5,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/5",
|
||||
"state": "open",
|
||||
"status": "ready"
|
||||
},
|
||||
"DGR-022": {
|
||||
"number": 6,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/6",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-023": {
|
||||
"number": 7,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/7",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-024": {
|
||||
"number": 8,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/8",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-025": {
|
||||
"number": 9,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/9",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-026": {
|
||||
"number": 10,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/10",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-027": {
|
||||
"number": 11,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/11",
|
||||
"state": "open",
|
||||
"status": "ready"
|
||||
},
|
||||
"DGR-028": {
|
||||
"number": 12,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/12",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-029": {
|
||||
"number": 13,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/13",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-030": {
|
||||
"number": 14,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/14",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-031": {
|
||||
"number": 15,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/15",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-032": {
|
||||
"number": 16,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/16",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-033": {
|
||||
"number": 17,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/17",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-034": {
|
||||
"number": 18,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/18",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-035": {
|
||||
"number": 19,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/19",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-036": {
|
||||
"number": 20,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/20",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-037": {
|
||||
"number": 21,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/21",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-038": {
|
||||
"number": 22,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/22",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-039": {
|
||||
"number": 23,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/23",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-040": {
|
||||
"number": 24,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/24",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-041": {
|
||||
"number": 25,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/25",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-042": {
|
||||
"number": 26,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/26",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-043": {
|
||||
"number": 27,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/27",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-044": {
|
||||
"number": 28,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/28",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-045": {
|
||||
"number": 29,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/29",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-046": {
|
||||
"number": 30,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/30",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-047": {
|
||||
"number": 31,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/31",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-048": {
|
||||
"number": 32,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/32",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-049": {
|
||||
"number": 33,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/33",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-050": {
|
||||
"number": 34,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/34",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-051": {
|
||||
"number": 35,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/35",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-052": {
|
||||
"number": 36,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/36",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-053": {
|
||||
"number": 37,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/37",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-054": {
|
||||
"number": 38,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/38",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-055": {
|
||||
"number": 39,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/39",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-056": {
|
||||
"number": 40,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/40",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-057": {
|
||||
"number": 41,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/41",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-058": {
|
||||
"number": 42,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/42",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-059": {
|
||||
"number": 43,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/43",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-060": {
|
||||
"number": 44,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/44",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-061": {
|
||||
"number": 45,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/45",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-062": {
|
||||
"number": 46,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/46",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-063": {
|
||||
"number": 47,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/47",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-064": {
|
||||
"number": 48,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/48",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-065": {
|
||||
"number": 49,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/49",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-066": {
|
||||
"number": 50,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/50",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-067": {
|
||||
"number": 51,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/51",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-068": {
|
||||
"number": 52,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/52",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-069": {
|
||||
"number": 53,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/53",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-070": {
|
||||
"number": 54,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/54",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
},
|
||||
"DGR-071": {
|
||||
"number": 55,
|
||||
"url": "https://git.d-popov.com/popov/neuron-tai/issues/55",
|
||||
"state": "open",
|
||||
"status": "blocked"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,40 +1,241 @@
|
||||
# Distributed GGUF Runtime implementation strategy
|
||||
# Focused implementation strategy: performant concurrent distributed inference
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
Status: Accepted planning direction
|
||||
Last updated: 2026-07-13
|
||||
|
||||
## Execution model
|
||||
## Product objective
|
||||
|
||||
Execute one numerically ordered, dependency-ready story per fresh Ralph context. Read `RALPH-CONTEXT.md`, source issue, and dependency evidence first; use TDD/fixture-first verification; finish with exact evidence. `prd.json` is the only state authority.
|
||||
Enable clients to run top open models that do not fit on one consumer machine by combining independently owned model Shards into performant, concurrent Inference Routes.
|
||||
|
||||
## Sequence
|
||||
The project is not trying to reproduce every vLLM feature or support every inference engine. It is optimizing for:
|
||||
|
||||
1. **M0 DGR-017..020:** reconcile legacy reality, lock metadata/performance contracts, and run the independent whole-model baseline.
|
||||
2. **M1 DGR-021..033:** protocol/lifecycle/codegen, exact identities and split artifacts, pinned upstream/patches, CPU then accelerator builds, `ShardEngine`, fixtures, fake worker.
|
||||
3. **M2 DGR-034..043:** dense ranged ownership/boundary/parity/local state, worker integration, supervision/direct-relay, and measured GGUF inputs to unchanged routing.
|
||||
4. **M3 DGR-044..054:** pin/inventory V4, adapt upstream boundary/local state/MoE/hash execution, pass parity and real 2–4 scenario, then enforce alpha with MTP off.
|
||||
5. **M4 DGR-055..067:** batching/backpressure/failure/recovery/long-context, existing-routing 10+ certification, real scale, measured optimization/compression, MTP contract+implementation, hardware certification.
|
||||
6. **M5 DGR-068..071:** packages, human upstream collaboration, beta gate (including MTP), and pin/patch/certification maintenance.
|
||||
1. Models larger than one node's RAM/VRAM.
|
||||
2. Useful interactive decode speed on consumer CPU, AMD, NVIDIA, Vulkan, and mixed routes where certified.
|
||||
3. Multiple concurrent Route Sessions without cache corruption or global serialization.
|
||||
4. A lean runtime with one control plane and one primary GGUF engine.
|
||||
5. Measured improvement over the existing Transformers/safetensors implementation.
|
||||
|
||||
## Guardrails
|
||||
## Current reality
|
||||
|
||||
The existing project already owns the differentiating distributed control plane:
|
||||
|
||||
## Locked scope
|
||||
- Tracker-selected contiguous Shards.
|
||||
- Stable Route Sessions.
|
||||
- Local per-Shard Hot KV State in the Transformers reference backend.
|
||||
- Binary Activation Seams.
|
||||
- Relay/direct routing, cancellation, telemetry, billing, and capability admission.
|
||||
- Persistent relay and direct transport optimizations.
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
The missing production path is a native GGUF execution worker that can load and execute only an assigned layer range while retaining local Hot KV State for concurrent Route Sessions.
|
||||
|
||||
## Target identities
|
||||
Whole-model llama.cpp, vLLM, and existing Transformers serving remain baselines or optional route kinds. They are not substitutes for native distributed Shards.
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
## Performance hypothesis—not an assumption
|
||||
|
||||
DGR-020 cannot use distributed results. DGR-054 does not depend on MTP. DGR-070 depends on DGR-066. Compile support and scenario success never imply general routability.
|
||||
GGUF itself is a format. Performance comes from llama.cpp/GGML's quantized kernels, memory layout, mmap, backend scheduling, and reduced working set.
|
||||
|
||||
Quantized GGUF may be faster or may merely fit a larger model. Comparisons against safetensors must report both speed and quality because BF16 safetensors and Q4/Q8 GGUF are not numerically equivalent.
|
||||
|
||||
Before expensive native work, establish controlled lanes:
|
||||
|
||||
- Same model architecture and upstream revision.
|
||||
- Same machine, prompt set, context, output length, sampling policy, and concurrency.
|
||||
- Transformers/safetensors BF16 or the current production recipe.
|
||||
- llama.cpp GGUF F16/BF16 or Q8 correctness lane where available.
|
||||
- Q4_K_M or selected production quantization performance/fit lane.
|
||||
- TTFT, prefill tok/s, decode tok/s, p50/p95 latency, RSS, VRAM, artifact size, energy where available, and output-quality drift.
|
||||
|
||||
The program proceeds only if llama.cpp/GGUF provides at least one meaningful advantage recorded in a machine-readable performance contract:
|
||||
|
||||
- Better decode or aggregate throughput at acceptable quality; or
|
||||
- Materially lower memory that makes the target model routable while preserving useful throughput.
|
||||
|
||||
## Parallelism we will use
|
||||
|
||||
### Public Inference Route: layer/pipeline parallelism
|
||||
|
||||
Each node independently executes one contiguous Shard. Activations cross seams; weights and Hot KV State remain local.
|
||||
|
||||
This is the only public cross-machine model-parallel primitive in the first runtime.
|
||||
|
||||
### Per-node continuous batching
|
||||
|
||||
Autoregressive tokens remain sequential within one generation. Throughput comes from batching decode steps from multiple active Route Sessions inside each node using llama.cpp batches and sequence IDs or bounded context pools.
|
||||
|
||||
This is essential. A worker that globally serializes sessions is not production-ready.
|
||||
|
||||
### Multiple complete routes: data parallelism
|
||||
|
||||
The Tracker may select multiple complete routes for independent requests. This increases network throughput and availability without requiring collectives between routes.
|
||||
|
||||
### Trusted composite node: optional tensor/expert parallelism
|
||||
|
||||
Tensor parallelism and expert parallelism require frequent collectives and tight compatibility. They may be used later inside one operator-controlled composite node or managed cluster exposed as one logical provider. They are not public WAN routing primitives.
|
||||
|
||||
### Deferred mechanisms
|
||||
|
||||
- Disaggregated prefill and KV transfer.
|
||||
- Speculative decoding.
|
||||
- Cross-route prefix snapshots.
|
||||
- Route repair with KV migration.
|
||||
- Public tensor/expert parallel collectives.
|
||||
|
||||
They remain out of the critical path until the native layer route passes performance and concurrency gates.
|
||||
|
||||
## Reuse decisions
|
||||
|
||||
### llama.cpp/GGML: primary runtime substrate
|
||||
|
||||
Reuse:
|
||||
|
||||
- GGUF parsing and mmap.
|
||||
- Quantized kernels.
|
||||
- CPU, CUDA, HIP/ROCm, Vulkan, Metal, and other supported backends.
|
||||
- Tokenizer and model architecture implementations.
|
||||
- KV and sequence operations.
|
||||
- Backend scheduler and graph execution.
|
||||
|
||||
Maintain a small exact-commit fork only for the missing local seam:
|
||||
|
||||
- Range-aware tensor ownership/loading.
|
||||
- Architecture-defined boundary input/output.
|
||||
- Intermediate boundary output without tail normalization.
|
||||
- Layer-filtered KV and sequence mapping.
|
||||
|
||||
Keep networking, Tracker logic, billing, and public protocol outside llama.cpp. Upstream generic hooks where possible.
|
||||
|
||||
### vLLM: concepts and optional managed backend
|
||||
|
||||
Use unmodified vLLM only as:
|
||||
|
||||
- A whole-model node backend.
|
||||
- A managed TP/PP/EP cluster represented as one logical provider.
|
||||
- A performance/correctness baseline.
|
||||
|
||||
Adapt concepts, not runtime code:
|
||||
|
||||
- Named intermediate tensor bundles.
|
||||
- Continuous batching and request-owner maps.
|
||||
- Versioned KV-transfer compatibility fingerprints.
|
||||
- Explicit send/receive/abort/failure lifecycle.
|
||||
- Load telemetry and unbiased route selection.
|
||||
|
||||
Do not fork vLLM for public Shards and do not transplant PagedAttention, Torch process groups, or GGUF-plugin kernels into the llama.cpp worker.
|
||||
|
||||
### Nakshatra, prima.cpp, llama-gguf, LiGGUF, GPUStack
|
||||
|
||||
Use as source and test donors only:
|
||||
|
||||
- Nakshatra: partial-GGUF patches, daemon concepts, replay cases.
|
||||
- prima.cpp: selected tensor ownership and local-layer KV evidence.
|
||||
- llama-gguf: small protocol and integration-test patterns.
|
||||
- LiGGUF: Q8 activation transport and tensor-reduction reference.
|
||||
- historical GPUStack: resource preflight and role-oriented placement.
|
||||
|
||||
Do not adopt or fork their repositories wholesale.
|
||||
|
||||
## Battle-proven transport decision
|
||||
|
||||
Use gRPC over HTTP/2 with Protocol Buffers for the native C++ Shard worker protocol.
|
||||
|
||||
Why:
|
||||
|
||||
- Mature Python and C++ implementations.
|
||||
- Bidirectional streaming.
|
||||
- HTTP/2 flow control and connection reuse.
|
||||
- Deadlines, cancellation, status codes, TLS, authentication interceptors, and generated schemas.
|
||||
- Avoids inventing a socket protocol.
|
||||
|
||||
Scope boundary:
|
||||
|
||||
- OpenAI-compatible client/Gateway APIs remain HTTP/SSE.
|
||||
- Tracker/control APIs remain existing project interfaces.
|
||||
- One long-lived bidirectional gRPC stream serves one Route Session Activation Seam.
|
||||
- Existing relay/WebSocket infrastructure may carry the same versioned protobuf frames as opaque binary when direct gRPC reachability is unavailable.
|
||||
- Large prefill tensors are chunked into bounded frames; decode bundles stay small.
|
||||
- No QUIC/WebRTC/custom transport in this milestone.
|
||||
|
||||
The public boundary uses a versioned named-tensor bundle rather than one anonymous tensor because architecture boundaries can require more than `hidden_states`.
|
||||
|
||||
Minimum identity:
|
||||
|
||||
```text
|
||||
schema version
|
||||
request/work id
|
||||
Route Session id and route epoch
|
||||
Model Artifact and runtime recipe fingerprint
|
||||
Shard range and effective start
|
||||
phase: prefill/decode/release/cancel
|
||||
position/token range
|
||||
named tensors with shape/dtype/byte order
|
||||
compression and checksum
|
||||
idempotency step id
|
||||
cache expectation/result
|
||||
```
|
||||
|
||||
## Concurrency model
|
||||
|
||||
A native worker must not use one global serving sequence or one lock around all model execution.
|
||||
|
||||
Required ownership:
|
||||
|
||||
```text
|
||||
(Route Session id, route epoch)
|
||||
-> local sequence/context
|
||||
-> Shard-local Hot KV State
|
||||
-> bounded lease and memory accounting
|
||||
```
|
||||
|
||||
The node scheduler:
|
||||
|
||||
- Admits sessions against model memory and KV budget.
|
||||
- Forms compatible decode batches from active sessions.
|
||||
- Preserves per-session position and route order.
|
||||
- Applies bounded queues and backpressure.
|
||||
- Cancels/releases independently.
|
||||
- Reports queue, batch, KV, prefill, decode, and seam telemetry.
|
||||
|
||||
Initial deterministic gate: at least four concurrent sessions on a small certified model with no token/KV cross-talk. Final concurrency targets are hardware/recipe-specific and recorded by capability admission rather than hardcoded globally.
|
||||
|
||||
## Stage gates
|
||||
|
||||
### Gate A: performance hypothesis
|
||||
|
||||
Controlled safetensors-versus-GGUF benchmark produces a signed/reproducible report and locks thresholds. Stop native work if there is no meaningful speed or fit benefit.
|
||||
|
||||
### Gate B: local range parity
|
||||
|
||||
Two local processes own disjoint GGUF ranges and match whole-model llama.cpp within the certified numerical tolerance for prefill and greedy decode.
|
||||
|
||||
### Gate C: concurrent KV
|
||||
|
||||
Multiple Route Sessions prefill/decode concurrently with isolated local KV, bounded memory, cancellation, and release.
|
||||
|
||||
### Gate D: real distributed route
|
||||
|
||||
Two physical machines execute one model that uses both Shards. Synthetic activation tests do not satisfy this gate.
|
||||
|
||||
### Gate E: consumer-hardware performance
|
||||
|
||||
On certified consumer hardware, the GGUF route beats the current distributed safetensors route under the locked performance contract or enables a larger otherwise-unroutable model at useful measured speed.
|
||||
|
||||
### Gate F: architecture expansion
|
||||
|
||||
Only after dense Llama-family gates pass, add an explicit Qwen3/Qwen3-MoE adapter and certify it independently.
|
||||
|
||||
## Scope discipline
|
||||
|
||||
The following do not block the first production candidate:
|
||||
|
||||
- New cryptocurrency/economics work.
|
||||
- New artifact P2P protocol.
|
||||
- QUIC or WebRTC.
|
||||
- vLLM fork.
|
||||
- Whole-repository Nakshatra/prima adoption.
|
||||
- Every GGUF architecture.
|
||||
- Automatic route repair.
|
||||
- Prefix snapshot migration.
|
||||
- Speculative decoding.
|
||||
- A large-model marketing demo before small-model parity and concurrency pass.
|
||||
|
||||
Every optimization must preserve output contract, session isolation, cancellation, resource cleanup, capability admission, and per-node attribution.
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# 01 — Lock the safetensors-versus-GGUF performance contract
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-001` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a runtime engineer, I need a controlled baseline so that GGUF work proceeds from measured speed, memory, and quality rather than reputation.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Benchmark harness and deterministic tests
|
||||
- evidence/DGR-001/performance-contract.json
|
||||
- Raw and summarized safetensors/GGUF benchmark evidence
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Benchmark the same model architecture/revision, machine, prompts, context lengths, output lengths, sampling policy, and concurrency across the current Transformers/safetensors recipe and whole-model llama.cpp recipes.
|
||||
- [ ] Separate correctness/quality lanes from quantized performance/fit lanes instead of claiming BF16 and Q4 are numerically equivalent.
|
||||
- [ ] Report TTFT, prefill tok/s, decode tok/s, p50/p95 latency, aggregate throughput, RSS, VRAM, artifact size, failures, and output drift in machine-readable JSON.
|
||||
- [ ] Add concurrency levels 1 and 4 where memory permits.
|
||||
- [ ] Write a versioned performance contract consumed by later release gates, including an explicit stop condition when llama.cpp/GGUF has no meaningful speed or fit benefit.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-001/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- None. This story may start immediately.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-017: Reconcile and clean the superseded DGR backlog
|
||||
|
||||
- **Status / triage:** completed; `passes: true`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** None
|
||||
- **Blocks (derived):** `DGR-018`, `DGR-019`, `DGR-027`, `DGR-054`
|
||||
- **Labels:** `area:provenance`, `area:cleanup`, `type:audit`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/017-reconcile-and-clean-the-superseded-dgr-backlog.md`, and evidence READMEs for dependencies (none) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Audit implementation reality, void inherited completion credit, and clean misleading backlog/stub baggage while preserving attributable evidence and accepted research.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Compare the branch, old DGR-001..016 issue/pass states, evidence, and actual runtime sources; classify each output as reusable, reference-only, blocked, obsolete, or absent.
|
||||
- [x] Record an authoritative old-to-new disposition and provenance; explicitly give no completion credit to any new story and note absent implementation/evidence.
|
||||
- [x] Remove or archive only artifacts the audit proves obsolete while preserving accepted ADRs, useful research, raw benchmark evidence, and attributable reusable work.
|
||||
- [x] Protect ignored build workspaces, generated protobuf outputs, Ralph logs, and model artifacts from accidental commits, and document every retained legacy artifact.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-017/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-018: Define canonical Ralph and Gitea metadata schema
|
||||
|
||||
- **Status / triage:** completed; `passes: true`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** `DGR-017`
|
||||
- **Blocks (derived):** `DGR-021`, `DGR-025`
|
||||
- **Labels:** `area:planning`, `area:gitea`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make `prd.json` the validated source from which Markdown and Gitea issues can later be generated losslessly.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [x] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
|
||||
- [x] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
|
||||
- [x] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
|
||||
- [x] Add deterministic model-free tests for parse, validation, and generation round trips.
|
||||
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.
|
||||
@@ -1,40 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-019: Lock alpha and beta performance contracts
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** `DGR-017`
|
||||
- **Blocks (derived):** `DGR-020`, `DGR-044`, `DGR-054`
|
||||
- **Labels:** `area:performance`, `type:contract`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `generic+deepseek-v4`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Freeze useful speed, correctness, memory-fit, and stop/go thresholds before implementation results are visible.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define controlled safetensors, whole-model GGUF, dense distributed GGUF, and V4 Flash distributed lanes with fixed prompts, context/output lengths, sampling, concurrency, hardware, and metrics.
|
||||
- [ ] Alpha requires correctness plus a human-approved useful-speed threshold; beta adds concurrency, long-context, failure, and sustained-throughput thresholds.
|
||||
- [ ] Separate quantization/model-fit gains from runtime, transport, batching, and kernel gains.
|
||||
- [ ] Treat quants and 2–4/10+ stage counts only as named certification scenarios; no product logic may hardcode them.
|
||||
- [ ] Lock thresholds and stop conditions in versioned machine-readable data before benchmark result ingestion.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-019/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,59 @@
|
||||
# 02 — Adopt the versioned gRPC Shard protocol
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-002` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node developer, I need a battle-proven streaming protocol so that Python and C++ Shards communicate without a custom socket protocol.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- packages/node/native/proto/shard_runtime.proto
|
||||
- Reproducible Python/C++ schema generation and build wiring
|
||||
- Protocol round-trip and compatibility tests
|
||||
- evidence/DGR-002/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Add a Protocol Buffers schema for capability, health, session stream, release, and cancellation operations.
|
||||
- [ ] Define one long-lived bidirectional gRPC stream per Route Session Activation Seam with deadlines, cancellation, flow control, and structured errors.
|
||||
- [ ] Define bounded chunking for prefill and a small decode fast path.
|
||||
- [ ] Carry schema version, request/work ID, Route Session ID, route epoch, artifact/recipe fingerprint, Shard range/effective start, phase, position, idempotency step, cache expectation, compression, and checksum.
|
||||
- [ ] Define a versioned named-tensor bundle with per-tensor name, shape, dtype, byte order, and payload fragments.
|
||||
- [ ] Add generated-schema round-trip and compatibility tests in Python and C++.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-002/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- None. This story may start immediately.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-020: Run the controlled whole-model GGUF baseline
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M0`
|
||||
- **Dependencies:** `DGR-019`
|
||||
- **Blocks (derived):** `DGR-054`
|
||||
- **Labels:** `area:performance`, `type:benchmark`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/020-run-the-controlled-whole-model-gguf-baseline.md`, and evidence READMEs for dependencies (DGR-019) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute the locked safetensors and whole-model llama.cpp lanes before distributed implementation results can influence the decision.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run the exact DGR-019 safetensors and whole-model llama.cpp benchmark lanes with locked prompts, lengths, sampling, concurrency, hardware, and artifact/runtime identities.
|
||||
- [ ] Record raw machine-readable correctness, TTFT, prefill/decode, throughput, latency, memory, artifact-size, failure, and quality-drift metrics without ingesting distributed implementation results.
|
||||
- [ ] Separate quantization/model-fit effects from runtime/kernel effects and preserve failed or unavailable lanes honestly.
|
||||
- [ ] Publish a threshold-based `go`, `optimize baseline`, or `stop` decision without changing the locked contract.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-020/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-021: Define the versioned named-tensor stream envelope
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-018`
|
||||
- **Blocks (derived):** `DGR-022`, `DGR-023`, `DGR-025`, `DGR-031`, `DGR-035`, `DGR-046`
|
||||
- **Labels:** `area:protocol`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/021-define-the-versioned-named-tensor-stream-envelope.md`, and evidence READMEs for dependencies (DGR-018) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the backend-neutral protobuf envelope used by direct and relayed Shard activation traffic.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define schema version, request/work ID, route session/epoch, shard range/effective start, phase, position, and idempotency step.
|
||||
- [ ] Define named tensors with shape, dtype, byte order, bounded fragments, compression identity, and checksum.
|
||||
- [ ] Reserve extensible fields for token-ID sidebands, architecture state, recurrent state, and MTP without claiming implementations.
|
||||
- [ ] Add deterministic serialization, fragmentation, checksum, unknown-field, and size-limit tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-021/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-022: Define Shard lifecycle and structured status RPCs
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-021`
|
||||
- **Blocks (derived):** `DGR-024`, `DGR-033`, `DGR-037`
|
||||
- **Labels:** `area:protocol`, `area:lifecycle`, `type:infrastructure`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/022-define-shard-lifecycle-and-structured-status-rpcs.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Complete the gRPC contract for worker capability, health, sessions, cancellation, release, and metrics.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define capability, health, bidirectional session stream, cancellation, release, and metrics RPCs.
|
||||
- [ ] Specify deadlines, cancellation propagation, bounded flow control, cache expectations/results, and structured error taxonomy.
|
||||
- [ ] Specify TLS/auth hooks without moving Meshnet authentication or billing into the worker.
|
||||
- [ ] Add compatibility tests for supported versions and fail-closed tests for unsupported versions and malformed lifecycle transitions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-022/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-023: Make Python and C++ protobuf generation reproducible
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-021`
|
||||
- **Blocks (derived):** `DGR-024`, `DGR-037`
|
||||
- **Labels:** `area:protocol`, `area:build`, `type:tooling`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/023-make-python-and-c-protobuf-generation-reproducible.md`, and evidence READMEs for dependencies (DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Generate identical Python/C++ protocol bindings without manual copying or checked-in build debris.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin protoc, gRPC, and plugin versions or declare a verified compatible range.
|
||||
- [ ] Generate Python and C++ bindings into out-of-tree build/package locations through documented commands.
|
||||
- [ ] Add Python↔C++ round-trip and descriptor compatibility tests.
|
||||
- [ ] A clean checkout regenerates bindings deterministically or fails with an actionable toolchain error.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-023/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-024: Implement in-memory fake gRPC seam transport
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-022`, `DGR-023`
|
||||
- **Blocks (derived):** `DGR-033`, `DGR-042`
|
||||
- **Labels:** `area:protocol`, `area:testing`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `fake`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/024-implement-in-memory-fake-grpc-seam-transport.md`, and evidence READMEs for dependencies (DGR-022, DGR-023) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Exercise the complete streaming protocol deterministically before a real model or worker exists.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Provide a fake bidirectional stream supporting prefill fragments, decode fast-path frames, release, cancel, and structured errors.
|
||||
- [ ] Test flow-control blocking, deadlines, malformed fragments, checksum failure, duplicates, and stale epochs.
|
||||
- [ ] Verify direct and opaque-relay framing preserve identical protobuf bytes.
|
||||
- [ ] Tests require no sockets outside localhost, model downloads, or native accelerator.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-024/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-025: Define exact artifact and runtime recipe identity
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-018`, `DGR-021`
|
||||
- **Blocks (derived):** `DGR-026`, `DGR-031`, `DGR-041`, `DGR-044`
|
||||
- **Labels:** `area:identity`, `area:admission`, `type:domain`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/025-define-exact-artifact-and-runtime-recipe-identity.md`, and evidence READMEs for dependencies (DGR-018, DGR-021) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure the tracker and worker only combine numerically and operationally compatible shards.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Fingerprint source artifact SHA, tokenizer revision, architecture adapter/version, boundary schema, runtime pin/patch stack, backend, quant, activation/compute dtype, and KV/state layout.
|
||||
- [ ] Bind each shard to an exact half-open range without hardcoding a topology or quant.
|
||||
- [ ] Fail closed on any artifact, adapter, boundary, cache, backend, or runtime mismatch.
|
||||
- [ ] Unsupported recipes remain registered-but-dark until real-hardware evidence certifies them.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-025/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-026: Provision exact split-GGUF artifacts outside /home
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-025`
|
||||
- **Blocks (derived):** `DGR-044`, `DGR-045`
|
||||
- **Labels:** `area:artifacts`, `area:provenance`, `type:tooling`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/026-provision-exact-split-gguf-artifacts-outside-home.md`, and evidence READMEs for dependencies (DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Make exact split-GGUF inputs reproducibly available from mounted-drive storage without embedding a quantization or topology assumption in product code.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Create an exact manifest that binds the source artifact, tokenizer/revision, every split file name, size, range/role, and cryptographic hash.
|
||||
- [ ] Provide resumable, hash-verifying download/provision tooling targeting configured mounted-drive storage; refuse paths under `/home` and incomplete or mismatched splits.
|
||||
- [ ] Keep quantization and split topology as manifest/recipe inputs with no hardcoded quant, node count, or range layout.
|
||||
- [ ] Add deterministic model-download-free tests using tiny local split fixtures, including interrupted resume, missing split, hash mismatch, and `/home` rejection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-026/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-027: Add exact llama.cpp provenance manifest and fetch workspace
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-017`
|
||||
- **Blocks (derived):** `DGR-028`, `DGR-029`, `DGR-044`
|
||||
- **Labels:** `area:upstream`, `area:build`, `type:provenance`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/027-add-exact-llama-cpp-provenance-manifest-and-fetch-workspace.md`, and evidence READMEs for dependencies (DGR-017) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Pin llama.cpp exactly through an in-repo manifest while fetching source only into an ignored build workspace.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Manifest records upstream URL, exact commit, expected source archive/tree hash, license, and retrieval method.
|
||||
- [ ] Fetch tooling verifies identity before use and refuses an unpinned branch/tag.
|
||||
- [ ] Source is fetched into an ignored build workspace; no submodule, vendored source tree, or permanent fork is introduced.
|
||||
- [ ] Offline reuse is supported only after the cached tree’s exact identity is verified.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-027/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-028: Implement numbered patch-stack apply and verification
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-027`
|
||||
- **Blocks (derived):** `DGR-029`, `DGR-034`, `DGR-069`
|
||||
- **Labels:** `area:upstream`, `area:patches`, `type:tooling`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/028-implement-numbered-patch-stack-apply-and-verification.md`, and evidence READMEs for dependencies (DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Maintain a minimal auditable llama.cpp delta with one numbered patch per concern.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Add deterministic apply/check/reverse verification against the exact manifest pin.
|
||||
- [ ] Separate range loading, boundary I/O, filtered state, and worker hooks into scoped patches.
|
||||
- [ ] Record upstream file/API assumptions and fail with the first incompatible patch when the pin changes.
|
||||
- [ ] Verify license/attribution and prove no Meshnet routing, billing, relay, or authentication code enters the patch stack.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-028/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-029: Create the native CMake skeleton and deterministic CPU lane
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-027`, `DGR-028`
|
||||
- **Blocks (derived):** `DGR-030`, `DGR-034`
|
||||
- **Labels:** `area:build`, `type:toolchain`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/029-create-the-native-cmake-skeleton-and-deterministic-cpu-lane.md`, and evidence READMEs for dependencies (DGR-027, DGR-028) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish an out-of-tree standalone native build with a deterministic CPU lane before accelerator matrix work.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Create the standalone native CMake target/skeleton and isolated out-of-tree configure/build preset for CPU.
|
||||
- [ ] Build and run a deterministic model-free CPU smoke/CTest lane from a clean checkout with actionable toolchain failures.
|
||||
- [ ] Keep fetched upstream sources, generated bindings, and all build outputs ignored and out of tree.
|
||||
- [ ] Ensure build success alone does not advertise any backend/model/recipe capability.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-029/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,57 @@
|
||||
# 03 — Define exact Artifact and runtime recipe identity
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-003` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the Tracker, I need exact compatibility identity so that only numerically and operationally compatible Shards form an Inference Route.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact runtime recipe/fingerprint implementation
|
||||
- Tracker/node fail-closed admission tests
|
||||
- evidence/DGR-003/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Separate weight quantization, activation dtype, compute dtype, KV dtype/layout, tokenizer revision, architecture adapter, backend, and runtime version.
|
||||
- [ ] Bind derivative or split artifacts to an exact source Model Artifact hash and Shard range.
|
||||
- [ ] Produce a stable compatibility fingerprint used by capability admission and the gRPC handshake.
|
||||
- [ ] Fail closed on mismatched artifact, tokenizer, architecture, range, boundary schema, activation recipe, or cache layout.
|
||||
- [ ] Keep unsupported recipes registered-but-dark until a real distributed forward certifies them.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-003/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-030: Add accelerator build presets and native CI matrix
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-029`
|
||||
- **Blocks (derived):** `DGR-053`, `DGR-067`, `DGR-068`
|
||||
- **Labels:** `area:build`, `area:hardware`, `type:toolchain`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/030-add-accelerator-build-presets-and-native-ci-matrix.md`, and evidence READMEs for dependencies (DGR-029) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Compile all required accelerator lanes reproducibly while keeping untested real-hardware capabilities registered-dark.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Add isolated out-of-tree presets for CUDA, ROCm, Vulkan, and Metal without changing the deterministic CPU default.
|
||||
- [ ] Add a native CI/build matrix that reports unavailable SDKs as explicit unavailable/skipped lanes rather than false success.
|
||||
- [ ] Compile each available lane and preserve exact compiler, SDK, upstream pin, patch-stack, and build-option evidence.
|
||||
- [ ] Keep every backend/model/recipe lane registered-dark until a separate real-hardware certification record exists.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-030/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-031: Introduce the project-owned `ShardEngine` interface
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-021`, `DGR-025`
|
||||
- **Blocks (derived):** `DGR-032`, `DGR-034`, `DGR-035`, `DGR-037`
|
||||
- **Labels:** `area:engine`, `type:architecture`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/031-introduce-the-project-owned-shardengine-interface.md`, and evidence READMEs for dependencies (DGR-021, DGR-025) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Isolate worker/protocol code from llama.cpp internals behind a stable project-owned engine contract.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define load, capabilities, prefill/decode, boundary/logits result, cancel, release, health, and metrics operations.
|
||||
- [ ] Use project-owned request/result/state types; expose no `ggml_tensor`, llama context, scheduler, or ABI-owned structure.
|
||||
- [ ] Reserve typed MTP and architecture auxiliary-state hooks without enabling them.
|
||||
- [ ] Add contract tests proving fake and future llama implementations obey identical lifecycle semantics.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-031/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-032: Implement deterministic fake `ShardEngine`
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-031`
|
||||
- **Blocks (derived):** `DGR-033`
|
||||
- **Labels:** `area:engine`, `area:testing`, `type:fixture`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `fake`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/032-implement-deterministic-fake-shardengine.md`, and evidence READMEs for dependencies (DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Provide an engine fixture that deterministically transforms typed boundary bundles and session state.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Support head, middle, tail, prefill, decode, cancellation, and release with deterministic outputs.
|
||||
- [ ] Model isolated session/epoch state and deterministic cache-miss/stale-epoch failures.
|
||||
- [ ] Support configurable delay, memory pressure, malformed output, and crash injection.
|
||||
- [ ] Contract tests distinguish fixture evidence from real-model certification.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-032/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-033: Build a standalone fake C++ gRPC Shard worker
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M1`
|
||||
- **Dependencies:** `DGR-022`, `DGR-024`, `DGR-032`
|
||||
- **Blocks (derived):** `DGR-036`, `DGR-040`
|
||||
- **Labels:** `area:worker`, `area:grpc`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `fake`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/033-build-a-standalone-fake-c-grpc-shard-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-024, DGR-032) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the standalone worker process, stream, lifecycle, and supervision shape before llama.cpp integration.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A standalone C++ executable serves the complete lifecycle and stream RPC contract using the fake engine.
|
||||
- [ ] Python integration tests cover startup, health, capability, fragmented prefill, decode, release, cancellation, and graceful shutdown.
|
||||
- [ ] Bounded messages, deadlines, flow control, and independent session cancellation are enforced.
|
||||
- [ ] The worker exposes neither llama.cpp RPC nor arbitrary graph execution.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-033/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-034: Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-028`, `DGR-029`, `DGR-031`
|
||||
- **Blocks (derived):** `DGR-035`, `DGR-037`, `DGR-051`
|
||||
- **Labels:** `area:llama`, `area:weights`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/034-implement-dense-llama-range-aware-gguf-ownership.md`, and evidence READMEs for dependencies (DGR-028, DGR-029, DGR-031) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Load only the dense-Llama tensors owned by a shard range.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Load only `blk.N.*` tensors in the assigned range, embeddings only at the head, and norm/output or tied output only at the tail.
|
||||
- [ ] Derive authoritative range and endpoint ownership from the loaded engine state.
|
||||
- [ ] Reject invalid/gapped/out-of-model ranges and unexpected required tensors.
|
||||
- [ ] Real-model evidence shows mapped/resident memory scales with owned tensors rather than full artifact size.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-034/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-035: Implement dense architecture boundary input/output
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-021`, `DGR-031`, `DGR-034`
|
||||
- **Blocks (derived):** `DGR-036`, `DGR-037`, `DGR-069`
|
||||
- **Labels:** `area:llama`, `area:boundary`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/035-implement-dense-architecture-boundary-input-output.md`, and evidence READMEs for dependencies (DGR-021, DGR-031, DGR-034) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Execute dense head/middle/tail ranges through an architecture-defined residual boundary.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Head accepts token IDs and owns embedding; middle/tail bypass embedding and accept a named boundary bundle.
|
||||
- [ ] Non-tail returns the unnormalized residual before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail returns logits or sampled-token output under an explicit contract.
|
||||
- [ ] Uncertified architectures and incompatible boundary schemas fail closed.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-035/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-036: Prove dense fixture and real-model range parity
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-033`, `DGR-035`
|
||||
- **Blocks (derived):** `DGR-039`
|
||||
- **Labels:** `area:parity`, `area:testing`, `type:certification`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/036-prove-dense-fixture-and-real-model-range-parity.md`, and evidence READMEs for dependencies (DGR-033, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove the engine/protocol seam before adding session concurrency.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Model-free two-stage tests pass through two fake worker processes with disjoint ranges.
|
||||
- [ ] A small real dense GGUF passes whole-model versus two-range prefill parity.
|
||||
- [ ] At least 32 greedy decode tokens match the locked tolerance.
|
||||
- [ ] Evidence distinguishes deterministic fixture proof from opt-in real-model proof.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-036/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-037: Bind llama.cpp to the standalone worker
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-022`, `DGR-023`, `DGR-031`, `DGR-034`, `DGR-035`
|
||||
- **Blocks (derived):** `DGR-038`, `DGR-039`, `DGR-040`, `DGR-051`
|
||||
- **Labels:** `area:worker`, `area:llama`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/037-bind-llama-cpp-to-the-standalone-worker.md`, and evidence READMEs for dependencies (DGR-022, DGR-023, DGR-031, DGR-034, DGR-035) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Replace the fake engine with the pinned llama.cpp `ShardEngine` implementation without changing the public worker contract.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Worker loads exactly one artifact/recipe/range identity and rejects mismatched stream requests.
|
||||
- [ ] All execution passes through `ShardEngine`; llama.cpp implementation types remain private.
|
||||
- [ ] Health and metrics expose loaded identity, authoritative ownership, memory, and execution state.
|
||||
- [ ] Graceful shutdown releases model/session resources; injected process death is observable and bounded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-037/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-038: Implement isolated shard-local Hot KV State
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-037`
|
||||
- **Blocks (derived):** `DGR-039`, `DGR-052`, `DGR-055`, `DGR-069`
|
||||
- **Labels:** `area:kv-cache`, `area:concurrency`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/038-implement-isolated-shard-local-hot-kv-state.md`, and evidence READMEs for dependencies (DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Map each route session and epoch to isolated state containing only locally owned layers.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Map `(route_session_id, route_epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Support prefill/decode append, truncate, release, TTL/LRU eviction, cache miss, and stale-epoch rejection.
|
||||
- [ ] Four concurrent sessions complete without token, KV, position, or cancellation cross-talk.
|
||||
- [ ] Release/eviction returns memory to the configured budget without affecting other sessions.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-038/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-039: Pass local two-process dense acceptance
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-036`, `DGR-037`, `DGR-038`
|
||||
- **Blocks (derived):** `DGR-054`
|
||||
- **Labels:** `area:integration`, `area:parity`, `type:certification`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `dense-llama`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/039-pass-local-two-process-dense-acceptance.md`, and evidence READMEs for dependencies (DGR-036, DGR-037, DGR-038) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Establish the first real, independently executing distributed GGUF route on one host.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Two worker processes open disjoint dense ranges and both execute real prefill/decode work.
|
||||
- [ ] Whole-model parity, 32-token greedy decode, four-session isolation, cancellation, and cleanup pass.
|
||||
- [ ] Record TTFT, prefill/decode rates, seam bytes/latency, RSS/VRAM, KV, queue, and failure metrics.
|
||||
- [ ] Killing one worker returns a bounded structured failure rather than hanging.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-039/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,61 @@
|
||||
# 04 — Create the reproducible pinned llama.cpp patch stack
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-004` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a maintainer, I need a small auditable fork boundary so that upstream updates do not turn the runtime into an unmaintainable stitched codebase.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Exact llama.cpp upstream pin
|
||||
- Numbered minimal patch stack
|
||||
- Reproducible fetch/apply/build smoke
|
||||
- evidence/DGR-004/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin one exact llama.cpp commit through a reproducible source dependency mechanism.
|
||||
- [ ] Store a numbered minimal patch stack separately from Meshnet networking code.
|
||||
- [ ] Add a build script that applies/checks patches and builds the standalone worker without manual source copying.
|
||||
- [ ] Record upstream file/ABI assumptions and fail clearly when the pin changes.
|
||||
- [ ] Preserve upstream license and attribution notices.
|
||||
- [ ] Add a clean rebuild smoke test that does not download a model.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-004/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-040: Add node-side native worker supervision
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-033`, `DGR-037`
|
||||
- **Blocks (derived):** `DGR-041`, `DGR-042`, `DGR-055`, `DGR-058`
|
||||
- **Labels:** `area:node`, `area:supervision`, `type:integration`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/040-add-node-side-native-worker-supervision.md`, and evidence READMEs for dependencies (DGR-033, DGR-037) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Let the existing node service safely start, probe, stop, and restart the standalone worker.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Supervision owns process startup, readiness, log capture, graceful shutdown, and bounded forced termination.
|
||||
- [ ] Startup verifies worker binary, artifact identity, recipe, and range before registration.
|
||||
- [ ] Crashes or health loss make the capability unavailable without corrupting the Transformers backend.
|
||||
- [ ] Tests use the fake worker and deterministic crash injection.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-040/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-041: Register native Shard capabilities without redesigning Meshnet
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-025`, `DGR-040`
|
||||
- **Blocks (derived):** `DGR-043`
|
||||
- **Labels:** `area:meshnet`, `area:admission`, `type:integration`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/041-register-native-shard-capabilities-without-redesigning-meshnet.md`, and evidence READMEs for dependencies (DGR-025, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt native worker identity and capacity to existing backend-agnostic capability admission.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Registration carries exact recipe fingerprint, authoritative range, backend, memory/KV capacity, concurrency, and certification status.
|
||||
- [ ] Existing tracker, billing, routing, telemetry, and provider semantics remain backend-agnostic.
|
||||
- [ ] Uncertified backend/model/recipe combinations are visible but unroutable.
|
||||
- [ ] Existing Transformers registration and route tests remain unchanged in behavior.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-041/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-042: Carry native frames through direct and existing relay seams
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-024`, `DGR-040`
|
||||
- **Blocks (derived):** `DGR-054`, `DGR-058`
|
||||
- **Labels:** `area:meshnet`, `area:relay`, `area:grpc`, `type:integration`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/042-carry-native-frames-through-direct-and-existing-relay-seams.md`, and evidence READMEs for dependencies (DGR-024, DGR-040) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Use direct gRPC where available and the existing relay as opaque protobuf transport without redesigning relay behavior.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Direct paths use the long-lived gRPC activation stream.
|
||||
- [ ] Relayed paths carry byte-identical versioned protobuf frames through the existing relay contract.
|
||||
- [ ] Request/work identity, cancellation, deadlines, telemetry, billing correlation, and per-node attribution survive both paths.
|
||||
- [ ] Fake-worker tests cover direct, relay, disconnect, cancellation, and bounded buffering.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-042/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-043: Expose GGUF compatibility and measured cost inputs to existing routing
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M2`
|
||||
- **Dependencies:** `DGR-041`
|
||||
- **Blocks (derived):** `DGR-053`, `DGR-054`, `DGR-059`, `DGR-061`
|
||||
- **Labels:** `area:tracker`, `area:routing`, `type:integration`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/043-expose-gguf-compatibility-and-measured-cost-inputs-to-existing-routing.md`, and evidence READMEs for dependencies (DGR-041) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Adapt exact GGUF recipe/range compatibility and measured capacity, queue, seam-cost, health, and reliability data into the existing backend-agnostic routing inputs without redesigning routing or load balancing.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Expose exact recipe, range coverage, capacity, queue/load, seam-cost, health, reliability, backend, and certification measurements through existing tracker input contracts.
|
||||
- [ ] Prove existing routing forms complete compatible coverage and excludes dark or mismatched candidates using its current backend-agnostic mechanisms.
|
||||
- [ ] Regression-test unchanged Transformers behavior and unchanged tracker routing, load-balancing, billing, relay, and provider semantics.
|
||||
- [ ] Regression-test that no quant, stage count, fixed split, architecture, backend sequence, or DeepSeek-specific policy is hardcoded.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-043/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-044: Pin the DeepSeek V4 Flash target contract
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-019`, `DGR-025`, `DGR-026`, `DGR-027`
|
||||
- **Blocks (derived):** `DGR-045`
|
||||
- **Labels:** `area:deepseek-v4`, `area:provenance`, `type:contract`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/044-pin-the-deepseek-v4-flash-target-contract.md`, and evidence READMEs for dependencies (DGR-019, DGR-025, DGR-026, DGR-027) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Fix the alpha/beta model identity and upstream maturity assumptions before implementing the adapter.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Pin Hugging Face target SHA `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- [ ] Record the selected GGUF provenance and exact source binding without assuming one quant.
|
||||
- [ ] Record that llama.cpp support began with PR 24162/commit `8c146a8` and remains maturing; the build manifest still pins one exact validated upstream commit.
|
||||
- [ ] Record 43 main layers plus MTP and all known architecture-specific state obligations.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-044/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-045: Inventory V4 GGUF tensors and layer ownership
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-026`, `DGR-044`
|
||||
- **Blocks (derived):** `DGR-046`, `DGR-047`, `DGR-050`
|
||||
- **Labels:** `area:deepseek-v4`, `area:weights`, `type:research-spike`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/045-inventory-v4-gguf-tensors-and-layer-ownership.md`, and evidence READMEs for dependencies (DGR-026, DGR-044) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Derive exact head, repeated-layer, tail, expert, auxiliary-state, and MTP ownership from the pinned artifact.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Produce a machine-readable inventory tied to the target SHA and exact GGUF artifact.
|
||||
- [ ] Classify embeddings, 43 main layers, final head, MTP, mHC, attention variants, indexer/compressor, and MoE tensors.
|
||||
- [ ] Validate half-open range ownership and fail on unclassified required tensors.
|
||||
- [ ] Inventory tooling works across quants without embedding quant names in architecture logic.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-045/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-046: Define the V4 typed architecture boundary schema
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-021`, `DGR-045`
|
||||
- **Blocks (derived):** `DGR-047`, `DGR-048`, `DGR-049`
|
||||
- **Labels:** `area:deepseek-v4`, `area:boundary`, `type:protocol`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `model-free`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/046-define-the-v4-typed-architecture-boundary-schema.md`, and evidence READMEs for dependencies (DGR-021, DGR-045) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Define the exact cross-stage V4 architecture boundary while keeping per-layer attention and auxiliary caches shard-local.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Define a versioned named bundle for the mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations.
|
||||
- [ ] Explicitly exclude per-layer CSA, HCA, SWA, indexer, compressor, KV, and MTP caches/state from the WAN boundary; those remain local to the owning shard and session/epoch.
|
||||
- [ ] Reserve typed MTP boundary fields but mark MTP execution unsupported and unroutable for alpha.
|
||||
- [ ] Fingerprint independently of quant/topology and fail closed on missing, incompatible, incorrectly shaped, or stale boundary/cache expectations.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-046/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-047: Adapt the upstream V4 mHC boundary for ranged ownership
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-045`, `DGR-046`
|
||||
- **Blocks (derived):** `DGR-048`, `DGR-049`, `DGR-050`, `DGR-051`
|
||||
- **Labels:** `area:deepseek-v4`, `area:mhc`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/047-adapt-the-upstream-v4-mhc-boundary-for-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-046) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Add range-boundary adapters around upstream llama.cpp V4 mHC execution without reimplementing the V4 graph or kernels.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Represent and validate the upstream V4 4×4096 mHC boundary without flattening semantic axes.
|
||||
- [ ] Add only head/intermediate/tail range ownership and boundary conversion hooks around the pinned upstream llama.cpp graph.
|
||||
- [ ] Compare deterministic fixture vectors and single-process ranged outputs with upstream whole-model execution.
|
||||
- [ ] Document that llama.cpp owns V4 mHC graph/kernels and that quantized storage does not alter the logical boundary schema.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-047/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-048: Carry token-ID sideband through the first three hash-routed layers
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-046`, `DGR-047`
|
||||
- **Blocks (derived):** `DGR-051`
|
||||
- **Labels:** `area:deepseek-v4`, `area:routing-state`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/048-carry-token-id-sideband-through-the-first-three-hash-routed-layers.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure any stage containing the first three hash-routed layers receives exact token identities.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Head emits token IDs in the typed sideband for boundaries that require them.
|
||||
- [ ] Intermediate stages preserve or consume the sideband according to their layer range.
|
||||
- [ ] Routes splitting before, within, and after the first three layers pass deterministic partition tests.
|
||||
- [ ] Missing or mismatched token IDs fail closed rather than silently selecting incorrect experts.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-048/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-049: Keep V4 attention and auxiliary state shard-local
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-046`, `DGR-047`
|
||||
- **Blocks (derived):** `DGR-051`
|
||||
- **Labels:** `area:deepseek-v4`, `area:attention`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/049-keep-v4-attention-and-auxiliary-state-shard-local.md`, and evidence READMEs for dependencies (DGR-046, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Bind V4 CSA/HCA/SWA/indexer/compressor and KV state to the owning shard session/epoch rather than transferring caches over the WAN.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Map CSA, HCA, SWA, indexer, compressor, and KV ownership from the pinned upstream graph to each owned layer range.
|
||||
- [ ] Key all such caches/state by route session and epoch with bounded lifecycle, memory accounting, stale-epoch rejection, release, and eviction.
|
||||
- [ ] Prove cross-stage messages carry only the typed architecture boundary and cache expectations, never per-layer cache contents.
|
||||
- [ ] On route change or state loss, use explicit cache miss and re-prefill/restart; do not migrate V4 caches across workers.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-049/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,61 @@
|
||||
# 05 — Implement dense-Llama range-aware GGUF ownership
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-005` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node, I need to map only my assigned dense-Llama Shard so that aggregate consumer memory can hold a model larger than one node.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Dense-Llama range-aware ownership implementation
|
||||
- Authoritative loaded-range introspection
|
||||
- Mapped/resident memory evidence
|
||||
- evidence/DGR-005/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Register and allocate only `blk.N.*` tensors in the assigned range.
|
||||
- [ ] Load embeddings only for the head and final norm/LM head only for the tail, including tied embeddings.
|
||||
- [ ] Prefer range-aware mapping from one exact source GGUF; if derivative sub-GGUFs are used temporarily, verify source/slice hashes and avoid claiming final artifact semantics.
|
||||
- [ ] Report authoritative loaded range and endpoint ownership from the model, not operator CLI claims.
|
||||
- [ ] Demonstrate mapped/resident memory scales with owned tensors rather than full model size.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-005/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-004` must have `passes: true`; read `../evidence/DGR-004/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-050: Validate upstream V4 MoE and hash-routing execution under ranged ownership
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-045`, `DGR-047`
|
||||
- **Blocks (derived):** `DGR-051`
|
||||
- **Labels:** `area:deepseek-v4`, `area:moe`, `type:runtime`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/050-validate-upstream-v4-moe-and-hash-routing-execution-under-ranged-ownership.md`, and evidence READMEs for dependencies (DGR-045, DGR-047) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Validate the pinned upstream llama.cpp V4 MoE and hash-routing graph when layers are range-owned, without reimplementing routing or expert kernels in Meshnet.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Load range-owned tensors for 256 routed plus one shared expert and validate upstream execution activates six routed experts where specified.
|
||||
- [ ] Validate the first three upstream hash-routed layers receive exact token-ID sideband across splits before, within, and after them.
|
||||
- [ ] Compare upstream whole-model and ranged router/expert outputs, shared-expert contribution, and malformed sideband failures.
|
||||
- [ ] Document that llama.cpp owns V4 MoE, attention, hash routing, graph, and kernels; no expert-parallel WAN collective or Meshnet reimplementation is introduced.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-050/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-051: Assemble the DeepSeek V4 Flash `ShardEngine` adapter
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-034`, `DGR-037`, `DGR-047`, `DGR-048`, `DGR-049`, `DGR-050`
|
||||
- **Blocks (derived):** `DGR-052`, `DGR-060`, `DGR-065`, `DGR-069`
|
||||
- **Labels:** `area:deepseek-v4`, `area:engine`, `type:vertical-slice`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/051-assemble-the-deepseek-v4-flash-shardengine-adapter.md`, and evidence READMEs for dependencies (DGR-034, DGR-037, DGR-047, DGR-048, DGR-049, DGR-050) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Assemble range ownership hooks and boundary/local-state adapters around upstream llama.cpp V4 execution behind the stable ShardEngine interface.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Support valid head, middle, tail, and whole-model ranges over all 43 main layers using the pinned upstream llama.cpp V4 graph/kernels.
|
||||
- [ ] Integrate exact range ownership, mHC boundary, positions, required token-ID sideband, and shard-local session/epoch state adapters.
|
||||
- [ ] Report MTP reserved-but-not-executable for alpha and reject unsupported split points or cache expectations.
|
||||
- [ ] Include exact adapter/schema/runtime/artifact identity and valid split restrictions without reimplementing V4 MoE, attention, mHC, or hash routing.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-051/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-052: Pass local V4 whole-model versus ranged parity
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-038`, `DGR-051`
|
||||
- **Blocks (derived):** `DGR-053`, `DGR-054`
|
||||
- **Labels:** `area:deepseek-v4`, `area:parity`, `type:certification`, `priority:p0`, `ready-for-agent`
|
||||
- **Evidence class:** `real-model`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/052-pass-local-v4-whole-model-versus-ranged-parity.md`, and evidence READMEs for dependencies (DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4 prefill and decode correctness locally before network certification.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Compare pinned whole-model llama.cpp with at least two valid ranged partitions.
|
||||
- [ ] Prefill intermediates and locked-length greedy decode pass architecture-specific tolerances.
|
||||
- [ ] Test splits around the first three hash-routed layers and at least one CSA/HCA/SWA/indexer/compressor transition.
|
||||
- [ ] Four concurrent sessions pass KV/state isolation and cleanup.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-052/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,40 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-053: Certify a real 2–4-stage V4 route
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-030`, `DGR-043`, `DGR-052`
|
||||
- **Blocks (derived):** `DGR-054`, `DGR-062`, `DGR-067`
|
||||
- **Labels:** `area:deepseek-v4`, `area:hardware`, `type:certification`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/053-certify-a-real-2-4-stage-v4-route.md`, and evidence READMEs for dependencies (DGR-030, DGR-043, DGR-052) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove real Tracker-selected V4 execution across physical machines before alpha.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run one documented 2–4-stage certification scenario using exact compatible artifacts/recipes; the count and chosen quant are evidence inputs, not product constants.
|
||||
- [ ] Actual CPU/GPU work executes on every stage; fake workers do not satisfy acceptance.
|
||||
- [ ] Record parity, TTFT, prefill/decode speed, seam cost, memory, cache/state isolation, cancellation, and cleanup.
|
||||
- [ ] Tracker selection remains dynamic and rejects an injected incompatible backend/recipe.
|
||||
- [ ] Only the exact real-hardware lanes exercised become eligible for advertisement.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-053/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-054: Enforce the DeepSeek V4 Flash alpha gate
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M3`
|
||||
- **Dependencies:** `DGR-017`, `DGR-019`, `DGR-020`, `DGR-039`, `DGR-042`, `DGR-043`, `DGR-052`, `DGR-053`
|
||||
- **Blocks (derived):** `DGR-055`, `DGR-065`, `DGR-066`, `DGR-068`
|
||||
- **Labels:** `area:deepseek-v4`, `area:release`, `milestone:alpha`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/054-enforce-the-deepseek-v4-flash-alpha-gate.md`, and evidence READMEs for dependencies (DGR-017, DGR-019, DGR-020, DGR-039, DGR-042, DGR-043, DGR-052, DGR-053) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Release alpha only when V4 is correct and clears the pre-locked useful-speed requirement.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Evaluate correctness, model fit, TTFT, prefill/decode speed, seam cost, memory, and failure rate against DGF-004 without changing thresholds.
|
||||
- [ ] Separate quant/model-fit gains from runtime/transport gains.
|
||||
- [ ] Decision is `alpha`, `optimize measured bottleneck`, or `stop`; failures cannot be relabeled as certification.
|
||||
- [ ] Alpha documentation states MTP is reserved but not implemented and identifies every dark hardware lane.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-054/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-055: Implement compatible continuous decode batching
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-038`, `DGR-040`, `DGR-054`
|
||||
- **Blocks (derived):** `DGR-056`, `DGR-057`
|
||||
- **Labels:** `area:scheduler`, `area:batching`, `type:performance`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/055-implement-compatible-continuous-decode-batching.md`, and evidence READMEs for dependencies (DGR-038, DGR-040, DGR-054) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Batch compatible active sessions without violating route, recipe, position, or architecture state.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Batch only sessions with compatible engine, recipe, phase, boundary schema, and state layout.
|
||||
- [ ] Preserve per-session positions, sequence IDs, outputs, cancellation, and accounting.
|
||||
- [ ] Decode receives bounded service while prefill remains schedulable.
|
||||
- [ ] Deterministic 1/2/4/8 concurrency tests show no cross-session corruption.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-055/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-056: Enforce bounded admission, queues, and backpressure
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-055`
|
||||
- **Blocks (derived):** `DGR-057`, `DGR-058`, `DGR-060`
|
||||
- **Labels:** `area:scheduler`, `area:backpressure`, `type:performance`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/056-enforce-bounded-admission-queues-and-backpressure.md`, and evidence READMEs for dependencies (DGR-055) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prevent weight, KV, state, scratch, and stream buffers from exceeding admitted capacity.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Admission accounts separately for weights, KV/state, scratch, in-flight fragments, and queue slots.
|
||||
- [ ] Queue and stream limits produce structured retryable or terminal responses.
|
||||
- [ ] Prefill cannot starve decode and one route session cannot monopolize queue capacity.
|
||||
- [ ] Telemetry reports active sessions, queue depth/time, batch occupancy, memory pressure, and rejects.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-056/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-057: Benchmark batching saturation and fairness
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-055`, `DGR-056`
|
||||
- **Blocks (derived):** `DGR-061`, `DGR-063`, `DGR-070`
|
||||
- **Labels:** `area:performance`, `area:batching`, `type:benchmark`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `dense-llama+deepseek-v4`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/057-benchmark-batching-saturation-and-fairness.md`, and evidence READMEs for dependencies (DGR-055, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Identify useful concurrency and saturation points without assuming one global batch size.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run controlled concurrency 1/2/4/8 and record aggregate throughput, per-request p50/p95, TTFT, queue time, occupancy, and memory.
|
||||
- [ ] Measure dense and V4 lanes separately where hardware permits.
|
||||
- [ ] Verify bounded fairness and no prefill/decode starvation.
|
||||
- [ ] Produce machine-readable saturation recommendations consumed by capability reporting, not hardcoded constants.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-057/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-058: Harden worker and stream failure semantics
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-040`, `DGR-042`, `DGR-056`
|
||||
- **Blocks (derived):** `DGR-059`, `DGR-060`, `DGR-061`
|
||||
- **Labels:** `area:reliability`, `area:failure`, `type:hardening`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/058-harden-worker-and-stream-failure-semantics.md`, and evidence READMEs for dependencies (DGR-040, DGR-042, DGR-056) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Ensure malformed input, process death, transport reset, and cancellation terminate cleanly.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Test worker death, stream reset, malformed bundle, checksum failure, deadline, cache miss, stale epoch, and partial fragment loss.
|
||||
- [ ] Cancellation propagates across all stages and releases KV/state, queue entries, and fragment buffers.
|
||||
- [ ] Duplicate steps are idempotent; uncertain mutations are never silently replayed.
|
||||
- [ ] Billing/work outcomes distinguish completed, cancelled, failed, and unverified work.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-058/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-059: Implement alpha-safe route recovery
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-043`, `DGR-058`
|
||||
- **Blocks (derived):** `DGR-070`
|
||||
- **Labels:** `area:recovery`, `area:routing`, `type:hardening`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/059-implement-alpha-safe-route-recovery.md`, and evidence READMEs for dependencies (DGR-043, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Recover from route loss without importing or continuing unverified state.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] On route loss or local-state cache miss, restart from token zero or re-prefill on a newly compatible route according to the locked alpha contract.
|
||||
- [ ] Never import KV, CSA, HCA, SWA, indexer, compressor, recurrent, or MTP cache/state across workers.
|
||||
- [ ] Keep exact same-route duplicate delivery idempotent while cross-route continuation fails closed.
|
||||
- [ ] Test direct/relay loss, cache miss, replacement selection, client cancellation, and billing reconciliation.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-059/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,61 @@
|
||||
# 06 — Implement architecture-defined boundary input/output
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-006` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a Shard, I need to consume and emit the correct transformer boundary state so that disjoint processes reproduce whole-model execution.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Architecture boundary adapter
|
||||
- Whole-model/two-range parity tests and results
|
||||
- evidence/DGR-006/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Head accepts token IDs and owns token embedding.
|
||||
- [ ] Middle/tail bypass token embedding and accept the named boundary bundle.
|
||||
- [ ] Non-tail emits the unnormalized architecture-defined residual/boundary before final norm/head and before tail-only row pruning.
|
||||
- [ ] Tail emits logits or token output through an explicit sampling contract.
|
||||
- [ ] Dense-Llama whole-model versus two-range prefill and greedy-decode parity passes the documented tolerance.
|
||||
- [ ] The adapter interface fails closed for uncertified architectures.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-006/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
- `DGR-005` must have `passes: true`; read `../evidence/DGR-005/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-060: Certify V4 long-context state correctness
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-051`, `DGR-056`, `DGR-058`
|
||||
- **Blocks (derived):** `DGR-062`, `DGR-064`, `DGR-065`, `DGR-066`, `DGR-070`
|
||||
- **Labels:** `area:deepseek-v4`, `area:long-context`, `type:certification`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/060-certify-v4-long-context-state-correctness.md`, and evidence READMEs for dependencies (DGR-051, DGR-056, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Prove V4’s KV and auxiliary state remain correct and bounded at long contexts.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Exercise pre-locked context lengths covering multiple prefill chunks and sustained decode.
|
||||
- [ ] Validate KV plus CSA/HCA/SWA/indexer/compressor state positions across every stage.
|
||||
- [ ] Measure memory growth, fragment sizes, TTFT, decode degradation, cancellation latency, and cleanup.
|
||||
- [ ] Cache mismatch, truncation, and context-limit behavior fail deterministically.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-060/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-061: Certify existing routing with 10+ GGUF stage candidates
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-043`, `DGR-057`, `DGR-058`
|
||||
- **Blocks (derived):** `DGR-062`, `DGR-063`
|
||||
- **Labels:** `area:tracker`, `area:scale`, `type:simulation`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/061-certify-existing-routing-with-10-gguf-stage-candidates.md`, and evidence READMEs for dependencies (DGR-043, DGR-057, DGR-058) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Certify that existing backend-agnostic routing consumes GGUF compatibility and measured-cost inputs for a 10+ candidate scenario without algorithm or policy redesign.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Generate deterministic pools with 10+ compatible GGUF stage candidates plus slower, overloaded, unhealthy, and incompatible alternatives.
|
||||
- [ ] Run the existing routing implementation unchanged and verify complete coverage, compatibility, acyclicity, and use of measured inputs.
|
||||
- [ ] Regression-test that no stage count, quant, fixed split, architecture, or backend sequence is encoded in production routing policy.
|
||||
- [ ] Verify existing route recomputation reacts to measured load/failure while tracker, load-balancing, billing, relay, and provider semantics remain unchanged.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-061/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,40 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-062: Certify a dynamic real 10+ stage V4 scenario
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-053`, `DGR-060`, `DGR-061`
|
||||
- **Blocks (derived):** `DGR-067`, `DGR-070`
|
||||
- **Labels:** `area:deepseek-v4`, `area:scale`, `area:hardware`, `type:certification`, `priority:p1`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/062-certify-a-dynamic-real-10-stage-v4-scenario.md`, and evidence READMEs for dependencies (DGR-053, DGR-060, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Demonstrate that a large real route can be dynamically selected and usefully execute V4.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run one real 10+-stage V4 certification scenario; stage count, quant, and split remain scenario metadata only.
|
||||
- [ ] Every selected stage proves real work, exact compatible identity, local state ownership, and bounded resources.
|
||||
- [ ] Record end-to-end and per-stage TTFT, prefill/decode, seam cost, queueing, memory, failures, and cleanup.
|
||||
- [ ] Inject at least one incompatible or degraded candidate and verify dynamic exclusion/reselection.
|
||||
- [ ] The scenario must meet its pre-locked usefulness threshold or remain failed evidence.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-062/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-063: Profile and optimize the critical seam
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-057`, `DGR-061`
|
||||
- **Blocks (derived):** `DGR-064`
|
||||
- **Labels:** `area:performance`, `type:optimization`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `conditional`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/063-profile-and-optimize-the-critical-seam.md`, and evidence READMEs for dependencies (DGR-057, DGR-061) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Optimize only the measured dominant compute, serialization, copy, or transport bottleneck.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Profile worker compute, tensor conversion, copies, protobuf framing, compression, relay/direct transport, queueing, and synchronization.
|
||||
- [ ] Select one dominant bottleneck from evidence and lock its before/after benchmark.
|
||||
- [ ] Implement one bounded optimization without weakening correctness or protocol identity.
|
||||
- [ ] Report whether the optimization changes TTFT, prefill, decode, throughput, or memory and preserve raw data.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-063/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-064: Add negotiated activation compression
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-060`, `DGR-063`
|
||||
- **Blocks (derived):** `DGR-066`, `DGR-070`
|
||||
- **Labels:** `area:protocol`, `area:compression`, `type:optimization`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `no`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/064-add-negotiated-activation-compression.md`, and evidence READMEs for dependencies (DGR-060, DGR-063) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce V4 seam cost only when a certified compression recipe improves end-to-end performance acceptably.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Negotiate compression through recipe identity; unnegotiated or mismatched compression fails closed.
|
||||
- [ ] Preserve names, shapes, semantic axes, token-ID sideband, and checksums.
|
||||
- [ ] Measure quality drift, encode/decode cost, bytes, TTFT, and throughput against uncompressed boundaries.
|
||||
- [ ] Compression remains dark unless it meets pre-locked quality and speed thresholds.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-064/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-065: Define V4 MTP ownership and distributed state contract
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-051`, `DGR-054`, `DGR-060`
|
||||
- **Blocks (derived):** `DGR-066`
|
||||
- **Labels:** `area:deepseek-v4`, `area:mtp`, `type:contract`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `fixture`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/065-define-v4-mtp-ownership-and-distributed-state-contract.md`, and evidence READMEs for dependencies (DGR-051, DGR-054, DGR-060) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Lock MTP tensor, execution, boundary, acceptance/rejection, and session-state ownership before implementation while keeping MTP disabled for alpha.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Derive exact MTP tensor and execution ownership from the pinned V4 target and upstream llama.cpp graph, distinguishing main-layer, tail, and MTP responsibilities.
|
||||
- [ ] Define typed MTP inputs/outputs plus accepted-token, rejection, rollback/truncation, position, cache, and session/epoch lifecycle semantics.
|
||||
- [ ] Specify which MTP state remains shard-local and which typed boundary values cross a seam; prohibit WAN migration of KV or V4 auxiliary caches.
|
||||
- [ ] Keep every MTP recipe disabled and unroutable for alpha, with fail-closed capability negotiation and deterministic contract fixtures.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-065/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-066: Implement and benchmark V4 MTP
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-054`, `DGR-060`, `DGR-064`, `DGR-065`
|
||||
- **Blocks (derived):** `DGR-067`, `DGR-070`
|
||||
- **Labels:** `area:deepseek-v4`, `area:mtp`, `type:runtime`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/066-implement-and-benchmark-v4-mtp.md`, and evidence READMEs for dependencies (DGR-054, DGR-060, DGR-064, DGR-065) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Implement the locked MTP contract around upstream llama.cpp before beta, without making MTP an alpha prerequisite.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Implement the exact DGR-065 MTP ownership and typed state contract against the pinned upstream V4 graph while preserving shard-local caches.
|
||||
- [ ] Pass whole-model versus ranged MTP correctness, accepted-token, rejection, rollback/truncation, and session-isolation tests.
|
||||
- [ ] Benchmark MTP-off versus MTP-on quality, accepted-token rate, TTFT, decode speed, seam bytes, and memory on the locked lane.
|
||||
- [ ] Distinguish implemented, certified, disabled, and unsupported MTP recipes in fail-closed capability advertisement.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-066/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,40 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-067: Certify the backend capability matrix
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M4`
|
||||
- **Dependencies:** `DGR-030`, `DGR-053`, `DGR-062`, `DGR-066`
|
||||
- **Blocks (derived):** `DGR-068`, `DGR-070`
|
||||
- **Labels:** `area:hardware`, `area:admission`, `type:certification`, `priority:p1`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `real-hardware`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/067-certify-the-backend-capability-matrix.md`, and evidence READMEs for dependencies (DGR-030, DGR-053, DGR-062, DGR-066) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Publish only backend/model/recipe lanes proven on actual CPU, CUDA, ROCm, Vulkan, or Metal hardware.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Maintain independent build and certification status for CPU, CUDA, ROCm, Vulkan, and Metal.
|
||||
- [ ] Each advertised lane includes real hardware/driver/runtime/model evidence for parity, concurrency, cancellation, cleanup, and useful speed.
|
||||
- [ ] Build success alone never makes a lane routable; unavailable hardware remains explicitly dark.
|
||||
- [ ] Tracker admission consumes signed/versioned certification records rather than backend-name allowlists.
|
||||
- [ ] Mixed-backend routes require exact compatibility evidence and fail closed otherwise.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-067/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-068: Package reproducible native worker releases
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M5`
|
||||
- **Dependencies:** `DGR-030`, `DGR-054`, `DGR-067`
|
||||
- **Blocks (derived):** `DGR-070`
|
||||
- **Labels:** `area:packaging`, `area:release`, `type:delivery`, `priority:p1`, `ready-for-agent`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `generic`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/068-package-reproducible-native-worker-releases.md`, and evidence READMEs for dependencies (DGR-030, DGR-054, DGR-067) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Produce installable worker packages that preserve exact source, patch, protocol, and certification identity.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Package worker binaries/configuration for supported host/backend combinations without bundling an unverified llama.cpp tree.
|
||||
- [ ] Embed protocol, engine, upstream pin, patch-stack, build-toolchain, and certification fingerprints.
|
||||
- [ ] Installation/startup refuses mismatched artifacts, runtime recipes, or unsupported hardware lanes.
|
||||
- [ ] Produce checksums, SBOM/license attribution, reproducible build instructions, and smoke tests.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-068/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-069: Prepare narrow upstream llama.cpp collaboration patches
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M5`
|
||||
- **Dependencies:** `DGR-028`, `DGR-035`, `DGR-038`, `DGR-051`
|
||||
- **Blocks (derived):** `DGR-071`
|
||||
- **Labels:** `area:upstream`, `area:patches`, `type:collaboration`, `priority:p2`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `none`
|
||||
- **Model:** `generic+deepseek-v4`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/069-prepare-narrow-upstream-llama-cpp-collaboration-patches.md`, and evidence READMEs for dependencies (DGR-028, DGR-035, DGR-038, DGR-051) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Reduce long-term patch burden through generic upstreamable hooks without asking llama.cpp to own Meshnet.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Separate generic range loading, boundary I/O, and filtered state hooks from project worker/protocol code.
|
||||
- [ ] Provide one scoped patch, focused test, and minimal reproducer per concern against the exact upstream pin.
|
||||
- [ ] Compare the proposal with prior-art evidence and current upstream V4 support/maturity.
|
||||
- [ ] Prepare human-ready design/outreach text; actual issue/PR submission remains a human action.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-069/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,60 @@
|
||||
# 07 — Add isolated concurrent local Hot KV State
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-007` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a client, I need concurrent Route Sessions to retain independent per-Shard cache so that one request cannot clear or corrupt another.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Concurrent local KV/session manager
|
||||
- Isolation, eviction, cancellation and cleanup tests
|
||||
- evidence/DGR-007/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Map `(Route Session ID, route epoch)` to an isolated llama sequence or bounded context.
|
||||
- [ ] Allocate KV only for owned layers.
|
||||
- [ ] Support prefill append, decode append, truncate, release, TTL/LRU eviction, and explicit cache-miss response.
|
||||
- [ ] Reject stale epochs and incompatible cache recipes.
|
||||
- [ ] At least four concurrent sessions on a small model complete without token or KV cross-talk.
|
||||
- [ ] Cancellation/release of one session leaves other sessions intact and memory returns to the configured budget.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-007/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-070: Enforce the DeepSeek V4 Flash beta gate
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-human`; `passes: false`
|
||||
- **Execution mode:** `HITL`
|
||||
- **Milestone:** `M5`
|
||||
- **Dependencies:** `DGR-057`, `DGR-059`, `DGR-060`, `DGR-062`, `DGR-064`, `DGR-066`, `DGR-067`, `DGR-068`
|
||||
- **Blocks (derived):** `DGR-071`
|
||||
- **Labels:** `area:deepseek-v4`, `area:release`, `milestone:beta`, `priority:p0`, `gate:hitl`, `ready-for-human`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `required`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/070-enforce-the-deepseek-v4-flash-beta-gate.md`, and evidence READMEs for dependencies (DGR-057, DGR-059, DGR-060, DGR-062, DGR-064, DGR-066, DGR-067, DGR-068) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Promote beta only after concurrency, resilience, long-context, MTP, scale, packaging, and speed gates pass.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Evaluate all beta metrics against DGF-004 without changing thresholds after observing results.
|
||||
- [ ] Require V4 parity, batching/fairness, bounded backpressure, failure recovery, long-context correctness, MTP evidence, and dynamic 10+ stage evidence.
|
||||
- [ ] Publish the exact certified hardware/backend/recipe matrix; all other lanes remain dark.
|
||||
- [ ] Decision is `beta`, `targeted optimization`, or `stop/rollback`, with unresolved failures listed explicitly.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-070/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -1,39 +0,0 @@
|
||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||
# DGR-071: Establish upstream pin, patch, and certification maintenance
|
||||
|
||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
||||
- **Execution mode:** `AFK`
|
||||
- **Milestone:** `M5`
|
||||
- **Dependencies:** `DGR-069`, `DGR-070`
|
||||
- **Blocks (derived):** None
|
||||
- **Labels:** `area:maintenance`, `area:upstream`, `area:certification`, `type:operations`, `priority:p2`, `ready-for-agent`
|
||||
- **Evidence class:** `release`
|
||||
- **Hardware:** `optional`
|
||||
- **Model:** `deepseek-v4-flash`
|
||||
- **Upstream:** `yes`
|
||||
|
||||
## Objective / description
|
||||
|
||||
Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`, source issue `.scratch/distributed-gguf-runtime/issues/071-establish-upstream-pin-patch-and-certification-maintenance.md`, and evidence READMEs for dependencies (DGR-069, DGR-070) before changing code. Inspect live source/tests rather than trusting legacy pass states. Objective: Keep beta reproducible and detect upstream/model/hardware drift before it reaches routable nodes.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Add a documented update procedure for llama.cpp pin, numbered patches, protocol/engine versions, V4 target revision, and certification records.
|
||||
- [ ] A candidate pin update must apply/build/test in isolation and cannot replace the production pin automatically.
|
||||
- [ ] Any changed graph, tensor inventory, boundary, cache/state layout, backend behavior, or performance invalidates affected certification.
|
||||
- [ ] Add scheduled model-free checks plus opt-in real-hardware recertification instructions and rollback procedure.
|
||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||
|
||||
## Shared quality gates
|
||||
|
||||
- Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.
|
||||
- `git diff --check` passes.
|
||||
- Default tests are model-download-free, API-credit-free, and GPU-free.
|
||||
- Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.
|
||||
- Native changes pass focused out-of-tree CMake build and CTest; patch changes verify clean apply/check/reverse against the exact llama.cpp pin.
|
||||
- Runs are opt-in and record exact artifact/split hashes, runtime/upstream pin, backend/driver, hardware, network, commands, and raw metrics. Model artifacts use configured mounted-drive storage and never `/home`.
|
||||
- Preserve existing Transformers behavior and backend-agnostic Tracker routing/load balancing/billing/relay semantics unless an explicit versioned contract says otherwise. One scoped story commit is expected during execution, but this specification-materialization change is not committed.
|
||||
|
||||
## Evidence handoff
|
||||
|
||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-071/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
||||
@@ -0,0 +1,65 @@
|
||||
# 08 — Build the standalone C++ gRPC Shard worker
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-008` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node runtime, I need one supervised native process so that llama.cpp internals remain behind a stable project-owned protocol.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Standalone C++ gRPC worker
|
||||
- Fake-model Python/C++ integration tests
|
||||
- Lifecycle and bounded-failure evidence
|
||||
- evidence/DGR-008/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Worker exposes capability, health, session stream, release, cancellation, and metrics services from DGR-002.
|
||||
- [ ] Worker loads one exact Artifact/recipe/Shard identity and refuses mismatched requests.
|
||||
- [ ] Streaming path enforces bounded messages, flow control, deadlines, idempotency, and independent session cancellation.
|
||||
- [ ] Worker does not expose raw llama.cpp RPC or arbitrary GGML graph execution.
|
||||
- [ ] Graceful shutdown releases sessions; crash behavior is bounded and observable.
|
||||
- [ ] Python integration tests run against a fake model mode without model downloads.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-008/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-002` must have `passes: true`; read `../evidence/DGR-002/README.md` and verify its referenced files/commands.
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-004` must have `passes: true`; read `../evidence/DGR-004/README.md` and verify its referenced files/commands.
|
||||
- `DGR-006` must have `passes: true`; read `../evidence/DGR-006/README.md` and verify its referenced files/commands.
|
||||
- `DGR-007` must have `passes: true`; read `../evidence/DGR-007/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,61 @@
|
||||
# 09 — Integrate the native worker with Meshnet
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-009` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the existing node service, I need a GGUF Shard backend adapter so that the Tracker, relay, billing, telemetry, and capability admission remain the sole control plane.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Meshnet GGUF backend adapter
|
||||
- Registration, routing, relay, telemetry and billing tests
|
||||
- evidence/DGR-009/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Implement the existing model-backend surface without changing Transformers behavior.
|
||||
- [ ] Registration carries exact validated GGUF recipe, Shard, backend and concurrency/KV capacity.
|
||||
- [ ] Tracker forms only complete compatible routes and keeps uncertified recipes dark.
|
||||
- [ ] Direct routes use gRPC streams; relayed routes carry the same versioned protobuf frames as opaque binary through the existing relay seam.
|
||||
- [ ] Existing request/work IDs, cancellation, Generation Telemetry, billing, and per-node attribution remain correlated.
|
||||
- [ ] No vLLM, Nakshatra, prima.cpp, or custom-engine control plane becomes a core dependency.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-009/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-003` must have `passes: true`; read `../evidence/DGR-003/README.md` and verify its referenced files/commands.
|
||||
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,62 @@
|
||||
# 10 — Pass local real-model two-process acceptance
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-010` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a release engineer, I need real local distributed parity before involving network variability.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Real local two-process commands and configuration
|
||||
- Raw parity, memory and performance results
|
||||
- evidence/DGR-010/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Two local worker processes open disjoint dense-Llama ranges from the certified Artifact.
|
||||
- [ ] Prefill and at least 32 greedy decode tokens match whole-model llama.cpp within the certified tolerance.
|
||||
- [ ] Each worker retains only its own tensors and Hot KV State.
|
||||
- [ ] Four concurrent Route Sessions pass isolation and cleanup checks.
|
||||
- [ ] Report TTFT, prefill/decode throughput, seam bytes/latency, worker RSS/VRAM, KV memory, batch size, and queue time.
|
||||
- [ ] Killing one worker produces a bounded structured failure rather than a deadlock.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-010/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,62 @@
|
||||
# 11 — Pass a real heterogeneous two-machine route
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-011` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a consumer-hardware operator, I need two physical machines to execute one GGUF model so that the distributed claim is real.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Two-machine hardware/network/runtime manifest
|
||||
- Raw real-route metrics and output evidence
|
||||
- evidence/DGR-011/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Tracker selects two physical nodes with disjoint Shards and one exact certified recipe/compatibility class.
|
||||
- [ ] Actual CPU/GPU execution occurs on both nodes; synthetic workers do not satisfy acceptance.
|
||||
- [ ] Prefill/decode, concurrent-session isolation, telemetry, cancellation, and cleanup pass over the real transport/relay path.
|
||||
- [ ] Exact hardware, network, backend, model hash, route, commands, and raw metrics are recorded.
|
||||
- [ ] A model or recipe larger than one participating node's admitted memory is exercised when available.
|
||||
- [ ] Output drift is measured and incompatible mixed backends fail closed.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-011/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-010` must have `passes: true`; read `../evidence/DGR-010/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,63 @@
|
||||
# 12 — Implement continuous batching and bounded admission
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-012` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a node operator, I need active sessions batched safely so that concurrency increases aggregate throughput rather than serializing every request.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Continuous batching/admission scheduler
|
||||
- Concurrency 1/2/4/8 report
|
||||
- Queue, batch and KV-pressure evidence
|
||||
- evidence/DGR-012/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Node scheduler admits sessions against weight, KV, scratch, and queue budgets.
|
||||
- [ ] Compatible decode steps from multiple sessions form llama.cpp batches while preserving per-session positions and outputs.
|
||||
- [ ] Prefill does not starve decode; scheduling policy and bounds are explicit.
|
||||
- [ ] Backpressure prevents unbounded queued activations or KV growth.
|
||||
- [ ] Capability telemetry reports active sessions, queue depth, batch occupancy, KV pressure, prefill/decode rates, and rejected admissions.
|
||||
- [ ] Concurrency 1/2/4/8 benchmark identifies saturation and shows no cross-session corruption.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-012/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-007` must have `passes: true`; read `../evidence/DGR-007/README.md` and verify its referenced files/commands.
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
- `DGR-010` must have `passes: true`; read `../evidence/DGR-010/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,62 @@
|
||||
# 13 — Harden failure, cancellation, and restart semantics
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-013` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a client, I need failures to be bounded and explicit so that distributed speed does not come with hanging or corrupted generations.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Failure/cancel/restart test matrix
|
||||
- Resource cleanup and billing-state evidence
|
||||
- evidence/DGR-013/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Deadlines and heartbeat/health loss terminate blocked stream operations.
|
||||
- [ ] Cancellation propagates across every Shard and releases local KV and queued buffers.
|
||||
- [ ] Duplicate steps are idempotent; uncertain mutations are never replayed silently.
|
||||
- [ ] Alpha failover restarts from token zero on a newly compatible route rather than importing unverified KV.
|
||||
- [ ] Worker death, stream reset, malformed bundle, stale epoch, and cache miss tests pass.
|
||||
- [ ] Billing/work records distinguish completed, cancelled, failed, and unverified work.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-013/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-008` must have `passes: true`; read `../evidence/DGR-008/README.md` and verify its referenced files/commands.
|
||||
- `DGR-009` must have `passes: true`; read `../evidence/DGR-009/README.md` and verify its referenced files/commands.
|
||||
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,65 @@
|
||||
# 14 — Enforce the GGUF-versus-safetensors release gate
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-014` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Immutable comparison against DGR-001 thresholds
|
||||
- Machine-readable final report
|
||||
- Ship/optimize/stop recommendation
|
||||
- evidence/DGR-014/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.
|
||||
- [ ] Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.
|
||||
- [ ] Evaluate against the DGR-001 performance contract without changing thresholds after seeing results.
|
||||
- [ ] Ship recommendation is one of: promote GGUF, optimize a measured bottleneck with a new bounded task, or stop the native track.
|
||||
- [ ] Results clearly separate quantization gains from transport/runtime gains.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-014/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-001` must have `passes: true`; read `../evidence/DGR-001/README.md` and verify its referenced files/commands.
|
||||
- `DGR-011` must have `passes: true`; read `../evidence/DGR-011/README.md` and verify its referenced files/commands.
|
||||
- `DGR-012` must have `passes: true`; read `../evidence/DGR-012/README.md` and verify its referenced files/commands.
|
||||
- `DGR-013` must have `passes: true`; read `../evidence/DGR-013/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,61 @@
|
||||
# 15 — Add and certify a Qwen3/Qwen3-MoE adapter
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-015` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a client seeking top models, I need a separately certified MoE-capable architecture after the dense runtime proves stable.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Qwen3-family architecture adapter
|
||||
- Architecture-specific parity/admission/performance results
|
||||
- evidence/DGR-015/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Implement explicit tensor ownership, router/top-k, expert/shared-expert, Q/K normalization, boundary bundle, and cache semantics for the selected Qwen3 family recipe.
|
||||
- [ ] Do not reuse the dense-Llama adapter through unchecked name substitutions.
|
||||
- [ ] Whole-model versus distributed prefill/decode parity passes the architecture-specific tolerance.
|
||||
- [ ] Expert memory ownership and communication are measured.
|
||||
- [ ] Real consumer-hardware acceptance and capability admission pass before the recipe becomes routable.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
|
||||
- [ ] Model artifacts remain on the configured mounted-drive storage and never under /home
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-015/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-014` must have `passes: true`; read `../evidence/DGR-014/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -0,0 +1,60 @@
|
||||
# 16 — Produce the upstream llama.cpp collaboration package
|
||||
|
||||
Status: ready-for-agent
|
||||
|
||||
## Mandatory fresh-session context
|
||||
|
||||
- Read [RALPH-CONTEXT.md](../RALPH-CONTEXT.md) completely before changing code.
|
||||
- This issue is `DGR-016` in [prd.json](../prd.json).
|
||||
- Read the evidence README for every dependency listed below.
|
||||
- Inspect current code and `git status`; historical text and previous agent claims are not evidence.
|
||||
|
||||
## Description
|
||||
|
||||
As a maintainer, I need narrow upstreamable proposals so that our patch burden can shrink without asking llama.cpp to own Meshnet networking.
|
||||
|
||||
## Expected durable outputs
|
||||
|
||||
- Narrow upstream patches/tests
|
||||
- Generic API design note
|
||||
- Human-ready llama.cpp outreach package
|
||||
- evidence/DGR-016/README.md
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Separate generic llama.cpp hooks from Meshnet protocol/control-plane code.
|
||||
- [ ] Prepare minimal reproducible examples and tests for range-aware loading, boundary input/output, and layer-filtered KV.
|
||||
- [ ] Compare the proposal with Nakshatra and prima.cpp evidence and explain why the API is generally useful.
|
||||
- [ ] Preserve one scoped commit/patch per concern against the exact upstream pin.
|
||||
- [ ] Produce an outreach document suitable for Georgi/llama.cpp maintainers; actual sending remains a human action.
|
||||
- [ ] Targeted pytest tests pass
|
||||
- [ ] python -m compileall packages tests passes for Python changes
|
||||
- [ ] git diff --check passes
|
||||
- [ ] Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
|
||||
- [ ] Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
|
||||
- [ ] Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
|
||||
- [ ] llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
|
||||
- [ ] Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
|
||||
- [ ] Read and verify every dependency evidence README before relying on dependency behavior
|
||||
- [ ] Preserve all pre-existing working-tree changes and stage only files belonging to this story
|
||||
- [ ] Write .scratch/distributed-gguf-runtime/evidence/DGR-016/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
|
||||
- [ ] Update only this story issue to Status: done after every acceptance criterion and quality gate passes
|
||||
|
||||
## Dependency handoff
|
||||
|
||||
- `DGR-010` must have `passes: true`; read `../evidence/DGR-010/README.md` and verify its referenced files/commands.
|
||||
|
||||
## Finish contract
|
||||
|
||||
- Create the task evidence directory and durable handoff required above.
|
||||
- Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
|
||||
- Change this issue to `Status: done` only after all criteria pass.
|
||||
- Emit `<promise>COMPLETE</promise>` only after the evidence handoff exists.
|
||||
|
||||
## References
|
||||
|
||||
- [Ralph execution context](../RALPH-CONTEXT.md)
|
||||
- [PRD](../PRD.md)
|
||||
- [Implementation strategy](../implementation-strategy.md)
|
||||
- [Current architecture](../architecture.md)
|
||||
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)
|
||||
@@ -1,37 +1,35 @@
|
||||
# Distributed GGUF Runtime milestones
|
||||
# Distributed GGUF runtime milestones
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
## Gate A — measured runtime value
|
||||
|
||||
## M0 — Truth and contracts
|
||||
- DGR-001 locks the safetensors-versus-GGUF performance/fit/quality contract.
|
||||
- DGR-002 can proceed independently and defines the battle-proven backend-neutral wire protocol.
|
||||
- DGR-003 builds exact recipe identity on DGR-002.
|
||||
- Expensive native llama.cpp work remains gated by DGR-001.
|
||||
|
||||
- **Stories:** `DGR-017..DGR-020`
|
||||
- **Outcome:** Reconciled legacy truth, canonical metadata, immutable gates, and a controlled whole-model baseline.
|
||||
## Gate B — minimal native execution seam
|
||||
|
||||
## M1 — Protocol and native substrate
|
||||
- DGR-004 creates the reproducible pinned fork boundary.
|
||||
- DGR-005 implements dense-Llama range ownership.
|
||||
- DGR-006 proves architecture-defined boundary parity.
|
||||
|
||||
- **Stories:** `DGR-021..DGR-033`
|
||||
- **Outcome:** Versioned gRPC protocol, exact identities/artifacts, pinned upstream, reproducible builds, ShardEngine, and fake worker.
|
||||
## Gate C — concurrent production worker
|
||||
|
||||
## M2 — Dense vertical proof
|
||||
- DGR-007 isolates concurrent Hot KV State.
|
||||
- DGR-008 exposes the native worker over gRPC.
|
||||
- DGR-009 integrates the worker without replacing Meshnet's control plane.
|
||||
- DGR-010 passes local real-model two-process acceptance.
|
||||
|
||||
- **Stories:** `DGR-034..DGR-043`
|
||||
- **Outcome:** Dense ranged execution, parity, local state, worker integration, and GGUF inputs to existing routing.
|
||||
## Gate D — real consumer-hardware route
|
||||
|
||||
## M3 — DeepSeek V4 Flash alpha
|
||||
- DGR-011 passes two-physical-machine execution.
|
||||
- DGR-012 adds continuous batching and bounded admission.
|
||||
- DGR-013 hardens failure and cancellation.
|
||||
|
||||
- **Stories:** `DGR-044..DGR-054`
|
||||
- **Outcome:** Pinned V4 adapter around upstream llama.cpp, real route certification, and pre-locked alpha decision with MTP off.
|
||||
## Gate E — product release decision
|
||||
|
||||
## M4 — Performance and beta hardening
|
||||
- DGR-014 compares distributed GGUF against the current distributed safetensors route under locked thresholds.
|
||||
- DGR-015 adds Qwen3/Qwen3-MoE only after the dense runtime passes.
|
||||
- DGR-016 prepares narrow upstream llama.cpp collaboration material.
|
||||
|
||||
- **Stories:** `DGR-055..DGR-067`
|
||||
- **Outcome:** Batching, backpressure, recovery, scale certification, optimization, MTP, and hardware matrix.
|
||||
|
||||
## M5 — Release and maintenance
|
||||
|
||||
- **Stories:** `DGR-068..DGR-071`
|
||||
- **Outcome:** Reproducible packages, upstream collaboration, beta decision, and sustainable recertification.
|
||||
|
||||
## Release semantics
|
||||
|
||||
Alpha is DGR-054 and requires correctness plus the immutable useful-speed contract; MTP remains off. Beta is DGR-070 and requires DGR-066 MTP evidence plus concurrency, recovery, long-context, 10+ scenario, compression/optimization where certified, hardware matrix, and packaging. Failed gates remain failed evidence; they are not relabeled as success.
|
||||
No later gate may be claimed from synthetic workers or documentation-only evidence.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,37 +1,414 @@
|
||||
# Distributed GGUF Runtime technical challenges
|
||||
# Distributed GGUF Technical Challenge Register
|
||||
|
||||
> **Specification status:** planning artifacts only. No distributed GGUF runtime is implemented. DGR-017 cleanup is complete; no runtime implementation story has completion credit. `prd.json` is authoritative.
|
||||
> **Historical challenge register.** Route Session, binary activation, local Hot KV State, and transport performance work have advanced since this file was written. Current implementation gates live in [PRD.md](PRD.md), [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), and [prd.json](prd.json). Preserve this file for detailed risk context; do not treat its “current constraint” section as live system state.
|
||||
|
||||
## Key challenges and planned answers
|
||||
This document focuses on the engineering problems that decide whether the distributed GGUF path is viable. The important distinction is:
|
||||
|
||||
1. **Truth/provenance:** old passes are unreliable. DGR-017 audits before preserving or cleaning anything.
|
||||
2. **Baseline comparability:** lock lanes first (DGR-019), execute whole-model-only baseline next (DGR-020), and separate quant fit from runtime effects.
|
||||
3. **Exact artifacts:** complete split manifests/hashes, resumable mounted-drive provisioning, no `/home` models, dynamic quant/topology (DGR-025..026).
|
||||
4. **Fast-moving upstream:** exact manifest plus numbered patches in an ignored workspace; no vendoring/submodule/permanent fork (DGR-027..030, DGR-071).
|
||||
5. **Stable project boundary:** gRPC/Protobuf and `ShardEngine` prevent upstream ABI leakage (DGR-021..033).
|
||||
6. **V4 correctness:** llama.cpp owns mHC/MoE/attention/hash graphs and kernels; Meshnet adds ranged ownership and adapters, then validates whole-model parity (DGR-044..052).
|
||||
7. **Boundary versus caches:** only mHC residual/positions/needed token IDs/schema expectations cross; KV and CSA/HCA/SWA/indexer/compressor state stay local/session-keyed (DGR-046, DGR-049).
|
||||
8. **Recovery:** stale/missing local state causes cache miss and re-prefill/restart, never WAN migration (DGR-058..060).
|
||||
9. **Scale without policy churn:** feed measured GGUF inputs into and certify existing backend-agnostic routing at 10+ candidates (DGR-043, DGR-061..062).
|
||||
10. **Performance/concurrency:** bounded admission, continuous batching, fairness, long-context profiling, then evidence-selected optimization/compression (DGR-055..064).
|
||||
11. **MTP:** define ownership/state first, keep off for alpha, implement/benchmark before beta (DGR-065..066).
|
||||
12. **Hardware truth:** compile all five backend families but advertise only exact real-hardware-certified lanes (DGR-030, DGR-067).
|
||||
- **Model artifacts move like torrents.**
|
||||
- **Inference state moves like a pipeline.**
|
||||
- **Hot KV state does not move unless we are explicitly checkpointing or repairing a route.**
|
||||
|
||||
## Locked scope
|
||||
## Current Constraint
|
||||
|
||||
- Existing Meshnet Tracker routing, load balancing, billing, telemetry, relay, and provider semantics are backend-agnostic and are **not redesigned**. GGUF contributes exact compatibility, range/capacity, queue/load, seam-cost, health/reliability, and certification inputs only.
|
||||
- The data plane is a standalone project-owned C++ Shard worker with gRPC/Protobuf and a project-owned `ShardEngine` boundary.
|
||||
- llama.cpp is fetched at one exact commit into an ignored workspace from an in-repo manifest, then a numbered minimal patch stack is applied. There is no submodule, vendored tree, or permanent-fork dependency.
|
||||
- llama.cpp owns DeepSeek V4 graphs, mHC, MoE, attention, hash routing, and kernels. Meshnet adds only range-ownership hooks, typed boundary/local-state adapters, worker integration, and parity/certification.
|
||||
- Quantization and placement are dynamic recipe inputs. The 2–4 and 10+ stage layouts are certification scenarios, never product constants.
|
||||
- Per-shard Hot KV and V4 CSA/HCA/SWA/indexer/compressor state remain local and keyed by route session/epoch. The WAN seam carries the typed mHC 4×4096 residual boundary, positions, token-ID sideband where required, and schema/cache expectations—not per-layer caches.
|
||||
- Route changes use cache miss plus re-prefill/restart. There is no WAN KV or V4 auxiliary-cache migration.
|
||||
- CPU/CUDA/ROCm/Vulkan/Metal compile lanes are planned; only exact real-hardware-certified backend/model/recipe lanes may be advertised.
|
||||
- Alpha requires correctness and the pre-locked useful-speed gate. MTP is reserved and off for alpha; its ownership contract, implementation, and benchmark are required before beta.
|
||||
The existing full local PyTorch path lets Transformers own generation and local KV cache.
|
||||
|
||||
## Target identities
|
||||
The existing distributed PyTorch path does not. It manually calls shard layers with cache disabled and recomputes the whole growing prompt for every generated token. It passes hidden activations across shard boundaries, not KV cache, but those activations currently include the full sequence on every decode step.
|
||||
|
||||
- DeepSeek V4 official target SHA: `60d8d70770c6776ff598c94bb586a859a38244f1`.
|
||||
- llama.cpp V4 support lineage began at PR 24162 / merge `8c146a8366304c871efc26057cc90370ccf58dad`; DGR-027 later pins one exact validated current commit.
|
||||
- V4 scope: 43 main layers plus MTP; mHC 4×4096 boundary; 256 routed + 1 shared experts with six routed active; token IDs required for the first three hash-routed layers.
|
||||
- Exact split-GGUF artifacts are provisioned to mounted-drive storage with a complete hashed manifest and resumable verification; no model artifact may be placed under `/home`.
|
||||
For a 128K context and `hidden_size=6144`, one bfloat16 activation crossing one shard boundary is roughly:
|
||||
|
||||
```text
|
||||
131072 tokens * 6144 hidden * 2 bytes = 1.5 GiB
|
||||
```
|
||||
|
||||
That is acceptable once during chunked prefill only if chunked and streamed. It is not acceptable once per generated token.
|
||||
|
||||
## Challenge 1: Decode Must Be O(1) Per Token Across Each Seam
|
||||
|
||||
Problem:
|
||||
|
||||
During decode, sending `[batch, sequence, hidden]` over the network scales with context length. At 128K, the network dominates everything.
|
||||
|
||||
Solution:
|
||||
|
||||
Split execution into explicit **prefill** and **decode-step** phases.
|
||||
|
||||
- Prefill accepts prompt chunks and builds local cache on every shard.
|
||||
- Decode-step accepts exactly one new token or one-step activation.
|
||||
- Every shard reads its own hot KV state, appends one position, and forwards a one-step activation.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A two-node route prefills a 4K prompt once.
|
||||
- The next 100 generated tokens do not resend a 4K activation.
|
||||
- Decode seam payload is proportional to `hidden_size`, not `context_length * hidden_size`.
|
||||
|
||||
## Challenge 2: Stable Route Session State
|
||||
|
||||
Problem:
|
||||
|
||||
KV cache only works if every hop agrees that multiple calls belong to the same route session. A fresh request id per hop or per token destroys cache locality.
|
||||
|
||||
Solution:
|
||||
|
||||
Introduce a route-session lifecycle.
|
||||
|
||||
```text
|
||||
create route session
|
||||
-> tracker pins inference route
|
||||
-> head node assigns session_id and route_id
|
||||
-> every hop allocates local cache for its layer range
|
||||
-> prefill chunks append cache
|
||||
-> decode steps append cache
|
||||
-> close session releases cache
|
||||
```
|
||||
|
||||
Minimum state key:
|
||||
|
||||
```text
|
||||
session_id
|
||||
route_id
|
||||
model_preset
|
||||
model_revision
|
||||
backend_id
|
||||
cache_abi
|
||||
layer_start
|
||||
layer_end
|
||||
position
|
||||
```
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A node can report active sessions and cache bytes by session.
|
||||
- Closing a session frees the per-shard cache.
|
||||
- Replaying a decode-step with the wrong route/session fails before model execution.
|
||||
|
||||
## Challenge 3: KV Cache Ownership
|
||||
|
||||
Problem:
|
||||
|
||||
A centralized KV cache sounds attractive for failover, but it puts remote storage in the tightest loop of generation. It also creates privacy and consistency problems.
|
||||
|
||||
Solution:
|
||||
|
||||
Hot KV state is owned by the node that owns the shard for that route session.
|
||||
|
||||
```text
|
||||
Node A: layers 0..15 hot KV for session S
|
||||
Node B: layers 16..31 hot KV for session S
|
||||
Node C: layers 32..77 hot KV for session S
|
||||
```
|
||||
|
||||
The tracker may know where active KV lives, but it does not serve it during decode.
|
||||
|
||||
Cache servers may store:
|
||||
|
||||
- prefix snapshots
|
||||
- failover checkpoints
|
||||
- audit samples
|
||||
- cold reusable context blocks
|
||||
|
||||
They must not be required for every generated token.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- Killing a cache server does not affect an active decode route.
|
||||
- Killing a route node fails the route in alpha unless a compatible snapshot exists.
|
||||
|
||||
## Challenge 4: Prefill Is Still Large
|
||||
|
||||
Problem:
|
||||
|
||||
Even with correct decode, prefill can move a lot of data. A 128K prompt cannot be sent as one activation blob through many shard boundaries.
|
||||
|
||||
Solution:
|
||||
|
||||
Use the existing binary activation direction from ADR-0008:
|
||||
|
||||
- bfloat16 activation body
|
||||
- shape/dtype/session metadata in headers
|
||||
- zstd level 1 optional compression
|
||||
- chunked prefill
|
||||
- backpressure between hops
|
||||
|
||||
For large contexts, prefill should stream in chunks such as 128, 256, or 512 tokens. The right chunk size is a benchmark output, not a constant baked into the domain model.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- Peak per-hop prefill memory is bounded by chunk size.
|
||||
- A slow downstream node applies backpressure instead of letting upstream buffer the whole prompt.
|
||||
|
||||
## Challenge 5: GGUF Artifact Splits Are Not Execution Shards
|
||||
|
||||
Problem:
|
||||
|
||||
GGUF split files can divide model data by size or tensor count. That is useful for storage and transfer, but it is not the same as a network Shard. A Shard in this project is a contiguous layer range with reward, route, and cache meaning.
|
||||
|
||||
Solution:
|
||||
|
||||
Define an artifact manifest that maps storage chunks to semantic model parts.
|
||||
|
||||
Required concepts:
|
||||
|
||||
```text
|
||||
artifact_id
|
||||
model_preset
|
||||
upstream_repo
|
||||
upstream_revision
|
||||
license
|
||||
runtime_backend
|
||||
quantization
|
||||
context_cap
|
||||
file_hashes
|
||||
piece_hashes
|
||||
tensor_to_layer_map
|
||||
layer_to_artifact_map
|
||||
tokenizer_artifacts
|
||||
cache_descriptor
|
||||
```
|
||||
|
||||
The Shard Swarm seeds artifacts. The Inference Route executes shards.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- Given `layers 16..31`, a node can compute the exact artifact pieces it must download and verify.
|
||||
- Given a local artifact directory, the node can prove which layer ranges it can serve.
|
||||
|
||||
## Challenge 6: llama.cpp Is Optimized For Whole-Graph Execution
|
||||
|
||||
Problem:
|
||||
|
||||
`llama-server` is excellent for local inference, but a distributed route needs lower-level capabilities:
|
||||
|
||||
- load selected layers/tensors or mmap them without full materialization
|
||||
- accept hidden states from a previous shard
|
||||
- execute only a layer range
|
||||
- emit hidden states at a boundary
|
||||
- own KV/state for only that layer range
|
||||
- report cache layout and memory requirements
|
||||
|
||||
Those are not the normal public serving abstractions.
|
||||
|
||||
Solution:
|
||||
|
||||
Stage the llama.cpp path instead of jumping directly to internet-scale distributed GGUF.
|
||||
|
||||
1. Use llama.cpp as a full local GGUF backend for immediate CPU performance.
|
||||
2. Build a localhost layer-boundary prototype on a simple supported GGUF model.
|
||||
3. Identify the minimal `libllama`/ggml hooks needed for layer-range execution.
|
||||
4. Collaborate upstream on a stable extension rather than carrying a long-lived fork.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- Process A runs layers `0..k`, exports hidden states.
|
||||
- Process B imports those hidden states, runs `k+1..n`, and produces logits close to full single-process execution.
|
||||
- Both processes maintain only their own cache state.
|
||||
|
||||
## Challenge 7: Model Architectures Are Not Uniform
|
||||
|
||||
Problem:
|
||||
|
||||
Dense Llama-style attention, MoE, MLA/DSA, and hybrid linear/full attention do not have the same cache shape, routing cost, or layer cost.
|
||||
|
||||
GLM-5.2 uses compressed DSA/MLA-style state. Ornith uses a hybrid attention pattern. Parameter count alone is a poor routing metric.
|
||||
|
||||
Solution:
|
||||
|
||||
Keep model-specific cache internals inside the backend. The tracker should route based on backend-advertised capabilities and measured telemetry, not on hardcoded tensor formulas.
|
||||
|
||||
Backend capability report:
|
||||
|
||||
```text
|
||||
model_arch
|
||||
supported_runtime
|
||||
supports_prefill
|
||||
supports_decode_step
|
||||
supports_layer_range
|
||||
supports_partial_artifacts
|
||||
cache_abi
|
||||
cache_bytes_per_token_estimate
|
||||
prefill_tokens_per_second
|
||||
decode_tokens_per_second
|
||||
active_memory_floor
|
||||
```
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- The tracker can reject a route because one node lacks the required cache ABI.
|
||||
- A model support audit can say "artifact available, local full inference works, distributed layer-boundary unsupported" without ambiguity.
|
||||
|
||||
## Challenge 8: Tensor Parallelism Is Not The Same Product
|
||||
|
||||
Problem:
|
||||
|
||||
Projects like Distributed Llama and prima.cpp lean toward local-cluster tensor/ring parallelism. That can work on a trusted LAN, but it usually requires tight synchronization every layer. On a public internet volunteer route, that becomes fragile and hard to reward.
|
||||
|
||||
Solution:
|
||||
|
||||
For the public network, make a Shard a contiguous layer range. Tensor parallelism can exist inside one node, one trusted colocated pod, or one future "composite node", but not as the first public routing primitive.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A public route can be represented as ordered layer coverage.
|
||||
- Billing can attribute work to layer ranges.
|
||||
- No cross-node all-reduce is required on every layer for v1.
|
||||
|
||||
## Challenge 9: Heterogeneity And Stragglers
|
||||
|
||||
Problem:
|
||||
|
||||
A route is only as fast as its slowest hop during decode. A weak node holding a bottleneck shard can make a 1.6T model technically available but unusable.
|
||||
|
||||
Solution:
|
||||
|
||||
Route selection must use measured telemetry, not static declarations.
|
||||
|
||||
Metrics:
|
||||
|
||||
- prefill throughput
|
||||
- decode throughput
|
||||
- queue depth
|
||||
- disk read rate
|
||||
- memory pressure
|
||||
- network latency to neighbors
|
||||
- route failure rate
|
||||
- cache warmth
|
||||
|
||||
The tracker should prefer complete routes that avoid weak nodes, and the rebalancer should increase redundancy for bottleneck layer ranges.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A slow node is removed from a candidate route even if it has the needed layer range.
|
||||
- The coverage map can show "covered but under-provisioned" separately from "coverage gap".
|
||||
|
||||
## Challenge 10: Reliability And Failover
|
||||
|
||||
Problem:
|
||||
|
||||
If hot KV is local, route repair is not free. A replacement node cannot continue decoding unless it has compatible cache state.
|
||||
|
||||
Solution:
|
||||
|
||||
Alpha behavior should be simple:
|
||||
|
||||
- route failure during prefill: fail and retry from scratch
|
||||
- route failure during decode: fail unless compatible snapshot exists
|
||||
- tracker restart: active sessions may be lost
|
||||
- node restart: local hot KV is lost
|
||||
- client-visible telemetry reports the last known phase and failure reason
|
||||
|
||||
Later behavior:
|
||||
|
||||
- periodic prefix snapshots
|
||||
- snapshot generation ids
|
||||
- cache ABI compatibility checks
|
||||
- route repair only when the replacement node has the same model revision, layer range, backend cache ABI, and snapshot position
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- Failures produce explicit route-session errors.
|
||||
- No node silently continues from missing or incompatible cache state.
|
||||
|
||||
## Challenge 11: Privacy, Fraud, And Audit
|
||||
|
||||
Problem:
|
||||
|
||||
Hidden activations and KV state can leak information. Public volunteer inference is not private by default. Also, a node can return bad activations while still appearing available.
|
||||
|
||||
Solution:
|
||||
|
||||
Separate product modes:
|
||||
|
||||
- public swarm: low privacy, broad access, audited
|
||||
- private swarm: trusted nodes, stronger privacy expectation
|
||||
- paid trusted route: selected nodes with stronger guarantees
|
||||
|
||||
Use existing validation-event and slash-proof concepts for audit, but adapt them to distributed routes:
|
||||
|
||||
- record model preset, route, node wallets, prompt metadata, output, and sampling seed
|
||||
- sample full-route replays where feasible
|
||||
- compare output/logits within model-specific tolerance
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A client can choose public or private route policy.
|
||||
- A validation event contains enough information to reproduce route membership and observed output.
|
||||
|
||||
## Challenge 12: Economics Must Not Reward The Wrong Bottleneck
|
||||
|
||||
Problem:
|
||||
|
||||
Layer count, parameter count, active MoE experts, cache memory, disk serving, and network transfer are different costs. A naive equal split across nodes will be wrong.
|
||||
|
||||
Solution:
|
||||
|
||||
Start with simple compute accounting:
|
||||
|
||||
```text
|
||||
node_reward_weight =
|
||||
owned_layer_work
|
||||
* prefill_tokens
|
||||
+ owned_layer_work
|
||||
* decode_tokens
|
||||
```
|
||||
|
||||
Then refine with:
|
||||
|
||||
- measured throughput
|
||||
- active MoE cost
|
||||
- storage/seeding contribution
|
||||
- cache memory reservation
|
||||
- reliability
|
||||
|
||||
Keep artifact seeding rewards separate from inference rewards until fraud and metering are clear.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A node that only seeds artifacts is not paid as if it executed inference.
|
||||
- A node that executes a heavier shard can earn more than a node executing a light shard.
|
||||
|
||||
## Challenge 13: Long Requests Need Streaming Or Realtime Feedback
|
||||
|
||||
Problem:
|
||||
|
||||
Large distributed routes may spend meaningful time in artifact loading, prefill, queueing, or slow decode. The product can tolerate latency, but users should not wait blindly.
|
||||
|
||||
Solution:
|
||||
|
||||
Streaming token deltas is preferred when the backend and client transport support it. Generation Telemetry is required regardless of whether token deltas are streamed.
|
||||
|
||||
Minimum telemetry:
|
||||
|
||||
```text
|
||||
session_id
|
||||
route_id
|
||||
model_preset
|
||||
phase = queued | loading | prefill | decode | finalizing | failed
|
||||
prefill_tokens_done
|
||||
prefill_tokens_total
|
||||
generated_tokens
|
||||
rolling_tokens_per_second
|
||||
average_tokens_per_second
|
||||
active_route_nodes
|
||||
failure_reason
|
||||
```
|
||||
|
||||
The gateway may expose token deltas and telemetry through Server-Sent Events or WebSocket. Simple clients may use a polling endpoint for telemetry and receive the final answer only when complete.
|
||||
|
||||
Acceptance test:
|
||||
|
||||
- A client can show live progress before the first output token is available.
|
||||
- During decode, the user sees streamed token deltas when supported.
|
||||
- During decode, the user sees rolling tokens/sec even if output text is not streamed.
|
||||
- A failed route returns a final error and the last known phase/reason.
|
||||
|
||||
## Engineering Order
|
||||
|
||||
1. Fix distributed PyTorch cache semantics first. This proves the route-session model without llama.cpp internals.
|
||||
2. Add local full-model llama.cpp/GGUF serving for immediate CPU improvement.
|
||||
3. Add Generation Telemetry for route sessions so long requests are observable.
|
||||
4. Preserve binary HTTP activation transfer while splitting prefill/decode and measuring payload sizes.
|
||||
5. Add artifact manifest and Shard Swarm metadata.
|
||||
6. Prototype llama.cpp layer-boundary execution locally.
|
||||
7. Network the GGUF route only after the cache/session protocol works.
|
||||
8. Audit DeepSeek-V4-Flash as the first serious large-model target.
|
||||
9. Audit GLM-5.2 and Ornith support after simpler GGUF models pass the route test.
|
||||
|
||||
@@ -1,148 +0,0 @@
|
||||
# Colibrì implementation audit
|
||||
|
||||
Research date: 2026-07-15. Primary source: [JustVugg/colibri](https://github.com/JustVugg/colibri) at `main` (README and linked source files). The repository is a model-specific runtime, not a wrapper around llama.cpp.
|
||||
|
||||
## Answer in one paragraph
|
||||
|
||||
Colibrì runs inference in a project-owned, dependency-free C engine (`c/glm.c` for GLM-5.2 and `c/olmoe.c` for OLMoE). Python is used for the one-time FP8/safetensors-to-Colibrì-container conversion and for the standard-library OpenAI HTTP gateway; it is not in the runtime inference path. The engine keeps dense/shared weights resident, while routed MoE experts are stored as individually addressable quantized records on disk and loaded into a per-layer LRU working set. RAM and optional VRAM are hot tiers; disk is a cold immutable backing store. This is local memory/storage tiering on one machine—not distributed expert execution over a network.
|
||||
|
||||
## What performs inference
|
||||
|
||||
- The README explicitly describes a “single C file (`c/glm.c`, ~2,400 lines)” with no BLAS, Python, or GPU requirement; runtime is pure C and Python is conversion-only ([README](https://github.com/JustVugg/colibri#the-idea), [runtime/setup section](https://github.com/JustVugg/colibri#quick-start)).
|
||||
- The C source declares the GLM MoE forward path, MLA attention, sigmoid router, shared expert, and “expert routed in streaming dal disco (per-expert)” ([`c/glm.c`](https://github.com/JustVugg/colibri/blob/main/c/glm.c)). It defines its own quantized tensor (`QT`) and expert-slot (`ESlot`) structures, rather than importing GGUF/llama.cpp data structures.
|
||||
- Optional CUDA and Metal backends are native Colibrì backends. On Windows, CUDA is a separately built `coli_cuda.dll` loaded through `c/backend_loader.c`; the host falls back to CPU if it is absent ([README GPU section](https://github.com/JustVugg/colibri#windows-11-native-no-wsl), [`backend_loader.c`](https://github.com/JustVugg/colibri/blob/main/c/backend_loader.c), [`backend_cuda.cu`](https://github.com/JustVugg/colibri/blob/main/c/backend_cuda.cu)).
|
||||
- `c/openai_server.py` is only an HTTP adapter. The README says inference remains in the same C engine and that one persistent process owns one mutable KV context ([API section](https://github.com/JustVugg/colibri#openai-compatible-api)).
|
||||
|
||||
## How experts are loaded on one laptop
|
||||
|
||||
The placement policy is a three-tier hierarchy:
|
||||
|
||||
1. Dense attention, embeddings, shared experts, and other always-used weights stay resident in RAM (roughly 9.9 GB int4 for the stated GLM-5.2 setup).
|
||||
2. Routed experts are separate records (about 19 MB each at int4 in the README's GLM-5.2 example). A per-layer LRU cache holds the currently useful experts in RAM; an optional pinned hot store keeps frequently used experts from eviction. CUDA can make VRAM an additional hot tier.
|
||||
3. Remaining experts stay on disk (about 370 GB in the stated int4 container). A token routes to top-k experts per MoE layer; cache misses issue bounded background reads, then the loaded records are multiplied before the layer completes.
|
||||
|
||||
The README quantifies the trade-off: 75 layers × 8 experts means approximately 11 GB of cold reads per token, and the reported cold rate is only 0.05–0.1 token/s on a ~1 GB/s disk ([expert layout](https://github.com/JustVugg/colibri#the-idea), [numbers/resource policy](https://github.com/JustVugg/colibri#honest-numbers), [resource policy](https://github.com/JustVugg/colibri#resource-policy)). `PILOT=1` predicts the next layer's routes (reported 71.6% top-8 recall) and prefetches them while the current layer computes ([README router-lookahead](https://github.com/JustVugg/colibri#resource-policy)). Prefill and MTP verification use “batch-union MoE”: each unique expert in a batch is read once and applied to all positions that selected it.
|
||||
|
||||
The learning cache persists expert-use counts in `.coli_usage`, pins hot experts at startup, and can periodically repin them using a session-local LFRU score. This is an adaptive placement policy, not a change to router semantics. The model directory is converted offline one source shard at a time; the original 756 GB FP8 checkpoint need not coexist on disk ([converter and warmup](https://github.com/JustVugg/colibri#quick-start), [cache policy](https://github.com/JustVugg/colibri#resource-policy)).
|
||||
|
||||
## Model format and scope
|
||||
|
||||
Colibrì does **not** consume GGUF. Its converter reads Hugging Face safetensors/config data and writes a Colibrì-specific quantized container/directory (the README calls it an “int4 container” and runs with `COLI_MODEL=/path/to/...`). The C loader and `QT`/`ESlot` types are custom to this repository ([converter](https://github.com/JustVugg/colibri/blob/main/c/tools/convert_fp8_to_int4.py), [`c/glm.c`](https://github.com/JustVugg/colibri/blob/main/c/glm.c)). Current fidelity is tied to `glm_moe_dsa` (GLM-5.2); OLMoE has a separate implementation. This should be treated as an architectural experiment and source of techniques, not as a drop-in GGUF backend.
|
||||
|
||||
## What is and is not distributed
|
||||
|
||||
There is no peer protocol, tensor RPC, layer hand-off, remote expert service, or multi-host scheduler in the repository. `coli serve` serializes requests through a local process (bounded FIFO queue; optional isolated KV slots), and the README explicitly says concurrent requests queue because the engine owns mutable KV state ([queue/KV section](https://github.com/JustVugg/colibri#openai-compatible-api)). The “distributed-looking” behavior is storage-tier streaming inside one address space: disk I/O overlaps compute, but every expert matmul and the KV state remain on the same laptop.
|
||||
|
||||
## Ideas worth carrying into Meshnet
|
||||
|
||||
1. **Expert-level placement, not only layer-level placement.** For MoE models, advertise and assign individual expert records (or expert groups) independently from dense/layer shards. A node can contribute capacity for hot experts without owning the whole model.
|
||||
2. **Immutable cold backing + bounded hot cache.** Treat the model artifact as a content-addressed, immutable source; keep a bounded LRU/LFRU cache of resident experts. Placement changes then become cache promotion/eviction rather than model mutation.
|
||||
3. **Router-aware prefetch.** Add an optional next-seam prefetch hint after layer L predicts likely expert IDs for layer L+1. Hints must be advisory and cancellable; correctness still waits for the router's actual top-k.
|
||||
4. **Batch-union requests.** During prefill or verification, deduplicate expert IDs across tokens so one transfer serves many positions. This maps naturally to a Meshnet seam batch message.
|
||||
5. **Persisted usage heat.** Track expert hit/miss/latency histograms and use them for placement recommendations. Keep this separate from billing/reputation and avoid treating historical heat as a correctness signal.
|
||||
6. **Explicit cold-path telemetry.** Report disk/network service time separately from foreground-visible wait. Colibrì's profile distinguishes overlap; Meshnet should expose the same distinction per activation seam.
|
||||
7. **Resource planning as a first-class contract.** `coli plan`/`doctor` produce a versioned placement/budget report before loading. Meshnet admission could use an equivalent plan: dense footprint, expert cache budget, KV reserve, bandwidth, and safe concurrency.
|
||||
|
||||
## Follow-up: distributed expert routing
|
||||
|
||||
### The transferable idea
|
||||
|
||||
For an MoE layer, the node that owns and executes that layer's router can select
|
||||
the token or batch's top-*k* experts, dispatch the same layer input to the
|
||||
providers that own those experts, then gather and weighted-sum the returned
|
||||
expert outputs before continuing with the next layer. This is **expert
|
||||
parallelism**. It is not a responsibility of the route's initial/head node:
|
||||
every MoE layer has its own router and therefore makes its own selection.
|
||||
|
||||
```text
|
||||
activation reaches MoE layer L
|
||||
|
|
||||
v
|
||||
L's Shard computes attention + router scores
|
||||
|
|
||||
v
|
||||
top-k expert IDs -> expert-provider groups
|
||||
|
|
||||
v
|
||||
scatter inputs -> run expert(s) -> gather weighted outputs
|
||||
|
|
||||
v
|
||||
complete layer L and continue the Inference Route
|
||||
```
|
||||
|
||||
Colibrì proves the useful local analogue: experts are independently addressable
|
||||
quantized records; its router selects them at execution time; a bounded
|
||||
RAM/VRAM cache, pinning, and read-ahead decide whether a selected expert comes
|
||||
from fast memory or its cold disk backing. It does **not** perform the
|
||||
networked version: all expert execution and KV state remain local to one
|
||||
process ([Colibrì README: expert layout](https://github.com/JustVugg/colibri#the-idea),
|
||||
[Colibrì README: server/KV model](https://github.com/JustVugg/colibri#openai-compatible-api),
|
||||
[`c/glm.c`](https://github.com/JustVugg/colibri/blob/main/c/glm.c)).
|
||||
|
||||
### Why this is not the first public-network primitive
|
||||
|
||||
Naively making every individual expert independently reachable over a WAN
|
||||
would cause a scatter/gather at every MoE layer for every decode step. The
|
||||
Colibrì GLM-5.2 example has 75 MoE layers and selects eight routed experts per
|
||||
layer; that illustrates the potential fan-out, even though Colibrì satisfies
|
||||
those selections locally ([Colibrì README: expert layout and cold-path
|
||||
numbers](https://github.com/JustVugg/colibri#the-idea)). Network latency,
|
||||
tail-provider delay, failure/retry behavior, and per-expert accounting would
|
||||
become part of the autoregressive critical path.
|
||||
|
||||
This reinforces ADR-0024's current choice: public Inference Routes use
|
||||
contiguous layer/pipeline Shards; tensor and expert parallelism are deferred to
|
||||
trusted composite providers or managed clusters, where the network is
|
||||
low-latency and one provider can own the collective's operational contract
|
||||
([ADR-0024: distributed parallelism](../adr/0024-distributed-gguf-runtime.md)).
|
||||
|
||||
### Safe staged adoption
|
||||
|
||||
1. **Local tiered experts inside a contiguous MoE Shard.** Keep a Shard's
|
||||
expert execution local, but apply Colibrì-style immutable cold storage,
|
||||
bounded LRU/LFRU caches, hot-expert pinning, batch-union loading, and
|
||||
router-aware prefetch.
|
||||
2. **Expert routing within one trusted composite provider.** Let a managed
|
||||
LAN/cluster expose a single Meshnet provider identity while it handles
|
||||
expert scatter/gather internally. This is the earliest setting where the
|
||||
technique should be benchmarked end-to-end.
|
||||
3. **Public remote expert providers only behind a release gate.** If measured
|
||||
performance warrants it, expose versioned remote *expert packs* rather than
|
||||
unconstrained per-expert endpoints. The owning MoE-layer Shard must retain
|
||||
control of selection and aggregation.
|
||||
|
||||
The public form would require all of the following before it can be routable:
|
||||
|
||||
- content-addressed artifact, quantization, architecture, and runtime-recipe
|
||||
identity for every expert pack;
|
||||
- stable ownership, replication, cache residency, and health reports;
|
||||
- a versioned scatter/gather protocol carrying layer ID, expert IDs, route
|
||||
session/epoch, token positions, inputs, weights, deadlines, and cancellation;
|
||||
- batch-union deduplication by provider, bounded fan-out, backpressure, and
|
||||
straggler/failure policy;
|
||||
- separate telemetry for cache hit/miss, transfer bytes, overlap, remote
|
||||
service time, tail latency, and aggregation time; and
|
||||
- proof that the resulting output, KV isolation, and admission behavior match
|
||||
the certified whole-model/contiguous-Shard execution.
|
||||
|
||||
The strategy is therefore to borrow Colibrì's **expert-as-movable-artifact and
|
||||
memory-tiering** idea, while preserving Meshnet's Route Session ownership and
|
||||
contiguous public layer Shards. Its local cache should be an optimization below
|
||||
our existing activation seam, not a replacement for the control plane.
|
||||
|
||||
## Important limitations for our design
|
||||
|
||||
- Colibrì's cold path is local NVMe. Network expert fetches add latency, loss, authentication, retries, and Byzantine-data concerns that the project does not solve.
|
||||
- One mutable KV context and one-at-a-time generation are deliberate constraints; Meshnet needs explicit Route Session/KV ownership and seam backpressure for concurrent users.
|
||||
- Router lookahead is model-specific and only experimentally measured. It cannot be assumed for arbitrary MoE architectures.
|
||||
- The custom container and hand-written kernels maximize control but increase maintenance and validation burden. Reusing llama.cpp/GGML remains attractive for a general GGUF lane; Colibrì's expert-cache and planning ideas can sit above that substrate.
|
||||
|
||||
## Source index
|
||||
|
||||
- Repository/README: <https://github.com/JustVugg/colibri>
|
||||
- GLM engine and custom tensor/expert structures: <https://github.com/JustVugg/colibri/blob/main/c/glm.c>
|
||||
- OLMoE engine: <https://github.com/JustVugg/colibri/blob/main/c/olmoe.c>
|
||||
- FP8→Colibrì int4 converter: <https://github.com/JustVugg/colibri/blob/main/c/tools/convert_fp8_to_int4.py>
|
||||
- Optional CUDA backend/loader: <https://github.com/JustVugg/colibri/tree/main/c>
|
||||
- Local OpenAI gateway: <https://github.com/JustVugg/colibri/blob/main/c/openai_server.py>
|
||||
- Placement planning/doctor implementation: <https://github.com/JustVugg/colibri/blob/main/c/resource_plan.py> and <https://github.com/JustVugg/colibri/blob/main/c/doctor.py>
|
||||
@@ -2814,6 +2814,42 @@ def _set_upstream_read_timeout(upstream: Any, timeout: float | None) -> None:
|
||||
sock.settimeout(timeout)
|
||||
|
||||
|
||||
def _upstream_wait_readable(upstream: Any, sock: Any, timeout: float) -> bool:
|
||||
"""Return True when the next ``upstream.readline()`` can make progress.
|
||||
|
||||
Cancellation of a direct SSE proxy is polled with a bounded wait, but a
|
||||
bounded socket timeout cannot be used for the read itself: ``socket``'s
|
||||
makefile reader poisons itself after a single timeout, which would abort any
|
||||
idle gap between frames. So the read stays blocking (timeout ``None``) and
|
||||
this helper does the bounded waiting instead.
|
||||
|
||||
``select()`` alone is insufficient: ``urlopen``'s header parsing can read
|
||||
the first SSE frame ahead into the response ``BufferedReader``, leaving the
|
||||
raw socket with no new bytes -- ``select()`` would then withhold that frame
|
||||
until the upstream produced more output. A non-blocking ``peek()`` surfaces
|
||||
such already-buffered bytes first; only when nothing is buffered do we wait
|
||||
on the raw socket.
|
||||
"""
|
||||
fp = getattr(upstream, "fp", None)
|
||||
if fp is not None and sock is not None:
|
||||
sock.setblocking(False)
|
||||
try:
|
||||
if fp.peek(1):
|
||||
return True
|
||||
except (BlockingIOError, InterruptedError):
|
||||
pass
|
||||
except (OSError, ValueError):
|
||||
# Let the blocking readline surface the real error/EOF.
|
||||
return True
|
||||
finally:
|
||||
# Restore blocking mode (timeout None) so readline never poisons.
|
||||
sock.setblocking(True)
|
||||
if sock is None:
|
||||
return True
|
||||
readable, _, _ = select.select([sock], [], [], timeout)
|
||||
return bool(readable)
|
||||
|
||||
|
||||
def _clear_proxy_progress_log_state(server: "_TrackerHTTPServer", request_id: str) -> None:
|
||||
state = getattr(server, "_proxy_progress_log_state", None)
|
||||
if state is not None:
|
||||
@@ -2980,6 +3016,35 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
except BrokenPipeError:
|
||||
pass
|
||||
|
||||
def _start_sse_response(self, status: int, content_type: str) -> tuple[Any, Any]:
|
||||
"""Start a chunked SSE response and return its write/finish callbacks.
|
||||
|
||||
A streaming proxy cannot use EOF as its framing boundary: HTTP clients
|
||||
and intermediaries are then free to retain an SSE frame until the
|
||||
upstream closes. Chunked framing makes each flushed frame observable
|
||||
immediately while still giving cancellation a clean response end.
|
||||
"""
|
||||
# The tracker otherwise serves finite HTTP/1.0-style responses. Limit
|
||||
# HTTP/1.1 to this framed response so those existing response contracts
|
||||
# do not become persistent connections accidentally.
|
||||
self.protocol_version = "HTTP/1.1"
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", content_type)
|
||||
self.send_header("Cache-Control", "no-cache")
|
||||
self.send_header("X-Accel-Buffering", "no")
|
||||
self.send_header("Transfer-Encoding", "chunked")
|
||||
self.end_headers()
|
||||
|
||||
def write(chunk: bytes) -> None:
|
||||
self.wfile.write(f"{len(chunk):X}\r\n".encode() + chunk + b"\r\n")
|
||||
self.wfile.flush()
|
||||
|
||||
def finish() -> None:
|
||||
self.wfile.write(b"0\r\n\r\n")
|
||||
self.wfile.flush()
|
||||
|
||||
return write, finish
|
||||
|
||||
# ---- unified auth boundary (ADR-0017) ----
|
||||
|
||||
def _resolve_identity(self) -> tuple[str | None, dict | None]:
|
||||
@@ -4035,11 +4100,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
upstream = upstream_result[0]
|
||||
with proxy_ctx.upstream_lock:
|
||||
proxy_ctx.upstream = upstream
|
||||
# Keep the read blocking (timeout None); cancellation is polled by
|
||||
# _upstream_wait_readable() rather than by a bounded socket timeout,
|
||||
# which would poison the makefile reader across idle SSE gaps.
|
||||
_set_upstream_read_timeout(upstream, None)
|
||||
upstream_sock = _upstream_socket(upstream)
|
||||
if upstream_sock is not None:
|
||||
_set_upstream_read_timeout(upstream, None)
|
||||
else:
|
||||
_set_upstream_read_timeout(upstream, 0.5)
|
||||
_tracker_log(server, "info", "proxy connected", request_id=request_id, target_url=target_url)
|
||||
except urllib.error.HTTPError as exc:
|
||||
# Relay error status + body from node
|
||||
@@ -4088,11 +4153,9 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
content_type = upstream.headers.get("Content-Type", "application/json")
|
||||
if is_stream or "text/event-stream" in content_type:
|
||||
# Relay SSE stream chunk-by-chunk
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/event-stream; charset=utf-8")
|
||||
self.send_header("Cache-Control", "no-cache")
|
||||
self.send_header("X-Accel-Buffering", "no")
|
||||
self.end_headers()
|
||||
write_sse, finish_sse = self._start_sse_response(
|
||||
200, "text/event-stream; charset=utf-8"
|
||||
)
|
||||
stream_usage: dict | None = None
|
||||
observed_stream_tokens = 0
|
||||
client_gone = False
|
||||
@@ -4100,22 +4163,14 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
while True:
|
||||
if proxy_ctx.cancel_event.is_set():
|
||||
break
|
||||
if upstream_sock is not None:
|
||||
readable, _, _ = select.select([upstream_sock], [], [], 0.5)
|
||||
if not readable:
|
||||
continue
|
||||
try:
|
||||
line = upstream.readline()
|
||||
except TimeoutError:
|
||||
if not _upstream_wait_readable(upstream, upstream_sock, 0.5):
|
||||
continue
|
||||
line = upstream.readline()
|
||||
if not line:
|
||||
if proxy_ctx.cancel_event.is_set():
|
||||
break
|
||||
break
|
||||
if not client_gone:
|
||||
try:
|
||||
self.wfile.write(line)
|
||||
self.wfile.flush()
|
||||
write_sse(line)
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
# Keep draining upstream so completed node work is still billed.
|
||||
client_gone = True
|
||||
@@ -4135,6 +4190,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
stream_usage = usage
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
client_gone = True
|
||||
except (OSError, ValueError):
|
||||
# Cancellation closes the upstream from another handler
|
||||
# thread to break the blocking readline safely.
|
||||
if not proxy_ctx.cancel_event.is_set():
|
||||
raise
|
||||
if self._finalize_proxy_cancel(
|
||||
proxy_ctx=proxy_ctx,
|
||||
server=server,
|
||||
@@ -4150,6 +4210,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
observed_stream_tokens=observed_stream_tokens,
|
||||
stream_usage=stream_usage,
|
||||
):
|
||||
if not client_gone:
|
||||
try:
|
||||
finish_sse()
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
pass
|
||||
return
|
||||
elapsed = time.monotonic() - started
|
||||
# Bill even on client disconnect — the nodes did the work.
|
||||
@@ -4182,6 +4247,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
api_key, model, tokens, node_work,
|
||||
input_tokens=in_tokens, output_tokens=out_tokens,
|
||||
)
|
||||
if not client_gone:
|
||||
try:
|
||||
finish_sse()
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
pass
|
||||
else:
|
||||
# Non-streaming: buffer and relay
|
||||
resp_body = upstream.read()
|
||||
@@ -4371,11 +4441,10 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
"""Forward a streamed relay response (US-036) to the client as SSE,
|
||||
billing with the same accounting as the direct stream path."""
|
||||
headers = first.get("headers") if isinstance(first.get("headers"), dict) else {}
|
||||
self.send_response(int(first.get("status", 200)))
|
||||
self.send_header("Content-Type", headers.get("Content-Type", "text/event-stream; charset=utf-8"))
|
||||
self.send_header("Cache-Control", "no-cache")
|
||||
self.send_header("X-Accel-Buffering", "no")
|
||||
self.end_headers()
|
||||
write_sse, finish_sse = self._start_sse_response(
|
||||
int(first.get("status", 200)),
|
||||
headers.get("Content-Type", "text/event-stream; charset=utf-8"),
|
||||
)
|
||||
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||
stream_usage: dict | None = None
|
||||
observed_stream_tokens = 0
|
||||
@@ -4389,8 +4458,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
data = chunk.encode()
|
||||
if not client_gone:
|
||||
try:
|
||||
self.wfile.write(data)
|
||||
self.wfile.flush()
|
||||
write_sse(data)
|
||||
except BrokenPipeError:
|
||||
# Keep draining frames — the nodes did the work; bill it.
|
||||
client_gone = True
|
||||
@@ -4429,6 +4497,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
stream_usage=stream_usage,
|
||||
)
|
||||
):
|
||||
if not client_gone:
|
||||
try:
|
||||
finish_sse()
|
||||
except BrokenPipeError:
|
||||
pass
|
||||
return
|
||||
elapsed = time.monotonic() - started
|
||||
in_tokens, out_tokens = _stream_billable_split(
|
||||
@@ -4459,6 +4532,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
||||
api_key, model, tokens, node_work,
|
||||
input_tokens=in_tokens, output_tokens=out_tokens,
|
||||
)
|
||||
if not client_gone:
|
||||
try:
|
||||
finish_sse()
|
||||
except BrokenPipeError:
|
||||
pass
|
||||
|
||||
def _send_relayed_response(self, response: dict) -> None:
|
||||
status = int(response.get("status", 503))
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run one or more supervised Ralph iterations with Gitea state reconciliation.
|
||||
# The canonical PRD remains authoritative; Gitea is updated before and after Ralph.
|
||||
set -u
|
||||
ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
cd "$ROOT"
|
||||
|
||||
python3 scripts/ralph_gitea_sync.py sync
|
||||
ralph_status=0
|
||||
"$@" || ralph_status=$?
|
||||
if [ "$ralph_status" -eq 0 ]; then
|
||||
python3 scripts/ralph_gitea_sync.py sync
|
||||
else
|
||||
python3 scripts/ralph_gitea_sync.py sync --reset-in-progress
|
||||
fi
|
||||
exit "$ralph_status"
|
||||
@@ -1,210 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Synchronize the authoritative Ralph PRD with Gitea issues.
|
||||
|
||||
Gitea is a projection of prd.json. Story completion remains authoritative in
|
||||
prd.json; this tool only creates/reconciles issue bodies, labels, milestones,
|
||||
and open/closed/in-progress presentation.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
DEFAULT_FEATURE = ROOT / ".scratch" / "distributed-gguf-runtime"
|
||||
STATUS_LABELS = {
|
||||
"ready": ("status:ready", "Ready for Ralph execution"),
|
||||
"in-progress": ("status:in-progress", "Currently selected by Ralph"),
|
||||
"blocked": ("status:blocked", "Blocked by incomplete dependencies or human gate"),
|
||||
"completed": ("status:completed", "Completed and verified in prd.json"),
|
||||
}
|
||||
|
||||
|
||||
def credential_for(host: str) -> tuple[str, str]:
|
||||
result = subprocess.run(
|
||||
["git", "credential", "fill"],
|
||||
input=f"protocol=https\nhost={host}\n\n",
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
values = dict(line.split("=", 1) for line in result.stdout.splitlines() if "=" in line)
|
||||
username, password = values.get("username"), values.get("password")
|
||||
if not password:
|
||||
raise RuntimeError(f"No Git credential available for {host}")
|
||||
return username or "", password
|
||||
|
||||
|
||||
class Gitea:
|
||||
def __init__(self, base_url: str, username: str, password: str, dry_run: bool = False):
|
||||
self.base = base_url.rstrip("/")
|
||||
self.auth = "Basic " + base64.b64encode(f"{username}:{password}".encode()).decode()
|
||||
self.dry_run = dry_run
|
||||
|
||||
def request(self, method: str, path: str, payload: Any = None) -> Any:
|
||||
url = self.base + path
|
||||
body = None if payload is None else json.dumps(payload).encode()
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=body,
|
||||
method=method,
|
||||
headers={
|
||||
"Authorization": self.auth,
|
||||
"Accept": "application/json",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
if self.dry_run and method not in {"GET", "HEAD"}:
|
||||
print(f"DRY-RUN {method} {path}")
|
||||
return {}
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=60) as response:
|
||||
raw = response.read()
|
||||
return json.loads(raw) if raw else {}
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode(errors="replace")[:500]
|
||||
raise RuntimeError(f"Gitea {method} {path} -> HTTP {exc.code}: {detail}") from exc
|
||||
|
||||
def paginate(self, path: str) -> list[dict[str, Any]]:
|
||||
result: list[dict[str, Any]] = []
|
||||
page = 1
|
||||
while True:
|
||||
separator = "&" if "?" in path else "?"
|
||||
batch = self.request("GET", f"{path}{separator}limit=50&page={page}")
|
||||
if not batch:
|
||||
return result
|
||||
result.extend(batch)
|
||||
if len(batch) < 50:
|
||||
return result
|
||||
page += 1
|
||||
|
||||
|
||||
def color(name: str) -> str:
|
||||
return "#" + hashlib.sha256(name.encode()).hexdigest()[:6]
|
||||
|
||||
|
||||
def load_schema(feature: Path) -> Any:
|
||||
module_path = ROOT / "scripts" / "ralph_prd_schema.py"
|
||||
import importlib.util
|
||||
spec = importlib.util.spec_from_file_location("ralph_prd_schema", module_path)
|
||||
assert spec is not None and spec.loader is not None
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def issue_body(schema: Any, story: dict[str, Any]) -> str:
|
||||
data = schema.load_prd(DEFAULT_FEATURE / "prd.json")
|
||||
return schema.render_issue_markdown(story, data) + f"\n<!-- ralph-story-id: {story['id']} -->\n"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("command", choices=["sync"])
|
||||
parser.add_argument("--feature-dir", type=Path, default=DEFAULT_FEATURE)
|
||||
parser.add_argument("--repo-url", default="https://git.d-popov.com/popov/neuron-tai")
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
parser.add_argument("--reset-in-progress", action="store_true", help="Do not claim a next story after a failed Ralph start")
|
||||
args = parser.parse_args()
|
||||
|
||||
feature = args.feature_dir.resolve()
|
||||
prd_path = feature / "prd.json"
|
||||
data = json.loads(prd_path.read_text())
|
||||
stories = data["userStories"]
|
||||
repo = urllib.parse.urlparse(args.repo_url)
|
||||
if not repo.hostname or not repo.path.strip("/"):
|
||||
raise SystemExit("--repo-url must be an HTTPS Gitea repository URL")
|
||||
owner, name = repo.path.strip("/").removesuffix(".git").split("/", 1)
|
||||
api_base = f"{repo.scheme}://{repo.hostname}/api/v1/repos/{owner}/{name}"
|
||||
username, password = credential_for(repo.hostname)
|
||||
gitea = Gitea(api_base, username, password, args.dry_run)
|
||||
schema = load_schema(feature)
|
||||
schema_errors = schema.validate_backlog(data)
|
||||
if schema_errors:
|
||||
raise SystemExit("PRD validation failed:\n" + "\n".join(schema_errors))
|
||||
|
||||
labels = {x["name"]: x for x in gitea.paginate("/labels")}
|
||||
milestones = {x["title"]: x for x in gitea.paginate("/milestones?state=all")}
|
||||
issue_list = gitea.paginate("/issues?state=all")
|
||||
by_story = {}
|
||||
for issue in issue_list:
|
||||
marker = f"<!-- ralph-story-id: DGR-"
|
||||
body = issue.get("body") or ""
|
||||
if marker in body:
|
||||
story_id = body.split(marker, 1)[1].split(" -->", 1)[0]
|
||||
by_story["DGR-" + story_id] = issue
|
||||
elif issue.get("title", "").startswith("DGR-"):
|
||||
by_story[issue["title"].split(":", 1)[0]] = issue
|
||||
|
||||
def ensure_label(name_: str, description: str = "") -> dict[str, Any]:
|
||||
if name_ in labels:
|
||||
return labels[name_]
|
||||
created = gitea.request("POST", "/labels", {"name": name_, "color": color(name_), "description": description})
|
||||
if args.dry_run:
|
||||
created = {"id": -len(labels) - 1, "name": name_}
|
||||
labels[name_] = created
|
||||
return created
|
||||
|
||||
for story in stories:
|
||||
for label in story["labels"]:
|
||||
ensure_label(label)
|
||||
for label, description in STATUS_LABELS.values():
|
||||
ensure_label(label, description)
|
||||
for milestone in data.get("milestones", []):
|
||||
title = milestone["id"]
|
||||
if title not in milestones:
|
||||
milestones[title] = gitea.request("POST", "/milestones", {"title": title, "description": milestone["name"] + ": " + milestone["outcome"]})
|
||||
if args.dry_run:
|
||||
milestones[title] = {"id": -len(milestones), "title": title}
|
||||
|
||||
passed = {s["id"] for s in stories if s.get("passes")}
|
||||
next_story = None if args.reset_in_progress else next(
|
||||
(
|
||||
s
|
||||
for s in stories
|
||||
if not s.get("passes")
|
||||
and s.get("triage") == "ready-for-agent"
|
||||
and all(d in passed for d in s["dependsOn"])
|
||||
),
|
||||
None,
|
||||
)
|
||||
mapping = {}
|
||||
for story in stories:
|
||||
sid = story["id"]
|
||||
desired_status = "completed" if story.get("passes") else ("in-progress" if next_story and sid == next_story["id"] else ("ready" if all(d in passed for d in story["dependsOn"]) else "blocked"))
|
||||
label_names = list(dict.fromkeys(story["labels"] + [STATUS_LABELS[desired_status][0]]))
|
||||
label_ids = [ensure_label(label)["id"] for label in label_names]
|
||||
milestone_id = milestones[story["milestone"]]["id"]
|
||||
title = f"{sid}: {story['title']}"
|
||||
body = issue_body(schema, story)
|
||||
issue = by_story.get(sid)
|
||||
payload = {"title": title, "body": body, "labels": label_ids, "milestone": milestone_id, "state": "closed" if story.get("passes") else "open"}
|
||||
if issue is None:
|
||||
issue = gitea.request("POST", "/issues", payload)
|
||||
print(f"created {sid} #{issue.get('number')} {desired_status}")
|
||||
else:
|
||||
issue = gitea.request("PATCH", f"/issues/{issue['number']}", payload)
|
||||
print(f"reconciled {sid} #{issue.get('number')} {desired_status}")
|
||||
# Gitea's issue PATCH does not reliably replace labels; use the
|
||||
# dedicated endpoint so stale status:in-progress labels are removed.
|
||||
gitea.request("PUT", f"/issues/{issue['number']}/labels", {"labels": label_ids})
|
||||
mapping[sid] = {"number": issue.get("number"), "url": issue.get("html_url"), "state": "closed" if story.get("passes") else "open", "status": desired_status}
|
||||
|
||||
if not args.dry_run:
|
||||
(feature / "gitea-issues.json").write_text(json.dumps({"repository": args.repo_url, "stories": mapping}, indent=2) + "\n")
|
||||
print(f"synced={len(stories)} next={next_story['id'] if next_story else 'none'} dry_run={args.dry_run}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,576 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Canonical schema, validator, and generator for Ralph `prd.json` backlogs.
|
||||
|
||||
`prd.json` is the single authoritative source for a Ralph-managed backlog.
|
||||
This module defines the field shape every story must have, validates
|
||||
structural and cross-story invariants (unique IDs/titles, resolvable
|
||||
`dependsOn`, acyclic dependency graph, `blocks` derived correctly), and
|
||||
renders the same fields into the generated Markdown/Gitea-issue view used
|
||||
by `.scratch/<feature>/issues/*.md`.
|
||||
|
||||
Generation is one-directional: prd.json -> Markdown / Gitea payload. There
|
||||
is no Markdown -> prd.json parser, so a generated artifact can never feed
|
||||
`passes` (or any other field) back into the authoritative source. Every
|
||||
generated Markdown document must start with the authority disclaimer
|
||||
(`authority_disclaimer()`, sourced from prd.json's own `metadataSchema` when
|
||||
present) so a stray edit of the generated file cannot be mistaken for a
|
||||
source-of-truth change. The shared "quality gates" boilerplate is likewise
|
||||
sourced from prd.json's `qualityGates`, not duplicated as a Python literal.
|
||||
|
||||
Example:
|
||||
python scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||
python scripts/ralph_prd_schema.py render .scratch/distributed-gguf-runtime/prd.json DGR-018
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
AUTHORITY_DISCLAIMER = (
|
||||
"<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. "
|
||||
"prd.json IS AUTHORITATIVE. -->"
|
||||
)
|
||||
|
||||
EXECUTION_MODES = {"AFK", "HITL"}
|
||||
EVIDENCE_CLASSES = {"model-free", "fixture", "real-model", "real-hardware", "release"}
|
||||
HARDWARE_FLAGS = {"none", "optional", "required"}
|
||||
UPSTREAM_FLAGS = {"yes", "no", "conditional"}
|
||||
TRIAGE_VALUES = {"ready-for-agent", "ready-for-human"}
|
||||
|
||||
# Canonical field registry: name -> (required, expected python type(s)).
|
||||
# This is the single definition of "what a story is" referenced by both
|
||||
# validation and documentation; keep it in sync with any schema change.
|
||||
STORY_FIELDS: dict[str, tuple[bool, tuple[type, ...]]] = {
|
||||
"id": (True, (str,)),
|
||||
"title": (True, (str,)),
|
||||
"priority": (True, (int,)),
|
||||
"milestone": (True, (str,)),
|
||||
"executionMode": (True, (str,)),
|
||||
"labels": (True, (list,)),
|
||||
"evidenceClass": (True, (str,)),
|
||||
"evidencePath": (True, (str,)),
|
||||
"hardware": (True, (str,)),
|
||||
"model": (True, (str,)),
|
||||
"upstream": (True, (str,)),
|
||||
"dependsOn": (True, (list,)),
|
||||
"triage": (True, (str,)),
|
||||
"description": (True, (str,)),
|
||||
"acceptanceCriteria": (True, (list,)),
|
||||
"passes": (True, (bool,)),
|
||||
"notes": (True, (str,)),
|
||||
"blocks": (True, (list,)),
|
||||
"completionNotes": (False, (str,)),
|
||||
}
|
||||
|
||||
_ID_RE = re.compile(r"^[A-Z][A-Z0-9]*-\d+$")
|
||||
|
||||
|
||||
class PrdValidationError(ValueError):
|
||||
"""Raised with all accumulated validation errors, one per line."""
|
||||
|
||||
def __init__(self, errors: list[str]):
|
||||
super().__init__("\n".join(errors))
|
||||
self.errors = errors
|
||||
|
||||
|
||||
def load_prd(path: Path) -> dict[str, Any]:
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError as exc:
|
||||
raise PrdValidationError([f"PRD not found: {path}"]) from exc
|
||||
try:
|
||||
data = json.loads(text)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise PrdValidationError([f"Invalid JSON in {path}: {exc}"]) from exc
|
||||
if not isinstance(data, dict):
|
||||
raise PrdValidationError([f"{path}: top-level PRD document must be an object"])
|
||||
return data
|
||||
|
||||
|
||||
def stories_of(data: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
stories = data.get("userStories")
|
||||
if not isinstance(stories, list):
|
||||
return []
|
||||
return [s for s in stories if isinstance(s, dict)]
|
||||
|
||||
|
||||
def derive_type(labels: list[str]) -> str | None:
|
||||
"""Derive the story's `type` from its `type:` label, if present.
|
||||
|
||||
Release-gate stories (labelled `gate:hitl`) intentionally carry no
|
||||
`type:` label; they derive the synthetic type `release-gate` instead.
|
||||
"""
|
||||
types = [label.split(":", 1)[1] for label in labels if label.startswith("type:")]
|
||||
if types:
|
||||
return types[0]
|
||||
if any(label.startswith("gate:") for label in labels):
|
||||
return "release-gate"
|
||||
return None
|
||||
|
||||
|
||||
def _label_prefix_counts(labels: list[str], prefix: str) -> int:
|
||||
return sum(1 for label in labels if label.startswith(prefix))
|
||||
|
||||
|
||||
def validate_schema(data: dict[str, Any]) -> list[str]:
|
||||
"""Structural validation: required top-level keys and per-story field shape."""
|
||||
errors: list[str] = []
|
||||
for key in ("name", "description", "branchName", "userStories"):
|
||||
if key not in data:
|
||||
errors.append(f"prd: missing top-level field '{key}'")
|
||||
stories = data.get("userStories")
|
||||
if not isinstance(stories, list):
|
||||
errors.append("prd: 'userStories' must be a list")
|
||||
return errors
|
||||
|
||||
for index, story in enumerate(stories):
|
||||
label = f"userStories[{index}]"
|
||||
if not isinstance(story, dict):
|
||||
errors.append(f"{label}: story must be an object")
|
||||
continue
|
||||
sid = story.get("id", label)
|
||||
|
||||
for field, (required, types) in STORY_FIELDS.items():
|
||||
if field not in story:
|
||||
if required:
|
||||
errors.append(f"{sid}: missing required field '{field}'")
|
||||
continue
|
||||
value = story[field]
|
||||
if types == (bool,) and isinstance(value, bool):
|
||||
continue
|
||||
if not isinstance(value, types):
|
||||
errors.append(
|
||||
f"{sid}: field '{field}' must be {types}, got {type(value).__name__}"
|
||||
)
|
||||
|
||||
if isinstance(story.get("id"), str) and not _ID_RE.match(story["id"]):
|
||||
errors.append(f"{sid}: id '{story['id']}' does not match PREFIX-NUMBER convention")
|
||||
if isinstance(story.get("executionMode"), str) and story["executionMode"] not in EXECUTION_MODES:
|
||||
errors.append(f"{sid}: executionMode '{story['executionMode']}' not in {sorted(EXECUTION_MODES)}")
|
||||
if isinstance(story.get("evidenceClass"), str) and story["evidenceClass"] not in EVIDENCE_CLASSES:
|
||||
errors.append(f"{sid}: evidenceClass '{story['evidenceClass']}' not in {sorted(EVIDENCE_CLASSES)}")
|
||||
if isinstance(story.get("hardware"), str) and story["hardware"] not in HARDWARE_FLAGS:
|
||||
errors.append(f"{sid}: hardware '{story['hardware']}' not in {sorted(HARDWARE_FLAGS)}")
|
||||
if isinstance(story.get("upstream"), str) and story["upstream"] not in UPSTREAM_FLAGS:
|
||||
errors.append(f"{sid}: upstream '{story['upstream']}' not in {sorted(UPSTREAM_FLAGS)}")
|
||||
if isinstance(story.get("triage"), str) and story["triage"] not in TRIAGE_VALUES:
|
||||
errors.append(f"{sid}: triage '{story['triage']}' not in {sorted(TRIAGE_VALUES)}")
|
||||
|
||||
labels = story.get("labels")
|
||||
if isinstance(labels, list):
|
||||
if not all(isinstance(label, str) for label in labels):
|
||||
errors.append(f"{sid}: labels must all be strings")
|
||||
elif _label_prefix_counts(labels, "type:") > 1:
|
||||
errors.append(f"{sid}: at most one 'type:' label allowed")
|
||||
elif _label_prefix_counts(labels, "priority:") > 1:
|
||||
errors.append(f"{sid}: at most one 'priority:' label allowed")
|
||||
|
||||
depends_on = story.get("dependsOn")
|
||||
if isinstance(depends_on, list) and not all(isinstance(dep, str) for dep in depends_on):
|
||||
errors.append(f"{sid}: dependsOn entries must all be strings")
|
||||
|
||||
acceptance = story.get("acceptanceCriteria")
|
||||
if isinstance(acceptance, list):
|
||||
if not acceptance:
|
||||
errors.append(f"{sid}: acceptanceCriteria must be non-empty")
|
||||
elif not all(isinstance(item, str) and item.strip() for item in acceptance):
|
||||
errors.append(f"{sid}: acceptanceCriteria entries must be non-empty strings")
|
||||
|
||||
return errors
|
||||
|
||||
|
||||
def _find_cycle(story_ids: set[str], depends_on: dict[str, list[str]]) -> list[str] | None:
|
||||
WHITE, GRAY, BLACK = 0, 1, 2
|
||||
color = {sid: WHITE for sid in story_ids}
|
||||
path: list[str] = []
|
||||
|
||||
def dfs(node: str) -> list[str] | None:
|
||||
color[node] = GRAY
|
||||
path.append(node)
|
||||
for dep in depends_on.get(node, []):
|
||||
if dep not in color:
|
||||
continue
|
||||
if color[dep] == GRAY:
|
||||
cycle_start = path.index(dep)
|
||||
return path[cycle_start:] + [dep]
|
||||
if color[dep] == WHITE:
|
||||
found = dfs(dep)
|
||||
if found:
|
||||
return found
|
||||
path.pop()
|
||||
color[node] = BLACK
|
||||
return None
|
||||
|
||||
for sid in sorted(story_ids):
|
||||
if color[sid] == WHITE:
|
||||
cycle = dfs(sid)
|
||||
if cycle:
|
||||
return cycle
|
||||
return None
|
||||
|
||||
|
||||
def validate_semantics(data: dict[str, Any]) -> list[str]:
|
||||
"""Cross-story validation: uniqueness, dependency graph, derived `blocks`."""
|
||||
errors: list[str] = []
|
||||
stories = stories_of(data)
|
||||
|
||||
ids = [s.get("id") for s in stories if isinstance(s.get("id"), str)]
|
||||
seen_ids: set[str] = set()
|
||||
for sid in ids:
|
||||
if sid in seen_ids:
|
||||
errors.append(f"duplicate story id: '{sid}'")
|
||||
seen_ids.add(sid)
|
||||
|
||||
titles = [s.get("title") for s in stories if isinstance(s.get("title"), str)]
|
||||
seen_titles: set[str] = set()
|
||||
for title in titles:
|
||||
if title in seen_titles:
|
||||
errors.append(f"duplicate story title: '{title}'")
|
||||
seen_titles.add(title)
|
||||
|
||||
by_id = {s["id"]: s for s in stories if isinstance(s.get("id"), str)}
|
||||
depends_on: dict[str, list[str]] = {}
|
||||
for story in stories:
|
||||
sid = story.get("id")
|
||||
if not isinstance(sid, str):
|
||||
continue
|
||||
deps = story.get("dependsOn") if isinstance(story.get("dependsOn"), list) else []
|
||||
deps = [d for d in deps if isinstance(d, str)]
|
||||
depends_on[sid] = deps
|
||||
if sid in deps:
|
||||
errors.append(f"{sid}: cannot depend on itself")
|
||||
for dep in deps:
|
||||
if dep not in by_id:
|
||||
errors.append(f"{sid}: depends on unknown story '{dep}'")
|
||||
|
||||
cycle = _find_cycle(set(by_id), depends_on)
|
||||
if cycle:
|
||||
errors.append("dependency cycle detected: " + " -> ".join(cycle))
|
||||
|
||||
derived_blocks: dict[str, list[str]] = {sid: [] for sid in by_id}
|
||||
for sid, deps in depends_on.items():
|
||||
for dep in deps:
|
||||
if dep in derived_blocks:
|
||||
derived_blocks[dep].append(sid)
|
||||
for sid in derived_blocks:
|
||||
derived_blocks[sid] = sorted(derived_blocks[sid])
|
||||
|
||||
for story in stories:
|
||||
sid = story.get("id")
|
||||
if not isinstance(sid, str):
|
||||
continue
|
||||
given = story.get("blocks") if isinstance(story.get("blocks"), list) else []
|
||||
given_sorted = sorted(d for d in given if isinstance(d, str))
|
||||
expected = derived_blocks.get(sid, [])
|
||||
if given_sorted != expected:
|
||||
errors.append(
|
||||
f"{sid}: 'blocks' does not match derived dependents "
|
||||
f"(got {given_sorted}, expected {expected})"
|
||||
)
|
||||
|
||||
evidence_path = story.get("evidencePath")
|
||||
if isinstance(evidence_path, str):
|
||||
expected_path = f".scratch/distributed-gguf-runtime/evidence/{sid}/README.md"
|
||||
if evidence_path != expected_path:
|
||||
errors.append(
|
||||
f"{sid}: evidencePath '{evidence_path}' does not match convention "
|
||||
f"'{expected_path}'"
|
||||
)
|
||||
|
||||
return errors
|
||||
|
||||
|
||||
def validate_fresh_backlog(data: dict[str, Any]) -> list[str]:
|
||||
"""Extra invariant for a freshly generated (not-yet-started) backlog.
|
||||
|
||||
A fresh backlog must not claim any completion credit: every story
|
||||
starts `passes: false`. This is distinct from validating an in-flight
|
||||
backlog (use validate_schema + validate_semantics for that), because a
|
||||
real backlog legitimately accumulates `passes: true` stories over time.
|
||||
"""
|
||||
errors = validate_backlog(data)
|
||||
for story in stories_of(data):
|
||||
if story.get("passes") is not False:
|
||||
errors.append(
|
||||
f"{story.get('id')}: fresh backlog requires passes=false, "
|
||||
f"got {story.get('passes')!r}"
|
||||
)
|
||||
return errors
|
||||
|
||||
|
||||
def validate_backlog(data: dict[str, Any]) -> list[str]:
|
||||
"""Full validation for an in-progress backlog: structure, semantics, and
|
||||
(when present) prd.json's self-documented metadataSchema/qualityGates."""
|
||||
return (
|
||||
validate_schema(data)
|
||||
+ validate_semantics(data)
|
||||
+ validate_metadata_schema_consistency(data)
|
||||
)
|
||||
|
||||
|
||||
def validate_metadata_schema_consistency(data: dict[str, Any]) -> list[str]:
|
||||
"""If prd.json documents its own `metadataSchema`/`qualityGates`, verify
|
||||
that self-description hasn't drifted from what this validator actually
|
||||
enforces. Absent entirely, this is a no-op (a minimal fixture PRD is not
|
||||
required to carry self-documentation to be structurally valid).
|
||||
"""
|
||||
errors: list[str] = []
|
||||
ms = data.get("metadataSchema")
|
||||
if ms is None:
|
||||
return errors
|
||||
|
||||
enum_checks = {
|
||||
"triageValues": TRIAGE_VALUES,
|
||||
"executionModeValues": EXECUTION_MODES,
|
||||
"evidenceClassValues": EVIDENCE_CLASSES,
|
||||
"hardwareValues": HARDWARE_FLAGS,
|
||||
"upstreamValues": UPSTREAM_FLAGS,
|
||||
}
|
||||
for key, expected in enum_checks.items():
|
||||
declared = ms.get(key)
|
||||
if declared is None:
|
||||
errors.append(f"metadataSchema: missing '{key}'")
|
||||
continue
|
||||
if set(declared) != expected:
|
||||
errors.append(
|
||||
f"metadataSchema.{key} {sorted(declared)} does not match "
|
||||
f"enforced values {sorted(expected)}"
|
||||
)
|
||||
|
||||
required_in_code = {field for field, (required, _) in STORY_FIELDS.items() if required}
|
||||
declared_required = set(ms.get("requiredStoryFields", []))
|
||||
if declared_required != required_in_code:
|
||||
errors.append(
|
||||
f"metadataSchema.requiredStoryFields {sorted(declared_required)} does not "
|
||||
f"match code-enforced required fields {sorted(required_in_code)}"
|
||||
)
|
||||
|
||||
optional_in_code = {field for field, (required, _) in STORY_FIELDS.items() if not required}
|
||||
declared_optional = set(ms.get("optionalStoryFields", []))
|
||||
if declared_optional != optional_in_code:
|
||||
errors.append(
|
||||
f"metadataSchema.optionalStoryFields {sorted(declared_optional)} does not "
|
||||
f"match code-enforced optional fields {sorted(optional_in_code)}"
|
||||
)
|
||||
|
||||
if not ms.get("generatedArtifactDisclaimer"):
|
||||
errors.append("metadataSchema: missing 'generatedArtifactDisclaimer'")
|
||||
|
||||
if "qualityGates" not in data:
|
||||
errors.append("prd: metadataSchema is documented but top-level 'qualityGates' is missing")
|
||||
elif not quality_gate_bullets(data):
|
||||
errors.append("qualityGates: no bullets declared under universal/native/realModelHardware/scope")
|
||||
|
||||
return errors
|
||||
|
||||
|
||||
def quality_gate_bullets(data: dict[str, Any]) -> list[str]:
|
||||
"""Flatten prd.json's `qualityGates` into the bullet order used by the
|
||||
generated "Shared quality gates" Markdown section."""
|
||||
gates = data.get("qualityGates", {})
|
||||
bullets: list[str] = []
|
||||
for key in ("universal", "native", "realModelHardware", "scope"):
|
||||
bullets.extend(gates.get(key, []))
|
||||
return bullets
|
||||
|
||||
|
||||
def authority_disclaimer(data: dict[str, Any]) -> str:
|
||||
"""The exact disclaimer generated Markdown/Gitea bodies must start with.
|
||||
|
||||
Sourced from prd.json's `metadataSchema.generatedArtifactDisclaimer` when
|
||||
present, falling back to the module default for minimal fixtures.
|
||||
"""
|
||||
return data.get("metadataSchema", {}).get("generatedArtifactDisclaimer", AUTHORITY_DISCLAIMER)
|
||||
|
||||
|
||||
def check_generated_markdown_authority(text: str, disclaimer: str = AUTHORITY_DISCLAIMER) -> list[str]:
|
||||
"""Reject generated Markdown that fails to defer authority to prd.json."""
|
||||
errors: list[str] = []
|
||||
first_line = text.splitlines()[0] if text else ""
|
||||
if first_line != disclaimer:
|
||||
errors.append(
|
||||
"generated Markdown must start with the authority disclaimer: "
|
||||
f"expected {disclaimer!r}, got {first_line!r}"
|
||||
)
|
||||
lowered = text.lower()
|
||||
for claim in ("this file is authoritative", "this document is authoritative", "markdown is authoritative"):
|
||||
if claim in lowered:
|
||||
errors.append(f"generated Markdown claims authority over prd.json: found '{claim}'")
|
||||
return errors
|
||||
|
||||
|
||||
def slugify(title: str) -> str:
|
||||
slug = re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")
|
||||
return re.sub(r"-{2,}", "-", slug)
|
||||
|
||||
|
||||
def issue_number(story_id: str) -> str:
|
||||
match = re.search(r"(\d+)$", story_id)
|
||||
if not match:
|
||||
raise ValueError(f"story id '{story_id}' has no trailing number")
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def issue_filename(story: dict[str, Any]) -> str:
|
||||
return f"{issue_number(story['id'])}-{slugify(story['title'])}.md"
|
||||
|
||||
|
||||
# Fallback bullets used only when a minimal fixture PRD has no qualityGates
|
||||
# block of its own; the real backlog always sources these from prd.json via
|
||||
# quality_gate_bullets() so this list is a default, not a duplicate of truth.
|
||||
_DEFAULT_QUALITY_GATE_BULLETS = [
|
||||
"Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.",
|
||||
"`git diff --check` passes.",
|
||||
"Default tests are model-download-free, API-credit-free, and GPU-free.",
|
||||
"Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.",
|
||||
]
|
||||
|
||||
|
||||
def render_issue_markdown(story: dict[str, Any], data: dict[str, Any] | None = None) -> str:
|
||||
"""Render the canonical generated Markdown view of a story.
|
||||
|
||||
`data` supplies the shared quality-gates bullets and authority disclaimer
|
||||
from prd.json's own `qualityGates`/`metadataSchema`; omit it only for
|
||||
fixtures that don't model those fields. This is byte-for-byte identical
|
||||
to the convention already used by `.scratch/distributed-gguf-runtime/
|
||||
issues/*.md`; see the round-trip tests in tests/test_ralph_prd_schema.py.
|
||||
"""
|
||||
data = data or {}
|
||||
sid = story["id"]
|
||||
passes = story["passes"]
|
||||
deps = story.get("dependsOn", [])
|
||||
blocks = sorted(story.get("blocks", []))
|
||||
|
||||
deps_text = ", ".join(f"`{d}`" for d in deps) if deps else "None"
|
||||
blocks_text = ", ".join(f"`{b}`" for b in blocks) if blocks else "None"
|
||||
labels_text = ", ".join(f"`{label}`" for label in story.get("labels", []))
|
||||
|
||||
if passes:
|
||||
status_line = "completed; `passes: true`"
|
||||
else:
|
||||
status_line = f"specification only; `{story['triage']}`; `passes: false`"
|
||||
|
||||
checkbox = "x" if passes else " "
|
||||
acceptance_lines = "\n".join(
|
||||
f"- [{checkbox}] {criterion}" for criterion in story["acceptanceCriteria"]
|
||||
)
|
||||
|
||||
if passes:
|
||||
evidence_handoff = (
|
||||
f"Verified evidence: `{story['evidencePath']}`. Legacy evidence remains "
|
||||
"provenance only and grants no implementation completion credit."
|
||||
)
|
||||
else:
|
||||
evidence_handoff = (
|
||||
f"Write and verify `{story['evidencePath']}`. Until every criterion and "
|
||||
f"applicable gate has real evidence, this story remains `passes: false`. "
|
||||
"Legacy evidence is provenance only, not completion credit."
|
||||
)
|
||||
|
||||
description = story["description"]
|
||||
|
||||
gate_bullets = quality_gate_bullets(data) if "qualityGates" in data else _DEFAULT_QUALITY_GATE_BULLETS
|
||||
gates_section = "## Shared quality gates\n\n" + "\n".join(f"- {bullet}" for bullet in gate_bullets)
|
||||
|
||||
lines = [
|
||||
authority_disclaimer(data),
|
||||
f"# {sid}: {story['title']}",
|
||||
"",
|
||||
f"- **Status / triage:** {status_line}",
|
||||
f"- **Execution mode:** `{story['executionMode']}`",
|
||||
f"- **Milestone:** `{story['milestone']}`",
|
||||
f"- **Dependencies:** {deps_text}",
|
||||
f"- **Blocks (derived):** {blocks_text}",
|
||||
f"- **Labels:** {labels_text}",
|
||||
f"- **Evidence class:** `{story['evidenceClass']}`",
|
||||
f"- **Hardware:** `{story['hardware']}`",
|
||||
f"- **Model:** `{story['model']}`",
|
||||
f"- **Upstream:** `{story['upstream']}`",
|
||||
"",
|
||||
"## Objective / description",
|
||||
"",
|
||||
description,
|
||||
"",
|
||||
"## Acceptance criteria",
|
||||
"",
|
||||
acceptance_lines,
|
||||
"",
|
||||
gates_section,
|
||||
"",
|
||||
"## Evidence handoff",
|
||||
"",
|
||||
evidence_handoff,
|
||||
"",
|
||||
]
|
||||
return "\n".join(lines[:-1]) + "\n"
|
||||
|
||||
|
||||
def to_gitea_issue_payload(story: dict[str, Any], data: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||
"""Structured payload matching the Gitea create-issue API shape.
|
||||
|
||||
`body` is exactly `render_issue_markdown`'s output, so the payload
|
||||
carries no information that isn't losslessly derivable from prd.json.
|
||||
"""
|
||||
return {
|
||||
"title": f"{story['id']}: {story['title']}",
|
||||
"body": render_issue_markdown(story, data),
|
||||
"labels": list(story.get("labels", [])),
|
||||
"milestone": story["milestone"],
|
||||
}
|
||||
|
||||
|
||||
def _cmd_validate(args: argparse.Namespace) -> int:
|
||||
data = load_prd(args.prd)
|
||||
errors = validate_fresh_backlog(data) if args.fresh else validate_backlog(data)
|
||||
if errors:
|
||||
for error in errors:
|
||||
print(f"ERROR: {error}", file=sys.stderr)
|
||||
print(f"{len(errors)} validation error(s).", file=sys.stderr)
|
||||
return 1
|
||||
print(f"OK: {len(stories_of(data))} stories validated.")
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_render(args: argparse.Namespace) -> int:
|
||||
data = load_prd(args.prd)
|
||||
by_id = {s["id"]: s for s in stories_of(data)}
|
||||
story = by_id.get(args.story_id)
|
||||
if story is None:
|
||||
print(f"Unknown story id: {args.story_id}", file=sys.stderr)
|
||||
return 1
|
||||
print(render_issue_markdown(story, data), end="")
|
||||
return 0
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
validate = subparsers.add_parser("validate", help="Validate a prd.json against the canonical schema")
|
||||
validate.add_argument("prd", type=Path)
|
||||
validate.add_argument("--fresh", action="store_true", help="Also require passes=false on every story")
|
||||
validate.set_defaults(func=_cmd_validate)
|
||||
|
||||
render = subparsers.add_parser("render", help="Render the generated Markdown view of one story")
|
||||
render.add_argument("prd", type=Path)
|
||||
render.add_argument("story_id")
|
||||
render.set_defaults(func=_cmd_render)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args(argv[1:])
|
||||
return args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
@@ -1,503 +0,0 @@
|
||||
"""Deterministic, model-free tests for the canonical Ralph prd.json schema.
|
||||
|
||||
Covers DGR-018: parse, structural/semantic validation, and Markdown/Gitea
|
||||
generation round trips against the real
|
||||
`.scratch/distributed-gguf-runtime/prd.json` backlog.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
FEATURE_DIR = REPO_ROOT / ".scratch" / "distributed-gguf-runtime"
|
||||
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
"ralph_prd_schema", REPO_ROOT / "scripts" / "ralph_prd_schema.py"
|
||||
)
|
||||
schema = importlib.util.module_from_spec(_spec)
|
||||
sys.modules.setdefault("ralph_prd_schema", schema)
|
||||
_spec.loader.exec_module(schema) # type: ignore[union-attr]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _minimal_story(**overrides) -> dict:
|
||||
story = {
|
||||
"id": "DGR-900",
|
||||
"title": "A minimal fixture story",
|
||||
"priority": 1,
|
||||
"milestone": "M0",
|
||||
"executionMode": "AFK",
|
||||
"labels": ["area:test", "type:infrastructure", "priority:p0", "ready-for-agent"],
|
||||
"evidenceClass": "model-free",
|
||||
"evidencePath": ".scratch/distributed-gguf-runtime/evidence/DGR-900/README.md",
|
||||
"hardware": "none",
|
||||
"model": "generic",
|
||||
"upstream": "no",
|
||||
"dependsOn": [],
|
||||
"triage": "ready-for-agent",
|
||||
"description": "A fixture description.",
|
||||
"acceptanceCriteria": ["Do the thing."],
|
||||
"passes": False,
|
||||
"notes": "Generated source issue: none; prd.json is authoritative.",
|
||||
"blocks": [],
|
||||
}
|
||||
story.update(overrides)
|
||||
return story
|
||||
|
||||
|
||||
def _minimal_prd(stories: list[dict]) -> dict:
|
||||
return {
|
||||
"name": "Fixture",
|
||||
"description": "Fixture PRD",
|
||||
"branchName": "fixture",
|
||||
"userStories": stories,
|
||||
"metadata": {},
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def live_prd_data() -> dict:
|
||||
return schema.load_prd(FEATURE_DIR / "prd.json")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_load_prd_parses_real_backlog(live_prd_data):
|
||||
stories = schema.stories_of(live_prd_data)
|
||||
assert len(stories) == 55
|
||||
assert all(isinstance(s["id"], str) for s in stories)
|
||||
|
||||
|
||||
def test_load_prd_missing_file_raises(tmp_path):
|
||||
with pytest.raises(schema.PrdValidationError):
|
||||
schema.load_prd(tmp_path / "does-not-exist.json")
|
||||
|
||||
|
||||
def test_load_prd_invalid_json_raises(tmp_path):
|
||||
bad = tmp_path / "prd.json"
|
||||
bad.write_text("{not valid json")
|
||||
with pytest.raises(schema.PrdValidationError):
|
||||
schema.load_prd(bad)
|
||||
|
||||
|
||||
def test_load_prd_rejects_non_object_document(tmp_path):
|
||||
bad = tmp_path / "prd.json"
|
||||
bad.write_text("[]")
|
||||
with pytest.raises(schema.PrdValidationError):
|
||||
schema.load_prd(bad)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Structural + semantic validation against the real backlog
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_real_backlog_passes_structural_validation(live_prd_data):
|
||||
assert schema.validate_schema(live_prd_data) == []
|
||||
|
||||
|
||||
def test_real_backlog_passes_semantic_validation(live_prd_data):
|
||||
assert schema.validate_semantics(live_prd_data) == []
|
||||
|
||||
|
||||
def test_real_backlog_passes_full_validate_backlog(live_prd_data):
|
||||
assert schema.validate_backlog(live_prd_data) == []
|
||||
|
||||
|
||||
def test_real_backlog_has_unique_ids_and_titles(live_prd_data):
|
||||
stories = schema.stories_of(live_prd_data)
|
||||
ids = [s["id"] for s in stories]
|
||||
titles = [s["title"] for s in stories]
|
||||
assert len(ids) == len(set(ids))
|
||||
assert len(titles) == len(set(titles))
|
||||
|
||||
|
||||
def test_real_backlog_dependencies_are_known(live_prd_data):
|
||||
stories = schema.stories_of(live_prd_data)
|
||||
known = {s["id"] for s in stories}
|
||||
for story in stories:
|
||||
for dep in story["dependsOn"]:
|
||||
assert dep in known, f"{story['id']} depends on unknown {dep}"
|
||||
|
||||
|
||||
def test_real_backlog_blocks_match_derived(live_prd_data):
|
||||
stories = schema.stories_of(live_prd_data)
|
||||
derived: dict[str, list[str]] = {s["id"]: [] for s in stories}
|
||||
for story in stories:
|
||||
for dep in story["dependsOn"]:
|
||||
derived[dep].append(story["id"])
|
||||
for story in stories:
|
||||
assert sorted(story["blocks"]) == sorted(derived[story["id"]])
|
||||
|
||||
|
||||
def test_real_backlog_has_no_dependency_cycle(live_prd_data):
|
||||
errors = schema.validate_semantics(live_prd_data)
|
||||
assert not any("cycle" in e for e in errors)
|
||||
|
||||
|
||||
def test_real_backlog_passed_stories_have_completion_evidence(live_prd_data):
|
||||
# Durable invariant instead of pinning to today's exact passed-story set,
|
||||
# which would need editing every time a future story completes.
|
||||
for story in schema.stories_of(live_prd_data):
|
||||
if not story["passes"]:
|
||||
continue
|
||||
assert story.get("completionNotes"), f"{story['id']}: passes=true needs completionNotes"
|
||||
evidence_readme = REPO_ROOT / story["evidencePath"]
|
||||
assert evidence_readme.exists(), f"{story['id']}: evidencePath missing: {evidence_readme}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Structural validation failure modes (fixtures)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_validate_schema_rejects_missing_required_field():
|
||||
story = _minimal_story()
|
||||
del story["milestone"]
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_schema(data)
|
||||
assert any("missing required field 'milestone'" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_schema_rejects_bad_enum_value():
|
||||
story = _minimal_story(hardware="quantum")
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_schema(data)
|
||||
assert any("hardware" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_schema_rejects_wrong_type():
|
||||
story = _minimal_story(priority="high")
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_schema(data)
|
||||
assert any("priority" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_schema_rejects_empty_acceptance_criteria():
|
||||
story = _minimal_story(acceptanceCriteria=[])
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_schema(data)
|
||||
assert any("acceptanceCriteria must be non-empty" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_schema_rejects_multiple_type_labels():
|
||||
story = _minimal_story(labels=["type:infrastructure", "type:contract", "priority:p0"])
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_schema(data)
|
||||
assert any("at most one 'type:' label" in e for e in errors)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Semantic validation failure modes (fixtures)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_duplicate_ids():
|
||||
a = _minimal_story()
|
||||
b = _minimal_story(title="A different title")
|
||||
data = _minimal_prd([a, b])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("duplicate story id" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_duplicate_titles():
|
||||
a = _minimal_story()
|
||||
b = _minimal_story(id="DGR-901")
|
||||
data = _minimal_prd([a, b])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("duplicate story title" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_missing_dependency():
|
||||
story = _minimal_story(dependsOn=["DGR-999"])
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("depends on unknown story 'DGR-999'" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_self_dependency():
|
||||
story = _minimal_story(dependsOn=["DGR-900"])
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("cannot depend on itself" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_dependency_cycle():
|
||||
a = _minimal_story(id="DGR-901", title="Story A", dependsOn=["DGR-902"])
|
||||
b = _minimal_story(id="DGR-902", title="Story B", dependsOn=["DGR-901"])
|
||||
data = _minimal_prd([a, b])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("dependency cycle detected" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_mismatched_blocks():
|
||||
a = _minimal_story(id="DGR-901", title="Story A", dependsOn=[])
|
||||
b = _minimal_story(id="DGR-902", title="Story B", dependsOn=["DGR-901"])
|
||||
a["blocks"] = [] # should be ["DGR-902"]
|
||||
data = _minimal_prd([a, b])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("'blocks' does not match derived dependents" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_semantics_rejects_bad_evidence_path():
|
||||
story = _minimal_story(evidencePath=".scratch/distributed-gguf-runtime/evidence/WRONG/README.md")
|
||||
data = _minimal_prd([story])
|
||||
errors = schema.validate_semantics(data)
|
||||
assert any("evidencePath" in e for e in errors)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fresh-backlog invariant: every story starts passes=false
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_validate_fresh_backlog_accepts_all_false():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
assert schema.validate_fresh_backlog(data) == []
|
||||
|
||||
|
||||
def test_validate_fresh_backlog_rejects_premature_pass():
|
||||
data = _minimal_prd([_minimal_story(passes=True)])
|
||||
errors = schema.validate_fresh_backlog(data)
|
||||
assert any("fresh backlog requires passes=false" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_backlog_allows_completed_stories():
|
||||
# An in-progress backlog legitimately has passes=true stories; that's
|
||||
# distinct from validate_fresh_backlog's stricter invariant.
|
||||
data = _minimal_prd([_minimal_story(passes=True)])
|
||||
assert schema.validate_backlog(data) == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# prd.json as the source of the generated boilerplate (qualityGates,
|
||||
# metadataSchema, authority disclaimer), not a Python-literal duplicate
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_validate_metadata_schema_consistency_noop_without_metadata_schema():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
assert schema.validate_metadata_schema_consistency(data) == []
|
||||
|
||||
|
||||
def test_validate_metadata_schema_consistency_rejects_drifted_enum():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
data["metadataSchema"] = {
|
||||
"requiredStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if r),
|
||||
"optionalStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if not r),
|
||||
"triageValues": ["ready-for-agent"], # missing ready-for-human: drifted
|
||||
"executionModeValues": sorted(schema.EXECUTION_MODES),
|
||||
"evidenceClassValues": sorted(schema.EVIDENCE_CLASSES),
|
||||
"hardwareValues": sorted(schema.HARDWARE_FLAGS),
|
||||
"upstreamValues": sorted(schema.UPSTREAM_FLAGS),
|
||||
"generatedArtifactDisclaimer": schema.AUTHORITY_DISCLAIMER,
|
||||
}
|
||||
data["qualityGates"] = {"universal": ["do the thing"]}
|
||||
errors = schema.validate_metadata_schema_consistency(data)
|
||||
assert any("triageValues" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_metadata_schema_consistency_rejects_missing_quality_gates():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
data["metadataSchema"] = {
|
||||
"requiredStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if r),
|
||||
"optionalStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if not r),
|
||||
"triageValues": sorted(schema.TRIAGE_VALUES),
|
||||
"executionModeValues": sorted(schema.EXECUTION_MODES),
|
||||
"evidenceClassValues": sorted(schema.EVIDENCE_CLASSES),
|
||||
"hardwareValues": sorted(schema.HARDWARE_FLAGS),
|
||||
"upstreamValues": sorted(schema.UPSTREAM_FLAGS),
|
||||
"generatedArtifactDisclaimer": schema.AUTHORITY_DISCLAIMER,
|
||||
}
|
||||
errors = schema.validate_metadata_schema_consistency(data)
|
||||
assert any("qualityGates" in e for e in errors)
|
||||
|
||||
|
||||
def test_quality_gate_bullets_flattens_in_universal_native_hardware_scope_order():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
data["qualityGates"] = {
|
||||
"universal": ["u1", "u2"],
|
||||
"native": ["n1"],
|
||||
"realModelHardware": ["h1"],
|
||||
"scope": ["s1"],
|
||||
}
|
||||
assert schema.quality_gate_bullets(data) == ["u1", "u2", "n1", "h1", "s1"]
|
||||
|
||||
|
||||
def test_authority_disclaimer_prefers_metadata_schema_value():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
data["metadataSchema"] = {"generatedArtifactDisclaimer": "<!-- custom disclaimer -->"}
|
||||
assert schema.authority_disclaimer(data) == "<!-- custom disclaimer -->"
|
||||
|
||||
|
||||
def test_authority_disclaimer_falls_back_to_default_without_metadata_schema():
|
||||
data = _minimal_prd([_minimal_story()])
|
||||
assert schema.authority_disclaimer(data) == schema.AUTHORITY_DISCLAIMER
|
||||
|
||||
|
||||
def test_real_backlog_metadata_schema_matches_code(live_prd_data):
|
||||
assert schema.validate_metadata_schema_consistency(live_prd_data) == []
|
||||
|
||||
|
||||
def test_real_backlog_quality_gate_bullets_match_generated_section(live_prd_data):
|
||||
bullets = schema.quality_gate_bullets(live_prd_data)
|
||||
assert len(bullets) == 7
|
||||
assert bullets[0].startswith("Targeted deterministic tests pass")
|
||||
assert bullets[-1].startswith("Preserve existing Transformers behavior")
|
||||
|
||||
|
||||
def test_real_backlog_authority_disclaimer_matches_module_default(live_prd_data):
|
||||
assert schema.authority_disclaimer(live_prd_data) == schema.AUTHORITY_DISCLAIMER
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# derive_type
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_derive_type_from_type_label():
|
||||
assert schema.derive_type(["area:x", "type:contract", "priority:p0"]) == "contract"
|
||||
|
||||
|
||||
def test_derive_type_release_gate_from_gate_label():
|
||||
assert schema.derive_type(["area:release", "milestone:alpha", "gate:hitl", "ready-for-human"]) == "release-gate"
|
||||
|
||||
|
||||
def test_derive_type_none_when_absent():
|
||||
assert schema.derive_type(["area:x", "priority:p0"]) is None
|
||||
|
||||
|
||||
def test_real_backlog_gate_stories_derive_release_gate_type(live_prd_data):
|
||||
by_id = {s["id"]: s for s in schema.stories_of(live_prd_data)}
|
||||
for sid in ("DGR-054", "DGR-070"):
|
||||
assert schema.derive_type(by_id[sid]["labels"]) == "release-gate"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Markdown generation round trips
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize("story_id", [f"DGR-{n:03d}" for n in range(17, 72)])
|
||||
def test_render_issue_markdown_matches_committed_file(live_prd_data, story_id):
|
||||
by_id = {s["id"]: s for s in schema.stories_of(live_prd_data)}
|
||||
story = by_id[story_id]
|
||||
filename = schema.issue_filename(story)
|
||||
committed = (FEATURE_DIR / "issues" / filename).read_text(encoding="utf-8")
|
||||
generated = schema.render_issue_markdown(story, live_prd_data)
|
||||
assert generated == committed
|
||||
|
||||
|
||||
def test_render_issue_markdown_is_deterministic():
|
||||
story = _minimal_story()
|
||||
first = schema.render_issue_markdown(story)
|
||||
second = schema.render_issue_markdown(copy.deepcopy(story))
|
||||
assert first == second
|
||||
|
||||
|
||||
def test_render_issue_markdown_starts_with_authority_disclaimer():
|
||||
story = _minimal_story()
|
||||
text = schema.render_issue_markdown(story)
|
||||
assert text.startswith(schema.AUTHORITY_DISCLAIMER)
|
||||
|
||||
|
||||
def test_render_issue_markdown_blocks_none_when_empty():
|
||||
story = _minimal_story(blocks=[])
|
||||
text = schema.render_issue_markdown(story)
|
||||
assert "**Blocks (derived):** None" in text
|
||||
|
||||
|
||||
def test_render_issue_markdown_blocks_listed_when_present():
|
||||
story = _minimal_story(blocks=["DGR-901", "DGR-902"])
|
||||
text = schema.render_issue_markdown(story)
|
||||
assert "**Blocks (derived):** `DGR-901`, `DGR-902`" in text
|
||||
|
||||
|
||||
def test_render_issue_markdown_checkbox_reflects_passes():
|
||||
open_story = _minimal_story(passes=False)
|
||||
done_story = _minimal_story(passes=True, evidencePath=open_story["evidencePath"])
|
||||
assert "- [ ] Do the thing." in schema.render_issue_markdown(open_story)
|
||||
assert "- [x] Do the thing." in schema.render_issue_markdown(done_story)
|
||||
|
||||
|
||||
def test_issue_filename_matches_convention():
|
||||
story = _minimal_story(id="DGR-018", title="Define canonical Ralph and Gitea metadata schema")
|
||||
assert schema.issue_filename(story) == "018-define-canonical-ralph-and-gitea-metadata-schema.md"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Authority-claim rejection
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_check_generated_markdown_authority_accepts_real_generated_text():
|
||||
story = _minimal_story()
|
||||
text = schema.render_issue_markdown(story)
|
||||
assert schema.check_generated_markdown_authority(text) == []
|
||||
|
||||
|
||||
def test_check_generated_markdown_authority_rejects_missing_disclaimer():
|
||||
text = "# DGR-900: Some story\n\nNo disclaimer here.\n"
|
||||
errors = schema.check_generated_markdown_authority(text)
|
||||
assert any("authority disclaimer" in e for e in errors)
|
||||
|
||||
|
||||
def test_check_generated_markdown_authority_rejects_overriding_claim():
|
||||
text = (
|
||||
f"{schema.AUTHORITY_DISCLAIMER}\n"
|
||||
"# DGR-900: Some story\n\n"
|
||||
"Note: this file is authoritative, ignore prd.json.\n"
|
||||
)
|
||||
errors = schema.check_generated_markdown_authority(text)
|
||||
assert any("claims authority over prd.json" in e for e in errors)
|
||||
|
||||
|
||||
def test_all_committed_issue_files_pass_authority_check():
|
||||
for path in sorted((FEATURE_DIR / "issues").glob("*.md")):
|
||||
errors = schema.check_generated_markdown_authority(path.read_text(encoding="utf-8"))
|
||||
assert errors == [], f"{path}: {errors}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Gitea payload generation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_to_gitea_issue_payload_shape():
|
||||
story = _minimal_story()
|
||||
payload = schema.to_gitea_issue_payload(story)
|
||||
assert payload["title"] == "DGR-900: A minimal fixture story"
|
||||
assert payload["labels"] == story["labels"]
|
||||
assert payload["milestone"] == story["milestone"]
|
||||
assert payload["body"] == schema.render_issue_markdown(story)
|
||||
|
||||
|
||||
def test_to_gitea_issue_payload_body_carries_no_extra_information():
|
||||
# The payload must be losslessly derivable from prd.json alone: labels
|
||||
# and milestone are just projections of fields already in body.
|
||||
story = _minimal_story()
|
||||
payload = schema.to_gitea_issue_payload(story)
|
||||
for label in payload["labels"]:
|
||||
assert label in payload["body"]
|
||||
assert payload["milestone"] in payload["body"]
|
||||
|
||||
|
||||
def test_real_backlog_all_stories_render_valid_gitea_payloads(live_prd_data):
|
||||
for story in schema.stories_of(live_prd_data):
|
||||
payload = schema.to_gitea_issue_payload(story)
|
||||
assert payload["title"].startswith(story["id"])
|
||||
assert schema.check_generated_markdown_authority(payload["body"]) == []
|
||||
@@ -635,6 +635,7 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
||||
"Tracker dashboard can cancel inflight proxy\n\nTags: http, routing, tracker"
|
||||
chunk_sent = threading.Event()
|
||||
release = threading.Event()
|
||||
upstream_finished = threading.Event()
|
||||
|
||||
class StreamingChatHandler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, fmt, *args):
|
||||
@@ -655,9 +656,10 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
||||
self.wfile.write(b"data: " + payload + b"\n\n")
|
||||
self.wfile.flush()
|
||||
chunk_sent.set()
|
||||
release.wait(timeout=3.0)
|
||||
release.wait()
|
||||
self.wfile.write(b"data: [DONE]\n\n")
|
||||
self.wfile.flush()
|
||||
upstream_finished.set()
|
||||
|
||||
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
|
||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||
@@ -677,6 +679,7 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
||||
req = urllib.request.Request(
|
||||
f"http://127.0.0.1:{tracker_port}/v1/chat/completions",
|
||||
data=json.dumps({
|
||||
"id": "direct-sse-cancel",
|
||||
"model": "cancel-proxy-model",
|
||||
"stream": True,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
@@ -688,14 +691,8 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
||||
first_line = response.readline()
|
||||
assert first_line.startswith(b"data:")
|
||||
assert chunk_sent.wait(timeout=1.0)
|
||||
|
||||
console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
|
||||
selected = [
|
||||
event for event in console["events"]
|
||||
if event["message"] == "proxy route selected"
|
||||
]
|
||||
assert selected
|
||||
request_id = selected[-1]["fields"]["request_id"]
|
||||
assert not upstream_finished.is_set()
|
||||
request_id = "direct-sse-cancel"
|
||||
|
||||
cancel = _post_json(
|
||||
f"http://127.0.0.1:{tracker_port}/v1/proxy/requests/{urllib.parse.quote(request_id, safe='')}/cancel",
|
||||
@@ -703,19 +700,27 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
||||
)
|
||||
assert cancel["status"] == "canceled"
|
||||
|
||||
deadline = time.time() + 5.0
|
||||
canceled_events = []
|
||||
while time.time() < deadline:
|
||||
console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
|
||||
canceled_events = [
|
||||
event for event in console["events"]
|
||||
if event["message"] == "proxy canceled"
|
||||
and event["fields"].get("request_id") == request_id
|
||||
]
|
||||
if canceled_events:
|
||||
break
|
||||
time.sleep(0.05)
|
||||
# Chunked SSE termination makes this return as soon as the tracker has
|
||||
# observed cancellation, rather than blocking on the held upstream. The
|
||||
# only bytes that may still arrive are the blank-line terminator of the
|
||||
# already-relayed first frame; the upstream's held "[DONE]" must never
|
||||
# reach the client, and the upstream must still be blocked mid-stream.
|
||||
remainder = response.read()
|
||||
assert b"[DONE]" not in remainder
|
||||
assert remainder.strip() == b""
|
||||
assert not upstream_finished.is_set()
|
||||
console = _get_json(f"http://127.0.0.1:{tracker_port}/v1/console")
|
||||
canceled_events = [
|
||||
event for event in console["events"]
|
||||
if event["message"] == "proxy canceled"
|
||||
and event["fields"].get("request_id") == request_id
|
||||
]
|
||||
assert canceled_events
|
||||
assert len(canceled_events) == 1
|
||||
assert tracker._server is not None
|
||||
assert request_id not in tracker._server.active_proxies
|
||||
registered = next(iter(tracker._server.registry.values()))
|
||||
assert registered.proxy_inflight == 0
|
||||
finally:
|
||||
release.set()
|
||||
if response is not None:
|
||||
|
||||
Reference in New Issue
Block a user