# DGR-019 evidence — lock alpha and beta performance contracts

**Completed:** 2026-07-22
**Branch:** `ralph/distributed-gguf-runtime`
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
**Dependency:** DGR-017 (`evidence/DGR-017/README.md`) — cleaned backlog reconciled to `origin/master`; no old pass state transferred.

## Objective

Freeze useful-speed, correctness, memory-fit, and stop/go thresholds for the DeepSeek V4 Flash
distributed GGUF track *before* any distributed implementation produces a benchmark result, per
`.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`.

## Pre-existing state found (not caused by this story)

Before any change in this session, `git status` showed `.scratch/distributed-gguf-runtime/prd.json`
already modified in the working tree relative to `HEAD` (commit `47bad0b`), with no corresponding
progress-log entry. Diffing against `HEAD` showed the working copy had **dropped** prd.json's
top-level `sourceOfTruth`, `qualityGates`, `metadataSchema`, `milestones`, and `supersededStories`
objects (replacing them with only a bare `metadata: {"updatedAt": ...}` stamp), while `userStories`
itself was byte-identical to `HEAD`. Running `tests/test_ralph_prd_schema.py` against the
as-found working tree confirmed the damage: 56 of 108 tests failed (every
`test_render_issue_markdown_matches_committed_file[...]` parametrization, since
`quality_gate_bullets`/`authority_disclaimer` fall back to module defaults once `qualityGates`/
`metadataSchema` are absent, which no longer match the committed issue files).

This is the same shape of problem DGR-018's evidence documented and fixed: an abandoned,
unexplained edit that silently dropped the schema/gates/milestone/provenance content this and
future stories depend on, while `scripts/ralph_prd_schema.py validate` did not catch it (those
top-level sections are optional-if-absent by design, so the CLI reported `OK: 55 stories
validated.` even with them missing). The most likely cause is `ralph-tui`'s own read/write of
`prd.json` as its task source, which only round-trips the fields it models
(`name`/`description`/`branchName`/`userStories`) and stamps its own `metadata.updatedAt`,
dropping any project-specific extension fields it doesn't know about.

Per `RALPH-CONTEXT.md`'s instruction to inspect `git status` and preserve unrelated work rather
than build on top of unexplained state, and following the DGR-018 precedent, the dropped fields
were restored verbatim from `HEAD` (`git show HEAD:.scratch/distributed-gguf-runtime/prd.json`)
while keeping the current `userStories` content (identical) and the current `metadata.updatedAt`
stamp. `tests/test_ralph_prd_schema.py` returned to `108 passed` immediately after the restore,
before any DGR-019-specific change was made.

## Changes

### `packages/node/meshnet_node/dgr_performance/` (new package)

- **`data/alpha-beta-contract-v1.json`** — the locked, versioned, machine-readable contract.
  `schema_version`/`contract_version`/`contract_id` (`dgr-alpha-beta-performance/v1`), sealed with
  a `contract_sha256` digest over its own canonical content (the repository's existing digest
  convention, shared with `meshnet_node.glm_alpha.contract`). Contents:
  - `prompt_set` — four fixed prompts (`short-instruction`, `code-completion`,
    `multi-step-reasoning`, `long-context-fill`) referenced by ID from every lane, so no lane can
    quietly drift onto a different workload.
  - `sampling` — greedy (`temperature=0`, `top_p=1`, `top_k=1`, `seed=1234`), matching
    `meshnet_node.recipe_benchmark.SamplingPolicy` defaults.
  - `lanes` — all four lanes named in the acceptance criteria. `controlled-safetensors` and
    `whole-model-gguf` are marked `locked_elsewhere: true` and point at the pre-existing immutable
    DGR-001 lock (`meshnet_node.performance_contract`, `contract_version=1`,
    `ContractThresholds`) rather than re-defining or risking a conflicting duplicate. Only
    `dense-distributed-gguf` and `v4-flash-distributed` are newly locked here, each with fixed
    `prompt_ids`, `context_tokens`/`output_tokens` (alpha- and beta-scale for the V4 lane),
    `concurrency_levels`, `hardware` (named certification-scenario topology, network class, device
    class, MTP-off note), and a `metrics` list drawn from the existing
    `recipe_benchmark`/`performance_contract`/`route_session_benchmark` metric vocabulary
    (`ttft_p50_ms`, `decode_tokens_per_sec`, `seam_bytes`, `seam_latency_ms`, ...).
  - `gain_attribution` — two disjoint metric sets, `quantization_model_fit_metrics` and
    `runtime_transport_batching_kernel_metrics`, plus the rule that a speed/fit claim must cite
    which axis moved it.
  - `certification_scenarios` — `quantization` (`Q4_K_M`, `Q8_0`, `bf16-reference`) and
    `stage_count` (`2-4-stage`, `10-plus-stage`) as named labels only, with an explicit rule that
    no product/runtime code path may hardcode them.
  - `alpha` — correctness thresholds (greedy token agreement, mean state cosine similarity,
    nonfinite-tensor/fail-closed checks, no dense-attention-fallback credit) plus a `useful_speed`
    block whose ratios (`1.25`/`0.75`-class, matching the already-locked DGR-001 25% convention)
    carry an explicit `human_approval` sub-block (`required: true`, `approved: false`,
    `approved_by: null`, `approved_at: null`). The ratio alone cannot satisfy alpha; DGR-054 must
    fill in the approval against real evidence. `mtp.reserved=true`/`enabled_for_alpha=false` per
    `RALPH-CONTEXT.md`. `verdicts: ["alpha", "optimize", "stop"]`.
  - `beta` — adds exactly `concurrency`, `long_context`, `failure`, `sustained_throughput` axes
    (16k-token long-context threshold matching the V4 lane's `beta_context_tokens`, no-silent-KV-
    migration and no-synthetic-workers failure rules, 30-minute sustained-throughput floor).
    `verdicts: ["beta", "targeted-optimization", "stop-rollback"]`.
  - `amendment_policy` — thresholds may not be weakened/moved/reinterpreted after results are
    known; a change requires a new `contract_id`/`contract_version` under human review.
- **`contract.py`** — loader/validator mirroring the proven
  `meshnet_node.glm_alpha.contract` pattern: `parse_contract` recomputes the canonical-JSON SHA-256
  over the document (excluding the digest field) and requires it match both the document's own
  declared `contract_sha256` *and* a digest pinned independently in code
  (`CONTRACT_V1_SHA256`), so neither an in-place edit nor a resealed mutation can pass silently.
  Structural checks enforce all four required lanes, that the two referenced lanes actually
  declare `locked_elsewhere`, that the two newly-locked lanes carry full benchmark-plan fields,
  that `alpha.verdicts`/`beta.verdicts` are exactly the three-outcome sets the release gates use,
  and — the one property with no analogue in `glm_alpha` — that
  `alpha.useful_speed.human_approval.required` is `true`. `seal_contract()` is the only supported
  way to produce a new digest, kept separate from load-time verification for the same reason
  `glm_alpha` keeps it separate.
- **`__init__.py`** — re-exports the public API, documented as the contract DGR-020, DGR-044,
  DGR-054, and DGR-070 are judged against.

### `tests/test_dgr_performance_contract.py` (new, 28 tests)

Deterministic, offline, GPU-free, model-download-free. Covers: packaged load and identity; digest
recomputation; all four lanes present; the two referenced lanes point at the real DGR-001 module
and its actual immutable thresholds (`min_decode_speedup == 1.25`, `max_resident_memory_ratio ==
0.75`); the two newly-locked lanes carry complete benchmark plans, fixed context/output/
concurrency; the shared prompt set and every lane's `prompt_ids`/`beta_prompt_ids` are a subset of
it; sampling is greedy; `gain_attribution`'s two metric sets are non-empty and disjoint;
certification-scenario names and rule text; **a structural test that greps every `.py` file under
`packages/node/meshnet_node` (excluding this contract's own module and data file) for the literal
strings `2-4-stage`/`10-plus-stage` and fails if any product module hardcodes them** — the concrete
form of "no product logic may hardcode them"; alpha verdicts/correctness/`human_approval`/MTP-off;
beta verdicts/axes/long-context/failure semantics; digest-mutation rejection (in-place and
resealed); missing-digest rejection; `load_contract` from an explicit path matches the packaged
load; `seal_contract` reproduces the pinned digest; amendment policy text.

### `.scratch/distributed-gguf-runtime/prd.json`

- Restored the top-level `sourceOfTruth`/`qualityGates`/`metadataSchema`/`milestones`/
  `supersededStories` objects dropped by the pre-existing unrelated edit (see above); kept the
  current `metadata.updatedAt` tooling stamp.
- Marked `DGR-019.passes = true` with `completionNotes` summarizing this outcome.

### `.scratch/distributed-gguf-runtime/issues/019-lock-alpha-and-beta-performance-contracts.md`

Regenerated via `python scripts/ralph_prd_schema.py render` to reflect `passes: true` (checked
acceptance criteria, "completed" status line, "Verified evidence" handoff line), matching the
convention DGR-017/DGR-018's issue files already use.

## Commands and results

```bash
.venv-rocm/bin/python -m pytest -q tests/test_dgr_performance_contract.py
```
```text
28 passed in 0.14s
```

```bash
.venv-rocm/bin/python -m pytest -q tests/test_ralph_prd_schema.py tests/test_dgr_performance_contract.py \
  tests/test_glm_alpha_target.py tests/test_recipe_benchmark.py tests/test_route_session_benchmark.py
```
```text
270 passed in 1.04s
```

```bash
.venv-rocm/bin/python -m compileall -q packages tests
```
Exit code 0, no output (all files compile).

```bash
git diff --check
```
Exit code 0 (no whitespace errors).

```bash
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
```
```text
OK: 55 stories validated.
```

```bash
.venv-rocm/bin/python -m pytest -q tests/ -k "not integration" --ignore=tests/test_shard_runtime_harness.py
```
```text
7 failed, 1146 passed, 11 skipped, 4 deselected, 3 warnings in 261.71s (0:04:21)
```
This full sweep was launched in the background while `prd.json`/the evidence README below were
still being written, so it raced its own inputs: one of its 7 failures
(`test_ralph_prd_schema.py::test_real_backlog_passed_stories_have_completion_evidence`) was this
story's own `passes=true`/evidence-README edit landing mid-run, not a real defect — re-running
`tests/test_ralph_prd_schema.py` alone afterward, against the finalized tree, gives
`108 passed`. The other 6 failures (`test_billing_ledger.py::
test_tracker_enables_billing_with_default_db`, `test_dynamic_routing.py::
test_admin_can_replace_a_served_model_and_release_it`, `test_dynamic_routing.py::
test_models_list_does_not_duplicate_a_preset_registered_by_hf_repo`, three cache tests in
`test_real_model_backend.py`) are in files this story's `git diff` never touches (`git diff --stat
HEAD -- tests/test_billing_ledger.py tests/test_dynamic_routing.py tests/test_real_model_backend.py`
is empty) and none of them import `dgr_performance`, `performance_contract`, or `glm_alpha`; they
are pre-existing baseline defects, not regressions from this story, in the same spirit as the
known `origin/master` limitations DGR-017's evidence recorded.

## Known limitations

- `tests/test_shard_runtime_harness.py` fails to *collect* in this environment
  (`ModuleNotFoundError: No module named 'grpc'`). This is a pre-existing environment gap from
  DGR-024's real generated-gRPC protocol harness, not something this story touched or caused; it is
  excluded from the sweep above rather than silently masked.
- Alpha's `useful_speed` ratios (`1.25`/`0.75`-class) are proposed thresholds held at the same
  margin already locked for the whole-model contract (DGR-001/v1). They are locked numbers, but
  `human_approval.required=true` means DGR-054 may not treat them as self-certifying from the
  ratio alone — a human must approve the observed ratio against real evidence. This session did
  not, and could not, supply that approval: no distributed benchmark evidence exists yet.
- `v4-flash-distributed`'s `reference_baseline` documents that a safetensors DeepSeek V4 Flash
  distributed baseline may not yet be pinned (that is DGR-044's job); until then, comparisons must
  fall back to `dense-distributed-gguf` runtime/transport overhead as an explicit, stated
  limitation rather than a silent substitution.
- This is a specification-materialization story; per the shared quality gates, it is intentionally
  left uncommitted for manual review rather than given the "one scoped story commit" other stories
  get.

## Dependency handoff

DGR-020 (run the controlled whole-model baseline) consumes the DGR-001 lock referenced — not
redefined — by this contract's `controlled-safetensors`/`whole-model-gguf` lanes.

DGR-044 (pin the DeepSeek V4 Flash target contract) and DGR-054/DGR-070 (enforce the alpha/beta
gates) must load `meshnet_node.dgr_performance.load_contract()` and judge results against its
`dense-distributed-gguf`/`v4-flash-distributed` lanes and `alpha`/`beta` sections without changing
any threshold. DGR-054 specifically must populate `alpha.useful_speed.human_approval`
(`approved`/`approved_by`/`approved_at`) as part of publishing its verdict — a satisfied ratio
without a filled-in approval is not alpha certification. Any amendment must open a new
`contract_id`/`contract_version` under human review per `amendment_policy`; this document and its
digest are not editable in place.
