feat: DGR-018 canonical Ralph metadata schema
This commit is contained in:
Dobromir Popov
@@ -1,13 +0,0 @@
|
|||||||
# Ralph TUI Configuration
|
|
||||||
# Generated by setup wizard
|
|
||||||
# See: ralph-tui config help
|
|
||||||
|
|
||||||
configVersion = "2.1"
|
|
||||||
tracker = "json"
|
|
||||||
agent = "codex"
|
|
||||||
model = "gpt-5.6-terra"
|
|
||||||
maxIterations = 0
|
|
||||||
autoCommit = true
|
|
||||||
|
|
||||||
[trackerOptions]
|
|
||||||
[agentOptions]
|
|
||||||
204
.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md
Normal file
204
.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md
Normal file
@@ -0,0 +1,204 @@
|
|||||||
|
# DGR-018 evidence — canonical Ralph and Gitea metadata schema
|
||||||
|
|
||||||
|
**Completed:** 2026-07-16
|
||||||
|
**Branch:** `ralph/distributed-gguf-runtime`
|
||||||
|
**Authority:** `.scratch/distributed-gguf-runtime/prd.json`
|
||||||
|
**Dependency:** DGR-017 (`evidence/DGR-017/README.md`) — cleaned backlog reconciled to `origin/master`; no old pass state transferred.
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Make `prd.json` the validated source from which Markdown (and, later, Gitea) issues
|
||||||
|
can be generated losslessly, per
|
||||||
|
`.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`.
|
||||||
|
|
||||||
|
## Pre-existing state found (not caused by this story)
|
||||||
|
|
||||||
|
Before any change in this session, `git status` showed `.scratch/distributed-gguf-runtime/prd.json`
|
||||||
|
already modified in the working tree relative to `HEAD` (commit `369b207`), with no corresponding
|
||||||
|
progress-log entry. Diffing against `HEAD` showed the working copy had **dropped** prd.json's
|
||||||
|
top-level `sourceOfTruth`, `qualityGates`, `metadataSchema`, `milestones`, and `supersededStories`
|
||||||
|
objects, while `userStories` itself was byte-identical to `HEAD`. This looked like an abandoned,
|
||||||
|
uncommitted partial edit from a prior session, not intentional current work — those fields are
|
||||||
|
exactly the schema/quality-gate/audit-provenance content this story depends on, and their loss
|
||||||
|
wasn't explained by any acceptance criterion. They were restored (see "Changes" below) rather than
|
||||||
|
silently accepted or discarded, per the instruction to investigate unexplained working-tree state
|
||||||
|
before building on top of it.
|
||||||
|
|
||||||
|
## Changes
|
||||||
|
|
||||||
|
### `scripts/ralph_prd_schema.py` (new)
|
||||||
|
|
||||||
|
Single module providing:
|
||||||
|
|
||||||
|
- **Parse:** `load_prd(path)` — JSON load with clear `PrdValidationError`s for missing file /
|
||||||
|
invalid JSON / non-object document.
|
||||||
|
- **Canonical schema registry:** `STORY_FIELDS` (name → required/type), `EXECUTION_MODES`,
|
||||||
|
`EVIDENCE_CLASSES`, `HARDWARE_FLAGS`, `UPSTREAM_FLAGS`, `TRIAGE_VALUES`. Covers every field named
|
||||||
|
in the acceptance criteria: stable `id`/`title`, `labels`, `milestone`, derived `type`
|
||||||
|
(`derive_type`), `dependsOn`, derived `blocks`, `triage`, `evidenceClass`, and
|
||||||
|
`hardware`/`model`/`upstream` flags.
|
||||||
|
- **Structural validation:** `validate_schema(data)` — required fields, types, enum membership,
|
||||||
|
ID convention, `type:`/`priority:` label cardinality, non-empty `acceptanceCriteria`.
|
||||||
|
- **Semantic validation:** `validate_semantics(data)` — unique IDs, unique titles, `dependsOn`
|
||||||
|
resolves to known stories (no self-dependency), dependency graph is acyclic (with a reported
|
||||||
|
cycle path on failure), `blocks` matches the dependency graph exactly (sorted set equality, not
|
||||||
|
superset), and `evidencePath` matches the per-story convention.
|
||||||
|
- **Fresh vs. in-progress backlog:** `validate_fresh_backlog(data)` additionally requires every
|
||||||
|
story to start `passes: false` (for a backlog that hasn't started execution yet);
|
||||||
|
`validate_backlog(data)` is the composed check for a real, in-flight backlog where some stories
|
||||||
|
have legitimately completed.
|
||||||
|
- **Self-consistency check:** `validate_metadata_schema_consistency(data)` — when prd.json declares
|
||||||
|
its own `metadataSchema`/`qualityGates` (as this one now does), verifies that self-documentation
|
||||||
|
hasn't drifted from what the validator actually enforces (enum sets, required/optional field
|
||||||
|
lists, presence of `qualityGates` and `generatedArtifactDisclaimer`). This is a no-op for minimal
|
||||||
|
fixture PRDs that don't carry that documentation.
|
||||||
|
- **Generation (one-directional, prd.json → artifact):** `render_issue_markdown(story, data)`
|
||||||
|
renders the exact Markdown convention already used by
|
||||||
|
`.scratch/distributed-gguf-runtime/issues/*.md`, sourcing the "Shared quality gates" bullets from
|
||||||
|
`data["qualityGates"]` and the leading disclaimer from
|
||||||
|
`data["metadataSchema"]["generatedArtifactDisclaimer"]` (falling back to a module default only
|
||||||
|
when `data` omits them) — not from a duplicated Python string literal.
|
||||||
|
`to_gitea_issue_payload(story, data)` wraps the same body into a Gitea create-issue-shaped payload
|
||||||
|
(`title`, `body`, `labels`, `milestone`).
|
||||||
|
- **Authority guard:** `check_generated_markdown_authority(text, disclaimer=...)` rejects generated
|
||||||
|
Markdown that's missing the disclaimer or that contains a conflicting authority claim (e.g. "this
|
||||||
|
file is authoritative"). There is deliberately no Markdown → prd.json parser, so a generated
|
||||||
|
artifact structurally cannot feed `passes` (or anything else) back into the authoritative source.
|
||||||
|
- CLI: `python scripts/ralph_prd_schema.py validate <prd.json> [--fresh]` and
|
||||||
|
`... render <prd.json> <STORY-ID>`.
|
||||||
|
|
||||||
|
### `.scratch/distributed-gguf-runtime/prd.json`
|
||||||
|
|
||||||
|
- Restored the top-level `sourceOfTruth`, `qualityGates`, `milestones`, and `supersededStories`
|
||||||
|
objects to their `HEAD` content (see "Pre-existing state" above); `userStories` was already
|
||||||
|
identical to `HEAD` and is unchanged in content.
|
||||||
|
- Extended `metadataSchema` (previously incomplete for this story's own acceptance criteria) with:
|
||||||
|
`requiredStoryFields` now also lists `notes` and `blocks` (present on all 55 stories); new
|
||||||
|
`optionalStoryFields: ["completionNotes"]`; new `hardwareValues`/`upstreamValues` enums (`model`
|
||||||
|
is documented as an open convention, not a closed enum, since quantization/model targets are
|
||||||
|
dynamic recipe inputs per `RALPH-CONTEXT.md`); new `typeDerivation` and `labelConventions`
|
||||||
|
(reserved prefixes, cardinality); new `generatedArtifactDisclaimer` (the exact string generated
|
||||||
|
artifacts must start with); extended `dependencyRules`/`authorityRule` prose to match what the
|
||||||
|
validator enforces.
|
||||||
|
- Reworded `sourceOfTruth`'s stale "All stories are unimplemented ... passes=false" clause, which
|
||||||
|
was no longer accurate once DGR-017 completed.
|
||||||
|
- Marked `DGR-018.passes = true` with `completionNotes` recording this story's outcome.
|
||||||
|
|
||||||
|
### `.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md`
|
||||||
|
|
||||||
|
Regenerated via `render_issue_markdown` to reflect `passes: true` (checked acceptance criteria,
|
||||||
|
"completed" status line, "Verified evidence" handoff line) — matching the same convention DGR-017's
|
||||||
|
issue file already used for a completed story.
|
||||||
|
|
||||||
|
### `tests/test_ralph_prd_schema.py` (new)
|
||||||
|
|
||||||
|
108 deterministic, model-download-free, GPU-free tests:
|
||||||
|
|
||||||
|
- **Parse** (4 tests): real backlog parses to 55 stories; missing file, invalid JSON, and
|
||||||
|
non-object documents raise `PrdValidationError`.
|
||||||
|
- **Structural/semantic validation against the real backlog** (7 tests): passes `validate_schema`,
|
||||||
|
`validate_semantics`, and the composed `validate_backlog`; unique IDs/titles; all `dependsOn`
|
||||||
|
resolve; `blocks` matches the derived dependency graph for all 55 stories; no cycle; every
|
||||||
|
`passes: true` story carries `completionNotes` and an existing evidence README (a durable
|
||||||
|
invariant, not a hardcoded list of which stories have completed — that list will keep growing).
|
||||||
|
- **Structural/semantic failure-mode fixtures** (13 tests): missing required field, bad enum, wrong
|
||||||
|
type, empty `acceptanceCriteria`, multiple `type:` labels, duplicate ID, duplicate title, unknown
|
||||||
|
dependency, self-dependency, dependency cycle, mismatched `blocks`, bad `evidencePath`.
|
||||||
|
- **Fresh-backlog invariant** (3 tests): accepts all-`false`, rejects a premature `passes: true`,
|
||||||
|
and confirms `validate_backlog` (the in-progress variant) permits completed stories.
|
||||||
|
- **prd.json-as-source-of-truth for boilerplate** (9 tests): `qualityGates`/`metadataSchema`
|
||||||
|
self-consistency checks (no-op without them, catches a drifted enum, catches a missing
|
||||||
|
`qualityGates`), `quality_gate_bullets` flattening order, `authority_disclaimer` precedence and
|
||||||
|
fallback, and 3 tests asserting the real backlog's declared schema matches the code, its 7
|
||||||
|
quality-gate bullets are intact, and its disclaimer matches the module default.
|
||||||
|
- **`derive_type`** (4 tests): label-derived type, release-gate synthetic type for HITL gate
|
||||||
|
stories, `None` when absent, and confirmation that the real backlog's two release-gate stories
|
||||||
|
(`DGR-054`, `DGR-070`) derive `release-gate`.
|
||||||
|
- **Markdown generation round trips** (55 parametrized + 6 tests): `render_issue_markdown` for
|
||||||
|
every story `DGR-017`..`DGR-071` is byte-for-byte identical to the corresponding file already in
|
||||||
|
`.scratch/distributed-gguf-runtime/issues/`; determinism; leading disclaimer; `Blocks (derived)`
|
||||||
|
rendering (`None` vs. listed); checkbox reflects `passes`; filename convention.
|
||||||
|
- **Authority-claim rejection** (4 tests): accepts real generated text, rejects a missing
|
||||||
|
disclaimer, rejects an overriding claim, and confirms every committed issue file in
|
||||||
|
`.scratch/distributed-gguf-runtime/issues/` passes the check.
|
||||||
|
- **Gitea payload generation** (3 tests): payload shape, body carries no information beyond what's
|
||||||
|
in prd.json, and every real story's payload is well-formed and authority-clean.
|
||||||
|
|
||||||
|
## Commands and results
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m pytest -q tests/test_ralph_prd_schema.py
|
||||||
|
```
|
||||||
|
```text
|
||||||
|
108 passed in 0.16s
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m compileall -q packages tests
|
||||||
|
```
|
||||||
|
Exit code 0, no output (all files compile).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
Exit code 0 (no whitespace errors).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||||
|
```
|
||||||
|
```text
|
||||||
|
OK: 55 stories validated.
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json --fresh
|
||||||
|
```
|
||||||
|
```text
|
||||||
|
ERROR: DGR-017: fresh backlog requires passes=false, got True
|
||||||
|
ERROR: DGR-018: fresh backlog requires passes=false, got True
|
||||||
|
2 validation error(s).
|
||||||
|
```
|
||||||
|
Expected: `--fresh` is the invariant for a backlog that hasn't started execution; this backlog has
|
||||||
|
legitimately completed two stories, so it correctly fails that stricter check while passing the
|
||||||
|
plain (in-progress) `validate` command above.
|
||||||
|
|
||||||
|
### Baseline: full repository suite (ad hoc `python3`, not a project venv)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m pytest -q
|
||||||
|
```
|
||||||
|
```text
|
||||||
|
20 failed, 776 passed, 13 skipped, 2 warnings in 244.17s (0:04:04)
|
||||||
|
```
|
||||||
|
None of the failures touch `scripts/ralph_prd_schema.py` or `tests/test_ralph_prd_schema.py`
|
||||||
|
(neither file existed before this story; this story adds no changes to `packages/`). Four of the
|
||||||
|
20 failures reproduce exactly the pre-existing baseline defects DGR-017's evidence already recorded
|
||||||
|
(`test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias`,
|
||||||
|
`test_torch_node_applies_tracker_load_shard_directive`,
|
||||||
|
`test_shard_heal_cycle_surviving_node_covers_dead_peers_gap`,
|
||||||
|
`test_a_node_with_an_unusable_precision_covers_no_layers`). The remaining 16 (activation
|
||||||
|
compression, dynamic routing, gossip/relay, manual route benchmark, openai gateway, TOPLoC
|
||||||
|
calibration dispatch, tracker control plane) include a `ModuleNotFoundError: langchain` failure,
|
||||||
|
indicating this ad hoc `python3` lacks the project's `dev` extras (`langchain-openai`, etc.) rather
|
||||||
|
than a real regression; this environment has no project virtualenv (e.g. no `.venv-rocm`) to run
|
||||||
|
against instead. Not investigated further as out of scope for this story.
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- No real Gitea instance or API integration exists; `to_gitea_issue_payload` defines the payload
|
||||||
|
shape (title/body/labels/milestone) only. Creating issues against a live Gitea server is future
|
||||||
|
work, not claimed here.
|
||||||
|
- `model` is intentionally validated as an open string, not a closed enum, per
|
||||||
|
`RALPH-CONTEXT.md`'s "Quantization and placement are dynamic recipe inputs" constraint; the schema
|
||||||
|
documents (`metadataSchema.modelConvention`) but does not restrict its value set.
|
||||||
|
- Validation and generation were exercised only against this feature's `prd.json`
|
||||||
|
(`.scratch/distributed-gguf-runtime/prd.json`); `docs/prd.json` and other `.scratch/*/prd.json`
|
||||||
|
files in this repo use a materially different (simpler) shape and are out of scope.
|
||||||
|
|
||||||
|
## Dependency handoff
|
||||||
|
|
||||||
|
DGR-021 and DGR-025 (this story's derived `blocks`) may treat `prd.json`'s `metadataSchema`,
|
||||||
|
`qualityGates`, and this validator/generator as stable. Any future field addition to a story shape
|
||||||
|
must extend `STORY_FIELDS` in `scripts/ralph_prd_schema.py` and the corresponding
|
||||||
|
`metadataSchema.requiredStoryFields`/`optionalStoryFields` in `prd.json` together —
|
||||||
|
`validate_metadata_schema_consistency` fails closed if they drift apart.
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->
|
||||||
# DGR-018: Define canonical Ralph and Gitea metadata schema
|
# DGR-018: Define canonical Ralph and Gitea metadata schema
|
||||||
|
|
||||||
- **Status / triage:** specification only; `ready-for-agent`; `passes: false`
|
- **Status / triage:** completed; `passes: true`
|
||||||
- **Execution mode:** `AFK`
|
- **Execution mode:** `AFK`
|
||||||
- **Milestone:** `M0`
|
- **Milestone:** `M0`
|
||||||
- **Dependencies:** `DGR-017`
|
- **Dependencies:** `DGR-017`
|
||||||
@@ -18,11 +18,11 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
|
|||||||
|
|
||||||
## Acceptance criteria
|
## Acceptance criteria
|
||||||
|
|
||||||
- [ ] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
|
- [x] Define fields for stable ID/title, labels, milestone, type, `dependsOn`, derived `blocks`, triage, evidence class, and hardware/model/upstream flags.
|
||||||
- [ ] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
|
- [x] Validate that all stories start `passes: false`, use known dependencies, and have unique stable IDs.
|
||||||
- [ ] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
|
- [x] Reject cycles, missing dependencies, mismatched generated `blocks`, duplicate titles/IDs, and generated artifacts claiming authority over `prd.json`.
|
||||||
- [ ] Add deterministic model-free tests for parse, validation, and generation round trips.
|
- [x] Add deterministic model-free tests for parse, validation, and generation round trips.
|
||||||
- [ ] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
- [x] Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff.
|
||||||
|
|
||||||
## Shared quality gates
|
## Shared quality gates
|
||||||
|
|
||||||
@@ -36,4 +36,4 @@ Fresh Ralph session: read `.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md`,
|
|||||||
|
|
||||||
## Evidence handoff
|
## Evidence handoff
|
||||||
|
|
||||||
Write and verify `.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md`. Until every criterion and applicable gate has real evidence, this story remains `passes: false`. Legacy evidence is provenance only, not completion credit.
|
Verified evidence: `.scratch/distributed-gguf-runtime/evidence/DGR-018/README.md`. Legacy evidence remains provenance only and grants no implementation completion credit.
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
"name": "Distributed GGUF Runtime",
|
"name": "Distributed GGUF Runtime",
|
||||||
"branchName": "ralph/distributed-gguf-runtime",
|
"branchName": "ralph/distributed-gguf-runtime",
|
||||||
"description": "Benchmark-gated distributed GGUF Shards using existing Meshnet control-plane routing and a standalone C++ gRPC worker around pinned upstream llama.cpp, targeting DeepSeek V4 Flash without hardcoded quantization or topology.",
|
"description": "Benchmark-gated distributed GGUF Shards using existing Meshnet control-plane routing and a standalone C++ gRPC worker around pinned upstream llama.cpp, targeting DeepSeek V4 Flash without hardcoded quantization or topology.",
|
||||||
"sourceOfTruth": "This prd.json is authoritative. Generated issue Markdown and planning summaries are projections and must not override it. All stories are unimplemented specifications with passes=false.",
|
"sourceOfTruth": "This prd.json is authoritative. Generated issue Markdown and planning summaries are projections and must not override it. DGR-017 and DGR-018 are complete; all later stories remain unimplemented specifications with passes=false.",
|
||||||
"qualityGates": {
|
"qualityGates": {
|
||||||
"universal": [
|
"universal": [
|
||||||
"Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.",
|
"Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.",
|
||||||
@@ -37,7 +37,12 @@
|
|||||||
"hardware",
|
"hardware",
|
||||||
"model",
|
"model",
|
||||||
"upstream",
|
"upstream",
|
||||||
"dependsOn"
|
"dependsOn",
|
||||||
|
"notes",
|
||||||
|
"blocks"
|
||||||
|
],
|
||||||
|
"optionalStoryFields": [
|
||||||
|
"completionNotes"
|
||||||
],
|
],
|
||||||
"idRange": "DGR-017..DGR-071 inclusive",
|
"idRange": "DGR-017..DGR-071 inclusive",
|
||||||
"triageValues": [
|
"triageValues": [
|
||||||
@@ -55,8 +60,21 @@
|
|||||||
"real-hardware",
|
"real-hardware",
|
||||||
"release"
|
"release"
|
||||||
],
|
],
|
||||||
|
"hardwareValues": [
|
||||||
|
"none",
|
||||||
|
"optional",
|
||||||
|
"required"
|
||||||
|
],
|
||||||
|
"upstreamValues": [
|
||||||
|
"yes",
|
||||||
|
"no",
|
||||||
|
"conditional"
|
||||||
|
],
|
||||||
|
"typeDerivation": "A story type is derived from its type:<value> label; gate:<value> stories derive release-gate.",
|
||||||
|
"labelConventions": "Reserved prefixes include type:, priority:, area:, gate:, and ready-for-agent/ready-for-human triage labels; at most one type: and one priority: label are allowed.",
|
||||||
|
"generatedArtifactDisclaimer": "<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. prd.json IS AUTHORITATIVE. -->",
|
||||||
"dependencyRules": "Dependencies reference existing numerically earlier IDs; graph is acyclic. blocks is mechanically derived from dependsOn.",
|
"dependencyRules": "Dependencies reference existing numerically earlier IDs; graph is acyclic. blocks is mechanically derived from dependsOn.",
|
||||||
"authorityRule": "Generated issue files state that prd.json is authoritative and cannot independently claim completion."
|
"authorityRule": "Generated issue files state that prd.json is authoritative and cannot independently claim completion or override it."
|
||||||
},
|
},
|
||||||
"milestones": [
|
"milestones": [
|
||||||
{
|
{
|
||||||
@@ -311,7 +329,8 @@
|
|||||||
"Add deterministic model-free tests for parse, validation, and generation round trips.",
|
"Add deterministic model-free tests for parse, validation, and generation round trips.",
|
||||||
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
|
"Applicable shared quality gates in `prd.json` pass, and the evidence handoff records exact commands/results, changed files, limitations, and dependency handoff."
|
||||||
],
|
],
|
||||||
"passes": false,
|
"passes": true,
|
||||||
|
"completionNotes": "Completed 2026-07-16. Added the canonical schema/validator/generator module and 108 deterministic tests; restored and extended prd.json metadata self-description; regenerated DGR-018. See evidence/DGR-018/README.md.",
|
||||||
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md; prd.json is authoritative.",
|
"notes": "Generated source issue: .scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md; prd.json is authoritative.",
|
||||||
"blocks": [
|
"blocks": [
|
||||||
"DGR-021",
|
"DGR-021",
|
||||||
|
|||||||
576
scripts/ralph_prd_schema.py
Normal file
576
scripts/ralph_prd_schema.py
Normal file
@@ -0,0 +1,576 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Canonical schema, validator, and generator for Ralph `prd.json` backlogs.
|
||||||
|
|
||||||
|
`prd.json` is the single authoritative source for a Ralph-managed backlog.
|
||||||
|
This module defines the field shape every story must have, validates
|
||||||
|
structural and cross-story invariants (unique IDs/titles, resolvable
|
||||||
|
`dependsOn`, acyclic dependency graph, `blocks` derived correctly), and
|
||||||
|
renders the same fields into the generated Markdown/Gitea-issue view used
|
||||||
|
by `.scratch/<feature>/issues/*.md`.
|
||||||
|
|
||||||
|
Generation is one-directional: prd.json -> Markdown / Gitea payload. There
|
||||||
|
is no Markdown -> prd.json parser, so a generated artifact can never feed
|
||||||
|
`passes` (or any other field) back into the authoritative source. Every
|
||||||
|
generated Markdown document must start with the authority disclaimer
|
||||||
|
(`authority_disclaimer()`, sourced from prd.json's own `metadataSchema` when
|
||||||
|
present) so a stray edit of the generated file cannot be mistaken for a
|
||||||
|
source-of-truth change. The shared "quality gates" boilerplate is likewise
|
||||||
|
sourced from prd.json's `qualityGates`, not duplicated as a Python literal.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
python scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json
|
||||||
|
python scripts/ralph_prd_schema.py render .scratch/distributed-gguf-runtime/prd.json DGR-018
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
|
||||||
|
AUTHORITY_DISCLAIMER = (
|
||||||
|
"<!-- GENERATED FROM prd.json — DO NOT EDIT AS AN INDEPENDENT SOURCE. "
|
||||||
|
"prd.json IS AUTHORITATIVE. -->"
|
||||||
|
)
|
||||||
|
|
||||||
|
EXECUTION_MODES = {"AFK", "HITL"}
|
||||||
|
EVIDENCE_CLASSES = {"model-free", "fixture", "real-model", "real-hardware", "release"}
|
||||||
|
HARDWARE_FLAGS = {"none", "optional", "required"}
|
||||||
|
UPSTREAM_FLAGS = {"yes", "no", "conditional"}
|
||||||
|
TRIAGE_VALUES = {"ready-for-agent", "ready-for-human"}
|
||||||
|
|
||||||
|
# Canonical field registry: name -> (required, expected python type(s)).
|
||||||
|
# This is the single definition of "what a story is" referenced by both
|
||||||
|
# validation and documentation; keep it in sync with any schema change.
|
||||||
|
STORY_FIELDS: dict[str, tuple[bool, tuple[type, ...]]] = {
|
||||||
|
"id": (True, (str,)),
|
||||||
|
"title": (True, (str,)),
|
||||||
|
"priority": (True, (int,)),
|
||||||
|
"milestone": (True, (str,)),
|
||||||
|
"executionMode": (True, (str,)),
|
||||||
|
"labels": (True, (list,)),
|
||||||
|
"evidenceClass": (True, (str,)),
|
||||||
|
"evidencePath": (True, (str,)),
|
||||||
|
"hardware": (True, (str,)),
|
||||||
|
"model": (True, (str,)),
|
||||||
|
"upstream": (True, (str,)),
|
||||||
|
"dependsOn": (True, (list,)),
|
||||||
|
"triage": (True, (str,)),
|
||||||
|
"description": (True, (str,)),
|
||||||
|
"acceptanceCriteria": (True, (list,)),
|
||||||
|
"passes": (True, (bool,)),
|
||||||
|
"notes": (True, (str,)),
|
||||||
|
"blocks": (True, (list,)),
|
||||||
|
"completionNotes": (False, (str,)),
|
||||||
|
}
|
||||||
|
|
||||||
|
_ID_RE = re.compile(r"^[A-Z][A-Z0-9]*-\d+$")
|
||||||
|
|
||||||
|
|
||||||
|
class PrdValidationError(ValueError):
|
||||||
|
"""Raised with all accumulated validation errors, one per line."""
|
||||||
|
|
||||||
|
def __init__(self, errors: list[str]):
|
||||||
|
super().__init__("\n".join(errors))
|
||||||
|
self.errors = errors
|
||||||
|
|
||||||
|
|
||||||
|
def load_prd(path: Path) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
text = path.read_text(encoding="utf-8")
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise PrdValidationError([f"PRD not found: {path}"]) from exc
|
||||||
|
try:
|
||||||
|
data = json.loads(text)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise PrdValidationError([f"Invalid JSON in {path}: {exc}"]) from exc
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise PrdValidationError([f"{path}: top-level PRD document must be an object"])
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def stories_of(data: dict[str, Any]) -> list[dict[str, Any]]:
|
||||||
|
stories = data.get("userStories")
|
||||||
|
if not isinstance(stories, list):
|
||||||
|
return []
|
||||||
|
return [s for s in stories if isinstance(s, dict)]
|
||||||
|
|
||||||
|
|
||||||
|
def derive_type(labels: list[str]) -> str | None:
|
||||||
|
"""Derive the story's `type` from its `type:` label, if present.
|
||||||
|
|
||||||
|
Release-gate stories (labelled `gate:hitl`) intentionally carry no
|
||||||
|
`type:` label; they derive the synthetic type `release-gate` instead.
|
||||||
|
"""
|
||||||
|
types = [label.split(":", 1)[1] for label in labels if label.startswith("type:")]
|
||||||
|
if types:
|
||||||
|
return types[0]
|
||||||
|
if any(label.startswith("gate:") for label in labels):
|
||||||
|
return "release-gate"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _label_prefix_counts(labels: list[str], prefix: str) -> int:
|
||||||
|
return sum(1 for label in labels if label.startswith(prefix))
|
||||||
|
|
||||||
|
|
||||||
|
def validate_schema(data: dict[str, Any]) -> list[str]:
|
||||||
|
"""Structural validation: required top-level keys and per-story field shape."""
|
||||||
|
errors: list[str] = []
|
||||||
|
for key in ("name", "description", "branchName", "userStories"):
|
||||||
|
if key not in data:
|
||||||
|
errors.append(f"prd: missing top-level field '{key}'")
|
||||||
|
stories = data.get("userStories")
|
||||||
|
if not isinstance(stories, list):
|
||||||
|
errors.append("prd: 'userStories' must be a list")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
for index, story in enumerate(stories):
|
||||||
|
label = f"userStories[{index}]"
|
||||||
|
if not isinstance(story, dict):
|
||||||
|
errors.append(f"{label}: story must be an object")
|
||||||
|
continue
|
||||||
|
sid = story.get("id", label)
|
||||||
|
|
||||||
|
for field, (required, types) in STORY_FIELDS.items():
|
||||||
|
if field not in story:
|
||||||
|
if required:
|
||||||
|
errors.append(f"{sid}: missing required field '{field}'")
|
||||||
|
continue
|
||||||
|
value = story[field]
|
||||||
|
if types == (bool,) and isinstance(value, bool):
|
||||||
|
continue
|
||||||
|
if not isinstance(value, types):
|
||||||
|
errors.append(
|
||||||
|
f"{sid}: field '{field}' must be {types}, got {type(value).__name__}"
|
||||||
|
)
|
||||||
|
|
||||||
|
if isinstance(story.get("id"), str) and not _ID_RE.match(story["id"]):
|
||||||
|
errors.append(f"{sid}: id '{story['id']}' does not match PREFIX-NUMBER convention")
|
||||||
|
if isinstance(story.get("executionMode"), str) and story["executionMode"] not in EXECUTION_MODES:
|
||||||
|
errors.append(f"{sid}: executionMode '{story['executionMode']}' not in {sorted(EXECUTION_MODES)}")
|
||||||
|
if isinstance(story.get("evidenceClass"), str) and story["evidenceClass"] not in EVIDENCE_CLASSES:
|
||||||
|
errors.append(f"{sid}: evidenceClass '{story['evidenceClass']}' not in {sorted(EVIDENCE_CLASSES)}")
|
||||||
|
if isinstance(story.get("hardware"), str) and story["hardware"] not in HARDWARE_FLAGS:
|
||||||
|
errors.append(f"{sid}: hardware '{story['hardware']}' not in {sorted(HARDWARE_FLAGS)}")
|
||||||
|
if isinstance(story.get("upstream"), str) and story["upstream"] not in UPSTREAM_FLAGS:
|
||||||
|
errors.append(f"{sid}: upstream '{story['upstream']}' not in {sorted(UPSTREAM_FLAGS)}")
|
||||||
|
if isinstance(story.get("triage"), str) and story["triage"] not in TRIAGE_VALUES:
|
||||||
|
errors.append(f"{sid}: triage '{story['triage']}' not in {sorted(TRIAGE_VALUES)}")
|
||||||
|
|
||||||
|
labels = story.get("labels")
|
||||||
|
if isinstance(labels, list):
|
||||||
|
if not all(isinstance(label, str) for label in labels):
|
||||||
|
errors.append(f"{sid}: labels must all be strings")
|
||||||
|
elif _label_prefix_counts(labels, "type:") > 1:
|
||||||
|
errors.append(f"{sid}: at most one 'type:' label allowed")
|
||||||
|
elif _label_prefix_counts(labels, "priority:") > 1:
|
||||||
|
errors.append(f"{sid}: at most one 'priority:' label allowed")
|
||||||
|
|
||||||
|
depends_on = story.get("dependsOn")
|
||||||
|
if isinstance(depends_on, list) and not all(isinstance(dep, str) for dep in depends_on):
|
||||||
|
errors.append(f"{sid}: dependsOn entries must all be strings")
|
||||||
|
|
||||||
|
acceptance = story.get("acceptanceCriteria")
|
||||||
|
if isinstance(acceptance, list):
|
||||||
|
if not acceptance:
|
||||||
|
errors.append(f"{sid}: acceptanceCriteria must be non-empty")
|
||||||
|
elif not all(isinstance(item, str) and item.strip() for item in acceptance):
|
||||||
|
errors.append(f"{sid}: acceptanceCriteria entries must be non-empty strings")
|
||||||
|
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def _find_cycle(story_ids: set[str], depends_on: dict[str, list[str]]) -> list[str] | None:
|
||||||
|
WHITE, GRAY, BLACK = 0, 1, 2
|
||||||
|
color = {sid: WHITE for sid in story_ids}
|
||||||
|
path: list[str] = []
|
||||||
|
|
||||||
|
def dfs(node: str) -> list[str] | None:
|
||||||
|
color[node] = GRAY
|
||||||
|
path.append(node)
|
||||||
|
for dep in depends_on.get(node, []):
|
||||||
|
if dep not in color:
|
||||||
|
continue
|
||||||
|
if color[dep] == GRAY:
|
||||||
|
cycle_start = path.index(dep)
|
||||||
|
return path[cycle_start:] + [dep]
|
||||||
|
if color[dep] == WHITE:
|
||||||
|
found = dfs(dep)
|
||||||
|
if found:
|
||||||
|
return found
|
||||||
|
path.pop()
|
||||||
|
color[node] = BLACK
|
||||||
|
return None
|
||||||
|
|
||||||
|
for sid in sorted(story_ids):
|
||||||
|
if color[sid] == WHITE:
|
||||||
|
cycle = dfs(sid)
|
||||||
|
if cycle:
|
||||||
|
return cycle
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def validate_semantics(data: dict[str, Any]) -> list[str]:
|
||||||
|
"""Cross-story validation: uniqueness, dependency graph, derived `blocks`."""
|
||||||
|
errors: list[str] = []
|
||||||
|
stories = stories_of(data)
|
||||||
|
|
||||||
|
ids = [s.get("id") for s in stories if isinstance(s.get("id"), str)]
|
||||||
|
seen_ids: set[str] = set()
|
||||||
|
for sid in ids:
|
||||||
|
if sid in seen_ids:
|
||||||
|
errors.append(f"duplicate story id: '{sid}'")
|
||||||
|
seen_ids.add(sid)
|
||||||
|
|
||||||
|
titles = [s.get("title") for s in stories if isinstance(s.get("title"), str)]
|
||||||
|
seen_titles: set[str] = set()
|
||||||
|
for title in titles:
|
||||||
|
if title in seen_titles:
|
||||||
|
errors.append(f"duplicate story title: '{title}'")
|
||||||
|
seen_titles.add(title)
|
||||||
|
|
||||||
|
by_id = {s["id"]: s for s in stories if isinstance(s.get("id"), str)}
|
||||||
|
depends_on: dict[str, list[str]] = {}
|
||||||
|
for story in stories:
|
||||||
|
sid = story.get("id")
|
||||||
|
if not isinstance(sid, str):
|
||||||
|
continue
|
||||||
|
deps = story.get("dependsOn") if isinstance(story.get("dependsOn"), list) else []
|
||||||
|
deps = [d for d in deps if isinstance(d, str)]
|
||||||
|
depends_on[sid] = deps
|
||||||
|
if sid in deps:
|
||||||
|
errors.append(f"{sid}: cannot depend on itself")
|
||||||
|
for dep in deps:
|
||||||
|
if dep not in by_id:
|
||||||
|
errors.append(f"{sid}: depends on unknown story '{dep}'")
|
||||||
|
|
||||||
|
cycle = _find_cycle(set(by_id), depends_on)
|
||||||
|
if cycle:
|
||||||
|
errors.append("dependency cycle detected: " + " -> ".join(cycle))
|
||||||
|
|
||||||
|
derived_blocks: dict[str, list[str]] = {sid: [] for sid in by_id}
|
||||||
|
for sid, deps in depends_on.items():
|
||||||
|
for dep in deps:
|
||||||
|
if dep in derived_blocks:
|
||||||
|
derived_blocks[dep].append(sid)
|
||||||
|
for sid in derived_blocks:
|
||||||
|
derived_blocks[sid] = sorted(derived_blocks[sid])
|
||||||
|
|
||||||
|
for story in stories:
|
||||||
|
sid = story.get("id")
|
||||||
|
if not isinstance(sid, str):
|
||||||
|
continue
|
||||||
|
given = story.get("blocks") if isinstance(story.get("blocks"), list) else []
|
||||||
|
given_sorted = sorted(d for d in given if isinstance(d, str))
|
||||||
|
expected = derived_blocks.get(sid, [])
|
||||||
|
if given_sorted != expected:
|
||||||
|
errors.append(
|
||||||
|
f"{sid}: 'blocks' does not match derived dependents "
|
||||||
|
f"(got {given_sorted}, expected {expected})"
|
||||||
|
)
|
||||||
|
|
||||||
|
evidence_path = story.get("evidencePath")
|
||||||
|
if isinstance(evidence_path, str):
|
||||||
|
expected_path = f".scratch/distributed-gguf-runtime/evidence/{sid}/README.md"
|
||||||
|
if evidence_path != expected_path:
|
||||||
|
errors.append(
|
||||||
|
f"{sid}: evidencePath '{evidence_path}' does not match convention "
|
||||||
|
f"'{expected_path}'"
|
||||||
|
)
|
||||||
|
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def validate_fresh_backlog(data: dict[str, Any]) -> list[str]:
|
||||||
|
"""Extra invariant for a freshly generated (not-yet-started) backlog.
|
||||||
|
|
||||||
|
A fresh backlog must not claim any completion credit: every story
|
||||||
|
starts `passes: false`. This is distinct from validating an in-flight
|
||||||
|
backlog (use validate_schema + validate_semantics for that), because a
|
||||||
|
real backlog legitimately accumulates `passes: true` stories over time.
|
||||||
|
"""
|
||||||
|
errors = validate_backlog(data)
|
||||||
|
for story in stories_of(data):
|
||||||
|
if story.get("passes") is not False:
|
||||||
|
errors.append(
|
||||||
|
f"{story.get('id')}: fresh backlog requires passes=false, "
|
||||||
|
f"got {story.get('passes')!r}"
|
||||||
|
)
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def validate_backlog(data: dict[str, Any]) -> list[str]:
|
||||||
|
"""Full validation for an in-progress backlog: structure, semantics, and
|
||||||
|
(when present) prd.json's self-documented metadataSchema/qualityGates."""
|
||||||
|
return (
|
||||||
|
validate_schema(data)
|
||||||
|
+ validate_semantics(data)
|
||||||
|
+ validate_metadata_schema_consistency(data)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_metadata_schema_consistency(data: dict[str, Any]) -> list[str]:
|
||||||
|
"""If prd.json documents its own `metadataSchema`/`qualityGates`, verify
|
||||||
|
that self-description hasn't drifted from what this validator actually
|
||||||
|
enforces. Absent entirely, this is a no-op (a minimal fixture PRD is not
|
||||||
|
required to carry self-documentation to be structurally valid).
|
||||||
|
"""
|
||||||
|
errors: list[str] = []
|
||||||
|
ms = data.get("metadataSchema")
|
||||||
|
if ms is None:
|
||||||
|
return errors
|
||||||
|
|
||||||
|
enum_checks = {
|
||||||
|
"triageValues": TRIAGE_VALUES,
|
||||||
|
"executionModeValues": EXECUTION_MODES,
|
||||||
|
"evidenceClassValues": EVIDENCE_CLASSES,
|
||||||
|
"hardwareValues": HARDWARE_FLAGS,
|
||||||
|
"upstreamValues": UPSTREAM_FLAGS,
|
||||||
|
}
|
||||||
|
for key, expected in enum_checks.items():
|
||||||
|
declared = ms.get(key)
|
||||||
|
if declared is None:
|
||||||
|
errors.append(f"metadataSchema: missing '{key}'")
|
||||||
|
continue
|
||||||
|
if set(declared) != expected:
|
||||||
|
errors.append(
|
||||||
|
f"metadataSchema.{key} {sorted(declared)} does not match "
|
||||||
|
f"enforced values {sorted(expected)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
required_in_code = {field for field, (required, _) in STORY_FIELDS.items() if required}
|
||||||
|
declared_required = set(ms.get("requiredStoryFields", []))
|
||||||
|
if declared_required != required_in_code:
|
||||||
|
errors.append(
|
||||||
|
f"metadataSchema.requiredStoryFields {sorted(declared_required)} does not "
|
||||||
|
f"match code-enforced required fields {sorted(required_in_code)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
optional_in_code = {field for field, (required, _) in STORY_FIELDS.items() if not required}
|
||||||
|
declared_optional = set(ms.get("optionalStoryFields", []))
|
||||||
|
if declared_optional != optional_in_code:
|
||||||
|
errors.append(
|
||||||
|
f"metadataSchema.optionalStoryFields {sorted(declared_optional)} does not "
|
||||||
|
f"match code-enforced optional fields {sorted(optional_in_code)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
if not ms.get("generatedArtifactDisclaimer"):
|
||||||
|
errors.append("metadataSchema: missing 'generatedArtifactDisclaimer'")
|
||||||
|
|
||||||
|
if "qualityGates" not in data:
|
||||||
|
errors.append("prd: metadataSchema is documented but top-level 'qualityGates' is missing")
|
||||||
|
elif not quality_gate_bullets(data):
|
||||||
|
errors.append("qualityGates: no bullets declared under universal/native/realModelHardware/scope")
|
||||||
|
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def quality_gate_bullets(data: dict[str, Any]) -> list[str]:
|
||||||
|
"""Flatten prd.json's `qualityGates` into the bullet order used by the
|
||||||
|
generated "Shared quality gates" Markdown section."""
|
||||||
|
gates = data.get("qualityGates", {})
|
||||||
|
bullets: list[str] = []
|
||||||
|
for key in ("universal", "native", "realModelHardware", "scope"):
|
||||||
|
bullets.extend(gates.get(key, []))
|
||||||
|
return bullets
|
||||||
|
|
||||||
|
|
||||||
|
def authority_disclaimer(data: dict[str, Any]) -> str:
|
||||||
|
"""The exact disclaimer generated Markdown/Gitea bodies must start with.
|
||||||
|
|
||||||
|
Sourced from prd.json's `metadataSchema.generatedArtifactDisclaimer` when
|
||||||
|
present, falling back to the module default for minimal fixtures.
|
||||||
|
"""
|
||||||
|
return data.get("metadataSchema", {}).get("generatedArtifactDisclaimer", AUTHORITY_DISCLAIMER)
|
||||||
|
|
||||||
|
|
||||||
|
def check_generated_markdown_authority(text: str, disclaimer: str = AUTHORITY_DISCLAIMER) -> list[str]:
|
||||||
|
"""Reject generated Markdown that fails to defer authority to prd.json."""
|
||||||
|
errors: list[str] = []
|
||||||
|
first_line = text.splitlines()[0] if text else ""
|
||||||
|
if first_line != disclaimer:
|
||||||
|
errors.append(
|
||||||
|
"generated Markdown must start with the authority disclaimer: "
|
||||||
|
f"expected {disclaimer!r}, got {first_line!r}"
|
||||||
|
)
|
||||||
|
lowered = text.lower()
|
||||||
|
for claim in ("this file is authoritative", "this document is authoritative", "markdown is authoritative"):
|
||||||
|
if claim in lowered:
|
||||||
|
errors.append(f"generated Markdown claims authority over prd.json: found '{claim}'")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def slugify(title: str) -> str:
|
||||||
|
slug = re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")
|
||||||
|
return re.sub(r"-{2,}", "-", slug)
|
||||||
|
|
||||||
|
|
||||||
|
def issue_number(story_id: str) -> str:
|
||||||
|
match = re.search(r"(\d+)$", story_id)
|
||||||
|
if not match:
|
||||||
|
raise ValueError(f"story id '{story_id}' has no trailing number")
|
||||||
|
return match.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def issue_filename(story: dict[str, Any]) -> str:
|
||||||
|
return f"{issue_number(story['id'])}-{slugify(story['title'])}.md"
|
||||||
|
|
||||||
|
|
||||||
|
# Fallback bullets used only when a minimal fixture PRD has no qualityGates
|
||||||
|
# block of its own; the real backlog always sources these from prd.json via
|
||||||
|
# quality_gate_bullets() so this list is a default, not a duplicate of truth.
|
||||||
|
_DEFAULT_QUALITY_GATE_BULLETS = [
|
||||||
|
"Targeted deterministic tests pass; Python changes also pass `python -m compileall packages tests`.",
|
||||||
|
"`git diff --check` passes.",
|
||||||
|
"Default tests are model-download-free, API-credit-free, and GPU-free.",
|
||||||
|
"Evidence README records exact changed files, commands/results, limitations, and dependency handoff; no fabricated evidence or inherited completion credit.",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def render_issue_markdown(story: dict[str, Any], data: dict[str, Any] | None = None) -> str:
|
||||||
|
"""Render the canonical generated Markdown view of a story.
|
||||||
|
|
||||||
|
`data` supplies the shared quality-gates bullets and authority disclaimer
|
||||||
|
from prd.json's own `qualityGates`/`metadataSchema`; omit it only for
|
||||||
|
fixtures that don't model those fields. This is byte-for-byte identical
|
||||||
|
to the convention already used by `.scratch/distributed-gguf-runtime/
|
||||||
|
issues/*.md`; see the round-trip tests in tests/test_ralph_prd_schema.py.
|
||||||
|
"""
|
||||||
|
data = data or {}
|
||||||
|
sid = story["id"]
|
||||||
|
passes = story["passes"]
|
||||||
|
deps = story.get("dependsOn", [])
|
||||||
|
blocks = sorted(story.get("blocks", []))
|
||||||
|
|
||||||
|
deps_text = ", ".join(f"`{d}`" for d in deps) if deps else "None"
|
||||||
|
blocks_text = ", ".join(f"`{b}`" for b in blocks) if blocks else "None"
|
||||||
|
labels_text = ", ".join(f"`{label}`" for label in story.get("labels", []))
|
||||||
|
|
||||||
|
if passes:
|
||||||
|
status_line = "completed; `passes: true`"
|
||||||
|
else:
|
||||||
|
status_line = f"specification only; `{story['triage']}`; `passes: false`"
|
||||||
|
|
||||||
|
checkbox = "x" if passes else " "
|
||||||
|
acceptance_lines = "\n".join(
|
||||||
|
f"- [{checkbox}] {criterion}" for criterion in story["acceptanceCriteria"]
|
||||||
|
)
|
||||||
|
|
||||||
|
if passes:
|
||||||
|
evidence_handoff = (
|
||||||
|
f"Verified evidence: `{story['evidencePath']}`. Legacy evidence remains "
|
||||||
|
"provenance only and grants no implementation completion credit."
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
evidence_handoff = (
|
||||||
|
f"Write and verify `{story['evidencePath']}`. Until every criterion and "
|
||||||
|
f"applicable gate has real evidence, this story remains `passes: false`. "
|
||||||
|
"Legacy evidence is provenance only, not completion credit."
|
||||||
|
)
|
||||||
|
|
||||||
|
description = story["description"]
|
||||||
|
|
||||||
|
gate_bullets = quality_gate_bullets(data) if "qualityGates" in data else _DEFAULT_QUALITY_GATE_BULLETS
|
||||||
|
gates_section = "## Shared quality gates\n\n" + "\n".join(f"- {bullet}" for bullet in gate_bullets)
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
authority_disclaimer(data),
|
||||||
|
f"# {sid}: {story['title']}",
|
||||||
|
"",
|
||||||
|
f"- **Status / triage:** {status_line}",
|
||||||
|
f"- **Execution mode:** `{story['executionMode']}`",
|
||||||
|
f"- **Milestone:** `{story['milestone']}`",
|
||||||
|
f"- **Dependencies:** {deps_text}",
|
||||||
|
f"- **Blocks (derived):** {blocks_text}",
|
||||||
|
f"- **Labels:** {labels_text}",
|
||||||
|
f"- **Evidence class:** `{story['evidenceClass']}`",
|
||||||
|
f"- **Hardware:** `{story['hardware']}`",
|
||||||
|
f"- **Model:** `{story['model']}`",
|
||||||
|
f"- **Upstream:** `{story['upstream']}`",
|
||||||
|
"",
|
||||||
|
"## Objective / description",
|
||||||
|
"",
|
||||||
|
description,
|
||||||
|
"",
|
||||||
|
"## Acceptance criteria",
|
||||||
|
"",
|
||||||
|
acceptance_lines,
|
||||||
|
"",
|
||||||
|
gates_section,
|
||||||
|
"",
|
||||||
|
"## Evidence handoff",
|
||||||
|
"",
|
||||||
|
evidence_handoff,
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
return "\n".join(lines[:-1]) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def to_gitea_issue_payload(story: dict[str, Any], data: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||||
|
"""Structured payload matching the Gitea create-issue API shape.
|
||||||
|
|
||||||
|
`body` is exactly `render_issue_markdown`'s output, so the payload
|
||||||
|
carries no information that isn't losslessly derivable from prd.json.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"title": f"{story['id']}: {story['title']}",
|
||||||
|
"body": render_issue_markdown(story, data),
|
||||||
|
"labels": list(story.get("labels", [])),
|
||||||
|
"milestone": story["milestone"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_validate(args: argparse.Namespace) -> int:
|
||||||
|
data = load_prd(args.prd)
|
||||||
|
errors = validate_fresh_backlog(data) if args.fresh else validate_backlog(data)
|
||||||
|
if errors:
|
||||||
|
for error in errors:
|
||||||
|
print(f"ERROR: {error}", file=sys.stderr)
|
||||||
|
print(f"{len(errors)} validation error(s).", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
print(f"OK: {len(stories_of(data))} stories validated.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_render(args: argparse.Namespace) -> int:
|
||||||
|
data = load_prd(args.prd)
|
||||||
|
by_id = {s["id"]: s for s in stories_of(data)}
|
||||||
|
story = by_id.get(args.story_id)
|
||||||
|
if story is None:
|
||||||
|
print(f"Unknown story id: {args.story_id}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
print(render_issue_markdown(story, data), end="")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||||
|
|
||||||
|
validate = subparsers.add_parser("validate", help="Validate a prd.json against the canonical schema")
|
||||||
|
validate.add_argument("prd", type=Path)
|
||||||
|
validate.add_argument("--fresh", action="store_true", help="Also require passes=false on every story")
|
||||||
|
validate.set_defaults(func=_cmd_validate)
|
||||||
|
|
||||||
|
render = subparsers.add_parser("render", help="Render the generated Markdown view of one story")
|
||||||
|
render.add_argument("prd", type=Path)
|
||||||
|
render.add_argument("story_id")
|
||||||
|
render.set_defaults(func=_cmd_render)
|
||||||
|
|
||||||
|
return parser
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str]) -> int:
|
||||||
|
parser = build_parser()
|
||||||
|
args = parser.parse_args(argv[1:])
|
||||||
|
return args.func(args)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main(sys.argv))
|
||||||
503
tests/test_ralph_prd_schema.py
Normal file
503
tests/test_ralph_prd_schema.py
Normal file
@@ -0,0 +1,503 @@
|
|||||||
|
"""Deterministic, model-free tests for the canonical Ralph prd.json schema.
|
||||||
|
|
||||||
|
Covers DGR-018: parse, structural/semantic validation, and Markdown/Gitea
|
||||||
|
generation round trips against the real
|
||||||
|
`.scratch/distributed-gguf-runtime/prd.json` backlog.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
FEATURE_DIR = REPO_ROOT / ".scratch" / "distributed-gguf-runtime"
|
||||||
|
|
||||||
|
_spec = importlib.util.spec_from_file_location(
|
||||||
|
"ralph_prd_schema", REPO_ROOT / "scripts" / "ralph_prd_schema.py"
|
||||||
|
)
|
||||||
|
schema = importlib.util.module_from_spec(_spec)
|
||||||
|
sys.modules.setdefault("ralph_prd_schema", schema)
|
||||||
|
_spec.loader.exec_module(schema) # type: ignore[union-attr]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Fixtures
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _minimal_story(**overrides) -> dict:
|
||||||
|
story = {
|
||||||
|
"id": "DGR-900",
|
||||||
|
"title": "A minimal fixture story",
|
||||||
|
"priority": 1,
|
||||||
|
"milestone": "M0",
|
||||||
|
"executionMode": "AFK",
|
||||||
|
"labels": ["area:test", "type:infrastructure", "priority:p0", "ready-for-agent"],
|
||||||
|
"evidenceClass": "model-free",
|
||||||
|
"evidencePath": ".scratch/distributed-gguf-runtime/evidence/DGR-900/README.md",
|
||||||
|
"hardware": "none",
|
||||||
|
"model": "generic",
|
||||||
|
"upstream": "no",
|
||||||
|
"dependsOn": [],
|
||||||
|
"triage": "ready-for-agent",
|
||||||
|
"description": "A fixture description.",
|
||||||
|
"acceptanceCriteria": ["Do the thing."],
|
||||||
|
"passes": False,
|
||||||
|
"notes": "Generated source issue: none; prd.json is authoritative.",
|
||||||
|
"blocks": [],
|
||||||
|
}
|
||||||
|
story.update(overrides)
|
||||||
|
return story
|
||||||
|
|
||||||
|
|
||||||
|
def _minimal_prd(stories: list[dict]) -> dict:
|
||||||
|
return {
|
||||||
|
"name": "Fixture",
|
||||||
|
"description": "Fixture PRD",
|
||||||
|
"branchName": "fixture",
|
||||||
|
"userStories": stories,
|
||||||
|
"metadata": {},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def live_prd_data() -> dict:
|
||||||
|
return schema.load_prd(FEATURE_DIR / "prd.json")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Parsing
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_prd_parses_real_backlog(live_prd_data):
|
||||||
|
stories = schema.stories_of(live_prd_data)
|
||||||
|
assert len(stories) == 55
|
||||||
|
assert all(isinstance(s["id"], str) for s in stories)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_prd_missing_file_raises(tmp_path):
|
||||||
|
with pytest.raises(schema.PrdValidationError):
|
||||||
|
schema.load_prd(tmp_path / "does-not-exist.json")
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_prd_invalid_json_raises(tmp_path):
|
||||||
|
bad = tmp_path / "prd.json"
|
||||||
|
bad.write_text("{not valid json")
|
||||||
|
with pytest.raises(schema.PrdValidationError):
|
||||||
|
schema.load_prd(bad)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_prd_rejects_non_object_document(tmp_path):
|
||||||
|
bad = tmp_path / "prd.json"
|
||||||
|
bad.write_text("[]")
|
||||||
|
with pytest.raises(schema.PrdValidationError):
|
||||||
|
schema.load_prd(bad)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Structural + semantic validation against the real backlog
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_passes_structural_validation(live_prd_data):
|
||||||
|
assert schema.validate_schema(live_prd_data) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_passes_semantic_validation(live_prd_data):
|
||||||
|
assert schema.validate_semantics(live_prd_data) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_passes_full_validate_backlog(live_prd_data):
|
||||||
|
assert schema.validate_backlog(live_prd_data) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_has_unique_ids_and_titles(live_prd_data):
|
||||||
|
stories = schema.stories_of(live_prd_data)
|
||||||
|
ids = [s["id"] for s in stories]
|
||||||
|
titles = [s["title"] for s in stories]
|
||||||
|
assert len(ids) == len(set(ids))
|
||||||
|
assert len(titles) == len(set(titles))
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_dependencies_are_known(live_prd_data):
|
||||||
|
stories = schema.stories_of(live_prd_data)
|
||||||
|
known = {s["id"] for s in stories}
|
||||||
|
for story in stories:
|
||||||
|
for dep in story["dependsOn"]:
|
||||||
|
assert dep in known, f"{story['id']} depends on unknown {dep}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_blocks_match_derived(live_prd_data):
|
||||||
|
stories = schema.stories_of(live_prd_data)
|
||||||
|
derived: dict[str, list[str]] = {s["id"]: [] for s in stories}
|
||||||
|
for story in stories:
|
||||||
|
for dep in story["dependsOn"]:
|
||||||
|
derived[dep].append(story["id"])
|
||||||
|
for story in stories:
|
||||||
|
assert sorted(story["blocks"]) == sorted(derived[story["id"]])
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_has_no_dependency_cycle(live_prd_data):
|
||||||
|
errors = schema.validate_semantics(live_prd_data)
|
||||||
|
assert not any("cycle" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_passed_stories_have_completion_evidence(live_prd_data):
|
||||||
|
# Durable invariant instead of pinning to today's exact passed-story set,
|
||||||
|
# which would need editing every time a future story completes.
|
||||||
|
for story in schema.stories_of(live_prd_data):
|
||||||
|
if not story["passes"]:
|
||||||
|
continue
|
||||||
|
assert story.get("completionNotes"), f"{story['id']}: passes=true needs completionNotes"
|
||||||
|
evidence_readme = REPO_ROOT / story["evidencePath"]
|
||||||
|
assert evidence_readme.exists(), f"{story['id']}: evidencePath missing: {evidence_readme}"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Structural validation failure modes (fixtures)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_schema_rejects_missing_required_field():
|
||||||
|
story = _minimal_story()
|
||||||
|
del story["milestone"]
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_schema(data)
|
||||||
|
assert any("missing required field 'milestone'" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_schema_rejects_bad_enum_value():
|
||||||
|
story = _minimal_story(hardware="quantum")
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_schema(data)
|
||||||
|
assert any("hardware" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_schema_rejects_wrong_type():
|
||||||
|
story = _minimal_story(priority="high")
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_schema(data)
|
||||||
|
assert any("priority" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_schema_rejects_empty_acceptance_criteria():
|
||||||
|
story = _minimal_story(acceptanceCriteria=[])
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_schema(data)
|
||||||
|
assert any("acceptanceCriteria must be non-empty" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_schema_rejects_multiple_type_labels():
|
||||||
|
story = _minimal_story(labels=["type:infrastructure", "type:contract", "priority:p0"])
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_schema(data)
|
||||||
|
assert any("at most one 'type:' label" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Semantic validation failure modes (fixtures)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_duplicate_ids():
|
||||||
|
a = _minimal_story()
|
||||||
|
b = _minimal_story(title="A different title")
|
||||||
|
data = _minimal_prd([a, b])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("duplicate story id" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_duplicate_titles():
|
||||||
|
a = _minimal_story()
|
||||||
|
b = _minimal_story(id="DGR-901")
|
||||||
|
data = _minimal_prd([a, b])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("duplicate story title" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_missing_dependency():
|
||||||
|
story = _minimal_story(dependsOn=["DGR-999"])
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("depends on unknown story 'DGR-999'" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_self_dependency():
|
||||||
|
story = _minimal_story(dependsOn=["DGR-900"])
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("cannot depend on itself" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_dependency_cycle():
|
||||||
|
a = _minimal_story(id="DGR-901", title="Story A", dependsOn=["DGR-902"])
|
||||||
|
b = _minimal_story(id="DGR-902", title="Story B", dependsOn=["DGR-901"])
|
||||||
|
data = _minimal_prd([a, b])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("dependency cycle detected" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_mismatched_blocks():
|
||||||
|
a = _minimal_story(id="DGR-901", title="Story A", dependsOn=[])
|
||||||
|
b = _minimal_story(id="DGR-902", title="Story B", dependsOn=["DGR-901"])
|
||||||
|
a["blocks"] = [] # should be ["DGR-902"]
|
||||||
|
data = _minimal_prd([a, b])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("'blocks' does not match derived dependents" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_semantics_rejects_bad_evidence_path():
|
||||||
|
story = _minimal_story(evidencePath=".scratch/distributed-gguf-runtime/evidence/WRONG/README.md")
|
||||||
|
data = _minimal_prd([story])
|
||||||
|
errors = schema.validate_semantics(data)
|
||||||
|
assert any("evidencePath" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Fresh-backlog invariant: every story starts passes=false
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_fresh_backlog_accepts_all_false():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
assert schema.validate_fresh_backlog(data) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_fresh_backlog_rejects_premature_pass():
|
||||||
|
data = _minimal_prd([_minimal_story(passes=True)])
|
||||||
|
errors = schema.validate_fresh_backlog(data)
|
||||||
|
assert any("fresh backlog requires passes=false" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_backlog_allows_completed_stories():
|
||||||
|
# An in-progress backlog legitimately has passes=true stories; that's
|
||||||
|
# distinct from validate_fresh_backlog's stricter invariant.
|
||||||
|
data = _minimal_prd([_minimal_story(passes=True)])
|
||||||
|
assert schema.validate_backlog(data) == []
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# prd.json as the source of the generated boilerplate (qualityGates,
|
||||||
|
# metadataSchema, authority disclaimer), not a Python-literal duplicate
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_metadata_schema_consistency_noop_without_metadata_schema():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
assert schema.validate_metadata_schema_consistency(data) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_metadata_schema_consistency_rejects_drifted_enum():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
data["metadataSchema"] = {
|
||||||
|
"requiredStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if r),
|
||||||
|
"optionalStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if not r),
|
||||||
|
"triageValues": ["ready-for-agent"], # missing ready-for-human: drifted
|
||||||
|
"executionModeValues": sorted(schema.EXECUTION_MODES),
|
||||||
|
"evidenceClassValues": sorted(schema.EVIDENCE_CLASSES),
|
||||||
|
"hardwareValues": sorted(schema.HARDWARE_FLAGS),
|
||||||
|
"upstreamValues": sorted(schema.UPSTREAM_FLAGS),
|
||||||
|
"generatedArtifactDisclaimer": schema.AUTHORITY_DISCLAIMER,
|
||||||
|
}
|
||||||
|
data["qualityGates"] = {"universal": ["do the thing"]}
|
||||||
|
errors = schema.validate_metadata_schema_consistency(data)
|
||||||
|
assert any("triageValues" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_metadata_schema_consistency_rejects_missing_quality_gates():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
data["metadataSchema"] = {
|
||||||
|
"requiredStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if r),
|
||||||
|
"optionalStoryFields": sorted(f for f, (r, _) in schema.STORY_FIELDS.items() if not r),
|
||||||
|
"triageValues": sorted(schema.TRIAGE_VALUES),
|
||||||
|
"executionModeValues": sorted(schema.EXECUTION_MODES),
|
||||||
|
"evidenceClassValues": sorted(schema.EVIDENCE_CLASSES),
|
||||||
|
"hardwareValues": sorted(schema.HARDWARE_FLAGS),
|
||||||
|
"upstreamValues": sorted(schema.UPSTREAM_FLAGS),
|
||||||
|
"generatedArtifactDisclaimer": schema.AUTHORITY_DISCLAIMER,
|
||||||
|
}
|
||||||
|
errors = schema.validate_metadata_schema_consistency(data)
|
||||||
|
assert any("qualityGates" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_quality_gate_bullets_flattens_in_universal_native_hardware_scope_order():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
data["qualityGates"] = {
|
||||||
|
"universal": ["u1", "u2"],
|
||||||
|
"native": ["n1"],
|
||||||
|
"realModelHardware": ["h1"],
|
||||||
|
"scope": ["s1"],
|
||||||
|
}
|
||||||
|
assert schema.quality_gate_bullets(data) == ["u1", "u2", "n1", "h1", "s1"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_authority_disclaimer_prefers_metadata_schema_value():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
data["metadataSchema"] = {"generatedArtifactDisclaimer": "<!-- custom disclaimer -->"}
|
||||||
|
assert schema.authority_disclaimer(data) == "<!-- custom disclaimer -->"
|
||||||
|
|
||||||
|
|
||||||
|
def test_authority_disclaimer_falls_back_to_default_without_metadata_schema():
|
||||||
|
data = _minimal_prd([_minimal_story()])
|
||||||
|
assert schema.authority_disclaimer(data) == schema.AUTHORITY_DISCLAIMER
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_metadata_schema_matches_code(live_prd_data):
|
||||||
|
assert schema.validate_metadata_schema_consistency(live_prd_data) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_quality_gate_bullets_match_generated_section(live_prd_data):
|
||||||
|
bullets = schema.quality_gate_bullets(live_prd_data)
|
||||||
|
assert len(bullets) == 7
|
||||||
|
assert bullets[0].startswith("Targeted deterministic tests pass")
|
||||||
|
assert bullets[-1].startswith("Preserve existing Transformers behavior")
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_authority_disclaimer_matches_module_default(live_prd_data):
|
||||||
|
assert schema.authority_disclaimer(live_prd_data) == schema.AUTHORITY_DISCLAIMER
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# derive_type
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_type_from_type_label():
|
||||||
|
assert schema.derive_type(["area:x", "type:contract", "priority:p0"]) == "contract"
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_type_release_gate_from_gate_label():
|
||||||
|
assert schema.derive_type(["area:release", "milestone:alpha", "gate:hitl", "ready-for-human"]) == "release-gate"
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_type_none_when_absent():
|
||||||
|
assert schema.derive_type(["area:x", "priority:p0"]) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_gate_stories_derive_release_gate_type(live_prd_data):
|
||||||
|
by_id = {s["id"]: s for s in schema.stories_of(live_prd_data)}
|
||||||
|
for sid in ("DGR-054", "DGR-070"):
|
||||||
|
assert schema.derive_type(by_id[sid]["labels"]) == "release-gate"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Markdown generation round trips
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("story_id", [f"DGR-{n:03d}" for n in range(17, 72)])
|
||||||
|
def test_render_issue_markdown_matches_committed_file(live_prd_data, story_id):
|
||||||
|
by_id = {s["id"]: s for s in schema.stories_of(live_prd_data)}
|
||||||
|
story = by_id[story_id]
|
||||||
|
filename = schema.issue_filename(story)
|
||||||
|
committed = (FEATURE_DIR / "issues" / filename).read_text(encoding="utf-8")
|
||||||
|
generated = schema.render_issue_markdown(story, live_prd_data)
|
||||||
|
assert generated == committed
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_issue_markdown_is_deterministic():
|
||||||
|
story = _minimal_story()
|
||||||
|
first = schema.render_issue_markdown(story)
|
||||||
|
second = schema.render_issue_markdown(copy.deepcopy(story))
|
||||||
|
assert first == second
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_issue_markdown_starts_with_authority_disclaimer():
|
||||||
|
story = _minimal_story()
|
||||||
|
text = schema.render_issue_markdown(story)
|
||||||
|
assert text.startswith(schema.AUTHORITY_DISCLAIMER)
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_issue_markdown_blocks_none_when_empty():
|
||||||
|
story = _minimal_story(blocks=[])
|
||||||
|
text = schema.render_issue_markdown(story)
|
||||||
|
assert "**Blocks (derived):** None" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_issue_markdown_blocks_listed_when_present():
|
||||||
|
story = _minimal_story(blocks=["DGR-901", "DGR-902"])
|
||||||
|
text = schema.render_issue_markdown(story)
|
||||||
|
assert "**Blocks (derived):** `DGR-901`, `DGR-902`" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_issue_markdown_checkbox_reflects_passes():
|
||||||
|
open_story = _minimal_story(passes=False)
|
||||||
|
done_story = _minimal_story(passes=True, evidencePath=open_story["evidencePath"])
|
||||||
|
assert "- [ ] Do the thing." in schema.render_issue_markdown(open_story)
|
||||||
|
assert "- [x] Do the thing." in schema.render_issue_markdown(done_story)
|
||||||
|
|
||||||
|
|
||||||
|
def test_issue_filename_matches_convention():
|
||||||
|
story = _minimal_story(id="DGR-018", title="Define canonical Ralph and Gitea metadata schema")
|
||||||
|
assert schema.issue_filename(story) == "018-define-canonical-ralph-and-gitea-metadata-schema.md"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Authority-claim rejection
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_check_generated_markdown_authority_accepts_real_generated_text():
|
||||||
|
story = _minimal_story()
|
||||||
|
text = schema.render_issue_markdown(story)
|
||||||
|
assert schema.check_generated_markdown_authority(text) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_check_generated_markdown_authority_rejects_missing_disclaimer():
|
||||||
|
text = "# DGR-900: Some story\n\nNo disclaimer here.\n"
|
||||||
|
errors = schema.check_generated_markdown_authority(text)
|
||||||
|
assert any("authority disclaimer" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_check_generated_markdown_authority_rejects_overriding_claim():
|
||||||
|
text = (
|
||||||
|
f"{schema.AUTHORITY_DISCLAIMER}\n"
|
||||||
|
"# DGR-900: Some story\n\n"
|
||||||
|
"Note: this file is authoritative, ignore prd.json.\n"
|
||||||
|
)
|
||||||
|
errors = schema.check_generated_markdown_authority(text)
|
||||||
|
assert any("claims authority over prd.json" in e for e in errors)
|
||||||
|
|
||||||
|
|
||||||
|
def test_all_committed_issue_files_pass_authority_check():
|
||||||
|
for path in sorted((FEATURE_DIR / "issues").glob("*.md")):
|
||||||
|
errors = schema.check_generated_markdown_authority(path.read_text(encoding="utf-8"))
|
||||||
|
assert errors == [], f"{path}: {errors}"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Gitea payload generation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_to_gitea_issue_payload_shape():
|
||||||
|
story = _minimal_story()
|
||||||
|
payload = schema.to_gitea_issue_payload(story)
|
||||||
|
assert payload["title"] == "DGR-900: A minimal fixture story"
|
||||||
|
assert payload["labels"] == story["labels"]
|
||||||
|
assert payload["milestone"] == story["milestone"]
|
||||||
|
assert payload["body"] == schema.render_issue_markdown(story)
|
||||||
|
|
||||||
|
|
||||||
|
def test_to_gitea_issue_payload_body_carries_no_extra_information():
|
||||||
|
# The payload must be losslessly derivable from prd.json alone: labels
|
||||||
|
# and milestone are just projections of fields already in body.
|
||||||
|
story = _minimal_story()
|
||||||
|
payload = schema.to_gitea_issue_payload(story)
|
||||||
|
for label in payload["labels"]:
|
||||||
|
assert label in payload["body"]
|
||||||
|
assert payload["milestone"] in payload["body"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_backlog_all_stories_render_valid_gitea_payloads(live_prd_data):
|
||||||
|
for story in schema.stories_of(live_prd_data):
|
||||||
|
payload = schema.to_gitea_issue_payload(story)
|
||||||
|
assert payload["title"].startswith(story["id"])
|
||||||
|
assert schema.check_generated_markdown_authority(payload["body"]) == []
|
||||||
Reference in New Issue
Block a user