neuron-tai/.scratch/distributed-gguf-runtime/evidence/DGR-018

DGR-018 evidence — canonical Ralph and Gitea metadata schema

Completed: 2026-07-16 Branch: ralph/distributed-gguf-runtime Authority: .scratch/distributed-gguf-runtime/prd.json Dependency: DGR-017 (evidence/DGR-017/README.md) — cleaned backlog reconciled to origin/master; no old pass state transferred.

Objective

Make prd.json the validated source from which Markdown (and, later, Gitea) issues can be generated losslessly, per .scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md.

Pre-existing state found (not caused by this story)

Before any change in this session, git status showed .scratch/distributed-gguf-runtime/prd.json already modified in the working tree relative to HEAD (commit 369b207), with no corresponding progress-log entry. Diffing against HEAD showed the working copy had dropped prd.json's top-level sourceOfTruth, qualityGates, metadataSchema, milestones, and supersededStories objects, while userStories itself was byte-identical to HEAD. This looked like an abandoned, uncommitted partial edit from a prior session, not intentional current work — those fields are exactly the schema/quality-gate/audit-provenance content this story depends on, and their loss wasn't explained by any acceptance criterion. They were restored (see "Changes" below) rather than silently accepted or discarded, per the instruction to investigate unexplained working-tree state before building on top of it.

Changes

scripts/ralph_prd_schema.py (new)

Single module providing:

• Parse: load_prd(path) — JSON load with clear PrdValidationErrors for missing file / invalid JSON / non-object document.
• Canonical schema registry: STORY_FIELDS (name → required/type), EXECUTION_MODES, EVIDENCE_CLASSES, HARDWARE_FLAGS, UPSTREAM_FLAGS, TRIAGE_VALUES. Covers every field named in the acceptance criteria: stable id/title, labels, milestone, derived type (derive_type), dependsOn, derived blocks, triage, evidenceClass, and hardware/model/upstream flags.
• Structural validation: validate_schema(data) — required fields, types, enum membership, ID convention, type:/priority: label cardinality, non-empty acceptanceCriteria.
• Semantic validation: validate_semantics(data) — unique IDs, unique titles, dependsOn resolves to known stories (no self-dependency), dependency graph is acyclic (with a reported cycle path on failure), blocks matches the dependency graph exactly (sorted set equality, not superset), and evidencePath matches the per-story convention.
• Fresh vs. in-progress backlog: validate_fresh_backlog(data) additionally requires every story to start passes: false (for a backlog that hasn't started execution yet); validate_backlog(data) is the composed check for a real, in-flight backlog where some stories have legitimately completed.
• Self-consistency check: validate_metadata_schema_consistency(data) — when prd.json declares its own metadataSchema/qualityGates (as this one now does), verifies that self-documentation hasn't drifted from what the validator actually enforces (enum sets, required/optional field lists, presence of qualityGates and generatedArtifactDisclaimer). This is a no-op for minimal fixture PRDs that don't carry that documentation.
• Generation (one-directional, prd.json → artifact): render_issue_markdown(story, data) renders the exact Markdown convention already used by .scratch/distributed-gguf-runtime/issues/*.md, sourcing the "Shared quality gates" bullets from data["qualityGates"] and the leading disclaimer from data["metadataSchema"]["generatedArtifactDisclaimer"] (falling back to a module default only when data omits them) — not from a duplicated Python string literal. to_gitea_issue_payload(story, data) wraps the same body into a Gitea create-issue-shaped payload (title, body, labels, milestone).
• Authority guard: check_generated_markdown_authority(text, disclaimer=...) rejects generated Markdown that's missing the disclaimer or that contains a conflicting authority claim (e.g. "this file is authoritative"). There is deliberately no Markdown → prd.json parser, so a generated artifact structurally cannot feed passes (or anything else) back into the authoritative source.
• CLI: python scripts/ralph_prd_schema.py validate <prd.json> [--fresh] and ... render <prd.json> <STORY-ID>.

.scratch/distributed-gguf-runtime/prd.json

• Restored the top-level sourceOfTruth, qualityGates, milestones, and supersededStories objects to their HEAD content (see "Pre-existing state" above); userStories was already identical to HEAD and is unchanged in content.
• Extended metadataSchema (previously incomplete for this story's own acceptance criteria) with: requiredStoryFields now also lists notes and blocks (present on all 55 stories); new optionalStoryFields: ["completionNotes"]; new hardwareValues/upstreamValues enums (model is documented as an open convention, not a closed enum, since quantization/model targets are dynamic recipe inputs per RALPH-CONTEXT.md); new typeDerivation and labelConventions (reserved prefixes, cardinality); new generatedArtifactDisclaimer (the exact string generated artifacts must start with); extended dependencyRules/authorityRule prose to match what the validator enforces.
• Reworded sourceOfTruth's stale "All stories are unimplemented ... passes=false" clause, which was no longer accurate once DGR-017 completed.
• Marked DGR-018.passes = true with completionNotes recording this story's outcome.

.scratch/distributed-gguf-runtime/issues/018-define-canonical-ralph-and-gitea-metadata-schema.md

Regenerated via render_issue_markdown to reflect passes: true (checked acceptance criteria, "completed" status line, "Verified evidence" handoff line) — matching the same convention DGR-017's issue file already used for a completed story.

tests/test_ralph_prd_schema.py (new)

108 deterministic, model-download-free, GPU-free tests:

• Parse (4 tests): real backlog parses to 55 stories; missing file, invalid JSON, and non-object documents raise PrdValidationError.
• Structural/semantic validation against the real backlog (7 tests): passes validate_schema, validate_semantics, and the composed validate_backlog; unique IDs/titles; all dependsOn resolve; blocks matches the derived dependency graph for all 55 stories; no cycle; every passes: true story carries completionNotes and an existing evidence README (a durable invariant, not a hardcoded list of which stories have completed — that list will keep growing).
• Structural/semantic failure-mode fixtures (13 tests): missing required field, bad enum, wrong type, empty acceptanceCriteria, multiple type: labels, duplicate ID, duplicate title, unknown dependency, self-dependency, dependency cycle, mismatched blocks, bad evidencePath.
• Fresh-backlog invariant (3 tests): accepts all-false, rejects a premature passes: true, and confirms validate_backlog (the in-progress variant) permits completed stories.
• prd.json-as-source-of-truth for boilerplate (9 tests): qualityGates/metadataSchema self-consistency checks (no-op without them, catches a drifted enum, catches a missing qualityGates), quality_gate_bullets flattening order, authority_disclaimer precedence and fallback, and 3 tests asserting the real backlog's declared schema matches the code, its 7 quality-gate bullets are intact, and its disclaimer matches the module default.
• derive_type (4 tests): label-derived type, release-gate synthetic type for HITL gate stories, None when absent, and confirmation that the real backlog's two release-gate stories (DGR-054, DGR-070) derive release-gate.
• Markdown generation round trips (55 parametrized + 6 tests): render_issue_markdown for every story DGR-017..DGR-071 is byte-for-byte identical to the corresponding file already in .scratch/distributed-gguf-runtime/issues/; determinism; leading disclaimer; Blocks (derived) rendering (None vs. listed); checkbox reflects passes; filename convention.
• Authority-claim rejection (4 tests): accepts real generated text, rejects a missing disclaimer, rejects an overriding claim, and confirms every committed issue file in .scratch/distributed-gguf-runtime/issues/ passes the check.
• Gitea payload generation (3 tests): payload shape, body carries no information beyond what's in prd.json, and every real story's payload is well-formed and authority-clean.

Commands and results

python3 -m pytest -q tests/test_ralph_prd_schema.py

108 passed in 0.16s

python3 -m compileall -q packages tests

Exit code 0, no output (all files compile).

git diff --check

Exit code 0 (no whitespace errors).

python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json

OK: 55 stories validated.

python3 scripts/ralph_prd_schema.py validate .scratch/distributed-gguf-runtime/prd.json --fresh

ERROR: DGR-017: fresh backlog requires passes=false, got True
ERROR: DGR-018: fresh backlog requires passes=false, got True
2 validation error(s).

Expected: --fresh is the invariant for a backlog that hasn't started execution; this backlog has legitimately completed two stories, so it correctly fails that stricter check while passing the plain (in-progress) validate command above.

Baseline: full repository suite (ad hoc python3, not a project venv)

python3 -m pytest -q

20 failed, 776 passed, 13 skipped, 2 warnings in 244.17s (0:04:04)

None of the failures touch scripts/ralph_prd_schema.py or tests/test_ralph_prd_schema.py (neither file existed before this story; this story adds no changes to packages/). Four of the 20 failures reproduce exactly the pre-existing baseline defects DGR-017's evidence already recorded (test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias, test_torch_node_applies_tracker_load_shard_directive, test_shard_heal_cycle_surviving_node_covers_dead_peers_gap, test_a_node_with_an_unusable_precision_covers_no_layers). The remaining 16 (activation compression, dynamic routing, gossip/relay, manual route benchmark, openai gateway, TOPLoC calibration dispatch, tracker control plane) include a ModuleNotFoundError: langchain failure, indicating this ad hoc python3 lacks the project's dev extras (langchain-openai, etc.) rather than a real regression; this environment has no project virtualenv (e.g. no .venv-rocm) to run against instead. Not investigated further as out of scope for this story.

Limitations

• No real Gitea instance or API integration exists; to_gitea_issue_payload defines the payload shape (title/body/labels/milestone) only. Creating issues against a live Gitea server is future work, not claimed here.
• model is intentionally validated as an open string, not a closed enum, per RALPH-CONTEXT.md's "Quantization and placement are dynamic recipe inputs" constraint; the schema documents (metadataSchema.modelConvention) but does not restrict its value set.
• Validation and generation were exercised only against this feature's prd.json (.scratch/distributed-gguf-runtime/prd.json); docs/prd.json and other .scratch/*/prd.json files in this repo use a materially different (simpler) shape and are out of scope.

Dependency handoff

DGR-021 and DGR-025 (this story's derived blocks) may treat prd.json's metadataSchema, qualityGates, and this validator/generator as stable. Any future field addition to a story shape must extend STORY_FIELDS in scripts/ralph_prd_schema.py and the corresponding metadataSchema.requiredStoryFields/optionalStoryFields in prd.json together — validate_metadata_schema_consistency fails closed if they drift apart.