neuron-tai/.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md

14 — Enforce the GGUF-versus-safetensors release gate

Status: ready-for-agent

Mandatory fresh-session context

• Read RALPH-CONTEXT.md completely before changing code.
• This issue is DGR-014 in prd.json.
• Read the evidence README for every dependency listed below.
• Inspect current code and git status; historical text and previous agent claims are not evidence.

Description

As the product owner, I need an end-to-end comparison so that the native runtime ships only if it advances model access or performance.

Expected durable outputs

• Immutable comparison against DGR-001 thresholds
• Machine-readable final report
• Ship/optimize/stop recommendation
• evidence/DGR-014/README.md

Acceptance criteria

• Run current distributed safetensors and distributed GGUF routes on the same certified model/hardware/network scenario where technically comparable.
• Report quality, TTFT, prefill/decode throughput, aggregate concurrency throughput, p95 latency, seam cost, memory, KV pressure, failures, and cleanup.
• Evaluate against the DGR-001 performance contract without changing thresholds after seeing results.
• Ship recommendation is one of: promote GGUF, optimize a measured bottleneck with a new bounded task, or stop the native track.
• Results clearly separate quantization gains from transport/runtime gains.
• Targeted pytest tests pass
• python -m compileall packages tests passes for Python changes
• git diff --check passes
• Default tests remain deterministic, model-download-free, API-credit-free, and GPU-free
• Full deterministic pytest -q passes, or the exact pre-existing unrelated failure is recorded with a clean-tree reproduction
• Pinned native C++ target builds and focused CTest/protocol tests pass where native code is touched
• llama.cpp patch stack applies cleanly to the exact pinned commit where patch code is touched
• Real-model execution is opt-in through MESHNET_ENABLE_REAL_INFERENCE_TESTS=1 and records exact artifact/runtime/hardware evidence
• Model artifacts remain on the configured mounted-drive storage and never under /home
• Read .scratch/distributed-gguf-runtime/RALPH-CONTEXT.md and this story issue completely before changing code
• Read and verify every dependency evidence README before relying on dependency behavior
• Preserve all pre-existing working-tree changes and stage only files belonging to this story
• Write .scratch/distributed-gguf-runtime/evidence/DGR-014/README.md with files changed, exact commands and real results, limitations, compatibility notes, and dependent-story handoff
• Update only this story issue to Status: done after every acceptance criterion and quality gate passes

Dependency handoff

• DGR-001 must have passes: true; read ../evidence/DGR-001/README.md and verify its referenced files/commands.
• DGR-011 must have passes: true; read ../evidence/DGR-011/README.md and verify its referenced files/commands.
• DGR-012 must have passes: true; read ../evidence/DGR-012/README.md and verify its referenced files/commands.
• DGR-013 must have passes: true; read ../evidence/DGR-013/README.md and verify its referenced files/commands.

Finish contract

• Create the task evidence directory and durable handoff required above.
• Preserve real failures and blockers; never fabricate benchmark, model, test or hardware output.
• Change this issue to Status: done only after all criteria pass.
• Emit <promise>COMPLETE</promise> only after the evidence handoff exists.

References

• Ralph execution context
• PRD
• Implementation strategy
• Current architecture
• Architecture decision