1.5 KiB
1.5 KiB
DGR-001 real-benchmark blocker
Status: blocked only for the required real-model measurement. The deterministic harness, report schema, and immutable contract are implemented and tested; this file deliberately does not turn an unrun benchmark into a passing result.
Verified environment state (2026-07-13)
- Mounted GGUF artifacts exist under
/run/media/popov/DATA/llm/. llama-serveris not onPATH.- The available Python test environment has neither
torchnortransformers. - No matching local safetensors snapshot was found for an installed GGUF recipe.
Therefore this session cannot run the controlled same-model, same-revision, same-machine comparison without downloading/installing new runtime/model assets. That is intentionally not inferred from the story request.
Continuation
- Put a matching safetensors snapshot and near-lossless plus quantized GGUF
artifacts below one mounted-drive root, never
/home. - Install or build the pinned
llama-server, and use.venv-rocmwhen testing the Radeon backend. - Compute each artifact SHA-256 and create a config declaring the same
source_model_idandsource_model_revisionfor every recipe. - Run the command in
commands.txtwithMESHNET_ENABLE_REAL_INFERENCE_TESTS=1; save its JSON report and summary in this directory, then evaluate it againstperformance-contract.json. - Only after those results and all quality gates pass may DGR-001 be marked done and DGR-004 consume the baseline.