feat: DGR-001 - Lock the safetensors-versus-GGUF performance contract

This commit is contained in:
Dobromir Popov
2026-07-13 17:55:55 +03:00
parent 59f2486bf2
commit e24db7854f
8 changed files with 248 additions and 4 deletions

View File

@@ -0,0 +1,30 @@
# DGR-001 real-benchmark blocker
Status: blocked only for the required real-model measurement. The deterministic
harness, report schema, and immutable contract are implemented and tested; this
file deliberately does not turn an unrun benchmark into a passing result.
## Verified environment state (2026-07-13)
- Mounted GGUF artifacts exist under `/run/media/popov/DATA/llm/`.
- `llama-server` is not on `PATH`.
- The available Python test environment has neither `torch` nor `transformers`.
- No matching local safetensors snapshot was found for an installed GGUF recipe.
Therefore this session cannot run the controlled same-model, same-revision,
same-machine comparison without downloading/installing new runtime/model assets.
That is intentionally not inferred from the story request.
## Continuation
1. Put a matching safetensors snapshot and near-lossless plus quantized GGUF
artifacts below one mounted-drive root, never `/home`.
2. Install or build the pinned `llama-server`, and use `.venv-rocm` when testing
the Radeon backend.
3. Compute each artifact SHA-256 and create a config declaring the same
`source_model_id` and `source_model_revision` for every recipe.
4. Run the command in `commands.txt` with
`MESHNET_ENABLE_REAL_INFERENCE_TESTS=1`; save its JSON report and summary in
this directory, then evaluate it against `performance-contract.json`.
5. Only after those results and all quality gates pass may DGR-001 be marked
done and DGR-004 consume the baseline.