feat: DGR-001 - Lock the safetensors-versus-GGUF performance contract
This commit is contained in:
@@ -0,0 +1,30 @@
|
||||
# DGR-001 real-benchmark blocker
|
||||
|
||||
Status: blocked only for the required real-model measurement. The deterministic
|
||||
harness, report schema, and immutable contract are implemented and tested; this
|
||||
file deliberately does not turn an unrun benchmark into a passing result.
|
||||
|
||||
## Verified environment state (2026-07-13)
|
||||
|
||||
- Mounted GGUF artifacts exist under `/run/media/popov/DATA/llm/`.
|
||||
- `llama-server` is not on `PATH`.
|
||||
- The available Python test environment has neither `torch` nor `transformers`.
|
||||
- No matching local safetensors snapshot was found for an installed GGUF recipe.
|
||||
|
||||
Therefore this session cannot run the controlled same-model, same-revision,
|
||||
same-machine comparison without downloading/installing new runtime/model assets.
|
||||
That is intentionally not inferred from the story request.
|
||||
|
||||
## Continuation
|
||||
|
||||
1. Put a matching safetensors snapshot and near-lossless plus quantized GGUF
|
||||
artifacts below one mounted-drive root, never `/home`.
|
||||
2. Install or build the pinned `llama-server`, and use `.venv-rocm` when testing
|
||||
the Radeon backend.
|
||||
3. Compute each artifact SHA-256 and create a config declaring the same
|
||||
`source_model_id` and `source_model_revision` for every recipe.
|
||||
4. Run the command in `commands.txt` with
|
||||
`MESHNET_ENABLE_REAL_INFERENCE_TESTS=1`; save its JSON report and summary in
|
||||
this directory, then evaluate it against `performance-contract.json`.
|
||||
5. Only after those results and all quality gates pass may DGR-001 be marked
|
||||
done and DGR-004 consume the baseline.
|
||||
Reference in New Issue
Block a user