fix: cryptographically bind DGR-001 evidence

This commit is contained in:
Dobromir Popov
2026-07-13 19:38:14 +03:00
parent 9e67b829e3
commit b1c9deeb01
11 changed files with 1349 additions and 903 deletions

View File

@@ -45,17 +45,17 @@ All recipes completed every request with zero failures.
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|---|---:|---:|---:|
| Decode tok/s, c=1 | 46.1 | 88.0 | 170.1 |
| Aggregate decode tok/s, c=4 | 47.1 | 211.4 | 206.4 |
| TTFT p50, c=1 | 37.5 ms | 43.9 ms | 23.8 ms |
| Decode tok/s, c=1 | 40.8 | 98.5 | 207.7 |
| Aggregate decode tok/s, c=4 | 46.5 | 222.8 | 195.7 |
| TTFT p50, c=1 | 40.0 ms | 15.1 ms | 21.6 ms |
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
| Failures | 0 | 0 | 0 |
Against the reference, the eligible Q4_K_M lane measured:
- single-request decode speedup: **3.69×**;
- concurrency-4 aggregate throughput speedup: **4.38×**;
- single-request decode speedup: **5.10×**;
- concurrency-4 aggregate throughput speedup: **4.20×**;
- resident-memory ratio: **0.279×**;
- artifact-size ratio: **0.398×**.
@@ -99,9 +99,13 @@ Thresholds were not changed after observing these results.
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
after it; the driver subtracts that token so decode throughput matches the
Transformers inter-token convention.
- `performance_contract.py` rejects wrong plans, synthetic evidence, missing
recipes/concurrency, mixed model revisions, incomplete quality coverage,
failed references, and missing artifact/host provenance.
- `performance_contract.py` rejects wrong plans, unsigned or incorrectly signed
real evidence, wrong config/artifact/runtime/backend/host bindings, missing
recipes/concurrency, mixed model revisions, incomplete quality coverage, and
failed references.
- Every non-synthetic report is Ed25519-signed over the complete canonical JSON,
including raw outcomes and metrics. The contract pins the public key and exact
config SHA-256; the private key remains outside Git at mode `0600`.
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
quality gate, and only performance-fit recipes can earn speed/fit benefits.
@@ -120,8 +124,8 @@ Thresholds were not changed after observing these results.
## Verification
```text
Targeted: 22 passed
Full suite: 749 passed, 13 skipped
Targeted: 24 passed
Full suite: 751 passed, 13 skipped
Earlier cancellation retry matrix, DGR-001: 4/5 passed
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
compileall: passed
@@ -129,9 +133,9 @@ git diff --check: passed
Evidence JSON parse/integrity checks: passed
```
The full-suite exception is documented in `known-unrelated-failure.md` and
satisfies the issue's explicit clean-tree reproduction clause. DGR-001 changes
no tracker/proxy files.
An earlier intermittent tracker cancellation race reproduced at the same rate on
the clean base and is retained in `known-unrelated-failure.md`; the final suite
completed green. DGR-001 changes no tracker/proxy files.
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
invalid: it used Hermes Agent's internal venv. Verification above used the