fix: cryptographically bind DGR-001 evidence
This commit is contained in:
@@ -45,17 +45,17 @@ All recipes completed every request with zero failures.
|
||||
|
||||
| Metric | Transformers BF16 | llama.cpp BF16 | llama.cpp Q4_K_M |
|
||||
|---|---:|---:|---:|
|
||||
| Decode tok/s, c=1 | 46.1 | 88.0 | 170.1 |
|
||||
| Aggregate decode tok/s, c=4 | 47.1 | 211.4 | 206.4 |
|
||||
| TTFT p50, c=1 | 37.5 ms | 43.9 ms | 23.8 ms |
|
||||
| Decode tok/s, c=1 | 40.8 | 98.5 | 207.7 |
|
||||
| Aggregate decode tok/s, c=4 | 46.5 | 222.8 | 195.7 |
|
||||
| TTFT p50, c=1 | 40.0 ms | 15.1 ms | 21.6 ms |
|
||||
| Peak resident memory, c=1 | 1.94 GB | 1.11 GB | 0.54 GB |
|
||||
| Artifact size | 1.00 GB | 0.99 GB | 0.40 GB |
|
||||
| Failures | 0 | 0 | 0 |
|
||||
|
||||
Against the reference, the eligible Q4_K_M lane measured:
|
||||
|
||||
- single-request decode speedup: **3.69×**;
|
||||
- concurrency-4 aggregate throughput speedup: **4.38×**;
|
||||
- single-request decode speedup: **5.10×**;
|
||||
- concurrency-4 aggregate throughput speedup: **4.20×**;
|
||||
- resident-memory ratio: **0.279×**;
|
||||
- artifact-size ratio: **0.398×**.
|
||||
|
||||
@@ -99,9 +99,13 @@ Thresholds were not changed after observing these results.
|
||||
- llama.cpp's `predicted_n` includes the first token while `predicted_ms` begins
|
||||
after it; the driver subtracts that token so decode throughput matches the
|
||||
Transformers inter-token convention.
|
||||
- `performance_contract.py` rejects wrong plans, synthetic evidence, missing
|
||||
recipes/concurrency, mixed model revisions, incomplete quality coverage,
|
||||
failed references, and missing artifact/host provenance.
|
||||
- `performance_contract.py` rejects wrong plans, unsigned or incorrectly signed
|
||||
real evidence, wrong config/artifact/runtime/backend/host bindings, missing
|
||||
recipes/concurrency, mixed model revisions, incomplete quality coverage, and
|
||||
failed references.
|
||||
- Every non-synthetic report is Ed25519-signed over the complete canonical JSON,
|
||||
including raw outcomes and metrics. The contract pins the public key and exact
|
||||
config SHA-256; the private key remains outside Git at mode `0600`.
|
||||
- Quantized drift remains advisory. Only the near-lossless lane can satisfy the
|
||||
quality gate, and only performance-fit recipes can earn speed/fit benefits.
|
||||
|
||||
@@ -120,8 +124,8 @@ Thresholds were not changed after observing these results.
|
||||
## Verification
|
||||
|
||||
```text
|
||||
Targeted: 22 passed
|
||||
Full suite: 749 passed, 13 skipped
|
||||
Targeted: 24 passed
|
||||
Full suite: 751 passed, 13 skipped
|
||||
Earlier cancellation retry matrix, DGR-001: 4/5 passed
|
||||
Earlier cancellation retry matrix, clean d904c40: 4/5 passed
|
||||
compileall: passed
|
||||
@@ -129,9 +133,9 @@ git diff --check: passed
|
||||
Evidence JSON parse/integrity checks: passed
|
||||
```
|
||||
|
||||
The full-suite exception is documented in `known-unrelated-failure.md` and
|
||||
satisfies the issue's explicit clean-tree reproduction clause. DGR-001 changes
|
||||
no tracker/proxy files.
|
||||
An earlier intermittent tracker cancellation race reproduced at the same rate on
|
||||
the clean base and is retained in `known-unrelated-failure.md`; the final suite
|
||||
completed green. DGR-001 changes no tracker/proxy files.
|
||||
|
||||
The earlier Ralph claim that the full suite was blocked by Protobuf 6.33.6 was
|
||||
invalid: it used Hermes Agent's internal venv. Verification above used the
|
||||
|
||||
Reference in New Issue
Block a user