Add alpha-hardening ADRs and issue plan from pre-release audit.
Lock alpha scope, tracker auth, TOPLOC fraud verification, and deferred multi-tracker money-path work; supersede legacy fraud issues with ADR-0018. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
Status: ready-for-human
|
||||
|
||||
# 21 — Honest-noise TOPLOC calibration corpus
|
||||
|
||||
## What to build
|
||||
|
||||
Before enabling production TOPLOC audit thresholds, collect an **honest-noise baseline** across the heterogeneous volunteer fleet. Run identical inference jobs on every active node/GPU combo; measure the divergence envelope (TOPLOC exponent/mantissa deltas, logprob-rank spread) under real hardware variance.
|
||||
|
||||
Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and-reputation.md): threshold calibration requires an honest-noise corpus across the fleet before production thresholds.
|
||||
|
||||
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Scripted benchmark job (fixed prompt, model preset, seed policy) runnable on all nodes
|
||||
- [ ] Aggregated corpus artifact (per node: GPU model, dtype, TOPLOC deltas vs reference)
|
||||
- [ ] Recommended tolerance thresholds documented (p99 honest envelope + safety margin)
|
||||
- [ ] Gate checklist: production audit enable blocked until corpus covers ≥N distinct hardware profiles (define N in runbook, suggest ≥3)
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Corpus collected from current fleet (or documented subset + extrapolation note)
|
||||
- [ ] Threshold constants in validator config derived from corpus, not guessed
|
||||
- [ ] False-positive rate estimate documented at chosen thresholds
|
||||
- [ ] README / runbook cross-link: **do not enable production audits** until this issue closes
|
||||
|
||||
## ADR links
|
||||
|
||||
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) — Consequences (honest-noise corpus)
|
||||
|
||||
## Blocked by
|
||||
|
||||
- `06-fraud-toploc-integration.md` (TOPLOC wired; calibration uses same primitive)
|
||||
|
||||
## Blocks (prod gate)
|
||||
|
||||
- Production enable of adaptive audit thresholds (issues 09–10 in prod)
|
||||
Reference in New Issue
Block a user