48 lines
2.4 KiB
Markdown
48 lines
2.4 KiB
Markdown
Status: done
|
||
|
||
# 06 — FRAUD: TOPLOC integration (teacher-forced audit primitive)
|
||
|
||
## What to build
|
||
|
||
Adopt [TOPLOC](https://github.com/PrimeIntellect-ai/toploc) (MIT, `pip install toploc`) for activation fingerprint commit and verify. Replace string-equality validator checks with teacher-forced prefill + TOPLOC tolerance matching.
|
||
|
||
**Estimated effort:** 2+ sessions. First landing should be the validator-only TOPLOC primitive and docs; node runtime commitments/on-demand capture can follow in issue 07 if this grows.
|
||
|
||
| Subtask | Owner package | Deliverable |
|
||
|---|---|---|
|
||
| Validator audit primitive | `packages/validator/` | Teacher-forced prefill, TOPLOC verify, unit tests with stub tensors |
|
||
| Node runtime commitments | `packages/node/` (if prover-side) | On-demand activation fingerprint generation on audit-selected requests; move to issue 07 if it blocks the validator primitive |
|
||
|
||
**Code refs:**
|
||
|
||
- `packages/validator/meshnet_validator/__init__.py` — `_run_reference`, `_outputs_match` (~92–148)
|
||
- `packages/validator/README.md` — deterrence math (update for 19× at p=0.05)
|
||
- Research: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layers 1–2, build-vs-adopt table
|
||
|
||
Pin one canonical precision/quantization per model preset. Add `toploc` to validator (and node if prover-side) dependencies.
|
||
|
||
## Test-first
|
||
|
||
1. Red: validator compares final text strings — fails on cross-GPU honest divergence (document expected).
|
||
2. Green: stub activation tensors + TOPLOC proofs round-trip in unit test.
|
||
3. Integration: reference node teacher-forces tokens; verify accepts honest proof, rejects swapped precision.
|
||
|
||
## Acceptance criteria
|
||
|
||
- [x] `toploc` dependency declared; `build_proofs_*` / `verify_proofs_*` wired
|
||
- [x] Validator re-runs claimed token sequence as prefill, not free generation
|
||
- [x] Model preset documents canonical dtype/quantization
|
||
- [x] README updated: 19× deterrence at 5% audit (research §1.1)
|
||
- [x] Tests with deterministic stub tensors (no GPU required in CI)
|
||
|
||
## ADR links
|
||
|
||
- [ADR-0018](../../docs/adr/0018-fraud-detection-verification-and-reputation.md) §2
|
||
- Research: [research-verifiable-inference.md](../research-verifiable-inference.md) §8, §9 build-vs-adopt
|
||
|
||
## Blocked by
|
||
|
||
- `05-a1-a5-persist-strike-ban-reputation_completed.md`
|
||
|
||
**Prod gate:** do not enable production audit thresholds until `21-honest-noise-calibration-corpus.md` completes (see README Phase 2 note).
|