neuron-tai/docs/adr/0018-fraud-detection-verification-and-reputation.md

ADR-0018: Fraud detection, verification, and reputation

Status: Accepted

Context

ADR-0003 established optimistic sampling with stake slashing; ADR-0015 replaced stake with pending-balance forfeiture as collateral. Pre-alpha audits identified two distinct fraud types:

1. Correctness fraud — wrong model, layer-skipping, garbage outputs.
2. Accounting fraud — inflated token counts or shard-span work units reported by nodes.

The validator today compares final text only and always blames the last hop (_final_text_node in packages/validator/meshnet_validator/__init__.py ~137–140) — wrong for multi-hop pipelines.

Research (.scratch/alpha-hardening/research-verifiable-inference.md, 2026-07-04) grounds the alpha design in deployed patterns (Prime Intellect TOPLOC, Hyperbolic PoSP, Gensyn Verde blame patterns). This ADR is the flagship decision record for alpha hardening.

Decisions

1. Anchor technique: optimistic accept + teacher-forced audit

• Default audit probability p ≈ 5% — a budget target, not a hard cap. Anomalies, low reputation, and disputes escalate rate; veterans floor at ≥ 2% (research §6, §8).
• Deterrence condition: at p = 0.05, penalty L must exceed L > g·(1−p)/p ≈ 19× per-job gain g (research §1.1). Full pending forfeiture is the primary penalty; three strikes → ban. The ×0.8-per-strike multiplier applies to routing/payout weight (reputation decay), not to the forfeiture amount.
• Single-tracker alpha: the tracker (or a designated reference node) is the auditor — no verifier market, no verifier's dilemma (research §1.1).

2. Detection primitive: ADOPT TOPLOC

• pip install toploc (MIT, PrimeIntellect-ai/toploc) for activation fingerprint commit + verify (research build-vs-adopt table).
• Teacher-forced prefill re-verification — compare in logit/activation space with tolerances, never free-running token equality (research §2).
• Pin one canonical precision/quantization per served model; TOPLOC detects precision drift by design.
• Per-hop boundary fingerprints extend TOPLOC's final-hidden-state encoding for multi-hop blame (research §1.2, §8 layer 1).

3. Commit layer: on-demand activation commitments

• Nodes commit compact TOPLOC-style fingerprints of output boundary activations per hop when selected for audit (on-demand, not every request — lower serving latency; brief retention window for recent activations).
• Commitments are audit pins, not proofs — correctness requires independent recomputation (research §4).

4. Blame layer: hop-boundary bisection (adapt Verde pattern)

On audit failure:

1. Referee (tracker) teacher-forces claimed token sequence through reference model.
2. Compare committed hop-boundary fingerprints to reference at each cut-point.
3. First divergent hop is the culprit — fixes _final_text_node last-hop-only bug.
4. Full interactive Truebit/Verde on-chain game and bitwise RepOps kernels: roadmap-only (research §1.2, §9).

5. Accounting fraud: tracker-authoritative metering

• Token counts come from the tracker's proxied stream/non-stream response parsing (server.py ~1890–1943), not node self-reports.
• Work units derive from tracker-assigned shard span at route construction (server.py ~1776–1782), not node-declared ranges at billing time.
• See issue H2.

6. Reputation model (graduated, persisted)

Reputation derives only from tracker-verified audit outcomes + uptime/latency — never peer ratings (research §6, collusion surface).

Signal	Effect
Clean audits	Slow reputation build; higher routing weight
Strike	×0.8 routing multiplier per strike (graduated decay)
Failed audit	Full pending forfeiture + strike; audit rate → maximum
Ban (3 strikes)	Registration rejected; excluded from routes; pending never paid
New/low reputation	Elevated audit rate (20–30% target for newcomers)
Inactivity	Reputation decay

Persist strike/ban/reputation in SQLite alongside billing (issue A1/A5). Probation (first N jobs unpaid) retained as re-entry cost.

7. Passive tripwires

Perplexity/repetition/truncation heuristics on all traffic raise audit rate without direct punishment (research §8 layer 5).

8. Roadmap-only (explicitly NOT alpha)

• zkML proofs of LLM inference (research §1.3)
• GPU TEE attestation on consumer cards (research §1.4)
• Fully trustless Verde interactive games + RepOps bitwise kernels (research §9)
• Decentralized verifier markets
• Peer-rating reputation (EigenTrust)
• PoW as correctness proof — registration-time hardware attestation only, optional (research §4)

Consequences

• Validator must be rewired: TOPLOC verify, hop blame, tracker-authoritative events — not string compare on final text alone.
• Threshold calibration requires an honest-noise corpus across the volunteer fleet before production thresholds (research §8).
• ADR-0003 remains historical; penalty mechanics follow ADR-0015 + this ADR.
• Implementation order: auth + persistence → accounting → TOPLOC → bisection → reputation routing (.scratch/alpha-hardening/README.md).

Related

• Research: .scratch/alpha-hardening/research-verifiable-inference.md (§8 recommended scheme, §9 roadmap, build-vs-adopt table)
• ADR-0015 (forfeiture collateral)
• ADR-0016 (alpha scope)
• ADR-0017 (validator/forfeit auth)
• Issues: 06-fraud-toploc-integration.md through 10-fraud-penalty-calibration-wiring.md