neuron-tai/packages/validator/README.md

meshnet-validator

Optimistic fraud detection (ADR-0003, penalty amended by ADR-0015): the validator re-runs a random ~5% sample of completed inference requests against a trusted reference node. Audit-capable events are checked by teacher-forcing the claimed token sequence through the reference node and verifying the claimed TOPLOC activation proof. Legacy events without TOPLOC metadata still fall back to text comparison until node-side proof capture lands.

Why the penalty deters cheating

There is no upfront stake. Settlement is periodic (US-033), so a node always has an unpaid pending balance — that balance is the collateral.

At a sampling rate p, a cheater who gains G per fraudulent job and loses L when caught has expected value:

(1 - p) * G - p * L < 0
L > ((1 - p) / p) * G        # p = 0.05  ->  L > 19 x G

With the production settlement period of 24h, the pending balance at any moment approximates a full day's earnings — hundreds to thousands of jobs — which is far above the 19× bar. Each catch also records a strike; three strikes ban the wallet (registration rejected, excluded from routes, unpaid pending never settled), and the probationary period (first N jobs unpaid) makes re-entry with a fresh wallet costly.

TOPLOC audit contract

The validator expects audit-capable events to carry:

• claimed_token_ids: the final token sequence claimed by the prover.
• toploc_proof: compact TOPLOC proof data built from prover activations.

On audit the validator calls the reference node's POST /v1/audit/toploc endpoint with the original messages plus claimed_token_ids. The reference node must run a teacher-forced prefill over exactly that token sequence and return the activations for TOPLOC verification. It must not free-generate a second answer for audit.

Canonical audit parameters for the current alpha preset are:

dtype = "bfloat16"
quantization = "bfloat16"
decode_batching_size = 32
topk = 8
skip_prefill = true
encoding = "base64"

verify_activation_proofs_detailed() (meshnet_validator.audit) surfaces the raw TOPLOC divergence — exp_intersections (worst-case across chunks), mant_err_mean, mant_err_median — alongside the pass/fail bool. This is what the calibration corpus below is built from; existing callers that only need the bool keep using verify_activation_proofs().

Do not enable production audit thresholds before issue 21 closes. Production audit thresholds remain gated on the honest-noise calibration corpus in issue 21: the tracker's POST /v1/calibration/toploc/run (admin/validator-only, mirrors POST /v1/benchmark/hop-penalty) dispatches a fixed prompt to every solo-capable registered node, verifies each node's on-demand commitment against a teacher-forced reference replay, and records the divergence into a SQLite corpus (meshnet_tracker.calibration. ToplocCalibrationStore) keyed by node wallet + GPU model + dtype. GET /v1/calibration/toploc/results reports the corpus plus:

• envelope: p99 honest-noise value per metric with a 20% safety margin — the recommended (not yet wired) tolerance constants.
• gate_status.ready: whether the corpus covers enough distinct hardware profiles (--toploc-calibration-gate-min-hardware-profiles, default 1). Alpha exception: with the hired-VPS-only launch fleet, ready may legitimately mean "covers every node we currently operate" — this must be revisited (raise the minimum) before a public/volunteer launch broadens the hardware mix, since a new corpus is required whenever the fleet's hardware composition changes.

Two operational notes:

• Shortening the settlement period shrinks the collateral. Period changes must weigh chain overhead against deterrence.
• A cheater immediately after a payout has little to forfeit — the strike/ban ladder covers that window.

Usage

ValidatorProcess(
    contracts=contracts,          # registry/validation boundary
    billing=ledger,               # BillingLedger — enables forfeiture
    reference_node_url="http://...",
    sample_rate=0.05,
)

Remote validators can instead call the tracker's privileged POST /v1/billing/forfeit endpoint (non-empty Authorization header).

Reputation-weighted audit rate (ADR-0018 §1, §6-7)

sample_rate is a flat coin flip: every wallet audited at the same rate. Pass audit_sampler=AdaptiveAuditSampler(...) instead to make the audit probability a function of each wallet's tenure and reputation — newcomers and low-reputation wallets sampled at 20–30%, veterans in good standing floor at ≥2% — while a running budget balance keeps the fleet-wide realized rate anchored to AuditRateConfig.target_rate (default 5%) regardless of the wallet mix. Passive tripwires (detect_output_tripwire) bump only that one request's odds; they never strike, ban, or affect other wallets' rates.

from meshnet_validator import AdaptiveAuditSampler, detect_output_tripwire

ValidatorProcess(
    contracts=contracts,
    reference_node_url="http://...",
    audit_sampler=AdaptiveAuditSampler(random_seed=42),
)

When audit_sampler is set, sample_rate is ignored — the sampler decides per event, keyed on whichever route wallet has the lowest reputation.