neuron-tai/packages/validator/README.md

# meshnet-validator

Optimistic fraud detection (ADR-0003, penalty amended by ADR-0015): the
validator re-runs a random ~5% sample of completed inference requests against
a trusted reference node. Audit-capable events are checked by teacher-forcing
the claimed token sequence through the reference node and verifying the
claimed TOPLOC activation proof. Legacy events without TOPLOC metadata still
fall back to text comparison until node-side proof capture lands.

## Why the penalty deters cheating

There is no upfront stake. Settlement is periodic (US-033), so a node always
has an unpaid **pending balance** — that balance *is* the collateral.

At a sampling rate `p`, a cheater who gains `G` per fraudulent job and loses
`L` when caught has expected value:

```
(1 - p) * G - p * L < 0
L > ((1 - p) / p) * G        # p = 0.05  ->  L > 19 x G
```

With the production settlement period of 24h, the pending balance at any
moment approximates a full day's earnings — hundreds to thousands of jobs —
which is far above the 19× bar. Each catch also records a strike; three
strikes ban the wallet (registration rejected, excluded from routes, unpaid
pending never settled), and the probationary period (first N jobs unpaid)
makes re-entry with a fresh wallet costly.

## TOPLOC audit contract

The validator expects audit-capable events to carry:

- `claimed_token_ids`: the final token sequence claimed by the prover.
- `toploc_proof`: compact TOPLOC proof data built from prover activations.

On audit the validator calls the reference node's `POST /v1/audit/toploc`
endpoint with the original messages plus `claimed_token_ids`. The reference
node must run a teacher-forced prefill over exactly that token sequence and
return the activations for TOPLOC verification. It must not free-generate a
second answer for audit.

Canonical audit parameters for the current alpha preset are:

```
dtype = "bfloat16"
quantization = "bfloat16"
decode_batching_size = 32
topk = 8
skip_prefill = true
encoding = "base64"
```

`verify_activation_proofs_detailed()` (`meshnet_validator.audit`) surfaces the
raw TOPLOC divergence — `exp_intersections` (worst-case across chunks),
`mant_err_mean`, `mant_err_median` — alongside the pass/fail bool. This is
what the calibration corpus below is built from; existing callers that only
need the bool keep using `verify_activation_proofs()`.

**Do not enable production audit thresholds before issue 21 closes.**
Production audit thresholds remain gated on the honest-noise calibration
corpus in issue 21: the tracker's `POST /v1/calibration/toploc/run`
(admin/validator-only, mirrors `POST /v1/benchmark/hop-penalty`) dispatches a
fixed prompt to every solo-capable registered node, verifies each node's
on-demand commitment against a teacher-forced reference replay, and records
the divergence into a SQLite corpus (`meshnet_tracker.calibration.
ToplocCalibrationStore`) keyed by node wallet + GPU model + dtype.
`GET /v1/calibration/toploc/results` reports the corpus plus:

- `envelope`: p99 honest-noise value per metric with a 20% safety margin —
  the recommended (not yet wired) tolerance constants.
- `gate_status.ready`: whether the corpus covers enough distinct hardware
  profiles (`--toploc-calibration-gate-min-hardware-profiles`, default 1).
  **Alpha exception:** with the hired-VPS-only launch fleet, `ready` may
  legitimately mean "covers every node we currently operate" — this must be
  revisited (raise the minimum) before a public/volunteer launch broadens
  the hardware mix, since a new corpus is required whenever the fleet's
  hardware composition changes.

Two operational notes:

- Shortening the settlement period shrinks the collateral. Period changes
  must weigh chain overhead against deterrence.
- A cheater immediately after a payout has little to forfeit — the
  strike/ban ladder covers that window.

## Usage

```python
ValidatorProcess(
    contracts=contracts,          # registry/validation boundary
    billing=ledger,               # BillingLedger — enables forfeiture
    reference_node_url="http://...",
    sample_rate=0.05,
)
```

Remote validators can instead call the tracker's privileged
`POST /v1/billing/forfeit` endpoint (non-empty Authorization header).

## Reputation-weighted audit rate (ADR-0018 §1, §6-7)

`sample_rate` is a flat coin flip: every wallet audited at the same rate.
Pass `audit_sampler=AdaptiveAuditSampler(...)` instead to make the audit
probability a function of each wallet's tenure and reputation — newcomers
and low-reputation wallets sampled at 20–30%, veterans in good standing
floor at ≥2% — while a running budget balance keeps the fleet-wide realized
rate anchored to `AuditRateConfig.target_rate` (default 5%) regardless of
the wallet mix. Passive tripwires (`detect_output_tripwire`) bump only that
one request's odds; they never strike, ban, or affect other wallets' rates.

```python
from meshnet_validator import AdaptiveAuditSampler, detect_output_tripwire

ValidatorProcess(
    contracts=contracts,
    reference_node_url="http://...",
    audit_sampler=AdaptiveAuditSampler(random_seed=42),
)
```

When `audit_sampler` is set, `sample_rate` is ignored — the sampler decides
per event, keyed on whichever route wallet has the lowest reputation.