neuron-tai/.scratch/alpha-hardening/research-verifiable-inference.md

Verifiable Inference & Fraud Layer — Research Findings

Date: 2026-07-04 Scope: Ground the neuron-tai fraud/verification layer in 2026 state-of-the-art and in what comparable decentralized compute networks actually do. Alpha context assumed throughout: volunteer consumer GPUs (8 GB class, e.g. RTX 4060), pipeline-parallel multi-hop inference where different nodes hold different layer ranges, a single trusted tracker (coordinator), devnet mock-USDT payments but reputation that carries forward, and an owner-approved verification budget of roughly 5% redundant recomputation.

---

Executive summary

1. The proven, deployed pattern for exactly our setting is: optimistic acceptance + random teacher-forced re-verification + slashing/reputation penalties. This is what Prime Intellect runs in production for INTELLECT-2 (TOPLOC validators, random sampling, slash-and-evict) and what Hyperbolic formalized game-theoretically (Proof of Sampling). At a 5% audit rate the math works if the penalty for a caught cheat exceeds ~19× the per-job gain — which carried-forward reputation plus forfeited pending balance easily provides.
2. Teacher-forced logit/activation re-verification is a recognized, robust technique — it is the core of TOPLOC (ICML 2025), Token-DiFR (2025), and log-probability auditing papers. It sidesteps autoregressive divergence entirely and is up to ~100× cheaper than the original generation because verification prefill is compute-bound while generation is memory-bound.
3. Bitwise output equality is unattainable across heterogeneous volunteer GPUs; honest nodes on different hardware/batch conditions produce different tokens even at temperature 0. All robust schemes compare in logit/activation space with tolerances (TOPLOC exponent/mantissa thresholds, DiFR divergence scores) or force bitwise reproducibility with special kernels (Gensyn RepOps) at a performance cost.
4. zkML and GPU TEEs are roadmap-only. zkML in 2025–2026 proves GPT-2-scale models in tens of seconds per inference (100–10,000× overhead); nothing near serving speed for multi-billion-parameter models. GPU confidential computing exists only on H100/Blackwell-class datacenter hardware paired with server CPUs (SEV-SNP/TDX) — categorically unavailable on consumer volunteer cards.
5. Layer-skipping is NOT reliably detectable by output quality. Careful pruning of ~25% of layers keeps ~90% of benchmark scores (ShortGPT); models degrade gracefully until a sharp collapse around 20–55% removal depending on family. But any layer skipping changes hidden states and logits drastically, so per-position activation/logit comparison catches it essentially always (TOPLOC reports 100% detection of model modifications in its evals). Verify in logit space, not by eyeballing text quality.
6. Classic PoW is Sybil/hardware admission control, not correctness proof — io.net's hourly PoW is the cautionary tale (it verifies a GPU exists and has claimed VRAM, nothing about job correctness). Self-computed activation checksums are binding commitments, not correctness proofs: a cheater simply commits to its wrong values. Commitments earn their keep only when a referee later recomputes and compares — i.e., for audit pinning and dispute bisection.

---

1. Technique-by-technique review

1.1 Optimistic verification + spot-checking (probabilistic deterrence)

Mechanism. Accept results by default; re-run a random, unpredictable subset with an independent replica; punish divergences. Hyperbolic's Proof of Sampling (PoSP) paper proves a pure-strategy Nash equilibrium in which rational nodes are honest, and claims the spot-check approach adds well under 1% overhead when everyone is rational (PoSP, arXiv:2405.00295; Hyperbolic spML breakdown). VeriLLM adds VRF-selected audit indices over Merkle-committed hidden states so verifiers can't be lazy or predictable (VeriLLM, arXiv:2509.24257). Prime Intellect's production deployment: validators randomly sample committed batches, and "since the Inference Provider does not know which generations will be checked, they are incentivized to be honest on all generations" (INTELLECT-2, arXiv:2505.07291).

Sampling math. With independent audit probability p per job:

• A node that cheats on every job survives N jobs undetected with probability (1−p)^N. At p = 0.05: ~36% survive 20 jobs, ~4.6% survive 60 jobs, ~0.6% survive 100 jobs. Detection is near-certain over any meaningful volume.
• A node that cheats on a fraction f of jobs is caught per job with probability p·f; expected jobs until caught = 1/(p·f). At p = 0.05, f = 0.2: caught within ~100 jobs on average. Low-rate cheating stretches time-to-detection linearly — this is why the penalty must scale with accumulated stake/reputation, not per-incident.
• Deterrence condition (rational node): cheating gains g per job and loses L when caught, so expected value of cheating is (1−p)·g − p·L. Honesty dominates iff L > g·(1−p)/p ≈ 19·g at p = 0.05. If a caught node forfeits its pending balance and a reputation that gates future routing (i.e., discounted future earnings), L is orders of magnitude above 19× a single job's margin. This is exactly the structure PoSP formalizes and the existing neuron-tai forfeiture design (pending-balance forfeiture + strikes + ban) already implements.

The verifier-incentive caveat. Truebit identified the classic failure mode: if the system works and nobody cheats, verifiers never earn anything and stop checking; Truebit's answer was "forced errors" with jackpot payouts, and it estimated the verification tax at 500–5000% of task cost to keep independent verifiers attentive (Truebit whitepaper §2, §4). This whole problem disappears in a single-tracker alpha: the tracker audits as a protocol function paid from the protocol cut, not as a profit-seeking third party. It becomes relevant again only when verification is decentralized.

Feasibility for alpha: HIGH. This is the anchor technique. 5% audit budget is in the same range as deployed systems (OTR proposes ρ ≈ 1% spot-checks as sufficient deterrent in a TEE-hybrid design — arXiv:2512.20176); 5% gives comfortable margin for a young network with thin reputational stakes.

1.2 Interactive verification / refereed delegation (Truebit-style bisection)

Mechanism. Solver commits Merkle roots of intermediate execution states. On dispute, challenger and solver binary-search over the committed states to the first step where they diverge; a referee (smart contract or trusted party) recomputes only that single step and rules. One honest challenger suffices; the game costs O(log n) rounds (Truebit whitepaper; Truebit explainer, Medium).

ML adaptation. Gensyn's Verde adapts this to neural networks with a two-phase bisection: first narrow to the training/inference step, then to the single operator in the computational graph; the referee recomputes one operator, needing ~two orders of magnitude less compute than the full job. Crucially, Verde only works because RepOps makes execution bitwise reproducible across heterogeneous hardware by fixing floating-point operation order — otherwise honest nodes diverge and bisection finds spurious "fraud" (Verde paper, arXiv:2502.19405; Gensyn Verde blog).

Mapping to our multi-hop pipeline. Our pipeline is naturally pre-bisected: each hop's boundary activations are the "intermediate states." A workable scheme:

1. Each node commits a compact fingerprint (TOPLOC-style top-k encoding, not a raw hash — see §2) of its input and output boundary activations per request.
2. When an end-to-end audit fails, the referee (tracker) teacher-forces the full claimed token sequence through a reference model and computes reference boundary activations at each hop cut-point in one forward pass.
3. The first hop whose committed output fingerprint diverges from the reference (beyond tolerance) while its input fingerprint matched is the culprit. No interactive game is needed at hop granularity because the number of hops is small — the referee can check all cut-points in a single replay.
4. Bisection within a hop (to a layer or operator, Verde-style) is only needed if you must prove fault to a third party at fine granularity; for internal blame assignment, hop granularity is enough since payment and reputation are per node.

The main adaptation cost is the referee needing enough VRAM to run the full model (or to re-run hop layer-ranges one at a time, which any 8 GB card can do for its own range).

Feasibility for alpha: MEDIUM-HIGH in the simplified "referee replays and compares committed hop boundaries" form (no on-chain game, tracker as referee). The full trustless interactive game with bitwise-reproducible kernels is roadmap.

1.3 zkML (zero-knowledge proofs of inference)

2025–2026 state. The frontier moved fast but remains far from LLM serving speed:

• EZKL (ONNX→Halo2) is the most-used toolkit; generic overhead is characterized as 100–10,000× native execution, and much real usage is verifiable data science rather than LLMs (Equilibrium Labs survey).
• zkGPT (USENIX Security 2025) proves a GPT-2 inference in under 25 seconds on a CPU server — 185× faster than the prior ZKML system (zkGPT).
• NanoZK (2026) proves GPT-2-scale transformer blocks in ~43 s with 6.9 KB proofs via layerwise decomposition, 52–228× faster than EZKL (NanoZK, arXiv:2603.18046).
• For a 7B model, one estimate puts pure-zkML verification at over 20 minutes per inference (Optimistic TEE-Rollups, arXiv:2512.20176).
• Project reality check: Modulus Labs (authored the field-defining benchmark "The Cost of Intelligence", demonstrated proving of a multi-billion-parameter LLM as a feat, not a service) was acquired by Tools for Humanity in 2024 and now does World ID cryptography (announcement). Giza uses ZK proofs for small, simple DeFi-agent models on Starknet (Equilibrium survey). EZKL itself has drifted toward verifiable data science and co-SNARKs.

Honest read: proving is ~GPT-2 scale (~1.5B params max in tens of seconds), on beefy servers, per short inference. For a volunteer network serving multi-billion-parameter models interactively, zkML is 3+ orders of magnitude away, and proof generation would crush an 8 GB consumer card regardless.

Feasibility for alpha: NONE (roadmap-only).

1.4 TEE / confidential computing on GPUs

What it gives. NVIDIA H100 is the first GPU with a hardware TEE anchored in an on-die root of trust: measured/secure boot, SPDM session with a driver inside a CPU confidential VM, and a signed attestation report covering GPU firmware/state that a remote party verifies against NVIDIA's attestation service before releasing work (NVIDIA blog; CACM: Creating the First Confidential GPUs). Combined with CPU TEE attestation (AMD SEV-SNP or Intel TDX measuring the VM image), you get a verifiable claim that a specific measured software stack loaded a specific model and ran on genuine hardware, with only ~2–6% overhead (Blackwell figures, GPUYard setup guide) or ~5–10% generally (Equilibrium survey). Phala runs exactly this to sell "verifiable LLMs" through OpenRouter (Phala blog).

Caveats. Attestation proves the environment, not the mathematics — you trust NVIDIA/AMD/Intel hardware and are exposed to side channels and firmware bugs (the OTR paper layers ZK spot-checks on top of TEEs precisely because of compromised-TEE risk). And critically for us: CC mode requires Hopper/Blackwell datacenter GPUs plus server platforms with SEV-SNP/TDX enabled in BIOS and the open kernel modules (NVIDIA deployment guide). No RTX consumer card supports it. Volunteer nodes on 4060-class hardware categorically cannot provide GPU TEE attestation; Intel SGX on client CPUs is likewise not a path (deprecated on consumer parts, and it would only cover CPU-side code anyway).

Feasibility for alpha: NONE for volunteer nodes. Roadmap option: a TEE tier — if datacenter H100/Blackwell operators ever join, attest them and route confidentiality-sensitive or high-value jobs there; also usable for the tracker's own reference validator.

---

2. The non-determinism problem for output comparison

2.1 Why honest nodes disagree

• Floating-point non-associativity is the root cause, but the mechanism that surfaces it is batch-variant kernels: inference servers pick different kernels/split strategies depending on batch size and load, changing reduction order and hence results — the same request returns different logits depending on what else is in the batch (Thinking Machines: Defeating Nondeterminism in LLM Inference).
• Even greedy (temperature-0) decoding diverges across configurations: changing batch size, GPU count, GPU architecture, or tensor-parallel size measurably changes outputs under greedy decoding, and small numerical differences get amplified autoregressively — one flipped token early in a long generation cascades into a completely different continuation (arXiv:2506.09501; arXiv:2511.17826).
• Temperature > 0 sampling adds outright randomness on top unless seeds and the sampling algorithm (e.g. Gumbel-max in vLLM) are synchronized.

Consequence: naive "re-run the prompt, compare the text" audits will flag honest heterogeneous nodes as cheaters. Never compare free-running generations token-for-token across different hardware.

2.2 Techniques that make honest-vs-honest comparison robust

1. Teacher-forced re-verification (per-position logit/activation check) — yes, this is a recognized, state-of-the-art technique.
   • TOPLOC (Prime Intellect, ICML 2025): the prover commits top-k values/indices of the last hidden state per decode chunk (258 bytes per 32 tokens, ~1000× smaller than raw activations); the validator re-runs the claimed token sequence as a single prefill (teacher forcing) and checks the committed top-k against recomputed values using exponent/mantissa error thresholds. Reported: 100% detection of model, prompt, or precision substitutions with zero false positives across different GPUs, tensor-parallel layouts, and attention implementations, and validation up to 100× faster than the original inference because prefill is compute-bound while decode is memory-bound (TOPLOC, arXiv:2501.16007; GitHub).
   • Token-DiFR / Activation-DiFR (2025): synchronize the sampling seed (Gumbel-max), then compare the provider's tokens against a reference re-generation — with a shared seed, >98% of tokens match exactly for honest providers, so token divergence itself becomes the audit signal; detects 4-bit quantization with AUC > 0.999 within 300 tokens. Activation-DiFR compresses activations with random orthogonal projections and detects the same with 2 tokens. Released as a vLLM integration (DiFR, arXiv:2511.20621; author explainer).
   • Log-probability auditing: comparing per-token logprob distributions against a reference model detects even single-step fine-tuning or quantization changes; simple statistical tests (permutation test on per-token mean logprobs, KS tests) overcome logprob noise (Logprob Tracking, arXiv:2512.03816; model-substitution audit, arXiv:2504.04715).
2. Tolerance-based matching, not bitwise equality. TOPLOC accepts bounded exponent-intersection and mantissa-error deviations; DiFR scores divergence-from-reference against an honest-noise baseline. Both are explicitly designed so cross-GPU numerical noise passes while quantization/model swaps fail.
3. Beware scalar aggregate thresholds alone. The DiFR authors note that a single statistic like mean cross-entropy can be gamed — a malicious provider can tune its sampling temperature until the aggregate matches expectation. Seed-synchronized token matching or per-position top-k activation checks leave far fewer degrees of freedom (DiFR explainer).
4. Bitwise-deterministic kernels (Gensyn RepOps; Thinking Machines' batch-invariant kernels; TBIK for cross-tensor-parallel invariance) make exact comparison possible but require replacing the whole operator stack and sacrifice performance — practical for a controlled verification environment, unrealistic to impose on volunteer nodes running stock llama.cpp/transformers (Gensyn; Thinking Machines; arXiv:2511.17826).

Practical recipe for our audits: record prompt, claimed output tokens, sampling params, and per-hop activation fingerprints. Audit = one teacher-forced forward pass of the claimed tokens on a reference node; compare per-position: (a) hop-boundary fingerprints TOPLOC-style, and/or (b) whether each claimed token is plausible under the reference distribution (rank/logprob within tolerance; exact match if seeds are synchronized). This is cheap (single prefill), robust to hardware noise, and immune to autoregressive divergence because the token sequence is fixed.

---

3. Layer-skipping detectability

How gracefully do transformers degrade when layers are dropped?

• Careful pruning degrades subtly. ShortGPT removes ~25% of layers (chosen by Block Influence, training-free) and keeps ~90% of benchmark performance — e.g. LLaMA2-13B MMLU drops only 55.0→52.2 after removing 10 of 40 layers (ShortGPT, arXiv:2403.03853). Gromov et al. find a "characteristic flat region of robust performance" on QA benchmarks followed by a sharp collapse to random accuracy at a model-dependent threshold: ~45–55% of layers for Llama-2 family, ~35% for Mistral-7B, ~25% for Phi-2, ~20% for Qwen — and up to ~half of Llama-2-70B's layers with healing finetuning (The Unreasonable Ineffectiveness of the Deeper Layers, arXiv:2403.17887). LayerDrop showed a decade of precedent: networks trained with layer dropout can be pruned at inference with modest loss (arXiv:1909.11556).
• So: would skipping ~36 of 100 layers produce gibberish? Probably not gibberish. For a Llama-like model, 36% removal of well-chosen layers sits below the collapse threshold — output would be degraded but often fluent, and on easy prompts might look acceptable. A lazy cheater skipping a contiguous middle block without choosing layers carefully does worse — Gromov et al. show the naive heuristic's loss "begins to increase very rapidly even with small amounts of pruning" before healing — but "worse" still means plausible-looking text on many prompts, not obvious garbage. Human-eyeball or coarse quality checks are not a reliable fraud detector.
• Statistical detection: yes, and easily. Without healing, C4 validation loss rises sharply with pruning fraction (Gromov et al. Fig. 3), so perplexity of the node's outputs under a reference model shifts detectably given enough tokens. But the far stronger signal is direct: skipping any layer changes the hidden states and output logits at every position by amounts vastly exceeding floating-point noise. TOPLOC's evaluation explicitly includes detecting modified models with 100% accuracy; DiFR detects even 4-bit quantization (a much subtler perturbation than deleting layers) with AUC > 0.999 in ≤300 tokens. A single teacher-forced audit pass catches layer-skipping essentially deterministically.
• Pipeline nuance: in multi-hop inference, a node that skips layers inside its assigned range corrupts its output boundary activations; the hop-boundary fingerprint comparison in §1.2 localizes exactly which node did it.

---

4. Why classic PoW does not solve verifiable useful computation

• PoW's defining property is verification asymmetry over an arbitrary puzzle: work is hard to produce, trivially cheap to check (hash preimage with leading zeros), and the puzzle content is irrelevant — it exists purely to price Sybil identities and block production (Wikipedia: Proof of work). Useful computation generally lacks this asymmetry: checking an ML result naively requires re-executing it, which is the whole problem. This is the central, repeatedly confirmed obstacle in the Proof-of-Useful-Work literature — "the main weakness that all presented PoUW approaches have in common is the verification of results" (Challenges of PoUW, arXiv:2209.03865; SoK: Is Proof-of-Useful-Work Really Useful?, IACR 2025/1814). Expensive verification then triggers the verifier's dilemma: rational verifiers skip checking, and security collapses (Truebit whitepaper §1).
• Live case study: io.net's hourly "Proof-of-Work" makes GPUs solve hash puzzles and proves the hardware exists, has the claimed VRAM, and is online (io.net PoW docs). It was bolted on after the April 2024 Sybil attack (~1.8M fake GPUs spoofed via a shared auth token to farm airdrops) and says nothing about whether any customer job was computed correctly (io.net 2026 analysis). PoW = admission control and capacity attestation; correctness needs a separate mechanism.
• Self-computed checksums/commitments: confirmed — no standalone correctness guarantee. A hash or Merkle commitment over activations is binding (the node can't later change its story) and possibly hiding, but the committed values are whatever the node computed — a malicious node honestly commits to its dishonest activations, and the checksum verifies perfectly. Correctness only enters when an independent party recomputes and compares against the commitment. That is precisely how the serious systems use commitments: Truebit's Merkle state roots and Verde's checkpoint hashes exist to make dispute bisection possible and to pin claims before an unpredictable audit (so a node can't retroactively fix its answer once it learns it's being checked), never as proof by themselves (Truebit; Verde, arXiv:2502.19405; TOPLOC commitments are verified by validator recomputation — arXiv:2501.16007).

---

5. What comparable networks actually do

Network	Verification of outputs	Sybil / fraud handling	Notes
Prime Intellect	TOPLOC activation commitments per sequence; validators teacher-force-recompute (sampled, up to 100× faster than generation); plus sampling-sanity and data-sanity checks	Invalid batches → node slashed and evicted from compute pool; hardware checks at registration	Deployed at scale for INTELLECT-2 (32B RL run over permissionless nodes) (arXiv:2505.07291)
Gensyn	Verde refereed delegation: bisect disputes to a single operator; referee recomputes it; requires RepOps bitwise-reproducible kernels	Correct result guaranteed if ≥1 of the assigned providers is honest; economic penalties on losers of disputes	The most rigorous trustless design; reproducibility stack is the price (Verde; docs)
Bittensor	No direct output verification: subnet validators score miners; Yuma Consensus aggregates stake-weighted scores into emissions	Registration costs + stake; the notorious weight-copying problem (validators free-riding by copying consensus weights) countered by Commit-Reveal v3: timelock-encrypted weights revealed epochs later so copiers only get stale data (weight copying; commit-reveal)	Verification quality is per-subnet and heuristic; consensus rewards conformity, not proven correctness
io.net	None for job outputs	Hourly hash-puzzle PoW + VRAM check + Proof of Time-Lock (uptime), added after the Apr 2024 fake-GPU Sybil attack; failed checks → excluded from rewards/hiring (PoW docs; Messari overview)	Marketplace model: hardware attestation only, correctness left to the customer
Ritual (Infernet)	Modular: nodes may attach ZK, optimistic, or TEE proofs; consumer contracts opt in (Ritual blog)	ChainLight's security review: base Infernet has no aggregation of multiple nodes, no reputation, no penalty for malicious nodes returning wrong results (ChainLight)	"Verification-optional" in practice
Hyperbolic	PoSP/spML: random spot-check by a second node; disagreement → arbitration; slashing of the dishonest party (PoSP, arXiv:2405.00295)	Nash-equilibrium argument that rational nodes stay honest; validator anonymity until finalization to resist collusion	Closest published formalization of our intended scheme
Akash	None (general compute marketplace)	Auditor-attested provider tiers: governance-approved auditors post on-chain attestations; no provider staking/slashing — ChainLight flags malicious-provider risk as an accepted gap (ChainLight; Akash docs)	Reputation = audit badge + market history
Petals (closest architectural cousin: volunteer, pipeline-parallel, consumer GPUs)	None — the paper and docs openly state malicious peers can alter outputs; mitigation is "use a private swarm of people you trust"; devs planned a centralized points system for nodes that "consistently return correct results" (GitHub; dev comments on HN)	Proof that the architecture works, and that the fraud layer is the missing piece we're building
Together AI	Centralized provider; no public trustless verification protocol — trust is contractual/reputational	n/a	Included for contrast: the "trusted provider" baseline
Atoma / Ambient (emerging)	Atoma: TEE where available, else user-configurable redundant sampling; Ambient: "Proof of Logits" fingerprinting with ~0.1% claimed overhead but no open implementation (Equilibrium survey)	—	Signal of where the field is converging: logit fingerprints + sampling

Pattern: nobody in production uses zkML for LLM serving; nobody offers TEE on consumer GPUs; every serious inference-verification deployment converges on commit → sampled teacher-forced recompute → slash/reputation, with refereed delegation as the trustless escalation path.

---

6. Reputation systems for routing and audit-rate weighting

Foundations. EigenTrust computes global trust as the principal eigenvector of normalized local trust ratings and remains the canonical design; in simulation it suppresses malicious collectives up to large fractions of the network, but it depends on pre-trusted peers and majority-honest aggregation (EigenTrust, WWW'03).

Documented pitfalls (from EigenTrust's own threat models and the P2P reputation-attack literature — survey):

• Sybil flooding / whitewashing: free identities let attackers dominate the "unknown newcomer" pool and shed bad reputations by rejoining. Defense: entry cost (stake, invite, proof-of-hardware), newcomer probation with low routing weight and elevated audit rates.
• Reputation farming / milking: behave honestly on cheap jobs to build score, then cash in on expensive ones ("moles" that interleave honest work are documented in real systems like Maze). Defense: value-weighted reputation (a job's reputation contribution ∝ its audited value), and audit probability that never floors at zero for anyone.
• Collusion rings: peers mutually inflating ratings. Defense: in our alpha, reputation should derive only from tracker-verified audit outcomes, never from peer ratings — this deletes the collusion surface EigenTrust suffers from. (EigenTrust++ hardens propagation with feedback-similarity for when peer ratings do get introduced — EigenTrust++.)
• Weight-copying (Bittensor's lesson): any published evaluation signal will be free-ridden by lazy evaluators; the fix is delaying/encrypting evaluations (commit-reveal timelock) until they're stale (Bittensor docs). Relevant to us only when third-party validators exist; a single tracker has no one to copy from.

Reputation-weighted routing + audit rates in practice. io.net gates hiring and block rewards on rolling verification status; Akash routes via audit-tier filters; Bittensor routes emissions via consensus-weighted scores. The clean design for us, consistent with PoSP-style deterrence:

• Routing weight increases with reputation (and observed TPS, already tracked), so honest nodes earn more — making L (loss-on-catch, §1.1) grow with tenure.
• Audit rate decreases with reputation but has a hard floor (e.g. new node 20–30% → veteran 2–3%, floor ≥ 2%), keeping the overall budget ≈ 5% while making cheating-as-a-newcomer maximally risky and reputation-then-defect still irrational.
• Asymmetric dynamics: slow to build, instant to lose (existing 3-strike forfeiture design already matches this), plus decay on inactivity so parked identities can't hoard trust.
• Unpredictability is load-bearing: audit selection must be indistinguishable from normal traffic at request time (same request format, decided by tracker RNG/VRF after commitment), or nodes cheat only on non-audited requests — the exact property both PoSP and INTELLECT-2 rely on.

---

7. Feasibility table for the neuron-tai alpha

Alpha constraints: volunteer 8 GB consumer GPUs, single trusted tracker, mock-USDT but real carried-forward reputation, ~5% audit budget, small models today (0.5B-class) scaling up.

Technique	Alpha feasibility	Overhead	What it buys	Verdict
Optimistic accept + random teacher-forced audit (~5%)	High — tracker (or a trusted validator node) re-runs claimed tokens as one prefill	~5% of serving compute; audits themselves up to ~100× cheaper than original generation	Detects wrong model, quantization, layer-skipping, garbage outputs; rational-cheating deterrence at L > 19× per-job gain	Do now (anchor)
Per-hop activation fingerprints (TOPLOC-style top-k, tolerance matching)	High — few hundred bytes per response per hop; nodes already stream activations	Negligible at inference; storage ~258 B/32 tokens/hop	Pins claims before audit; localizes blame to the exact hop on audit failure	Do now
Hop-boundary replay "bisection" (referee = tracker, non-interactive)	Medium-high — needs a reference node able to run the full model or per-hop ranges	Only on audit failure / dispute	Culprit identification in multi-hop pipelines; fair slashing	Do now (simplified form)
Seed-synchronized sampling (Token-DiFR-style exact-token audits)	Medium — requires controlling the sampler (Gumbel-max w/ shared seed) in node runtime	Zero at serve time	Strongest per-token evidence; >98% exact match for honest nodes	Do if node runtime is ours; else logprob-rank tolerance
Statistical logprob/perplexity monitoring (passive, on all traffic)	High — reference-free heuristics (output perplexity, repetition, truncation) + periodic logprob tests	Tiny	Cheap tripwire that raises a node's audit rate when outputs look off	Do now (cheap complement)
Reputation-weighted routing + reputation-weighted audit rate with floor	High — extends existing tracker reputation/strikes	None	Makes deterrence math work; concentrates audits on new/suspect nodes	Do now
Full interactive verification game (Truebit/Verde) with bitwise RepOps kernels	Low — requires replacing node operator stacks, decentralized referees	High engineering; runtime penalty for reproducible kernels	Trustless dispute resolution without a trusted tracker	Roadmap (multi-tracker era)
zkML proofs of inference	None — GPT-2-scale, tens of seconds per proof, server-class hardware (zkGPT; NanoZK)	100–10,000×	Cryptographic soundness	Roadmap-only; re-evaluate yearly
GPU TEE attestation	None on consumer cards (H100/Blackwell + SEV-SNP/TDX servers only — NVIDIA guide)	~2–10% where available	Hardware-rooted "right model+code ran"	Roadmap: optional attested tier for datacenter contributors
PoW-style hardware challenges (io.net-like)	Medium — trivial to add	Idle-time only	Sybil/VRAM/capacity attestation at registration; NOT output correctness	Optional, registration-time only; don't confuse with fraud layer

---

8. Recommended layered scheme for alpha

1. Commit layer (on-demand, audit-selected): when a request is selected for audit, each hop signs and reports a TOPLOC-style top-k fingerprint of its output boundary activations plus the sampling metadata (params, seed if controlled). Client-visible response carries the full claimed token sequence. Commitments are cheap and make retroactive lying impossible — they are audit pins, not proofs (§4).¹

2. Audit layer (~5% of requests, VRF/tracker-RNG selected, indistinguishable ex ante): a reference executor teacher-forces the claimed tokens in one prefill and checks (a) per-position token plausibility under the reference distribution (exact match if seed-synced; logprob-rank tolerance otherwise) and (b) hop-boundary fingerprints within TOPLOC-style exponent/mantissa tolerances. New/low-reputation nodes get 20–30% audit rates, veterans 2–3%, floor ≥ 2%, budget-balanced to ≈5% overall.
3. Blame layer (on audit failure): replay comparison across hop cut-points identifies the first divergent hop; that node eats the penalty (forfeit pending balance + strike, per the existing forfeiture design), and its audit rate snaps to maximum. Honest-noise false positives are handled by tolerance calibration against an honest-node baseline corpus (collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds).
4. Reputation layer: reputation derives exclusively from audit outcomes + uptime/latency, never peer ratings (no collusion surface). It weights routing (earnings) and inversely weights audit probability. Slow build, instant loss, decay on inactivity, entry probation for Sybil/whitewash resistance.
5. Tripwire layer (all traffic, passive): perplexity/repetition/truncation anomaly scoring on outputs; anomalies don't punish directly, they escalate the node's audit rate.

This is, deliberately, the Prime Intellect / Hyperbolic pattern adapted to multi-hop pipelines with the tracker as referee — every component has a deployed or peer-reviewed precedent.

9. Explicitly NOT feasible for alpha (roadmap-only)

• zkML proofs of LLM inference — 3+ orders of magnitude too slow for multi-billion-parameter serving; provers won't run on 8 GB cards. Revisit if layerwise/GKR provers (NanoZK, zkGPT lineage) reach real-time on 7B+ models.
• GPU TEE / confidential-computing attestation on volunteer nodes — hardware doesn't exist on consumer GPUs. Keep as an attested datacenter tier concept.
• Fully trustless interactive verification games (Truebit/Verde with on-chain referees and bitwise-reproducible kernels) — pointless while a single trusted tracker exists; becomes the correct escalation path when trackers decentralize. Requires shipping a reproducible-operator runtime to nodes.
• Decentralized third-party verifier markets (and their incentive pathologies: verifier's dilemma, forced-error jackpots, Truebit's 500–5000% verification tax) — single-tracker alpha sidesteps all of it; re-enters with multi-tracker.
• Peer-rating reputation (EigenTrust-style transitive trust) — collusion/Sybil surface with no benefit while the tracker is the sole verifier.
• Commit-reveal timelock machinery for evaluator honesty (Bittensor CRV3) — solves weight-copying among many validators; irrelevant until independent validators exist.
• PoW puzzles as a correctness mechanism — category error (§4); acceptable only as optional registration-time hardware attestation.

---

Sources

Papers / peer-reviewed:

• Truebit: A scalable verification solution for blockchains (Teutsch & Reitwießner)
• Gensyn Verde: arXiv:2502.19405; Gensyn blog; Gensyn docs
• TOPLOC: arXiv:2501.16007 (ICML 2025); GitHub
• INTELLECT-2: arXiv:2505.07291
• Proof of Sampling (Hyperbolic): arXiv:2405.00295; spML blog
• DiFR (Token/Activation): arXiv:2511.20621; explainer
• Logprob tracking: arXiv:2512.03816; Model-substitution auditing: arXiv:2504.04715
• VeriLLM: arXiv:2509.24257; Optimistic TEE-Rollups: arXiv:2512.20176
• zkGPT: USENIX Security 2025; NanoZK: arXiv:2603.18046; Modulus "Cost of Intelligence": IACR 2026/1063
• Nondeterminism: Thinking Machines blog; arXiv:2506.09501; arXiv:2511.17826; arXiv:2511.00025
• Layer pruning: arXiv:2403.17887 (Gromov et al., ICLR 2025); ShortGPT, arXiv:2403.03853 (ACL 2025); LayerDrop: arXiv:1909.11556
• PoUW: SoK, IACR 2025/1814; arXiv:2209.03865; Wikipedia: Proof of work
• EigenTrust: WWW'03; EigenTrust++; P2P reputation attack survey
• Petals: GitHub; petals.dev; dev statements on incentives/correctness, HN

Official docs / engineering blogs:

• NVIDIA CC: H100 CC blog; deployment guide (TDX/SNP); CACM article; Phala GPU-TEE on OpenRouter
• Bittensor: weight copying; commit reveal; OTF blog
• io.net: PoW docs; Messari overview; 2026 risk analysis incl. Sybil attack history
• Ritual: ritual.net blog; ChainLight security review of AI-chain projects (Ritual, Akash)
• Akash: provider docs
• Landscape survey: Equilibrium Labs — State of Verifiable Inference
• Modulus Labs acquisition: world.org announcement

---

Open-source reusability & build-vs-adopt

Added 2026-07-04. Focused follow-up: which of the projects above ship code we can actually reuse, and is verified decentralized inference a "solved problem"? All repo facts below checked against the GitHub repos/APIs on 2026-07-04.

TOPLOC (Prime Intellect) — a real, adoptable library

• Repo & license: PrimeIntellect-ai/toploc, MIT (both the repo license and the pyproject.toml declaration). Published on PyPI as pip install toploc. Separate toploc-experiments repo holds the paper's vLLM-integrated experiment code.
• Maintenance: actively maintained — created 2025-01-28, last push 2026-07-02 (two days before this writing), 56 stars / 12 forks / 12 open issues. Small single-purpose codebase (Python + a compiled extension; ships wheels, depends only on torch/numpy).
• What it actually is: a library, not a framework. Two function families (README):
  • build_proofs_bytes/base64(activations, decode_batching_size, topk, skip_prefill) — prover side: takes a list of activation tensors (prefill tensor + one last-hidden-state tensor per generated token, bf16 in the examples) and returns compact proofs (~10 bytes each; 258 bytes per 32 tokens per the paper).
  • verify_proofs_bytes/base64(recomputed_activations, proofs, ...) — verifier side: returns per-chunk VerificationResult(exp_intersections, mant_err_mean, mant_err_median); the accept/reject thresholds are ours to set and calibrate (the paper gives reference thresholds validated across GPU types).
• Assumptions: model- and runtime-agnostic — it operates on tensors you extract yourself. It does not hook into vLLM/transformers for you; capturing the last hidden state per decode step in the node runtime, and recomputing activations via teacher-forced prefill on the validator, is our integration work. Detects precision changes by design, so we must pin one canonical precision/quantization per served model or honest quantized volunteers will fail verification.
• Integration estimate for our pipeline-parallel case: modest. (1) Node runtime: capture final-hidden-state per token, call build_proofs_*, attach to response — days, not weeks. (2) Tracker/validator: teacher-forced prefill re-run + verify_proofs_* — days. (3) The genuinely new part is per-hop commitments: TOPLOC as published commits only the final hidden state (whole-pipeline check, catches that fraud happened); applying the same encoding to hop-boundary activations for per-node blame is our own straightforward extension since the functions are tensor-generic, but its thresholds across heterogeneous 8 GB cards must be calibrated by us — that empirical honest-noise calibration is the real work. (4) Threshold calibration corpus across the volunteer fleet — the long pole, but required for any tolerance-based scheme regardless of library.

Gensyn — papers and proprietary binaries, not a reusable verification library

What is actually public in github.com/gensyn-ai (org listing checked 2026-07-04):

• rl-swarm — MIT, 1.7k stars: a framework for RL training swarms, plus rl-swarm-contracts (coordination contracts). Not inference verification; not relevant to our fraud layer.
• Verde (dispute resolution): paper only. No verde repository exists in the org. The bisection/arbitration protocol from arXiv:2502.19405 has no published implementation. If we want it, we implement it from the paper.
• RepOps / reproducible execution: shipping, but proprietary. repops-demo is demo scripts with no OSS license file (only a LICENSE-LLAMA for the model weights) driving a prebuilt Docker image; supported targets include consumer RTX 3090/4070/4090 (CC 7.5–9, CUDA 12.6+). The successor ree (Reproducible Execution Environment) is explicit in its README: the SDK wrapper is MIT, but "the REE compiler binary and the REE reproducible-operators binary … are not licensed under the MIT License" — they fall under a proprietary REE Binary License. REE does reproducible LLM inference with run/verify receipts, pipeline parallelism up to 72B (v0.2.0 notes), and needs Docker + NVIDIA driver ≥570. So bitwise cross-GPU determinism is real and demonstrably achievable on consumer GPUs — but as a closed-source appliance, not an embeddable library.
• "Judge" (verifiable AI evaluator, docs) — no public library repo found in the org.
• Assessment of the RepOps requirement: deterministic cross-GPU execution means reimplementing/fixing the reduction order of every operator (matmul, attention, norms) for every hardware target — a compiler/kernel-engineering effort Gensyn keeps proprietary, and Thinking Machines' open batch-invariant kernels (blog) only address batch-invariance on one platform, not cross-GPU bitwise equality. This is a large lift with no off-the-shelf OSS solution; it is exactly what TOPLOC-style tolerance matching lets us avoid. Adopting REE would mean forcing volunteers into Gensyn's Docker appliance (proprietary binaries, driver floor, perf overhead) — a non-starter for our alpha; a possible future option for the validator/referee environment only.

Bittensor — an incentive market, not verification code (confirmed)

• Confirmed from primary sources: the whitepaper frames Bittensor as peers ranking peers ("intelligence measured by intelligence"), and the Yuma Consensus docs describe the on-chain mechanism precisely: each subnet validator submits a weight vector ranking miners; YC resolves the stake-weighted matrix into emissions, clipping any weight above the level supported by κ (default 0.5) of stake to punish out-of-consensus/collusive over-evaluation, and paying validators via EMA bonds that reward staying near consensus. There is no cryptographic verification of any computation anywhere in the mechanism — correctness of miner outputs is whatever each subnet's own validator code decides to score. The chain (opentensor/subtensor, Rust, The Unlicense, actively developed) implements the consensus/emissions math, not inference checking.
• What subnets actually do: subnet owners write off-chain validator logic (Python, typically on the MIT-licensed bittensor SDK) that queries miners with tasks and scores responses — reference-model comparison, heuristics, or LLM-judging, entirely subnet-specific. Quality of validation is therefore uneven by construction.
• Known weaknesses & mitigations (documented by Bittensor itself): validator weight-copying (free-riding on the public weight matrix) — mitigated by Commit-Reveal v3 timelock-encrypted weights revealed epochs later (weight copying; commit reveal); lazy/colluding validators — mitigated only economically via clipping and bond penalties; and the docs concede commit-reveal fails if miner rankings are too static.
• Reusable for us: the design patterns — stake-weighted score aggregation with clipping, EMA-smoothed trust, commit-reveal for future multi-validator honesty — not code. Subtensor's Rust consensus math is chain-embedded and solves a multi-validator problem we don't have while the single tracker is the only scorer.

Other OSS in this niche

• EZKL (zkonduit/ezkl) — real, mature library (Rust with Python/JS bindings, Apache-2.0 per its npm package and README, ~1.2k stars, last push 2026-02). It proves ONNX-graph inference in ZK and genuinely works — for small models (MLPs, small CNNs/regressors; §1.3 overheads apply). Irrelevant to serving multi-billion-parameter LLMs, but adoptable later if we ever need to prove a tiny model (e.g., a routing/scoring model) on-chain.
• Petals (bigscience-workshop/petals) — MIT, 10.3k stars, but effectively dormant (last push Sep 2024) and contains zero trust machinery: its own docs say malicious peers can alter outputs and recommend private swarms (§5). Its hivemind DHT/networking stack (MIT, also mirrored by Gensyn) is reusable infrastructure, not verification. Petals is validation of our architecture and a warning, not a component.
• Hyperbolic PoSP/spML — paper + marketing only. The HyperbolicLabs GitHub org contains agent kits, MCP servers, and a k8s OS — no PoSP/spML implementation is published. Adapt the pattern from arXiv:2405.00295; there is nothing to adopt.
• Ritual Infernet — the node (ritual-net/infernet-node, Python, BSD-3-Clause-Clear per its community port; direct repo access was flaky at check time) plus infernet-deploy/infernet-sdk are open coordination/oracle plumbing (request routing, container orchestration, on-chain delivery). Proofs are a pluggable slot, not shipped verification logic, and ChainLight's review found no output aggregation, reputation, or penalties in the base system (§5). Nothing here advances our fraud layer.
• DiFR — the Token/Activation-DiFR authors state they released an open-source vLLM integration (arXiv:2511.20621); worth tracking as a second adoptable audit primitive alongside TOPLOC, especially the seed-synchronized exact-token variant.

Bottom line: is it solved?

The audit primitive is solved and adoptable; the system around it is not. No OSS project ships an end-to-end "verified decentralized inference network in a box" — every network either built theirs in-house (Prime Intellect), kept the hard part proprietary (Gensyn REE binaries), or doesn't verify at all (Petals, Akash, base Infernet, Bittensor's chain). What is genuinely reusable is small, high-quality, and MIT-licensed: TOPLOC. Everything else we need is a pattern to adapt or thin logic to build on our tracker, which is appropriately bespoke (it's our economics).

Capability	Verdict	Component / reference	Rationale
Detection / audit (teacher-forced re-verification)	ADOPT	toploc (MIT, PyPI, maintained) — track DiFR's vLLM integration as a complement	Proven encoding + verifier with cross-GPU tolerance semantics; our work is runtime hooks + threshold calibration, not algorithm development
Blame attribution / bisection (per-hop)	ADAPT + BUILD	Verde's commit-then-recompute pattern (arXiv:2502.19405); reuse TOPLOC's encoding on hop-boundary activations	No published Verde code; full bisection is overkill for a handful of hops — a single referee replay over committed hop boundaries suffices (§1.2)
Reputation / incentive	ADAPT + BUILD	PoSP deterrence math (arXiv:2405.00295); Yuma patterns (clipping, EMA trust, commit-reveal) for the future multi-validator era; EigenTrust pitfalls list (§6)	No code to adopt (PoSP unpublished; Yuma is chain-embedded Rust for a different topology); our tracker-side logic is small and economics-specific
Cryptographic proof (zkML / TEE / bitwise-reproducible execution)	NEITHER (roadmap)	EZKL (Apache-2.0) if a tiny provable model ever needs it; Gensyn REE as a proprietary-appliance option for a future referee tier; GPU TEE per §1.4	All three are unusable on 8 GB volunteer cards today (§1.3–1.4); REE's reproducible operators are closed-source binaries

Direct answer to the owner: Gensyn and Bittensor are the right things to read but the wrong things to build on — Gensyn's verification core is a paper plus proprietary binaries aimed at training-grade trustlessness we don't need under a single trusted tracker, and Bittensor ships an incentive market that deliberately contains no computation verification at all. The one shelf-ready piece for our exact audit step is TOPLOC (MIT, pip-installable, updated this week), and the recommended alpha scheme in §8 remains: adopt TOPLOC for the audit primitive, adapt PoSP/Verde patterns for deterrence and blame, and build the thin tracker-side reputation/audit-rate logic ourselves.

---

1. Alpha implements on-demand commitments per ADR-0018 §3 — not every request. Nodes retain recent activations briefly; serving path stays uncommitted until audit selection. ↩︎