neuron-tai/CONTEXT.md

# Distributed Inference Network

A volunteer GPU network where nodes independently load model shards, a tracker routes inference through optimal node chains, and contributors earn tokens for serving compute.

## Language

### Nodes & compute

**Node**:
A volunteer machine that runs the node client, holds one or more shards on disk, and serves inference requests for those shards.
_Avoid_: worker, peer, miner, server

**Shard**:
A contiguous range of transformer layers from a model that a node loads and serves. Shards are the unit of storage, assignment, and reward.
_Avoid_: partition, slice, chunk, segment

**Shard Swarm**:
The P2P group of nodes that collectively seed and download a specific shard. One swarm exists per shard.
_Avoid_: torrent, cluster, pool

**Inference Route**:
An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request.
_Avoid_: pipeline, chain, path

**Gateway**:
The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results back.
_Avoid_: proxy, relay, orchestrator, primary

### Tracker

**Tracker**:
The coordinator service that maintains the node registry, scores nodes by throughput/latency, and assigns inference routes. Runs as a centralized service with a P2P gossip fallback.
_Avoid_: coordinator, scheduler, director

**Tracker Node**:
A node that serves at least the first-layer shard (`layers[0..k]`) for a model and acts as the inference entry point for that model. Tracker nodes own the tokenizer and `embed_tokens`, receive client requests directly, select the onward route from the coverage map, and stream results back. Any node advertising a new model to the network becomes its tracker node.
_Avoid_: primary node, master node, gateway node

**Coverage Map**:
The tracker's per-model mapping of layer ranges to node counts: `[(start_layer, end_layer, node_count), ...]`. A layer range with `node_count=0` is a coverage gap — the model is unroutable until the gap is filled. Coverage-first bin-packing fills all gaps before adding redundancy.
_Avoid_: shard map, assignment table, coverage report

**Rebalance Directive**:
A `LOAD_SHARD` or `DROP_SHARD` instruction the tracker issues to a node when the coverage map changes (node joins, node leaves, or load-balance reoptimization). Delivered as part of the node's heartbeat response.
_Avoid_: rebalance command, shard instruction, migration order

**Node Score**:
A throughput/latency rating the tracker maintains per node, used for route selection. Updated continuously from inference telemetry.
_Avoid_: reputation, rating, rank

### Payments & fraud

**Stake**:
Tokens a node locks as collateral that can be slashed for fraud. Stake protects the network economically, but route selection is not based on a node's token balance.
_Avoid_: deposit, bond, escrow

**Tax**:
The share of caller payments distributed to compute nodes as rewards. Taxes are weighted by completed work and historical node speed so faster, larger nodes earn proportionally more.
_Avoid_: fee, toll, commission

**Caller Credit**:
Free starting balance granted to a new caller/API key so they can try the network before topping up.
_Avoid_: signup bonus, faucet, airdrop

**Free Compute Job**:
Work a compute node performs without earning immediate rewards, usually during probation or bootstrap phases.
_Avoid_: unpaid labor, warmup request

**Slash**:
The act of reducing a node's stake as a penalty for a proven fraud incident.
_Avoid_: penalize, burn, fine

**Strike**:
A fraud incident recorded on-chain against a node. Enough strikes result in a ban.
_Avoid_: infraction, violation, flag

**Ban**:
Permanent exclusion of a wallet from the network after exceeding the strike threshold. Recorded on-chain.
_Avoid_: blacklist, block, suspension

**Probationary Period**:
The first N jobs a new wallet must complete without earning, to raise the cost of re-entering after a ban.
_Avoid_: trial period, warmup, grace period

**Token**:
Our native Solana L2 token. Used by nodes for staking and received as inference rewards. Clients never need to hold it.
_Avoid_: coin, reward token, native token

**Contract Boundary**:
The Python interface in `packages/contracts` that represents registry, payment, and settlement behavior. During the prototype it is implemented by deterministic local wrappers; later the same boundary is backed by real Solana programs.
_Avoid_: mock contract, fake chain, temporary hack

**Validator**:
A trusted node (or the tracker itself) that re-runs a sample of inference requests to detect fraud.
_Avoid_: auditor, checker, referee

**Validation Event**:
A completed inference record that contains enough information for a validator to decide whether to sample and re-run the request: session id, model preset, messages, inference route, node wallets, and observed output.
_Avoid_: audit log, trace, receipt

**Slash Proof**:
The record submitted by a validator when a sampled re-run diverges from the observed output beyond tolerance. In the prototype this is deterministic local contract state; later it maps to an on-chain proof transaction.
_Avoid_: accusation, report, claim

### Client-facing

**Client**:
Any application or user that sends inference requests to the gateway. Pays in SOL or USDC.
_Avoid_: user, caller, consumer

**Model Preset**:
A named, versioned model available on the network (e.g. `llama-3-70b`). The tracker knows which nodes hold which shards for each preset.
_Avoid_: model, checkpoint, version