2.9 KiB
Distributed Inference Network
A volunteer GPU network where nodes independently load model shards, a tracker routes inference through optimal node chains, and contributors earn tokens for serving compute.
Language
Nodes & compute
Node: A volunteer machine that runs the node client, holds one or more shards on disk, and serves inference requests for those shards. Avoid: worker, peer, miner, server
Shard: A contiguous range of transformer layers from a model that a node loads and serves. Shards are the unit of storage, assignment, and reward. Avoid: partition, slice, chunk, segment
Shard Swarm: The P2P group of nodes that collectively seed and download a specific shard. One swarm exists per shard. Avoid: torrent, cluster, pool
Inference Route: An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request. Avoid: pipeline, chain, path
Gateway: The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results back. Avoid: proxy, relay, orchestrator, primary
Tracker
Tracker: The coordinator service that maintains the node registry, scores nodes by throughput/latency, and assigns inference routes. Runs as a centralized service with a P2P gossip fallback. Avoid: coordinator, scheduler, director
Node Score: A throughput/latency rating the tracker maintains per node, used for route selection. Updated continuously from inference telemetry. Avoid: reputation, rating, rank
Payments & fraud
Stake: Tokens a node locks as collateral before it can serve paid inference. Slashed on fraud. Avoid: deposit, bond, escrow
Slash: The act of reducing a node's stake as a penalty for a proven fraud incident. Avoid: penalize, burn, fine
Strike: A fraud incident recorded on-chain against a node. Enough strikes result in a ban. Avoid: infraction, violation, flag
Ban: Permanent exclusion of a wallet from the network after exceeding the strike threshold. Recorded on-chain. Avoid: blacklist, block, suspension
Probationary Period: The first N jobs a new wallet must complete without earning, to raise the cost of re-entering after a ban. Avoid: trial period, warmup, grace period
Token: Our native Solana L2 token. Used by nodes for staking and received as inference rewards. Clients never need to hold it. Avoid: coin, reward token, native token
Validator: A trusted node (or the tracker itself) that re-runs a sample of inference requests to detect fraud. Avoid: auditor, checker, referee
Client-facing
Client: Any application or user that sends inference requests to the gateway. Pays in SOL or USDC. Avoid: user, caller, consumer
Model Preset:
A named, versioned model available on the network (e.g. llama-3-70b). The tracker knows which nodes hold which shards for each preset.
Avoid: model, checkpoint, version