94 lines
3.9 KiB
Markdown
94 lines
3.9 KiB
Markdown
# Distributed Inference Network
|
|
|
|
A volunteer GPU network where nodes independently load model shards, a tracker routes inference through optimal node chains, and contributors earn tokens for serving compute.
|
|
|
|
## Language
|
|
|
|
### Nodes & compute
|
|
|
|
**Node**:
|
|
A volunteer machine that runs the node client, holds one or more shards on disk, and serves inference requests for those shards.
|
|
_Avoid_: worker, peer, miner, server
|
|
|
|
**Shard**:
|
|
A contiguous range of transformer layers from a model that a node loads and serves. Shards are the unit of storage, assignment, and reward.
|
|
_Avoid_: partition, slice, chunk, segment
|
|
|
|
**Shard Swarm**:
|
|
The P2P group of nodes that collectively seed and download a specific shard. One swarm exists per shard.
|
|
_Avoid_: torrent, cluster, pool
|
|
|
|
**Inference Route**:
|
|
An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request.
|
|
_Avoid_: pipeline, chain, path
|
|
|
|
**Gateway**:
|
|
The network entry point that accepts client requests (OpenAI-compatible HTTP), selects an inference route from the tracker, and streams results back.
|
|
_Avoid_: proxy, relay, orchestrator, primary
|
|
|
|
### Tracker
|
|
|
|
**Tracker**:
|
|
The coordinator service that maintains the node registry, scores nodes by throughput/latency, and assigns inference routes. Runs as a centralized service with a P2P gossip fallback.
|
|
_Avoid_: coordinator, scheduler, director
|
|
|
|
**Node Score**:
|
|
A throughput/latency rating the tracker maintains per node, used for route selection. Updated continuously from inference telemetry.
|
|
_Avoid_: reputation, rating, rank
|
|
|
|
### Payments & fraud
|
|
|
|
**Stake**:
|
|
Tokens a node locks as collateral that can be slashed for fraud. Stake protects the network economically, but route selection is not based on a node's token balance.
|
|
_Avoid_: deposit, bond, escrow
|
|
|
|
**Tax**:
|
|
The share of caller payments distributed to compute nodes as rewards. Taxes are weighted by completed work and historical node speed so faster, larger nodes earn proportionally more.
|
|
_Avoid_: fee, toll, commission
|
|
|
|
**Caller Credit**:
|
|
Free starting balance granted to a new caller/API key so they can try the network before topping up.
|
|
_Avoid_: signup bonus, faucet, airdrop
|
|
|
|
**Free Compute Job**:
|
|
Work a compute node performs without earning immediate rewards, usually during probation or bootstrap phases.
|
|
_Avoid_: unpaid labor, warmup request
|
|
|
|
**Slash**:
|
|
The act of reducing a node's stake as a penalty for a proven fraud incident.
|
|
_Avoid_: penalize, burn, fine
|
|
|
|
**Strike**:
|
|
A fraud incident recorded on-chain against a node. Enough strikes result in a ban.
|
|
_Avoid_: infraction, violation, flag
|
|
|
|
**Ban**:
|
|
Permanent exclusion of a wallet from the network after exceeding the strike threshold. Recorded on-chain.
|
|
_Avoid_: blacklist, block, suspension
|
|
|
|
**Probationary Period**:
|
|
The first N jobs a new wallet must complete without earning, to raise the cost of re-entering after a ban.
|
|
_Avoid_: trial period, warmup, grace period
|
|
|
|
**Token**:
|
|
Our native Solana L2 token. Used by nodes for staking and received as inference rewards. Clients never need to hold it.
|
|
_Avoid_: coin, reward token, native token
|
|
|
|
**Contract Boundary**:
|
|
The Python interface in `packages/contracts` that represents registry, payment, and settlement behavior. During the prototype it is implemented by deterministic local wrappers; later the same boundary is backed by real Solana programs.
|
|
_Avoid_: mock contract, fake chain, temporary hack
|
|
|
|
**Validator**:
|
|
A trusted node (or the tracker itself) that re-runs a sample of inference requests to detect fraud.
|
|
_Avoid_: auditor, checker, referee
|
|
|
|
### Client-facing
|
|
|
|
**Client**:
|
|
Any application or user that sends inference requests to the gateway. Pays in SOL or USDC.
|
|
_Avoid_: user, caller, consumer
|
|
|
|
**Model Preset**:
|
|
A named, versioned model available on the network (e.g. `llama-3-70b`). The tracker knows which nodes hold which shards for each preset.
|
|
_Avoid_: model, checkpoint, version
|