Normalize line endings to LF via .gitattributes
Adds a committed .gitattributes so Windows and Linux checkouts converge on LF for all text files, overriding each developer's local core.autocrlf. Renormalizes existing blobs (server.py, dashboard.html, etc.) that had CRLF baked in, clearing the repo-wide phantom "modified" churn. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dobromir Popov
258
CONTEXT.md
258
CONTEXT.md
@@ -1,23 +1,23 @@
|
||||
# Distributed Inference Network
|
||||
|
||||
A volunteer GPU network where nodes independently load model shards, a tracker routes inference through optimal node chains, and contributors earn tokens for serving compute.
|
||||
|
||||
## Language
|
||||
|
||||
### Nodes & compute
|
||||
|
||||
**Node**:
|
||||
A volunteer machine that runs the node client, holds one or more shards on disk, and serves inference requests for those shards.
|
||||
_Avoid_: worker, peer, miner, server
|
||||
|
||||
**Shard**:
|
||||
A contiguous range of transformer layers from a model that a node loads and serves. Shards are the unit of storage, assignment, and reward.
|
||||
_Avoid_: partition, slice, chunk, segment
|
||||
|
||||
**Shard Swarm**:
|
||||
The P2P group of nodes that collectively seed and download a specific shard. One swarm exists per shard.
|
||||
_Avoid_: torrent, cluster, pool
|
||||
|
||||
# Distributed Inference Network
|
||||
|
||||
A volunteer GPU network where nodes independently load model shards, a tracker routes inference through optimal node chains, and contributors earn tokens for serving compute.
|
||||
|
||||
## Language
|
||||
|
||||
### Nodes & compute
|
||||
|
||||
**Node**:
|
||||
A volunteer machine that runs the node client, holds one or more shards on disk, and serves inference requests for those shards.
|
||||
_Avoid_: worker, peer, miner, server
|
||||
|
||||
**Shard**:
|
||||
A contiguous range of transformer layers from a model that a node loads and serves. Shards are the unit of storage, assignment, and reward.
|
||||
_Avoid_: partition, slice, chunk, segment
|
||||
|
||||
**Shard Swarm**:
|
||||
The P2P group of nodes that collectively seed and download a specific shard. One swarm exists per shard.
|
||||
_Avoid_: torrent, cluster, pool
|
||||
|
||||
**Inference Route**:
|
||||
An ordered sequence of nodes whose shards together cover all layers of a model. The tracker selects the optimal route per request.
|
||||
_Avoid_: pipeline, chain, path
|
||||
@@ -55,115 +55,115 @@ Realtime progress information for an active Route Session, including phase, gene
|
||||
_Avoid_: logs, debug output
|
||||
|
||||
### Tracker
|
||||
|
||||
**Tracker**:
|
||||
The coordinator service that maintains the node registry, scores nodes by throughput/latency, and assigns inference routes. Runs as a centralized service with a P2P gossip fallback.
|
||||
_Avoid_: coordinator, scheduler, director
|
||||
|
||||
|
||||
**Tracker**:
|
||||
The coordinator service that maintains the node registry, scores nodes by throughput/latency, and assigns inference routes. Runs as a centralized service with a P2P gossip fallback.
|
||||
_Avoid_: coordinator, scheduler, director
|
||||
|
||||
**Tracker Node**:
|
||||
A node that serves at least the first-layer shard (`layers[0..k]`) for a model and acts as the inference entry point for that model. Tracker nodes own the tokenizer and `embed_tokens`, receive client requests directly, select the onward route from the coverage map, and stream results and progress when possible. Any node advertising a new model to the network becomes its tracker node.
|
||||
_Avoid_: primary node, master node, gateway node
|
||||
|
||||
**Coverage Map**:
|
||||
The tracker's per-model mapping of layer ranges to node counts: `[(start_layer, end_layer, node_count), ...]`. A layer range with `node_count=0` is a coverage gap — the model is unroutable until the gap is filled. Coverage-first bin-packing fills all gaps before adding redundancy.
|
||||
_Avoid_: shard map, assignment table, coverage report
|
||||
|
||||
**Rebalance Directive**:
|
||||
A `LOAD_SHARD` or `DROP_SHARD` instruction the tracker issues to a node when the coverage map changes (node joins, node leaves, or load-balance reoptimization). Delivered as part of the node's heartbeat response.
|
||||
_Avoid_: rebalance command, shard instruction, migration order
|
||||
|
||||
**Node Score**:
|
||||
A throughput/latency rating the tracker maintains per node, used for route selection. Updated continuously from inference telemetry.
|
||||
_Avoid_: reputation, rating, rank
|
||||
|
||||
### Payments & fraud
|
||||
|
||||
**Stake**:
|
||||
Collateral a node stands to lose for fraud. In the current design the node's Pending Balance serves as stake — no upfront deposit is required. An optional USDT/TAI deposit may return later for routing priority.
|
||||
_Avoid_: deposit, bond, escrow
|
||||
|
||||
**Treasury**:
|
||||
The single project-owned Solana wallet that custodially holds client deposits, pays node payouts, and accumulates the Protocol Cut. Its keypair is loaded only on settlement-capable trackers.
|
||||
_Avoid_: escrow, vault, hot wallet
|
||||
|
||||
**Pending Balance**:
|
||||
A node's accrued, not-yet-paid USDT earnings on the tracker ledger. Doubles as the node's fraud collateral: it is forfeited in full when a validator catches a divergent output.
|
||||
_Avoid_: unpaid rewards, accrual, balance due
|
||||
|
||||
**Settlement Period**:
|
||||
The dynamic interval driving on-chain payouts: a node is paid when its Pending Balance exceeds the Payout Threshold or the period elapses, whichever comes first. Short in development (seconds), long in production (daily), configurable to grow with volume.
|
||||
_Avoid_: epoch, payout cycle, billing cycle
|
||||
|
||||
**Payout Threshold**:
|
||||
The minimum Pending Balance that triggers an immediate payout before the Settlement Period elapses. Includes a dust floor so payouts are never smaller than they are worth.
|
||||
_Avoid_: minimum payout, dust limit
|
||||
|
||||
**Protocol Cut**:
|
||||
The 10% of inference fees retained by the project for infrastructure; the remaining 90% goes to the nodes that served the request. Accumulates in the Treasury as the future TAI liquidity reserve.
|
||||
_Avoid_: spread, commission, house fee
|
||||
|
||||
**Deposit Watcher**:
|
||||
The tracker component that observes the Treasury's on-chain USDT deposits and credits the sending client's API-key ledger balance.
|
||||
_Avoid_: payment listener, chain scanner
|
||||
|
||||
**Mock USDT**:
|
||||
The self-created 6-decimal SPL mint that stands in for USDT on devnet, where real USDT does not exist. The mint address is configuration, so mainnet cutover is a config change.
|
||||
_Avoid_: test token, fake USDT, devnet dollar
|
||||
|
||||
**Tax**:
|
||||
The share of caller payments distributed to compute nodes as rewards. Taxes are weighted by completed work and historical node speed so faster, larger nodes earn proportionally more.
|
||||
_Avoid_: fee, toll, commission
|
||||
|
||||
**Caller Credit**:
|
||||
Free starting balance granted to a new caller/API key so they can try the network before topping up.
|
||||
_Avoid_: signup bonus, faucet, airdrop
|
||||
|
||||
**Free Compute Job**:
|
||||
Work a compute node performs without earning immediate rewards, usually during probation or bootstrap phases.
|
||||
_Avoid_: unpaid labor, warmup request
|
||||
|
||||
**Slash**:
|
||||
The penalty for a proven fraud incident: the node's entire Pending Balance is forfeited to the Treasury and a Strike is recorded.
|
||||
_Avoid_: penalize, burn, fine, forfeit
|
||||
|
||||
**Strike**:
|
||||
A fraud incident recorded on-chain against a node. Enough strikes result in a ban.
|
||||
_Avoid_: infraction, violation, flag
|
||||
|
||||
**Ban**:
|
||||
Permanent exclusion of a wallet from the network after exceeding the strike threshold. Recorded on-chain.
|
||||
_Avoid_: blacklist, block, suspension
|
||||
|
||||
**Probationary Period**:
|
||||
The first N jobs a new wallet must complete without earning, to raise the cost of re-entering after a ban.
|
||||
_Avoid_: trial period, warmup, grace period
|
||||
|
||||
**Token**:
|
||||
TAI, our native Solana SPL token. Deferred (ADR-0015): nodes are currently paid directly in USDT; TAI returns as the reward/upside layer once volume exists, funded by the accumulated Protocol Cut. Clients never need to hold it.
|
||||
_Avoid_: coin, reward token, native token
|
||||
|
||||
**Contract Boundary**:
|
||||
The Python interface in `packages/contracts` that represents registry, payment, and settlement behavior. During the prototype it is implemented by deterministic local wrappers; later the same boundary is backed by real Solana programs.
|
||||
_Avoid_: mock contract, fake chain, temporary hack
|
||||
|
||||
**Validator**:
|
||||
A trusted node (or the tracker itself) that re-runs a sample of inference requests to detect fraud.
|
||||
_Avoid_: auditor, checker, referee
|
||||
|
||||
**Validation Event**:
|
||||
A completed inference record that contains enough information for a validator to decide whether to sample and re-run the request: session id, model preset, messages, inference route, node wallets, and observed output.
|
||||
_Avoid_: audit log, trace, receipt
|
||||
|
||||
**Slash Proof**:
|
||||
The record submitted by a validator when a sampled re-run diverges from the observed output beyond tolerance. In the prototype this is deterministic local contract state; later it maps to an on-chain proof transaction.
|
||||
_Avoid_: accusation, report, claim
|
||||
|
||||
### Client-facing
|
||||
|
||||
**Client**:
|
||||
Any application or user that sends inference requests to the gateway. Prepays USDT into the Treasury; each request is metered against the resulting ledger balance at a per-1K-tokens price set per model.
|
||||
_Avoid_: user, caller, consumer
|
||||
|
||||
**Model Preset**:
|
||||
A named, versioned model available on the network (e.g. `llama-3-70b`). The tracker knows which nodes hold which shards for each preset.
|
||||
_Avoid_: model, checkpoint, version
|
||||
|
||||
**Coverage Map**:
|
||||
The tracker's per-model mapping of layer ranges to node counts: `[(start_layer, end_layer, node_count), ...]`. A layer range with `node_count=0` is a coverage gap — the model is unroutable until the gap is filled. Coverage-first bin-packing fills all gaps before adding redundancy.
|
||||
_Avoid_: shard map, assignment table, coverage report
|
||||
|
||||
**Rebalance Directive**:
|
||||
A `LOAD_SHARD` or `DROP_SHARD` instruction the tracker issues to a node when the coverage map changes (node joins, node leaves, or load-balance reoptimization). Delivered as part of the node's heartbeat response.
|
||||
_Avoid_: rebalance command, shard instruction, migration order
|
||||
|
||||
**Node Score**:
|
||||
A throughput/latency rating the tracker maintains per node, used for route selection. Updated continuously from inference telemetry.
|
||||
_Avoid_: reputation, rating, rank
|
||||
|
||||
### Payments & fraud
|
||||
|
||||
**Stake**:
|
||||
Collateral a node stands to lose for fraud. In the current design the node's Pending Balance serves as stake — no upfront deposit is required. An optional USDT/TAI deposit may return later for routing priority.
|
||||
_Avoid_: deposit, bond, escrow
|
||||
|
||||
**Treasury**:
|
||||
The single project-owned Solana wallet that custodially holds client deposits, pays node payouts, and accumulates the Protocol Cut. Its keypair is loaded only on settlement-capable trackers.
|
||||
_Avoid_: escrow, vault, hot wallet
|
||||
|
||||
**Pending Balance**:
|
||||
A node's accrued, not-yet-paid USDT earnings on the tracker ledger. Doubles as the node's fraud collateral: it is forfeited in full when a validator catches a divergent output.
|
||||
_Avoid_: unpaid rewards, accrual, balance due
|
||||
|
||||
**Settlement Period**:
|
||||
The dynamic interval driving on-chain payouts: a node is paid when its Pending Balance exceeds the Payout Threshold or the period elapses, whichever comes first. Short in development (seconds), long in production (daily), configurable to grow with volume.
|
||||
_Avoid_: epoch, payout cycle, billing cycle
|
||||
|
||||
**Payout Threshold**:
|
||||
The minimum Pending Balance that triggers an immediate payout before the Settlement Period elapses. Includes a dust floor so payouts are never smaller than they are worth.
|
||||
_Avoid_: minimum payout, dust limit
|
||||
|
||||
**Protocol Cut**:
|
||||
The 10% of inference fees retained by the project for infrastructure; the remaining 90% goes to the nodes that served the request. Accumulates in the Treasury as the future TAI liquidity reserve.
|
||||
_Avoid_: spread, commission, house fee
|
||||
|
||||
**Deposit Watcher**:
|
||||
The tracker component that observes the Treasury's on-chain USDT deposits and credits the sending client's API-key ledger balance.
|
||||
_Avoid_: payment listener, chain scanner
|
||||
|
||||
**Mock USDT**:
|
||||
The self-created 6-decimal SPL mint that stands in for USDT on devnet, where real USDT does not exist. The mint address is configuration, so mainnet cutover is a config change.
|
||||
_Avoid_: test token, fake USDT, devnet dollar
|
||||
|
||||
**Tax**:
|
||||
The share of caller payments distributed to compute nodes as rewards. Taxes are weighted by completed work and historical node speed so faster, larger nodes earn proportionally more.
|
||||
_Avoid_: fee, toll, commission
|
||||
|
||||
**Caller Credit**:
|
||||
Free starting balance granted to a new caller/API key so they can try the network before topping up.
|
||||
_Avoid_: signup bonus, faucet, airdrop
|
||||
|
||||
**Free Compute Job**:
|
||||
Work a compute node performs without earning immediate rewards, usually during probation or bootstrap phases.
|
||||
_Avoid_: unpaid labor, warmup request
|
||||
|
||||
**Slash**:
|
||||
The penalty for a proven fraud incident: the node's entire Pending Balance is forfeited to the Treasury and a Strike is recorded.
|
||||
_Avoid_: penalize, burn, fine, forfeit
|
||||
|
||||
**Strike**:
|
||||
A fraud incident recorded on-chain against a node. Enough strikes result in a ban.
|
||||
_Avoid_: infraction, violation, flag
|
||||
|
||||
**Ban**:
|
||||
Permanent exclusion of a wallet from the network after exceeding the strike threshold. Recorded on-chain.
|
||||
_Avoid_: blacklist, block, suspension
|
||||
|
||||
**Probationary Period**:
|
||||
The first N jobs a new wallet must complete without earning, to raise the cost of re-entering after a ban.
|
||||
_Avoid_: trial period, warmup, grace period
|
||||
|
||||
**Token**:
|
||||
TAI, our native Solana SPL token. Deferred (ADR-0015): nodes are currently paid directly in USDT; TAI returns as the reward/upside layer once volume exists, funded by the accumulated Protocol Cut. Clients never need to hold it.
|
||||
_Avoid_: coin, reward token, native token
|
||||
|
||||
**Contract Boundary**:
|
||||
The Python interface in `packages/contracts` that represents registry, payment, and settlement behavior. During the prototype it is implemented by deterministic local wrappers; later the same boundary is backed by real Solana programs.
|
||||
_Avoid_: mock contract, fake chain, temporary hack
|
||||
|
||||
**Validator**:
|
||||
A trusted node (or the tracker itself) that re-runs a sample of inference requests to detect fraud.
|
||||
_Avoid_: auditor, checker, referee
|
||||
|
||||
**Validation Event**:
|
||||
A completed inference record that contains enough information for a validator to decide whether to sample and re-run the request: session id, model preset, messages, inference route, node wallets, and observed output.
|
||||
_Avoid_: audit log, trace, receipt
|
||||
|
||||
**Slash Proof**:
|
||||
The record submitted by a validator when a sampled re-run diverges from the observed output beyond tolerance. In the prototype this is deterministic local contract state; later it maps to an on-chain proof transaction.
|
||||
_Avoid_: accusation, report, claim
|
||||
|
||||
### Client-facing
|
||||
|
||||
**Client**:
|
||||
Any application or user that sends inference requests to the gateway. Prepays USDT into the Treasury; each request is metered against the resulting ledger balance at a per-1K-tokens price set per model.
|
||||
_Avoid_: user, caller, consumer
|
||||
|
||||
**Model Preset**:
|
||||
A named, versioned model available on the network (e.g. `llama-3-70b`). The tracker knows which nodes hold which shards for each preset.
|
||||
_Avoid_: model, checkpoint, version
|
||||
|
||||
Reference in New Issue
Block a user