popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

documentation revision

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-13 18:14:21 +02:00
parent 180a7674e6
commit 95245be512
43 changed files with 78 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
docs/PRD.md
View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done (base program US-001…US-035 complete; see `docs/prd.json`. Post-035 work lives in `docs/issues/36+` and `.scratch/`. Payment/settlement superseded by ADR-0015; fraud by ADR-0018.)
# Distributed Inference Network — PRD
@@ -8,7 +8,7 @@ Running large language models requires expensive dedicated hardware that most pe
## Solution
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in SOL or USDC; node operators earn our native token. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
## User Stories
@@ -76,8 +76,8 @@ The codebase is organized as a Python monorepo with the following top-level pack
- `packages/contracts` — Solana L2 smart contracts (stake, slash, strike, ban, settlement)
- `packages/p2p` — P2P gossip layer and shard swarm seeding
### Inference engine (ADR-0001)
PyTorch with a Petals-style shard pipeline. Each node independently loads its assigned shard from local disk. At inference time, only activation tensors (~8 KB per layer boundary per token) travel between nodes — no model weights cross the network during serving.
### Inference engine (ADR-0001; native GGUF path ADR-0024)
PyTorch with a Petals-style shard pipeline remains the current production backend. A benchmark-gated llama.cpp/GGUF native path is planned in ADR-0024. Each node independently loads its assigned shard from local disk. At inference time, only activation tensors (~8 KB per layer boundary per token) travel between nodes — no model weights cross the network during serving.
### Inference route execution
The gateway receives a client request, asks the tracker for an inference route (ordered list of node endpoints covering all layers), opens a persistent TCP session to the first node in the route, streams activation tensors through each node in sequence, and returns the final logits as a streaming chat completion response.
@@ -91,11 +91,11 @@ The gateway receives a client request, asks the tracker for an inference route (
6. Register with tracker (wallet, hardware profile, shard, endpoint)
7. Begin accepting inference connections
### Payment flow
Clients pre-fund an API key with SOL/USDC. The gateway records per-request compute attribution. A settlement transaction runs on Solana L2 at the end of each epoch: client balance is debited, node operators receive our native token proportional to layers served, validators receive a reward share. Solana contracts are the authoritative source for all stake, slash, strike, and ban state (ADR-0002).
### Payment flow (ADR-0015 supersedes ADR-0002 settlement mechanics)
Clients pre-fund an API key with USDT. The tracker meters each request against the off-chain ledger. Periodic settlement batches USDT payouts from the custodial treasury to node operators proportional to work units. Fraud penalties forfeit pending balance (ADR-0018); strike/ban state persists in the tracker registry. TAI token emission remains deferred (ADR-0002 roadmap).
### Fraud detection (ADR-0003)
Validators re-run ~5% of completed requests. If a node's output diverges beyond floating-point tolerance from the reference, the validator submits a slash transaction on-chain. Strike count increments. At the configured strike threshold, the wallet is banned on-chain. New wallets complete N unpaid jobs before earning begins.
### Fraud detection (ADR-0018; historical ADR-0003)
Validators re-run ~5% of completed requests with TOPLOC activation verification. Caught cheaters forfeit pending balance and receive strikes; three strikes bans the wallet. Probation (first N unpaid jobs) remains the anti-sybil re-entry cost.
### Tracker architecture (ADR-0004)
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. Solana is the authoritative source of truth for all incentive-relevant state.