popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: harden node placement and partial model loading

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-13 21:58:08 +02:00
parent a6bcc69288
commit 5d87e81bc9
21 changed files with 497 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/PRD.md
View File

@@ -1,4 +1,4 @@
Status: done (base program US-001…US-035 complete; friends-test arc US-036…US-047 in `docs/prd.json`. Payment/settlement superseded by ADR-0015; fraud by ADR-0018.)
Status: done (US-001…US-035 complete; friends-test arc US-036…US-049 in `docs/prd.json`; US-048/050 tracked. See ADRs 0015–0018, 0023, 0025–0026.)
# Distributed Inference Network — PRD
@@ -8,10 +8,12 @@ Running large language models requires expensive dedicated hardware that most pe
## Solution
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning **USDT**. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
## User Stories
> **Status (2026-07-13):** Stories below are the original product intent. **Shipped behavior** is in [Implementation Decisions](#implementation-decisions) and ADRs 0015–0018, 0023, 0025–0026. Superseded lines are marked inline.
### Node Operator
1. As a node operator, I want to install the node client with a single command (`pip install meshnet-node`), so that I can start contributing without reading documentation.
@@ -21,10 +23,10 @@ A volunteer GPU network where anyone can share their GPU by running a single com
5. As a node operator, I want my assigned shard to download automatically from HuggingFace on first start, so that I don't have to manually find or download model weights.
6. As a node operator, I want to seed my shard to other nodes via P2P once I have it, so that new nodes with the same shard assignment don't need to download from HuggingFace.
7. As a node operator, I want the node client to register with the tracker automatically and begin serving inference requests, so that I start earning as soon as setup is complete.
8. As a node operator, I want to see my current node score, shard assignment, and token earnings in the terminal, so that I can verify my node is contributing correctly.
9. As a node operator, I want to stake tokens before serving paid inference, so that I have skin in the game and the network can trust my outputs.
8. As a node operator, I want to see my current node score, shard assignment, and USDT earnings in the terminal, so that I can verify my node is contributing correctly.
9. As a node operator, I want to serve paid inference without upfront stake deposits, with my accrued USDT pending balance as fraud collateral and probation as the anti-sybil cost, so that onboarding stays frictionless. *(Supersedes stake-before-serving; ADR-0015/0018.)*
10. As a node operator, I want my first N jobs to run without earning (probationary period), so that the network can establish trust before paying me.
11. As a node operator, I want to be notified immediately if my stake is slashed due to a fraud detection event, so that I can investigate and fix the issue.
11. As a node operator, I want to be notified when my pending balance is forfeited due to a failed audit, so that I can investigate and fix the issue. *(Supersedes stake slash; ADR-0018 forfeiture.)*
12. As a node operator, I want to receive a strike and a warning before being banned, so that accidental failures don't immediately end my participation.
13. As a node operator, I want to be automatically reassigned to a different shard when the tracker determines another shard is more in demand, so that my hardware is always optimally used.
14. As a node operator, I want the node client to reconnect automatically if the tracker is temporarily unavailable, so that transient network issues don't stop me from earning.
@@ -34,8 +36,8 @@ A volunteer GPU network where anyone can share their GPU by running a single com
### Client Developer
17. As a client developer, I want to send `POST /v1/chat/completions` requests to the gateway in the same format as the OpenAI API, so that I can switch to the network with a one-line code change.
18. As a client developer, I want to authenticate with an API key funded by SOL or USDC, so that I never need to acquire or hold our native token.
19. As a client developer, I want to top up my API key balance by sending SOL or USDC to a Solana address, so that payment is simple and familiar.
18. As a client developer, I want to authenticate with an API key funded by USDT, so that I never need to acquire or hold our native token. *(ADR-0015.)*
19. As a client developer, I want to top up my API key balance by sending USDT to the treasury Solana address, so that payment is simple and familiar. *(ADR-0015; wallet binding US-039/041.)*
20. As a client developer, I want to see a per-request cost estimate before sending a request, so that I can budget inference costs accurately.
21. As a client developer, I want to receive streaming responses (`text/event-stream`) in OpenAI-compatible format, so that I can build low-latency user experiences.
22. As a client developer, I want `GET /v1/models` to return the list of available model presets on the network, so that I know what I can request.
@@ -46,15 +48,15 @@ A volunteer GPU network where anyone can share their GPU by running a single com
### End User (via a client app)
27. As an end user, I want to buy SOL on any exchange and use it to pay for inference, so that I don't need to understand blockchain technology to use the service.
27. As an end user, I want to buy USDT on an exchange and use it to pay for inference via Solana, so that I don't need deep crypto knowledge to use the service. *(Clients pay USDT; SOL is only for network fees if they self-custody.)*
28. As an end user, I want responses of equivalent quality to centralised providers, so that I don't have to trade quality for cost savings.
29. As an end user, I want low latency on first token, so that conversational applications feel responsive.
### Validator
30. As a validator, I want to automatically re-run a random sample (~5%) of completed inference requests on a reference node, so that I can detect nodes returning fraudulent outputs.
31. As a validator, I want to submit a fraud proof on-chain when a node's output diverges beyond tolerance, so that the slash event is recorded trustlessly.
32. As a validator, I want to earn a reward for each successful fraud detection, so that there is an economic incentive to run validation.
30. As a validator, I want to automatically re-run a random sample (~5%) of completed inference requests on a reference node with TOPLOC activation verification, so that I can detect nodes returning fraudulent outputs. *(ADR-0018.)*
31. As a validator, I want the tracker to record forfeiture and strikes when an audit fails, so that penalties are applied consistently. *(Supersedes on-chain fraud proof in alpha; ADR-0018.)*
32. As a validator, I want economic incentive to run validation, so that fraud detection is not purely altruistic. *(Validator reward share deferred; forfeiture to protocol cut today.)*
### Network (tracker / system)
@@ -62,7 +64,7 @@ A volunteer GPU network where anyone can share their GPU by running a single com
34. As the tracker, I want to rebalance shard assignments across nodes when demand for a model preset changes, so that the network always covers the most-requested models.
35. As the tracker, I want to instruct a node to download a new shard when no other node covers it, so that model preset coverage is maintained automatically.
36. As the tracker, I want to exclude banned wallets from route selection, so that fraudulent nodes cannot serve paid inference.
37. As the tracker, I want to read stake, slash, strike, and ban state exclusively from Solana smart contracts, so that I cannot manipulate payouts even with full control of the routing layer.
37. As the tracker, I want strike and ban state persisted in the registry and enforced on route selection, so that fraudulent wallets cannot serve paid inference. *(Supersedes on-chain-only stake/slash registry; ADR-0018; on-chain deferred per ADR-0007/0015.)*
38. As the network, I want new model presets to be addable by submitting a HuggingFace model ID and shard count, so that the set of available models can grow without code changes.
## Implementation Decisions
@@ -73,7 +75,7 @@ The codebase is organized as a Python monorepo with the following top-level pack
- `packages/gateway` — OpenAI-compatible HTTP gateway and route orchestration
- `packages/tracker` — centralized tracker service (node registry, scoring, route selection)
- `packages/sdk` — `meshnet` Python SDK wrapping gateway + wallet controls
- `packages/contracts` — Solana L2 smart contracts (stake, slash, strike, ban, settlement)
- `packages/contracts` — Solana adapter boundary (custodial USDT treasury, local registry prototype)
- `packages/p2p` — P2P gossip layer and shard swarm seeding
### Inference engine (ADR-0001; native GGUF path ADR-0024)
@@ -98,7 +100,7 @@ Clients pre-fund an API key with USDT. The tracker meters each request against t
Validators re-run ~5% of completed requests with TOPLOC activation verification. Caught cheaters forfeit pending balance and receive strikes; three strikes bans the wallet. Probation (first N unpaid jobs) remains the anti-sybil re-entry cost.
### Tracker architecture (ADR-0004)
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. Solana is the authoritative source of truth for all incentive-relevant state.
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. **Alpha:** strike/ban/forfeiture state lives in the tracker registry (ADR-0018); USDT settlement via custodial treasury (ADR-0015). On-chain programs deferred (ADR-0007).
### Shard distribution (ADR-0005)
Shards are identified by `(model_preset, shard_index)`. On assignment, the node downloads the shard layers from HuggingFace using `huggingface_hub`. Once downloaded, the node joins the P2P shard swarm and seeds to other nodes requesting the same shard. Popular shards propagate entirely via P2P; cold shards fall back to HuggingFace.
@@ -115,9 +117,9 @@ The gateway exposes OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/mo
**Per-component seams:**
- **Tracker**: given a set of registered nodes with known shard coverage and node scores, assert `select_route(model_preset)` returns an optimal ordered list of node endpoints.
- **Node shard serving**: given an activation tensor for the node's layer range, assert the output tensor shape and dtype are correct.
- **Fraud detection**: given a validator that re-runs a known-bad node response, assert a slash transaction is submitted on-chain with correct attribution.
- **Fraud detection**: given a validator that re-runs a known-bad node response, assert strike/forfeiture state updates with correct attribution (ADR-0018; on-chain slash deferred).
- **Shard swarm**: given a node that has a shard, assert a second node with the same assignment downloads it via P2P rather than HuggingFace.
- **Payment settlement**: given a completed inference session with known compute attribution, assert token balances change by the expected amounts after epoch settlement.
- **Payment settlement**: given a completed inference session with known compute attribution, assert USDT ledger balances change by the expected amounts after epoch settlement (ADR-0015).
## Out of Scope
@@ -135,4 +137,4 @@ The gateway exposes OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/mo
- The `meshnet-node` CLI is the primary viral growth vector. Every friction point in the install/start sequence costs node operators. The startup sequence must complete without any manual configuration on a machine with a CUDA-capable GPU.
- The name "meshnet" is a working name. The actual package and token names are TBD.
- The Solana L2 chain selection (vs Base/Arbitrum) is not yet finalised — both are cheap, EVM-compatible fallbacks. The contracts package should abstract chain-specific details.
- The probationary period length (N free jobs) and slash amounts are economic parameters that will need tuning once the network has real usage data. Hardcode sensible defaults; make them on-chain governable.
- The probationary period length (N free jobs) and forfeiture amounts are economic parameters that will need tuning once the network has real usage data. Hardcode sensible defaults; governance TBD (ADR-0018).