2.3 KiB
name, description, metadata
| name | description | metadata | ||||||
|---|---|---|---|---|---|---|---|---|
| product-selling-points | Key differentiators and landing page angles for neuron-tai distributed inference network |
|
neuron-tai — Product Selling Points
Core pitch
Volunteer GPU network for distributed LLM inference. Small GPU owners contribute compute and earn TAI tokens. Clients get inference on models larger than any single machine can serve.
Confirmed technical differentiators (verified working)
Mixed hardware inference routes
The tracker can chain CPU nodes and GPU nodes into a single inference route. Shard A on a CPU node → Shard B on a GPU node → valid streamed response. Each participant in the route only needs to fit their shard in memory, not the whole model.
Angle for landing page: "Run a 70B model across three laptops and a gaming PC. Each machine only holds the layers it can fit."
Nuance to acknowledge: PyTorch/HuggingFace device_map="auto" already does CPU+GPU mixing on a single machine. Our value-add is doing this across machines over the network, democratizing access to models that no single volunteer machine could serve alone.
Hardware-aware routing
Tracker scores nodes by benchmark_tokens_per_sec / (queue_depth + 1) and always routes to the fastest available node per shard range. A GPU node at 11,200 throughput index beats a CPU node at 626 automatically — no user configuration needed.
Zero port-forwarding required
Nodes connect outbound to the relay via WebSocket. Works from behind NAT, WSL2, 5G, or a home router with no config. The public tracker at ai.neuron.d-popov.com handles discovery.
OpenAI-compatible API
Any app using the OpenAI Python SDK works by changing only base_url. No code changes for the client.
Landing page content TODO
- User asked to capture these points for the landing page copy (2026-07-01)
- No landing page file exists in the repo yet
- When writing copy, lead with the "run models bigger than your GPU" angle, then support with mixed-hardware routing, relay, and OpenAI compat
How to apply: When writing product descriptions, pitches, or landing page copy, use these as the primary hooks. The mixed-network inference route (CPU+GPU across machines) is the biggest differentiator vs. single-machine solutions.