39 lines
2.3 KiB
Markdown
39 lines
2.3 KiB
Markdown
---
|
|
name: product-selling-points
|
|
description: Key differentiators and landing page angles for neuron-tai distributed inference network
|
|
metadata:
|
|
node_type: memory
|
|
type: project
|
|
originSessionId: 8fb120ee-7b8e-45be-98c0-b5ae9c64d1ec
|
|
---
|
|
|
|
# neuron-tai — Product Selling Points
|
|
|
|
## Core pitch
|
|
Volunteer GPU network for distributed LLM inference. Small GPU owners contribute compute and earn TAI tokens. Clients get inference on models larger than any single machine can serve.
|
|
|
|
## Confirmed technical differentiators (verified working)
|
|
|
|
### Mixed hardware inference routes
|
|
The tracker can chain CPU nodes and GPU nodes into a single inference route. Shard A on a CPU node → Shard B on a GPU node → valid streamed response. Each participant in the route only needs to fit *their shard* in memory, not the whole model.
|
|
|
|
**Angle for landing page:** "Run a 70B model across three laptops and a gaming PC. Each machine only holds the layers it can fit."
|
|
|
|
**Nuance to acknowledge:** PyTorch/HuggingFace `device_map="auto"` already does CPU+GPU mixing on a single machine. Our value-add is doing this *across machines over the network*, democratizing access to models that no single volunteer machine could serve alone.
|
|
|
|
### Hardware-aware routing
|
|
Tracker scores nodes by `benchmark_tokens_per_sec / (queue_depth + 1)` and always routes to the fastest available node per shard range. A GPU node at 11,200 throughput index beats a CPU node at 626 automatically — no user configuration needed.
|
|
|
|
### Zero port-forwarding required
|
|
Nodes connect outbound to the relay via WebSocket. Works from behind NAT, WSL2, 5G, or a home router with no config. The public tracker at ai.neuron.d-popov.com handles discovery.
|
|
|
|
### OpenAI-compatible API
|
|
Any app using the OpenAI Python SDK works by changing only `base_url`. No code changes for the client.
|
|
|
|
## Landing page content TODO
|
|
- User asked to capture these points for the landing page copy (2026-07-01)
|
|
- No landing page file exists in the repo yet
|
|
- When writing copy, lead with the "run models bigger than your GPU" angle, then support with mixed-hardware routing, relay, and OpenAI compat
|
|
|
|
**How to apply:** When writing product descriptions, pitches, or landing page copy, use these as the primary hooks. The mixed-network inference route (CPU+GPU across machines) is the biggest differentiator vs. single-machine solutions.
|