12 KiB
Main Features
High-level product capabilities for neuron-tai. Each section describes the user-facing
outcome, current status, and how it fits the mass-adoption goal. Implementation detail
lives in QUICKSTART.md, ADRs, and package code; this file is the product map.
Ralph task sources (authoritative status lives in source issue headers, not always
passes in JSON):
| Source | Stories | Ralph branch | Notes |
|---|---|---|---|
docs/prd.json |
US-001…035 | ralph/distributed-inference-network |
35/35 done |
.scratch/alpha-hardening/prd.json |
AH-001…025 | ralph/alpha-hardening |
See status table below — JSON passes can be stale |
docs/issues/ US-036+ |
36…47 | not in Ralph yet | Filed after main PRD closed |
.scratch/distributed-gguf-runtime/ |
10 milestones | not in Ralph yet | Draft scratch package |
Node bootstrap installer
Status: Planned — early development. Manual install (QUICKSTART.md) is the
current path; a unified installer is the next step toward one-click node onboarding.
Why it matters: Mass adoption depends on volunteers joining without reading a 691-line quickstart or guessing which PyTorch wheel matches their GPU. Inspiration: NiceHash — detect hardware, pick the right runtime, install, run. Our version must support heterogeneous fleet hardware (NVIDIA CUDA, AMD ROCm including Strix Halo gfx1151, CPU-only laptops) and later wrap the same logic in a web-based GUI.
Scope
| Phase | Boundary | Installer owns | User still does |
|---|---|---|---|
| v1 (now) | B — Python + OS deps | Clone/update repo, venv, correct PyTorch index, meshnet packages, OS package checks, hardware smoke test, launch setup wizard | GPU driver install (often needs reboot), WSL2 enablement, accepting elevated prompts |
| v2 (target) | C — NiceHash-style | Single downloadable artifact; may bundle Python/conda; maximal auto-setup | Almost nothing — accept UAC/reboot where the OS requires it |
v1 explicitly does not silently paper over missing drivers. If --gpu is set and
the GPU path cannot be verified, the installer fails with a structured error and a
wiki slug — it does not fall back to CPU unless --cpu was passed.
Entry points (planned)
# Linux / WSL — auto-detect hardware, install, smoke-test, run wizard
curl -fsSL https://<host>/install.sh | bash
# Explicit device mode (early development — these two flags are enough for v1)
curl -fsSL https://<host>/install.sh | bash -s -- --gpu
curl -fsSL https://<host>/install.sh | bash -s -- --cpu
# Non-interactive / GUI-driven (same script, no prompts)
curl -fsSL https://<host>/install.sh | bash -s -- --gpu --yes
Windows equivalent: install.ps1 with the same flags.
--cpu / --gpu semantics (v1)
| Flag | Meaning |
|---|---|
| (none) | Auto-detect hardware, print detected profile, proceed with best match (interactive confirm unless --yes) |
--cpu |
Skip GPU wheels entirely; install CPU PyTorch and register as a CPU node — even if a discrete GPU is present |
--gpu |
Install and verify a GPU runtime; fail hard if GPU execution cannot be confirmed after install |
--yes |
Skip interactive confirm; for headless installs and future web GUI orchestration |
These flags set intent for the install environment. They do not replace runtime
hardware detection in meshnet_node.hardware.detect_hardware() — that profile still
feeds tracker registration and routing.
v1 install pipeline
-
Preflight — Python 3.11+ (3.12 recommended for Qwen3.6/FLA), git, disk space, network.
-
Hardware probe — reuse detection logic aligned with
packages/node/meshnet_node/hardware.py(nvidia-smi, Windows WMI, torch CUDA/HIP inventory, RAM). -
OS dependency checks (boundary B) — verify or install distro packages where safe (e.g.
python3-venv,build-essential); check GPU device nodes (/dev/kfd,/dev/dri/renderD*) and group membership (video,render) on Linux AMD; emit fix instructions, do not auto-modify kernel drivers. -
PyTorch variant selection — one wheel line per detected (or forced) profile:
Profile PyTorch source NVIDIA CUDA Default PyPI index CPU only download.pytorch.org/whl/cpuAMD ROCm (discrete, supported arch) download.pytorch.org/whl/rocm6.3AMD Strix Halo / gfx1151 rocm.nightlies.amd.com/v2/gfx1151/See
QUICKSTART.md§ PyTorch variant for host prerequisites and troubleshooting notes already validated on the fleet. -
Meshnet packages — editable install of
packages/node(+p2pas needed);transformers,accelerate, and model-specific extras (e.g.flash-linear-attentionon ROCm for Qwen3.6). -
Smoke test — short matmul on chosen device (same idea as
benchmark_throughput_checked()); must pass before declaring success. -
Hand off — run existing mining-style wizard (
packages/node/meshnet_node/wizard.py): tracker URL, wallet, model/shard assignment.
Keep ROCm and CPU envs separate when probing GPU paths so a failed ROCm attempt
does not break a known-good CPU venv (QUICKSTART.md already documents this pattern).
Failure telemetry and hardware wiki
Every failed install should report back structured diagnostics so support improves with fleet scale:
- Report payload (planned): OS, CPU model, RAM, GPU name/VRAM/arch, chosen
PyTorch index, failing step, stderr tail, installer version,
--cpu/--gpuflag. - Privacy: opt-in or anonymous fleet telemetry; no wallet keys or model paths.
- Hardware wiki / index: failed (and successful) profiles accumulate into a
searchable support index — e.g.
rocm-missing-kfd,gfx1151-wrong-wheel,wsl2-nvidia-smi-missing. Each slug links symptoms, detection rule, fix steps, and "works on" confirmations. Future GUI surfaces the same index when install fails.
This closes the loop NiceHash gets from millions of installs: uncommon hardware becomes documented automatically instead of repeating Discord support threads.
GUI integration (later)
The install script is the headless API for a future web-based node manager:
- GUI downloads or invokes
install.sh/install.ps1with--gpu --yesand streams log output. - Same failure payloads feed the hardware wiki and in-app "your GPU + Fedora 43" fix cards.
- Post-install, GUI wraps
meshnet-nodedashboard and tracker registration status.
Related code and docs
| Asset | Role |
|---|---|
packages/node/meshnet_node/hardware.py |
Runtime hardware detection and benchmark |
packages/node/meshnet_node/wizard.py |
Post-install interactive setup |
QUICKSTART.md |
Current manual install matrix (source of truth until installer ships) |
docs/INSTALL_WINDOWS.md |
WSL2 + CUDA passthrough path |
Open decisions (post-v1)
- Exact telemetry endpoint and opt-in UX.
- Whether v1 ships
install.shonly or also a pinned release tarball (no git required). - Conda vs venv default on Windows (today: both documented; installer should pick one happy path per platform).
Core network (docs/prd.json — 35/35 done)
Original distributed-inference Ralph arc. All stories status: done.
| Theme | Stories | Status |
|---|---|---|
| Scaffold + two-node pipeline | 01–02 | Done |
| Tracker registration & routing | 03, 13–14, 20–30 | Done |
| Node client + mining CLI | 04, 16, 21 | Done |
| OpenAI gateway + SDK | 05, 10 | Done |
| PyTorch backend + binary wire format | 11–12, 19 | Done |
| P2P swarm + relay/NAT | 09, 17, 29 | Done |
| Heartbeat, stats, smart assignment | 23–28 | Done |
| Billing, devnet treasury, settlement, dashboard | 31–35 | Done |
| Fraud / stake (superseded) | 06–08 | Done in PRD; alpha path replaced by ADR-0015/0018 + alpha-hardening |
| Ralph tooling | 15 | Done (scripts/ralph_progress.py) |
| Two-machine LAN test | 18 | Done |
User-facing capabilities this arc delivered: mixed CPU+GPU routes across machines,
hardware-aware routing, relay (no port-forward), OpenAI-compatible API, mining-style
meshnet-node wizard, billing ledger, devnet USDT, tracker web dashboard.
Alpha hardening (.scratch/alpha-hardening/ — AH-001…025)
Pre-release trust/money/fraud path. Index:
.scratch/alpha-hardening/README.md.
Done (engineering complete)
| ID | Feature |
|---|---|
| AH-001…005 | Hive gossip auth, unified auth boundary, zero starting credit, tracker-authoritative accounting, persisted strike/ban/reputation |
| AH-006…010 | TOPLOC integration, hop bisection, reputation model, adaptive audit routing, penalty wiring |
| AH-011, AH-020 | Wallet binding proof, validator service token |
| AH-016, AH-018…019, AH-022 | Doc hygiene: US-006 reconciliation, runbooks, test-env, memory index |
| AH-023 | Dynamic HF-benchmarked pricing (engineering done; hf_aliases curation is human sign-off) |
Open / not truly done
| ID | Feature | Status | Blocker |
|---|---|---|---|
| AH-021 | Honest-noise TOPLOC calibration corpus | ready-for-human | Alpha release blocker — run calibration job on live hired-VPS fleet; threshold/FPR write-up |
| AH-024 | Learned-routing telemetry + live-progress cleanup | ready-for-agent | server.py:1490 import crash; dashboard active-request telemetry |
| AH-025 | Sharded per-node KV cache | implemented — verify | Re-measure on live 2-node GPU + Qwen3.6 mixed topology (ADR-0022) |
Deferred (post-alpha, design tracked — ADR-0019)
| ID | Feature | Status |
|---|---|---|
| AH-012…015 | On-chain idempotency, consensus-gated settlement, durable Raft term/vote, commutative forfeit | ready-for-human |
| AH-017 | Duplicate US-020 issue dedup | ready-for-human |
Post-PRD backlog (docs/issues/ US-036+)
Filed after the main 35-story arc closed. Not yet in a Ralph prd.json.
| ID | Feature | Status | Priority note |
|---|---|---|---|
| US-036 | Streamed chat over relay RPC | planned | Critical — blocks public friends-test |
| US-037 | Relay bridge concurrency | planned | |
| US-038 | Tracker seed join | planned | |
| US-039…041 | Caller credit keys, dashboard top-up, account wallet keypair | planned | |
| US-042 | GGUF / llama.cpp node backend | planned | Pairs with distributed-gguf scratch |
| US-043 | Dashboard model search cards | planned | |
| US-044 | Tracker as shard file source (partial download) | in progress | High — multi-machine big models |
| US-045 | Dual-rate billing | in progress | |
| US-046 | Tracker env + first-node autojoin | in progress | |
| US-047 | Model source download visibility | in progress | |
| US-020b | Memory budget, shard slots, dropout relocation | ready-for-agent | Hardens US-013 capacity contract |
Distributed GGUF runtime (draft scratch)
Long-horizon runtime for torrent-distributed GGUF + llama.cpp multi-node routes.
Not in Ralph yet. See
.scratch/distributed-gguf-runtime/README.md.
| Milestone | Status |
|---|---|
| 01–10 (route session → networked GGUF → model audits) | Planned / not started |
| PyTorch distributed KV reference (04) | Partially addressed by AH-025 |
Feature status at a glance
| Feature | Status | Ralph / source |
|---|---|---|
| Mixed hardware inference routes | Working | US-002+, ADR-0020 |
| Hardware-aware + learned routing | Working (telemetry cleanup open) | US-027+, AH-024 |
| Zero port-forwarding (relay) | Working (streamed relay chat open) | US-017, US-029, US-036 |
| OpenAI-compatible API | Working | US-005 |
| Mining-style node CLI + wizard | Working (installer not built) | US-016 |
| Billing + devnet USDT | Working | US-031…033, alpha-hardening |
| Fraud / TOPLOC / reputation | Engineering done (calibration ops pending) | AH-006…010, AH-021 |
| Sharded per-node KV cache | Implemented — GPU verify pending | AH-025, ADR-0022 |
| Node bootstrap installer | Planned | This doc — not in Ralph yet |
| Dynamic HF pricing | Done (alias curation ongoing) | AH-023 |
| Distributed GGUF / llama.cpp | Draft | .scratch/distributed-gguf-runtime/ |
Narrative hooks for landing copy:
.claude/memory/product-selling-points.md.