256 lines
13 KiB
Markdown
256 lines
13 KiB
Markdown
# Main Features
|
||
|
||
High-level product capabilities for neuron-tai. Each section describes the user-facing
|
||
outcome, current status, and how it fits the mass-adoption goal. Implementation detail
|
||
lives in `QUICKSTART.md`, ADRs, and package code; this file is the product map.
|
||
|
||
**Ralph task sources** (authoritative status lives in source issue headers, not always
|
||
`passes` in JSON):
|
||
|
||
| Source | Stories | Ralph branch | Notes |
|
||
|--------|---------|--------------|-------|
|
||
| [`docs/prd.json`](docs/prd.json) | US-001…035 | `ralph/distributed-inference-network` | **35/35 done** |
|
||
| [`.scratch/alpha-hardening/prd.json`](.scratch/alpha-hardening/prd.json) | AH-001…025 | `ralph/alpha-hardening` | See status table below — JSON `passes` can be stale |
|
||
| [`docs/issues/`](docs/issues/) US-036+ | 36…47 | not in Ralph yet | Filed after main PRD closed |
|
||
| [`.scratch/distributed-gguf-runtime/`](.scratch/distributed-gguf-runtime/) | 10 milestones | not in Ralph yet | Draft scratch package |
|
||
|
||
---
|
||
|
||
## Node bootstrap installer
|
||
|
||
**Status:** Planned — early development. Manual install (`QUICKSTART.md`) is the
|
||
current path; a unified installer is the next step toward one-click node onboarding.
|
||
|
||
**Why it matters:** Mass adoption depends on volunteers joining without reading a
|
||
691-line quickstart or guessing which PyTorch wheel matches their GPU. Inspiration:
|
||
[NiceHash](https://www.nicehash.com/) — detect hardware, pick the right runtime,
|
||
install, run. Our version must support heterogeneous fleet hardware (NVIDIA CUDA,
|
||
AMD ROCm including Strix Halo gfx1151, CPU-only laptops) and later wrap the same
|
||
logic in a web-based GUI.
|
||
|
||
### Scope
|
||
|
||
| Phase | Boundary | Installer owns | User still does |
|
||
|-------|----------|----------------|-----------------|
|
||
| **v1 (now)** | **B — Python + OS deps** | Clone/update repo, venv, correct PyTorch index, meshnet packages, OS package checks, hardware smoke test, launch setup wizard | GPU driver install (often needs reboot), WSL2 enablement, accepting elevated prompts |
|
||
| **v2 (target)** | **C — NiceHash-style** | Single downloadable artifact; may bundle Python/conda; maximal auto-setup | Almost nothing — accept UAC/reboot where the OS requires it |
|
||
|
||
v1 explicitly does **not** silently paper over missing drivers. If `--gpu` is set and
|
||
the GPU path cannot be verified, the installer fails with a structured error and a
|
||
wiki slug — it does not fall back to CPU unless `--cpu` was passed.
|
||
|
||
### Entry points (planned)
|
||
|
||
```bash
|
||
# Linux / WSL — auto-detect hardware, install, smoke-test, run wizard
|
||
curl -fsSL https://<host>/install.sh | bash
|
||
|
||
# Explicit device mode (early development — these two flags are enough for v1)
|
||
curl -fsSL https://<host>/install.sh | bash -s -- --gpu
|
||
curl -fsSL https://<host>/install.sh | bash -s -- --cpu
|
||
|
||
# Non-interactive / GUI-driven (same script, no prompts)
|
||
curl -fsSL https://<host>/install.sh | bash -s -- --gpu --yes
|
||
```
|
||
|
||
Windows equivalent: `install.ps1` with the same flags.
|
||
|
||
### `--cpu` / `--gpu` semantics (v1)
|
||
|
||
| Flag | Meaning |
|
||
|------|---------|
|
||
| *(none)* | Auto-detect hardware, print detected profile, proceed with best match (interactive confirm unless `--yes`) |
|
||
| `--cpu` | Installer: CPU PyTorch wheel. **`meshnet-node --cpu` (implemented):** force CPU inference and CPU shard assignment even if a GPU is present |
|
||
| `--gpu` | Install and verify a GPU runtime; **fail hard** if GPU execution cannot be confirmed after install (installer only — not implemented on `meshnet-node` yet) |
|
||
| `--yes` | Skip interactive confirm; for headless installs and future web GUI orchestration |
|
||
|
||
Installer flags set install-time intent. At runtime, `meshnet-node` auto-uses GPU when
|
||
CUDA works; pass `--cpu` to ignore it. Hardware metadata (GPU name/VRAM) is still
|
||
detected for diagnostics.
|
||
|
||
### v1 install pipeline
|
||
|
||
1. **Preflight** — Python 3.11+ (3.12 recommended for Qwen3.6/FLA), git, disk space,
|
||
network.
|
||
2. **Hardware probe** — reuse detection logic aligned with
|
||
`packages/node/meshnet_node/hardware.py` (nvidia-smi, Windows WMI, torch CUDA/HIP
|
||
inventory, RAM).
|
||
3. **OS dependency checks (boundary B)** — verify or install distro packages where
|
||
safe (e.g. `python3-venv`, `build-essential`); **check** GPU device nodes
|
||
(`/dev/kfd`, `/dev/dri/renderD*`) and group membership (`video`, `render`) on
|
||
Linux AMD; emit fix instructions, do not auto-modify kernel drivers.
|
||
4. **PyTorch variant selection** — one wheel line per detected (or forced) profile:
|
||
|
||
| Profile | PyTorch source |
|
||
|---------|----------------|
|
||
| NVIDIA CUDA | Default PyPI index |
|
||
| CPU only | `download.pytorch.org/whl/cpu` |
|
||
| AMD ROCm (discrete, supported arch) | `download.pytorch.org/whl/rocm6.3` |
|
||
| AMD Strix Halo / gfx1151 | `rocm.nightlies.amd.com/v2/gfx1151/` |
|
||
|
||
See `QUICKSTART.md` § PyTorch variant for host prerequisites and troubleshooting
|
||
notes already validated on the fleet.
|
||
|
||
5. **Meshnet packages** — editable install of `packages/node` (+ `p2p` as needed);
|
||
`transformers`, `accelerate`, and model-specific extras (e.g. `flash-linear-attention`
|
||
on ROCm for Qwen3.6).
|
||
6. **Smoke test** — short matmul on chosen device (same idea as
|
||
`benchmark_throughput_checked()`); must pass before declaring success.
|
||
7. **Hand off** — run existing mining-style wizard (`packages/node/meshnet_node/wizard.py`):
|
||
tracker URL, wallet, model/shard assignment.
|
||
|
||
Keep ROCm and CPU envs **separate** when probing GPU paths so a failed ROCm attempt
|
||
does not break a known-good CPU venv (`QUICKSTART.md` already documents this pattern).
|
||
|
||
### Failure telemetry and hardware wiki
|
||
|
||
Every failed install should report back structured diagnostics so support improves
|
||
with fleet scale:
|
||
|
||
- **Report payload (planned):** OS, CPU model, RAM, GPU name/VRAM/arch, chosen
|
||
PyTorch index, failing step, stderr tail, installer version, `--cpu`/`--gpu` flag.
|
||
- **Privacy:** opt-in or anonymous fleet telemetry; no wallet keys or model paths.
|
||
- **Hardware wiki / index:** failed (and successful) profiles accumulate into a
|
||
searchable support index — e.g. `rocm-missing-kfd`, `gfx1151-wrong-wheel`,
|
||
`wsl2-nvidia-smi-missing`. Each slug links symptoms, detection rule, fix steps,
|
||
and "works on" confirmations. Future GUI surfaces the same index when install fails.
|
||
|
||
This closes the loop NiceHash gets from millions of installs: uncommon hardware
|
||
becomes documented automatically instead of repeating Discord support threads.
|
||
|
||
### GUI integration (later)
|
||
|
||
The install script is the **headless API** for a future web-based node manager:
|
||
|
||
- GUI downloads or invokes `install.sh` / `install.ps1` with `--gpu --yes` and streams
|
||
log output.
|
||
- Same failure payloads feed the hardware wiki and in-app "your GPU + Fedora 43"
|
||
fix cards.
|
||
- Post-install, GUI wraps `meshnet-node` dashboard and tracker registration status.
|
||
|
||
### Related code and docs
|
||
|
||
| Asset | Role |
|
||
|-------|------|
|
||
| `packages/node/meshnet_node/hardware.py` | Runtime hardware detection and benchmark |
|
||
| `packages/node/meshnet_node/wizard.py` | Post-install interactive setup |
|
||
| `QUICKSTART.md` | Current manual install matrix (source of truth until installer ships) |
|
||
| `docs/INSTALL_WINDOWS.md` | WSL2 + CUDA passthrough path |
|
||
|
||
### Open decisions (post-v1)
|
||
|
||
- Exact telemetry endpoint and opt-in UX.
|
||
- Whether v1 ships `install.sh` only or also a pinned release tarball (no git required).
|
||
- Conda vs venv default on Windows (today: both documented; installer should pick one
|
||
happy path per platform).
|
||
|
||
---
|
||
|
||
## Core network (`docs/prd.json` — 35/35 done)
|
||
|
||
Original distributed-inference Ralph arc. All stories `status: done`.
|
||
|
||
| Theme | Stories | Status |
|
||
|-------|---------|--------|
|
||
| Scaffold + two-node pipeline | 01–02 | Done |
|
||
| Tracker registration & routing | 03, 13–14, 20–30 | Done |
|
||
| Node client + mining CLI | 04, 16, 21 | Done |
|
||
| OpenAI gateway + SDK | 05, 10 | Done |
|
||
| PyTorch backend + binary wire format | 11–12, 19 | Done |
|
||
| P2P swarm + relay/NAT | 09, 17, 29 | Done |
|
||
| Heartbeat, stats, smart assignment | 23–28 | Done |
|
||
| Billing, devnet treasury, settlement, dashboard | 31–35 | Done |
|
||
| Fraud / stake (superseded) | 06–08 | Done in PRD; alpha path replaced by ADR-0015/0018 + alpha-hardening |
|
||
| Ralph tooling | 15 | Done (`scripts/ralph_progress.py`) |
|
||
| Two-machine LAN test | 18 | Done |
|
||
|
||
User-facing capabilities this arc delivered: mixed CPU+GPU routes across machines,
|
||
hardware-aware routing, relay (no port-forward), OpenAI-compatible API, mining-style
|
||
`meshnet-node` wizard, billing ledger, devnet USDT, tracker web dashboard.
|
||
|
||
---
|
||
|
||
## Alpha hardening (`.scratch/alpha-hardening/` — AH-001…025)
|
||
|
||
Pre-release trust/money/fraud path. Index:
|
||
[`.scratch/alpha-hardening/README.md`](.scratch/alpha-hardening/README.md).
|
||
|
||
### Done (engineering complete)
|
||
|
||
| ID | Feature |
|
||
|----|---------|
|
||
| AH-001…005 | Hive gossip auth, unified auth boundary, zero starting credit, tracker-authoritative accounting, persisted strike/ban/reputation |
|
||
| AH-006…010 | TOPLOC integration, hop bisection, reputation model, adaptive audit routing, penalty wiring |
|
||
| AH-011, AH-020 | Wallet binding proof, validator service token |
|
||
| AH-016, AH-018…019, AH-022 | Doc hygiene: US-006 reconciliation, runbooks, test-env, memory index |
|
||
| AH-023 | Dynamic HF-benchmarked pricing (engineering done; `hf_aliases` curation is human sign-off) |
|
||
|
||
### Open / not truly done
|
||
|
||
| ID | Feature | Status | Blocker |
|
||
|----|---------|--------|---------|
|
||
| AH-021 | Honest-noise TOPLOC calibration corpus | **ready-for-human** | **Alpha release blocker** — run calibration job on live hired-VPS fleet; threshold/FPR write-up |
|
||
| AH-024 | Learned-routing telemetry + live-progress cleanup | **ready-for-agent** | `server.py:1490` import crash; dashboard active-request telemetry |
|
||
| AH-025 | Sharded per-node KV cache | **implemented — verify** | Re-measure on live 2-node GPU + Qwen3.6 mixed topology ([ADR-0022](docs/adr/0022-sharded-per-node-kv-cache.md)) |
|
||
|
||
### Deferred (post-alpha, design tracked — ADR-0019)
|
||
|
||
| ID | Feature | Status |
|
||
|----|---------|--------|
|
||
| AH-012…015 | On-chain idempotency, consensus-gated settlement, durable Raft term/vote, commutative forfeit | ready-for-human |
|
||
| AH-017 | Duplicate US-020 issue dedup | ready-for-human |
|
||
|
||
---
|
||
|
||
## Post-PRD backlog (`docs/issues/` US-036+)
|
||
|
||
Filed after the main 35-story arc closed. Not yet in a Ralph `prd.json`.
|
||
|
||
| ID | Feature | Status | Priority note |
|
||
|----|---------|--------|---------------|
|
||
| US-036 | Streamed chat over relay RPC | planned | Critical — blocks public friends-test |
|
||
| US-037 | Relay bridge concurrency | planned | |
|
||
| US-038 | Tracker seed join | planned | |
|
||
| US-039…041 | Caller credit keys, dashboard top-up, account wallet keypair | planned | |
|
||
| US-042 | GGUF / llama.cpp node backend | planned | Pairs with distributed-gguf scratch |
|
||
| US-043 | Dashboard model search cards | planned | |
|
||
| US-044 | Tracker as shard file source (partial download) | **in progress** | High — multi-machine big models |
|
||
| US-045 | Dual-rate billing | **in progress** | |
|
||
| US-046 | Tracker env + first-node autojoin | **in progress** | |
|
||
| US-047 | Model source download visibility | **in progress** | |
|
||
| US-020b | Memory budget, shard slots, dropout relocation | ready-for-agent | Hardens US-013 capacity contract |
|
||
|
||
---
|
||
|
||
## Distributed GGUF runtime (draft scratch)
|
||
|
||
Long-horizon runtime for torrent-distributed GGUF + llama.cpp multi-node routes.
|
||
Not in Ralph yet. See
|
||
[`.scratch/distributed-gguf-runtime/README.md`](.scratch/distributed-gguf-runtime/README.md).
|
||
|
||
| Milestone | Status |
|
||
|-----------|--------|
|
||
| 01–10 (route session → networked GGUF → model audits) | Planned / not started |
|
||
| PyTorch distributed KV reference (04) | Partially addressed by AH-025 |
|
||
|
||
---
|
||
|
||
## Feature status at a glance
|
||
|
||
| Feature | Status | Ralph / source |
|
||
|---------|--------|----------------|
|
||
| Mixed hardware inference routes | **Working** | US-002+, ADR-0020 |
|
||
| Hardware-aware + learned routing | **Working** (telemetry cleanup open) | US-027+, AH-024 |
|
||
| Zero port-forwarding (relay) | **Working** (streamed relay chat open) | US-017, US-029, US-036 |
|
||
| OpenAI-compatible API | **Working** | US-005 |
|
||
| Mining-style node CLI + wizard | **Working** (`--cpu` forces CPU mode) | US-016 |
|
||
| Billing + devnet USDT | **Working** | US-031…033, alpha-hardening |
|
||
| Fraud / TOPLOC / reputation | **Engineering done** (calibration ops pending) | AH-006…010, AH-021 |
|
||
| Sharded per-node KV cache | **Implemented — GPU verify pending** | AH-025, ADR-0022 |
|
||
| Node bootstrap installer | **Planned** | This doc — not in Ralph yet |
|
||
| Dynamic HF pricing | **Done** (alias curation ongoing) | AH-023 |
|
||
| Distributed GGUF / llama.cpp | **Draft** | `.scratch/distributed-gguf-runtime/` |
|
||
|
||
Narrative hooks for landing copy:
|
||
[`.claude/memory/product-selling-points.md`](.claude/memory/product-selling-points.md).
|