d1e75ddded
Move issues (01–29) and PRD from .scratch/distributed-inference-network/ into docs/issues/ and docs/. Update ralph_progress.py DEFAULT_PRD path and rewrite docs/agents/issue-tracker.md to reflect the new layout. The distributed_inference_network.egg-info/docs/ mirror is a build artifact already covered by *.egg-info/ in .gitignore — not committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
6.7 KiB
6.7 KiB
US-016 — Mining-style node startup CLI + live dashboard
Goal
Replace the bare flag-driven meshnet-node start with a wizard-guided first-run experience modelled on GPU mining clients (like PhoenixMiner, lolMiner, etc.). After the wizard, the terminal switches to a live status dashboard showing real-time node health and earnings.
Wizard flow (first run only)
╔══════════════════════════════════════════════════════════╗
║ meshnet-node v0.1.0 ║
║ Distributed AI Inference — Node Setup ║
╚══════════════════════════════════════════════════════════╝
Detecting hardware...
GPU 0: NVIDIA RTX 4090 24 GB VRAM ✓
GPU 1: NVIDIA RTX 3090 24 GB VRAM ✓
Select a model to serve:
# Model Layers NF4 INT8 BF16
1 Llama-3-70B-Instruct 80 ✓18GB ✓40GB ✗80GB
2 Qwen-2.5-72B-Instruct 80 ✓19GB ✗41GB ✗81GB
3 Mixtral-8x7B-Instruct-v0.1 32 ✓ 7GB ✓14GB ✓27GB
4 Phi-3-medium-128k-instruct 40 ✓ 4GB ✓ 8GB ✓15GB
5 [Browse HuggingFace…]
Enter number [1]: _
Quantization [nf4/int8/bf16] (nf4 recommended for 24GB): _
Download directory [~/.meshnet/models]: _
Tracker URL [http://localhost:8080]: _
Wallet path [~/.config/meshnet/wallet.json] (new wallet will be created): _
Config saved to ~/.config/meshnet/config.json
Starting node…
Second run with existing config:
meshnet-node
Reading config from ~/.config/meshnet/config.json
Model: Llama-3-70B-Instruct Quant: nf4 Shard: layers 0–15
Tracker: http://192.168.1.10:8080
Starting…
Live dashboard (once running)
Renders every 2 seconds using rich.live. Fallback: plain-text status line if rich is unavailable or terminal is not a TTY (important for WSL2 / SSH).
meshnet-node Llama-3-70B-Instruct [nf4] shard 0–15/80 up 00:03:22
GPU 0 RTX 4090 GPU ████████░░ 73% VRAM 18.2/24.0 GB 45°C
GPU 1 RTX 3090 GPU ███░░░░░░░ 28% VRAM 8.7/24.0 GB 38°C
Tokens/sec ▁▂▃▄▅▆▇█ 42.3 t/s (EMA 30s)
Requests 1,247 served 3 active
Peers 8 connected (tracker: ✓ relay: ✓)
TAI earned 0.00 TAI (payments active after US-006)
Uptime 00:03:22
[q] quit [r] reset stats [c] compact view
Compact mode (--compact or pressing c) shows a single status line:
[43t/s VRAM18.2GB req1247 peers8 up3m22s]
Implementation notes
Hardware detection
import torch
def detect_gpus() -> list[dict]:
gpus = []
if torch.cuda.is_available():
for i in range(torch.cuda.device_count()):
props = torch.cuda.get_device_properties(i)
gpus.append({
"index": i,
"name": props.name,
"vram_gb": props.total_memory / 1e9,
"backend": "cuda"
})
# ROCm / Apple Silicon stubs for later
return gpus
Curated model list
packages/node/meshnet_node/model_catalog.py — a hardcoded list of ModelPreset dataclasses:
@dataclass
class ModelPreset:
name: str # display name
hf_repo: str # HuggingFace repo ID
num_layers: int
vram_gb: dict # {"nf4": 18, "int8": 40, "bf16": 80}
description: str # one-line description
Initial list (expand over time):
meta-llama/Meta-Llama-3-70B-Instruct— 80L, NF4 18GB, INT8 40GB, BF16 80GBQwen/Qwen2.5-72B-Instruct— 80L, NF4 19GB, INT8 41GB, BF16 81GBmistralai/Mixtral-8x7B-Instruct-v0.1— 32L, NF4 7GB, INT8 14GB, BF16 27GBmicrosoft/Phi-3-medium-128k-instruct— 40L, NF4 4GB, INT8 8GB, BF16 15GBgoogle/gemma-2-27b-it— 46L, NF4 10GB, INT8 20GB, BF16 40GB
HuggingFace Browse
from huggingface_hub import list_models
def browse_hf(top_n=20) -> list[dict]:
models = list_models(
pipeline_tag="text-generation",
library="transformers",
sort="downloads",
direction=-1,
limit=top_n,
cardData=True,
)
return [{"repo": m.modelId, "downloads": m.downloads} for m in models]
Persistent config
~/.config/meshnet/config.json:
{
"model_hf_repo": "meta-llama/Meta-Llama-3-70B-Instruct",
"quantization": "nf4",
"download_dir": "~/.meshnet/models",
"tracker_url": "http://192.168.1.10:8080",
"wallet_path": "~/.config/meshnet/wallet.json",
"shard_start": null,
"shard_end": null,
"updatedAt": "2026-06-29T..."
}
shard_start/shard_end: null means tracker auto-assigns. User can pin a range for dedicated partial-model nodes.
CLI flags
All wizard answers are overridable without re-running the wizard:
meshnet-node [start]
--model <hf-repo-id> # e.g. meta-llama/Meta-Llama-3-70B-Instruct
--quantization [bf16|int8|nf4]
--download-dir <path>
--tracker <url>
--wallet <path>
--shard-start <int> # pin shard range (optional)
--shard-end <int>
--reset-config # ignore saved config, re-run wizard
--no-tui # plain-text output (for CI / headless)
--compact # single-line status instead of full dashboard
meshnet-node models # list curated models and exit
meshnet-node models --browse # list HF Hub top-20 and exit
meshnet-node config # print current config and exit
WSL2 / non-TTY fallback
import sys, os
def is_interactive_tty() -> bool:
return sys.stdout.isatty() and os.environ.get("TERM") not in ("dumb", "")
if not is_interactive_tty():
# fall back to plain-text periodic status
run_plain_status_loop(node)
else:
run_rich_dashboard(node)
Do NOT use termios, fcntl, or /dev/tty — these break in Windows cmd.exe and some WSL2 terminal emulators.
Acceptance criteria
meshnet-nodewith no args and no config → wizard starts- Wizard detects GPU and marks
[too large]for models that exceed available VRAM meshnet-node modelsprints curated list and exitsmeshnet-node models --browsecalls HF Hub API, prints top-20, exits- Second run (config exists) → skips wizard, starts immediately
--reset-configre-runs wizard even with config present- All wizard inputs override-able via CLI flags
- Live rich dashboard renders and updates every 2s when running in a TTY
- Falls back to plain-text when not a TTY (CI / WSL2 without TERM set)
- Ctrl-C prints a clean summary line and exits 0
python -m pytestpasses from repo root- Commit only this story's changes