misc

QUICKSTART.md

@@ -3,7 +3,9 @@
 Get from zero to a live inference request in **three terminals**: install once, start
 the tracker, start a node, send a request.
 
-Tested on: AMD Ryzen AI Max (Strix Halo APU), 124 GB RAM, Linux, CPU inference.
+Tested on: AMD Ryzen AI Max (Strix Halo APU), 124 GB RAM, Linux CPU inference.
+ROCm GPU setup is covered below, but must be verified on the host because ROCm
+support depends on the exact AMD GPU/APU, kernel, driver, and ROCm runtime.
 
 **Active development models** (what we run day-to-day):
 
@@ -129,11 +131,110 @@ Install **one** torch line into the same env as `meshnet-node`:
 |----------|---------|
 | NVIDIA CUDA | `pip install torch` (default index) |
 | CPU only | `pip install torch --index-url https://download.pytorch.org/whl/cpu` |
-| AMD ROCm | `pip install torch --index-url https://download.pytorch.org/whl/rocm6.2` |
+| AMD ROCm | `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.3` |
 
 On Windows `.venv`, prefix with `.\.venv\Scripts\pip.exe`. Conda users with CUDA
 torch already installed can skip this step.
 
+### Linux AMD ROCm GPU install
+
+Use this when the node machine has an AMD GPU/APU and you want PyTorch to run on
+ROCm instead of CPU. The Python wheel is not enough by itself: the host must have
+working AMD GPU device access and a compatible ROCm runtime.
+
+**Host prerequisites:**
+
+1. Confirm the AMD GPU is visible:
+
+```bash
+lspci | grep -Ei 'vga|3d|display|amd|ati'
+ls -l /dev/kfd /dev/dri/renderD* 2>/dev/null
+```
+
+2. Make sure the node user can access GPU devices. AMD ROCm documents the normal
+   Linux permission path as membership in both `video` and `render`:
+
+```bash
+groups
+sudo usermod -a -G video,render "$LOGNAME"
+# log out and back in before continuing
+```
+
+3. Confirm the ROCm runtime tools work if they are installed:
+
+```bash
+rocminfo | head
+```
+
+If `rocminfo` is missing or cannot see the GPU, fix the host ROCm install first.
+Do not debug `meshnet-node` until this works.
+
+**Install ROCm PyTorch into the node env:**
+
+```bash
+cd /path/to/neuron-tai
+python3 -m venv .venv-rocm
+source .venv-rocm/bin/activate
+python -m pip install --upgrade pip setuptools wheel
+python -m pip install -e packages/tracker -e packages/node -e packages/p2p -e packages/gateway -e packages/relay
+python -m pip install "transformers>=5.12" accelerate safetensors
+python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.3
+```
+
+Keep this separate from a known-good CPU `.venv` until ROCm is verified on that
+machine. ROCm wheels are large and host-runtime-sensitive; a failed ROCm install
+should not break the CPU fallback environment.
+
+**Verify PyTorch sees ROCm:**
+
+```bash
+python - <<'PY'
+import torch
+print("torch", torch.__version__)
+print("hip", torch.version.hip)
+print("cuda api available", torch.cuda.is_available())
+if torch.cuda.is_available():
+    print("device", torch.cuda.get_device_name(0))
+    x = torch.ones((1,), device="cuda")
+    torch.cuda.synchronize()
+    print("tensor", x)
+PY
+```
+
+Expected: `torch.version.hip` is not `None`, `torch.cuda.is_available()` is
+`True`, and the tensor allocation succeeds. PyTorch intentionally exposes ROCm
+through the `torch.cuda` API.
+
+**Start an AMD ROCm node:**
+
+```bash
+HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
+  --tracker <tracker-url> \
+  --model Qwen/Qwen2.5-0.5B-Instruct \
+  --quantization bfloat16
+```
+
+For the Qwen3.6 alpha model on Linux ROCm, install the optional FLA ROCm fast
+path in the same env:
+
+```bash
+.venv-rocm/bin/pip install 'flash-linear-attention[rocm]'
+HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
+  --tracker <tracker-url> \
+  --model qwen3.6-35b-a3b \
+  --quantization bfloat16
+```
+
+**Troubleshooting notes:**
+
+- `torch.version.hip is None` means you installed a CPU/CUDA torch build, not ROCm.
+- `torch.cuda.is_available() == False` with a ROCm build usually means host driver,
+  permissions, unsupported hardware, or missing runtime libraries.
+- Missing libraries such as `libamdhip64.so`, `libMIOpen.so`, `librocsolver.so`,
+  or `libroctx64.so` are host ROCm runtime problems, not meshnet-node problems.
+- Some AMD APUs and consumer GPUs require newer ROCm/Radeon support than server
+  Instinct cards. Check AMD's ROCm Radeon/Ryzen support matrix for the exact model.
+
 ### Qwen3.5/3.6-MoE notes
 
 Applies to **`qwen3.6-35b-a3b`** and other hybrid linear-attention models. **`Qwen2.5-0.5B`**
@@ -355,13 +456,20 @@ meshnet-node start --tracker http://192.168.0.179:8080 --model qwen3.6-35b-a3b -
 
 Do not add `causal-conv1d` or `flash-linear-attention[cuda]` on Windows (see Qwen3.5/3.6 notes).
 
-**Alpha model (Qwen3.6, Linux GPU — with fast path):**
+**Alpha model (Qwen3.6, Linux NVIDIA GPU — with fast path):**
 
 ```bash
 HF_HOME=/path/to/models .venv/bin/meshnet-node start --tracker <tracker-url> --model qwen3.6-35b-a3b --quantization bfloat16
 # Install once on that machine: pip install flash-linear-attention[cuda]
 ```
 
+**Alpha model (Qwen3.6, Linux AMD ROCm GPU — with fast path):**
+
+```bash
+HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start --tracker <tracker-url> --model qwen3.6-35b-a3b --quantization bfloat16
+# Install once on that machine: .venv-rocm/bin/pip install 'flash-linear-attention[rocm]'
+```
+
 After the first node registers a model, later nodes can join with only the tracker
 URL (shard auto-assigned):