popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[verified] feat: complete Ralph task workstreams

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-12 11:17:03 +03:00
parent 9a1b15c020
commit 377346c301
37 changed files with 5862 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files

130
QUICKSTART.md
View File

@@ -7,7 +7,15 @@ Tested on: AMD Ryzen AI Max (Strix Halo APU), 124 GB RAM, Linux CPU inference.
ROCm GPU setup is covered below, but must be verified on the host because ROCm
support depends on the exact AMD GPU/APU, kernel, driver, and ROCm runtime.
**Active development models** (what we run day-to-day):
**Support here is evidence, not a promise.** No combination of GPU, torch build,
optional accelerator package and model is "supported" until `meshnet-node doctor`
has pushed a real forward through the exact shard you intend to serve, on that
machine. Detecting a GPU proves nothing; installing an optional package proves
nothing. See [Validate before you serve](#validate-before-you-serve-meshnet-node-doctor).
**Active development models** — what we run day-to-day, so their setup notes are the
most exercised. This is *not* a support matrix: any model the node can load is fair
game, and any model on this list can still fail `doctor` on your hardware.
| Role | `--model` / alias | HF repo | Notes |
|------|-------------------|---------|-------|
@@ -288,8 +296,8 @@ HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
--quantization bfloat16
```
For the Qwen3.6 alpha model on Linux ROCm, install the optional FLA ROCm fast
path in the same env:
A model whose recipe uses an optional accelerator needs that package in the *same*
env. For example, `qwen3.6-35b-a3b` on Linux ROCm can use FLA:
```bash
.venv-rocm/bin/pip install 'flash-linear-attention[rocm]'
@@ -299,6 +307,12 @@ HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
--quantization bfloat16
```
This is a per-model, per-platform example. A successful `pip install` is not evidence
that the kernel runs on your GPU — optional-kernel support varies by architecture, and
we make no universal claim here. Prove it with
`.venv-rocm/bin/meshnet-node doctor --model <model>` before starting the node; startup
will refuse to register anyway if the shard cannot execute.
### Linux ROCm: Triton JIT compiler prerequisite
Some model/runtime paths invoke Triton at the first real forward. Triton builds a local HIP
@@ -530,8 +544,11 @@ curl -sI https://ai.neuron.d-popov.com/rpc/test-peer
no HuggingFace gating. Best for first-time setup.
**Alpha model:** `qwen3.6-35b-a3b` — 40 layers, ~72 GB BF16 download, MoE with hybrid
linear attention. On Windows install `triton-windows` + `flash-linear-attention`; on Linux
GPU use `flash-linear-attention[cuda]`. Tracker accepts the alias or full repo id (`unsloth/Qwen3.6-35B-A3B`).
linear attention. Tracker accepts the alias or full repo id (`unsloth/Qwen3.6-35B-A3B`).
Some models have an *optional* accelerator path (for this one, Triton +
`flash-linear-attention`); those installs are per-model, per-platform examples, not a
requirement the node imposes and not a guarantee the path will work — see
[Optional accelerator packages](#optional-accelerator-packages).
Downloads cache under `~/.meshnet/models/` (or `$HF_HOME` / `$env:HF_HOME`).
@@ -539,6 +556,79 @@ Shard range is auto-detected from the curated catalog. For unknown repos the nod
fetches only `config.json`. Override with `--shard-start` / `--shard-end` for partial
shards or multi-node splits.
### Validate before you serve: `meshnet-node doctor`
`doctor` resolves exactly the model, shard, quantization and recipe that `start` would
load, loads it, and pushes one bounded **real forward** through it. It is offline — it
never contacts the tracker — and it is the only thing that turns a guess into evidence.
```bash
HF_HOME=/path/to/models .venv/bin/meshnet-node doctor --model <model> --quantization bfloat16
```
```
meshnet-node doctor
Model: <model>
Shard: layers 0–23; 24 of 24
Quantization: bfloat16
[PASS] recipe default (v1) on cuda — 412 ms
OK — the selected shard ran a real forward for 1 recipe.
Capability report: ~/.meshnet/capability.json
```
Exit code is 0 on pass, 1 on failure. It writes a **capability report** either way — a
failure is evidence too. Useful flags: `--recipe <id>` to validate one named recipe,
`--all-recipes` to try every recipe for the selection (CI and diagnosis), `--report PATH`
to choose where the report lands, `--json` for machine-readable output, `--debug` for the
full traceback behind a failure. The selection flags (`--model`, `--quantization`,
`--shard-start`/`--shard-end`, `--cpu`) work the same as on `start`.
**Three states — do not confuse them:**
| State | What it means | How it is established |
|-------|---------------|-----------------------|
| **Detected hardware** | A GPU, a torch build, and an optional package are *present*. | Inventory/`torch.cuda.is_available()`. Proves nothing about whether your shard runs. |
| **Validated recipe** | This machine executed a real forward for *this* model + shard + recipe + device. | A passing `doctor` run → a capability report. |
| **Routable Node** | The tracker will send paid work here. | Startup re-proves the loaded shard and registers the report; the tracker admits it. |
Each state is strictly stronger than the one above it. A machine can have a working GPU
(detected) and still fail to execute a shard (no validated recipe), and a node can hold a
valid report and still not route (the tracker moved it to a shard range it never proved).
**Startup is fail-closed.** `meshnet-node start` runs the same validation against the
backend it just loaded, *before* it opens a port or registers. If the shard cannot execute,
the node exits non-zero — it never binds a socket and never registers. There is no
production bypass; a node that cannot do the work never advertises that it can.
**The tracker admits, it does not re-run.** The node ships its report with registration.
Registration always succeeds, so a broken node is visible rather than invisible — but only
a report that *covers what the node advertises* makes it routable. The verdict comes back
in the registration response, is logged, and is exposed per node on `GET /v1/network/map`
under `capability` (`admitted`, `absent`, `invalid`, `failed`, `stale`, `model-mismatch`,
`shard-mismatch`, `recipe-mismatch`, `catalogue-incompatible`). A node that is registered
but dark is showing you one of those. Full semantics, and the transitional `compat` vs
`enforce` policy, are in [ADR-0023](docs/adr/0023-model-agnostic-node-capability-admission.md).
**When `doctor` fails** it prints a category and an actionable hint, not a traceback:
| Category | Do this |
|----------|---------|
| `no-model-selected` | Pass `--model`, or run `meshnet-node` once to save a config. |
| `missing-dependency` | Install the node's model extras (torch, transformers, safetensors, accelerate). |
| `model-unavailable` | Check the model id, `--download-dir`, and that the artifact is downloaded/reachable. |
| `insufficient-memory` | Serve fewer layers (`--shard-start`/`--shard-end`) or a smaller quantization (`-q int8`, `-q nf4`). |
| `invalid-shard` | The layer range does not exist in this model — check it against the layer count. |
| `unsupported-recipe` | This backend cannot apply that execution setting; pick another `--recipe`. |
| `load-failed` | The shard would not load; re-run with `--debug`. |
| `forward-failed` | It loaded but cannot execute. This machine cannot serve this shard; re-run with `--debug`. |
**Scope:** the node validates recipes it already ships. It does **not** download executable
recipes, install Python/OS packages, or update drivers for you — every install on this page
is something you run deliberately. Signed node updates are a deliberate follow-up, not part
of this release.
### Core command
Replace `<tracker-url>` and adjust the prefix for your shell (see table above).
@@ -570,31 +660,45 @@ HF_HOME=/path/to/models .venv/bin/meshnet-node start --tracker http://localhost:
.venv/bin/meshnet-node start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
```
**Alpha model (Qwen3.6, Windows GPU — enable fast path):**
#### Optional accelerator packages
Some models have an optional fast path that a *specific* model's recipe can use if the
package is importable. The examples below are exactly that — **worked examples for one
development model on one platform**, not a supported-configuration list. Installing the
package does not mean the fast path works on your GPU: a package can import cleanly and
still fail to execute a kernel on your architecture. `doctor` is what tells you, and the
node serves the recipe that actually validated. If none of this is installed, the model
still runs on the standard path.
**Example — `qwen3.6-35b-a3b` (Triton + FLA), Windows GPU:**
```powershell
$env:HF_HOME = "D:\DEV\models"
pip install triton-windows
pip install -U flash-linear-attention
meshnet-node doctor --model qwen3.6-35b-a3b --quantization bfloat16
meshnet-node start --tracker http://192.168.0.179:8080 --model qwen3.6-35b-a3b --quantization bfloat16
```
Do not add `causal-conv1d` or `flash-linear-attention[cuda]` on Windows (see Qwen3.5/3.6 notes).
**Alpha model (Qwen3.6, Linux NVIDIA GPU — with fast path):**
**Example — `qwen3.6-35b-a3b`, Linux NVIDIA GPU:**
```bash
HF_HOME=/path/to/models .venv/bin/meshnet-node start --tracker <tracker-url> --model qwen3.6-35b-a3b --quantization bfloat16
# Install once on that machine: pip install flash-linear-attention[cuda]
HF_HOME=/path/to/models .venv/bin/meshnet-node start --tracker <tracker-url> --model qwen3.6-35b-a3b --quantization bfloat16
```
**Alpha model (Qwen3.6, Linux AMD ROCm GPU — with fast path):**
**Example — `qwen3.6-35b-a3b`, Linux AMD ROCm GPU:**
```bash
HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start --tracker <tracker-url> --model qwen3.6-35b-a3b --quantization bfloat16
# Install once on that machine: .venv-rocm/bin/pip install 'flash-linear-attention[rocm]'
HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start --tracker <tracker-url> --model qwen3.6-35b-a3b --quantization bfloat16
```
Run `doctor` after any of these installs: it is the only way to learn whether the optional
path executes here, and it costs one bounded forward.
After the first node registers a model, later nodes can join with only the tracker
URL (shard auto-assigned):
@@ -786,3 +890,9 @@ failure the node logs a warning and falls back to direct HTTP before erroring.
```bash
.venv/bin/python -m pytest -q
```
Tests marked `integration` download models or need a GPU; the default lane is
`pytest -m "not integration"`. The opt-in real-model doctor check takes its model from the
environment and skips without it — see
[docs/dev/certified-hardware-lanes.md](docs/dev/certified-hardware-lanes.md) for the
release contract that certifies a hardware lane.