install jit on liunux fedora, TPS in chat

This commit is contained in:
Dobromir Popov
2026-07-10 01:08:26 +03:00
parent 0195ba08e3
commit 23dd0c3219
4 changed files with 100 additions and 15 deletions

View File

@@ -244,6 +244,38 @@ HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
--quantization bfloat16
```
### Linux ROCm: Triton JIT compiler prerequisite
Some model/runtime paths invoke Triton at the first real forward. Triton builds a local HIP
support module before that kernel can run, so a host C compiler must be discoverable on
`PATH`. A successful PyTorch allocation or Node startup does not prove this prerequisite;
without it, the first `/forward` can fail with `Failed to find C compiler`.
On Fedora, install Clang once:
```bash
sudo dnf install -y clang
```
Restart the Node from a shell where `clang` is on `PATH`. If a custom shell or service does
not inherit the system path, set the compiler explicitly for that launch:
```bash
CC=/usr/bin/clang CXX=/usr/bin/clang++ \
HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
--tracker <tracker-url> --model <selected-model>
```
`CC`/`CXX` are normally unnecessary after Clang is installed; they are a diagnostic override,
not a Python dependency. Other Linux distributions should install their system `clang` package
through the OS package manager.
**Windows NVIDIA/Triton:** use native PowerShell and install `triton-windows`, then install or
upgrade `flash-linear-attention` when the selected model uses it. `triton-windows` supplies the
supported Windows compiler path; do not apply Linux `dnf`/`CC` instructions to Windows. If a
Windows Node still reports a compiler error, capture `python -c "import triton; print(triton.__version__)"`
and the exact error before installing arbitrary CUDA toolkits or `causal-conv1d`.
**Troubleshooting notes:**
- `torch.version.hip is None` means you installed a CPU/CUDA torch build, not ROCm.
@@ -261,6 +293,9 @@ HF_HOME=/path/to/models .venv-rocm/bin/meshnet-node start \
your GPU arch. Compare `torch.cuda.get_device_properties(0).gcnArchName`
against `torch.cuda.get_arch_list()`; if your arch is missing, install a wheel
built for it (see the Strix Halo/gfx1151 note above).
- `Failed to find C compiler` during a real forward is a Triton JIT host-toolchain failure.
On Fedora install `clang` as shown above, confirm `command -v clang`, and restart the Node;
it is separate from ROCm driver and device-access troubleshooting.
### Qwen3.5/3.6-MoE notes