wip -more responsive UI, better routing

This commit is contained in:
Dobromir Popov
2026-07-08 09:07:54 +02:00
parent 518c259cd3
commit 3d82188dc1
14 changed files with 506 additions and 39 deletions

View File

@@ -38,12 +38,30 @@ python3 -m venv .venv
`python -c "import transformers; print(transformers.__version__)"` and upgrade
with `pip install -U transformers` in the environment that runs `meshnet-node`
(conda/miniforge users: upgrade inside that env, not a layered `.venv`).
- The startup warning
`The fast path is not available because one of the required library is not installed`
is **harmless** — transformers falls back to a pure-torch implementation of the
linear-attention layers. The fast-path packages (`flash-linear-attention`,
`causal-conv1d`) are CUDA-only kernels: install them for GPU speed if you want,
skip them entirely on CPU nodes.
- **Linear-attention fast path (GPU only).** Qwen3.5/3.6 use hybrid linear-attention
layers; without optional CUDA kernels, Transformers falls back to slower pure-PyTorch
code and prints `The fast path is not available…` at startup. That warning is
harmless — inference still works. On native Windows, install `triton-windows` in
the same env as `meshnet-node`; otherwise `flash-linear-attention` can fail during
import with `Could not import module 'Qwen3_5MoeForCausalLM'`. Install the
acceleration packages into the same env as `meshnet-node` for GPU speed; skip on
CPU-only nodes:
```bash
# Native Windows
pip install triton-windows
# NVIDIA (CUDA)
pip install flash-linear-attention[cuda] causal-conv1d
# AMD (ROCm) — match your torch index, then:
pip install flash-linear-attention[rocm] causal-conv1d
```
Restart the node after install; the warning should disappear. Expect the largest
gain on GPU nodes serving linear-attention layers (roughly three quarters of
Qwen3.6 layers); end-to-end chat speed still depends on the slowest hop in a
split route.
- `pip install nvidia-ml-py` silences the pynvml deprecation warning on NVIDIA hosts.
## Bootstrap a tracker on a new machine