wip -more responsive UI, better routing
This commit is contained in:
@@ -38,12 +38,30 @@ python3 -m venv .venv
|
||||
`python -c "import transformers; print(transformers.__version__)"` and upgrade
|
||||
with `pip install -U transformers` in the environment that runs `meshnet-node`
|
||||
(conda/miniforge users: upgrade inside that env, not a layered `.venv`).
|
||||
- The startup warning
|
||||
`The fast path is not available because one of the required library is not installed`
|
||||
is **harmless** — transformers falls back to a pure-torch implementation of the
|
||||
linear-attention layers. The fast-path packages (`flash-linear-attention`,
|
||||
`causal-conv1d`) are CUDA-only kernels: install them for GPU speed if you want,
|
||||
skip them entirely on CPU nodes.
|
||||
- **Linear-attention fast path (GPU only).** Qwen3.5/3.6 use hybrid linear-attention
|
||||
layers; without optional CUDA kernels, Transformers falls back to slower pure-PyTorch
|
||||
code and prints `The fast path is not available…` at startup. That warning is
|
||||
harmless — inference still works. On native Windows, install `triton-windows` in
|
||||
the same env as `meshnet-node`; otherwise `flash-linear-attention` can fail during
|
||||
import with `Could not import module 'Qwen3_5MoeForCausalLM'`. Install the
|
||||
acceleration packages into the same env as `meshnet-node` for GPU speed; skip on
|
||||
CPU-only nodes:
|
||||
|
||||
```bash
|
||||
# Native Windows
|
||||
pip install triton-windows
|
||||
|
||||
# NVIDIA (CUDA)
|
||||
pip install flash-linear-attention[cuda] causal-conv1d
|
||||
|
||||
# AMD (ROCm) — match your torch index, then:
|
||||
pip install flash-linear-attention[rocm] causal-conv1d
|
||||
```
|
||||
|
||||
Restart the node after install; the warning should disappear. Expect the largest
|
||||
gain on GPU nodes serving linear-attention layers (roughly three quarters of
|
||||
Qwen3.6 layers); end-to-end chat speed still depends on the slowest hop in a
|
||||
split route.
|
||||
- `pip install nvidia-ml-py` silences the pynvml deprecation warning on NVIDIA hosts.
|
||||
|
||||
## Bootstrap a tracker on a new machine
|
||||
|
||||
Reference in New Issue
Block a user