popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

md

Browse Source
This commit is contained in:
Dobromir Popov
2026-07-08 20:01:31 +02:00
parent 1ecc599f7f
commit 0b39d80375
3 changed files with 20 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
QUICKSTART.md
View File

@@ -144,11 +144,14 @@ does not need any of this — it is a standard transformer with no FLA fast path
`python -c "import transformers; print(transformers.__version__)"`.
- **GPU fast path (optional)** — without it inference still works; startup prints
`The fast path is not available…` and linear-attention layers use a slower PyTorch
fallback. Install **only for your platform**:
fallback. **The fast path runs on NVIDIA CUDA GPUs on both Linux and native
Windows** — the FLA kernels are Triton-compiled, and `triton-windows` compiles them
for CUDA on Windows just like Linux Triton does. Only the pip command differs per
platform. Install **only for your platform**:
| Platform | Install | Notes |
|----------|---------|-------|
| **Native Windows + NVIDIA** | `pip install triton-windows` then `pip install flash-linear-attention` | **Fast path works.** FLA [officially supports `triton-windows`](https://github.com/fla-org/flash-linear-attention/pull/757) (tested Win11, PyTorch 2.10, triton-windows 3.6). Do **not** use the `[cuda]` extra on Windows — pip looks for Linux `triton` and fails. Do **not** install `causal-conv1d` — FLA ≥0.3.2 ships Triton conv1d; the separate package is Linux-only and breaks on Windows (`bare_metal_version` / nvcc errors). |
| **Native Windows + NVIDIA CUDA** | `pip install triton-windows` then `pip install flash-linear-attention` | **Fast path works on the CUDA GPU** — no CUDA toolkit / `nvcc` needed; `triton-windows` bundles its own compiler. FLA [officially supports `triton-windows`](https://github.com/fla-org/flash-linear-attention/pull/757) (tested Win11, PyTorch 2.10, triton-windows 3.6). Do **not** use the `[cuda]` extra on Windows — that extra only pins Linux PyPI `triton` and fails; it is a packaging name, not a GPU requirement. Do **not** install `causal-conv1d` — FLA ≥0.3.2 ships Triton conv1d; the separate package is Linux-only and breaks on Windows (`bare_metal_version` / nvcc errors). |
| **Linux + NVIDIA CUDA** | `pip install flash-linear-attention[cuda]` | `causal-conv1d` optional (same FLA built-in conv1d note). Needs CUDA toolkit (`nvcc`) matching torch, or a prebuilt wheel. |
| **Linux + AMD ROCm** | `pip install flash-linear-attention[rocm]` | Same optional `causal-conv1d` note. |