diff --git a/QUICKSTART.md b/QUICKSTART.md index eabb88f..e7959a6 100644 --- a/QUICKSTART.md +++ b/QUICKSTART.md @@ -144,11 +144,14 @@ does not need any of this — it is a standard transformer with no FLA fast path `python -c "import transformers; print(transformers.__version__)"`. - **GPU fast path (optional)** — without it inference still works; startup prints `The fast path is not available…` and linear-attention layers use a slower PyTorch - fallback. Install **only for your platform**: + fallback. **The fast path runs on NVIDIA CUDA GPUs on both Linux and native + Windows** — the FLA kernels are Triton-compiled, and `triton-windows` compiles them + for CUDA on Windows just like Linux Triton does. Only the pip command differs per + platform. Install **only for your platform**: | Platform | Install | Notes | |----------|---------|-------| - | **Native Windows + NVIDIA** | `pip install triton-windows` then `pip install flash-linear-attention` | **Fast path works.** FLA [officially supports `triton-windows`](https://github.com/fla-org/flash-linear-attention/pull/757) (tested Win11, PyTorch 2.10, triton-windows 3.6). Do **not** use the `[cuda]` extra on Windows — pip looks for Linux `triton` and fails. Do **not** install `causal-conv1d` — FLA ≥0.3.2 ships Triton conv1d; the separate package is Linux-only and breaks on Windows (`bare_metal_version` / nvcc errors). | + | **Native Windows + NVIDIA CUDA** | `pip install triton-windows` then `pip install flash-linear-attention` | **Fast path works on the CUDA GPU** — no CUDA toolkit / `nvcc` needed; `triton-windows` bundles its own compiler. FLA [officially supports `triton-windows`](https://github.com/fla-org/flash-linear-attention/pull/757) (tested Win11, PyTorch 2.10, triton-windows 3.6). Do **not** use the `[cuda]` extra on Windows — that extra only pins Linux PyPI `triton` and fails; it is a packaging name, not a GPU requirement. Do **not** install `causal-conv1d` — FLA ≥0.3.2 ships Triton conv1d; the separate package is Linux-only and breaks on Windows (`bare_metal_version` / nvcc errors). | | **Linux + NVIDIA CUDA** | `pip install flash-linear-attention[cuda]` | `causal-conv1d` optional (same FLA built-in conv1d note). Needs CUDA toolkit (`nvcc`) matching torch, or a prebuilt wheel. | | **Linux + AMD ROCm** | `pip install flash-linear-attention[rocm]` | Same optional `causal-conv1d` note. | diff --git a/docs/INSTALL_WINDOWS.md b/docs/INSTALL_WINDOWS.md index b7730dc..d7ce3db 100644 --- a/docs/INSTALL_WINDOWS.md +++ b/docs/INSTALL_WINDOWS.md @@ -111,10 +111,10 @@ with `'Qwen3_5MoeConfig' object has no attribute 'vocab_size'`). If you install into an existing conda/miniforge env instead of a fresh venv, run `pip install -U transformers` there. The startup warning about `flash-linear-attention` / `causal-conv1d` ("fast path is not available") is -harmless on CPU — those are optional CUDA-only kernels. +harmless on CPU — those are optional GPU kernels. -If you run the node from native Windows instead of WSL2, install the Triton shim -in the same environment: +If you run the node from native Windows instead of WSL2, install Triton for +Windows in the same environment: ```powershell python -m pip install triton-windows @@ -123,6 +123,12 @@ python -m pip install triton-windows Without it, Qwen3.5/3.6-MoE startup can fail with the misleading message `Could not import module 'Qwen3_5MoeForCausalLM'`. +**NVIDIA GPU on native Windows:** the CUDA fast path works — after +`triton-windows`, install FLA with plain `pip install flash-linear-attention` +(no `[cuda]` extra, no `causal-conv1d`; both are Linux-only packaging and fail +on Windows). No CUDA toolkit / `nvcc` is needed. See the platform table in +[QUICKSTART.md](../QUICKSTART.md#qwen3536-moe-notes) for details. + --- ## Step 6 — Pre-download the model shard diff --git a/packages/node/meshnet_node/startup.py b/packages/node/meshnet_node/startup.py index c119474..5f6730e 100644 --- a/packages/node/meshnet_node/startup.py +++ b/packages/node/meshnet_node/startup.py @@ -829,7 +829,8 @@ def run_startup( shard_label = _format_shard_label(shard_start, shard_end, total_layers) public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host) endpoint = f"http://{public_host}:{actual_port}" - node.set_advertised_endpoint(endpoint) + if hasattr(node, "set_advertised_endpoint"): + node.set_advertised_endpoint(endpoint) local_base_url = f"http://127.0.0.1:{actual_port}" relay_bridge, relay_fields = _start_relay_bridge_if_available( tracker_url, @@ -978,7 +979,8 @@ def run_startup( actual_port = node.start() public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host) endpoint = f"http://{public_host}:{actual_port}" - node.set_advertised_endpoint(endpoint) + if hasattr(node, "set_advertised_endpoint"): + node.set_advertised_endpoint(endpoint) local_base_url = f"http://127.0.0.1:{actual_port}" relay_bridge, relay_fields = _start_relay_bridge_if_available( tracker_url, @@ -1156,7 +1158,8 @@ def run_startup( shard_label = f"{shard_label} (pinned)" public_host = advertise_host or (socket.getfqdn() if host == "0.0.0.0" else host) endpoint = f"http://{public_host}:{actual_port}" - node.set_advertised_endpoint(endpoint) + if hasattr(node, "set_advertised_endpoint"): + node.set_advertised_endpoint(endpoint) local_base_url = f"http://127.0.0.1:{actual_port}" relay_bridge, relay_fields = _start_relay_bridge_if_available( tracker_url,