popov/neuron-tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

qol improvements

Browse Source
This commit is contained in:
Dobromir Popov
2026-06-30 19:27:46 +02:00
parent d8a723a4c7
commit f1e4870124
4 changed files with 317 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
QUICKSTART.md
View File

@@ -129,8 +129,7 @@ $env:HF_HOME = "D:\DEV\models"
One-line variants:
```powershell
// only this works - when not behind NAT(wsl), and via IP. Revisit when relaying/RPC is implemented
.\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
.\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
.\.venv\Scripts\meshnet-node.exe start --tracker http://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
```
@@ -186,6 +185,22 @@ Then a node only needs the public tracker address:
--model Qwen/Qwen2.5-0.5B-Instruct
```
No authentication is required in the prototype. The first public node for a model
must still choose that model with `--model` or a saved wizard config. After at
least one HF model node is registered, later nodes can join the public network
with only the tracker URL; the tracker assigns an uncovered shard if one exists:
```bash
.venv/bin/meshnet-node start --tracker https://ai.neuron.d-popov.com
```
Use the public tracker to verify registration and routing:
```bash
curl -s "https://ai.neuron.d-popov.com/v1/network/map" | python3 -m json.tool
curl -s "https://ai.neuron.d-popov.com/v1/route?model=qwen2.5-0.5b" | python3 -m json.tool
```
---
## Step 1 — Start the tracker (Terminal 1)
@@ -258,6 +273,9 @@ For gated models (Llama), run `huggingface-cli login` first.
## Step 3 — Send an inference request (Terminal 3)
If you started the node with `--port 8001`, send the request directly to that
head node:
```bash Qwen2.5-0.5B-Instruct
curl -s http://localhost:8001/v1/chat/completions \
-H "Content-Type: application/json" \
@@ -268,6 +286,22 @@ curl -s http://localhost:8001/v1/chat/completions \
}' | python3 -m json.tool
```
If you did not pass `--port`, `meshnet-node start` uses the first free port at
or above `7000`. Use the `Endpoint:` printed by the node instead of `8001`.
To test tracker routing/proxying, send the same OpenAI-compatible request to the
tracker, using either the full HuggingFace repo or the quick alias:
```bash
curl -s http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen2.5-0.5b",
"messages": [{"role": "user", "content": "What is 7 times 8? Answer in one word."}],
"stream": false
}' | python3 -m json.tool
```
Or use the test script:
```bash
@@ -313,6 +347,12 @@ activations to Node B, and streams the final response back.
See `docs/TWO_MACHINE_TEST.md` (created by US-018).
For WSL2 nodes, registration only proves the node can reach the tracker
outbound. Tracker-routed inference also requires the tracker to reach the node's
advertised endpoint inbound. Either run the node in native Windows PowerShell,
configure Windows port forwarding into WSL for the node port, or start the
tracker with a relay URL so the node registers a `relay_addr`.
---
## Browse available models