dash works !!! good data. billing seems to work
This commit is contained in:
Dobromir Popov
171
docs/DEPLOY_PORTAINER.md
Normal file
171
docs/DEPLOY_PORTAINER.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# One-shot Portainer deploy: public tracker + relay
|
||||
|
||||
No Dockerfile, no custom image, no registry. Portainer runs `python:3.12-slim`, downloads this repo tarball, installs tracker/relay, and starts both services.
|
||||
|
||||
## 1. One-time host setup
|
||||
|
||||
DNS points to the Docker host, e.g. `cloud.neuron.d-popov.com`.
|
||||
Firewall exposes only `80`, `443`, and admin SSH.
|
||||
Nginx Proxy Manager and this stack share a Docker network, e.g. `npm_proxy`.
|
||||
|
||||
If needed:
|
||||
|
||||
```bash
|
||||
docker network create npm_proxy
|
||||
docker network connect npm_proxy <nginx-proxy-manager-container>
|
||||
```
|
||||
|
||||
## 2. Portainer stack
|
||||
|
||||
Portainer → Stacks → Add stack → Repository.
|
||||
|
||||
Stack file:
|
||||
|
||||
```text
|
||||
deploy/portainer/meshnet-tracker-nobuild-stack.yml
|
||||
```
|
||||
|
||||
Deploy only after the current branch has been pushed. The stack passes the
|
||||
current tracker billing flags (`--billing-db`, and optional treasury flags), so
|
||||
an older remote `master.tar.gz` will crash on startup with unrecognized
|
||||
arguments.
|
||||
|
||||
Variables:
|
||||
|
||||
```text
|
||||
SOURCE_TARBALL_URL=https://git.d-popov.com/<owner>/<repo>/archive/master.tar.gz
|
||||
PUBLIC_TRACKER_URL=https://cloud.neuron.d-popov.com
|
||||
PUBLIC_PROXY_NETWORK=npm_proxy
|
||||
CLUSTER_PEERS=https://ai.neuron.d-popov.com
|
||||
PUBLIC_RELAY_URL=wss://cloud.neuron.d-popov.com/ws
|
||||
HEARTBEAT_TIMEOUT=30
|
||||
ENABLE_BILLING_DB=1
|
||||
```
|
||||
|
||||
For first cloud-only test, use `CLUSTER_PEERS=`. Click **Deploy the stack**.
|
||||
|
||||
`ENABLE_BILLING_DB=1` makes billing public behavior active: `/v1/chat/completions`
|
||||
requires `Authorization: Bearer <api-key>`. Any key is accepted and starts with
|
||||
the configured starter credit; calls return `402` after that balance is
|
||||
exhausted. Set `ENABLE_BILLING_DB=0` if existing unauthenticated clients must
|
||||
keep working during the first redeploy.
|
||||
|
||||
Optional Solana treasury settlement variables:
|
||||
|
||||
```text
|
||||
SOLANA_RPC_URL=https://api.devnet.solana.com
|
||||
USDT_MINT=<mock-usdt-mint-from-scripts/devnet_setup.py>
|
||||
MESHNET_TREASURY_KEYPAIR_B64=<base64 of solana keypair JSON>
|
||||
SETTLE_PERIOD=86400
|
||||
PAYOUT_THRESHOLD=5
|
||||
PAYOUT_DUST_FLOOR=0.01
|
||||
```
|
||||
|
||||
Set all three treasury identity values together (`SOLANA_RPC_URL`, `USDT_MINT`,
|
||||
`MESHNET_TREASURY_KEYPAIR_B64`) or leave all three empty. The stack writes the
|
||||
decoded keypair into the tracker data volume at startup and passes it to
|
||||
`meshnet-tracker`; do not put this key on relay-only hosts or non-settlement
|
||||
trackers.
|
||||
|
||||
Expected containers:
|
||||
|
||||
```text
|
||||
meshnet-tracker internal 8081
|
||||
meshnet-relay internal 8765
|
||||
```
|
||||
|
||||
## 3. Nginx Proxy Manager
|
||||
|
||||
One Proxy Host for `cloud.neuron.d-popov.com`:
|
||||
|
||||
```text
|
||||
Forward Hostname/IP: meshnet-tracker
|
||||
Forward Port: 8081
|
||||
Websockets Support: ON
|
||||
SSL: Let's Encrypt + Force SSL
|
||||
```
|
||||
|
||||
Custom locations on the same proxy host:
|
||||
|
||||
```text
|
||||
/ws -> http://meshnet-relay:8765
|
||||
/rpc -> http://meshnet-relay:8765
|
||||
```
|
||||
|
||||
Leave sub-folder forwarding empty.
|
||||
|
||||
If WebSockets fail, Advanced:
|
||||
|
||||
```nginx
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $http_connection;
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
```
|
||||
|
||||
## 4. Smoke test
|
||||
|
||||
```bash
|
||||
curl -s https://cloud.neuron.d-popov.com/v1/health
|
||||
curl -s https://cloud.neuron.d-popov.com/v1/network/map | python3 -m json.tool
|
||||
curl -s https://cloud.neuron.d-popov.com/v1/raft/status | python3 -m json.tool
|
||||
```
|
||||
|
||||
Plain curl to `/ws` or `/rpc/test-peer` may show `426 Upgrade Required`; OK. It must not show nginx `502`.
|
||||
|
||||
Dashboard:
|
||||
|
||||
```bash
|
||||
curl -s https://cloud.neuron.d-popov.com/dashboard | head
|
||||
```
|
||||
|
||||
Then open:
|
||||
|
||||
```text
|
||||
https://cloud.neuron.d-popov.com/dashboard
|
||||
```
|
||||
|
||||
The dashboard is served by the tracker through the same proxy host; no extra NPM
|
||||
location is required.
|
||||
|
||||
If you previously deployed the build-image variant before `/var/lib/meshnet`
|
||||
was created as the `meshnet` user, the named volume may already be root-owned.
|
||||
Recreate that volume or chown it once before retrying the fixed image.
|
||||
|
||||
## 5. Start a node
|
||||
|
||||
`meshnet-node` is the worker/miner process. It will ask for a model and load a
|
||||
model shard. Do not use it to test the tracker dashboard.
|
||||
|
||||
```bash
|
||||
meshnet-node start --tracker https://cloud.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
Relay advertised by tracker — using outbound tunnel wss://cloud.neuron.d-popov.com/ws
|
||||
Relay connected — wss://cloud.neuron.d-popov.com/rpc/<peer_id>
|
||||
```
|
||||
|
||||
## 6. Two tracker sync
|
||||
|
||||
Cloud stack:
|
||||
|
||||
```text
|
||||
PUBLIC_TRACKER_URL=https://cloud.neuron.d-popov.com
|
||||
CLUSTER_PEERS=https://ai.neuron.d-popov.com
|
||||
PUBLIC_RELAY_URL=wss://cloud.neuron.d-popov.com/ws
|
||||
```
|
||||
|
||||
Local tracker:
|
||||
|
||||
```bash
|
||||
meshnet-tracker start --host 0.0.0.0 --port 8081 \
|
||||
--self-url https://ai.neuron.d-popov.com \
|
||||
--cluster-peers https://cloud.neuron.d-popov.com \
|
||||
--relay-url wss://ai.neuron.d-popov.com/ws
|
||||
```
|
||||
|
||||
Two Raft peers are enough for sync testing; real HA needs three trackers.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Coverage-first shard assignment and tracker-as-first-layer-node
|
||||
# Coverage-first shard assignment and tracker-routed inference
|
||||
|
||||
The tracker assigns shard ranges to nodes using a coverage-first, speed-weighted bin-packing algorithm. Tracker nodes must host at least the first layer shard of every model they coordinate, making them the natural inference entry point. Any node serving layers[0..k] can become a tracker node for that model.
|
||||
The tracker assigns shard ranges to worker nodes using a coverage-first, speed-weighted bin-packing algorithm. The tracker is a control-plane service and public inference API endpoint: it stores registry state, selects routes, enforces billing, and proxies OpenAI-compatible requests to the selected head worker. It does not download or load model weights.
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -23,24 +23,29 @@ Example: 700B NF4 model (~350GB weights). Node A has 128GB, Node B and C each ha
|
||||
- Node C gets layers[k_b..N] (the remainder)
|
||||
- If Node B benchmarks 2× faster than Node C, the tracker shifts the B/C boundary so B carries more layers
|
||||
|
||||
### Tracker-as-first-layer-node
|
||||
### Tracker-routed head worker
|
||||
|
||||
Any node that advertises a new model to the network becomes a **tracker node** for that model. Tracker nodes have one hard requirement: they must hold and serve `layers[0..k]` (the first-layer shard) for every model they coordinate.
|
||||
A worker that serves `layers[0..k]` is the **head worker** for that model. The tracker forwards `/v1/chat/completions` to a live head worker and injects the remaining downstream route. The worker, not the tracker process:
|
||||
|
||||
The reason is functional: a tracker node is also the inference entry point. When a client request arrives, the tracker node:
|
||||
When a client request arrives, the tracker:
|
||||
1. Authenticates/bills the request
|
||||
2. Selects a live head worker and full downstream route from the coverage map
|
||||
3. Proxies the request to that head worker
|
||||
4. Records usage and credits node shares after completion
|
||||
|
||||
The head worker:
|
||||
1. Tokenizes the input (owns the tokenizer)
|
||||
2. Runs `model.embed_tokens` + `model.layers[0..k]`
|
||||
3. Selects the optimal onward route from the coverage map
|
||||
4. Forwards activations to the next node in the route
|
||||
5. Receives the final hidden state back and streams tokens to the client
|
||||
3. Forwards activations to the next node in the route
|
||||
4. Receives the final hidden state back and streams tokens to the client
|
||||
|
||||
This collapses the separate "gateway" role: the tracker node that starts an inference request IS the gateway for that request. A standalone HTTP proxy/load-balancer may sit in front to pick which tracker node handles the request, but it carries no model weights.
|
||||
This keeps the public tracker lightweight: a standalone HTTP proxy/load-balancer may sit in front to pick which tracker handles the request, but neither proxy nor tracker carries model weights.
|
||||
|
||||
Multiple tracker nodes for the same model = multiple entry points = horizontal scale for both routing decisions and the first-layer compute.
|
||||
Multiple head workers for the same model = multiple inference entry points = horizontal scale for first-layer compute. Multiple trackers scale routing and billing, not model execution.
|
||||
|
||||
### Last-layer node (tail)
|
||||
|
||||
The node assigned `layers[N-k..N]` also runs `model.norm` and `model.lm_head`. It returns decoded token IDs (not hidden states) to the tracker node, which assembles the response. The tail shard assignment is marked `is_tail: true` in the shard registry.
|
||||
The node assigned `layers[N-k..N]` also runs `model.norm` and `model.lm_head`. It returns decoded token IDs (not hidden states) to the head worker, which assembles the response. The tail shard assignment is marked `is_tail: true` in the shard registry.
|
||||
|
||||
### Adaptive quantization
|
||||
|
||||
@@ -78,8 +83,8 @@ Nodes obey directives asynchronously; the tracker waits up to a configurable tim
|
||||
|
||||
## Consequences
|
||||
|
||||
- The standalone `meshnet-gateway` service from US-005 becomes a thin load-balancer that routes to tracker nodes; tracker nodes do the actual inference orchestration
|
||||
- Tracker nodes must download more model data (tokenizer + first-layer shard) — this is the price of being an entry point
|
||||
- The standalone `meshnet-gateway` service from US-005 becomes a thin compatibility proxy/load-balancer; the public tracker can also serve the OpenAI-compatible endpoint directly
|
||||
- Tracker processes do not download or load model data. Only worker nodes load model shards.
|
||||
- Benchmark data is self-reported by nodes at registration; the validator can detect fraudulent benchmarks (a node claiming 100 tokens/sec but delivering 2 gets slashed for under-performance)
|
||||
- VRAM reservation for KV cache means nodes can host fewer layers than their raw VRAM suggests — this is intentional; running out of KV cache during inference causes OOM crashes
|
||||
- New CONTEXT.md terms: **Tracker Node** (node serving first-layer shard + inference routing for a model), **Coverage Map** (tracker's per-model layer-range → node-count mapping), **Rebalance Directive** (tracker instruction to a node to load or drop a shard)
|
||||
- New CONTEXT.md terms: **Head Worker** (worker node serving first-layer shard for a model), **Coverage Map** (tracker's per-model layer-range → node-count mapping), **Rebalance Directive** (tracker instruction to a node to load or drop a shard)
|
||||
|
||||
Reference in New Issue
Block a user