diff --git a/deploy/portainer/README.md b/deploy/portainer/README.md new file mode 100644 index 0000000..8677c8d --- /dev/null +++ b/deploy/portainer/README.md @@ -0,0 +1,265 @@ +# Portainer deployment + +Start here if you want the public alpha tracker online from Portainer. + +Recommended first alpha path: + +1. If you do **not** have a Docker image in Gitea yet, use `meshnet-tracker-nobuild-stack.yml`. +2. After that works, create a Gitea **Container package** and switch Portainer to an image-based stack. +3. Do **not** create an npm package for deployment. This service is Python + Docker. For Portainer, the useful package is a Docker/OCI container image in Gitea Packages. + +This folder contains: + +| File | Use when | +| --- | --- | +| `meshnet-tracker-nobuild-stack.yml` | Easiest first deployment. No Docker registry. Downloads a repo tarball and installs at container start. | +| `meshnet-tracker-stack.yml` | Cleaner long-term deployment. Uses `deploy/docker/Dockerfile` / a prebuilt container image. | +| `meshnet-relay-only-stack.yml` | Optional relay-only deployment for a separate relay host/container. Not needed for the default alpha stack because the tracker embeds the relay. | +| `../docker/Dockerfile` | Builds one image containing tracker + relay + contracts packages. | + +Recommended alpha architecture: + +- One `meshnet-tracker` container. +- The tracker runs the relay in-process via `--embedded-relay`. +- The relay implementation is still the shared `meshnet_relay.RelayServer` class, so a future relay-only node can be split out without changing the protocol. +- Nginx Proxy Manager (or nginx/Caddy/Traefik) terminates TLS and routes `/v1`, `/dashboard`, `/ws`, and `/rpc` to the container. + +Important: the separate `meshnet-relay` container was not dropped as a capability. We removed it from the default alpha stack only to make first deployment simpler. Relay-only deployment remains supported via `meshnet-relay-only-stack.yml` or by running `meshnet-relay` from the same image. + +## Option A — easiest today: no registry / no package + +Use `meshnet-tracker-nobuild-stack.yml` in Portainer. + +It starts from `python:3.12-slim`, downloads a source `.tar.gz`, installs `packages/tracker`, `packages/relay`, and `packages/contracts`, then starts the tracker with embedded relay. First boot is slower, but it avoids creating/pushing a package. + +Required Portainer environment variables: + +```text +SOURCE_TARBALL_URL=https://git.d-popov.com/popov/neuron-tai/archive/master.tar.gz +PUBLIC_TRACKER_URL=https://ai.neuron.d-popov.com +PUBLIC_PROXY_NETWORK=npm_proxy +``` + +If your Gitea archive URL requires auth, either make an alpha release tarball downloadable to the Portainer host, or move to Option B and push a container image. + +Optional alpha/devnet variables: + +```text +STARTING_CREDIT=1 +DEVNET_TOPUP=1 +HEARTBEAT_TIMEOUT=30 +``` + +Set `STARTING_CREDIT=0` and `DEVNET_TOPUP=0` before any mainnet / real-money deployment. + +## Option B — recommended long-term: Gitea Container package + +Gitea Packages supports a Docker/OCI container registry. The package to create is a **Container Registry** package, not npm. + +Gitea docs: + +- Overview: https://docs.gitea.com/usage/packages/overview/ +- Container Registry: https://docs.gitea.com/usage/packages/container/ + +For this repo, use an image name like: + +```text +git.d-popov.com/popov/neuron-tai-tracker:alpha +``` + +or, if you prefer nested image names: + +```text +git.d-popov.com/popov/neuron-tai/meshnet-tracker-relay:alpha +``` + +Gitea image naming rule is: + +```text +{registry}/{owner}/{image}:{tag} +``` + +For us: + +```text +registry = git.d-popov.com +owner = popov +image = neuron-tai-tracker +label = alpha +``` + +### 1. Create a Gitea token + +In Gitea: + +1. Open user settings. +2. Go to Applications / Access Tokens. +3. Create a token that can write packages for `popov`. +4. Copy it once and store it safely. + +Do not commit the token into this repo or into the Portainer stack file. + +### 2. Login to the Gitea container registry + +From a machine with Docker and this repo checked out: + +```bash +docker login git.d-popov.com +``` + +Username: your Gitea username. +Password: the Gitea access token. + +If using 2FA/OAuth, Gitea docs recommend using a personal access token instead of your password. + +### 3. Build the image + +Run from repo root: + +```bash +docker build \ + -f deploy/docker/Dockerfile \ + -t git.d-popov.com/popov/neuron-tai-tracker:alpha \ + . +``` + +### 4. Push the image package to Gitea + +```bash +docker push git.d-popov.com/popov/neuron-tai-tracker:alpha +``` + +After this, Gitea should show the package under the `popov` user/org packages. + +### 5. Use the image in Portainer + +In `meshnet-tracker-stack.yml`, replace the local build block: + +```yaml +build: + context: ../.. + dockerfile: deploy/docker/Dockerfile +image: meshnet-tracker-relay:local +``` + +with: + +```yaml +image: git.d-popov.com/popov/neuron-tai-tracker:alpha +``` + +If the package is private, configure Portainer registry credentials for `git.d-popov.com`: + +1. Portainer → Registries → Add registry. +2. Type: Custom registry. +3. Registry URL: `git.d-popov.com`. +4. Username: your Gitea username. +5. Password/token: the Gitea access token. +6. Save, then deploy the stack. + +## Nginx Proxy Manager routing + +Create one Proxy Host for the public tracker domain. + +Default location `/`: + +```text +Scheme: http +Forward Hostname/IP: meshnet-tracker +Forward Port: 8081 +Websockets Support: ON +``` + +Custom locations: + +| Location | Forward hostname | Forward port | WebSockets | +| --- | --- | --- | --- | +| `/ws` | `meshnet-tracker` | `8765` | ON | +| `/rpc` | `meshnet-tracker` | `8765` | ON | + +Advanced tab if WebSocket upgrades fail: + +```nginx +proxy_http_version 1.1; +proxy_set_header Upgrade $http_upgrade; +proxy_set_header Connection $http_connection; +proxy_read_timeout 3600s; +proxy_send_timeout 3600s; +``` + +## Portainer variables + +For both stacks: + +```text +PUBLIC_TRACKER_URL=https://ai.neuron.d-popov.com +PUBLIC_PROXY_NETWORK=npm_proxy +``` + +For `meshnet-tracker-nobuild-stack.yml` only: + +```text +SOURCE_TARBALL_URL=https://git.d-popov.com/popov/neuron-tai/archive/master.tar.gz +SOURCE_STRIP_COMPONENTS=1 +``` + +Useful optional variables: + +```text +PUBLIC_RELAY_URL=wss://ai.neuron.d-popov.com/ws +HEARTBEAT_TIMEOUT=30 +ENABLE_BILLING_DB=1 +STARTING_CREDIT=1 +DEVNET_TOPUP=1 +``` + +`PUBLIC_RELAY_URL` can usually be omitted; the stack derives it from `PUBLIC_TRACKER_URL` by changing `https://` to `wss://` and appending `/ws`. + +## Verify deployment + +From outside the Docker host: + +```bash +curl -s https://ai.neuron.d-popov.com/v1/health +curl -s https://ai.neuron.d-popov.com/v1/network/map | python3 -m json.tool +``` + +Expected in `/v1/network/map`: + +```json +{ + "relay_url": "wss://ai.neuron.d-popov.com/ws", + "relay": { + "mode": "embedded", + "url": "wss://ai.neuron.d-popov.com/ws", + "bind_host": "0.0.0.0", + "bind_port": 8765 + } +} +``` + +Then start a node from any NAT/WSL2 machine: + +```bash +meshnet-node start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct +``` + +The node should print: + +```text +Relay advertised by tracker — using outbound tunnel wss://ai.neuron.d-popov.com/ws +Relay connected — wss://ai.neuron.d-popov.com/rpc/ +``` + +## Quick answer: npm or Gitea package? + +Use a Gitea **Container package** for Portainer. + +Do not use npm unless we later ship a JavaScript frontend package or Node.js CLI. It would not help the tracker/relay deployment. + +Recommended sequence: + +1. Deploy now with `meshnet-tracker-nobuild-stack.yml`. +2. Build/push `git.d-popov.com/popov/neuron-tai-tracker:alpha` as a Gitea Container package. +3. Switch Portainer to the image-based stack. +4. Later automate build/push in CI. diff --git a/deploy/portainer/meshnet-relay-only-stack.yml b/deploy/portainer/meshnet-relay-only-stack.yml new file mode 100644 index 0000000..280e13e --- /dev/null +++ b/deploy/portainer/meshnet-relay-only-stack.yml @@ -0,0 +1,35 @@ +# Meshnet relay-only stack for Portainer. +# +# Use this when you want to run a relay-only node separately from the tracker. +# The default alpha tracker stack embeds the same relay implementation in the +# tracker process, so this file is optional until relay traffic needs its own +# host/container. +# +# Intended topology for a relay-only public host: +# https://YOUR_DOMAIN/ws -> meshnet-relay:8765 (WebSocket) +# https://YOUR_DOMAIN/rpc/* -> meshnet-relay:8765 (WebSocket) +# +# If the tracker is separate, start it with: +# --relay-url wss://YOUR_DOMAIN/ws + +services: + meshnet-relay: + image: ${MESHNET_IMAGE:-git.d-popov.com/popov/neuron-tai-tracker:alpha} + container_name: meshnet-relay + restart: unless-stopped + command: ["meshnet-relay", "--host", "0.0.0.0", "--port", "8765", "--log-level", "INFO"] + expose: + - "8765" + healthcheck: + test: ["CMD", "python", "-c", "import socket; s=socket.create_connection(('127.0.0.1', 8765), 3); s.close()"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 10s + networks: + - public-proxy + +networks: + public-proxy: + external: true + name: ${PUBLIC_PROXY_NETWORK:-npm_proxy} diff --git a/deploy/portainer/meshnet-tracker-nobuild-stack.yml b/deploy/portainer/meshnet-tracker-nobuild-stack.yml index 950628e..441c8a7 100644 --- a/deploy/portainer/meshnet-tracker-nobuild-stack.yml +++ b/deploy/portainer/meshnet-tracker-nobuild-stack.yml @@ -1,9 +1,9 @@ -# Meshnet public tracker + relay stack for Portainer without a custom image. +# Meshnet public tracker stack for Portainer without a custom image. # # This stack does NOT use deploy/docker/Dockerfile and does NOT require pushing an # image to a registry. Each service starts from the public python:3.12-slim image, # downloads a source tarball, installs the tracker/relay packages into a named -# venv volume, then starts the service. +# venv volume, then starts the tracker with an embedded relay. # # Required Portainer variables: # SOURCE_TARBALL_URL URL to a .tar.gz archive of this repo @@ -88,6 +88,9 @@ services: --heartbeat-timeout "$${HEARTBEAT_TIMEOUT}" \ --self-url "$${PUBLIC_TRACKER_URL}" \ --relay-url "$${RELAY_URL}" \ + --embedded-relay \ + --relay-host 0.0.0.0 \ + --relay-port 8765 \ --stats-db /var/lib/meshnet/tracker-stats.sqlite \ --accounts-db /var/lib/meshnet/accounts.sqlite \ --starting-credit "$${STARTING_CREDIT:-1}" \ @@ -100,49 +103,9 @@ services: - meshnet-tracker-venv:/opt/meshnet-venv expose: - "8081" - healthcheck: - test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8081/v1/health', timeout=3).read()"] - interval: 30s - timeout: 5s - retries: 3 - start_period: 60s - networks: - - public-proxy - - meshnet-relay: - image: python:3.12-slim - container_name: meshnet-relay - restart: unless-stopped - environment: - SOURCE_TARBALL_URL: ${SOURCE_TARBALL_URL:?set SOURCE_TARBALL_URL} - SOURCE_STRIP_COMPONENTS: ${SOURCE_STRIP_COMPONENTS:-1} - command: - - /bin/sh - - -lc - - | - set -eu - apt-get update - apt-get install -y --no-install-recommends ca-certificates curl tar - rm -rf /var/lib/apt/lists/* - - rm -rf /opt/meshnet-src - mkdir -p /opt/meshnet-src - curl -fsSL "$${SOURCE_TARBALL_URL}" -o /tmp/meshnet-src.tar.gz - tar -xzf /tmp/meshnet-src.tar.gz -C /opt/meshnet-src --strip-components "$${SOURCE_STRIP_COMPONENTS:-1}" - - python -m venv /opt/meshnet-venv - /opt/meshnet-venv/bin/python -m pip install --upgrade pip setuptools wheel - /opt/meshnet-venv/bin/pip install \ - -e /opt/meshnet-src/packages/tracker \ - -e /opt/meshnet-src/packages/relay - - exec /opt/meshnet-venv/bin/meshnet-relay --host 0.0.0.0 --port 8765 --log-level INFO - volumes: - - meshnet-relay-venv:/opt/meshnet-venv - expose: - "8765" healthcheck: - test: ["CMD", "python", "-c", "import socket; s=socket.create_connection(('127.0.0.1', 8765), 3); s.close()"] + test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8081/v1/health', timeout=3).read()"] interval: 30s timeout: 5s retries: 3 @@ -153,7 +116,6 @@ services: volumes: meshnet-tracker-data: meshnet-tracker-venv: - meshnet-relay-venv: networks: public-proxy: diff --git a/deploy/portainer/meshnet-tracker-stack.yml b/deploy/portainer/meshnet-tracker-stack.yml index fdcc9ac..c494a09 100644 --- a/deploy/portainer/meshnet-tracker-stack.yml +++ b/deploy/portainer/meshnet-tracker-stack.yml @@ -1,10 +1,10 @@ -# Meshnet public tracker + relay stack for Portainer. +# Meshnet public tracker stack for Portainer. # # Intended topology when Nginx Proxy Manager (or another nginx reverse proxy) # runs on the same Docker host: # https://YOUR_DOMAIN/v1/* -> meshnet-tracker:8081 -# https://YOUR_DOMAIN/ws -> meshnet-relay:8765 (WebSocket) -# https://YOUR_DOMAIN/rpc/* -> meshnet-relay:8765 (WebSocket) +# https://YOUR_DOMAIN/ws -> meshnet-tracker:8765 (embedded relay WebSocket) +# https://YOUR_DOMAIN/rpc/* -> meshnet-tracker:8765 (embedded relay WebSocket) # # Before deploying, create or identify the Docker network shared with nginx/NPM, # then set PUBLIC_PROXY_NETWORK to its name in Portainer environment variables. @@ -64,6 +64,9 @@ services: --heartbeat-timeout "$${HEARTBEAT_TIMEOUT}" \ --self-url "$${PUBLIC_TRACKER_URL}" \ --relay-url "$${RELAY_URL}" \ + --embedded-relay \ + --relay-host 0.0.0.0 \ + --relay-port 8765 \ --stats-db /var/lib/meshnet/tracker-stats.sqlite \ --accounts-db /var/lib/meshnet/accounts.sqlite \ $${BILLING_ARGS} \ @@ -73,27 +76,9 @@ services: - meshnet-tracker-data:/var/lib/meshnet expose: - "8081" - healthcheck: - test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8081/v1/health', timeout=3).read()"] - interval: 30s - timeout: 5s - retries: 3 - start_period: 10s - networks: - - public-proxy - - meshnet-relay: - image: meshnet-tracker-relay:local - container_name: meshnet-relay - restart: unless-stopped - depends_on: - meshnet-tracker: - condition: service_started - command: ["meshnet-relay", "--host", "0.0.0.0", "--port", "8765", "--log-level", "INFO"] - expose: - "8765" healthcheck: - test: ["CMD", "python", "-c", "import socket; s=socket.create_connection(('127.0.0.1', 8765), 3); s.close()"] + test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8081/v1/health', timeout=3).read()"] interval: 30s timeout: 5s retries: 3