19 Commits

Author SHA1 Message Date
Dobromir Popov
b661590ac7 log window bigger 2026-07-14 17:47:20 +02:00
Dobromir Popov
21e6c86147 fix: let admin placement recover joined nodes 2026-07-14 16:37:42 +02:00
Dobromir Popov
def47f1a42 Merge branch 'master' of https://git.d-popov.com/popov/neuron-tai 2026-07-14 16:11:26 +02:00
Dobromir Popov
8cb00e951f feat: show admin node pool capacity 2026-07-14 16:11:18 +02:00
Dobromir Popov
7b3399760e chore: wrap up completed story metadata 2026-07-14 17:09:04 +03:00
Dobromir Popov
22467f145c merge: distributed performance baseline benchmark 2026-07-14 17:01:08 +03:00
Dobromir Popov
35af1e21de fix: make model placement controls observable 2026-07-14 16:00:37 +02:00
Dobromir Popov
905ea16ce0 feat: complete route session baseline benchmark 2026-07-14 16:55:52 +03:00
Dobromir Popov
348b003d6e fix: restore responsive dashboard panel grid 2026-07-14 15:55:24 +02:00
Dobromir Popov
1e64a5b2b9 new dash update 2026-07-14 15:29:11 +02:00
Dobromir Popov
e2f3ae32b8 feat: let admins manage model placement 2026-07-14 15:16:23 +02:00
Dobromir Popov
29351d6217 chore: ignore local model cache 2026-07-14 14:05:37 +02:00
Dobromir Popov
5c9a2f6c97 dash style fix 2026-07-14 13:29:51 +02:00
Dobromir Popov
13d82f8032 dash, tests 2026-07-14 12:26:10 +02:00
Dobromir Popov
d1a1400db9 Move tracker hive to admin and expand nodes panel.
Give Nodes & coverage full width on overview with inference prices and live speed, and expose model pricing on /v1/models.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 12:19:25 +02:00
Dobromir Popov
5d87e81bc9 feat: harden node placement and partial model loading 2026-07-13 21:58:08 +02:00
Dobromir Popov
a6bcc69288 sol mainnet payouts tasks 2026-07-13 18:51:40 +02:00
Dobromir Popov
c938d38031 more docs review 2026-07-13 18:37:07 +02:00
Dobromir Popov
95245be512 documentation revision 2026-07-13 18:14:21 +02:00
83 changed files with 1733 additions and 319 deletions

View File

@@ -2,9 +2,9 @@
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
- [User profile](user-profile.md) — who Dobromir is and how to work with him
- [Project status](project-status.md) — 35/35 stories done; alpha hardening next
- [Project status](project-status.md) — US-001…US-035 done; US-036…US-050 in docs/prd.json; alpha hardening + scratch features next
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 00160019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan; [ADR-0023](../../docs/adr/0023-model-agnostic-node-capability-admission.md), [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md))
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).

View File

@@ -20,13 +20,13 @@ Active workstream (started 2026-07-04): alpha hardening of the money/trust path.
**Launch-readiness grilling (2026-07-06):** Locked launch plan — devnet dev/test run now, then **real mainnet SOL/USDT** (not devnet, not a new public token) for the first cohort: friends (API clients) + hired VPS/VPC hosts (our own test infra, not third-party volunteers — stake-free, risk-free if something breaks, not a long-term topology). Pricing: clients are the only party spending real money; nodes only accumulate off-chain credit and get paid in batches (30min dev / 24h later) — a failed distribution leaves funds parked, not lost, so mainnet-vs-devnet mixups are lower-risk than initially assumed. TAI token: do NOT issue/list now — ADR-0002 already locks listing behind $50k volume + 25 nodes/15 wallets plus an unresolved securities-review gate; only a dormant mainnet mint (cheap, ~few $ SOL) for name/branding reservation is in scope, bundled with treasury-key work, not before it. Treasury custody: bare keypair file (current runbook 02) is not acceptable for real funds — plan is **free native SPL multisig** (`spl-token create-multisig`, no protocol fee unlike Squads' 0.5 SOL), 2-of-3 signers, at least one cold/offline, others one-per-hired-VPS-provider to avoid correlated compromise (not yet built — ops task, no issue filed). Stake/slash asymmetry (registry/slash is a local Python adapter per ADR-0007, not on-chain) accepted for now since hired hosts are our own infra and friends aren't node operators — revisit before opening to real third-party node operators. A mainnet-vs-devnet boot guardrail was proposed and explicitly declined by the owner given the safe-by-default money flow above.
**Two new issues from this session, both `ready-for-agent`:**
- **21 — Honest-noise calibration corpus** (`.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`) rescoped from "prod gate" to a **hard alpha-release blocker**. Confirmed by code read: `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns bool only, no raw divergence value; fleet-dispatch exists but wrong shape (`server.py:2998-3104`, pinned routes + latency, not full-fleet + TOPLOC divergence); storage wrong shape (`registry_events` has no divergence/hardware columns). Three-part build: (1) surface raw TOPLOC distance from audit.py, (2) extend dispatch to hit every registered node with fixed prompt/seed, (3) new SQLite table keyed by node+GPU+dtype. Small-fleet exception granted (N = actual hired-VPS fleet size). Hired VPS hosts stay stake-free until this closes.
- **23 — Dynamic HF-benchmarked pricing** (`.scratch/alpha-hardening/issues/23-dynamic-hf-pricing.md`), high priority but not a release blocker. Pricing today is 100% static (`DEFAULT_PRICE_PER_1K_TOKENS = 0.02`, `billing.py:21`; `model_presets.json` has no per-model price). Target: 80% of cheapest comparable provider on `https://huggingface.co/inference/models` (per-provider-per-model marketplace, `?search=` query param works, no confirmed JSON API — plain scrape attempted first, escalate to headless browser only if the table isn't in raw HTML). Human-verified `hf_aliases` + `hf_verified_match_note` (params/quantization) per model, not auto-discovered matching. Reuses the `_settlement_loop` daemon-thread pattern for a daily refresh; falls back silently to the static default on any failure.
**Two new issues from this session:**
- **21 — Honest-noise calibration corpus** `Status: ready-for-human` (engineering done 2026-07-06; blocked on human fleet calibration run before mainnet launch).
- **23 — Dynamic HF-benchmarked pricing** `Status: done` (see `23-dynamic-hf-pricing_completed.md`).
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (35/35 done) and does NOT see `.scratch/alpha-hardening/issues/`. No ralph loop is running and no `.ralph-tui/` state exists. `.scratch/alpha-hardening/prd.json` now has 23 stories (AH-001…AH-023); point Ralph at that file for the alpha-hardening branch. Do NOT use `ralph auto --parallel` on server.py-touching issues — 21 and 23 both touch `server.py`/`billing.py`/`audit.py`; if run in the same Ralph pass, run them serially, not in parallel (merge-conflict risk, same lesson as 03/04 previously).
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (US-001…US-047; base 35/35 done, friends-test arc 3647 open/in-progress). Alpha hardening uses `.scratch/alpha-hardening/prd.json` (AH-001…AH-023). Point Ralph at the prd.json for the branch you're running.
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].

View File

@@ -6,7 +6,13 @@ metadata:
type: project
---
# Project Status (2026-07-02)
# Project Status (2026-07-13)
## Distributed inference performance (2026-07-14)
`DIP-001` is done in `.scratch/distributed-inference-performance/`: the deterministic two-node Route Session stub benchmark covers direct/relay plus cached/stateless prefill and decode. Its JSON and concise summary explicitly attribute model execution, activation encode/decode, compression, connection setup, relay queueing, local HTTP forwarding, and end-to-end seam latency. `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` passed (7); the fixture assertion checks output-token identity and connection attempts.
> Doc reconciliation 2026-07-13: `docs/prd.json` tracks US-001…US-050 (048 memory budget, 049 mainnet pilot, 050 Qwen demand placement). ADRs 00250026 added (TAI phase B/C, assignment ownership).
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:

1
.gitignore vendored
View File

@@ -20,6 +20,7 @@ dist/
!.env.testnet
.rocm-local/*
.pytest-tmp/*
.cache/
# Local tracker/node sqlite databases (never commit runtime state)
*.sqlite

View File

@@ -2,9 +2,9 @@
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet SOL/USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers, stake-free). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers; no upfront stake, probation only). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch.
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch. Runbook: [04-toploc-calibration-run](./runbooks/04-toploc-calibration-run.md).
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
@@ -77,6 +77,7 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
| [04 TOPLOC calibration run](./runbooks/04-toploc-calibration-run.md) (issue 21 ops) |
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |

View File

@@ -8,7 +8,7 @@
## 1. Mission / where we are
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer and a **broken money/trust path**. Three independent audits agreed: unauthenticated gossip, free-credit faucet, double-pay risks, ephemeral bans, and node self-reported accounting undermine alpha release. The owner locked alpha scope (single settlement tracker, open node join, devnet mock-USDT, carried-forward reputation) and a fraud/verification design (TOPLOC adoption, adaptive audits, on-demand hop bisection, persisted graduated reputation, tracker-authoritative accounting). **Research and planning artifacts are complete** (ADRs 00160019, 22 issue files, README index). Next: implement Bucket 1 blockers test-first.
neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer. Pre-release audits found the money/trust path was not alpha-ready; **Bucket 1 alpha blockers are implemented** (see `.scratch/alpha-hardening/README.md`). Remaining launch gates: issue **21** (human calibration run), post-alpha Bucket 2 (1215), and active scratch tracks (NCA, perf, distributed GGUF).
---
@@ -42,7 +42,7 @@ Point to artifacts — do not re-derive from this handoff.
| Path | What it contains |
|---|---|
| `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order |
| `.scratch/alpha-hardening/issues/` | 22 work items (Buckets 13) |
| `.scratch/alpha-hardening/issues/` | 25 work items (Buckets 13 + perf follow-ups) |
| `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) |
| `docs/adr/00160019` | Alpha scope, auth, fraud, multi-tracker design |
| `docs/agents/issue-tracker.md` | Issue file conventions |

View File

@@ -1,6 +1,6 @@
Status: ready-for-human
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money (mainnet SOL/USDT) traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money mainnet USDT traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
@@ -14,9 +14,9 @@ Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay stake-free until it's done, then move to real staking once thresholds derive from their own hardware.
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay on probation (no upfront stake) until it's done, then move to paid USDT serving once thresholds derive from their own hardware.
**Current gap (confirmed 2026-07-06 by code read):** none of the three pieces below exist yet.
**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet.
1. `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns a **plain bool** — no raw TOPLOC divergence/distance value is ever computed or surfaced. Every "done" fraud-detection issue (0610) currently runs on a guessed threshold baked into that bool, not a calibrated one.
2. Fleet dispatch exists but is the wrong shape: `_handle_benchmark_hop_penalty` / `_handle_benchmark_results` (`packages/tracker/meshnet_tracker/server.py:2998-3104`, from the old US-030 latency work) targets pinned 13-node *routes* and measures latency, not TOPLOC divergence across *every* registered node.
@@ -36,7 +36,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section.
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section; [runbook 04](../runbooks/04-toploc-calibration-run.md).
## ADR links

View File

@@ -440,12 +440,12 @@
"Run relevant pytest tests; run the full suite when practical or document why not"
],
"priority": 21,
"passes": true,
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money friends+hired-VPS launch) — rescoped 2026-07-06, no longer a Ralph-skip.",
"passes": false,
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money mainnet USDT). Operator runbook: .scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md",
"dependsOn": [
"AH-006"
],
"completionNotes": "Engineering complete and unit-tested (validator audit.py detailed-verify aggregation, tracker calibration.py corpus store, calibration dispatch endpoints). Marked ready-for-human, not done: real corpus collection against the live hired-VPS fleet, and the threshold/FPR write-up that depends on its output, need a human operator — see .ralph-tui/progress.md and packages/validator/README.md."
"completionNotes": "Engineering complete and unit-tested. Remaining: human runs POST /v1/calibration/toploc/run on live hired-VPS fleet, records envelope/FPR, wires thresholds — see runbook 04 and packages/validator/README.md."
},
{
"id": "AH-022",

View File

@@ -0,0 +1,70 @@
# Runbook 04 — Honest-noise TOPLOC calibration (issue 21)
**Status:** engineering complete; **operator action required** before production audit thresholds.
**Blocks:** enabling calibrated TOPLOC thresholds on a mainnet / friends-test fleet (issue 21, ADR-0018).
## When to run
- Before first real-money traffic with audit enforcement enabled.
- Again whenever the fleets **hardware mix** changes materially (new GPU generation, CPU-only nodes added, precision/recipe change per model).
Alpha exception: with a **small hired-VPS-only** fleet, `gate_status.ready` may mean “covers every node we operate today” (`--toploc-calibration-gate-min-hardware-profiles 1`).
## Prerequisites
- Tracker running with billing + registry + `--toploc-calibration-db PATH` (or default under tracker cwd).
- At least one **solo-capable** node per hardware profile you want in the corpus (full model coverage — partial shards are skipped).
- Admin or validator credentials (`Authorization` header or validator service token per ADR-0017).
- Reference validator can replay the fixed calibration prompt (same model/seed as dispatch uses).
## Steps
1. **Register the fleet** — all nodes you intend to pay on mainnet should be up, admitted (NCA when enabled), and solo-serving the calibration model.
2. **Dispatch the job** (admin/validator only):
```bash
curl -X POST "https://<tracker>/v1/calibration/toploc/run" \
-H "Authorization: Bearer <admin-or-validator-token>" \
-H "Content-Type: application/json" \
-d '{}'
```
Partial-shard nodes appear under `skipped_partial_shard_node_ids`. Per-node failures appear under `skipped` with reasons.
3. **Wait for completion** — watch tracker logs and node consoles until every solo-capable node has a row in the corpus.
4. **Fetch results**:
```bash
curl "https://<tracker>/v1/calibration/toploc/results" \
-H "Authorization: Bearer <admin-or-validator-token>"
```
Record:
- `envelope` — p99 metrics + 20% safety margin (recommended tolerances).
- `gate_status.ready` and `gate_status.hardware_profiles`.
- `estimated_false_positive_rate` (in-sample sanity check only).
5. **Write up thresholds** — paste envelope values into operator notes / issue 21 comment. Do **not** wire into production `ToplocAuditConfig` until you have reviewed FPR on this fleet.
6. **Mark issue 21 done** — when corpus covers the launch fleet and thresholds are documented.
## Two-wallet / minimal pilot variant
If your “fleet” is one node machine + one client:
- Run calibration against the **node** profile only (one hardware row is enough for `gate_status` with min profiles = 1).
- Client wallet is irrelevant to calibration — it never serves inference.
## Do not
- Enable stricter production audit thresholds before this completes.
- Reuse a corpus collected on devnet/mock hardware for a different mainnet GPU mix without re-running.
## References
- Issue: `.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`
- Code: `packages/tracker/meshnet_tracker/calibration.py`, `POST /v1/calibration/toploc/run`
- Validator: `packages/validator/README.md` — TOPLOC audit contract

View File

@@ -12,4 +12,10 @@ Provide an opt-in, admin-only tracker Dashboard Testing tab that dynamically dis
- One active run.
- Real inference stays separately environment-gated and excluded from default suites.
## Operator workflow
See [`docs/dev/dashboard-test-runner.md`](../../docs/dev/dashboard-test-runner.md)
for launch configuration, default safe suites vs the gated real-inference suite,
and required environment variables.
See `prd.json` for executable Ralph user stories and acceptance criteria.

View File

@@ -51,15 +51,16 @@
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
],
"priority": 3,
"passes": false,
"passes": true,
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
"dependsOn": [
"US-001",
"US-002"
]
],
"completionNotes": "Completed by agent"
}
],
"metadata": {
"updatedAt": "2026-07-11T17:02:30.520Z"
"updatedAt": "2026-07-12T01:58:06.286Z"
}
}

View File

@@ -9,7 +9,7 @@ Before changing code, every Ralph agent must:
1. Read this file completely.
2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`.
3. Read `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
3. Read `docs/adr/0024-distributed-gguf-runtime.md` and the relevant part of `architecture.md`.
4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints.
5. Inspect the current implementation and tests; do not assume historical scratch text describes live code.
6. Read the evidence/handoff directories for every declared dependency.
@@ -296,7 +296,7 @@ Active decisions:
- `.scratch/distributed-gguf-runtime/README.md`
- `.scratch/distributed-gguf-runtime/implementation-strategy.md`
- `.scratch/distributed-gguf-runtime/architecture.md`
- `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md`
- `docs/adr/0024-distributed-gguf-runtime.md`
- `.scratch/distributed-gguf-runtime/PRD.md`
- `.scratch/distributed-gguf-runtime/prd.json`

View File

@@ -25,7 +25,7 @@ Transformers/safetensors remains the correctness baseline. vLLM remains an optio
- [Current architecture](architecture.md)
- [PRD](PRD.md)
- [Ralph backlog](prd.json)
- [ADR-0020](ADR-0020-distributed-gguf-runtime.md)
- [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md)
- [Milestones](milestones.md)
- [Issues](issues/)
- [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md)

View File

@@ -1,6 +1,6 @@
# Distributed GGUF Decision Framework
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0020](ADR-0020-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale.
This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language.

View File

@@ -56,4 +56,4 @@ As a runtime engineer, I need a controlled baseline so that GGUF work proceeds f
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -56,4 +56,4 @@ As a node developer, I need a battle-proven streaming protocol so that Python an
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -54,4 +54,4 @@ As the Tracker, I need exact compatibility identity so that only numerically and
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -58,4 +58,4 @@ As a maintainer, I need a small auditable fork boundary so that upstream updates
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -58,4 +58,4 @@ As a node, I need to map only my assigned dense-Llama Shard so that aggregate co
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -58,4 +58,4 @@ As a Shard, I need to consume and emit the correct transformer boundary state so
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -57,4 +57,4 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -62,4 +62,4 @@ As a node runtime, I need one supervised native process so that llama.cpp intern
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -58,4 +58,4 @@ As the existing node service, I need a GGUF Shard backend adapter so that the Tr
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -59,4 +59,4 @@ As a release engineer, I need real local distributed parity before involving net
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -59,4 +59,4 @@ As a consumer-hardware operator, I need two physical machines to execute one GGU
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -60,4 +60,4 @@ As a node operator, I need active sessions batched safely so that concurrency in
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -59,4 +59,4 @@ As a client, I need failures to be bounded and explicit so that distributed spee
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -62,4 +62,4 @@ As the product owner, I need an end-to-end comparison so that the native runtime
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -58,4 +58,4 @@ As a client seeking top models, I need a separately certified MoE-capable archit
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -57,4 +57,4 @@ As a maintainer, I need narrow upstreamable proposals so that our patch burden c
- [PRD](../PRD.md)
- [Implementation strategy](../implementation-strategy.md)
- [Current architecture](../architecture.md)
- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md)
- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md)

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done (2026-07-14)
# 01 — Baseline and profiling harness
@@ -12,16 +12,15 @@ sizes and connection counts without requiring a real model or external host.
## Acceptance criteria
- [ ] The harness runs a fixed prompt and fixed generated-token count through a
- [x] The harness runs a fixed prompt and fixed generated-token count through a
two-node route in direct and relay modes.
- [ ] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
- [x] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
compression ratio, connection attempts, and queue wait.
- [ ] It distinguishes prefill from decode and cached from stateless mode.
- [ ] It emits machine-readable JSON suitable for CI artifacts and a concise
- [x] It distinguishes prefill from decode and cached from stateless mode.
- [x] It emits machine-readable JSON suitable for CI artifacts and a concise
human-readable summary.
- [ ] A test fixture can assert connection attempts and output token identity.
- [x] A test fixture can assert connection attempts and output token identity.
## Blocked by
None - can start immediately.
None - completed. Verified with `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` (7 passed).

View File

@@ -15,9 +15,10 @@
"Can assert connection count and output token identity"
],
"priority": 1,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/01-baseline-profiling-harness.md",
"dependsOn": []
"dependsOn": [],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-002",
@@ -31,9 +32,12 @@
"Tests cover binary, JSON, timeout, disconnect, cancellation, and cleanup"
],
"priority": 2,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/02-relay-session-compatibility.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-003",
@@ -47,9 +51,12 @@
"Benchmark shows healthy-session connection count independent of token count"
],
"priority": 3,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/03-http-keepalive.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-004",
@@ -63,9 +70,12 @@
"Tests verify cadence and cleanup"
],
"priority": 4,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/04-seam-telemetry.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-005",
@@ -79,9 +89,12 @@
"Tests cover compressible, incompressible, threshold, malformed, and legacy bodies"
],
"priority": 5,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/05-adaptive-compression.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-006",
@@ -95,9 +108,12 @@
"Wire and token-output regression tests pass"
],
"priority": 6,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/06-activation-framing-copies.md",
"dependsOn": ["DIP-001"]
"dependsOn": [
"DIP-001"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-007",
@@ -111,9 +127,13 @@
"Tests cover chunking, slow consumers, failure, and legacy peers"
],
"priority": 7,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/07-prefill-backpressure.md",
"dependsOn": ["DIP-001", "DIP-004"]
"dependsOn": [
"DIP-001",
"DIP-004"
],
"completionNotes": "Completed by agent"
},
{
"id": "DIP-008",
@@ -127,9 +147,20 @@
"Gate verifies token identity, session stability, and resource cleanup"
],
"priority": 8,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/distributed-inference-performance/issues/08-end-to-end-performance-gate.md",
"dependsOn": ["DIP-002", "DIP-003", "DIP-004", "DIP-005", "DIP-006", "DIP-007"]
"dependsOn": [
"DIP-002",
"DIP-003",
"DIP-004",
"DIP-005",
"DIP-006",
"DIP-007"
],
"completionNotes": "Completed by agent"
}
]
}
],
"metadata": {
"updatedAt": "2026-07-12T02:35:28.752Z"
}
}

View File

@@ -71,6 +71,8 @@ As an operator and release engineer, I need clear doctor output and opt-in hardw
Add a small generic capability domain object in the node package. `doctor` loads the requested generic model path through the same backend startup uses, executes a bounded real forward at the assigned Shard, and emits the report. Startup gates routable registration on the successful report. Registration carries validated capabilities; the tracker persists/exposes them and filters route candidates at the model/shard/recipe seam.
**Assignment ownership:** NCA validates whatever the node loads; it does not assign models. Pinned vs tracker-managed assignment rules are in [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md). Demand-driven managed placement (Qwen scratch PRD) may only consume spare capacity; admission applies equally to pinned and managed loads.
The future signed-update contract is represented only by a local manifest version and generic schema in P0. A future Tracker Model Artifact Manifest may be signed data, but Node executable behavior remains supplied by signed Node releases.
## Success measures

View File

@@ -7,6 +7,7 @@ This P0 makes a Node prove it can serve its selected Model Artifact and Shard be
## Locked decisions
- A Node explicitly asked to serve a Model Preset fails closed when no validated recipe can execute it; it must not register as ready or accept paid inference.
- **Assignment ownership:** startup/`--model` loads are **pinned**; tracker-managed demand placement (Qwen US-050) may use **spare capacity only** — [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md).
- Default validation covers the selected model/shard only. `meshnet-node doctor --all-recipes` is reserved for support and CI.
- A Model Preset may have multiple named recipes. Each independently proves a real forward; the Tracker schedules only validated recipes while considering measured performance.
- Compatibility schemas are generic. A future Tracker may publish signed, data-only Model Artifact Manifests, but executable recipes arrive only through signed Node releases.

View File

@@ -35,11 +35,12 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 2,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/02-doctor-real-forward.md",
"dependsOn": [
"NCA-001"
]
],
"completionNotes": "Completed by agent"
},
{
"id": "NCA-003",
@@ -54,12 +55,13 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 3,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/03-fail-closed-startup-admission.md",
"dependsOn": [
"NCA-001",
"NCA-002"
]
],
"completionNotes": "Completed by agent"
},
{
"id": "NCA-004",
@@ -76,12 +78,13 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 4,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/04-tracker-validated-capability-routing.md",
"dependsOn": [
"NCA-001",
"NCA-003"
]
],
"completionNotes": "Completed by agent"
},
{
"id": "NCA-005",
@@ -96,15 +99,16 @@
"Full pytest passes or an exact unrelated blocker is recorded"
],
"priority": 5,
"passes": false,
"passes": true,
"notes": "Source issue: .scratch/node-capability-admission/issues/05-docs-hardware-lane-contract.md",
"dependsOn": [
"NCA-002",
"NCA-004"
]
],
"completionNotes": "Completed by agent"
}
],
"metadata": {
"updatedAt": "2026-07-11T19:16:52.768Z"
"updatedAt": "2026-07-12T01:54:03.030Z"
}
}

View File

@@ -46,13 +46,12 @@ model rather than waiting for an operator to request a load.
## Node ownership
- A startup-assigned `(model, shard range, quantization)` is pinned and never
changed by the tracker.
- Spare capacity on a pinned node, and all capacity on a model-less node, is
available for tracker-managed assignments.
- Tracker-added assignments are explicitly marked managed and may be moved or
removed by the tracker under the safety policy. Runtime UI controls are a
later feature.
Reconciled with [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md) and NCA (ADR-0023):
- A **startup-assigned** `(model, shard range, quantization)` from explicit `--model` or accepted bootstrap assign is **pinned** until the operator restarts.
- **Tracker-managed** assignments (this feature) use only **spare capacity** — model-less nodes or (future, US-048) unused shard slots — and are marked `managed: true`.
- The tracker may move or remove managed assignments under the safety policy below; it must not retarget a pinned serving assignment to satisfy demand.
- Every assignment, pinned or managed, must pass NCA `doctor` before becoming routable when admission is enabled.
## Pricing

10
CONTEXT-MAP.md Normal file
View File

@@ -0,0 +1,10 @@
# Context map
Multi-context layout is not yet split. Use the root domain vocabulary:
- **[CONTEXT.md](../CONTEXT.md)** — ubiquitous language for the distributed inference network
- **`docs/adr/`** — system-wide architectural decisions
- **`.scratch/<feature>/`** — active feature plans and issues
- **`.claude/memory/MEMORY.md`** — agent session index and current workstreams
Per-context `src/<context>/docs/adr/` ADRs will be added when bounded contexts graduate out of the monorepo packages layout.

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done (US-001…US-035 complete; friends-test arc US-036…US-049 in `docs/prd.json`; US-048/050 tracked. See ADRs 00150018, 0023, 00250026.)
# Distributed Inference Network — PRD
@@ -8,10 +8,12 @@ Running large language models requires expensive dedicated hardware that most pe
## Solution
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in SOL or USDC; node operators earn our native token. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning **USDT**. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
## User Stories
> **Status (2026-07-13):** Stories below are the original product intent. **Shipped behavior** is in [Implementation Decisions](#implementation-decisions) and ADRs 00150018, 0023, 00250026. Superseded lines are marked inline.
### Node Operator
1. As a node operator, I want to install the node client with a single command (`pip install meshnet-node`), so that I can start contributing without reading documentation.
@@ -21,10 +23,10 @@ A volunteer GPU network where anyone can share their GPU by running a single com
5. As a node operator, I want my assigned shard to download automatically from HuggingFace on first start, so that I don't have to manually find or download model weights.
6. As a node operator, I want to seed my shard to other nodes via P2P once I have it, so that new nodes with the same shard assignment don't need to download from HuggingFace.
7. As a node operator, I want the node client to register with the tracker automatically and begin serving inference requests, so that I start earning as soon as setup is complete.
8. As a node operator, I want to see my current node score, shard assignment, and token earnings in the terminal, so that I can verify my node is contributing correctly.
9. As a node operator, I want to stake tokens before serving paid inference, so that I have skin in the game and the network can trust my outputs.
8. As a node operator, I want to see my current node score, shard assignment, and USDT earnings in the terminal, so that I can verify my node is contributing correctly.
9. As a node operator, I want to serve paid inference without upfront stake deposits, with my accrued USDT pending balance as fraud collateral and probation as the anti-sybil cost, so that onboarding stays frictionless. *(Supersedes stake-before-serving; ADR-0015/0018.)*
10. As a node operator, I want my first N jobs to run without earning (probationary period), so that the network can establish trust before paying me.
11. As a node operator, I want to be notified immediately if my stake is slashed due to a fraud detection event, so that I can investigate and fix the issue.
11. As a node operator, I want to be notified when my pending balance is forfeited due to a failed audit, so that I can investigate and fix the issue. *(Supersedes stake slash; ADR-0018 forfeiture.)*
12. As a node operator, I want to receive a strike and a warning before being banned, so that accidental failures don't immediately end my participation.
13. As a node operator, I want to be automatically reassigned to a different shard when the tracker determines another shard is more in demand, so that my hardware is always optimally used.
14. As a node operator, I want the node client to reconnect automatically if the tracker is temporarily unavailable, so that transient network issues don't stop me from earning.
@@ -34,8 +36,8 @@ A volunteer GPU network where anyone can share their GPU by running a single com
### Client Developer
17. As a client developer, I want to send `POST /v1/chat/completions` requests to the gateway in the same format as the OpenAI API, so that I can switch to the network with a one-line code change.
18. As a client developer, I want to authenticate with an API key funded by SOL or USDC, so that I never need to acquire or hold our native token.
19. As a client developer, I want to top up my API key balance by sending SOL or USDC to a Solana address, so that payment is simple and familiar.
18. As a client developer, I want to authenticate with an API key funded by USDT, so that I never need to acquire or hold our native token. *(ADR-0015.)*
19. As a client developer, I want to top up my API key balance by sending USDT to the treasury Solana address, so that payment is simple and familiar. *(ADR-0015; wallet binding US-039/041.)*
20. As a client developer, I want to see a per-request cost estimate before sending a request, so that I can budget inference costs accurately.
21. As a client developer, I want to receive streaming responses (`text/event-stream`) in OpenAI-compatible format, so that I can build low-latency user experiences.
22. As a client developer, I want `GET /v1/models` to return the list of available model presets on the network, so that I know what I can request.
@@ -46,15 +48,15 @@ A volunteer GPU network where anyone can share their GPU by running a single com
### End User (via a client app)
27. As an end user, I want to buy SOL on any exchange and use it to pay for inference, so that I don't need to understand blockchain technology to use the service.
27. As an end user, I want to buy USDT on an exchange and use it to pay for inference via Solana, so that I don't need deep crypto knowledge to use the service. *(Clients pay USDT; SOL is only for network fees if they self-custody.)*
28. As an end user, I want responses of equivalent quality to centralised providers, so that I don't have to trade quality for cost savings.
29. As an end user, I want low latency on first token, so that conversational applications feel responsive.
### Validator
30. As a validator, I want to automatically re-run a random sample (~5%) of completed inference requests on a reference node, so that I can detect nodes returning fraudulent outputs.
31. As a validator, I want to submit a fraud proof on-chain when a node's output diverges beyond tolerance, so that the slash event is recorded trustlessly.
32. As a validator, I want to earn a reward for each successful fraud detection, so that there is an economic incentive to run validation.
30. As a validator, I want to automatically re-run a random sample (~5%) of completed inference requests on a reference node with TOPLOC activation verification, so that I can detect nodes returning fraudulent outputs. *(ADR-0018.)*
31. As a validator, I want the tracker to record forfeiture and strikes when an audit fails, so that penalties are applied consistently. *(Supersedes on-chain fraud proof in alpha; ADR-0018.)*
32. As a validator, I want economic incentive to run validation, so that fraud detection is not purely altruistic. *(Validator reward share deferred; forfeiture to protocol cut today.)*
### Network (tracker / system)
@@ -62,7 +64,7 @@ A volunteer GPU network where anyone can share their GPU by running a single com
34. As the tracker, I want to rebalance shard assignments across nodes when demand for a model preset changes, so that the network always covers the most-requested models.
35. As the tracker, I want to instruct a node to download a new shard when no other node covers it, so that model preset coverage is maintained automatically.
36. As the tracker, I want to exclude banned wallets from route selection, so that fraudulent nodes cannot serve paid inference.
37. As the tracker, I want to read stake, slash, strike, and ban state exclusively from Solana smart contracts, so that I cannot manipulate payouts even with full control of the routing layer.
37. As the tracker, I want strike and ban state persisted in the registry and enforced on route selection, so that fraudulent wallets cannot serve paid inference. *(Supersedes on-chain-only stake/slash registry; ADR-0018; on-chain deferred per ADR-0007/0015.)*
38. As the network, I want new model presets to be addable by submitting a HuggingFace model ID and shard count, so that the set of available models can grow without code changes.
## Implementation Decisions
@@ -73,11 +75,11 @@ The codebase is organized as a Python monorepo with the following top-level pack
- `packages/gateway` — OpenAI-compatible HTTP gateway and route orchestration
- `packages/tracker` — centralized tracker service (node registry, scoring, route selection)
- `packages/sdk``meshnet` Python SDK wrapping gateway + wallet controls
- `packages/contracts` — Solana L2 smart contracts (stake, slash, strike, ban, settlement)
- `packages/contracts` — Solana adapter boundary (custodial USDT treasury, local registry prototype)
- `packages/p2p` — P2P gossip layer and shard swarm seeding
### Inference engine (ADR-0001)
PyTorch with a Petals-style shard pipeline. Each node independently loads its assigned shard from local disk. At inference time, only activation tensors (~8 KB per layer boundary per token) travel between nodes — no model weights cross the network during serving.
### Inference engine (ADR-0001; native GGUF path ADR-0024)
PyTorch with a Petals-style shard pipeline remains the current production backend. A benchmark-gated llama.cpp/GGUF native path is planned in ADR-0024. Each node independently loads its assigned shard from local disk. At inference time, only activation tensors (~8 KB per layer boundary per token) travel between nodes — no model weights cross the network during serving.
### Inference route execution
The gateway receives a client request, asks the tracker for an inference route (ordered list of node endpoints covering all layers), opens a persistent TCP session to the first node in the route, streams activation tensors through each node in sequence, and returns the final logits as a streaming chat completion response.
@@ -91,14 +93,14 @@ The gateway receives a client request, asks the tracker for an inference route (
6. Register with tracker (wallet, hardware profile, shard, endpoint)
7. Begin accepting inference connections
### Payment flow
Clients pre-fund an API key with SOL/USDC. The gateway records per-request compute attribution. A settlement transaction runs on Solana L2 at the end of each epoch: client balance is debited, node operators receive our native token proportional to layers served, validators receive a reward share. Solana contracts are the authoritative source for all stake, slash, strike, and ban state (ADR-0002).
### Payment flow (ADR-0015 supersedes ADR-0002 settlement mechanics)
Clients pre-fund an API key with USDT. The tracker meters each request against the off-chain ledger. Periodic settlement batches USDT payouts from the custodial treasury to node operators proportional to work units (default: every 24 h or when pending ≥ 5 USDT). Fraud penalties forfeit pending balance (ADR-0018); strike/ban state persists in the tracker registry. TAI reward accrual is deferred — see ADR-0025 for reserved-mint / off-chain phase B/C; ADR-0002 roadmap for public listing.
### Fraud detection (ADR-0003)
Validators re-run ~5% of completed requests. If a node's output diverges beyond floating-point tolerance from the reference, the validator submits a slash transaction on-chain. Strike count increments. At the configured strike threshold, the wallet is banned on-chain. New wallets complete N unpaid jobs before earning begins.
### Fraud detection (ADR-0018; historical ADR-0003)
Validators re-run ~5% of completed requests with TOPLOC activation verification. Caught cheaters forfeit pending balance and receive strikes; three strikes bans the wallet. Probation (first N unpaid jobs) remains the anti-sybil re-entry cost.
### Tracker architecture (ADR-0004)
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. Solana is the authoritative source of truth for all incentive-relevant state.
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. **Alpha:** strike/ban/forfeiture state lives in the tracker registry (ADR-0018); USDT settlement via custodial treasury (ADR-0015). On-chain programs deferred (ADR-0007).
### Shard distribution (ADR-0005)
Shards are identified by `(model_preset, shard_index)`. On assignment, the node downloads the shard layers from HuggingFace using `huggingface_hub`. Once downloaded, the node joins the P2P shard swarm and seeds to other nodes requesting the same shard. Popular shards propagate entirely via P2P; cold shards fall back to HuggingFace.
@@ -115,9 +117,9 @@ The gateway exposes OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/mo
**Per-component seams:**
- **Tracker**: given a set of registered nodes with known shard coverage and node scores, assert `select_route(model_preset)` returns an optimal ordered list of node endpoints.
- **Node shard serving**: given an activation tensor for the node's layer range, assert the output tensor shape and dtype are correct.
- **Fraud detection**: given a validator that re-runs a known-bad node response, assert a slash transaction is submitted on-chain with correct attribution.
- **Fraud detection**: given a validator that re-runs a known-bad node response, assert strike/forfeiture state updates with correct attribution (ADR-0018; on-chain slash deferred).
- **Shard swarm**: given a node that has a shard, assert a second node with the same assignment downloads it via P2P rather than HuggingFace.
- **Payment settlement**: given a completed inference session with known compute attribution, assert token balances change by the expected amounts after epoch settlement.
- **Payment settlement**: given a completed inference session with known compute attribution, assert USDT ledger balances change by the expected amounts after epoch settlement (ADR-0015).
## Out of Scope
@@ -135,4 +137,4 @@ The gateway exposes OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/mo
- The `meshnet-node` CLI is the primary viral growth vector. Every friction point in the install/start sequence costs node operators. The startup sequence must complete without any manual configuration on a machine with a CUDA-capable GPU.
- The name "meshnet" is a working name. The actual package and token names are TBD.
- The Solana L2 chain selection (vs Base/Arbitrum) is not yet finalised — both are cheap, EVM-compatible fallbacks. The contracts package should abstract chain-specific details.
- The probationary period length (N free jobs) and slash amounts are economic parameters that will need tuning once the network has real usage data. Hardcode sensible defaults; make them on-chain governable.
- The probationary period length (N free jobs) and forfeiture amounts are economic parameters that will need tuning once the network has real usage data. Hardcode sensible defaults; governance TBD (ADR-0018).

View File

@@ -1,5 +1,7 @@
# PyTorch over llama.cpp for the inference engine
> **Runtime direction update (2026-07-13):** PyTorch/safetensors remains the current production backend and correctness reference. A benchmark-gated native GGUF path is defined in [ADR-0024](0024-distributed-gguf-runtime.md); it does not replace this ADR until release gates pass.
We started with llama.cpp RPC as the distributed backend (following kyuz0/amd-strix-halo-toolboxes), but switched to PyTorch with a Petals-style shard pipeline. llama.cpp RPC requires the primary node to load the full model and distribute weights over the network at every session start — for a 70B model that's ~70GB over LAN per launch, making tracker-driven node rebalancing prohibitively expensive. PyTorch/Petals lets each node load its shard independently from local disk; only activations (~8KB per layer boundary per token) cross the network at inference time. PyTorch also has same-day support for new model architectures, training support (required for the planned torrent-style fine-tuning feature), and is the engine Petals itself uses for this exact use case.
## Considered Options

View File

@@ -1,5 +1,7 @@
# Optimistic trust with stake slashing and strike-based bans
> **Settlement update (2026-07-04):** Alpha uses pending-balance forfeiture instead of stake slashing ([ADR-0015](0015-usdt-custodial-settlement.md)). Fraud detection, TOPLOC audits, and persisted reputation are specified in [ADR-0018](0018-fraud-detection-verification-and-reputation.md). The text below is the historical prototype design.
All inference responses are trusted by default. Validators re-run a random sample (~5%) of requests on reference nodes and compare outputs. Nodes that fail are slashed (stake reduced). Enough strikes result in a permanent on-chain ban.
For the prototype, the gateway emits validation events after completed requests. A validation event records the session id, model preset, request messages, observed output, and the route metadata for each node that served the request. The validator samples events with a configurable rate and deterministic seed for tests. Sampled events are re-run against a trusted reference node/reference function; string outputs must match exactly for stub models, while future tensor/model outputs use a configurable floating-point tolerance.

View File

@@ -1,6 +1,6 @@
# ADR-0020: Dashboard chat streaming, live request progress, and the mixed-topology routing flaw
## Status: Accepted (chat/streaming/styles implemented); routing flaw documented, fix pending
## Status: Accepted (chat/streaming/styles and mixed-topology routing fix implemented)
## Context
@@ -94,7 +94,7 @@ head + full-model downstream is a topology the planner never had to handle befor
prior split tests used disjoint shards (011 + 1223) where `shard_start` happened to
equal the correct continuation layer.
### Required fix (not yet implemented)
### Required fix (implemented 2026-07-07 — commits `518c259`, `e44abc9`, `1ecc599`; see ADR-0021)
1. **Correct continuation layer:** when hop N ends at layer `e`, hop N+1 must execute
from `start_layer = e + 1` regardless of the downstream node's own `shard_start`

View File

@@ -1,6 +1,8 @@
# ADR-0022: Sharded per-node generation cache for distributed PyTorch routes
## Status: Accepted
## Status: Superseded — see [0022-sharded-per-node-kv-cache.md](0022-sharded-per-node-kv-cache.md)
> Draft alternate header names (`X-Meshnet-Cache-Mode`, `X-Meshnet-Seq-Len`) were not implemented. The accepted wire protocol and implementation use `X-Meshnet-Cache` and `X-Meshnet-Past-Len` per the linked ADR.
## Context

View File

@@ -1,8 +1,12 @@
# ADR-0020: Lean Native Distributed GGUF Runtime
# ADR-0024: Lean Native Distributed GGUF Runtime
Status: Accepted
Date: 2026-07-13
> **Numbering note:** ADR-0020 is reserved for dashboard chat streaming and mixed-topology routing (`docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md`). This record was originally drafted as ADR-0020 in `.scratch/distributed-gguf-runtime/` and renumbered to avoid the collision.
>
> **Relation to ADR-0001:** PyTorch/safetensors remains the correctness reference and current production backend. This ADR defines a benchmark-gated native GGUF path; it does not revoke ADR-0001 until release gates pass.
## Context
The project currently uses Transformers/safetensors as its real model execution backend. This provides broad architecture coverage and a correctness reference, but reported and observed consumer CPU/GPU inference performance motivates evaluating llama.cpp/GGML and quantized GGUF.
@@ -119,3 +123,11 @@ Rejected. gRPC/HTTP2 already provides mature streaming, flow control, deadlines,
4. Real two-machine execution using both Shards.
5. End-to-end performance/fit advantage over the current distributed route.
6. Separate Qwen3-family architecture certification.
## Relationship to US-042 (whole-model GGUF shortcut)
[US-042](../issues/42-gguf-llamacpp-node-backend.md) **phase C** ships first: a node with enough RAM serves a **full** GGUF via llama.cpp on a single-hop Inference Route using the existing HTTP activation seam and PyTorch-era tracker integration. That is intentionally small and does not require this ADR's gRPC worker or llama.cpp patch stack.
This ADR's track starts only after **DGR-001** (controlled safetensors-vs-GGUF benchmark) shows a meaningful speed or fit benefit. Then implement the native worker (DGR-002+) — which subsumes US-042 direction A (layer-range GGUF + boundary tensors) if the benchmark warrants it.
Do not run US-042 phase C and DGR-008+ in parallel on the same node backend without an explicit integration plan; phase C uses llama-cpp-python (or equivalent) whole-model path; ADR-0024 uses the standalone C++ worker.

View File

@@ -0,0 +1,52 @@
# ADR-0025: TAI reserved mint and off-chain accrual (phase B/C)
## Status: Accepted
## Context
ADR-0015 chose **USDT-direct custodial settlement** for alpha and near-term production. Clients pay USDT; nodes receive batched USDT SPL payouts. ADR-0002's TAI reward token, revenue-backed floor, and open-market listing gates remain the long-term design but are **not** the live payment path.
The owner wants TAI to exist without the cost and legal surface of a public launch: no AMM, no open listing, no client-facing TAI, no on-chain stake machinery.
## Decision
### Phase B — Reserved mainnet mint (cheap, optional early)
- Create a fixed-supply TAI SPL mint on **mainnet** when treasury work happens (~0.002 SOL).
- Entire initial supply sits in a **team-controlled** wallet (same custody posture as the USDT treasury today).
- **No public emission, no market, no client UX.** Mint exists for name reservation and future programmatic rewards only.
- Document mint address in operator config; do not advertise to users.
### Phase C — Off-chain TAI accrual alongside USDT (before automatic on-chain TAI payouts)
- Extend the billing ledger with **`tai_pending[wallet]`** accrued from completed inference work using a simple rule (e.g. USDT node share × configurable TAI-per-USDT rate, or fixed TAI per work unit).
- TAI accrual is **display-only + ledger-persisted** initially; nodes see pending TAI in dashboard/CLI.
- **Clients never pay or hold TAI.** USDT remains the only client-facing asset.
- Optional manual or scheduled **TAI SPL batch transfers** from the team wallet (same batching pattern as USDT `send_payouts`) — operator-triggered until automatic emission is justified by volume.
- The existing **10% protocol USDT cut** continues to accumulate as future TAI liquidity per ADR-0015/0002; do not redirect it until a deliberate liquidity event.
### Explicit non-goals (this ADR)
- Open-market listing, AMM, or DEX liquidity
- Buyback floor endpoint or backing-price oracle (ADR-0002 machinery)
- On-chain stake deposits or slash contracts
- Paying clients rebates or accepting TAI for inference
- Replacing USDT node payouts with TAI-only payouts before volume gates in ADR-0002 pass
## Relation to ADR-0002 listing gates
Public TAI listing stays gated on **$50k cumulative USDT volume** and **25+ nodes / 15+ wallets**. Phase B/C may proceed **below** those gates because they do not create a public market — only reserved supply and off-chain accounting.
Securities review remains required before any **public** distribution or listing; off-chain accrual to hired/known operators with manual SPL transfers is an operator discretion, not a product promise.
## Consequences
- USDT mainnet pilot (two-wallet setup) is unblocked without TAI complexity.
- TAI narrative is preserved at minimal cost (mint + ledger column + optional manual transfers).
- Automatic TAI emission can later reuse the US-033 settlement loop shape with a second mint and separate pending bucket.
- Dashboard and APIs must label TAI balances as **non-withdrawable** until an on-chain payout batch confirms.
## Verification
- USDT settlement tests remain authoritative for production payouts (`tests/test_settlement_loop.py`).
- When phase C lands: ledger tests for `tai_pending` accrual, idempotent gossip replication, and optional TAI batch payout adapter tests mirroring USDT.

View File

@@ -0,0 +1,51 @@
# ADR-0026: Node assignment ownership — pinned startup vs managed demand placement
## Status: Accepted
## Context
Three features define how a node gets its `(model, shard range, recipe/quantization)`:
1. **ADR-0011 / US-013** — tracker suggests a gap from coverage map on startup or auto-join.
2. **Node capability admission (ADR-0023 / NCA)** — a node must pass `doctor` + real forward before becoming routable; startup-assigned work is validated, not blindly trusted.
3. **Qwen demand placement** (`.scratch/qwen3.6-27b-demand-placement/`) — tracker deploys a model when chat demand appears and spare capacity exists.
These looked contradictory: NCA and the Qwen PRD both say startup assignments are "pinned," while demand placement wants the tracker to assign models dynamically.
## Decision
### Three assignment tiers
| Tier | How it is created | Mutable by tracker? | Admission |
|---|---|---|---|
| **Operator-initiated** | Node starts with explicit `--model` / shard flags | **No** — pinned until operator restarts or explicitly reloads | Must pass NCA `doctor` before routable |
| **Network bootstrap** | `/v1/network/assign` or `/v1/nodes/assign` on first join (ADR-0011) | **No** for the active loaded shard — treated as operator-equivalent once accepted at startup | Must pass NCA before routable |
| **Tracker-managed** | Demand-driven placement (Qwen PRD) on spare capacity | **Yes** — marked `managed: true`; subject to cooldown / safety policy | Must pass NCA for the new assignment before routable |
### Spare capacity rule (unifies NCA + Qwen)
- A nodes **active** `(model, shard, recipe)` from startup is **pinned** — the tracker does not silently retarget a serving node to a different model.
- **Spare capacity** — memory/slots not holding the pinned assignment, or a node registered without a model — may receive **tracker-managed** assignments to satisfy demand.
- Until multi-shard runtime exists (US-048), “spare capacity” effectively means **model-less nodes** or nodes explicitly registered for managed placement; do not overload a single-shard node with a second assignment.
### Demand placement interaction
- First chat request for an unrouted model queues **demand**; leader tracker may assign **managed** nodes only when eligible spare capacity exists (Qwen PRD).
- Until complete coverage + validated recipes exist, return retryable `503 model_loading` with coverage metadata.
- Managed assignments must not evict pinned assignments on other nodes without the Qwen safety policy (≥3 copies, 1.5× demand multiplier, cooldown).
### NCA is not optional for any tier
Regardless of assignment source, registration carries **validated capability** only after `doctor` succeeds. The tracker excludes nodes with absent, stale, or failed capability reports (ADR-0023).
## Consequences
- NCA and Qwen demand placement are complementary: NCA gates *quality*; demand placement gates *where new coverage comes from*.
- US-048 (multi-shard slots) extends spare capacity — until then, demand placement primarily targets nodes that join without `--model`.
- Rebalance / dropout relocation (US-013, US-048) applies to **coverage gaps**, not retroactive retargeting of pinned nodes for demand convenience.
## Verification
- NCA tests: unvalidated nodes never routed.
- Demand-placement tests (when implemented): managed flag set; pinned nodes unchanged.
- Documented in Qwen scratch PRD and NCA README cross-links.

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 01 — Monorepo scaffold + single-node smoke test

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 02 — Two-node shard pipeline

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 03 — Tracker: node registration + route selection

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 04 — Node client startup flow (`meshnet-node start`)

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 05 — OpenAI-compatible gateway

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done (on-chain registry mechanics superseded — probation/ban enforcement uses tracker registry + ADR-0015/0018)
# 08 — Node probationary period + ban enforcement

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 09 — P2P shard swarm

View File

@@ -1,4 +1,4 @@
Status: ready-for-agent
Status: done
# 10 — `meshnet` Python SDK

View File

@@ -1,8 +1,6 @@
# US-019 — Binary data plane and optional peer weight transfer
Status: needs-triage
Priority: Low
Stage: Design parking lot
Status: done (design parking lot; binary activation path shipped in US-011/US-019)
## Context

View File

@@ -1,66 +1,10 @@
Status: ready-for-agent
Status: superseded
# US-020 - Memory budget, shard slots, and dropout relocation hardening
# Superseded — renumbered to US-048
## Goal
This issue slot was a duplicate of tracker-node-hardening (US-020). Memory budget / shard slots / dropout relocation work lives at:
Make node capacity limits explicit and enforce them consistently when the tracker assigns, rebalances, and relocates shards after a node dropout.
- **Issue:** [48-memory-budget-shard-slots-and-dropout-relocation.md](./48-memory-budget-shard-slots-and-dropout-relocation.md)
- **PRD:** `docs/prd.json``US-048`
This is a follow-up to US-013, not a replacement. US-013 owns the coverage-first assignment and rebalance algorithm. This issue hardens the capacity contract around that algorithm: operator memory budget, maximum loaded shard slots, and relocation behavior when one node must absorb or split ranges after another node disappears.
## Context
Recent work added the first part of the contract:
- `meshnet-node --memory MB` is registered with the tracker as `vram_bytes` when explicitly set.
- CPU nodes without `--memory` keep the tracker default capacity, preserving old behavior.
- `meshnet-node --max-shards N` is accepted and registered as `max_loaded_shards`.
- Tracker registration validates `max_loaded_shards >= 1`.
The current runtime still effectively has one active backend shard per node. A node may advertise `max_loaded_shards`, but the tracker does not yet use multiple shard slots in bin-packing, and the node does not yet host multiple concurrently loaded shard ranges.
## Scope
- Make tracker rebalance logic account for `max_loaded_shards` as a capacity multiplier or explicit shard-slot list.
- Ensure a node is never assigned more total layers than its memory budget can support across all loaded shard slots.
- Decide and implement the runtime behavior for multiple loaded shards:
- either support multiple concurrently loaded shard backends on one node, or
- keep one backend active and treat `max_loaded_shards` as future metadata, with tracker enforcement preventing multi-range assignment for now.
- On heartbeat timeout, relocate the dropped node's uncovered layer range to eligible managed nodes while respecting both memory and shard-slot limits.
- Surface the effective memory budget and shard slot count in tracker/network inspection output so operators can diagnose why a node did or did not receive a range.
## Non-Goals
- Do not redesign the US-013 coverage-first algorithm from scratch.
- Do not change relay, `/ws`, or `/rpc` behavior.
- Do not change the token/reward model.
- Do not require public internet verification; all behavior must be locally testable.
## Acceptance Criteria
- Tracker stores and exposes `max_loaded_shards` for registered nodes.
- Assignment/rebalance never exceeds:
- `assigned_layers_total <= floor((vram_bytes * 0.8) / bytes_per_layer_at_quant)`
- `assigned_range_count <= max_loaded_shards`
- A managed node with `max_loaded_shards=1` only receives one active shard range.
- A managed node with `max_loaded_shards=2` can absorb two non-contiguous uncovered ranges only if the node runtime supports serving both; otherwise tracker must keep assigning at most one range and document `max_loaded_shards` as reserved.
- Dropout test: register nodes covering a model, let a middle/tail node heartbeat-expire, and assert the tracker queues `LOAD_SHARD` directives that restore full coverage without violating memory or shard-slot limits.
- CLI test: `--memory` and `--max-shards` are reflected in the registration payload.
- `python -m pytest tests/test_tracker_routing.py tests/test_node_startup.py` passes in the project virtualenv, aside from any pre-existing platform-specific wallet permission assertion documented in the final notes.
## Implementation Notes
- Existing files likely involved:
- `packages/node/meshnet_node/cli.py`
- `packages/node/meshnet_node/startup.py`
- `packages/node/meshnet_node/torch_server.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/test_tracker_routing.py`
- `tests/test_node_startup.py`
- Keep backward compatibility: nodes that omit `vram_bytes` default to tracker defaults; nodes that omit `max_loaded_shards` default to `1`.
- Prefer a small internal representation for assigned ranges if multiple ranges become real, for example `assigned_shards: list[tuple[int, int]]`, while preserving `shard_start`/`shard_end` in public responses for single-range nodes.
## Comments
- 2026-06-30: Created after implementing the initial registration plumbing in commit `f1e4ed6` (`--memory`, `--max-shards`, tracker validation). This issue captures the remaining end-to-end behavior so it does not conflict with US-013.
- 2026-06-30: Implementation decision: `max_loaded_shards` is currently a validated and exposed capacity field, but multi-range assignment remains reserved because `TorchNodeServer` serves one active backend shard. The tracker therefore emits at most one active range per node while exposing `vram_bytes`, `ram_bytes`, `max_loaded_shards`, quantization, throughput, and computed `max_assignable_layers` in inspection endpoints.
Do not implement from this file.

View File

@@ -1,8 +1,6 @@
# US-036 — Streamed chat completions over the relay RPC path
Status: planned
Priority: Critical (blocks public friends-test deployment)
Stage: Designed
Status: done (implemented — `_stream_relayed_frames` in `server.py`; verify on public NAT relay before friends-test)
## Context

View File

@@ -1,9 +1,16 @@
# US-042 — GGUF/llama.cpp node backend
Status: planned
Priority: High (unlocks big MoE models on volunteer hardware — the pool's core value)
Priority: High (unlocks DeepSeek-V4-Flash on volunteer hardware — the pool's core value)
Stage: Draft design
## Goal
Run **DeepSeek-V4-Flash** as the first real large-model target on volunteer
hardware via GGUF/llama.cpp. This epic is no longer GLM-oriented: the initial
objective is to prove that DeepSeek-V4-Flash can load and serve correctly on
consumer/unified-memory nodes, then expand from there.
## Context
The node backend is transformers-only (`model_backend.py`

View File

@@ -86,10 +86,10 @@ What exists already (build on it, don't duplicate):
- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
layers 0k; machine B joins with no model and receives **only** the files
for its assigned range from A — nothing fetched from HF
- [ ] Machine B's resident memory scales with its shard size, not model size
- [ ] Checksums verified end-to-end; corrupted transfer falls back cleanly
- [x] Machine B's resident memory scales with its shard size, not model size
- [x] Checksums verified end-to-end; corrupted transfer falls back cleanly
- [x] Single-node/full-model flows unchanged
- [ ] `python -m pytest` passes from repo root
- [x] `python -m pytest` passes from repo root
## Implementation notes
@@ -98,6 +98,13 @@ What exists already (build on it, don't duplicate):
`full_url`; HuggingFace remains fallback-only, and when it is used the node
computes `allow_patterns` from the repo's remote SafeTensors index so it
stays layer-filtered even without tracker-cached files. Remaining hard half
is true partial model materialization: the backend can prefer a downloaded
is partial model materialization: the backend can prefer a downloaded
local model directory, but Transformers still needs a `meta`-device load
path that materializes only assigned layers.
- 2026-07-13: Partial LOAD implemented. `_load_partial_model_from_snapshot` builds
on `meta` via `init_empty_weights`, materializes only layer-scoped checkpoint
tensors, and finalizes device placement without copying unmaterialized meta
weights (`_finalize_active_shard_modules_on_device`). Tests cover memory
scaling (`test_partial_snapshot_resident_weight_numel_scales_with_shard`)
and real-torch meta-vs-materialized counts. Remaining: live two-machine LAN
verification.

View File

@@ -0,0 +1,69 @@
Status: planned
# US-048 — Memory budget, shard slots, and dropout relocation hardening
> Renumbered from duplicate slot `20` (which belongs to tracker-node-hardening / US-020 in `docs/prd.json`).
## Goal
Make node capacity limits explicit and enforce them consistently when the tracker assigns, rebalances, and relocates shards after a node dropout.
This is a follow-up to US-013, not a replacement. US-013 owns the coverage-first assignment and rebalance algorithm. This issue hardens the capacity contract around that algorithm: operator memory budget, maximum loaded shard slots, and relocation behavior when one node must absorb or split ranges after another node disappears.
## Context
Recent work added the first part of the contract:
- `meshnet-node --memory MB` is registered with the tracker as `vram_bytes` when explicitly set.
- CPU nodes without `--memory` keep the tracker default capacity, preserving old behavior.
- `meshnet-node --max-shards N` is accepted and registered as `max_loaded_shards`.
- Tracker registration validates `max_loaded_shards >= 1`.
The current runtime still effectively has one active backend shard per node. A node may advertise `max_loaded_shards`, but the tracker does not yet use multiple shard slots in bin-packing, and the node does not yet host multiple concurrently loaded shard ranges.
## Scope
- Make tracker rebalance logic account for `max_loaded_shards` as a capacity multiplier or explicit shard-slot list.
- Ensure a node is never assigned more total layers than its memory budget can support across all loaded shard slots.
- Decide and implement the runtime behavior for multiple loaded shards:
- either support multiple concurrently loaded shard backends on one node, or
- keep one backend active and treat `max_loaded_shards` as future metadata, with tracker enforcement preventing multi-range assignment for now.
- On heartbeat timeout, relocate the dropped node's uncovered layer range to eligible managed nodes while respecting both memory and shard-slot limits.
- Surface the effective memory budget and shard slot count in tracker/network inspection output so operators can diagnose why a node did or did not receive a range.
## Non-Goals
- Do not redesign the US-013 coverage-first algorithm from scratch.
- Do not change relay, `/ws`, or `/rpc` behavior.
- Do not change the token/reward model.
- Do not require public internet verification; all behavior must be locally testable.
## Acceptance Criteria
- Tracker stores and exposes `max_loaded_shards` for registered nodes.
- Assignment/rebalance never exceeds:
- `assigned_layers_total <= floor((vram_bytes * 0.8) / bytes_per_layer_at_quant)`
- `assigned_range_count <= max_loaded_shards`
- A managed node with `max_loaded_shards=1` only receives one active shard range.
- A managed node with `max_loaded_shards=2` can absorb two non-contiguous uncovered ranges only if the node runtime supports serving both; otherwise tracker must keep assigning at most one range and document `max_loaded_shards` as reserved.
- Dropout test: register nodes covering a model, let a middle/tail node heartbeat-expire, and assert the tracker queues `LOAD_SHARD` directives that restore full coverage without violating memory or shard-slot limits.
- CLI test: `--memory` and `--max-shards` are reflected in the registration payload.
- `python -m pytest tests/test_tracker_routing.py tests/test_node_startup.py` passes in the project virtualenv, aside from any pre-existing platform-specific wallet permission assertion documented in the final notes.
## Implementation Notes
- Existing files likely involved:
- `packages/node/meshnet_node/cli.py`
- `packages/node/meshnet_node/startup.py`
- `packages/node/meshnet_node/torch_server.py`
- `packages/tracker/meshnet_tracker/server.py`
- `tests/test_tracker_routing.py`
- `tests/test_node_startup.py`
- Keep backward compatibility: nodes that omit `vram_bytes` default to tracker defaults; nodes that omit `max_loaded_shards` default to `1`.
- Prefer a small internal representation for assigned ranges if multiple ranges become real, for example `assigned_shards: list[tuple[int, int]]`, while preserving `shard_start`/`shard_end` in public responses for single-range nodes.
## Comments
- 2026-06-30: Created after implementing the initial registration plumbing in commit `f1e4ed6` (`--memory`, `--max-shards`, tracker validation). This issue captures the remaining end-to-end behavior so it does not conflict with US-013.
- 2026-06-30: Implementation decision: `max_loaded_shards` is currently a validated and exposed capacity field, but multi-range assignment remains reserved because `TorchNodeServer` serves one active backend shard. The tracker therefore emits at most one active range per node while exposing `vram_bytes`, `ram_bytes`, `max_loaded_shards`, quantization, throughput, and computed `max_assignable_layers` in inspection endpoints.
- 2026-07-13: Renumbered from `docs/issues/20-memory-budget-…` to resolve duplicate issue slot 20.

View File

@@ -0,0 +1,101 @@
Status: ready-for-agent
# US-049 — Mainnet USDT cutover: two-wallet pilot checklist
Priority: High (first real-money friends test)
Stage: Operator runbook + config verification
## Goal
Move from **Solana devnet + mock-USDT** to **Solana mainnet + real USDT** for a minimal pilot: **one client wallet** (inference payer) and **one node-operator wallet** (payout recipient). Treasury holds USDT and pays SOL fees. TAI stays phase B/C per [ADR-0025](../adr/0025-tai-off-chain-accrual-and-reserved-mint.md).
## Wallet roles
| Role | Keypair | On-chain use |
|---|---|---|
| **Treasury** | Operator `treasury-keypair.json` (multisig when ready) | Holds USDT float + SOL for fees; sends batched node payouts |
| **Client** | Your inference-user wallet | SPL USDT → treasury; bound to API key for ledger credit |
| **Node** | Your node-operator wallet | Receives USDT payout batches from treasury |
The node process already creates/loads a Solana wallet at startup; the client wallet is bound via accounts/dashboard (`POST /v1/wallet/register` or US-041 flows).
## Pre-flight (devnet smoke — do not skip)
- [ ] Tracker with `--solana-rpc-url https://api.devnet.solana.com`, mock mint, treasury keypair
- [ ] `--settle-period 60 --payout-threshold 0` — confirm payout appears on dashboard **Settlement history** with explorer link
- [ ] Run `python -m pytest tests/test_settlement_loop.py -q` — includes prod 24h/5 USDT gate tests
- [ ] One inference request → node pending → settlement tx → node wallet balance increases
## Mainnet config change (config-only cutover)
Replace devnet values; **no code deploy required** beyond what is already on the branch.
```bash
# Example — use your mainnet RPC provider
meshnet-tracker start \
--solana-rpc-url https://api.mainnet-beta.solana.com \
--usdt-mint EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v \
--treasury-keypair ~/.config/solana/meshnet-treasury-mainnet.json \
--settle-period 86400 \
--payout-threshold 5.0 \
--payout-dust-floor 0.01 \
--starting-credit 0 \
--devnet-topup 0
```
| Flag | Devnet (test) | Mainnet (pilot) |
|---|---|---|
| RPC | `api.devnet.solana.com` | Mainnet RPC (Helius/QuickNode/etc.) |
| `--usdt-mint` | mock mint from `devnet_setup.py` | Real USDT mint (`EPjF…` on Solana) |
| `--settle-period` | `60` (fast verify) | `86400` (24 h) |
| `--payout-threshold` | `0` | `5.0` USDT |
| `--starting-credit` | `1.0` (optional) | `0` |
| `--devnet-topup` | `1.0` | `0` |
## Treasury funding
- [ ] Fund treasury wallet with **SOL** for fees (~0.10.5 SOL to start; ~$0.001 per daily batch + ~$0.30 once per new node ATA)
- [ ] Fund treasury with **USDT** for node payouts (your float — e.g. first week of expected node earnings)
- [ ] Client wallet holds USDT; send a test SPL transfer to treasury ATA; confirm deposit watcher credits API key within one poll interval
## Two-wallet pilot steps
1. **Start tracker** on mainnet config above (single settlement tracker per ADR-0016).
2. **Client path:** register account → create API key → bind **client wallet** → deposit USDT to treasury → verify ledger balance on dashboard.
3. **Node path:** start `meshnet-node` with **node wallet** keypair → register → serve inference.
4. **Inference:** client sends `POST /v1/chat/completions` with API key; verify 402 before deposit, success after.
5. **Accrual:** confirm node **pending USDT** on dashboard rises; client balance debits.
6. **Payout (24 h):** wait for `--settle-period` **or** temporarily lower to `300` for first pilot verification, then restore `86400`.
7. **Threshold path:** alternatively, accumulate ≥ `5` USDT pending in one session to trigger immediate batch without waiting 24 h.
8. **Verify on-chain:** settlement history shows mainnet tx signature; node wallet USDT ATA balance increased; pending zeroed.
## Safety checks
- [ ] `--devnet-topup 0` — no faucet on mainnet
- [ ] `--starting-credit 0` — no free inference credit
- [ ] Treasury keypair not committed to git; file mode 600
- [ ] Plan multisig migration before large float ([alpha runbook](../../.scratch/alpha-hardening/runbooks/02-treasury-key-rotation.md) intent)
- [ ] Issue **21** (TOPLOC calibration) before production audit thresholds on untrusted nodes — runbook: [04-toploc-calibration-run](../../.scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md)
## Cost estimate (this pilot)
- **SOL fees:** pennies per day at 1 batch × 1 node
- **USDT:** whatever clients deposit and nodes earn (treasury is passthrough for payouts)
- **TAI mint (optional, ADR-0025 phase B):** ~0.002 SOL one-time — defer if not needed for pilot
## Acceptance criteria
- [ ] Devnet checklist completed once
- [ ] Mainnet tracker serves dashboard; billing enabled
- [ ] Client wallet deposit → ledger credit → inference → debit
- [ ] Node wallet receives ≥1 confirmed USDT payout batch on mainnet
- [ ] 24 h period enforced: sub-threshold pending not paid before period (covered by `tests/test_settlement_loop.py`)
- [ ] ≥5 USDT pending triggers payout without waiting full period (covered by tests)
- [ ] Rollback documented: switch RPC + mint back to devnet if needed
## Related
- ADR-0015 (USDT custodial settlement)
- ADR-0025 (TAI reserved mint / off-chain accrual — not blocking this pilot)
- US-033 / US-032 (settlement + deposits)
- `scripts/devnet_setup.py` (devnet only)

View File

@@ -0,0 +1,15 @@
Status: in-design
# US-050 — Qwen3.6-27B demand-driven managed placement
> Full spec: [.scratch/qwen3.6-27b-demand-placement/PRD.md](../../.scratch/qwen3.6-27b-demand-placement/PRD.md)
> Assignment rules: [ADR-0026](../adr/0026-node-assignment-ownership-and-managed-placement.md)
> Admission: [ADR-0023](../adr/0023-model-agnostic-node-capability-admission.md)
## Summary
Deploy `Qwen/Qwen3.6-27B` when chat demand appears and **spare** fleet capacity exists. Startup `--model` assignments stay **pinned**; tracker-managed loads fill gaps on model-less or (future US-048) unused slot capacity only.
## Acceptance criteria
See scratch PRD and `docs/prd.json` US-050.

View File

@@ -1,6 +1,6 @@
{
"name": "Distributed Inference Network",
"description": "Build a distributed inference network with node, gateway, tracker, SDK, contracts, and P2P shard distribution components from the grill session PRD.",
"description": "Distributed inference network: base program US-001…US-035 complete; friends-test arc US-036…US-049; capacity/placement US-048/050. Scratch features (alpha hardening AH-001…AH-025, NCA, distributed GGUF, Qwen demand) have separate prd.json files under .scratch/.",
"branchName": "ralph/distributed-inference-network",
"userStories": [
{
@@ -814,10 +814,307 @@
"US-033"
],
"completionNotes": "GET /dashboard served from embedded dashboard.html (package-data, no build step) by any tracker. Panels: hive/leader (raft status), nodes+coverage grouped by model, client balances, node pending + protocol cut, settlement history with devnet explorer links, strikes/bans/forfeitures (GET /v1/registry/wallets + snapshot forfeits), RPM stats. 4s auto-refresh via fetch polling. 3 tests in tests/test_dashboard.py."
},
{
"id": "US-036",
"title": "36 — Streamed chat completions over the relay RPC path",
"description": "Public NAT deployments proxy every chat request tracker → relay → head node. Implement true multi-frame SSE streaming over the relay WebSocket so clients see live tokens and relayed streams bill through the same SSE accounting loop as direct proxy streams. Inter-node /forward activation hops stay single-frame (ADR-0014).",
"acceptanceCriteria": [
"stream: true chat via relay delivers SSE chunks incrementally (≥2 distinct frame arrivals before [DONE])",
"Relayed streamed request records nonzero billed tokens and node work credit",
"Non-streamed relayed requests and /forward binary hops behave exactly as before (single frame, body_base64 intact)",
"Legacy single-frame response from an old node is accepted as terminal",
"Idle stream (no frame for 120 s) returns 504 and cleans up the relay-side queue",
"Extend tests/test_gossip_and_relay.py alongside test_relay_rpc_round_trips_http_request_to_peer",
"python -m pytest passes from repo root"
],
"priority": 36,
"status": "needs-review",
"notes": "Source issue: docs/issues/36-relay-streamed-chat.md. Implemented via _stream_relayed_frames in server.py; verify on public NAT relay before friends-test.",
"dependsOn": [
"US-029",
"US-031"
],
"completionNotes": "Multi-frame relay-http-response protocol; node relay_bridge line-by-line SSE emit; relay server per-request asyncio.Queue; tracker _stream_relayed_frames with SSE billing parity. Client mid-stream disconnect accepted limitation for alpha."
},
{
"id": "US-037",
"title": "37 — Concurrent request handling in the node relay bridge",
"description": "RelayHttpBridge currently handles relay-http-request envelopes serially, blocking up to 300 s per request. Off-LAN a node can be head of one route and downstream hop of another — overlapping routes through a shared node break. Dispatch on a bounded ThreadPoolExecutor (default 8, configurable) with per-frame WS send locking compatible with US-036 streaming.",
"acceptanceCriteria": [
"While one relayed request is in flight, a second relay-http-request to the same node completes without waiting for the first",
"Responses are correctly matched by request_id when they complete out of order",
"More than N simultaneous requests queue and all eventually complete; thread count never exceeds N workers",
"Bridge survives a relay reconnect with workers still in flight (no crash, no deadlock; orphaned responses dropped)",
"Configurable via meshnet-node start --relay-concurrency N (env MESHNET_RELAY_CONCURRENCY)",
"Extend tests/test_gossip_and_relay.py",
"python -m pytest passes from repo root"
],
"priority": 37,
"status": "done",
"notes": "Source issue: docs/issues/37-relay-bridge-concurrency.md. Critical for public friends-test; blocks concurrent head + hop on same node.",
"dependsOn": [
"US-036"
],
"completionNotes": "ThreadPoolExecutor dispatch in relay_bridge.py; per-frame WS send lock; test_relay_bridge_serves_concurrent_requests; --relay-concurrency CLI flag sets MESHNET_RELAY_CONCURRENCY."
},
{
"id": "US-038",
"title": "38 — Tracker cluster join via a single seed peer",
"description": "Tracker cluster membership is static today — a newcomer configured with only one existing peer is never learned by the rest of the hive and quorum math diverges. A joining tracker configured with any one live seed announces via hive-HMAC-signed POST /v1/cluster/join; membership changes replicate through the Raft log and persist across restarts.",
"acceptanceCriteria": [
"Start trackers A+B; start C with only A as seed → within one election timeout A, B, and C report the same 3-peer membership on GET /v1/cluster/peers, and a value proposed on C commits on A and B",
"Join without a valid hive signature is rejected with 403; join to a follower is forwarded to the leader",
"Restarting C with its seed offline rejoins from persisted membership",
"Standalone tracker (no seeds) behaves exactly as today",
"python -m pytest passes from repo root"
],
"priority": 38,
"status": "open",
"notes": "Source issue: docs/issues/38-tracker-seed-join.md. Out of scope: peer removal, joint consensus, automatic seed retry.",
"dependsOn": [
"US-013",
"US-017"
]
},
{
"id": "US-039",
"title": "39 — Caller Credit granted once per account; chat requires account keys",
"description": "DEFAULT_STARTING_CREDIT=0 and no grant path leaves every fresh public tracker request at 402. Grant Caller Credit once per account on first API key creation via deterministic event id caller-credit-{account_id}; chat on accounts-enabled trackers requires a real active sk-mesh- key (401 for invented bearers).",
"acceptanceCriteria": [
"Fresh account → first key → key has --starting-credit balance; chat succeeds",
"Second key on the same account → no additional credit",
"Revoke-and-recreate keys → still no additional credit (deterministic event id)",
"Random bearer string on an accounts-enabled tracker → 401, never 402/free work",
"Tracker without accounts store: gate behavior unchanged",
"--starting-credit 0 disables the grant entirely (mainnet posture)",
"python -m pytest passes from repo root"
],
"priority": 39,
"status": "done",
"notes": "Source issue: docs/issues/39-caller-credit-account-keys.md. Critical for friends-test inference.",
"dependsOn": [
"US-031",
"US-035"
],
"completionNotes": "Caller credit granted once per account on first API key via deterministic event id; tests/test_accounts.py covers grant, revoke, and invented-bearer rejection."
},
{
"id": "US-040",
"title": "40 — Devnet top-up button on the dashboard",
"description": "After Caller Credit (US-039) is spent, devnet friends need a dashboard faucet refill without on-chain USDT deposits. POST /v1/account/topup (session-authenticated) credits a configured fixed amount per click; flag off returns 404 and hides the button.",
"acceptanceCriteria": [
"Flag off: endpoint 404s, dashboard shows no top-up button",
"Flag on: logged-in user tops up own key, balance rises by exactly N",
"Topping up another account's key → 403",
"python -m pytest passes from repo root"
],
"priority": 40,
"status": "done",
"notes": "Source issue: docs/issues/40-devnet-dashboard-topup.md. Mainnet deployments set --devnet-topup 0.",
"dependsOn": [
"US-039"
],
"completionNotes": "POST /v1/account/topup with session auth and flag gating; tests/test_accounts.py covers flag off/on, own-account credit, and cross-account 403."
},
{
"id": "US-041",
"title": "41 — Account wallet: browser-extension signing, in-browser generation, export-only",
"description": "Accounts need a visible wallet for deposit attribution without the tracker ever holding private keys. Dashboard integrates Solana wallet-adapter connect+nonce proof, or in-browser keypair generation with one-time export; no private-key import endpoint.",
"acceptanceCriteria": [
"Connect-extension flow stores a verified pubkey (rejects unsigned/mismatched nonce proofs)",
"Generate flow: pubkey lands on the account; private key is never sent to the tracker, export works",
"No endpoint or UI accepts a private key",
"Deposits to the shown address credit the account's keys via the existing watcher",
"Address visible on the account panel after either flow",
"python -m pytest passes from repo root"
],
"priority": 41,
"status": "open",
"notes": "Source issue: docs/issues/41-account-wallet-keypair.md. Not needed for devnet friends test; needed before mainnet.",
"dependsOn": [
"US-032",
"US-039"
]
},
{
"id": "US-042",
"title": "42 — GGUF/llama.cpp node backend (phase C whole-model first)",
"description": "Node backend is transformers-only today; large MoE models on consumer hardware require GGUF via llama.cpp. Phase C: whole-model GGUF nodes (single-hop routes) first; partial-layer distributed GGUF deferred to ADR-0024. Also: GGUF catalog entries, Strix Halo/Vulkan hardware detection, download dir applies to GGUF files.",
"acceptanceCriteria": [
"A node with --gguf <repo-or-path> --quant IQ3_XXS serves /v1/chat/completions via llama.cpp with GPU offload where available",
"Tracker treats it as a full-coverage node (single-hop routes, billing works)",
"Streamed responses work through the tracker proxy and the relay (US-036)",
"python -m pytest passes from repo root (llama.cpp behind an optional extra)"
],
"priority": 42,
"status": "in-design",
"notes": "Source issue: docs/issues/42-gguf-llamacpp-node-backend.md. Phase C before ADR-0024 distributed worker; see runtime sequencing in issue file.",
"dependsOn": [
"US-036"
]
},
{
"id": "US-043",
"title": "43 — Dashboard model search and model cards",
"description": "Dashboard lacks model-centric discovery. Add server-side HF search proxy merged with tracker presets and live coverage; model cards show architecture, coverage gaps, pricing, memory per quant, and a request-this-model action. Featured section driven by CURATED_MODELS including GGUF once US-042 lands.",
"acceptanceCriteria": [
"Searching a HF repo id or free text returns results without the browser calling HF directly",
"A served model's card shows live coverage and a working chat-now state",
"An unserved model's card shows the request action and estimated memory per quant",
"python -m pytest passes from repo root"
],
"priority": 43,
"status": "open",
"notes": "Source issue: docs/issues/43-dashboard-model-search-cards.md. Post-deploy polish.",
"dependsOn": [
"US-035"
]
},
{
"id": "US-044",
"title": "44 — Tracker as model-file source; nodes download only their shard",
"description": "Second nodes joining a fleet today download entire HF snapshots even for small shard assignments. Tracker --models-dir advertises layer-scoped safetensors subsets; nodes race tracker/peer sources before HF allow_patterns fallback. Hard half remaining: meta-device partial model materialization so resident memory scales with shard size, not full model size.",
"acceptanceCriteria": [
"Tracker started with --models-dir / MESHNET_MODELS_DIR advertises local model-file sources in assignment responses",
"Tracker serves a tar stream (or per-file API) containing only safetensors files for the assigned layer range plus config/tokenizer/index metadata",
"Node downloader tries exact-shard peers, then tracker/peer file subsets, then HF snapshot_download with allow_patterns — never silently full-repo when layer index is available",
"Two-machine test: machine B receives only its assigned range from machine A — nothing fetched from HF",
"Machine B resident memory scales with its shard size, not model size",
"Checksums verified end-to-end; corrupted transfer falls back cleanly",
"Single-node/full-model flows unchanged",
"python -m pytest passes from repo root"
],
"priority": 44,
"status": "in-progress",
"notes": "Source issue: docs/issues/44-tracker-shard-source-partial-download.md. Download path and partial LOAD implemented; live two-machine LAN verification remains.",
"dependsOn": [
"US-004",
"US-012"
],
"completionNotes": "Tracker models-dir indexing, layer-scoped tar stream, HF allow_patterns client-side from remote index, per-file download API with retries, symlink dereference in tar writers. Partial LOAD via init_empty_weights + layer-scoped safetensors materialization; memory-scaling and checksum fallback tests pass. Remaining: live two-machine test (machine B receives only assigned files from A, no HF)."
},
{
"id": "US-045",
"title": "45 — Dual-rate billing: separate input and output token prices",
"description": "Ledger has one price_per_1k_tokens and stream vs non-stream paths disagree on input vs output counting. Charge both input and output tokens at separate rates per model; HF pricing refresher applies 80% of each marketplace side separately.",
"acceptanceCriteria": [
"Streamed and non-streamed requests for the same exchange bill the same split (input charged in both)",
"A model with asymmetric provider rates bills input and output differently; usage_for / billing events expose the split",
"Old persisted billing events replay byte-identically (balances unchanged)",
"HF refresh sets both rates from the marketplace row, not the average",
"Spend cap (--max-charge-per-request) uses the dual rates",
"python -m pytest passes from repo root"
],
"priority": 45,
"status": "in-progress",
"notes": "Source issue: docs/issues/45-dual-rate-billing.md. Billing correctness before friends test.",
"dependsOn": [
"US-031"
]
},
{
"id": "US-046",
"title": "46 — Tracker .env awareness + first-node auto-join bootstrap",
"description": "Fresh trackers return 503 on auto-join because deployability ignores the joining caller's hardware, and meshnet-tracker ignores .env MESHNET_DOWNLOAD_DIR. Fix empty-registry bootstrap, tracker env loading parity with node CLI, models-dir fallback chain, and tar dereference for HF symlink snapshots.",
"acceptanceCriteria": [
"Fresh tracker (empty registry) + caller with enough memory for a recommended preset → /v1/network/assign returns 200 with model_sources populated when tracker holds a local snapshot",
"Fresh tracker + caller too small for any recommended preset → still 503",
"meshnet-tracker start in a directory with .env setting MESHNET_DOWNLOAD_DIR serves /v1/model-files/download from that dir with no extra flags",
"Explicit --models-dir and MESHNET_MODELS_DIR still take precedence",
"python -m pytest passes from repo root"
],
"priority": 46,
"status": "needs-review",
"notes": "Source issue: docs/issues/46-tracker-env-and-first-node-autojoin.md. Verified live 2026-07-06.",
"dependsOn": [
"US-044"
],
"completionNotes": "Empty-registry synthesizes caller as candidate node; tracker loads .env; models_dir falls back MESHNET_MODELS_DIR → MESHNET_DOWNLOAD_DIR; tar dereference=True. Pytest passes aside from known port-7000 env conflict."
},
{
"id": "US-047",
"title": "47 — Tracker-first model downloads: visibility, sane timeouts, RAM-based sizing",
"description": "Explicit --model startup should skip pointless auto-join; tracker/peer sources preferred over HF with visible progress and 30 s socket timeouts; client abort during tar stream logs one line; CPU nodes size shards from RAM not phantom GPU VRAM; per-file downloads for robustness over fragile multi-GB tar streams.",
"acceptanceCriteria": [
"Node started with explicit --model never queries /v1/network/assign and never prints auto-join unavailable",
"Tracker/peer model source preferred; HF only when no source, all sources fail, or --tracker-source-disabled",
"Tracker-source downloads print progress every 512 MB and print exception + URL on failure",
"A ≥2 s read stall no longer aborts a tracker model-source download (30 s socket timeout)",
"Client disconnect during /v1/model-files/download logs one line on the tracker, no traceback",
"CPU node with big RAM gets a RAM-sized shard: sizing paths ignore VRAM unless device=cuda",
"Live two-machine retest: Windows node downloads from tracker at LAN speed with RAM-sized shard assignment",
"python -m pytest passes from repo root"
],
"priority": 47,
"status": "in-progress",
"notes": "Source issue: docs/issues/47-model-source-download-visibility.md. Engineering largely complete 2026-07-06; live two-machine retest pending.",
"dependsOn": [
"US-044",
"US-046"
],
"completionNotes": "Skip auto-join when model explicit; sequential source try with progress; 30 s model-source timeout; assignment_vram_mb for CPU; per-file /v1/model-files/download with manifest and retries. Remaining: live Windows two-machine retest."
},
{
"id": "US-049",
"title": "49 — Mainnet USDT cutover: two-wallet pilot checklist",
"description": "Operator runbook to move from Solana devnet + mock-USDT to mainnet + real USDT for a minimal pilot: one client wallet (deposits USDT, pays for inference) and one node wallet (receives batched payouts). Treasury holds USDT float and SOL for fees. TAI deferred per ADR-0025.",
"acceptanceCriteria": [
"Devnet smoke completed: settlement loop pays with --settle-period 60 and mock mint",
"python -m pytest tests/test_settlement_loop.py -q passes (includes 24 h / 5 USDT gate tests)",
"Mainnet tracker configured: real USDT mint, --starting-credit 0, --devnet-topup 0, --settle-period 86400",
"Client wallet deposit credits API key ledger; inference debits balance",
"Node wallet receives at least one confirmed mainnet USDT payout batch",
"Sub-threshold pending not paid before 24 h; ≥5 USDT pending triggers immediate payout"
],
"priority": 49,
"status": "open",
"notes": "Source issue: docs/issues/49-mainnet-usdt-cutover-two-wallet-pilot.md. ADR-0025 covers optional TAI mint; not blocking this pilot.",
"dependsOn": [
"US-032",
"US-033",
"US-039"
]
},
{
"id": "US-048",
"title": "48 — Memory budget, shard slots, and dropout relocation hardening",
"description": "Harden the capacity contract around US-013 coverage-first assignment: enforce memory budget and max_loaded_shards in rebalance/dropout relocation, and decide whether one node may host multiple concurrent shard backends or max_loaded_shards remains metadata until runtime support lands.",
"acceptanceCriteria": [
"Assignment/rebalance never exceeds memory budget or max_loaded_shards",
"Dropout test restores full coverage without violating capacity limits",
"CLI --memory and --max-shards reflected in registration payload",
"python -m pytest tests/test_tracker_routing.py tests/test_node_startup.py passes"
],
"priority": 48,
"status": "open",
"notes": "Source issue: docs/issues/48-memory-budget-shard-slots-and-dropout-relocation.md. Renumbered from duplicate slot 20. Enables spare shard slots for ADR-0026 managed placement.",
"dependsOn": [
"US-013"
]
},
{
"id": "US-050",
"title": "50 — Qwen3.6-27B demand-driven managed placement",
"description": "Offer pinned Qwen/Qwen3.6-27B as a recommended text-only chat model. Valid chat requests prove demand; when spare fleet capacity exists, the tracker assigns managed nodes to reach complete coverage. Pinned startup assignments remain immutable per ADR-0026; NCA admission required before routable.",
"acceptanceCriteria": [
"First valid request for an uncovered variant queues demand and returns 503 model_loading until complete validated coverage exists",
"Managed assignments use only spare capacity and carry managed: true",
"Pinned startup assignments are never silently retargeted",
"Optional quantization field (bfloat16/int8/nf4) with coverage-vote UI semantics per scratch PRD",
"python -m pytest passes from repo root"
],
"priority": 50,
"status": "in-design",
"notes": "Source: .scratch/qwen3.6-27b-demand-placement/PRD.md and docs/issues/50-qwen3.6-27b-demand-placement.md. Reconciled with ADR-0026 and ADR-0023.",
"dependsOn": [
"US-035",
"US-048"
]
}
],
"metadata": {
"updatedAt": "2026-07-01T00:00:00.000Z",
"updatedAt": "2026-07-13T17:00:00.000Z",
"statusVocabulary": {
"open": "Not started",
"in-design": "Decisions pending before implementation can begin",

View File

@@ -36,6 +36,12 @@ def _load_env_file(path: Path) -> None:
os.environ[key] = value
def _apply_relay_concurrency_flag(value: int | None) -> None:
"""Expose relay bridge worker cap via CLI (env MESHNET_RELAY_CONCURRENCY)."""
if value is not None:
os.environ["MESHNET_RELAY_CONCURRENCY"] = str(max(1, value))
def _load_env_defaults() -> None:
"""Load machine-specific, local, and user-level node env defaults."""
machine = socket.gethostname().strip()
@@ -189,6 +195,8 @@ def _cmd_default(args) -> int:
if getattr(args, "cpu", False):
overrides["force_cpu"] = True
_apply_relay_concurrency_flag(getattr(args, "relay_concurrency", None))
if overrides:
cfg = merge_cli_overrides(cfg, **overrides)
@@ -349,6 +357,8 @@ def _cmd_start(args) -> int:
if getattr(args, "node_name", None):
cfg["node_name"] = args.node_name
_apply_relay_concurrency_flag(getattr(args, "relay_concurrency", None))
# Legacy start: just run without the dashboard (keep original blocking loop)
from .startup import run_startup
@@ -433,6 +443,8 @@ def main() -> None:
help="Set PyTorch inter-op CPU worker threads")
parser.add_argument("--cpu", action="store_true",
help="Force CPU inference even when a GPU is available")
parser.add_argument("--relay-concurrency", type=int, metavar="N",
help="Max concurrent relay-http-request workers (env MESHNET_RELAY_CONCURRENCY)")
parser.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
parser.add_argument("--no-tui", action="store_true", help="Plain-text output (no rich dashboard)")
parser.add_argument("--compact", action="store_true", help="Single-line status output")
@@ -510,6 +522,8 @@ def main() -> None:
help="Set PyTorch inter-op CPU worker threads")
start_cmd.add_argument("--cpu", action="store_true",
help="Force CPU inference even when a GPU is available")
start_cmd.add_argument("--relay-concurrency", type=int, metavar="N",
help="Max concurrent relay-http-request workers (env MESHNET_RELAY_CONCURRENCY)")
start_cmd.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
start_cmd.add_argument("--tracker-source-disabled", action="store_true",
help="Skip tracker/peer model-file sources and download from HuggingFace directly")

View File

@@ -2,6 +2,7 @@
import json
import os
import shutil
import subprocess
import time
@@ -183,6 +184,17 @@ def with_forced_cpu(hw: dict) -> dict:
return forced
def _with_model_drive(profile: dict) -> dict:
"""Attach free space for the default model cache drive to tracker diagnostics."""
try:
cache_root = os.path.expanduser("~/.cache/meshnet/shards")
profile["model_drive_free_bytes"] = shutil.disk_usage(os.path.expanduser("~")).free
profile["model_drive_path"] = cache_root
except OSError:
pass
return profile
def detect_hardware() -> dict:
"""Detect GPU model and available VRAM. Returns hardware profile dict."""
ram_mb = _detect_ram_mb()
@@ -208,23 +220,23 @@ def detect_hardware() -> dict:
}
if torch_gpu is not None and torch_gpu.get("gcn_arch"):
profile["gcn_arch"] = torch_gpu["gcn_arch"]
return profile
return _with_model_drive(profile)
except ImportError:
pass
torch_inventory = _gpu_inventory_profile(torch_gpu, ram_mb)
if torch_inventory is not None:
return torch_inventory
return _with_model_drive(torch_inventory)
nvidia_gpu = _gpu_inventory_profile(_detect_nvidia_smi_gpu_memory(), ram_mb)
if nvidia_gpu is not None:
return nvidia_gpu
return _with_model_drive(nvidia_gpu)
windows_gpu = _gpu_inventory_profile(_detect_windows_gpu_memory(), ram_mb)
if windows_gpu is not None:
return windows_gpu
return _with_model_drive(windows_gpu)
return {
return _with_model_drive({
"device": "cpu",
"gpu_name": None,
"vram_mb": 0,
@@ -232,7 +244,7 @@ def detect_hardware() -> dict:
"shared_vram_mb": 0,
"ram_mb": ram_mb,
"cuda_available": False,
}
})
def benchmark_throughput_checked(device_str: str = "cpu") -> tuple[float, bool, str | None]:

View File

@@ -899,12 +899,41 @@ def _load_partial_model_from_snapshot(
dtype=dtype,
)
for module in _active_modules_for_shard(model, shard_start, shard_end):
if hasattr(module, "to"):
module.to(device)
_finalize_active_shard_modules_on_device(model, shard_start, shard_end, device)
return model
def _finalize_active_shard_modules_on_device(
model: Any, shard_start: int, shard_end: int, device: Any
) -> None:
"""Place active shard modules on device without copying unmaterialized meta weights."""
for module in _active_modules_for_shard(model, shard_start, shard_end):
parameters = getattr(module, "parameters", None)
if not callable(parameters):
if hasattr(module, "to"):
module.to(device)
continue
params = list(parameters(recurse=True))
buffers_fn = getattr(module, "buffers", None)
buffers = list(buffers_fn(recurse=True)) if callable(buffers_fn) else []
tensors = params + buffers
if not tensors:
if hasattr(module, "to"):
module.to(device)
continue
if all(tensor.device.type == "meta" for tensor in tensors):
to_empty = getattr(module, "to_empty", None)
if callable(to_empty):
to_empty(device)
continue
if all(tensor.device.type != "meta" for tensor in tensors):
if hasattr(module, "to"):
module.to(device)
continue
# Partially materialized: set_module_tensor_to_device already placed loaded
# weights on the target device; leave remaining meta parameters untouched.
def _model_load_plan(
auto_config: Any,
model_id: str,

View File

@@ -44,6 +44,7 @@ class SeamSample:
cache_mode: CacheMode
model_ms: float
encode_ms: float
activation_decode_ms: float
framing_ms: float
metadata_ms: float
copy_allocation_ms: float
@@ -52,6 +53,7 @@ class SeamSample:
decompression_ms: float
connection_setup_ms: float
queue_wait_ms: float
local_http_forwarding_ms: float
transport_ms: float
seam_latency_ms: float
payload_bytes: int
@@ -120,6 +122,10 @@ def _summary(samples: list[SeamSample]) -> dict[str, float | int]:
"compression_cpu_ms": round(
sum(sample.compression_ms + sample.decompression_ms for sample in samples), 4
),
"model_execution_ms": round(sum(sample.model_ms for sample in samples), 4),
"activation_encoding_ms": round(sum(sample.encode_ms for sample in samples), 4),
"activation_decoding_ms": round(sum(sample.activation_decode_ms for sample in samples), 4),
"local_http_forwarding_ms": round(sum(sample.local_http_forwarding_ms for sample in samples), 4),
"peak_buffered_bytes": max((sample.copy_allocation_bytes for sample in samples), default=0),
}
@@ -159,6 +165,7 @@ class _StubTransport:
queue_wait_ms = 0.0 if self.mode == "direct" else 0.18 + (0.05 if token_index is not None and token_index % 2 else 0.0)
model_ms = 1.6 if phase == "prefill" else 0.45
encode_ms = 0.16 if phase == "prefill" else 0.06
activation_decode_ms = 0.055 if phase == "prefill" else 0.02
# Keep framing/metadata/copy costs explicit rather than hiding them in
# serialization or transport time. The stub owns one binary frame and
# one response body per hop; no base64 body is modeled.
@@ -168,20 +175,26 @@ class _StubTransport:
copy_allocation_bytes = wire_bytes + payload_bytes
compression_ms = 0.09 if self.scenario.compression else 0.0
decompression_ms = 0.07 if self.scenario.compression else 0.0
# Both routes finish by forwarding the decoded activation to the local
# tail-node HTTP handler; relay adds its own queue before that hop.
local_http_forwarding_ms = 0.11 if self.mode == "direct" else 0.16
transport_ms = (0.32 if self.mode == "direct" else 0.61) + wire_bytes / 100_000
seam_latency_ms = round(
model_ms + encode_ms + framing_ms + metadata_ms + copy_allocation_ms
+ compression_ms + decompression_ms + connection_setup_ms + queue_wait_ms + transport_ms,
model_ms + encode_ms + activation_decode_ms + framing_ms + metadata_ms + copy_allocation_ms
+ compression_ms + decompression_ms + connection_setup_ms + queue_wait_ms + transport_ms
+ local_http_forwarding_ms,
4,
)
return SeamSample(
phase=phase, token_index=token_index, session_id=self.session_id,
activation_id=f"benchmark-activation-{self._activation_count}", seam="head->tail", mode=self.mode,
cache_mode=self.cache_mode, model_ms=model_ms, encode_ms=encode_ms,
activation_decode_ms=activation_decode_ms,
framing_ms=framing_ms, metadata_ms=metadata_ms,
copy_allocation_ms=copy_allocation_ms, copy_allocation_bytes=copy_allocation_bytes,
compression_ms=compression_ms, decompression_ms=decompression_ms,
connection_setup_ms=connection_setup_ms, queue_wait_ms=queue_wait_ms,
local_http_forwarding_ms=local_http_forwarding_ms,
transport_ms=round(transport_ms, 4), seam_latency_ms=seam_latency_ms,
payload_bytes=payload_bytes, wire_bytes=wire_bytes,
compression_ratio=round(payload_bytes / wire_bytes, 4), connection_attempted=connection_attempted,
@@ -329,9 +342,10 @@ def run_real_model_lan_benchmark(url: str, *, model: str, timeout: float = 120.0
sample = SeamSample(
phase="decode", token_index=0, session_id=session_id, activation_id="lan-activation-1",
seam="head->tail", mode="direct", cache_mode="cached", model_ms=0.0, encode_ms=0.0,
activation_decode_ms=0.0,
framing_ms=0.0, metadata_ms=0.0, copy_allocation_ms=0.0, copy_allocation_bytes=0,
compression_ms=0.0, decompression_ms=0.0, connection_setup_ms=elapsed_ms,
queue_wait_ms=0.0, transport_ms=elapsed_ms, seam_latency_ms=elapsed_ms,
queue_wait_ms=0.0, local_http_forwarding_ms=0.0, transport_ms=elapsed_ms, seam_latency_ms=elapsed_ms,
payload_bytes=len(body), wire_bytes=len(body) + len(response_body), compression_ratio=1.0,
connection_attempted=True,
)
@@ -354,6 +368,10 @@ def format_summary(report: dict) -> str:
f"{decode['tokens_per_sec']:.1f} tok/s; {decode['bytes_per_token']:.0f} B/tok; "
f"seam {seam['payload_bytes']}/{seam['wire_bytes']} B "
f"({seam['compression_ratio']:.2f}x); connections {run['connections']['attempts']}; "
f"model/encode/decode {decode['model_execution_ms']:.2f}/"
f"{decode['activation_encoding_ms']:.2f}/{decode['activation_decoding_ms']:.2f} ms; "
f"compression {decode['compression_cpu_ms']:.2f} ms; "
f"HTTP {decode['local_http_forwarding_ms']:.2f} ms; "
f"queue p95 {decode['p95_queue_wait_ms']:.2f} ms"
)
return "\n".join(lines)

View File

@@ -1545,6 +1545,26 @@ class TorchNodeServer:
def apply_tracker_directives(self, directives: list[dict]) -> dict | None:
"""Apply tracker shard directives (LOAD_SHARD replace, ADD_SHARD load-more)."""
drop_directive = next(
(directive for directive in reversed(directives) if directive.get("action") == "DROP_SHARD"),
None,
)
if drop_directive is not None:
model_id = str(drop_directive.get("model") or "")
removed = self._backends.pop(model_id, None)
if removed is None:
return None
if self._backends:
self._backend = next(iter(self._backends.values()))
self._tracker_mode = self._backend.shard_start == 0
else:
self._backend = None
self._tracker_mode = False
if self._server is not None:
self._server.backends = dict(self._backends)
self._server.backend = self._backend
self._server.tracker_mode = self._tracker_mode
return {"action": "DROP_SHARD", "model": model_id}
add_directive = next(
(directive for directive in reversed(directives) if directive.get("action") == "ADD_SHARD"),
None,
@@ -1574,6 +1594,8 @@ class TorchNodeServer:
flush=True,
)
try:
if replacing:
self._backends.clear()
new_backend = _load_backend(model_id, shard_start, shard_end, quantization, self._cache_dir)
except TypeError:
new_backend = _load_backend(model_id, shard_start, shard_end, quantization)

View File

@@ -21,6 +21,9 @@ import uuid
DEFAULT_PRICE_PER_1K_TOKENS = 0.02 # USDT
DEFAULT_STARTING_CREDIT = 0.0 # USDT; accounts must be funded before inference
DEFAULT_BILLING_DB_PATH = "billing.sqlite"
DEFAULT_SETTLE_PERIOD = 86400.0 # seconds — max time between node payouts (24 h prod)
DEFAULT_PAYOUT_THRESHOLD = 5.0 # USDT — immediate payout when pending exceeds this
DEFAULT_PAYOUT_DUST_FLOOR = 0.01 # USDT — never pay less than this
NODE_REVENUE_SHARE = 0.90 # nodes 90% / protocol cut 10% (ADR-0015)

View File

@@ -8,7 +8,12 @@ import time
from pathlib import Path
from .accounts import DEFAULT_ACCOUNTS_DB_PATH
from .billing import DEFAULT_BILLING_DB_PATH
from .billing import (
DEFAULT_BILLING_DB_PATH,
DEFAULT_PAYOUT_DUST_FLOOR,
DEFAULT_PAYOUT_THRESHOLD,
DEFAULT_SETTLE_PERIOD,
)
from .capability import ALL_POLICIES as ALL_CAPABILITY_POLICIES
from .hf_pricing import DEFAULT_HF_PRICING_LOG_DB_PATH
from .logging_setup import (
@@ -237,19 +242,19 @@ def main() -> None:
common.add_argument(
"--settle-period",
type=float,
default=86400.0,
default=DEFAULT_SETTLE_PERIOD,
help="Max seconds between payouts to a node (dev: 60, prod: 86400)",
)
common.add_argument(
"--payout-threshold",
type=float,
default=5.0,
default=DEFAULT_PAYOUT_THRESHOLD,
help="Pending USDT that triggers an immediate payout (dev: 0)",
)
common.add_argument(
"--payout-dust-floor",
type=float,
default=0.01,
default=DEFAULT_PAYOUT_DUST_FLOOR,
help="Never pay out less than this many USDT",
)
common.add_argument(

View File

@@ -22,8 +22,9 @@
border-bottom:1px solid var(--border); flex-shrink:0; }
header h1 { font-size:16px; margin:0; color:var(--accent); }
header .meta { color:var(--dim); font-size:12px; }
main { display:grid; grid-template-columns:repeat(auto-fit,minmax(340px,1fr));
gap:14px; padding:14px 20px; }
main { display:grid; grid-template-columns:1fr;
gap:14px; padding:14px 20px; }
main > section { width:100%; min-width:0; }
body.chat-tab-active main {
flex:1; min-height:0; display:flex; flex-direction:column;
padding:0; gap:0; overflow:hidden;
@@ -32,6 +33,7 @@
border-radius:8px; padding:12px 14px; min-height:80px; }
section h2 { margin:0 0 8px; font-size:12px; text-transform:uppercase;
letter-spacing:.08em; color:var(--dim); }
.panel-heading { display:flex; align-items:center; justify-content:space-between; gap:8px; }
table { width:100%; border-collapse:collapse; font-size:12px; }
th { text-align:left; color:var(--dim); font-weight:normal;
border-bottom:1px solid var(--border); padding:2px 6px 4px 0; }
@@ -70,6 +72,12 @@
background:transparent; color:var(--dim); padding:5px 0 8px; }
.dashboard-tabs button.active { color:var(--accent); border-bottom-color:var(--accent); }
.wide { grid-column:1 / -1; }
/* Compact status cards fan out on desktop; tables remain readable at half width. */
@media (min-width:900px) {
main { grid-template-columns:repeat(4,minmax(0,1fr)); }
main > section { grid-column:span 1; }
.wide { grid-column:span 2; }
}
section[hidden] { display:none !important; }
section.chat-section {
padding:0; border:0; border-radius:0; background:var(--bg); min-height:0;
@@ -204,7 +212,7 @@
.chat-compose button:disabled { opacity:.45; cursor:not-allowed; }
.console {
background:var(--bg); border:1px solid var(--border); border-radius:6px;
min-height:160px; max-height:280px; overflow:auto; padding:7px 9px;
min-height:160px; max-height:520px; overflow-y:auto; overflow-x:auto; padding:7px 9px;
white-space:pre-wrap; word-break:break-word; font-size:11px;
}
.console-line { padding:1px 0; border-bottom:1px solid #161b22; }
@@ -247,8 +255,7 @@
</nav>
<main>
<section data-tab="overview" id="account-section"><h2>Account</h2><div id="account">loading…</div></section>
<section data-tab="overview"><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
<section data-tab="overview"><h2>Nodes &amp; coverage</h2><div id="nodes" class="empty">loading…</div></section>
<section data-tab="overview" class="wide"><h2>Nodes &amp; coverage</h2><div id="nodes" class="empty">loading…</div></section>
<section data-tab="overview"><h2>Model usage (RPM)</h2><div id="stats" class="empty">loading…</div></section>
<section data-tab="overview" class="wide"><h2>Routing (learned)</h2><div id="routing" class="empty">loading…</div></section>
<section data-tab="overview" class="wide"><h2>Model inference speed</h2><div id="model-speed-chart" class="empty">loading…</div></section>
@@ -287,10 +294,13 @@
<section data-tab="billing"><h2>Request history</h2><div id="billing-usage" class="empty">login required</div></section>
<section data-tab="billing" data-admin-only><h2>Node pending payouts</h2><div id="pending" class="empty">admin login required</div></section>
<section data-tab="billing" data-admin-only><h2>Settlement history</h2><div id="settlements" class="empty">admin login required</div></section>
<section data-tab="admin"><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
<section data-tab="admin" class="wide"><h2>Model placement</h2><div id="admin-model-placement-status" class="dim">Choose a model to load or release.</div><div id="admin-model-placement" class="empty">admin login required</div></section>
<section data-tab="admin" class="wide"><h2>Total node pool</h2><div id="admin-node-pool" class="empty">admin login required</div></section>
<section data-tab="admin" id="admin-section"><h2>All accounts (admin)</h2><div id="admin" class="empty"></div></section>
<section data-tab="admin" data-admin-only><h2>Strikes / bans / forfeitures</h2><div id="fraud" class="empty">admin login required</div></section>
<section data-tab="admin"><h2>Client balances</h2><div id="clients" class="empty">admin login required</div></section>
<section data-tab="admin" class="wide"><h2>Console output</h2><div id="console" class="console empty">admin login required</div></section>
<section data-tab="admin" class="wide"><h2 class="panel-heading">Console output <button class="small" type="button" onclick="clearConsole()">clear</button></h2><div id="console" class="console empty">admin login required</div></section>
<section data-tab="testing" data-admin-only>
<h2>Test run status</h2>
<div id="testing-error" class="testing-error" hidden></div>
@@ -524,7 +534,67 @@ function renderHive(raft) {
(raft.peers ? `<br>peers: ${esc(Array.isArray(raft.peers) ? raft.peers.join(", ") : raft.peers)}` : "");
}
function renderNodes(map) {
function nodeLiveTps(node) {
const reqs = (node.stats && node.stats.current_requests) || [];
let best = null;
for (const req of reqs) {
const rate = req.tokens_per_sec;
if (rate == null) continue;
if (best === null || rate > best) best = rate;
}
return best;
}
function buildModelPriceMap(modelsResp) {
const prices = new Map();
for (const model of (modelsResp && modelsResp.data) || []) {
if (!model.pricing) continue;
for (const key of modelLookupKeys(model)) {
prices.set(key, model.pricing);
prices.set(modelAliasKey(key), model.pricing);
}
}
return prices;
}
function lookupModelPricing(modelKey, aliasMap, priceMap) {
const candidates = [modelKey];
const alias = aliasMap.get(modelAliasKey(modelKey));
if (alias) candidates.push(alias);
for (const candidate of candidates) {
const hit = priceMap.get(candidate) || priceMap.get(modelAliasKey(candidate));
if (hit) return hit;
}
return null;
}
function formatInferencePrice(pricing) {
if (!pricing) return '<span class="dim">—</span>';
const input = pricing.input_per_1k_usdt;
const output = pricing.output_per_1k_usdt;
if (input == null && output == null) return '<span class="dim">—</span>';
if (input === output || output == null) {
return `<span class="num">${usdt(input ?? output)}/1k</span>`;
}
return `<span class="num">${usdt(input)}/${usdt(output)}</span> <span class="dim">in/out per 1k</span>`;
}
function modelGroupTypicalTps(modelKey, aliasMap, routing) {
const sampleNode = { hf_repo: modelKey, model: modelKey };
const resolved = resolveModelGroup(sampleNode, aliasMap);
const model = {
id: resolved,
name: resolved,
hf_repo: modelKey.includes("/") ? modelKey : null,
aliases: [modelKey, resolved].filter(Boolean),
};
return chatModelTypicalTps(model, routing);
}
function renderNodes(payload) {
const map = payload && payload.map;
const routing = payload && payload.routing;
const priceMap = payload && payload.priceMap;
const nodes = (map && map.nodes) || [];
if (!nodes.length) {
$("nodes").innerHTML = '<div class="empty">no nodes registered</div>'; return;
@@ -543,16 +613,22 @@ function renderNodes(map) {
.map(n => n.model_supply && n.model_supply.served_model_copies)
.filter(v => v !== null && v !== undefined);
const served = servedValues.length ? Math.max(...servedValues) : undefined;
html += `<div><b>${esc(model)}</b> <span class="dim">(${group.length} node${group.length===1?"":"s"} · ${esc(copies(served))} served)</span></div>`;
html += table(["node", "shard", "tps (1h)", "queue", "served"], group.map(n => {
const typicalTps = modelGroupTypicalTps(model, aliasMap, routing);
const pricing = lookupModelPricing(model, aliasMap, priceMap);
html += `<div style="margin-top:8px"><b>${esc(model)}</b> <span class="dim">(` +
`${group.length} node${group.length === 1 ? "" : "s"} · ${esc(copies(served))} served · ` +
`${esc(tps(typicalTps))} tok/s · price ${formatInferencePrice(pricing)})</span></div>`;
html += table(["node", "shard", "last tps", "tps (1h)", "queue", "served"], group.map(n => {
const modelStats = (n.throughput && (n.throughput[n.hf_repo] || n.throughput[n.model])) || {};
return [
nodeDisplayCell(n),
esc(`${n.shard_start ?? "?"}-${n.shard_end ?? "?"}`),
`<span class="num">${esc(tps(modelStats.tokens_per_sec_last_hour))}</span>`,
esc(String((n.stats && n.stats.queue_depth) ?? 0)),
`<span class="num">${esc(copies(n.model_supply && n.model_supply.served_model_copies))}</span>`,
]; }));
nodeDisplayCell(n),
esc(`${n.shard_start ?? "?"}-${n.shard_end ?? "?"}`),
`<span class="num">${esc(tps(nodeLiveTps(n)))}</span>`,
`<span class="num">${esc(tps(modelStats.tokens_per_sec_last_hour))}</span>`,
esc(String((n.stats && n.stats.queue_depth) ?? 0)),
`<span class="num">${esc(copies(n.model_supply && n.model_supply.served_model_copies))}</span>`,
];
}));
}
$("nodes").innerHTML = html;
}
@@ -1006,13 +1082,22 @@ function renderBillingUsage(records) {
table(["time", "model", "api key", "tokens", "cost (USDT)"], rows);
}
let consoleClearedAt = 0;
const CONSOLE_MAX_LINES = 1000;
function clearConsole() {
consoleClearedAt = Date.now() / 1000;
panelSig.console = null;
renderConsole({ events: [] });
}
function renderConsole(data) {
const events = (data && data.events) || [];
const events = ((data && data.events) || []).filter(event => event.ts > consoleClearedAt);
if (!events.length) {
$("console").innerHTML = '<div class="empty">no console events</div>';
return;
}
$("console").innerHTML = events.slice(-120).map(e => {
$("console").innerHTML = events.slice(-CONSOLE_MAX_LINES).map(e => {
const level = String(e.level || "info");
const cls = level === "error" ? "console-level-error" : level === "warn" ? "console-level-warn" : "console-level-info";
const fields = e.fields && Object.keys(e.fields).length ? " " + JSON.stringify(e.fields) : "";
@@ -1706,7 +1791,7 @@ async function requestSelectedModelLoad() {
if (!selectedChatModel) return;
const button = $("request-model-load");
if (button) button.disabled = true;
const result = await apiCall("/v1/models/load", "POST", { model: selectedChatModel });
const result = await apiCall("/v1/models/load", "POST", { model: selectedChatModel, force: isAdmin });
if (button) button.disabled = false;
if (!result.ok) {
alert(result.data.error || "model load request failed");
@@ -1716,6 +1801,75 @@ async function requestSelectedModelLoad() {
$("chat-status").textContent = `load queued on ${short(assignment.node_id || "node")} for layers ${assignment.shard_start}-${assignment.shard_end}`;
}
async function requestAdminModelLoad(model) {
const result = await apiCall("/v1/models/load", "POST", { model, force: true });
if (!result.ok) return showAdminModelPlacementStatus(result.data.error || "model load request failed", true);
const assignment = result.data.assignment || {};
showAdminModelPlacementStatus(`Load queued on ${short(assignment.node_id || "node")} for ${model}.`);
await refreshActiveTab(true);
}
async function releaseAdminModel(model) {
const result = await apiCall("/v1/models/release", "POST", { model });
if (!result.ok) return showAdminModelPlacementStatus(result.data.error || "model release request failed", true);
showAdminModelPlacementStatus(`Release queued for ${result.data.released || 0} node(s) serving ${model}.`);
await refreshActiveTab(true);
}
function showAdminModelPlacementStatus(message, isError) {
const status = $("admin-model-placement-status");
status.textContent = message;
status.className = isError ? "bad" : "ok";
}
function gib(bytes) { return bytes == null ? "not reported" : `${(Number(bytes) / 1073741824).toFixed(1)} GiB`; }
function renderAdminNodePool(map) {
const groups = {};
for (const node of (map && map.nodes) || []) {
const account = node.wallet_address || "unbound account";
(groups[account] = groups[account] || []).push(node);
}
let html = "";
for (const [account, nodes] of Object.entries(groups).sort(([a], [b]) => a.localeCompare(b))) {
html += `<div style="margin-top:10px"><b>${esc(short(account, 20))}</b> <span class="dim">${nodes.length} node(s)</span></div>`;
html += table(["node", "assignment", "state / slots", "RAM", "GPU / VRAM", "model drive"], nodes.map(node => {
const hw = node.hardware_profile || {};
const cap = node.capacity || {};
const disk = hw.model_drive_free_bytes ?? hw.model_path_free_bytes ?? hw.disk_free_bytes;
const gpu = hw.gpu_name || (hw.cuda_available ? "CUDA GPU" : "CPU only");
return [nodeDisplayCell(node), esc(node.hf_repo || node.model || "unassigned"),
esc(`${node.stats?.status || "?"} · ${cap.loaded_slots ?? "?"}/${cap.max_loaded_shards ?? node.max_loaded_shards ?? "?"} slots`),
esc(gib(node.ram_bytes || (hw.ram_mb && hw.ram_mb * 1048576))),
esc(`${gpu} · ${gib(node.vram_bytes || (hw.vram_mb && hw.vram_mb * 1048576))}`), esc(gib(disk))];
}));
}
$("admin-node-pool").innerHTML = html || '<div class="empty">no nodes registered</div>';
}
function renderAdminModelPlacement(models, map) {
const nodes = (map && map.nodes) || [];
const rows = ((models && models.data) || []).map(model => {
const aliases = new Set([model.id, model.hf_repo, ...(model.aliases || [])].filter(Boolean));
const serving = nodes.filter(node => aliases.has(node.model) || aliases.has(node.hf_repo)).length;
const downloaded = nodes.filter(node => aliases.has(node.model) || aliases.has(node.hf_repo) ||
(node.downloaded_models || []).some(item => aliases.has(item.model) || aliases.has(item.hf_repo))).length;
const actions = `<button class="small" data-admin-model-load="${esc(model.id)}">load</button> ` +
`<button class="small" data-admin-model-release="${esc(model.id)}"${serving ? "" : " disabled"}>release</button>`;
return [esc(model.name || model.id), String(serving), String(downloaded), actions];
});
$("admin-model-placement").innerHTML = rows.length
? table(["model", "serving nodes", "downloaded on nodes", "admin action"], rows)
: '<div class="empty">no model presets configured</div>';
}
$("admin-model-placement").addEventListener("click", event => {
const load = event.target.closest("[data-admin-model-load]");
const release = event.target.closest("[data-admin-model-release]");
if (load) void requestAdminModelLoad(load.dataset.adminModelLoad);
if (release) void releaseAdminModel(release.dataset.adminModelRelease);
});
function chatAuthToken() {
if (accountApiKeys.length) return accountApiKeys[0];
return null;
@@ -2293,21 +2447,21 @@ function markRefreshed() {
async function fetchOverviewTab() {
const fetches = [
fetchJson("/v1/raft/status"),
fetchJson("/v1/network/map"),
fetchJson("/v1/models"),
fetchJson("/v1/stats"),
fetchJson("/v1/routing"),
fetchJson("/v1/console"),
];
if (isLoggedIn) fetches.push(apiCall("/v1/account"));
const results = await Promise.all(fetches);
const [raft, map, stats, routing, consoleData, accountResp] = results;
const [map, models, stats, routing, consoleData, accountResp] = results;
lastStats = stats;
lastRouting = routing;
lastNetworkMap = map;
if (accountResp && accountResp.ok) applyAccountSummary(accountResp.data, false);
renderIfChanged("hive", raft, renderHive);
renderIfChanged("nodes", map, renderNodes);
const nodesPayload = { map, routing, priceMap: buildModelPriceMap(models) };
renderIfChanged("nodes", nodesPayload, renderNodes);
renderIfChanged("stats", stats, renderStats);
renderIfChanged("routing", routing, renderRouting);
const speedModels = Object.keys((routing && routing.models) || {});
@@ -2348,16 +2502,22 @@ async function fetchBillingTab() {
async function fetchAdminTab() {
const fetches = [
fetchJson("/v1/raft/status"),
fetchJson("/v1/console"),
fetchJson("/v1/billing/summary"),
fetchJson("/v1/registry/wallets"),
fetchJson("/v1/models"),
fetchJson("/v1/network/map"),
];
if (isAdmin) fetches.push(apiCall("/v1/admin/accounts"));
const results = await Promise.all(fetches);
const [consoleData, summary, wallets, adminResp] = results;
const [raft, consoleData, summary, wallets, models, map, adminResp] = results;
renderIfChanged("hive", raft, renderHive);
renderIfChanged("console", consoleData, renderConsole);
renderIfChanged("billing-summary", summary, data => renderBilling(data));
renderIfChanged("fraud", { wallets, summary }, data => renderFraud(data.wallets, data.summary));
renderIfChanged("admin-model-placement", { models, map }, data => renderAdminModelPlacement(data.models, data.map));
renderIfChanged("admin-node-pool", map, renderAdminNodePool);
if (adminResp && adminResp.ok) {
renderIfChanged("admin", adminResp.data.accounts || [], accounts => {
const rows = accounts.map(a => {

View File

@@ -86,7 +86,7 @@ from .model_files import files_for_layer_range, snapshot_dir_for_repo
from .raft import RaftNode
_CONSOLE_LIMIT = 300
_CONSOLE_LIMIT = 1000
_PROXY_PROGRESS_LOG_INTERVAL = 5.0
_SESSION_COOKIE_NAME = "meshnet_session"
@@ -1528,6 +1528,49 @@ def _request_model_load_locked(server: "_TrackerHTTPServer", model_key: str) ->
return None
def _force_model_load_locked(server: "_TrackerHTTPServer", model_key: str) -> dict | None:
"""Replace the fastest ready assignment after an explicit admin eviction."""
resolved_name, preset = _resolve_model_preset(server.model_presets, model_key)
if preset is None or not preset.get("hf_repo"):
return None
start, end = _preset_layer_bounds(preset)
# An explicit admin eviction is permitted to recover a stuck/loading node
# and to use the preset default precision. It must only avoid a node that
# already has another assignment in flight.
candidates = [node for node in server.registry.values()
if node.pending_new_assignment is None]
if not candidates:
return None
node = max(candidates, key=lambda item: item.benchmark_tokens_per_sec)
shard_end = min(end, start + max(1, min(_node_layer_capacity(node, preset), end - start + 1)) - 1)
quantization = _node_quantization(node, preset)
directive = _load_directive(node, str(preset["hf_repo"]), start, shard_end, quantization)
replaced = node.hf_repo or node.model
node.model, node.hf_repo = resolved_name, str(preset["hf_repo"])
node.shard_start, node.shard_end, node.quantization = start, shard_end, quantization
node.managed_assignment, node.pending_new_assignment = True, directive
node.pending_directives.append(directive)
_tracker_log(server, "warn", "model load forced", node_id=node.node_id,
model=resolved_name, replaced_model=replaced, shard=f"{start}-{shard_end}")
return {"node_id": node.node_id, "model": resolved_name, "hf_repo": preset["hf_repo"],
"shard_start": start, "shard_end": shard_end, "replaced_model": replaced}
def _release_model_locked(server: "_TrackerHTTPServer", model_key: str) -> int:
"""Queue DROP_SHARD for every served shard and remove it from routing immediately."""
resolved_name, preset = _resolve_model_preset(server.model_presets, model_key)
if preset is None:
return 0
released = 0
for node in server.registry.values():
if not _node_matches_preset(node, resolved_name, preset) or node.shard_start is None or node.shard_end is None:
continue
node.pending_directives.append(_drop_directive(node, str(preset.get("hf_repo") or resolved_name), node.shard_start, node.shard_end, node.quantization or "bfloat16"))
node.status = "loading"
released += 1
return released
def _preferred_node_quantization(
node: _NodeEntry,
preset: dict,
@@ -3043,6 +3086,9 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
if self.path == "/v1/models/load":
self._handle_model_load_request()
return
if self.path == "/v1/models/release":
self._handle_model_release_request()
return
if self.path == "/v1/models/vote":
self._handle_model_coverage_vote()
return
@@ -3145,6 +3191,16 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
self.send_response(404)
self.end_headers()
def _model_pricing_payload(self, model: str) -> dict | None:
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
if server.billing is None:
return None
in_rate, out_rate = server.billing.prices_for(model)
return {
"input_per_1k_usdt": in_rate,
"output_per_1k_usdt": out_rate,
}
def _handle_models(self):
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
created = int(time.time())
@@ -3160,8 +3216,6 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
seen_ids: set[str] = set()
for name, preset in server.model_presets.items():
model_nodes = [node for node in alive if _node_matches_preset(node, name, preset)]
if not model_nodes and not preset.get("recommended"):
continue
required_start, required_end = _preset_layer_bounds(preset)
coverage = _coverage_percentage(
model_nodes,
@@ -3200,6 +3254,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
"shard_coverage_percentage": coverage,
"served_model_copies": served_copies,
"quantizations": quantizations,
"pricing": self._model_pricing_payload(name),
})
seen_ids.add(name)
# Note: the preset's hf_repo is deliberately NOT added to seen_ids —
@@ -3251,6 +3306,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
required_start,
required_end,
),
"pricing": self._model_pricing_payload(model_id),
})
seen_ids.add(model_id)
self._send_json(200, {"object": "list", "data": data})
@@ -3310,6 +3366,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
"endpoint": node.endpoint,
"relay_addr": node.relay_addr,
"peer_id": node.peer_id,
"wallet_address": node.wallet_address,
"hardware_profile": dict(node.hardware_profile),
"ram_bytes": node.ram_bytes,
"vram_bytes": node.vram_bytes,
"max_loaded_shards": node.max_loaded_shards,
}
for node in tracker_nodes
],
@@ -4822,11 +4883,32 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
with server.lock:
self._purge_expired_nodes()
assignment = _request_model_load_locked(server, model)
if assignment is None and body.get("force") is True:
assignment = _force_model_load_locked(server, model)
if assignment is None:
self._send_json(409, {"error": "no ready joined node has an available model slot and sufficient capacity"})
return
self._send_json(202, {"status": "queued", "assignment": assignment})
def _handle_model_release_request(self):
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
if not self._require_role("admin", "validator"):
return
body = self._read_json_body()
if body is None:
return
model = body.get("model")
if not isinstance(model, str) or not model.strip():
self._send_json(400, {"error": "model is required"})
return
with server.lock:
self._purge_expired_nodes()
released = _release_model_locked(server, model)
if not released:
self._send_json(404, {"error": "no served shards found for model"})
return
self._send_json(202, {"status": "release_queued", "released": released})
def _handle_model_coverage_vote(self):
"""Record a rolling wish-list signal for an unavailable precision."""
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
@@ -6477,7 +6559,7 @@ class TrackerServer:
embedded_relay_port: int = 8765,
embedded_relay_max_peers: int = 500,
billing: BillingLedger | None = None,
enable_billing: bool = False,
enable_billing: bool = True,
billing_db: str | None = None,
accounts: AccountStore | None = None,
accounts_db: str | None = None,

View File

@@ -396,7 +396,7 @@ def test_accounts_gossip_endpoint_applies_events(account_tracker):
def test_accounts_endpoints_404_when_disabled():
"Accounts endpoints 404 when disabled\n\nTags: accounts, auth, http"
tracker = TrackerServer() # no accounts, no billing
tracker = TrackerServer(enable_billing=False) # no accounts, no billing
port = tracker.start()
try:
with pytest.raises(urllib.error.HTTPError) as exc_info:

View File

@@ -172,13 +172,13 @@ def test_restart_persistence(tmp_path):
assert reloaded.snapshot()["protocol_cut"] == pytest.approx(0.02 * 0.10)
def test_tracker_enables_billing_with_default_db_when_requested(tmp_path, monkeypatch):
"Tracker enables billing with default db when requested\n\nTags: billing, http, payments"
def test_tracker_enables_billing_with_default_db(tmp_path, monkeypatch):
"Tracker enables billing with default db by default\n\nTags: billing, http, payments"
from meshnet_tracker.billing import DEFAULT_BILLING_DB_PATH
monkeypatch.chdir(tmp_path)
tracker = TrackerServer(enable_billing=True)
tracker = TrackerServer()
port = tracker.start()
try:
# /v1/billing/summary is admin-gated now; just confirm the server is up.
@@ -329,6 +329,7 @@ def billed_tracker():
}
},
billing=ledger,
enable_billing=False,
hive_secret=HIVE_SECRET,
)
port = tracker.start()
@@ -457,6 +458,7 @@ def test_proxy_chat_caps_inflated_streaming_usage_by_observed_chunks():
}
},
billing=ledger,
enable_billing=False,
)
port = tracker.start()
tracker_url = f"http://127.0.0.1:{port}"
@@ -506,6 +508,7 @@ def test_proxy_chat_splits_payout_by_tracker_assigned_route_span():
}
},
billing=ledger,
enable_billing=False,
)
port = tracker.start()
tracker_url = f"http://127.0.0.1:{port}"
@@ -582,6 +585,7 @@ def test_proxy_chat_rejects_request_above_spend_cap_before_routing():
}
},
billing=ledger,
enable_billing=False,
max_charge_per_request=0.01,
)
port = tracker.start()
@@ -656,6 +660,7 @@ def test_proxy_chat_records_validation_event_with_plain_route_metadata():
}
},
contracts=contracts,
enable_billing=False,
)
port = tracker.start()
tracker_url = f"http://127.0.0.1:{port}"

View File

@@ -39,6 +39,14 @@ def test_dashboard_served_with_all_panels():
assert "resolveModelGroup" in html
assert "buildModelAliasMap" in html
assert "modelAliasKey(raw)" in html
assert "@media (min-width:900px)" in html
assert "grid-template-columns:repeat(4,minmax(0,1fr));" in html
assert ".wide { grid-column:span 2; }" in html
assert 'onclick="clearConsole()"' in html
assert "let consoleClearedAt = 0;" in html
assert "max-height:520px; overflow-y:auto; overflow-x:auto;" in html
assert "const CONSOLE_MAX_LINES = 1000;" in html
assert "events.slice(-CONSOLE_MAX_LINES)" in html
finally:
tracker.stop()
@@ -97,6 +105,23 @@ def test_dashboard_allows_admin_to_request_selected_model_load():
assert '$("request-model-load").style.display = enabled ? "" : "none"' in html
def test_dashboard_exposes_admin_model_inventory_and_release_controls():
"Admin placement controls show the full model inventory and can release capacity."
html = _dashboard_html()
assert 'id="admin-model-placement"' in html
assert "renderAdminModelPlacement" in html
assert '"/v1/models/release"' in html
assert "requestAdminModelLoad" in html
assert "releaseAdminModel" in html
assert 'data-admin-model-load=' in html
assert 'data-admin-model-release=' in html
assert "admin-model-placement-status" in html
assert 'id="admin-node-pool"' in html
assert "renderAdminNodePool" in html
assert "model drive" in html
def test_network_map_includes_node_friendly_name():
"Network map includes node friendly name\n\nTags: dashboard, http"
tracker = TrackerServer()

View File

@@ -253,7 +253,7 @@ def test_proxy_head_is_route_head_and_routing_endpoint_lists_routes():
threads.append(thread)
gpu_stub, cpu_stub = stubs
tracker = TrackerServer(model_presets={
tracker = TrackerServer(enable_billing=False, model_presets={
"qwen3.6-35b-a3b": {
"layers_start": 0,
"layers_end": 39,
@@ -321,7 +321,7 @@ def test_proxy_head_is_route_head_and_routing_endpoint_lists_routes():
def test_admin_model_load_request_queues_directive_on_joined_node():
"Admin model load request queues directive on joined node\n\nTags: http, performance, routing, tracker"
tracker = TrackerServer(validator_service_token="test-admin")
tracker = TrackerServer(enable_billing=False, validator_service_token="test-admin")
port = tracker.start()
try:
node = _post_json(
@@ -355,6 +355,39 @@ def test_admin_model_load_request_queues_directive_on_joined_node():
assert heartbeat["directives"][0]["model"] == "Qwen/Qwen2.5-0.5B-Instruct"
def test_admin_can_replace_a_served_model_and_release_it():
"Forced admin placement replaces a served shard; release queues DROP_SHARD."
tracker = TrackerServer(enable_billing=False, validator_service_token="test-admin")
port = tracker.start()
try:
node = _post_json(
f"http://127.0.0.1:{port}/v1/nodes/register",
{"endpoint": "http://127.0.0.1:9912", "model": "stub-model",
"shard_start": 0, "shard_end": 3, "managed_assignment": True,
"max_loaded_shards": 1, "memory_mb": 1,
"hardware_profile": {"host_id": "full-host"}},
)
headers = {"Content-Type": "application/json", "Authorization": "Bearer test-admin"}
load = urllib.request.Request(
f"http://127.0.0.1:{port}/v1/models/load",
data=json.dumps({"model": "qwen2.5-0.5b-instruct", "force": True}).encode(),
headers=headers, method="POST")
with urllib.request.urlopen(load) as response:
assert json.loads(response.read())["assignment"]["node_id"] == node["node_id"]
heartbeat = _post_json(f"http://127.0.0.1:{port}/v1/nodes/{node['node_id']}/heartbeat", {})
assert heartbeat["directives"][0]["action"] == "LOAD_SHARD"
release = urllib.request.Request(
f"http://127.0.0.1:{port}/v1/models/release",
data=json.dumps({"model": "qwen2.5-0.5b-instruct"}).encode(), headers=headers, method="POST")
with urllib.request.urlopen(release) as response:
assert json.loads(response.read())["released"] == 1
heartbeat = _post_json(f"http://127.0.0.1:{port}/v1/nodes/{node['node_id']}/heartbeat", {})
finally:
tracker.stop()
assert heartbeat["directives"][0]["action"] == "DROP_SHARD"
def test_endpoint_key_distinguishes_same_port_different_hosts():
"Endpoint key distinguishes same port different hosts\n\nTags: http, performance, routing, tracker"
from meshnet_node.torch_server import _clamp_downstream_hops, _endpoint_key

View File

@@ -203,6 +203,7 @@ def test_probation_earns_nothing_then_earning_begins():
},
contracts=contracts,
billing=ledger,
enable_billing=False,
)
port = tracker.start()
tracker_url = f"http://127.0.0.1:{port}"

View File

@@ -60,7 +60,7 @@ def test_bad_node_is_slashed_and_excluded_from_gateway_routes(capsys):
contracts = LocalSolanaContracts()
contracts.registry.submit_stake("wallet-good", 500)
contracts.registry.submit_stake("wallet-bad", 500)
tracker = TrackerServer(contracts=contracts)
tracker = TrackerServer(contracts=contracts, enable_billing=False)
tracker_port = tracker.start()
tracker_url = f"http://127.0.0.1:{tracker_port}"

View File

@@ -76,7 +76,7 @@ def test_tracker_records_increasing_hop_latency_for_model_and_hardware(model, ha
layer_count, hf_repo = MODELS[model]
nodes, threads = _start_latency_nodes(5)
tracker = TrackerServer(model_presets={
tracker = TrackerServer(enable_billing=False, model_presets={
model: {
"layers_start": 0,
"layers_end": layer_count - 1,
@@ -142,7 +142,7 @@ def test_tracker_records_increasing_hop_latency_for_model_and_hardware(model, ha
def test_model_speed_dashboard_includes_visualization_and_route_drilldown():
"Model speed dashboard includes visualization and route drilldown\n\nTags: model, node, routing"
tracker = TrackerServer()
tracker = TrackerServer(enable_billing=False)
port = tracker.start()
try:
html = urllib.request.urlopen(f"http://127.0.0.1:{port}/dashboard").read().decode()

View File

@@ -32,7 +32,7 @@ def _register_node(tracker_url: str, endpoint: str, shard_start: int, shard_end:
@pytest.fixture
def two_node_setup():
"""Start tracker, two stub nodes, and gateway; yield (gateway_url, node_a, node_b)."""
tracker = TrackerServer()
tracker = TrackerServer(enable_billing=False)
tracker_port = tracker.start()
tracker_url = f"http://127.0.0.1:{tracker_port}"

View File

@@ -1075,6 +1075,190 @@ def test_partial_snapshot_loader_materializes_only_assigned_tensors(tmp_path):
assert model.model.rotary_emb.to_calls == ["cpu:0"]
def _build_partial_snapshot_fixture(snapshot_dir: Path, *, num_layers: int = 4) -> dict[str, dict[str, int]]:
"""Minimal HF snapshot with per-tensor numel metadata for memory-scaling tests."""
weight_map: dict[str, str] = {
"model.embed_tokens.weight": "shard-head.safetensors",
"model.norm.weight": "shard-tail.safetensors",
"lm_head.weight": "shard-tail.safetensors",
}
tensor_numel: dict[str, dict[str, int]] = {
"shard-head.safetensors": {"model.embed_tokens.weight": 10_000},
"shard-tail.safetensors": {
"model.norm.weight": 512,
"lm_head.weight": 10_000,
},
}
for layer in range(num_layers):
rel = f"shard-layer-{layer}.safetensors"
weight_map[f"model.layers.{layer}.self_attn.q_proj.weight"] = rel
weight_map[f"model.layers.{layer}.mlp.down_proj.weight"] = rel
per_layer = 1_000 * (layer + 1)
tensor_numel[rel] = {
f"model.layers.{layer}.self_attn.q_proj.weight": per_layer,
f"model.layers.{layer}.mlp.down_proj.weight": per_layer,
}
(snapshot_dir / rel).write_bytes(b"stub")
(snapshot_dir / "config.json").write_text(json.dumps({"num_hidden_layers": num_layers}))
(snapshot_dir / "model.safetensors.index.json").write_text(json.dumps({"weight_map": weight_map}))
(snapshot_dir / "shard-head.safetensors").write_bytes(b"stub")
(snapshot_dir / "shard-tail.safetensors").write_bytes(b"stub")
return tensor_numel
def _partial_load_materialized_numel(
snapshot_dir: Path,
shard_start: int,
shard_end: int,
tensor_numel: dict[str, dict[str, int]],
) -> int:
"""Return the summed numel of checkpoint tensors assigned to one shard load."""
class FakeModule:
def to(self, device):
return self
class FakeModel:
def __init__(self, num_layers: int):
self.model = types.SimpleNamespace(
embed_tokens=FakeModule(),
layers=[FakeModule() for _ in range(num_layers)],
rotary_emb=FakeModule(),
norm=FakeModule(),
)
self.lm_head = FakeModule()
def tie_weights(self):
pass
class AutoConfigStub:
@staticmethod
def from_pretrained(model_id):
assert model_id == str(snapshot_dir)
return types.SimpleNamespace(num_hidden_layers=num_layers)
class AutoModelStub:
@staticmethod
def from_config(cfg, torch_dtype=None):
return FakeModel(cfg.num_hidden_layers)
class UnusedContext:
def __enter__(self):
return None
def __exit__(self, exc_type, exc, tb):
return False
num_layers = json.loads((snapshot_dir / "config.json").read_text())["num_hidden_layers"]
loaded_numel = 0
def fake_set_tensor(module, tensor_name, device, value=None, dtype=None):
nonlocal loaded_numel
loaded_numel += int(getattr(value, "numel", lambda: 0)())
class FakeTensor:
def __init__(self, numel: int):
self._numel = numel
def numel(self) -> int:
return self._numel
class FakeSafeOpen:
def __init__(self, filename, framework, device):
self.filename = Path(filename).name
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def get_tensor(self, tensor_name):
return FakeTensor(tensor_numel[self.filename][tensor_name])
_load_partial_model_from_snapshot(
AutoConfigStub,
AutoModelStub,
types.SimpleNamespace(),
str(snapshot_dir),
shard_start,
shard_end,
"bf16",
"cpu:0",
init_empty_weights_fn=UnusedContext,
set_tensor_fn=fake_set_tensor,
safe_open_fn=FakeSafeOpen,
)
return loaded_numel
def test_partial_snapshot_resident_weight_numel_scales_with_shard(tmp_path):
"Partial load materializes only assigned-layer weights, not the full checkpoint.\n\nTags: model, node, real-inference"
snapshot_dir = tmp_path / "snapshot"
snapshot_dir.mkdir()
tensor_numel = _build_partial_snapshot_fixture(snapshot_dir, num_layers=4)
middle_numel = _partial_load_materialized_numel(snapshot_dir, 1, 1, tensor_numel)
full_numel = _partial_load_materialized_numel(snapshot_dir, 0, 3, tensor_numel)
# Layer 1 only: two tensors at 2000 numel each.
assert middle_numel == 4_000
# Head embed + four layers (two tensors each, increasing sizes) + tail norm/lm_head.
assert full_numel == 10_000 + 512 + 10_000 + 2_000 + 4_000 + 6_000 + 8_000
assert middle_numel < full_numel // 4
@pytest.mark.integration
def test_partial_snapshot_materialized_param_count_with_real_torch(tmp_path):
"When torch is installed, partial load leaves unassigned params on meta device.\n\nTags: model, node, real-inference"
torch = _require_functional_torch()
pytest.importorskip("transformers")
pytest.importorskip("safetensors")
pytest.importorskip("accelerate")
from safetensors.torch import save_file
from transformers import AutoConfig, AutoModelForCausalLM, GPT2Config
n_layer = 4
n_embd = 16
snapshot_dir = tmp_path / "snapshot"
snapshot_dir.mkdir()
config = GPT2Config(n_layer=n_layer, n_embd=n_embd, n_head=2, n_positions=32)
(snapshot_dir / "config.json").write_text(config.to_json_string())
weight_map: dict[str, str] = {}
for layer in range(n_layer):
key = f"transformer.h.{layer}.attn.c_attn.weight"
rel = f"layer-{layer}.safetensors"
weight_map[key] = rel
save_file({key: torch.ones(n_embd, 3 * n_embd)}, snapshot_dir / rel)
(snapshot_dir / "model.safetensors.index.json").write_text(json.dumps({"weight_map": weight_map}))
def count_materialized(model) -> tuple[int, int]:
materialized = 0
meta = 0
for param in model.parameters():
if param.device.type == "meta":
meta += param.numel()
else:
materialized += param.numel()
return materialized, meta
middle = _load_partial_model_from_snapshot(
AutoConfig,
AutoModelForCausalLM,
torch,
str(snapshot_dir),
1,
1,
torch.float32,
torch.device("cpu"),
)
mid_mat, mid_meta = count_materialized(middle)
assert mid_mat == n_embd * 3 * n_embd
assert mid_meta > mid_mat * 10
def test_partial_snapshot_loader_requires_known_layer_count(tmp_path):
"Partial snapshot loader requires known layer count\n\nTags: model, node, real-inference"
snapshot_dir = tmp_path / "snapshot"

View File

@@ -32,12 +32,18 @@ def test_matrix_reports_direct_relay_prefill_decode_and_machine_readable_metrics
assert {"p50_latency_ms", "p95_latency_ms", "payload_bytes", "compression_ratio",
"connection_attempts", "p95_queue_wait_ms"} <= set(run["phases"]["decode"])
sample = run["samples"][0]
assert sample["model_ms"] > 0
assert sample["encode_ms"] > 0
assert sample["activation_decode_ms"] > 0
assert sample["framing_ms"] > 0
assert sample["metadata_ms"] > 0
assert sample["copy_allocation_ms"] > 0
assert sample["copy_allocation_bytes"] >= sample["payload_bytes"]
assert sample["local_http_forwarding_ms"] > 0
assert len(run["samples"]) == 1 + len(run["output_tokens"])
assert {"tokens_per_sec", "bytes_per_token", "compression_cpu_ms", "peak_buffered_bytes"} <= set(run["phases"]["decode"])
assert {"tokens_per_sec", "bytes_per_token", "compression_cpu_ms", "peak_buffered_bytes",
"model_execution_ms", "activation_encoding_ms", "activation_decoding_ms",
"local_http_forwarding_ms"} <= set(run["phases"]["decode"])
def test_cached_sessions_reuse_one_connection_and_preserve_stub_tokens():
@@ -74,7 +80,10 @@ def test_cli_writes_json_artifact_and_human_summary(tmp_path, capsys):
report = json.loads(output.read_text())
assert report["schema_version"] == 1
assert "Route Session benchmark" in capsys.readouterr().out
assert "relay" in format_summary(report)
summary = format_summary(report)
assert "relay" in summary
assert "model/encode/decode" in summary
assert "HTTP" in summary
def test_performance_gate_checks_comparison_identity_session_and_cleanup():

View File

@@ -12,7 +12,12 @@ import urllib.request
import pytest
from meshnet_contracts import LocalSolanaContracts
from meshnet_tracker.billing import BillingLedger
from meshnet_tracker.billing import (
BillingLedger,
DEFAULT_PAYOUT_DUST_FLOOR,
DEFAULT_PAYOUT_THRESHOLD,
DEFAULT_SETTLE_PERIOD,
)
from meshnet_tracker.server import TrackerServer
MODEL = "stub-model"
@@ -172,3 +177,115 @@ def test_non_leader_never_signs():
assert ledger.get_node_pending("wallet-a") > 0
finally:
tracker.stop()
# Production defaults — sourced from billing.py (cli.py uses the same constants)
PROD_SETTLE_PERIOD = DEFAULT_SETTLE_PERIOD
PROD_PAYOUT_THRESHOLD = DEFAULT_PAYOUT_THRESHOLD
PROD_DUST_FLOOR = DEFAULT_PAYOUT_DUST_FLOOR
def _set_pending_since(ledger: BillingLedger, wallet: str, ts: float) -> None:
with ledger._lock:
ledger._pending_since[wallet] = ts
def test_prod_settlement_constants():
"Production settlement policy is 24 h period, 5 USDT threshold, 0.01 dust floor.\n\nTags: billing, settlement"
assert DEFAULT_SETTLE_PERIOD == 86400.0
assert DEFAULT_PAYOUT_THRESHOLD == 5.0
assert DEFAULT_PAYOUT_DUST_FLOOR == 0.01
def test_payables_24h_period_not_due_before_elapsed():
"Sub-threshold pending is not payable until the 24 h period elapses.\n\nTags: billing, settlement"
ledger = BillingLedger(starting_credit=100.0, default_price_per_1k=0.02)
ledger.charge_request("client", MODEL, 1000, [("node-wallet", 12)])
assert ledger.get_node_pending("node-wallet") == pytest.approx(0.018)
t0 = 1_700_000_000.0
_set_pending_since(ledger, "node-wallet", t0)
due_early = ledger.payables(
threshold=PROD_PAYOUT_THRESHOLD,
max_period=PROD_SETTLE_PERIOD,
dust_floor=PROD_DUST_FLOOR,
now=t0 + PROD_SETTLE_PERIOD - 1.0,
)
assert due_early == []
def test_payables_24h_period_triggers_at_boundary():
"Sub-threshold pending becomes payable exactly after the 24 h period.\n\nTags: billing, settlement"
ledger = BillingLedger(starting_credit=100.0, default_price_per_1k=0.02)
ledger.charge_request("client", MODEL, 1000, [("node-wallet", 12)])
pending = ledger.get_node_pending("node-wallet")
t0 = 1_700_000_000.0
_set_pending_since(ledger, "node-wallet", t0)
due = ledger.payables(
threshold=PROD_PAYOUT_THRESHOLD,
max_period=PROD_SETTLE_PERIOD,
dust_floor=PROD_DUST_FLOOR,
now=t0 + PROD_SETTLE_PERIOD,
)
assert due == [("node-wallet", pytest.approx(pending))]
def test_payables_threshold_triggers_before_24h_period():
"Pending ≥ 5 USDT pays immediately without waiting the 24 h period.\n\nTags: billing, settlement"
ledger = BillingLedger(starting_credit=1000.0, default_price_per_1k=0.02)
# cost = 300k * 0.02 / 1k = 6.0 USDT → node share 5.4 USDT
ledger.charge_request("client", MODEL, 300_000, [("node-wallet", 12)])
assert ledger.get_node_pending("node-wallet") >= PROD_PAYOUT_THRESHOLD
due = ledger.payables(
threshold=PROD_PAYOUT_THRESHOLD,
max_period=PROD_SETTLE_PERIOD,
dust_floor=PROD_DUST_FLOOR,
now=time.time(),
)
assert len(due) == 1
assert due[0][0] == "node-wallet"
def test_prod_defaults_hold_sub_threshold_pending():
"Tracker with prod 24 h / 5 USDT settings does not pay tiny balances early.\n\nTags: billing, settlement"
ledger = BillingLedger(starting_credit=100.0, default_price_per_1k=0.02)
ledger.charge_request("client", MODEL, 1000, [("node-wallet", 12)])
treasury = _FakePayoutTreasury()
tracker = _make_tracker(
ledger,
treasury,
threshold=PROD_PAYOUT_THRESHOLD,
period=PROD_SETTLE_PERIOD,
dust=PROD_DUST_FLOOR,
)
tracker.start()
try:
time.sleep(0.5)
assert treasury.batches == []
assert ledger.get_node_pending("node-wallet") > 0
finally:
tracker.stop()
def test_prod_threshold_triggers_immediate_payout():
"Tracker with prod 24 h period still pays when pending ≥ 5 USDT.\n\nTags: billing, settlement"
ledger = BillingLedger(starting_credit=1000.0, default_price_per_1k=0.02)
ledger.charge_request("client", MODEL, 300_000, [("node-wallet", 12)])
treasury = _FakePayoutTreasury()
tracker = _make_tracker(
ledger,
treasury,
threshold=PROD_PAYOUT_THRESHOLD,
period=PROD_SETTLE_PERIOD,
dust=PROD_DUST_FLOOR,
)
tracker.start()
try:
assert _wait_for(lambda: treasury.batches)
assert ledger.get_node_pending("node-wallet") == pytest.approx(0.0)
assert treasury.batches[0][0][0] == "node-wallet"
assert treasury.batches[0][0][1] >= PROD_PAYOUT_THRESHOLD
finally:
tracker.stop()

View File

@@ -28,6 +28,11 @@ from meshnet_tracker.server import (
_TEST_HIVE_SECRET = "test-hive-secret"
def _tracker(**kwargs):
kwargs.setdefault("enable_billing", False)
return TrackerServer(**kwargs)
def _post_json(url: str, payload: dict) -> dict:
data = json.dumps(payload).encode()
req = urllib.request.Request(
@@ -67,7 +72,7 @@ def test_tracker_send_json_ignores_broken_pipe_after_client_disconnect():
def test_tracker_exposes_registered_model_metadata():
"Tracker exposes registered model metadata\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
try:
@@ -114,7 +119,7 @@ def test_tracker_exposes_registered_model_metadata():
def test_tracker_lists_recommended_kimi_before_nodes_register():
"Tracker lists recommended kimi before nodes register\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
try:
@@ -138,7 +143,7 @@ def test_tracker_lists_recommended_qwen_27b_before_nodes_register():
Tags: http, routing, tracker
"""
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
try:
@@ -177,7 +182,7 @@ def test_quantization_coverage_uses_higher_precision_shards():
def test_network_map_exposes_pool_size_and_speed_summary():
"Network map exposes pool size and speed summary\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
try:
@@ -219,7 +224,7 @@ def test_network_map_exposes_pool_size_and_speed_summary():
def test_network_map_exposes_served_model_copy_count():
"Network map exposes served model copy count\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
try:
@@ -273,7 +278,7 @@ def test_network_map_exposes_served_model_copy_count():
def test_recommended_kimi_becomes_deployable_when_pool_is_large_enough():
"Recommended kimi becomes deployable when pool is large enough\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
gib = 1024 * 1024 * 1024
@@ -302,7 +307,7 @@ def test_recommended_kimi_becomes_deployable_when_pool_is_large_enough():
def test_network_assign_can_start_recommended_kimi_by_short_name():
"Network assign can start recommended kimi by short name\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
url = f"http://127.0.0.1:{port}"
try:
@@ -344,7 +349,7 @@ def test_tracker_serves_health_while_proxy_request_is_in_flight():
slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
slow_thread.start()
tracker = TrackerServer(heartbeat_timeout=60.0)
tracker = _tracker(heartbeat_timeout=60.0)
tracker_port = tracker.start()
proxy_error = []
try:
@@ -410,7 +415,7 @@ def test_tracker_route_log_counts_proxy_inflight_requests():
slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
slow_thread.start()
tracker = TrackerServer(heartbeat_timeout=60.0)
tracker = _tracker(heartbeat_timeout=60.0)
tracker_port = tracker.start()
errors = []
try:
@@ -498,7 +503,7 @@ def test_tracker_logs_stream_progress_before_request_completes():
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer(heartbeat_timeout=60.0)
tracker = _tracker(heartbeat_timeout=60.0)
tracker_port = tracker.start()
response = None
try:
@@ -590,7 +595,7 @@ def test_tracker_stream_survives_idle_gap_between_sse_chunks():
node = http.server.HTTPServer(("127.0.0.1", 0), IdleStreamingChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer(heartbeat_timeout=60.0)
tracker = _tracker(heartbeat_timeout=60.0)
tracker_port = tracker.start()
response = None
try:
@@ -657,7 +662,7 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer(heartbeat_timeout=60.0)
tracker = _tracker(heartbeat_timeout=60.0)
tracker_port = tracker.start()
response = None
request_id = None
@@ -723,7 +728,7 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
def test_tracker_routes_hf_model_alias_from_quickstart():
"The documented qwen2.5-0.5b alias resolves a full HF repo registration.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
_post_json(
@@ -775,7 +780,7 @@ def test_tracker_proxy_accepts_hf_model_alias_from_quickstart():
node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
_post_json(
@@ -833,7 +838,7 @@ def test_tracker_proxy_routes_split_preset_nodes_by_alias():
head = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
head_thread = threading.Thread(target=head.serve_forever, daemon=True)
head_thread.start()
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"qwen3.6-35b-a3b": {
"layers_start": 0,
"layers_end": 39,
@@ -889,7 +894,7 @@ def test_tracker_proxy_routes_split_preset_nodes_by_alias():
def test_tracker_route_endpoint_routes_split_preset_nodes_by_alias():
"Tracker route endpoint routes split preset nodes by alias\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"qwen3.6-35b-a3b": {
"layers_start": 0,
"layers_end": 39,
@@ -936,7 +941,7 @@ def test_tracker_route_endpoint_routes_split_preset_nodes_by_alias():
def test_tracker_route_endpoint_ignores_model_case_and_outer_whitespace():
"Tracker route endpoint ignores model case and outer whitespace\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"qwen3.6-35b-a3b": {
"layers_start": 0,
"layers_end": 39,
@@ -970,7 +975,7 @@ def test_tracker_route_endpoint_ignores_model_case_and_outer_whitespace():
def test_tracker_route_prefers_distributed_over_single_full_shard():
"When a full 0-39 node and a partial 0-21 head coexist, /v1/route\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"qwen3.6-35b-a3b": {
"layers_start": 0,
"layers_end": 39,
@@ -1044,7 +1049,7 @@ def test_tracker_proxy_ignores_model_case_and_outer_whitespace():
node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"qwen3.6-35b-a3b": {
"layers_start": 0,
"layers_end": 39,
@@ -1083,7 +1088,7 @@ def test_tracker_proxy_ignores_model_case_and_outer_whitespace():
def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix():
"Tracker registration node id includes wallet prefix and stable suffix\n\nTags: http, routing, security, tracker, wallet"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
wallet = "7j77FsPY1evV8tuf4Z73AVrWwxBEW1pvKwi4EvcRD3g"
try:
@@ -1124,7 +1129,7 @@ def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix()
def test_tracker_node_registration():
"A node can register with the tracker and receives a node_id.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
resp = _post_json(
@@ -1146,7 +1151,7 @@ def test_tracker_node_registration():
def test_tracker_route_selection():
"Tracker returns ordered route when nodes collectively cover all layers.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
_post_json(
@@ -1174,7 +1179,7 @@ def test_tracker_route_selection():
def test_tracker_route_error_no_coverage():
"Tracker returns 503 when registered nodes do not cover all required layers.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
# Only half the layers covered — route must fail.
@@ -1197,7 +1202,7 @@ def test_tracker_route_error_no_coverage():
def test_tracker_coverage_endpoint_reports_uncovered_ranges():
"Coverage endpoint returns compressed layer ranges with node counts.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 6,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1242,7 +1247,7 @@ def test_tracker_coverage_endpoint_reports_uncovered_ranges():
def test_tracker_coverage_endpoint_accepts_registered_hf_repo_or_short_name():
"Coverage endpoint supports real HF models registered outside preset catalog.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
_post_json(
@@ -1279,7 +1284,7 @@ def test_tracker_coverage_endpoint_accepts_registered_hf_repo_or_short_name():
def test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias():
"Tracker models endpoint lists registered hf repo and short name alias\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
_post_json(
@@ -1302,7 +1307,7 @@ def test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias()
def test_tracker_auto_assigns_new_node_to_uncovered_range_first():
"Capability-driven registration fills the first uncovered layer gap.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 8,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1331,7 +1336,7 @@ def test_tracker_auto_assigns_new_node_to_uncovered_range_first():
def test_tracker_speed_weighted_vram_assignment_covers_model():
"Three auto-assigned nodes reach 100% coverage with widest range on largest VRAM.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 12,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1375,7 +1380,7 @@ def test_tracker_speed_weighted_vram_assignment_covers_model():
def test_tracker_speed_is_primary_when_both_nodes_can_cover_gap():
"Tracker speed is primary when both nodes can cover gap\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 12,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1410,7 +1415,7 @@ def test_tracker_speed_is_primary_when_both_nodes_can_cover_gap():
def test_tracker_registration_directive_is_not_replayed_on_heartbeat():
"Tracker registration directive is not replayed on heartbeat\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 2,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1434,7 +1439,7 @@ def test_tracker_registration_directive_is_not_replayed_on_heartbeat():
def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
"A second managed node with capacity for the full model keeps the first copy intact.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 4,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1472,7 +1477,7 @@ def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
def test_tracker_explicit_full_copy_join_preserves_existing_serving_node():
"--model style joins with explicit shards add redundancy instead of reshuffling.\n\nTags: http, routing, tracker"
tracker = TrackerServer(heartbeat_timeout=10.0)
tracker = _tracker(heartbeat_timeout=10.0)
tracker_port = tracker.start()
try:
base_reg = {
@@ -1511,7 +1516,7 @@ def test_tracker_explicit_full_copy_join_preserves_existing_serving_node():
def test_tracker_periodic_rebalance_purges_expired_nodes_without_requests():
"Tracker periodic rebalance purges expired nodes without requests\n\nTags: http, routing, tracker"
tracker = TrackerServer(
tracker = _tracker(
heartbeat_timeout=0.05,
rebalance_interval=0.02,
model_presets={"tiny-model": {"total_layers": 1, "bytes_per_layer": {"bfloat16": 1_000}}},
@@ -1533,7 +1538,7 @@ def test_tracker_periodic_rebalance_purges_expired_nodes_without_requests():
def test_tracker_faster_node_receives_wider_range_when_capacity_ties():
"Tracker faster node receives wider range when capacity ties\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 12,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1564,7 +1569,7 @@ def test_tracker_faster_node_receives_wider_range_when_capacity_ties():
def test_tracker_rebalances_after_middle_range_node_timeout():
"Killing a middle shard queues LOAD_SHARD and restores coverage.\n\nTags: http, routing, tracker"
tracker = TrackerServer(
tracker = _tracker(
heartbeat_timeout=0.15,
model_presets={
"tiny-model": {
@@ -1612,7 +1617,7 @@ def test_tracker_rebalances_after_middle_range_node_timeout():
def test_tracker_rebalances_managed_hf_node_after_peer_timeout():
"HF nodes auto-assigned by the tracker receive LOAD_SHARD after a peer dies.\n\nTags: http, routing, tracker"
tracker = TrackerServer(heartbeat_timeout=0.15, rebalance_interval=10.0)
tracker = _tracker(heartbeat_timeout=0.15, rebalance_interval=10.0)
tracker_port = tracker.start()
try:
managed = _post_json(
@@ -1648,7 +1653,7 @@ def test_tracker_rebalances_managed_hf_node_after_peer_timeout():
def test_tracker_route_error_no_nodes():
"Tracker returns 503 with clear error when the registry is empty.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
try:
@@ -1664,7 +1669,7 @@ def test_tracker_route_error_no_nodes():
def test_tracker_heartbeat_updates_node():
"Sending a heartbeat for a registered node succeeds.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
reg = _post_json(
@@ -1684,7 +1689,7 @@ def test_tracker_heartbeat_updates_node():
def test_tracker_heartbeat_stores_current_requests():
"Node-reported in-flight request snapshots appear on the network map.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
reg = _post_json(
@@ -1749,7 +1754,7 @@ def test_normalize_current_requests_sanitizes_payload():
def test_tracker_heartbeat_expiry():
"Nodes that miss their heartbeat window are excluded from routes.\n\nTags: http, routing, tracker"
tracker = TrackerServer(heartbeat_timeout=0.05) # 50 ms
tracker = _tracker(heartbeat_timeout=0.05) # 50 ms
tracker_port = tracker.start()
try:
_post_json(
@@ -1778,7 +1783,7 @@ def test_tracker_heartbeat_expiry():
def test_tracker_heartbeat_expiry_removes_node_from_registry():
"Expired nodes are removed, not merely hidden from route responses.\n\nTags: http, routing, tracker"
tracker = TrackerServer(heartbeat_timeout=0.05)
tracker = _tracker(heartbeat_timeout=0.05)
tracker_port = tracker.start()
try:
reg = _post_json(
@@ -1806,7 +1811,7 @@ def test_tracker_heartbeat_expiry_removes_node_from_registry():
def test_tracker_route_rejects_non_extending_overlap():
"Overlapping shards that do not extend coverage cannot form a complete route.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
_post_json(
@@ -1831,7 +1836,7 @@ def test_tracker_route_rejects_non_extending_overlap():
def test_tracker_console_records_model_not_available():
"Tracker console records model not available\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
try:
@@ -1856,7 +1861,7 @@ def test_tracker_console_records_model_not_available():
def test_tracker_registration_rejects_invalid_payload():
"Registration errors return a defined JSON 400 response.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
try:
@@ -1876,7 +1881,7 @@ def test_tracker_registration_rejects_invalid_payload():
def test_tracker_registration_rejects_invalid_max_loaded_shards():
"Tracker registration rejects invalid max loaded shards\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
try:
@@ -1896,7 +1901,7 @@ def test_tracker_registration_rejects_invalid_max_loaded_shards():
def test_network_map_exposes_node_capacity_limits():
"Network map exposes node capacity limits\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 12,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1927,7 +1932,7 @@ def test_network_map_exposes_node_capacity_limits():
def test_tracker_capacity_uses_ram_when_node_has_no_vram():
"CPU-only nodes should expose RAM-backed shard capacity, not default GPU capacity.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 20,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1956,7 +1961,7 @@ def test_tracker_capacity_uses_ram_when_node_has_no_vram():
def test_rebalance_keeps_one_active_range_even_when_multiple_slots_advertised():
"max_loaded_shards is exposed but reserved until node runtime supports multi-range serving.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 8,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -1983,7 +1988,7 @@ def test_rebalance_keeps_one_active_range_even_when_multiple_slots_advertised():
def test_tracker_routes_only_nodes_for_requested_model():
"A node registered for one model cannot satisfy another model's route.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"model-a": {"layers_start": 0, "layers_end": 31},
"model-b": {"layers_start": 0, "layers_end": 31},
})
@@ -2006,7 +2011,7 @@ def test_tracker_routes_only_nodes_for_requested_model():
def test_tracker_assignment_coverage_is_model_scoped():
"Shard assignment gaps are computed only from nodes serving the same model.\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"model-a": {"layers_start": 0, "layers_end": 31},
"model-b": {"layers_start": 0, "layers_end": 31},
})
@@ -2031,7 +2036,7 @@ def test_tracker_excludes_banned_wallets_from_routes():
contracts.registry.submit_stake("wallet-a", 1_000)
contracts.registry.submit_stake("wallet-b", 1_000)
contracts.registry.ban_wallet("wallet-b")
tracker = TrackerServer(contracts=contracts, minimum_stake=100)
tracker = _tracker(contracts=contracts, minimum_stake=100)
tracker_port = tracker.start()
try:
_post_json(
@@ -2065,7 +2070,7 @@ def test_tracker_rejects_banned_wallet_registration():
contracts = LocalSolanaContracts()
contracts.registry.submit_stake("wallet-b", 1_000)
contracts.registry.ban_wallet("wallet-b")
tracker = TrackerServer(contracts=contracts, minimum_stake=100)
tracker = _tracker(contracts=contracts, minimum_stake=100)
tracker_port = tracker.start()
try:
try:
@@ -2111,7 +2116,7 @@ def test_gateway_routes_even_when_compute_node_has_low_stake():
contracts = LocalSolanaContracts()
contracts.registry.submit_stake("wallet-a", 1_000)
contracts.registry.submit_stake("wallet-b", 5)
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
tracker_url = f"http://127.0.0.1:{tracker_port}"
@@ -2166,7 +2171,7 @@ def test_gateway_records_compute_attribution_after_inference_session():
contracts.registry.submit_stake("wallet-a", 1_000)
contracts.registry.submit_stake("wallet-b", 1_000)
contracts.payment.fund_api_key("api-key-a", lamports=10_000)
tracker = TrackerServer(contracts=contracts, minimum_stake=100)
tracker = _tracker(contracts=contracts, minimum_stake=100)
tracker_port = tracker.start()
tracker_url = f"http://127.0.0.1:{tracker_port}"
@@ -2294,7 +2299,7 @@ def test_gateway_returns_json_503_for_malformed_tracker_route():
def test_two_node_pipeline_via_tracker():
"End-to-end: nodes register with tracker; gateway discovers route dynamically.\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
tracker_url = f"http://127.0.0.1:{tracker_port}"
@@ -2351,7 +2356,7 @@ def test_two_node_pipeline_via_tracker():
def test_stats_endpoint_returns_empty_models_initially():
"Stats endpoint returns empty models initially\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
port = tracker.start()
try:
resp = _get_json(f"http://127.0.0.1:{port}/v1/stats")
@@ -2449,7 +2454,7 @@ def test_stats_sqlite_persistence_survives_restart(tmp_path):
def test_stats_gossip_endpoint_merges_peer_slice():
"Stats gossip endpoint merges peer slice\n\nTags: gossip, http, network, routing, tracker"
tracker = TrackerServer(hive_secret=_TEST_HIVE_SECRET)
tracker = _tracker(hive_secret=_TEST_HIVE_SECRET)
port = tracker.start()
try:
peer_payload = {
@@ -2610,7 +2615,7 @@ def test_select_route_reputation_never_overrides_coverage():
def test_two_stub_nodes_complete_pipeline_via_tracker():
"Integration: two StubNodeServer instances serving complementary shards\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"two-node-model": {"layers_start": 0, "layers_end": 3}
})
tracker_port = tracker.start()
@@ -2732,7 +2737,7 @@ def test_shard_heal_cycle_surviving_node_covers_dead_peers_gap(monkeypatch):
monkeypatch.setattr(torch_server, "_load_backend", fake_load)
# Use a very short timeout so Node A expires quickly.
tracker = TrackerServer(heartbeat_timeout=0.15, rebalance_interval=10.0)
tracker = _tracker(heartbeat_timeout=0.15, rebalance_interval=10.0)
tracker_port = tracker.start()
node_b = TorchNodeServer(backend=_FakeBackend(shard_start=12, shard_end=23))
@@ -2807,7 +2812,7 @@ def test_shard_heal_cycle_surviving_node_covers_dead_peers_gap(monkeypatch):
def test_network_map_exposes_memory_pool():
"Network map exposes memory pool\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"tiny-model": {
"total_layers": 8,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -2836,7 +2841,7 @@ def test_network_map_exposes_memory_pool():
def test_same_endpoint_can_register_multiple_models():
"Same endpoint can register multiple models\n\nTags: http, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker_port = tracker.start()
try:
base = {
@@ -2866,7 +2871,7 @@ def test_same_endpoint_can_register_multiple_models():
def test_scale_demanded_models_queues_add_shard_on_spare_host():
"Scale demanded models queues add shard on spare host\n\nTags: http, routing, tracker"
tracker = TrackerServer(model_presets={
tracker = _tracker(model_presets={
"model-a": {
"total_layers": 4,
"bytes_per_layer": {"bfloat16": 1_000},
@@ -2957,7 +2962,7 @@ def _proxy_chat_status(policy: str, quantization: str | None) -> tuple[int, dict
node = http.server.HTTPServer(("127.0.0.1", 0), _EchoChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer(capability_policy=policy)
tracker = _tracker(capability_policy=policy)
tracker_port = tracker.start()
try:
endpoint = f"http://127.0.0.1:{node.server_address[1]}"
@@ -3005,7 +3010,7 @@ def test_a_node_declaring_a_null_quantization_is_never_routed():
node = http.server.HTTPServer(("127.0.0.1", 0), _EchoChatHandler)
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
node_thread.start()
tracker = TrackerServer(capability_policy=POLICY_COMPAT)
tracker = _tracker(capability_policy=POLICY_COMPAT)
tracker_port = tracker.start()
try:
endpoint = f"http://127.0.0.1:{node.server_address[1]}"
@@ -3028,7 +3033,7 @@ def test_a_node_declaring_a_null_quantization_is_never_routed():
def test_raft_apply_preserves_a_declared_precision_for_follower_routing():
"A follower that drops the declared precision would route unlike the leader.\n\nTags: raft, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker.start()
try:
tracker._raft_apply("register", {
@@ -3051,7 +3056,7 @@ def test_raft_apply_preserves_a_declared_precision_for_follower_routing():
def test_raft_apply_grandfathers_a_legacy_registration_that_omits_quantization():
"The follower must reach the same legacy fallback the leader did, not a bare None.\n\nTags: raft, routing, tracker"
tracker = TrackerServer()
tracker = _tracker()
tracker.start()
try:
tracker._raft_apply("register", {
@@ -3071,7 +3076,7 @@ def test_raft_apply_grandfathers_a_legacy_registration_that_omits_quantization()
def test_rebalancing_does_not_resurrect_a_node_that_declared_an_unusable_precision():
"Managed assignment picks a node's precision, so it must not overwrite an explicit unusable one.\n\nTags: http, routing, tracker"
tracker = TrackerServer(heartbeat_timeout=0.15, rebalance_interval=10.0)
tracker = _tracker(heartbeat_timeout=0.15, rebalance_interval=10.0)
tracker_port = tracker.start()
try:
managed = _post_json(
@@ -3120,7 +3125,7 @@ def _register_shard(tracker_port: int, port: int, **overrides) -> dict:
def test_a_node_with_an_unusable_precision_covers_no_layers():
"A node no request can reach must not report its shards as served coverage.\n\nTags: http, routing, tracker"
tracker = TrackerServer(rebalance_interval=10.0)
tracker = _tracker(rebalance_interval=10.0)
tracker_port = tracker.start()
try:
_register_shard(
@@ -3136,7 +3141,7 @@ def test_a_node_with_an_unusable_precision_covers_no_layers():
def test_rebalancing_fills_the_layers_an_unusable_node_only_appears_to_cover():
"Counting a dark node as coverage hides the gap, so the rebalancer never fills it.\n\nTags: http, routing, tracker"
tracker = TrackerServer(rebalance_interval=10.0)
tracker = _tracker(rebalance_interval=10.0)
tracker_port = tracker.start()
try:
# The dark node claims the model's tail; the managed node holds the head.