Compare commits
19 Commits
95245be512
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ba7c656364 | ||
|
|
b661590ac7 | ||
|
|
21e6c86147 | ||
|
|
def47f1a42 | ||
|
|
8cb00e951f | ||
|
|
7b3399760e | ||
|
|
22467f145c | ||
|
|
35af1e21de | ||
|
|
905ea16ce0 | ||
|
|
348b003d6e | ||
|
|
1e64a5b2b9 | ||
|
|
e2f3ae32b8 | ||
|
|
29351d6217 | ||
|
|
5c9a2f6c97 | ||
|
|
13d82f8032 | ||
|
|
d1a1400db9 | ||
|
|
5d87e81bc9 | ||
|
|
a6bcc69288 | ||
|
|
c938d38031 |
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai
|
||||||
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
- [User profile](user-profile.md) — who Dobromir is and how to work with him
|
||||||
- [Project status](project-status.md) — 35/35 stories done; alpha hardening next
|
- [Project status](project-status.md) — US-001…US-035 done; US-036…US-050 in docs/prd.json; alpha hardening + scratch features next
|
||||||
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
|
- **Alpha hardening** — `.scratch/alpha-hardening/` (22 issues, ADRs 0016–0019, [README](../../.scratch/alpha-hardening/README.md), [handoff](../../.scratch/alpha-hardening/handoff.md))
|
||||||
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
|
- [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers
|
||||||
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023)
|
- **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan; [ADR-0023](../../docs/adr/0023-model-agnostic-node-capability-admission.md), [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md))
|
||||||
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
- **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning.
|
||||||
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md).
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Active workstream (started 2026-07-04): alpha hardening of the money/trust path.
|
|||||||
|
|
||||||
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
|
Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended.
|
||||||
|
|
||||||
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (35/35 done) and does NOT see `.scratch/alpha-hardening/issues/`. No ralph loop is running and no `.ralph-tui/` state exists. `.scratch/alpha-hardening/prd.json` now has 23 stories (AH-001…AH-023); point Ralph at that file for the alpha-hardening branch. Do NOT use `ralph auto --parallel` on server.py-touching issues — 21 and 23 both touch `server.py`/`billing.py`/`audit.py`; if run in the same Ralph pass, run them serially, not in parallel (merge-conflict risk, same lesson as 03/04 previously).
|
**Ralph note:** `scripts/ralph_progress.py` tracks `docs/prd.json` (US-001…US-047; base 35/35 done, friends-test arc 36–47 open/in-progress). Alpha hardening uses `.scratch/alpha-hardening/prd.json` (AH-001…AH-023). Point Ralph at the prd.json for the branch you're running.
|
||||||
|
|
||||||
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
|
**Why:** three audits agreed the alpha blockers are unauthenticated gossip (anyone can inject billing events), the free-credit faucet, and ephemeral bans.
|
||||||
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].
|
**How to apply:** work test-first per issue acceptance criteria; use `.venv`; `cryptography` belongs in node deps (wallet.py imports it — causes many of the 24 "failures" in a fresh env). See [[project-status]] and [[autonomous-work-style]].
|
||||||
|
|||||||
@@ -8,7 +8,16 @@ metadata:
|
|||||||
|
|
||||||
# Project Status (2026-07-13)
|
# Project Status (2026-07-13)
|
||||||
|
|
||||||
> Doc reconciliation 2026-07-13: `docs/PRD.md`, ADR numbering (0024 = distributed GGUF), and `docs/issues/` statuses updated to match `docs/prd.json` (US-001…US-035 done). Post-035 work remains in issues 36–48 and `.scratch/`.
|
## Selected-node model placement (2026-07-14)
|
||||||
|
|
||||||
|
- Admin Model placement now opens a node selector for load and release; the control-plane accepts optional `node_id` and targets only that registry assignment. Multi-model serving remains supported through `ADD_SHARD` and `max_loaded_shards`.
|
||||||
|
- Total node pool resource values are rendered from `/v1/network/map`'s `node.capacity` contract. Route selection remains assignment/capability/throughput/queue based; capacity is used for placement and falls back to tracker defaults only if a node truly omits it.
|
||||||
|
|
||||||
|
## Distributed inference performance (2026-07-14)
|
||||||
|
|
||||||
|
`DIP-001` is done in `.scratch/distributed-inference-performance/`: the deterministic two-node Route Session stub benchmark covers direct/relay plus cached/stateless prefill and decode. Its JSON and concise summary explicitly attribute model execution, activation encode/decode, compression, connection setup, relay queueing, local HTTP forwarding, and end-to-end seam latency. `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` passed (7); the fixture assertion checks output-token identity and connection attempts.
|
||||||
|
|
||||||
|
> Doc reconciliation 2026-07-13: `docs/prd.json` tracks US-001…US-050 (048 memory budget, 049 mainnet pilot, 050 Qwen demand placement). ADRs 0025–0026 added (TAI phase B/C, assignment ownership).
|
||||||
|
|
||||||
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:
|
All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02:
|
||||||
|
|
||||||
|
|||||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -20,6 +20,7 @@ dist/
|
|||||||
!.env.testnet
|
!.env.testnet
|
||||||
.rocm-local/*
|
.rocm-local/*
|
||||||
.pytest-tmp/*
|
.pytest-tmp/*
|
||||||
|
.cache/
|
||||||
|
|
||||||
# Local tracker/node sqlite databases (never commit runtime state)
|
# Local tracker/node sqlite databases (never commit runtime state)
|
||||||
*.sqlite
|
*.sqlite
|
||||||
|
|||||||
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
|
Pre-release alpha audit + grilling (2026-07-04). Bucket 1 trust-boundary blockers + fraud arc: **done** (16/22 original issues). Bucket 2 (12-15, multi-tracker) and 17 (doc dedup) remain deferred/human-gated — not launch blockers.
|
||||||
|
|
||||||
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet SOL/USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers, stake-free). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
|
**Launch-readiness grilling (2026-07-06):** locked plan is devnet dev/test run now, then real mainnet USDT for the first cohort — friends (API clients) + hired VPS/VPC hosts (own test infra, not third-party volunteers; no upfront stake, probation only). No new public token; TAI stays dormant per ADR-0002's existing volume/legal gates. Two new issues came out of this session:
|
||||||
|
|
||||||
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch.
|
- **[21 — Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md)** — rescoped from "prod gate" to a hard **alpha-release blocker**. `Status: ready-for-human` — engineering (audit.py raw divergence, tracker dispatch endpoint, SQLite corpus, p99 envelope) done 2026-07-06; blocked on a human running the calibration job against the real hired-VPS fleet before launch. Runbook: [04-toploc-calibration-run](./runbooks/04-toploc-calibration-run.md).
|
||||||
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
|
- **[23 — Dynamic HF-benchmarked pricing](./issues/23-dynamic-hf-pricing_completed.md)** — new, high priority but not a release blocker. `Status: done` — engineering complete 2026-07-06 (hf_pricing.py, opt-in daily refresh loop, GET /v1/pricing/hf/history); real `hf_aliases` curation per model is a follow-up human sign-off, not a completion blocker.
|
||||||
|
|
||||||
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
|
Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputation carries forward → fraud must be bounded. See [ADR-0016](../../docs/adr/0016-alpha-scope-and-known-limitations.md).
|
||||||
@@ -77,6 +77,7 @@ Locked scope: one settlement tracker, open node join, devnet mock-USDT, reputati
|
|||||||
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
|
| [17 Duplicate US-020 dedup](./issues/17-doc-duplicate-us020-dedup.md) |
|
||||||
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
|
| [18 Operational runbooks](./issues/18-doc-operational-runbooks_completed.md) |
|
||||||
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
|
| [19 Cryptography + test env](./issues/19-doc-cryptography-test-env_completed.md) |
|
||||||
|
| [04 TOPLOC calibration run](./runbooks/04-toploc-calibration-run.md) (issue 21 ops) |
|
||||||
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
|
| [22 MEMORY + project-status index](./issues/22-doc-memory-project-status_completed.md) (done) |
|
||||||
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
|
| [21 Honest-noise calibration corpus](./issues/21-honest-noise-calibration-corpus.md) (ops; prod gate for audits) |
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
Status: ready-for-human
|
Status: ready-for-human
|
||||||
|
|
||||||
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money (mainnet SOL/USDT) traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
|
**BLOCKS ALPHA RELEASE.** Scoped 2026-07-06 during alpha-launch-readiness grilling session — must complete before real-money mainnet USDT traffic goes live for the friends + hired-VPS-host launch. Loose/uncalibrated thresholds + manual admin slash-reversal are the stopgap only until this closes.
|
||||||
|
|
||||||
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
|
**Engineering complete 2026-07-06; blocked on a human running it against the real hired-VPS fleet before launch.** The three code gaps below are closed and unit-tested (see Deliverables), but nothing in a dev session can stand in for actually dispatching the job at real hardware — that step, plus the threshold/FPR write-up that depends on its output, needs an operator with the live fleet. See the validator README's "Honest-noise calibration corpus" section for the operational how-to.
|
||||||
|
|
||||||
@@ -14,7 +14,7 @@ Per [ADR-0018 consequences](../../docs/adr/0018-fraud-detection-verification-and
|
|||||||
|
|
||||||
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 layer 3 — "collect this first — run identical jobs across the current node fleet to measure the honest divergence envelope before setting thresholds."
|
||||||
|
|
||||||
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay stake-free until it's done, then move to real staking once thresholds derive from their own hardware.
|
**Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay on probation (no upfront stake) until it's done, then move to paid USDT serving once thresholds derive from their own hardware.
|
||||||
|
|
||||||
**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet.
|
**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet.
|
||||||
|
|
||||||
@@ -36,7 +36,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8
|
|||||||
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
|
- [ ] Threshold constants in validator config derived from corpus, not guessed — mechanically ready (`envelope()` returns them) but depends on the real corpus above; not yet wired into `ToplocAuditConfig` as enforced thresholds (deliberately — enforcing unvalidated thresholds would be worse than today's guessed bool).
|
||||||
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
|
- [ ] False-positive rate estimate documented at chosen thresholds — `envelope()` returns `estimated_false_positive_rate` (in-sample: fraction of the recorded corpus the recommended thresholds would themselves flag); needs the real corpus to be a meaningful number, and should be written up in the runbook once collected.
|
||||||
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
|
- [x] README / runbook cross-link: **do not enable production audits** until this issue closes — `packages/validator/README.md` "TOPLOC audit contract" section, updated with the full operational how-to.
|
||||||
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section.
|
- [x] Note in the runbook that this alpha corpus must be re-run once the fleet grows beyond the hired-VPS set (different hardware mix invalidates the envelope) — same README section; [runbook 04](../runbooks/04-toploc-calibration-run.md).
|
||||||
|
|
||||||
## ADR links
|
## ADR links
|
||||||
|
|
||||||
|
|||||||
@@ -440,12 +440,12 @@
|
|||||||
"Run relevant pytest tests; run the full suite when practical or document why not"
|
"Run relevant pytest tests; run the full suite when practical or document why not"
|
||||||
],
|
],
|
||||||
"priority": 21,
|
"priority": 21,
|
||||||
"passes": true,
|
"passes": false,
|
||||||
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money friends+hired-VPS launch) — rescoped 2026-07-06, no longer a Ralph-skip.",
|
"notes": "Source issue: .scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md. BLOCKS ALPHA RELEASE (real-money mainnet USDT). Operator runbook: .scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"AH-006"
|
"AH-006"
|
||||||
],
|
],
|
||||||
"completionNotes": "Engineering complete and unit-tested (validator audit.py detailed-verify aggregation, tracker calibration.py corpus store, calibration dispatch endpoints). Marked ready-for-human, not done: real corpus collection against the live hired-VPS fleet, and the threshold/FPR write-up that depends on its output, need a human operator — see .ralph-tui/progress.md and packages/validator/README.md."
|
"completionNotes": "Engineering complete and unit-tested. Remaining: human runs POST /v1/calibration/toploc/run on live hired-VPS fleet, records envelope/FPR, wires thresholds — see runbook 04 and packages/validator/README.md."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "AH-022",
|
"id": "AH-022",
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
# Runbook 04 — Honest-noise TOPLOC calibration (issue 21)
|
||||||
|
|
||||||
|
**Status:** engineering complete; **operator action required** before production audit thresholds.
|
||||||
|
|
||||||
|
**Blocks:** enabling calibrated TOPLOC thresholds on a mainnet / friends-test fleet (issue 21, ADR-0018).
|
||||||
|
|
||||||
|
## When to run
|
||||||
|
|
||||||
|
- Before first real-money traffic with audit enforcement enabled.
|
||||||
|
- Again whenever the fleet’s **hardware mix** changes materially (new GPU generation, CPU-only nodes added, precision/recipe change per model).
|
||||||
|
|
||||||
|
Alpha exception: with a **small hired-VPS-only** fleet, `gate_status.ready` may mean “covers every node we operate today” (`--toploc-calibration-gate-min-hardware-profiles 1`).
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Tracker running with billing + registry + `--toploc-calibration-db PATH` (or default under tracker cwd).
|
||||||
|
- At least one **solo-capable** node per hardware profile you want in the corpus (full model coverage — partial shards are skipped).
|
||||||
|
- Admin or validator credentials (`Authorization` header or validator service token per ADR-0017).
|
||||||
|
- Reference validator can replay the fixed calibration prompt (same model/seed as dispatch uses).
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
1. **Register the fleet** — all nodes you intend to pay on mainnet should be up, admitted (NCA when enabled), and solo-serving the calibration model.
|
||||||
|
|
||||||
|
2. **Dispatch the job** (admin/validator only):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "https://<tracker>/v1/calibration/toploc/run" \
|
||||||
|
-H "Authorization: Bearer <admin-or-validator-token>" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Partial-shard nodes appear under `skipped_partial_shard_node_ids`. Per-node failures appear under `skipped` with reasons.
|
||||||
|
|
||||||
|
3. **Wait for completion** — watch tracker logs and node consoles until every solo-capable node has a row in the corpus.
|
||||||
|
|
||||||
|
4. **Fetch results**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl "https://<tracker>/v1/calibration/toploc/results" \
|
||||||
|
-H "Authorization: Bearer <admin-or-validator-token>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Record:
|
||||||
|
- `envelope` — p99 metrics + 20% safety margin (recommended tolerances).
|
||||||
|
- `gate_status.ready` and `gate_status.hardware_profiles`.
|
||||||
|
- `estimated_false_positive_rate` (in-sample sanity check only).
|
||||||
|
|
||||||
|
5. **Write up thresholds** — paste envelope values into operator notes / issue 21 comment. Do **not** wire into production `ToplocAuditConfig` until you have reviewed FPR on this fleet.
|
||||||
|
|
||||||
|
6. **Mark issue 21 done** — when corpus covers the launch fleet and thresholds are documented.
|
||||||
|
|
||||||
|
## Two-wallet / minimal pilot variant
|
||||||
|
|
||||||
|
If your “fleet” is one node machine + one client:
|
||||||
|
|
||||||
|
- Run calibration against the **node** profile only (one hardware row is enough for `gate_status` with min profiles = 1).
|
||||||
|
- Client wallet is irrelevant to calibration — it never serves inference.
|
||||||
|
|
||||||
|
## Do not
|
||||||
|
|
||||||
|
- Enable stricter production audit thresholds before this completes.
|
||||||
|
- Reuse a corpus collected on devnet/mock hardware for a different mainnet GPU mix without re-running.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Issue: `.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`
|
||||||
|
- Code: `packages/tracker/meshnet_tracker/calibration.py`, `POST /v1/calibration/toploc/run`
|
||||||
|
- Validator: `packages/validator/README.md` — TOPLOC audit contract
|
||||||
@@ -12,4 +12,10 @@ Provide an opt-in, admin-only tracker Dashboard Testing tab that dynamically dis
|
|||||||
- One active run.
|
- One active run.
|
||||||
- Real inference stays separately environment-gated and excluded from default suites.
|
- Real inference stays separately environment-gated and excluded from default suites.
|
||||||
|
|
||||||
|
## Operator workflow
|
||||||
|
|
||||||
|
See [`docs/dev/dashboard-test-runner.md`](../../docs/dev/dashboard-test-runner.md)
|
||||||
|
for launch configuration, default safe suites vs the gated real-inference suite,
|
||||||
|
and required environment variables.
|
||||||
|
|
||||||
See `prd.json` for executable Ralph user stories and acceptance criteria.
|
See `prd.json` for executable Ralph user stories and acceptance criteria.
|
||||||
|
|||||||
@@ -51,15 +51,16 @@
|
|||||||
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
|
"uv run pytest tests/test_dashboard.py tests/test_dynamic_routing.py -q passes."
|
||||||
],
|
],
|
||||||
"priority": 3,
|
"priority": 3,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
|
"notes": "Do not reintroduce --enable-test-runner without implementing its CLI argument in US-001.",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"US-001",
|
"US-001",
|
||||||
"US-002"
|
"US-002"
|
||||||
]
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"updatedAt": "2026-07-11T17:02:30.520Z"
|
"updatedAt": "2026-07-12T01:58:06.286Z"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
Status: ready-for-agent
|
Status: done (2026-07-14)
|
||||||
|
|
||||||
# 01 — Baseline and profiling harness
|
# 01 — Baseline and profiling harness
|
||||||
|
|
||||||
@@ -12,16 +12,15 @@ sizes and connection counts without requiring a real model or external host.
|
|||||||
|
|
||||||
## Acceptance criteria
|
## Acceptance criteria
|
||||||
|
|
||||||
- [ ] The harness runs a fixed prompt and fixed generated-token count through a
|
- [x] The harness runs a fixed prompt and fixed generated-token count through a
|
||||||
two-node route in direct and relay modes.
|
two-node route in direct and relay modes.
|
||||||
- [ ] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
|
- [x] It reports p50/p95 per-token latency, per-hop latency, payload bytes,
|
||||||
compression ratio, connection attempts, and queue wait.
|
compression ratio, connection attempts, and queue wait.
|
||||||
- [ ] It distinguishes prefill from decode and cached from stateless mode.
|
- [x] It distinguishes prefill from decode and cached from stateless mode.
|
||||||
- [ ] It emits machine-readable JSON suitable for CI artifacts and a concise
|
- [x] It emits machine-readable JSON suitable for CI artifacts and a concise
|
||||||
human-readable summary.
|
human-readable summary.
|
||||||
- [ ] A test fixture can assert connection attempts and output token identity.
|
- [x] A test fixture can assert connection attempts and output token identity.
|
||||||
|
|
||||||
## Blocked by
|
## Blocked by
|
||||||
|
|
||||||
None - can start immediately.
|
None - completed. Verified with `PYTHONPATH=packages/node pytest -q tests/test_route_session_benchmark.py` (7 passed).
|
||||||
|
|
||||||
|
|||||||
@@ -15,9 +15,10 @@
|
|||||||
"Can assert connection count and output token identity"
|
"Can assert connection count and output token identity"
|
||||||
],
|
],
|
||||||
"priority": 1,
|
"priority": 1,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/01-baseline-profiling-harness.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/01-baseline-profiling-harness.md",
|
||||||
"dependsOn": []
|
"dependsOn": [],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-002",
|
"id": "DIP-002",
|
||||||
@@ -31,9 +32,12 @@
|
|||||||
"Tests cover binary, JSON, timeout, disconnect, cancellation, and cleanup"
|
"Tests cover binary, JSON, timeout, disconnect, cancellation, and cleanup"
|
||||||
],
|
],
|
||||||
"priority": 2,
|
"priority": 2,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/02-relay-session-compatibility.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/02-relay-session-compatibility.md",
|
||||||
"dependsOn": ["DIP-001"]
|
"dependsOn": [
|
||||||
|
"DIP-001"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-003",
|
"id": "DIP-003",
|
||||||
@@ -47,9 +51,12 @@
|
|||||||
"Benchmark shows healthy-session connection count independent of token count"
|
"Benchmark shows healthy-session connection count independent of token count"
|
||||||
],
|
],
|
||||||
"priority": 3,
|
"priority": 3,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/03-http-keepalive.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/03-http-keepalive.md",
|
||||||
"dependsOn": ["DIP-001"]
|
"dependsOn": [
|
||||||
|
"DIP-001"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-004",
|
"id": "DIP-004",
|
||||||
@@ -63,9 +70,12 @@
|
|||||||
"Tests verify cadence and cleanup"
|
"Tests verify cadence and cleanup"
|
||||||
],
|
],
|
||||||
"priority": 4,
|
"priority": 4,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/04-seam-telemetry.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/04-seam-telemetry.md",
|
||||||
"dependsOn": ["DIP-001"]
|
"dependsOn": [
|
||||||
|
"DIP-001"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-005",
|
"id": "DIP-005",
|
||||||
@@ -79,9 +89,12 @@
|
|||||||
"Tests cover compressible, incompressible, threshold, malformed, and legacy bodies"
|
"Tests cover compressible, incompressible, threshold, malformed, and legacy bodies"
|
||||||
],
|
],
|
||||||
"priority": 5,
|
"priority": 5,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/05-adaptive-compression.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/05-adaptive-compression.md",
|
||||||
"dependsOn": ["DIP-001"]
|
"dependsOn": [
|
||||||
|
"DIP-001"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-006",
|
"id": "DIP-006",
|
||||||
@@ -95,9 +108,12 @@
|
|||||||
"Wire and token-output regression tests pass"
|
"Wire and token-output regression tests pass"
|
||||||
],
|
],
|
||||||
"priority": 6,
|
"priority": 6,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/06-activation-framing-copies.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/06-activation-framing-copies.md",
|
||||||
"dependsOn": ["DIP-001"]
|
"dependsOn": [
|
||||||
|
"DIP-001"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-007",
|
"id": "DIP-007",
|
||||||
@@ -111,9 +127,13 @@
|
|||||||
"Tests cover chunking, slow consumers, failure, and legacy peers"
|
"Tests cover chunking, slow consumers, failure, and legacy peers"
|
||||||
],
|
],
|
||||||
"priority": 7,
|
"priority": 7,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/07-prefill-backpressure.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/07-prefill-backpressure.md",
|
||||||
"dependsOn": ["DIP-001", "DIP-004"]
|
"dependsOn": [
|
||||||
|
"DIP-001",
|
||||||
|
"DIP-004"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "DIP-008",
|
"id": "DIP-008",
|
||||||
@@ -127,9 +147,20 @@
|
|||||||
"Gate verifies token identity, session stability, and resource cleanup"
|
"Gate verifies token identity, session stability, and resource cleanup"
|
||||||
],
|
],
|
||||||
"priority": 8,
|
"priority": 8,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/distributed-inference-performance/issues/08-end-to-end-performance-gate.md",
|
"notes": "Source issue: .scratch/distributed-inference-performance/issues/08-end-to-end-performance-gate.md",
|
||||||
"dependsOn": ["DIP-002", "DIP-003", "DIP-004", "DIP-005", "DIP-006", "DIP-007"]
|
"dependsOn": [
|
||||||
|
"DIP-002",
|
||||||
|
"DIP-003",
|
||||||
|
"DIP-004",
|
||||||
|
"DIP-005",
|
||||||
|
"DIP-006",
|
||||||
|
"DIP-007"
|
||||||
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
}
|
}
|
||||||
]
|
],
|
||||||
}
|
"metadata": {
|
||||||
|
"updatedAt": "2026-07-12T02:35:28.752Z"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -71,6 +71,8 @@ As an operator and release engineer, I need clear doctor output and opt-in hardw
|
|||||||
|
|
||||||
Add a small generic capability domain object in the node package. `doctor` loads the requested generic model path through the same backend startup uses, executes a bounded real forward at the assigned Shard, and emits the report. Startup gates routable registration on the successful report. Registration carries validated capabilities; the tracker persists/exposes them and filters route candidates at the model/shard/recipe seam.
|
Add a small generic capability domain object in the node package. `doctor` loads the requested generic model path through the same backend startup uses, executes a bounded real forward at the assigned Shard, and emits the report. Startup gates routable registration on the successful report. Registration carries validated capabilities; the tracker persists/exposes them and filters route candidates at the model/shard/recipe seam.
|
||||||
|
|
||||||
|
**Assignment ownership:** NCA validates whatever the node loads; it does not assign models. Pinned vs tracker-managed assignment rules are in [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md). Demand-driven managed placement (Qwen scratch PRD) may only consume spare capacity; admission applies equally to pinned and managed loads.
|
||||||
|
|
||||||
The future signed-update contract is represented only by a local manifest version and generic schema in P0. A future Tracker Model Artifact Manifest may be signed data, but Node executable behavior remains supplied by signed Node releases.
|
The future signed-update contract is represented only by a local manifest version and generic schema in P0. A future Tracker Model Artifact Manifest may be signed data, but Node executable behavior remains supplied by signed Node releases.
|
||||||
|
|
||||||
## Success measures
|
## Success measures
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ This P0 makes a Node prove it can serve its selected Model Artifact and Shard be
|
|||||||
## Locked decisions
|
## Locked decisions
|
||||||
|
|
||||||
- A Node explicitly asked to serve a Model Preset fails closed when no validated recipe can execute it; it must not register as ready or accept paid inference.
|
- A Node explicitly asked to serve a Model Preset fails closed when no validated recipe can execute it; it must not register as ready or accept paid inference.
|
||||||
|
- **Assignment ownership:** startup/`--model` loads are **pinned**; tracker-managed demand placement (Qwen US-050) may use **spare capacity only** — [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md).
|
||||||
- Default validation covers the selected model/shard only. `meshnet-node doctor --all-recipes` is reserved for support and CI.
|
- Default validation covers the selected model/shard only. `meshnet-node doctor --all-recipes` is reserved for support and CI.
|
||||||
- A Model Preset may have multiple named recipes. Each independently proves a real forward; the Tracker schedules only validated recipes while considering measured performance.
|
- A Model Preset may have multiple named recipes. Each independently proves a real forward; the Tracker schedules only validated recipes while considering measured performance.
|
||||||
- Compatibility schemas are generic. A future Tracker may publish signed, data-only Model Artifact Manifests, but executable recipes arrive only through signed Node releases.
|
- Compatibility schemas are generic. A future Tracker may publish signed, data-only Model Artifact Manifests, but executable recipes arrive only through signed Node releases.
|
||||||
|
|||||||
@@ -35,11 +35,12 @@
|
|||||||
"Full pytest passes or an exact unrelated blocker is recorded"
|
"Full pytest passes or an exact unrelated blocker is recorded"
|
||||||
],
|
],
|
||||||
"priority": 2,
|
"priority": 2,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/node-capability-admission/issues/02-doctor-real-forward.md",
|
"notes": "Source issue: .scratch/node-capability-admission/issues/02-doctor-real-forward.md",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"NCA-001"
|
"NCA-001"
|
||||||
]
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "NCA-003",
|
"id": "NCA-003",
|
||||||
@@ -54,12 +55,13 @@
|
|||||||
"Full pytest passes or an exact unrelated blocker is recorded"
|
"Full pytest passes or an exact unrelated blocker is recorded"
|
||||||
],
|
],
|
||||||
"priority": 3,
|
"priority": 3,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/node-capability-admission/issues/03-fail-closed-startup-admission.md",
|
"notes": "Source issue: .scratch/node-capability-admission/issues/03-fail-closed-startup-admission.md",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"NCA-001",
|
"NCA-001",
|
||||||
"NCA-002"
|
"NCA-002"
|
||||||
]
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "NCA-004",
|
"id": "NCA-004",
|
||||||
@@ -76,12 +78,13 @@
|
|||||||
"Full pytest passes or an exact unrelated blocker is recorded"
|
"Full pytest passes or an exact unrelated blocker is recorded"
|
||||||
],
|
],
|
||||||
"priority": 4,
|
"priority": 4,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/node-capability-admission/issues/04-tracker-validated-capability-routing.md",
|
"notes": "Source issue: .scratch/node-capability-admission/issues/04-tracker-validated-capability-routing.md",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"NCA-001",
|
"NCA-001",
|
||||||
"NCA-003"
|
"NCA-003"
|
||||||
]
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "NCA-005",
|
"id": "NCA-005",
|
||||||
@@ -96,15 +99,16 @@
|
|||||||
"Full pytest passes or an exact unrelated blocker is recorded"
|
"Full pytest passes or an exact unrelated blocker is recorded"
|
||||||
],
|
],
|
||||||
"priority": 5,
|
"priority": 5,
|
||||||
"passes": false,
|
"passes": true,
|
||||||
"notes": "Source issue: .scratch/node-capability-admission/issues/05-docs-hardware-lane-contract.md",
|
"notes": "Source issue: .scratch/node-capability-admission/issues/05-docs-hardware-lane-contract.md",
|
||||||
"dependsOn": [
|
"dependsOn": [
|
||||||
"NCA-002",
|
"NCA-002",
|
||||||
"NCA-004"
|
"NCA-004"
|
||||||
]
|
],
|
||||||
|
"completionNotes": "Completed by agent"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"updatedAt": "2026-07-11T19:16:52.768Z"
|
"updatedAt": "2026-07-12T01:54:03.030Z"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -46,13 +46,12 @@ model rather than waiting for an operator to request a load.
|
|||||||
|
|
||||||
## Node ownership
|
## Node ownership
|
||||||
|
|
||||||
- A startup-assigned `(model, shard range, quantization)` is pinned and never
|
Reconciled with [ADR-0026](../../docs/adr/0026-node-assignment-ownership-and-managed-placement.md) and NCA (ADR-0023):
|
||||||
changed by the tracker.
|
|
||||||
- Spare capacity on a pinned node, and all capacity on a model-less node, is
|
- A **startup-assigned** `(model, shard range, quantization)` from explicit `--model` or accepted bootstrap assign is **pinned** until the operator restarts.
|
||||||
available for tracker-managed assignments.
|
- **Tracker-managed** assignments (this feature) use only **spare capacity** — model-less nodes or (future, US-048) unused shard slots — and are marked `managed: true`.
|
||||||
- Tracker-added assignments are explicitly marked managed and may be moved or
|
- The tracker may move or remove managed assignments under the safety policy below; it must not retarget a pinned serving assignment to satisfy demand.
|
||||||
removed by the tracker under the safety policy. Runtime UI controls are a
|
- Every assignment, pinned or managed, must pass NCA `doctor` before becoming routable when admission is enabled.
|
||||||
later feature.
|
|
||||||
|
|
||||||
## Pricing
|
## Pricing
|
||||||
|
|
||||||
|
|||||||
@@ -16,12 +16,9 @@
|
|||||||
|
|
||||||
|
|
||||||
.\.venv\Scripts\meshnet-node.exe start http://192.168.0.179:8081 --model-id Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
|
.\.venv\Scripts\meshnet-node.exe start http://192.168.0.179:8081 --model-id Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
|
||||||
.\.venv\Scripts\meshnet-node.exe start --tracker http://ai.neuron.d-popov.com --model-id Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
|
.\.venv\Scripts\meshnet-node.exe start --tracker http://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct --advertise-host 192.168.0.20
|
||||||
|
|
||||||
we .\.venv\Scripts\meshnet-node.exe start `
|
we .\.venv\Scripts\meshnet-node.exe start --tracker http://192.168.0.179:8081 --model Qwen/Qwen2.5-0.5B-Instruct
|
||||||
--tracker http://192.168.0.179:8081 `
|
|
||||||
--model Qwen/Qwen2.5-0.5B-Instruct `
|
|
||||||
--advertise-host 192.168.0.20
|
|
||||||
# trackers:
|
# trackers:
|
||||||
https://meshnet.2.d-popov.com
|
https://meshnet.2.d-popov.com
|
||||||
https://ai.neuron.d-popov.com
|
https://ai.neuron.d-popov.com
|
||||||
|
|||||||
38
docs/PRD.md
38
docs/PRD.md
@@ -1,4 +1,4 @@
|
|||||||
Status: done (base program US-001…US-035 complete; see `docs/prd.json`. Post-035 work lives in `docs/issues/36+` and `.scratch/`. Payment/settlement superseded by ADR-0015; fraud by ADR-0018.)
|
Status: done (US-001…US-035 complete; friends-test arc US-036…US-049 in `docs/prd.json`; US-048/050 tracked. See ADRs 0015–0018, 0023, 0025–0026.)
|
||||||
|
|
||||||
# Distributed Inference Network — PRD
|
# Distributed Inference Network — PRD
|
||||||
|
|
||||||
@@ -8,10 +8,12 @@ Running large language models requires expensive dedicated hardware that most pe
|
|||||||
|
|
||||||
## Solution
|
## Solution
|
||||||
|
|
||||||
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
|
A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning **USDT**. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start.
|
||||||
|
|
||||||
## User Stories
|
## User Stories
|
||||||
|
|
||||||
|
> **Status (2026-07-13):** Stories below are the original product intent. **Shipped behavior** is in [Implementation Decisions](#implementation-decisions) and ADRs 0015–0018, 0023, 0025–0026. Superseded lines are marked inline.
|
||||||
|
|
||||||
### Node Operator
|
### Node Operator
|
||||||
|
|
||||||
1. As a node operator, I want to install the node client with a single command (`pip install meshnet-node`), so that I can start contributing without reading documentation.
|
1. As a node operator, I want to install the node client with a single command (`pip install meshnet-node`), so that I can start contributing without reading documentation.
|
||||||
@@ -21,10 +23,10 @@ A volunteer GPU network where anyone can share their GPU by running a single com
|
|||||||
5. As a node operator, I want my assigned shard to download automatically from HuggingFace on first start, so that I don't have to manually find or download model weights.
|
5. As a node operator, I want my assigned shard to download automatically from HuggingFace on first start, so that I don't have to manually find or download model weights.
|
||||||
6. As a node operator, I want to seed my shard to other nodes via P2P once I have it, so that new nodes with the same shard assignment don't need to download from HuggingFace.
|
6. As a node operator, I want to seed my shard to other nodes via P2P once I have it, so that new nodes with the same shard assignment don't need to download from HuggingFace.
|
||||||
7. As a node operator, I want the node client to register with the tracker automatically and begin serving inference requests, so that I start earning as soon as setup is complete.
|
7. As a node operator, I want the node client to register with the tracker automatically and begin serving inference requests, so that I start earning as soon as setup is complete.
|
||||||
8. As a node operator, I want to see my current node score, shard assignment, and token earnings in the terminal, so that I can verify my node is contributing correctly.
|
8. As a node operator, I want to see my current node score, shard assignment, and USDT earnings in the terminal, so that I can verify my node is contributing correctly.
|
||||||
9. As a node operator, I want to stake tokens before serving paid inference, so that I have skin in the game and the network can trust my outputs.
|
9. As a node operator, I want to serve paid inference without upfront stake deposits, with my accrued USDT pending balance as fraud collateral and probation as the anti-sybil cost, so that onboarding stays frictionless. *(Supersedes stake-before-serving; ADR-0015/0018.)*
|
||||||
10. As a node operator, I want my first N jobs to run without earning (probationary period), so that the network can establish trust before paying me.
|
10. As a node operator, I want my first N jobs to run without earning (probationary period), so that the network can establish trust before paying me.
|
||||||
11. As a node operator, I want to be notified immediately if my stake is slashed due to a fraud detection event, so that I can investigate and fix the issue.
|
11. As a node operator, I want to be notified when my pending balance is forfeited due to a failed audit, so that I can investigate and fix the issue. *(Supersedes stake slash; ADR-0018 forfeiture.)*
|
||||||
12. As a node operator, I want to receive a strike and a warning before being banned, so that accidental failures don't immediately end my participation.
|
12. As a node operator, I want to receive a strike and a warning before being banned, so that accidental failures don't immediately end my participation.
|
||||||
13. As a node operator, I want to be automatically reassigned to a different shard when the tracker determines another shard is more in demand, so that my hardware is always optimally used.
|
13. As a node operator, I want to be automatically reassigned to a different shard when the tracker determines another shard is more in demand, so that my hardware is always optimally used.
|
||||||
14. As a node operator, I want the node client to reconnect automatically if the tracker is temporarily unavailable, so that transient network issues don't stop me from earning.
|
14. As a node operator, I want the node client to reconnect automatically if the tracker is temporarily unavailable, so that transient network issues don't stop me from earning.
|
||||||
@@ -34,8 +36,8 @@ A volunteer GPU network where anyone can share their GPU by running a single com
|
|||||||
### Client Developer
|
### Client Developer
|
||||||
|
|
||||||
17. As a client developer, I want to send `POST /v1/chat/completions` requests to the gateway in the same format as the OpenAI API, so that I can switch to the network with a one-line code change.
|
17. As a client developer, I want to send `POST /v1/chat/completions` requests to the gateway in the same format as the OpenAI API, so that I can switch to the network with a one-line code change.
|
||||||
18. As a client developer, I want to authenticate with an API key funded by SOL or USDC, so that I never need to acquire or hold our native token.
|
18. As a client developer, I want to authenticate with an API key funded by USDT, so that I never need to acquire or hold our native token. *(ADR-0015.)*
|
||||||
19. As a client developer, I want to top up my API key balance by sending SOL or USDC to a Solana address, so that payment is simple and familiar.
|
19. As a client developer, I want to top up my API key balance by sending USDT to the treasury Solana address, so that payment is simple and familiar. *(ADR-0015; wallet binding US-039/041.)*
|
||||||
20. As a client developer, I want to see a per-request cost estimate before sending a request, so that I can budget inference costs accurately.
|
20. As a client developer, I want to see a per-request cost estimate before sending a request, so that I can budget inference costs accurately.
|
||||||
21. As a client developer, I want to receive streaming responses (`text/event-stream`) in OpenAI-compatible format, so that I can build low-latency user experiences.
|
21. As a client developer, I want to receive streaming responses (`text/event-stream`) in OpenAI-compatible format, so that I can build low-latency user experiences.
|
||||||
22. As a client developer, I want `GET /v1/models` to return the list of available model presets on the network, so that I know what I can request.
|
22. As a client developer, I want `GET /v1/models` to return the list of available model presets on the network, so that I know what I can request.
|
||||||
@@ -46,15 +48,15 @@ A volunteer GPU network where anyone can share their GPU by running a single com
|
|||||||
|
|
||||||
### End User (via a client app)
|
### End User (via a client app)
|
||||||
|
|
||||||
27. As an end user, I want to buy SOL on any exchange and use it to pay for inference, so that I don't need to understand blockchain technology to use the service.
|
27. As an end user, I want to buy USDT on an exchange and use it to pay for inference via Solana, so that I don't need deep crypto knowledge to use the service. *(Clients pay USDT; SOL is only for network fees if they self-custody.)*
|
||||||
28. As an end user, I want responses of equivalent quality to centralised providers, so that I don't have to trade quality for cost savings.
|
28. As an end user, I want responses of equivalent quality to centralised providers, so that I don't have to trade quality for cost savings.
|
||||||
29. As an end user, I want low latency on first token, so that conversational applications feel responsive.
|
29. As an end user, I want low latency on first token, so that conversational applications feel responsive.
|
||||||
|
|
||||||
### Validator
|
### Validator
|
||||||
|
|
||||||
30. As a validator, I want to automatically re-run a random sample (~5%) of completed inference requests on a reference node, so that I can detect nodes returning fraudulent outputs.
|
30. As a validator, I want to automatically re-run a random sample (~5%) of completed inference requests on a reference node with TOPLOC activation verification, so that I can detect nodes returning fraudulent outputs. *(ADR-0018.)*
|
||||||
31. As a validator, I want to submit a fraud proof on-chain when a node's output diverges beyond tolerance, so that the slash event is recorded trustlessly.
|
31. As a validator, I want the tracker to record forfeiture and strikes when an audit fails, so that penalties are applied consistently. *(Supersedes on-chain fraud proof in alpha; ADR-0018.)*
|
||||||
32. As a validator, I want to earn a reward for each successful fraud detection, so that there is an economic incentive to run validation.
|
32. As a validator, I want economic incentive to run validation, so that fraud detection is not purely altruistic. *(Validator reward share deferred; forfeiture to protocol cut today.)*
|
||||||
|
|
||||||
### Network (tracker / system)
|
### Network (tracker / system)
|
||||||
|
|
||||||
@@ -62,7 +64,7 @@ A volunteer GPU network where anyone can share their GPU by running a single com
|
|||||||
34. As the tracker, I want to rebalance shard assignments across nodes when demand for a model preset changes, so that the network always covers the most-requested models.
|
34. As the tracker, I want to rebalance shard assignments across nodes when demand for a model preset changes, so that the network always covers the most-requested models.
|
||||||
35. As the tracker, I want to instruct a node to download a new shard when no other node covers it, so that model preset coverage is maintained automatically.
|
35. As the tracker, I want to instruct a node to download a new shard when no other node covers it, so that model preset coverage is maintained automatically.
|
||||||
36. As the tracker, I want to exclude banned wallets from route selection, so that fraudulent nodes cannot serve paid inference.
|
36. As the tracker, I want to exclude banned wallets from route selection, so that fraudulent nodes cannot serve paid inference.
|
||||||
37. As the tracker, I want to read stake, slash, strike, and ban state exclusively from Solana smart contracts, so that I cannot manipulate payouts even with full control of the routing layer.
|
37. As the tracker, I want strike and ban state persisted in the registry and enforced on route selection, so that fraudulent wallets cannot serve paid inference. *(Supersedes on-chain-only stake/slash registry; ADR-0018; on-chain deferred per ADR-0007/0015.)*
|
||||||
38. As the network, I want new model presets to be addable by submitting a HuggingFace model ID and shard count, so that the set of available models can grow without code changes.
|
38. As the network, I want new model presets to be addable by submitting a HuggingFace model ID and shard count, so that the set of available models can grow without code changes.
|
||||||
|
|
||||||
## Implementation Decisions
|
## Implementation Decisions
|
||||||
@@ -73,7 +75,7 @@ The codebase is organized as a Python monorepo with the following top-level pack
|
|||||||
- `packages/gateway` — OpenAI-compatible HTTP gateway and route orchestration
|
- `packages/gateway` — OpenAI-compatible HTTP gateway and route orchestration
|
||||||
- `packages/tracker` — centralized tracker service (node registry, scoring, route selection)
|
- `packages/tracker` — centralized tracker service (node registry, scoring, route selection)
|
||||||
- `packages/sdk` — `meshnet` Python SDK wrapping gateway + wallet controls
|
- `packages/sdk` — `meshnet` Python SDK wrapping gateway + wallet controls
|
||||||
- `packages/contracts` — Solana L2 smart contracts (stake, slash, strike, ban, settlement)
|
- `packages/contracts` — Solana adapter boundary (custodial USDT treasury, local registry prototype)
|
||||||
- `packages/p2p` — P2P gossip layer and shard swarm seeding
|
- `packages/p2p` — P2P gossip layer and shard swarm seeding
|
||||||
|
|
||||||
### Inference engine (ADR-0001; native GGUF path ADR-0024)
|
### Inference engine (ADR-0001; native GGUF path ADR-0024)
|
||||||
@@ -92,13 +94,13 @@ The gateway receives a client request, asks the tracker for an inference route (
|
|||||||
7. Begin accepting inference connections
|
7. Begin accepting inference connections
|
||||||
|
|
||||||
### Payment flow (ADR-0015 supersedes ADR-0002 settlement mechanics)
|
### Payment flow (ADR-0015 supersedes ADR-0002 settlement mechanics)
|
||||||
Clients pre-fund an API key with USDT. The tracker meters each request against the off-chain ledger. Periodic settlement batches USDT payouts from the custodial treasury to node operators proportional to work units. Fraud penalties forfeit pending balance (ADR-0018); strike/ban state persists in the tracker registry. TAI token emission remains deferred (ADR-0002 roadmap).
|
Clients pre-fund an API key with USDT. The tracker meters each request against the off-chain ledger. Periodic settlement batches USDT payouts from the custodial treasury to node operators proportional to work units (default: every 24 h or when pending ≥ 5 USDT). Fraud penalties forfeit pending balance (ADR-0018); strike/ban state persists in the tracker registry. TAI reward accrual is deferred — see ADR-0025 for reserved-mint / off-chain phase B/C; ADR-0002 roadmap for public listing.
|
||||||
|
|
||||||
### Fraud detection (ADR-0018; historical ADR-0003)
|
### Fraud detection (ADR-0018; historical ADR-0003)
|
||||||
Validators re-run ~5% of completed requests with TOPLOC activation verification. Caught cheaters forfeit pending balance and receive strikes; three strikes bans the wallet. Probation (first N unpaid jobs) remains the anti-sybil re-entry cost.
|
Validators re-run ~5% of completed requests with TOPLOC activation verification. Caught cheaters forfeit pending balance and receive strikes; three strikes bans the wallet. Probation (first N unpaid jobs) remains the anti-sybil re-entry cost.
|
||||||
|
|
||||||
### Tracker architecture (ADR-0004)
|
### Tracker architecture (ADR-0004)
|
||||||
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. Solana is the authoritative source of truth for all incentive-relevant state.
|
Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. **Alpha:** strike/ban/forfeiture state lives in the tracker registry (ADR-0018); USDT settlement via custodial treasury (ADR-0015). On-chain programs deferred (ADR-0007).
|
||||||
|
|
||||||
### Shard distribution (ADR-0005)
|
### Shard distribution (ADR-0005)
|
||||||
Shards are identified by `(model_preset, shard_index)`. On assignment, the node downloads the shard layers from HuggingFace using `huggingface_hub`. Once downloaded, the node joins the P2P shard swarm and seeds to other nodes requesting the same shard. Popular shards propagate entirely via P2P; cold shards fall back to HuggingFace.
|
Shards are identified by `(model_preset, shard_index)`. On assignment, the node downloads the shard layers from HuggingFace using `huggingface_hub`. Once downloaded, the node joins the P2P shard swarm and seeds to other nodes requesting the same shard. Popular shards propagate entirely via P2P; cold shards fall back to HuggingFace.
|
||||||
@@ -115,9 +117,9 @@ The gateway exposes OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/mo
|
|||||||
**Per-component seams:**
|
**Per-component seams:**
|
||||||
- **Tracker**: given a set of registered nodes with known shard coverage and node scores, assert `select_route(model_preset)` returns an optimal ordered list of node endpoints.
|
- **Tracker**: given a set of registered nodes with known shard coverage and node scores, assert `select_route(model_preset)` returns an optimal ordered list of node endpoints.
|
||||||
- **Node shard serving**: given an activation tensor for the node's layer range, assert the output tensor shape and dtype are correct.
|
- **Node shard serving**: given an activation tensor for the node's layer range, assert the output tensor shape and dtype are correct.
|
||||||
- **Fraud detection**: given a validator that re-runs a known-bad node response, assert a slash transaction is submitted on-chain with correct attribution.
|
- **Fraud detection**: given a validator that re-runs a known-bad node response, assert strike/forfeiture state updates with correct attribution (ADR-0018; on-chain slash deferred).
|
||||||
- **Shard swarm**: given a node that has a shard, assert a second node with the same assignment downloads it via P2P rather than HuggingFace.
|
- **Shard swarm**: given a node that has a shard, assert a second node with the same assignment downloads it via P2P rather than HuggingFace.
|
||||||
- **Payment settlement**: given a completed inference session with known compute attribution, assert token balances change by the expected amounts after epoch settlement.
|
- **Payment settlement**: given a completed inference session with known compute attribution, assert USDT ledger balances change by the expected amounts after epoch settlement (ADR-0015).
|
||||||
|
|
||||||
## Out of Scope
|
## Out of Scope
|
||||||
|
|
||||||
@@ -135,4 +137,4 @@ The gateway exposes OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/mo
|
|||||||
- The `meshnet-node` CLI is the primary viral growth vector. Every friction point in the install/start sequence costs node operators. The startup sequence must complete without any manual configuration on a machine with a CUDA-capable GPU.
|
- The `meshnet-node` CLI is the primary viral growth vector. Every friction point in the install/start sequence costs node operators. The startup sequence must complete without any manual configuration on a machine with a CUDA-capable GPU.
|
||||||
- The name "meshnet" is a working name. The actual package and token names are TBD.
|
- The name "meshnet" is a working name. The actual package and token names are TBD.
|
||||||
- The Solana L2 chain selection (vs Base/Arbitrum) is not yet finalised — both are cheap, EVM-compatible fallbacks. The contracts package should abstract chain-specific details.
|
- The Solana L2 chain selection (vs Base/Arbitrum) is not yet finalised — both are cheap, EVM-compatible fallbacks. The contracts package should abstract chain-specific details.
|
||||||
- The probationary period length (N free jobs) and slash amounts are economic parameters that will need tuning once the network has real usage data. Hardcode sensible defaults; make them on-chain governable.
|
- The probationary period length (N free jobs) and forfeiture amounts are economic parameters that will need tuning once the network has real usage data. Hardcode sensible defaults; governance TBD (ADR-0018).
|
||||||
|
|||||||
@@ -123,3 +123,11 @@ Rejected. gRPC/HTTP2 already provides mature streaming, flow control, deadlines,
|
|||||||
4. Real two-machine execution using both Shards.
|
4. Real two-machine execution using both Shards.
|
||||||
5. End-to-end performance/fit advantage over the current distributed route.
|
5. End-to-end performance/fit advantage over the current distributed route.
|
||||||
6. Separate Qwen3-family architecture certification.
|
6. Separate Qwen3-family architecture certification.
|
||||||
|
|
||||||
|
## Relationship to US-042 (whole-model GGUF shortcut)
|
||||||
|
|
||||||
|
[US-042](../issues/42-gguf-llamacpp-node-backend.md) **phase C** ships first: a node with enough RAM serves a **full** GGUF via llama.cpp on a single-hop Inference Route using the existing HTTP activation seam and PyTorch-era tracker integration. That is intentionally small and does not require this ADR's gRPC worker or llama.cpp patch stack.
|
||||||
|
|
||||||
|
This ADR's track starts only after **DGR-001** (controlled safetensors-vs-GGUF benchmark) shows a meaningful speed or fit benefit. Then implement the native worker (DGR-002+) — which subsumes US-042 direction A (layer-range GGUF + boundary tensors) if the benchmark warrants it.
|
||||||
|
|
||||||
|
Do not run US-042 phase C and DGR-008+ in parallel on the same node backend without an explicit integration plan; phase C uses llama-cpp-python (or equivalent) whole-model path; ADR-0024 uses the standalone C++ worker.
|
||||||
|
|||||||
52
docs/adr/0025-tai-off-chain-accrual-and-reserved-mint.md
Normal file
52
docs/adr/0025-tai-off-chain-accrual-and-reserved-mint.md
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
# ADR-0025: TAI reserved mint and off-chain accrual (phase B/C)
|
||||||
|
|
||||||
|
## Status: Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
ADR-0015 chose **USDT-direct custodial settlement** for alpha and near-term production. Clients pay USDT; nodes receive batched USDT SPL payouts. ADR-0002's TAI reward token, revenue-backed floor, and open-market listing gates remain the long-term design but are **not** the live payment path.
|
||||||
|
|
||||||
|
The owner wants TAI to exist without the cost and legal surface of a public launch: no AMM, no open listing, no client-facing TAI, no on-chain stake machinery.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
### Phase B — Reserved mainnet mint (cheap, optional early)
|
||||||
|
|
||||||
|
- Create a fixed-supply TAI SPL mint on **mainnet** when treasury work happens (~0.002 SOL).
|
||||||
|
- Entire initial supply sits in a **team-controlled** wallet (same custody posture as the USDT treasury today).
|
||||||
|
- **No public emission, no market, no client UX.** Mint exists for name reservation and future programmatic rewards only.
|
||||||
|
- Document mint address in operator config; do not advertise to users.
|
||||||
|
|
||||||
|
### Phase C — Off-chain TAI accrual alongside USDT (before automatic on-chain TAI payouts)
|
||||||
|
|
||||||
|
- Extend the billing ledger with **`tai_pending[wallet]`** accrued from completed inference work using a simple rule (e.g. USDT node share × configurable TAI-per-USDT rate, or fixed TAI per work unit).
|
||||||
|
- TAI accrual is **display-only + ledger-persisted** initially; nodes see pending TAI in dashboard/CLI.
|
||||||
|
- **Clients never pay or hold TAI.** USDT remains the only client-facing asset.
|
||||||
|
- Optional manual or scheduled **TAI SPL batch transfers** from the team wallet (same batching pattern as USDT `send_payouts`) — operator-triggered until automatic emission is justified by volume.
|
||||||
|
- The existing **10% protocol USDT cut** continues to accumulate as future TAI liquidity per ADR-0015/0002; do not redirect it until a deliberate liquidity event.
|
||||||
|
|
||||||
|
### Explicit non-goals (this ADR)
|
||||||
|
|
||||||
|
- Open-market listing, AMM, or DEX liquidity
|
||||||
|
- Buyback floor endpoint or backing-price oracle (ADR-0002 machinery)
|
||||||
|
- On-chain stake deposits or slash contracts
|
||||||
|
- Paying clients rebates or accepting TAI for inference
|
||||||
|
- Replacing USDT node payouts with TAI-only payouts before volume gates in ADR-0002 pass
|
||||||
|
|
||||||
|
## Relation to ADR-0002 listing gates
|
||||||
|
|
||||||
|
Public TAI listing stays gated on **$50k cumulative USDT volume** and **25+ nodes / 15+ wallets**. Phase B/C may proceed **below** those gates because they do not create a public market — only reserved supply and off-chain accounting.
|
||||||
|
|
||||||
|
Securities review remains required before any **public** distribution or listing; off-chain accrual to hired/known operators with manual SPL transfers is an operator discretion, not a product promise.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- USDT mainnet pilot (two-wallet setup) is unblocked without TAI complexity.
|
||||||
|
- TAI narrative is preserved at minimal cost (mint + ledger column + optional manual transfers).
|
||||||
|
- Automatic TAI emission can later reuse the US-033 settlement loop shape with a second mint and separate pending bucket.
|
||||||
|
- Dashboard and APIs must label TAI balances as **non-withdrawable** until an on-chain payout batch confirms.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- USDT settlement tests remain authoritative for production payouts (`tests/test_settlement_loop.py`).
|
||||||
|
- When phase C lands: ledger tests for `tai_pending` accrual, idempotent gossip replication, and optional TAI batch payout adapter tests mirroring USDT.
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
# ADR-0026: Node assignment ownership — pinned startup vs managed demand placement
|
||||||
|
|
||||||
|
## Status: Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Three features define how a node gets its `(model, shard range, recipe/quantization)`:
|
||||||
|
|
||||||
|
1. **ADR-0011 / US-013** — tracker suggests a gap from coverage map on startup or auto-join.
|
||||||
|
2. **Node capability admission (ADR-0023 / NCA)** — a node must pass `doctor` + real forward before becoming routable; startup-assigned work is validated, not blindly trusted.
|
||||||
|
3. **Qwen demand placement** (`.scratch/qwen3.6-27b-demand-placement/`) — tracker deploys a model when chat demand appears and spare capacity exists.
|
||||||
|
|
||||||
|
These looked contradictory: NCA and the Qwen PRD both say startup assignments are "pinned," while demand placement wants the tracker to assign models dynamically.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
### Three assignment tiers
|
||||||
|
|
||||||
|
| Tier | How it is created | Mutable by tracker? | Admission |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **Operator-initiated** | Node starts with explicit `--model` / shard flags | **No** — pinned until operator restarts or explicitly reloads | Must pass NCA `doctor` before routable |
|
||||||
|
| **Network bootstrap** | `/v1/network/assign` or `/v1/nodes/assign` on first join (ADR-0011) | **No** for the active loaded shard — treated as operator-equivalent once accepted at startup | Must pass NCA before routable |
|
||||||
|
| **Tracker-managed** | Demand-driven placement (Qwen PRD) on spare capacity | **Yes** — marked `managed: true`; subject to cooldown / safety policy | Must pass NCA for the new assignment before routable |
|
||||||
|
|
||||||
|
### Spare capacity rule (unifies NCA + Qwen)
|
||||||
|
|
||||||
|
- A node’s **active** `(model, shard, recipe)` from startup is **pinned** — the tracker does not silently retarget a serving node to a different model.
|
||||||
|
- **Spare capacity** — memory/slots not holding the pinned assignment, or a node registered without a model — may receive **tracker-managed** assignments to satisfy demand.
|
||||||
|
- Until multi-shard runtime exists (US-048), “spare capacity” effectively means **model-less nodes** or nodes explicitly registered for managed placement; do not overload a single-shard node with a second assignment.
|
||||||
|
|
||||||
|
### Demand placement interaction
|
||||||
|
|
||||||
|
- First chat request for an unrouted model queues **demand**; leader tracker may assign **managed** nodes only when eligible spare capacity exists (Qwen PRD).
|
||||||
|
- Until complete coverage + validated recipes exist, return retryable `503 model_loading` with coverage metadata.
|
||||||
|
- Managed assignments must not evict pinned assignments on other nodes without the Qwen safety policy (≥3 copies, 1.5× demand multiplier, cooldown).
|
||||||
|
|
||||||
|
### NCA is not optional for any tier
|
||||||
|
|
||||||
|
Regardless of assignment source, registration carries **validated capability** only after `doctor` succeeds. The tracker excludes nodes with absent, stale, or failed capability reports (ADR-0023).
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- NCA and Qwen demand placement are complementary: NCA gates *quality*; demand placement gates *where new coverage comes from*.
|
||||||
|
- US-048 (multi-shard slots) extends spare capacity — until then, demand placement primarily targets nodes that join without `--model`.
|
||||||
|
- Rebalance / dropout relocation (US-013, US-048) applies to **coverage gaps**, not retroactive retargeting of pinned nodes for demand convenience.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- NCA tests: unvalidated nodes never routed.
|
||||||
|
- Demand-placement tests (when implemented): managed flag set; pinned nodes unchanged.
|
||||||
|
- Documented in Qwen scratch PRD and NCA README cross-links.
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
Status: superseded
|
||||||
|
|
||||||
|
# Superseded — renumbered to US-048
|
||||||
|
|
||||||
|
This issue slot was a duplicate of tracker-node-hardening (US-020). Memory budget / shard slots / dropout relocation work lives at:
|
||||||
|
|
||||||
|
- **Issue:** [48-memory-budget-shard-slots-and-dropout-relocation.md](./48-memory-budget-shard-slots-and-dropout-relocation.md)
|
||||||
|
- **PRD:** `docs/prd.json` → `US-048`
|
||||||
|
|
||||||
|
Do not implement from this file.
|
||||||
@@ -1,9 +1,16 @@
|
|||||||
# US-042 — GGUF/llama.cpp node backend
|
# US-042 — GGUF/llama.cpp node backend
|
||||||
|
|
||||||
Status: planned
|
Status: planned
|
||||||
Priority: High (whole-model GGUF shortcut; distributed path in [ADR-0024](../adr/0024-distributed-gguf-runtime.md))
|
Priority: High (unlocks DeepSeek-V4-Flash on volunteer hardware — the pool's core value)
|
||||||
Stage: Draft design
|
Stage: Draft design
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Run **DeepSeek-V4-Flash** as the first real large-model target on volunteer
|
||||||
|
hardware via GGUF/llama.cpp. This epic is no longer GLM-oriented: the initial
|
||||||
|
objective is to prove that DeepSeek-V4-Flash can load and serve correctly on
|
||||||
|
consumer/unified-memory nodes, then expand from there.
|
||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The node backend is transformers-only (`model_backend.py` →
|
The node backend is transformers-only (`model_backend.py` →
|
||||||
|
|||||||
@@ -86,10 +86,10 @@ What exists already (build on it, don't duplicate):
|
|||||||
- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
|
- [ ] Two-machine test: machine A (tracker + node, holds full snapshot) serves
|
||||||
layers 0–k; machine B joins with no model and receives **only** the files
|
layers 0–k; machine B joins with no model and receives **only** the files
|
||||||
for its assigned range from A — nothing fetched from HF
|
for its assigned range from A — nothing fetched from HF
|
||||||
- [ ] Machine B's resident memory scales with its shard size, not model size
|
- [x] Machine B's resident memory scales with its shard size, not model size
|
||||||
- [ ] Checksums verified end-to-end; corrupted transfer falls back cleanly
|
- [x] Checksums verified end-to-end; corrupted transfer falls back cleanly
|
||||||
- [x] Single-node/full-model flows unchanged
|
- [x] Single-node/full-model flows unchanged
|
||||||
- [ ] `python -m pytest` passes from repo root
|
- [x] `python -m pytest` passes from repo root
|
||||||
|
|
||||||
## Implementation notes
|
## Implementation notes
|
||||||
|
|
||||||
@@ -98,6 +98,13 @@ What exists already (build on it, don't duplicate):
|
|||||||
`full_url`; HuggingFace remains fallback-only, and when it is used the node
|
`full_url`; HuggingFace remains fallback-only, and when it is used the node
|
||||||
computes `allow_patterns` from the repo's remote SafeTensors index so it
|
computes `allow_patterns` from the repo's remote SafeTensors index so it
|
||||||
stays layer-filtered even without tracker-cached files. Remaining hard half
|
stays layer-filtered even without tracker-cached files. Remaining hard half
|
||||||
is true partial model materialization: the backend can prefer a downloaded
|
is partial model materialization: the backend can prefer a downloaded
|
||||||
local model directory, but Transformers still needs a `meta`-device load
|
local model directory, but Transformers still needs a `meta`-device load
|
||||||
path that materializes only assigned layers.
|
path that materializes only assigned layers.
|
||||||
|
- 2026-07-13: Partial LOAD implemented. `_load_partial_model_from_snapshot` builds
|
||||||
|
on `meta` via `init_empty_weights`, materializes only layer-scoped checkpoint
|
||||||
|
tensors, and finalizes device placement without copying unmaterialized meta
|
||||||
|
weights (`_finalize_active_shard_modules_on_device`). Tests cover memory
|
||||||
|
scaling (`test_partial_snapshot_resident_weight_numel_scales_with_shard`)
|
||||||
|
and real-torch meta-vs-materialized counts. Remaining: live two-machine LAN
|
||||||
|
verification.
|
||||||
|
|||||||
101
docs/issues/49-mainnet-usdt-cutover-two-wallet-pilot.md
Normal file
101
docs/issues/49-mainnet-usdt-cutover-two-wallet-pilot.md
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
# US-049 — Mainnet USDT cutover: two-wallet pilot checklist
|
||||||
|
|
||||||
|
Priority: High (first real-money friends test)
|
||||||
|
Stage: Operator runbook + config verification
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Move from **Solana devnet + mock-USDT** to **Solana mainnet + real USDT** for a minimal pilot: **one client wallet** (inference payer) and **one node-operator wallet** (payout recipient). Treasury holds USDT and pays SOL fees. TAI stays phase B/C per [ADR-0025](../adr/0025-tai-off-chain-accrual-and-reserved-mint.md).
|
||||||
|
|
||||||
|
## Wallet roles
|
||||||
|
|
||||||
|
| Role | Keypair | On-chain use |
|
||||||
|
|---|---|---|
|
||||||
|
| **Treasury** | Operator `treasury-keypair.json` (multisig when ready) | Holds USDT float + SOL for fees; sends batched node payouts |
|
||||||
|
| **Client** | Your inference-user wallet | SPL USDT → treasury; bound to API key for ledger credit |
|
||||||
|
| **Node** | Your node-operator wallet | Receives USDT payout batches from treasury |
|
||||||
|
|
||||||
|
The node process already creates/loads a Solana wallet at startup; the client wallet is bound via accounts/dashboard (`POST /v1/wallet/register` or US-041 flows).
|
||||||
|
|
||||||
|
## Pre-flight (devnet smoke — do not skip)
|
||||||
|
|
||||||
|
- [ ] Tracker with `--solana-rpc-url https://api.devnet.solana.com`, mock mint, treasury keypair
|
||||||
|
- [ ] `--settle-period 60 --payout-threshold 0` — confirm payout appears on dashboard **Settlement history** with explorer link
|
||||||
|
- [ ] Run `python -m pytest tests/test_settlement_loop.py -q` — includes prod 24h/5 USDT gate tests
|
||||||
|
- [ ] One inference request → node pending → settlement tx → node wallet balance increases
|
||||||
|
|
||||||
|
## Mainnet config change (config-only cutover)
|
||||||
|
|
||||||
|
Replace devnet values; **no code deploy required** beyond what is already on the branch.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Example — use your mainnet RPC provider
|
||||||
|
meshnet-tracker start \
|
||||||
|
--solana-rpc-url https://api.mainnet-beta.solana.com \
|
||||||
|
--usdt-mint EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v \
|
||||||
|
--treasury-keypair ~/.config/solana/meshnet-treasury-mainnet.json \
|
||||||
|
--settle-period 86400 \
|
||||||
|
--payout-threshold 5.0 \
|
||||||
|
--payout-dust-floor 0.01 \
|
||||||
|
--starting-credit 0 \
|
||||||
|
--devnet-topup 0
|
||||||
|
```
|
||||||
|
|
||||||
|
| Flag | Devnet (test) | Mainnet (pilot) |
|
||||||
|
|---|---|---|
|
||||||
|
| RPC | `api.devnet.solana.com` | Mainnet RPC (Helius/QuickNode/etc.) |
|
||||||
|
| `--usdt-mint` | mock mint from `devnet_setup.py` | Real USDT mint (`EPjF…` on Solana) |
|
||||||
|
| `--settle-period` | `60` (fast verify) | `86400` (24 h) |
|
||||||
|
| `--payout-threshold` | `0` | `5.0` USDT |
|
||||||
|
| `--starting-credit` | `1.0` (optional) | `0` |
|
||||||
|
| `--devnet-topup` | `1.0` | `0` |
|
||||||
|
|
||||||
|
## Treasury funding
|
||||||
|
|
||||||
|
- [ ] Fund treasury wallet with **SOL** for fees (~0.1–0.5 SOL to start; ~$0.001 per daily batch + ~$0.30 once per new node ATA)
|
||||||
|
- [ ] Fund treasury with **USDT** for node payouts (your float — e.g. first week of expected node earnings)
|
||||||
|
- [ ] Client wallet holds USDT; send a test SPL transfer to treasury ATA; confirm deposit watcher credits API key within one poll interval
|
||||||
|
|
||||||
|
## Two-wallet pilot steps
|
||||||
|
|
||||||
|
1. **Start tracker** on mainnet config above (single settlement tracker per ADR-0016).
|
||||||
|
2. **Client path:** register account → create API key → bind **client wallet** → deposit USDT to treasury → verify ledger balance on dashboard.
|
||||||
|
3. **Node path:** start `meshnet-node` with **node wallet** keypair → register → serve inference.
|
||||||
|
4. **Inference:** client sends `POST /v1/chat/completions` with API key; verify 402 before deposit, success after.
|
||||||
|
5. **Accrual:** confirm node **pending USDT** on dashboard rises; client balance debits.
|
||||||
|
6. **Payout (24 h):** wait for `--settle-period` **or** temporarily lower to `300` for first pilot verification, then restore `86400`.
|
||||||
|
7. **Threshold path:** alternatively, accumulate ≥ `5` USDT pending in one session to trigger immediate batch without waiting 24 h.
|
||||||
|
8. **Verify on-chain:** settlement history shows mainnet tx signature; node wallet USDT ATA balance increased; pending zeroed.
|
||||||
|
|
||||||
|
## Safety checks
|
||||||
|
|
||||||
|
- [ ] `--devnet-topup 0` — no faucet on mainnet
|
||||||
|
- [ ] `--starting-credit 0` — no free inference credit
|
||||||
|
- [ ] Treasury keypair not committed to git; file mode 600
|
||||||
|
- [ ] Plan multisig migration before large float ([alpha runbook](../../.scratch/alpha-hardening/runbooks/02-treasury-key-rotation.md) intent)
|
||||||
|
- [ ] Issue **21** (TOPLOC calibration) before production audit thresholds on untrusted nodes — runbook: [04-toploc-calibration-run](../../.scratch/alpha-hardening/runbooks/04-toploc-calibration-run.md)
|
||||||
|
|
||||||
|
## Cost estimate (this pilot)
|
||||||
|
|
||||||
|
- **SOL fees:** pennies per day at 1 batch × 1 node
|
||||||
|
- **USDT:** whatever clients deposit and nodes earn (treasury is passthrough for payouts)
|
||||||
|
- **TAI mint (optional, ADR-0025 phase B):** ~0.002 SOL one-time — defer if not needed for pilot
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- [ ] Devnet checklist completed once
|
||||||
|
- [ ] Mainnet tracker serves dashboard; billing enabled
|
||||||
|
- [ ] Client wallet deposit → ledger credit → inference → debit
|
||||||
|
- [ ] Node wallet receives ≥1 confirmed USDT payout batch on mainnet
|
||||||
|
- [ ] 24 h period enforced: sub-threshold pending not paid before period (covered by `tests/test_settlement_loop.py`)
|
||||||
|
- [ ] ≥5 USDT pending triggers payout without waiting full period (covered by tests)
|
||||||
|
- [ ] Rollback documented: switch RPC + mint back to devnet if needed
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- ADR-0015 (USDT custodial settlement)
|
||||||
|
- ADR-0025 (TAI reserved mint / off-chain accrual — not blocking this pilot)
|
||||||
|
- US-033 / US-032 (settlement + deposits)
|
||||||
|
- `scripts/devnet_setup.py` (devnet only)
|
||||||
15
docs/issues/50-qwen3.6-27b-demand-placement.md
Normal file
15
docs/issues/50-qwen3.6-27b-demand-placement.md
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
Status: in-design
|
||||||
|
|
||||||
|
# US-050 — Qwen3.6-27B demand-driven managed placement
|
||||||
|
|
||||||
|
> Full spec: [.scratch/qwen3.6-27b-demand-placement/PRD.md](../../.scratch/qwen3.6-27b-demand-placement/PRD.md)
|
||||||
|
> Assignment rules: [ADR-0026](../adr/0026-node-assignment-ownership-and-managed-placement.md)
|
||||||
|
> Admission: [ADR-0023](../adr/0023-model-agnostic-node-capability-admission.md)
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Deploy `Qwen/Qwen3.6-27B` when chat demand appears and **spare** fleet capacity exists. Startup `--model` assignments stay **pinned**; tracker-managed loads fill gaps on model-less or (future US-048) unused slot capacity only.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
See scratch PRD and `docs/prd.json` US-050.
|
||||||
301
docs/prd.json
301
docs/prd.json
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "Distributed Inference Network",
|
"name": "Distributed Inference Network",
|
||||||
"description": "Build a distributed inference network with node, gateway, tracker, SDK, contracts, and P2P shard distribution components from the grill session PRD.",
|
"description": "Distributed inference network: base program US-001…US-035 complete; friends-test arc US-036…US-049; capacity/placement US-048/050. Scratch features (alpha hardening AH-001…AH-025, NCA, distributed GGUF, Qwen demand) have separate prd.json files under .scratch/.",
|
||||||
"branchName": "ralph/distributed-inference-network",
|
"branchName": "ralph/distributed-inference-network",
|
||||||
"userStories": [
|
"userStories": [
|
||||||
{
|
{
|
||||||
@@ -814,10 +814,307 @@
|
|||||||
"US-033"
|
"US-033"
|
||||||
],
|
],
|
||||||
"completionNotes": "GET /dashboard served from embedded dashboard.html (package-data, no build step) by any tracker. Panels: hive/leader (raft status), nodes+coverage grouped by model, client balances, node pending + protocol cut, settlement history with devnet explorer links, strikes/bans/forfeitures (GET /v1/registry/wallets + snapshot forfeits), RPM stats. 4s auto-refresh via fetch polling. 3 tests in tests/test_dashboard.py."
|
"completionNotes": "GET /dashboard served from embedded dashboard.html (package-data, no build step) by any tracker. Panels: hive/leader (raft status), nodes+coverage grouped by model, client balances, node pending + protocol cut, settlement history with devnet explorer links, strikes/bans/forfeitures (GET /v1/registry/wallets + snapshot forfeits), RPM stats. 4s auto-refresh via fetch polling. 3 tests in tests/test_dashboard.py."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-036",
|
||||||
|
"title": "36 — Streamed chat completions over the relay RPC path",
|
||||||
|
"description": "Public NAT deployments proxy every chat request tracker → relay → head node. Implement true multi-frame SSE streaming over the relay WebSocket so clients see live tokens and relayed streams bill through the same SSE accounting loop as direct proxy streams. Inter-node /forward activation hops stay single-frame (ADR-0014).",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"stream: true chat via relay delivers SSE chunks incrementally (≥2 distinct frame arrivals before [DONE])",
|
||||||
|
"Relayed streamed request records nonzero billed tokens and node work credit",
|
||||||
|
"Non-streamed relayed requests and /forward binary hops behave exactly as before (single frame, body_base64 intact)",
|
||||||
|
"Legacy single-frame response from an old node is accepted as terminal",
|
||||||
|
"Idle stream (no frame for 120 s) returns 504 and cleans up the relay-side queue",
|
||||||
|
"Extend tests/test_gossip_and_relay.py alongside test_relay_rpc_round_trips_http_request_to_peer",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 36,
|
||||||
|
"status": "needs-review",
|
||||||
|
"notes": "Source issue: docs/issues/36-relay-streamed-chat.md. Implemented via _stream_relayed_frames in server.py; verify on public NAT relay before friends-test.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-029",
|
||||||
|
"US-031"
|
||||||
|
],
|
||||||
|
"completionNotes": "Multi-frame relay-http-response protocol; node relay_bridge line-by-line SSE emit; relay server per-request asyncio.Queue; tracker _stream_relayed_frames with SSE billing parity. Client mid-stream disconnect accepted limitation for alpha."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-037",
|
||||||
|
"title": "37 — Concurrent request handling in the node relay bridge",
|
||||||
|
"description": "RelayHttpBridge currently handles relay-http-request envelopes serially, blocking up to 300 s per request. Off-LAN a node can be head of one route and downstream hop of another — overlapping routes through a shared node break. Dispatch on a bounded ThreadPoolExecutor (default 8, configurable) with per-frame WS send locking compatible with US-036 streaming.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"While one relayed request is in flight, a second relay-http-request to the same node completes without waiting for the first",
|
||||||
|
"Responses are correctly matched by request_id when they complete out of order",
|
||||||
|
"More than N simultaneous requests queue and all eventually complete; thread count never exceeds N workers",
|
||||||
|
"Bridge survives a relay reconnect with workers still in flight (no crash, no deadlock; orphaned responses dropped)",
|
||||||
|
"Configurable via meshnet-node start --relay-concurrency N (env MESHNET_RELAY_CONCURRENCY)",
|
||||||
|
"Extend tests/test_gossip_and_relay.py",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 37,
|
||||||
|
"status": "done",
|
||||||
|
"notes": "Source issue: docs/issues/37-relay-bridge-concurrency.md. Critical for public friends-test; blocks concurrent head + hop on same node.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-036"
|
||||||
|
],
|
||||||
|
"completionNotes": "ThreadPoolExecutor dispatch in relay_bridge.py; per-frame WS send lock; test_relay_bridge_serves_concurrent_requests; --relay-concurrency CLI flag sets MESHNET_RELAY_CONCURRENCY."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-038",
|
||||||
|
"title": "38 — Tracker cluster join via a single seed peer",
|
||||||
|
"description": "Tracker cluster membership is static today — a newcomer configured with only one existing peer is never learned by the rest of the hive and quorum math diverges. A joining tracker configured with any one live seed announces via hive-HMAC-signed POST /v1/cluster/join; membership changes replicate through the Raft log and persist across restarts.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Start trackers A+B; start C with only A as seed → within one election timeout A, B, and C report the same 3-peer membership on GET /v1/cluster/peers, and a value proposed on C commits on A and B",
|
||||||
|
"Join without a valid hive signature is rejected with 403; join to a follower is forwarded to the leader",
|
||||||
|
"Restarting C with its seed offline rejoins from persisted membership",
|
||||||
|
"Standalone tracker (no seeds) behaves exactly as today",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 38,
|
||||||
|
"status": "open",
|
||||||
|
"notes": "Source issue: docs/issues/38-tracker-seed-join.md. Out of scope: peer removal, joint consensus, automatic seed retry.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-013",
|
||||||
|
"US-017"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-039",
|
||||||
|
"title": "39 — Caller Credit granted once per account; chat requires account keys",
|
||||||
|
"description": "DEFAULT_STARTING_CREDIT=0 and no grant path leaves every fresh public tracker request at 402. Grant Caller Credit once per account on first API key creation via deterministic event id caller-credit-{account_id}; chat on accounts-enabled trackers requires a real active sk-mesh- key (401 for invented bearers).",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Fresh account → first key → key has --starting-credit balance; chat succeeds",
|
||||||
|
"Second key on the same account → no additional credit",
|
||||||
|
"Revoke-and-recreate keys → still no additional credit (deterministic event id)",
|
||||||
|
"Random bearer string on an accounts-enabled tracker → 401, never 402/free work",
|
||||||
|
"Tracker without accounts store: gate behavior unchanged",
|
||||||
|
"--starting-credit 0 disables the grant entirely (mainnet posture)",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 39,
|
||||||
|
"status": "done",
|
||||||
|
"notes": "Source issue: docs/issues/39-caller-credit-account-keys.md. Critical for friends-test inference.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-031",
|
||||||
|
"US-035"
|
||||||
|
],
|
||||||
|
"completionNotes": "Caller credit granted once per account on first API key via deterministic event id; tests/test_accounts.py covers grant, revoke, and invented-bearer rejection."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-040",
|
||||||
|
"title": "40 — Devnet top-up button on the dashboard",
|
||||||
|
"description": "After Caller Credit (US-039) is spent, devnet friends need a dashboard faucet refill without on-chain USDT deposits. POST /v1/account/topup (session-authenticated) credits a configured fixed amount per click; flag off returns 404 and hides the button.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Flag off: endpoint 404s, dashboard shows no top-up button",
|
||||||
|
"Flag on: logged-in user tops up own key, balance rises by exactly N",
|
||||||
|
"Topping up another account's key → 403",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 40,
|
||||||
|
"status": "done",
|
||||||
|
"notes": "Source issue: docs/issues/40-devnet-dashboard-topup.md. Mainnet deployments set --devnet-topup 0.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-039"
|
||||||
|
],
|
||||||
|
"completionNotes": "POST /v1/account/topup with session auth and flag gating; tests/test_accounts.py covers flag off/on, own-account credit, and cross-account 403."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-041",
|
||||||
|
"title": "41 — Account wallet: browser-extension signing, in-browser generation, export-only",
|
||||||
|
"description": "Accounts need a visible wallet for deposit attribution without the tracker ever holding private keys. Dashboard integrates Solana wallet-adapter connect+nonce proof, or in-browser keypair generation with one-time export; no private-key import endpoint.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Connect-extension flow stores a verified pubkey (rejects unsigned/mismatched nonce proofs)",
|
||||||
|
"Generate flow: pubkey lands on the account; private key is never sent to the tracker, export works",
|
||||||
|
"No endpoint or UI accepts a private key",
|
||||||
|
"Deposits to the shown address credit the account's keys via the existing watcher",
|
||||||
|
"Address visible on the account panel after either flow",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 41,
|
||||||
|
"status": "open",
|
||||||
|
"notes": "Source issue: docs/issues/41-account-wallet-keypair.md. Not needed for devnet friends test; needed before mainnet.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-032",
|
||||||
|
"US-039"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-042",
|
||||||
|
"title": "42 — GGUF/llama.cpp node backend (phase C whole-model first)",
|
||||||
|
"description": "Node backend is transformers-only today; large MoE models on consumer hardware require GGUF via llama.cpp. Phase C: whole-model GGUF nodes (single-hop routes) first; partial-layer distributed GGUF deferred to ADR-0024. Also: GGUF catalog entries, Strix Halo/Vulkan hardware detection, download dir applies to GGUF files.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"A node with --gguf <repo-or-path> --quant IQ3_XXS serves /v1/chat/completions via llama.cpp with GPU offload where available",
|
||||||
|
"Tracker treats it as a full-coverage node (single-hop routes, billing works)",
|
||||||
|
"Streamed responses work through the tracker proxy and the relay (US-036)",
|
||||||
|
"python -m pytest passes from repo root (llama.cpp behind an optional extra)"
|
||||||
|
],
|
||||||
|
"priority": 42,
|
||||||
|
"status": "in-design",
|
||||||
|
"notes": "Source issue: docs/issues/42-gguf-llamacpp-node-backend.md. Phase C before ADR-0024 distributed worker; see runtime sequencing in issue file.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-036"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-043",
|
||||||
|
"title": "43 — Dashboard model search and model cards",
|
||||||
|
"description": "Dashboard lacks model-centric discovery. Add server-side HF search proxy merged with tracker presets and live coverage; model cards show architecture, coverage gaps, pricing, memory per quant, and a request-this-model action. Featured section driven by CURATED_MODELS including GGUF once US-042 lands.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Searching a HF repo id or free text returns results without the browser calling HF directly",
|
||||||
|
"A served model's card shows live coverage and a working chat-now state",
|
||||||
|
"An unserved model's card shows the request action and estimated memory per quant",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 43,
|
||||||
|
"status": "open",
|
||||||
|
"notes": "Source issue: docs/issues/43-dashboard-model-search-cards.md. Post-deploy polish.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-035"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-044",
|
||||||
|
"title": "44 — Tracker as model-file source; nodes download only their shard",
|
||||||
|
"description": "Second nodes joining a fleet today download entire HF snapshots even for small shard assignments. Tracker --models-dir advertises layer-scoped safetensors subsets; nodes race tracker/peer sources before HF allow_patterns fallback. Hard half remaining: meta-device partial model materialization so resident memory scales with shard size, not full model size.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Tracker started with --models-dir / MESHNET_MODELS_DIR advertises local model-file sources in assignment responses",
|
||||||
|
"Tracker serves a tar stream (or per-file API) containing only safetensors files for the assigned layer range plus config/tokenizer/index metadata",
|
||||||
|
"Node downloader tries exact-shard peers, then tracker/peer file subsets, then HF snapshot_download with allow_patterns — never silently full-repo when layer index is available",
|
||||||
|
"Two-machine test: machine B receives only its assigned range from machine A — nothing fetched from HF",
|
||||||
|
"Machine B resident memory scales with its shard size, not model size",
|
||||||
|
"Checksums verified end-to-end; corrupted transfer falls back cleanly",
|
||||||
|
"Single-node/full-model flows unchanged",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 44,
|
||||||
|
"status": "in-progress",
|
||||||
|
"notes": "Source issue: docs/issues/44-tracker-shard-source-partial-download.md. Download path and partial LOAD implemented; live two-machine LAN verification remains.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-004",
|
||||||
|
"US-012"
|
||||||
|
],
|
||||||
|
"completionNotes": "Tracker models-dir indexing, layer-scoped tar stream, HF allow_patterns client-side from remote index, per-file download API with retries, symlink dereference in tar writers. Partial LOAD via init_empty_weights + layer-scoped safetensors materialization; memory-scaling and checksum fallback tests pass. Remaining: live two-machine test (machine B receives only assigned files from A, no HF)."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-045",
|
||||||
|
"title": "45 — Dual-rate billing: separate input and output token prices",
|
||||||
|
"description": "Ledger has one price_per_1k_tokens and stream vs non-stream paths disagree on input vs output counting. Charge both input and output tokens at separate rates per model; HF pricing refresher applies 80% of each marketplace side separately.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Streamed and non-streamed requests for the same exchange bill the same split (input charged in both)",
|
||||||
|
"A model with asymmetric provider rates bills input and output differently; usage_for / billing events expose the split",
|
||||||
|
"Old persisted billing events replay byte-identically (balances unchanged)",
|
||||||
|
"HF refresh sets both rates from the marketplace row, not the average",
|
||||||
|
"Spend cap (--max-charge-per-request) uses the dual rates",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 45,
|
||||||
|
"status": "in-progress",
|
||||||
|
"notes": "Source issue: docs/issues/45-dual-rate-billing.md. Billing correctness before friends test.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-031"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-046",
|
||||||
|
"title": "46 — Tracker .env awareness + first-node auto-join bootstrap",
|
||||||
|
"description": "Fresh trackers return 503 on auto-join because deployability ignores the joining caller's hardware, and meshnet-tracker ignores .env MESHNET_DOWNLOAD_DIR. Fix empty-registry bootstrap, tracker env loading parity with node CLI, models-dir fallback chain, and tar dereference for HF symlink snapshots.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Fresh tracker (empty registry) + caller with enough memory for a recommended preset → /v1/network/assign returns 200 with model_sources populated when tracker holds a local snapshot",
|
||||||
|
"Fresh tracker + caller too small for any recommended preset → still 503",
|
||||||
|
"meshnet-tracker start in a directory with .env setting MESHNET_DOWNLOAD_DIR serves /v1/model-files/download from that dir with no extra flags",
|
||||||
|
"Explicit --models-dir and MESHNET_MODELS_DIR still take precedence",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 46,
|
||||||
|
"status": "needs-review",
|
||||||
|
"notes": "Source issue: docs/issues/46-tracker-env-and-first-node-autojoin.md. Verified live 2026-07-06.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-044"
|
||||||
|
],
|
||||||
|
"completionNotes": "Empty-registry synthesizes caller as candidate node; tracker loads .env; models_dir falls back MESHNET_MODELS_DIR → MESHNET_DOWNLOAD_DIR; tar dereference=True. Pytest passes aside from known port-7000 env conflict."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-047",
|
||||||
|
"title": "47 — Tracker-first model downloads: visibility, sane timeouts, RAM-based sizing",
|
||||||
|
"description": "Explicit --model startup should skip pointless auto-join; tracker/peer sources preferred over HF with visible progress and 30 s socket timeouts; client abort during tar stream logs one line; CPU nodes size shards from RAM not phantom GPU VRAM; per-file downloads for robustness over fragile multi-GB tar streams.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Node started with explicit --model never queries /v1/network/assign and never prints auto-join unavailable",
|
||||||
|
"Tracker/peer model source preferred; HF only when no source, all sources fail, or --tracker-source-disabled",
|
||||||
|
"Tracker-source downloads print progress every 512 MB and print exception + URL on failure",
|
||||||
|
"A ≥2 s read stall no longer aborts a tracker model-source download (30 s socket timeout)",
|
||||||
|
"Client disconnect during /v1/model-files/download logs one line on the tracker, no traceback",
|
||||||
|
"CPU node with big RAM gets a RAM-sized shard: sizing paths ignore VRAM unless device=cuda",
|
||||||
|
"Live two-machine retest: Windows node downloads from tracker at LAN speed with RAM-sized shard assignment",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 47,
|
||||||
|
"status": "in-progress",
|
||||||
|
"notes": "Source issue: docs/issues/47-model-source-download-visibility.md. Engineering largely complete 2026-07-06; live two-machine retest pending.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-044",
|
||||||
|
"US-046"
|
||||||
|
],
|
||||||
|
"completionNotes": "Skip auto-join when model explicit; sequential source try with progress; 30 s model-source timeout; assignment_vram_mb for CPU; per-file /v1/model-files/download with manifest and retries. Remaining: live Windows two-machine retest."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-049",
|
||||||
|
"title": "49 — Mainnet USDT cutover: two-wallet pilot checklist",
|
||||||
|
"description": "Operator runbook to move from Solana devnet + mock-USDT to mainnet + real USDT for a minimal pilot: one client wallet (deposits USDT, pays for inference) and one node wallet (receives batched payouts). Treasury holds USDT float and SOL for fees. TAI deferred per ADR-0025.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Devnet smoke completed: settlement loop pays with --settle-period 60 and mock mint",
|
||||||
|
"python -m pytest tests/test_settlement_loop.py -q passes (includes 24 h / 5 USDT gate tests)",
|
||||||
|
"Mainnet tracker configured: real USDT mint, --starting-credit 0, --devnet-topup 0, --settle-period 86400",
|
||||||
|
"Client wallet deposit credits API key ledger; inference debits balance",
|
||||||
|
"Node wallet receives at least one confirmed mainnet USDT payout batch",
|
||||||
|
"Sub-threshold pending not paid before 24 h; ≥5 USDT pending triggers immediate payout"
|
||||||
|
],
|
||||||
|
"priority": 49,
|
||||||
|
"status": "open",
|
||||||
|
"notes": "Source issue: docs/issues/49-mainnet-usdt-cutover-two-wallet-pilot.md. ADR-0025 covers optional TAI mint; not blocking this pilot.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-032",
|
||||||
|
"US-033",
|
||||||
|
"US-039"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-048",
|
||||||
|
"title": "48 — Memory budget, shard slots, and dropout relocation hardening",
|
||||||
|
"description": "Harden the capacity contract around US-013 coverage-first assignment: enforce memory budget and max_loaded_shards in rebalance/dropout relocation, and decide whether one node may host multiple concurrent shard backends or max_loaded_shards remains metadata until runtime support lands.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"Assignment/rebalance never exceeds memory budget or max_loaded_shards",
|
||||||
|
"Dropout test restores full coverage without violating capacity limits",
|
||||||
|
"CLI --memory and --max-shards reflected in registration payload",
|
||||||
|
"python -m pytest tests/test_tracker_routing.py tests/test_node_startup.py passes"
|
||||||
|
],
|
||||||
|
"priority": 48,
|
||||||
|
"status": "open",
|
||||||
|
"notes": "Source issue: docs/issues/48-memory-budget-shard-slots-and-dropout-relocation.md. Renumbered from duplicate slot 20. Enables spare shard slots for ADR-0026 managed placement.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-013"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "US-050",
|
||||||
|
"title": "50 — Qwen3.6-27B demand-driven managed placement",
|
||||||
|
"description": "Offer pinned Qwen/Qwen3.6-27B as a recommended text-only chat model. Valid chat requests prove demand; when spare fleet capacity exists, the tracker assigns managed nodes to reach complete coverage. Pinned startup assignments remain immutable per ADR-0026; NCA admission required before routable.",
|
||||||
|
"acceptanceCriteria": [
|
||||||
|
"First valid request for an uncovered variant queues demand and returns 503 model_loading until complete validated coverage exists",
|
||||||
|
"Managed assignments use only spare capacity and carry managed: true",
|
||||||
|
"Pinned startup assignments are never silently retargeted",
|
||||||
|
"Optional quantization field (bfloat16/int8/nf4) with coverage-vote UI semantics per scratch PRD",
|
||||||
|
"python -m pytest passes from repo root"
|
||||||
|
],
|
||||||
|
"priority": 50,
|
||||||
|
"status": "in-design",
|
||||||
|
"notes": "Source: .scratch/qwen3.6-27b-demand-placement/PRD.md and docs/issues/50-qwen3.6-27b-demand-placement.md. Reconciled with ADR-0026 and ADR-0023.",
|
||||||
|
"dependsOn": [
|
||||||
|
"US-035",
|
||||||
|
"US-048"
|
||||||
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"updatedAt": "2026-07-01T00:00:00.000Z",
|
"updatedAt": "2026-07-13T17:00:00.000Z",
|
||||||
"statusVocabulary": {
|
"statusVocabulary": {
|
||||||
"open": "Not started",
|
"open": "Not started",
|
||||||
"in-design": "Decisions pending before implementation can begin",
|
"in-design": "Decisions pending before implementation can begin",
|
||||||
|
|||||||
@@ -36,6 +36,12 @@ def _load_env_file(path: Path) -> None:
|
|||||||
os.environ[key] = value
|
os.environ[key] = value
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_relay_concurrency_flag(value: int | None) -> None:
|
||||||
|
"""Expose relay bridge worker cap via CLI (env MESHNET_RELAY_CONCURRENCY)."""
|
||||||
|
if value is not None:
|
||||||
|
os.environ["MESHNET_RELAY_CONCURRENCY"] = str(max(1, value))
|
||||||
|
|
||||||
|
|
||||||
def _load_env_defaults() -> None:
|
def _load_env_defaults() -> None:
|
||||||
"""Load machine-specific, local, and user-level node env defaults."""
|
"""Load machine-specific, local, and user-level node env defaults."""
|
||||||
machine = socket.gethostname().strip()
|
machine = socket.gethostname().strip()
|
||||||
@@ -189,6 +195,8 @@ def _cmd_default(args) -> int:
|
|||||||
if getattr(args, "cpu", False):
|
if getattr(args, "cpu", False):
|
||||||
overrides["force_cpu"] = True
|
overrides["force_cpu"] = True
|
||||||
|
|
||||||
|
_apply_relay_concurrency_flag(getattr(args, "relay_concurrency", None))
|
||||||
|
|
||||||
if overrides:
|
if overrides:
|
||||||
cfg = merge_cli_overrides(cfg, **overrides)
|
cfg = merge_cli_overrides(cfg, **overrides)
|
||||||
|
|
||||||
@@ -349,6 +357,8 @@ def _cmd_start(args) -> int:
|
|||||||
if getattr(args, "node_name", None):
|
if getattr(args, "node_name", None):
|
||||||
cfg["node_name"] = args.node_name
|
cfg["node_name"] = args.node_name
|
||||||
|
|
||||||
|
_apply_relay_concurrency_flag(getattr(args, "relay_concurrency", None))
|
||||||
|
|
||||||
# Legacy start: just run without the dashboard (keep original blocking loop)
|
# Legacy start: just run without the dashboard (keep original blocking loop)
|
||||||
from .startup import run_startup
|
from .startup import run_startup
|
||||||
|
|
||||||
@@ -433,6 +443,8 @@ def main() -> None:
|
|||||||
help="Set PyTorch inter-op CPU worker threads")
|
help="Set PyTorch inter-op CPU worker threads")
|
||||||
parser.add_argument("--cpu", action="store_true",
|
parser.add_argument("--cpu", action="store_true",
|
||||||
help="Force CPU inference even when a GPU is available")
|
help="Force CPU inference even when a GPU is available")
|
||||||
|
parser.add_argument("--relay-concurrency", type=int, metavar="N",
|
||||||
|
help="Max concurrent relay-http-request workers (env MESHNET_RELAY_CONCURRENCY)")
|
||||||
parser.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
|
parser.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
|
||||||
parser.add_argument("--no-tui", action="store_true", help="Plain-text output (no rich dashboard)")
|
parser.add_argument("--no-tui", action="store_true", help="Plain-text output (no rich dashboard)")
|
||||||
parser.add_argument("--compact", action="store_true", help="Single-line status output")
|
parser.add_argument("--compact", action="store_true", help="Single-line status output")
|
||||||
@@ -510,6 +522,8 @@ def main() -> None:
|
|||||||
help="Set PyTorch inter-op CPU worker threads")
|
help="Set PyTorch inter-op CPU worker threads")
|
||||||
start_cmd.add_argument("--cpu", action="store_true",
|
start_cmd.add_argument("--cpu", action="store_true",
|
||||||
help="Force CPU inference even when a GPU is available")
|
help="Force CPU inference even when a GPU is available")
|
||||||
|
start_cmd.add_argument("--relay-concurrency", type=int, metavar="N",
|
||||||
|
help="Max concurrent relay-http-request workers (env MESHNET_RELAY_CONCURRENCY)")
|
||||||
start_cmd.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
|
start_cmd.add_argument("--debug", action="store_true", help="Enable verbose node debug logging")
|
||||||
start_cmd.add_argument("--tracker-source-disabled", action="store_true",
|
start_cmd.add_argument("--tracker-source-disabled", action="store_true",
|
||||||
help="Skip tracker/peer model-file sources and download from HuggingFace directly")
|
help="Skip tracker/peer model-file sources and download from HuggingFace directly")
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import time
|
import time
|
||||||
|
|
||||||
@@ -183,6 +184,17 @@ def with_forced_cpu(hw: dict) -> dict:
|
|||||||
return forced
|
return forced
|
||||||
|
|
||||||
|
|
||||||
|
def _with_model_drive(profile: dict) -> dict:
|
||||||
|
"""Attach free space for the default model cache drive to tracker diagnostics."""
|
||||||
|
try:
|
||||||
|
cache_root = os.path.expanduser("~/.cache/meshnet/shards")
|
||||||
|
profile["model_drive_free_bytes"] = shutil.disk_usage(os.path.expanduser("~")).free
|
||||||
|
profile["model_drive_path"] = cache_root
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
return profile
|
||||||
|
|
||||||
|
|
||||||
def detect_hardware() -> dict:
|
def detect_hardware() -> dict:
|
||||||
"""Detect GPU model and available VRAM. Returns hardware profile dict."""
|
"""Detect GPU model and available VRAM. Returns hardware profile dict."""
|
||||||
ram_mb = _detect_ram_mb()
|
ram_mb = _detect_ram_mb()
|
||||||
@@ -208,23 +220,23 @@ def detect_hardware() -> dict:
|
|||||||
}
|
}
|
||||||
if torch_gpu is not None and torch_gpu.get("gcn_arch"):
|
if torch_gpu is not None and torch_gpu.get("gcn_arch"):
|
||||||
profile["gcn_arch"] = torch_gpu["gcn_arch"]
|
profile["gcn_arch"] = torch_gpu["gcn_arch"]
|
||||||
return profile
|
return _with_model_drive(profile)
|
||||||
except ImportError:
|
except ImportError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
torch_inventory = _gpu_inventory_profile(torch_gpu, ram_mb)
|
torch_inventory = _gpu_inventory_profile(torch_gpu, ram_mb)
|
||||||
if torch_inventory is not None:
|
if torch_inventory is not None:
|
||||||
return torch_inventory
|
return _with_model_drive(torch_inventory)
|
||||||
|
|
||||||
nvidia_gpu = _gpu_inventory_profile(_detect_nvidia_smi_gpu_memory(), ram_mb)
|
nvidia_gpu = _gpu_inventory_profile(_detect_nvidia_smi_gpu_memory(), ram_mb)
|
||||||
if nvidia_gpu is not None:
|
if nvidia_gpu is not None:
|
||||||
return nvidia_gpu
|
return _with_model_drive(nvidia_gpu)
|
||||||
|
|
||||||
windows_gpu = _gpu_inventory_profile(_detect_windows_gpu_memory(), ram_mb)
|
windows_gpu = _gpu_inventory_profile(_detect_windows_gpu_memory(), ram_mb)
|
||||||
if windows_gpu is not None:
|
if windows_gpu is not None:
|
||||||
return windows_gpu
|
return _with_model_drive(windows_gpu)
|
||||||
|
|
||||||
return {
|
return _with_model_drive({
|
||||||
"device": "cpu",
|
"device": "cpu",
|
||||||
"gpu_name": None,
|
"gpu_name": None,
|
||||||
"vram_mb": 0,
|
"vram_mb": 0,
|
||||||
@@ -232,7 +244,7 @@ def detect_hardware() -> dict:
|
|||||||
"shared_vram_mb": 0,
|
"shared_vram_mb": 0,
|
||||||
"ram_mb": ram_mb,
|
"ram_mb": ram_mb,
|
||||||
"cuda_available": False,
|
"cuda_available": False,
|
||||||
}
|
})
|
||||||
|
|
||||||
|
|
||||||
def benchmark_throughput_checked(device_str: str = "cpu") -> tuple[float, bool, str | None]:
|
def benchmark_throughput_checked(device_str: str = "cpu") -> tuple[float, bool, str | None]:
|
||||||
|
|||||||
@@ -899,12 +899,41 @@ def _load_partial_model_from_snapshot(
|
|||||||
dtype=dtype,
|
dtype=dtype,
|
||||||
)
|
)
|
||||||
|
|
||||||
for module in _active_modules_for_shard(model, shard_start, shard_end):
|
_finalize_active_shard_modules_on_device(model, shard_start, shard_end, device)
|
||||||
if hasattr(module, "to"):
|
|
||||||
module.to(device)
|
|
||||||
return model
|
return model
|
||||||
|
|
||||||
|
|
||||||
|
def _finalize_active_shard_modules_on_device(
|
||||||
|
model: Any, shard_start: int, shard_end: int, device: Any
|
||||||
|
) -> None:
|
||||||
|
"""Place active shard modules on device without copying unmaterialized meta weights."""
|
||||||
|
for module in _active_modules_for_shard(model, shard_start, shard_end):
|
||||||
|
parameters = getattr(module, "parameters", None)
|
||||||
|
if not callable(parameters):
|
||||||
|
if hasattr(module, "to"):
|
||||||
|
module.to(device)
|
||||||
|
continue
|
||||||
|
params = list(parameters(recurse=True))
|
||||||
|
buffers_fn = getattr(module, "buffers", None)
|
||||||
|
buffers = list(buffers_fn(recurse=True)) if callable(buffers_fn) else []
|
||||||
|
tensors = params + buffers
|
||||||
|
if not tensors:
|
||||||
|
if hasattr(module, "to"):
|
||||||
|
module.to(device)
|
||||||
|
continue
|
||||||
|
if all(tensor.device.type == "meta" for tensor in tensors):
|
||||||
|
to_empty = getattr(module, "to_empty", None)
|
||||||
|
if callable(to_empty):
|
||||||
|
to_empty(device)
|
||||||
|
continue
|
||||||
|
if all(tensor.device.type != "meta" for tensor in tensors):
|
||||||
|
if hasattr(module, "to"):
|
||||||
|
module.to(device)
|
||||||
|
continue
|
||||||
|
# Partially materialized: set_module_tensor_to_device already placed loaded
|
||||||
|
# weights on the target device; leave remaining meta parameters untouched.
|
||||||
|
|
||||||
|
|
||||||
def _model_load_plan(
|
def _model_load_plan(
|
||||||
auto_config: Any,
|
auto_config: Any,
|
||||||
model_id: str,
|
model_id: str,
|
||||||
|
|||||||
@@ -44,6 +44,7 @@ class SeamSample:
|
|||||||
cache_mode: CacheMode
|
cache_mode: CacheMode
|
||||||
model_ms: float
|
model_ms: float
|
||||||
encode_ms: float
|
encode_ms: float
|
||||||
|
activation_decode_ms: float
|
||||||
framing_ms: float
|
framing_ms: float
|
||||||
metadata_ms: float
|
metadata_ms: float
|
||||||
copy_allocation_ms: float
|
copy_allocation_ms: float
|
||||||
@@ -52,6 +53,7 @@ class SeamSample:
|
|||||||
decompression_ms: float
|
decompression_ms: float
|
||||||
connection_setup_ms: float
|
connection_setup_ms: float
|
||||||
queue_wait_ms: float
|
queue_wait_ms: float
|
||||||
|
local_http_forwarding_ms: float
|
||||||
transport_ms: float
|
transport_ms: float
|
||||||
seam_latency_ms: float
|
seam_latency_ms: float
|
||||||
payload_bytes: int
|
payload_bytes: int
|
||||||
@@ -120,6 +122,10 @@ def _summary(samples: list[SeamSample]) -> dict[str, float | int]:
|
|||||||
"compression_cpu_ms": round(
|
"compression_cpu_ms": round(
|
||||||
sum(sample.compression_ms + sample.decompression_ms for sample in samples), 4
|
sum(sample.compression_ms + sample.decompression_ms for sample in samples), 4
|
||||||
),
|
),
|
||||||
|
"model_execution_ms": round(sum(sample.model_ms for sample in samples), 4),
|
||||||
|
"activation_encoding_ms": round(sum(sample.encode_ms for sample in samples), 4),
|
||||||
|
"activation_decoding_ms": round(sum(sample.activation_decode_ms for sample in samples), 4),
|
||||||
|
"local_http_forwarding_ms": round(sum(sample.local_http_forwarding_ms for sample in samples), 4),
|
||||||
"peak_buffered_bytes": max((sample.copy_allocation_bytes for sample in samples), default=0),
|
"peak_buffered_bytes": max((sample.copy_allocation_bytes for sample in samples), default=0),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -159,6 +165,7 @@ class _StubTransport:
|
|||||||
queue_wait_ms = 0.0 if self.mode == "direct" else 0.18 + (0.05 if token_index is not None and token_index % 2 else 0.0)
|
queue_wait_ms = 0.0 if self.mode == "direct" else 0.18 + (0.05 if token_index is not None and token_index % 2 else 0.0)
|
||||||
model_ms = 1.6 if phase == "prefill" else 0.45
|
model_ms = 1.6 if phase == "prefill" else 0.45
|
||||||
encode_ms = 0.16 if phase == "prefill" else 0.06
|
encode_ms = 0.16 if phase == "prefill" else 0.06
|
||||||
|
activation_decode_ms = 0.055 if phase == "prefill" else 0.02
|
||||||
# Keep framing/metadata/copy costs explicit rather than hiding them in
|
# Keep framing/metadata/copy costs explicit rather than hiding them in
|
||||||
# serialization or transport time. The stub owns one binary frame and
|
# serialization or transport time. The stub owns one binary frame and
|
||||||
# one response body per hop; no base64 body is modeled.
|
# one response body per hop; no base64 body is modeled.
|
||||||
@@ -168,20 +175,26 @@ class _StubTransport:
|
|||||||
copy_allocation_bytes = wire_bytes + payload_bytes
|
copy_allocation_bytes = wire_bytes + payload_bytes
|
||||||
compression_ms = 0.09 if self.scenario.compression else 0.0
|
compression_ms = 0.09 if self.scenario.compression else 0.0
|
||||||
decompression_ms = 0.07 if self.scenario.compression else 0.0
|
decompression_ms = 0.07 if self.scenario.compression else 0.0
|
||||||
|
# Both routes finish by forwarding the decoded activation to the local
|
||||||
|
# tail-node HTTP handler; relay adds its own queue before that hop.
|
||||||
|
local_http_forwarding_ms = 0.11 if self.mode == "direct" else 0.16
|
||||||
transport_ms = (0.32 if self.mode == "direct" else 0.61) + wire_bytes / 100_000
|
transport_ms = (0.32 if self.mode == "direct" else 0.61) + wire_bytes / 100_000
|
||||||
seam_latency_ms = round(
|
seam_latency_ms = round(
|
||||||
model_ms + encode_ms + framing_ms + metadata_ms + copy_allocation_ms
|
model_ms + encode_ms + activation_decode_ms + framing_ms + metadata_ms + copy_allocation_ms
|
||||||
+ compression_ms + decompression_ms + connection_setup_ms + queue_wait_ms + transport_ms,
|
+ compression_ms + decompression_ms + connection_setup_ms + queue_wait_ms + transport_ms
|
||||||
|
+ local_http_forwarding_ms,
|
||||||
4,
|
4,
|
||||||
)
|
)
|
||||||
return SeamSample(
|
return SeamSample(
|
||||||
phase=phase, token_index=token_index, session_id=self.session_id,
|
phase=phase, token_index=token_index, session_id=self.session_id,
|
||||||
activation_id=f"benchmark-activation-{self._activation_count}", seam="head->tail", mode=self.mode,
|
activation_id=f"benchmark-activation-{self._activation_count}", seam="head->tail", mode=self.mode,
|
||||||
cache_mode=self.cache_mode, model_ms=model_ms, encode_ms=encode_ms,
|
cache_mode=self.cache_mode, model_ms=model_ms, encode_ms=encode_ms,
|
||||||
|
activation_decode_ms=activation_decode_ms,
|
||||||
framing_ms=framing_ms, metadata_ms=metadata_ms,
|
framing_ms=framing_ms, metadata_ms=metadata_ms,
|
||||||
copy_allocation_ms=copy_allocation_ms, copy_allocation_bytes=copy_allocation_bytes,
|
copy_allocation_ms=copy_allocation_ms, copy_allocation_bytes=copy_allocation_bytes,
|
||||||
compression_ms=compression_ms, decompression_ms=decompression_ms,
|
compression_ms=compression_ms, decompression_ms=decompression_ms,
|
||||||
connection_setup_ms=connection_setup_ms, queue_wait_ms=queue_wait_ms,
|
connection_setup_ms=connection_setup_ms, queue_wait_ms=queue_wait_ms,
|
||||||
|
local_http_forwarding_ms=local_http_forwarding_ms,
|
||||||
transport_ms=round(transport_ms, 4), seam_latency_ms=seam_latency_ms,
|
transport_ms=round(transport_ms, 4), seam_latency_ms=seam_latency_ms,
|
||||||
payload_bytes=payload_bytes, wire_bytes=wire_bytes,
|
payload_bytes=payload_bytes, wire_bytes=wire_bytes,
|
||||||
compression_ratio=round(payload_bytes / wire_bytes, 4), connection_attempted=connection_attempted,
|
compression_ratio=round(payload_bytes / wire_bytes, 4), connection_attempted=connection_attempted,
|
||||||
@@ -329,9 +342,10 @@ def run_real_model_lan_benchmark(url: str, *, model: str, timeout: float = 120.0
|
|||||||
sample = SeamSample(
|
sample = SeamSample(
|
||||||
phase="decode", token_index=0, session_id=session_id, activation_id="lan-activation-1",
|
phase="decode", token_index=0, session_id=session_id, activation_id="lan-activation-1",
|
||||||
seam="head->tail", mode="direct", cache_mode="cached", model_ms=0.0, encode_ms=0.0,
|
seam="head->tail", mode="direct", cache_mode="cached", model_ms=0.0, encode_ms=0.0,
|
||||||
|
activation_decode_ms=0.0,
|
||||||
framing_ms=0.0, metadata_ms=0.0, copy_allocation_ms=0.0, copy_allocation_bytes=0,
|
framing_ms=0.0, metadata_ms=0.0, copy_allocation_ms=0.0, copy_allocation_bytes=0,
|
||||||
compression_ms=0.0, decompression_ms=0.0, connection_setup_ms=elapsed_ms,
|
compression_ms=0.0, decompression_ms=0.0, connection_setup_ms=elapsed_ms,
|
||||||
queue_wait_ms=0.0, transport_ms=elapsed_ms, seam_latency_ms=elapsed_ms,
|
queue_wait_ms=0.0, local_http_forwarding_ms=0.0, transport_ms=elapsed_ms, seam_latency_ms=elapsed_ms,
|
||||||
payload_bytes=len(body), wire_bytes=len(body) + len(response_body), compression_ratio=1.0,
|
payload_bytes=len(body), wire_bytes=len(body) + len(response_body), compression_ratio=1.0,
|
||||||
connection_attempted=True,
|
connection_attempted=True,
|
||||||
)
|
)
|
||||||
@@ -354,6 +368,10 @@ def format_summary(report: dict) -> str:
|
|||||||
f"{decode['tokens_per_sec']:.1f} tok/s; {decode['bytes_per_token']:.0f} B/tok; "
|
f"{decode['tokens_per_sec']:.1f} tok/s; {decode['bytes_per_token']:.0f} B/tok; "
|
||||||
f"seam {seam['payload_bytes']}/{seam['wire_bytes']} B "
|
f"seam {seam['payload_bytes']}/{seam['wire_bytes']} B "
|
||||||
f"({seam['compression_ratio']:.2f}x); connections {run['connections']['attempts']}; "
|
f"({seam['compression_ratio']:.2f}x); connections {run['connections']['attempts']}; "
|
||||||
|
f"model/encode/decode {decode['model_execution_ms']:.2f}/"
|
||||||
|
f"{decode['activation_encoding_ms']:.2f}/{decode['activation_decoding_ms']:.2f} ms; "
|
||||||
|
f"compression {decode['compression_cpu_ms']:.2f} ms; "
|
||||||
|
f"HTTP {decode['local_http_forwarding_ms']:.2f} ms; "
|
||||||
f"queue p95 {decode['p95_queue_wait_ms']:.2f} ms"
|
f"queue p95 {decode['p95_queue_wait_ms']:.2f} ms"
|
||||||
)
|
)
|
||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|||||||
@@ -1545,6 +1545,26 @@ class TorchNodeServer:
|
|||||||
|
|
||||||
def apply_tracker_directives(self, directives: list[dict]) -> dict | None:
|
def apply_tracker_directives(self, directives: list[dict]) -> dict | None:
|
||||||
"""Apply tracker shard directives (LOAD_SHARD replace, ADD_SHARD load-more)."""
|
"""Apply tracker shard directives (LOAD_SHARD replace, ADD_SHARD load-more)."""
|
||||||
|
drop_directive = next(
|
||||||
|
(directive for directive in reversed(directives) if directive.get("action") == "DROP_SHARD"),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if drop_directive is not None:
|
||||||
|
model_id = str(drop_directive.get("model") or "")
|
||||||
|
removed = self._backends.pop(model_id, None)
|
||||||
|
if removed is None:
|
||||||
|
return None
|
||||||
|
if self._backends:
|
||||||
|
self._backend = next(iter(self._backends.values()))
|
||||||
|
self._tracker_mode = self._backend.shard_start == 0
|
||||||
|
else:
|
||||||
|
self._backend = None
|
||||||
|
self._tracker_mode = False
|
||||||
|
if self._server is not None:
|
||||||
|
self._server.backends = dict(self._backends)
|
||||||
|
self._server.backend = self._backend
|
||||||
|
self._server.tracker_mode = self._tracker_mode
|
||||||
|
return {"action": "DROP_SHARD", "model": model_id}
|
||||||
add_directive = next(
|
add_directive = next(
|
||||||
(directive for directive in reversed(directives) if directive.get("action") == "ADD_SHARD"),
|
(directive for directive in reversed(directives) if directive.get("action") == "ADD_SHARD"),
|
||||||
None,
|
None,
|
||||||
@@ -1574,6 +1594,8 @@ class TorchNodeServer:
|
|||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
|
if replacing:
|
||||||
|
self._backends.clear()
|
||||||
new_backend = _load_backend(model_id, shard_start, shard_end, quantization, self._cache_dir)
|
new_backend = _load_backend(model_id, shard_start, shard_end, quantization, self._cache_dir)
|
||||||
except TypeError:
|
except TypeError:
|
||||||
new_backend = _load_backend(model_id, shard_start, shard_end, quantization)
|
new_backend = _load_backend(model_id, shard_start, shard_end, quantization)
|
||||||
|
|||||||
@@ -21,6 +21,9 @@ import uuid
|
|||||||
DEFAULT_PRICE_PER_1K_TOKENS = 0.02 # USDT
|
DEFAULT_PRICE_PER_1K_TOKENS = 0.02 # USDT
|
||||||
DEFAULT_STARTING_CREDIT = 0.0 # USDT; accounts must be funded before inference
|
DEFAULT_STARTING_CREDIT = 0.0 # USDT; accounts must be funded before inference
|
||||||
DEFAULT_BILLING_DB_PATH = "billing.sqlite"
|
DEFAULT_BILLING_DB_PATH = "billing.sqlite"
|
||||||
|
DEFAULT_SETTLE_PERIOD = 86400.0 # seconds — max time between node payouts (24 h prod)
|
||||||
|
DEFAULT_PAYOUT_THRESHOLD = 5.0 # USDT — immediate payout when pending exceeds this
|
||||||
|
DEFAULT_PAYOUT_DUST_FLOOR = 0.01 # USDT — never pay less than this
|
||||||
NODE_REVENUE_SHARE = 0.90 # nodes 90% / protocol cut 10% (ADR-0015)
|
NODE_REVENUE_SHARE = 0.90 # nodes 90% / protocol cut 10% (ADR-0015)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,12 @@ import time
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from .accounts import DEFAULT_ACCOUNTS_DB_PATH
|
from .accounts import DEFAULT_ACCOUNTS_DB_PATH
|
||||||
from .billing import DEFAULT_BILLING_DB_PATH
|
from .billing import (
|
||||||
|
DEFAULT_BILLING_DB_PATH,
|
||||||
|
DEFAULT_PAYOUT_DUST_FLOOR,
|
||||||
|
DEFAULT_PAYOUT_THRESHOLD,
|
||||||
|
DEFAULT_SETTLE_PERIOD,
|
||||||
|
)
|
||||||
from .capability import ALL_POLICIES as ALL_CAPABILITY_POLICIES
|
from .capability import ALL_POLICIES as ALL_CAPABILITY_POLICIES
|
||||||
from .hf_pricing import DEFAULT_HF_PRICING_LOG_DB_PATH
|
from .hf_pricing import DEFAULT_HF_PRICING_LOG_DB_PATH
|
||||||
from .logging_setup import (
|
from .logging_setup import (
|
||||||
@@ -237,19 +242,19 @@ def main() -> None:
|
|||||||
common.add_argument(
|
common.add_argument(
|
||||||
"--settle-period",
|
"--settle-period",
|
||||||
type=float,
|
type=float,
|
||||||
default=86400.0,
|
default=DEFAULT_SETTLE_PERIOD,
|
||||||
help="Max seconds between payouts to a node (dev: 60, prod: 86400)",
|
help="Max seconds between payouts to a node (dev: 60, prod: 86400)",
|
||||||
)
|
)
|
||||||
common.add_argument(
|
common.add_argument(
|
||||||
"--payout-threshold",
|
"--payout-threshold",
|
||||||
type=float,
|
type=float,
|
||||||
default=5.0,
|
default=DEFAULT_PAYOUT_THRESHOLD,
|
||||||
help="Pending USDT that triggers an immediate payout (dev: 0)",
|
help="Pending USDT that triggers an immediate payout (dev: 0)",
|
||||||
)
|
)
|
||||||
common.add_argument(
|
common.add_argument(
|
||||||
"--payout-dust-floor",
|
"--payout-dust-floor",
|
||||||
type=float,
|
type=float,
|
||||||
default=0.01,
|
default=DEFAULT_PAYOUT_DUST_FLOOR,
|
||||||
help="Never pay out less than this many USDT",
|
help="Never pay out less than this many USDT",
|
||||||
)
|
)
|
||||||
common.add_argument(
|
common.add_argument(
|
||||||
|
|||||||
@@ -22,8 +22,9 @@
|
|||||||
border-bottom:1px solid var(--border); flex-shrink:0; }
|
border-bottom:1px solid var(--border); flex-shrink:0; }
|
||||||
header h1 { font-size:16px; margin:0; color:var(--accent); }
|
header h1 { font-size:16px; margin:0; color:var(--accent); }
|
||||||
header .meta { color:var(--dim); font-size:12px; }
|
header .meta { color:var(--dim); font-size:12px; }
|
||||||
main { display:grid; grid-template-columns:repeat(auto-fit,minmax(340px,1fr));
|
main { display:grid; grid-template-columns:1fr;
|
||||||
gap:14px; padding:14px 20px; }
|
gap:14px; padding:14px 20px; }
|
||||||
|
main > section { width:100%; min-width:0; }
|
||||||
body.chat-tab-active main {
|
body.chat-tab-active main {
|
||||||
flex:1; min-height:0; display:flex; flex-direction:column;
|
flex:1; min-height:0; display:flex; flex-direction:column;
|
||||||
padding:0; gap:0; overflow:hidden;
|
padding:0; gap:0; overflow:hidden;
|
||||||
@@ -32,6 +33,7 @@
|
|||||||
border-radius:8px; padding:12px 14px; min-height:80px; }
|
border-radius:8px; padding:12px 14px; min-height:80px; }
|
||||||
section h2 { margin:0 0 8px; font-size:12px; text-transform:uppercase;
|
section h2 { margin:0 0 8px; font-size:12px; text-transform:uppercase;
|
||||||
letter-spacing:.08em; color:var(--dim); }
|
letter-spacing:.08em; color:var(--dim); }
|
||||||
|
.panel-heading { display:flex; align-items:center; justify-content:space-between; gap:8px; }
|
||||||
table { width:100%; border-collapse:collapse; font-size:12px; }
|
table { width:100%; border-collapse:collapse; font-size:12px; }
|
||||||
th { text-align:left; color:var(--dim); font-weight:normal;
|
th { text-align:left; color:var(--dim); font-weight:normal;
|
||||||
border-bottom:1px solid var(--border); padding:2px 6px 4px 0; }
|
border-bottom:1px solid var(--border); padding:2px 6px 4px 0; }
|
||||||
@@ -42,12 +44,15 @@
|
|||||||
.empty { color:var(--dim); font-style:italic; }
|
.empty { color:var(--dim); font-style:italic; }
|
||||||
.pill { display:inline-block; padding:0 7px; border-radius:9px;
|
.pill { display:inline-block; padding:0 7px; border-radius:9px;
|
||||||
border:1px solid var(--border); font-size:11px; }
|
border:1px solid var(--border); font-size:11px; }
|
||||||
input, button { font:inherit; color:var(--fg); background:var(--bg);
|
input, button, select { font:inherit; color:var(--fg); background:var(--bg);
|
||||||
border:1px solid var(--border); border-radius:6px; padding:5px 8px; }
|
border:1px solid var(--border); border-radius:6px; padding:5px 8px; }
|
||||||
input { width:100%; margin-bottom:6px; }
|
input { width:100%; margin-bottom:6px; }
|
||||||
button { cursor:pointer; color:var(--accent); }
|
button { cursor:pointer; color:var(--accent); }
|
||||||
button:hover { border-color:var(--accent); }
|
button:hover { border-color:var(--accent); }
|
||||||
button.small { font-size:11px; padding:1px 7px; }
|
button.small { font-size:11px; padding:1px 7px; }
|
||||||
|
dialog { color:var(--fg); background:var(--panel); border:1px solid var(--border); border-radius:8px; min-width:min(420px,calc(100vw - 32px)); }
|
||||||
|
dialog::backdrop { background:rgba(0,0,0,.55); }
|
||||||
|
.placement-dialog-actions { display:flex; justify-content:flex-end; gap:8px; margin-top:12px; }
|
||||||
.form-row { display:flex; gap:8px; }
|
.form-row { display:flex; gap:8px; }
|
||||||
.form-row button { white-space:nowrap; }
|
.form-row button { white-space:nowrap; }
|
||||||
.error-msg { color:var(--bad); font-size:12px; min-height:16px; }
|
.error-msg { color:var(--bad); font-size:12px; min-height:16px; }
|
||||||
@@ -70,6 +75,12 @@
|
|||||||
background:transparent; color:var(--dim); padding:5px 0 8px; }
|
background:transparent; color:var(--dim); padding:5px 0 8px; }
|
||||||
.dashboard-tabs button.active { color:var(--accent); border-bottom-color:var(--accent); }
|
.dashboard-tabs button.active { color:var(--accent); border-bottom-color:var(--accent); }
|
||||||
.wide { grid-column:1 / -1; }
|
.wide { grid-column:1 / -1; }
|
||||||
|
/* Compact status cards fan out on desktop; tables remain readable at half width. */
|
||||||
|
@media (min-width:900px) {
|
||||||
|
main { grid-template-columns:repeat(4,minmax(0,1fr)); }
|
||||||
|
main > section { grid-column:span 1; }
|
||||||
|
.wide { grid-column:span 2; }
|
||||||
|
}
|
||||||
section[hidden] { display:none !important; }
|
section[hidden] { display:none !important; }
|
||||||
section.chat-section {
|
section.chat-section {
|
||||||
padding:0; border:0; border-radius:0; background:var(--bg); min-height:0;
|
padding:0; border:0; border-radius:0; background:var(--bg); min-height:0;
|
||||||
@@ -204,7 +215,7 @@
|
|||||||
.chat-compose button:disabled { opacity:.45; cursor:not-allowed; }
|
.chat-compose button:disabled { opacity:.45; cursor:not-allowed; }
|
||||||
.console {
|
.console {
|
||||||
background:var(--bg); border:1px solid var(--border); border-radius:6px;
|
background:var(--bg); border:1px solid var(--border); border-radius:6px;
|
||||||
min-height:160px; max-height:280px; overflow:auto; padding:7px 9px;
|
min-height:160px; max-height:520px; overflow-y:auto; overflow-x:auto; padding:7px 9px;
|
||||||
white-space:pre-wrap; word-break:break-word; font-size:11px;
|
white-space:pre-wrap; word-break:break-word; font-size:11px;
|
||||||
}
|
}
|
||||||
.console-line { padding:1px 0; border-bottom:1px solid #161b22; }
|
.console-line { padding:1px 0; border-bottom:1px solid #161b22; }
|
||||||
@@ -247,8 +258,7 @@
|
|||||||
</nav>
|
</nav>
|
||||||
<main>
|
<main>
|
||||||
<section data-tab="overview" id="account-section"><h2>Account</h2><div id="account">loading…</div></section>
|
<section data-tab="overview" id="account-section"><h2>Account</h2><div id="account">loading…</div></section>
|
||||||
<section data-tab="overview"><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
|
<section data-tab="overview" class="wide"><h2>Nodes & coverage</h2><div id="nodes" class="empty">loading…</div></section>
|
||||||
<section data-tab="overview"><h2>Nodes & coverage</h2><div id="nodes" class="empty">loading…</div></section>
|
|
||||||
<section data-tab="overview"><h2>Model usage (RPM)</h2><div id="stats" class="empty">loading…</div></section>
|
<section data-tab="overview"><h2>Model usage (RPM)</h2><div id="stats" class="empty">loading…</div></section>
|
||||||
<section data-tab="overview" class="wide"><h2>Routing (learned)</h2><div id="routing" class="empty">loading…</div></section>
|
<section data-tab="overview" class="wide"><h2>Routing (learned)</h2><div id="routing" class="empty">loading…</div></section>
|
||||||
<section data-tab="overview" class="wide"><h2>Model inference speed</h2><div id="model-speed-chart" class="empty">loading…</div></section>
|
<section data-tab="overview" class="wide"><h2>Model inference speed</h2><div id="model-speed-chart" class="empty">loading…</div></section>
|
||||||
@@ -287,10 +297,13 @@
|
|||||||
<section data-tab="billing"><h2>Request history</h2><div id="billing-usage" class="empty">login required</div></section>
|
<section data-tab="billing"><h2>Request history</h2><div id="billing-usage" class="empty">login required</div></section>
|
||||||
<section data-tab="billing" data-admin-only><h2>Node pending payouts</h2><div id="pending" class="empty">admin login required</div></section>
|
<section data-tab="billing" data-admin-only><h2>Node pending payouts</h2><div id="pending" class="empty">admin login required</div></section>
|
||||||
<section data-tab="billing" data-admin-only><h2>Settlement history</h2><div id="settlements" class="empty">admin login required</div></section>
|
<section data-tab="billing" data-admin-only><h2>Settlement history</h2><div id="settlements" class="empty">admin login required</div></section>
|
||||||
|
<section data-tab="admin"><h2>Tracker hive</h2><div id="hive" class="empty">loading…</div></section>
|
||||||
|
<section data-tab="admin" class="wide"><h2>Model placement</h2><div id="admin-model-placement-status" class="dim">Choose a model to load or release.</div><div id="admin-model-placement" class="empty">admin login required</div></section>
|
||||||
|
<section data-tab="admin" class="wide"><h2>Total node pool</h2><div id="admin-node-pool" class="empty">admin login required</div></section>
|
||||||
<section data-tab="admin" id="admin-section"><h2>All accounts (admin)</h2><div id="admin" class="empty"></div></section>
|
<section data-tab="admin" id="admin-section"><h2>All accounts (admin)</h2><div id="admin" class="empty"></div></section>
|
||||||
<section data-tab="admin" data-admin-only><h2>Strikes / bans / forfeitures</h2><div id="fraud" class="empty">admin login required</div></section>
|
<section data-tab="admin" data-admin-only><h2>Strikes / bans / forfeitures</h2><div id="fraud" class="empty">admin login required</div></section>
|
||||||
<section data-tab="admin"><h2>Client balances</h2><div id="clients" class="empty">admin login required</div></section>
|
<section data-tab="admin"><h2>Client balances</h2><div id="clients" class="empty">admin login required</div></section>
|
||||||
<section data-tab="admin" class="wide"><h2>Console output</h2><div id="console" class="console empty">admin login required</div></section>
|
<section data-tab="admin" class="wide"><h2 class="panel-heading">Console output <button class="small" type="button" onclick="clearConsole()">clear</button></h2><div id="console" class="console empty">admin login required</div></section>
|
||||||
<section data-tab="testing" data-admin-only>
|
<section data-tab="testing" data-admin-only>
|
||||||
<h2>Test run status</h2>
|
<h2>Test run status</h2>
|
||||||
<div id="testing-error" class="testing-error" hidden></div>
|
<div id="testing-error" class="testing-error" hidden></div>
|
||||||
@@ -314,6 +327,14 @@
|
|||||||
<div id="testing-log" class="console empty">no test output yet</div>
|
<div id="testing-log" class="console empty">no test output yet</div>
|
||||||
</section>
|
</section>
|
||||||
</main>
|
</main>
|
||||||
|
<dialog id="model-placement-dialog">
|
||||||
|
<form method="dialog">
|
||||||
|
<div id="model-placement-dialog-title"></div>
|
||||||
|
<label for="model-placement-node">Node</label>
|
||||||
|
<select id="model-placement-node"></select>
|
||||||
|
<div class="placement-dialog-actions"><button value="cancel">Cancel</button><button value="confirm">Confirm</button></div>
|
||||||
|
</form>
|
||||||
|
</dialog>
|
||||||
<script>
|
<script>
|
||||||
"use strict";
|
"use strict";
|
||||||
const $ = id => document.getElementById(id);
|
const $ = id => document.getElementById(id);
|
||||||
@@ -524,7 +545,67 @@ function renderHive(raft) {
|
|||||||
(raft.peers ? `<br>peers: ${esc(Array.isArray(raft.peers) ? raft.peers.join(", ") : raft.peers)}` : "");
|
(raft.peers ? `<br>peers: ${esc(Array.isArray(raft.peers) ? raft.peers.join(", ") : raft.peers)}` : "");
|
||||||
}
|
}
|
||||||
|
|
||||||
function renderNodes(map) {
|
function nodeLiveTps(node) {
|
||||||
|
const reqs = (node.stats && node.stats.current_requests) || [];
|
||||||
|
let best = null;
|
||||||
|
for (const req of reqs) {
|
||||||
|
const rate = req.tokens_per_sec;
|
||||||
|
if (rate == null) continue;
|
||||||
|
if (best === null || rate > best) best = rate;
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildModelPriceMap(modelsResp) {
|
||||||
|
const prices = new Map();
|
||||||
|
for (const model of (modelsResp && modelsResp.data) || []) {
|
||||||
|
if (!model.pricing) continue;
|
||||||
|
for (const key of modelLookupKeys(model)) {
|
||||||
|
prices.set(key, model.pricing);
|
||||||
|
prices.set(modelAliasKey(key), model.pricing);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return prices;
|
||||||
|
}
|
||||||
|
|
||||||
|
function lookupModelPricing(modelKey, aliasMap, priceMap) {
|
||||||
|
const candidates = [modelKey];
|
||||||
|
const alias = aliasMap.get(modelAliasKey(modelKey));
|
||||||
|
if (alias) candidates.push(alias);
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
const hit = priceMap.get(candidate) || priceMap.get(modelAliasKey(candidate));
|
||||||
|
if (hit) return hit;
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatInferencePrice(pricing) {
|
||||||
|
if (!pricing) return '<span class="dim">—</span>';
|
||||||
|
const input = pricing.input_per_1k_usdt;
|
||||||
|
const output = pricing.output_per_1k_usdt;
|
||||||
|
if (input == null && output == null) return '<span class="dim">—</span>';
|
||||||
|
if (input === output || output == null) {
|
||||||
|
return `<span class="num">${usdt(input ?? output)}/1k</span>`;
|
||||||
|
}
|
||||||
|
return `<span class="num">${usdt(input)}/${usdt(output)}</span> <span class="dim">in/out per 1k</span>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function modelGroupTypicalTps(modelKey, aliasMap, routing) {
|
||||||
|
const sampleNode = { hf_repo: modelKey, model: modelKey };
|
||||||
|
const resolved = resolveModelGroup(sampleNode, aliasMap);
|
||||||
|
const model = {
|
||||||
|
id: resolved,
|
||||||
|
name: resolved,
|
||||||
|
hf_repo: modelKey.includes("/") ? modelKey : null,
|
||||||
|
aliases: [modelKey, resolved].filter(Boolean),
|
||||||
|
};
|
||||||
|
return chatModelTypicalTps(model, routing);
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderNodes(payload) {
|
||||||
|
const map = payload && payload.map;
|
||||||
|
const routing = payload && payload.routing;
|
||||||
|
const priceMap = payload && payload.priceMap;
|
||||||
const nodes = (map && map.nodes) || [];
|
const nodes = (map && map.nodes) || [];
|
||||||
if (!nodes.length) {
|
if (!nodes.length) {
|
||||||
$("nodes").innerHTML = '<div class="empty">no nodes registered</div>'; return;
|
$("nodes").innerHTML = '<div class="empty">no nodes registered</div>'; return;
|
||||||
@@ -543,16 +624,22 @@ function renderNodes(map) {
|
|||||||
.map(n => n.model_supply && n.model_supply.served_model_copies)
|
.map(n => n.model_supply && n.model_supply.served_model_copies)
|
||||||
.filter(v => v !== null && v !== undefined);
|
.filter(v => v !== null && v !== undefined);
|
||||||
const served = servedValues.length ? Math.max(...servedValues) : undefined;
|
const served = servedValues.length ? Math.max(...servedValues) : undefined;
|
||||||
html += `<div><b>${esc(model)}</b> <span class="dim">(${group.length} node${group.length===1?"":"s"} · ${esc(copies(served))} served)</span></div>`;
|
const typicalTps = modelGroupTypicalTps(model, aliasMap, routing);
|
||||||
html += table(["node", "shard", "tps (1h)", "queue", "served"], group.map(n => {
|
const pricing = lookupModelPricing(model, aliasMap, priceMap);
|
||||||
|
html += `<div style="margin-top:8px"><b>${esc(model)}</b> <span class="dim">(` +
|
||||||
|
`${group.length} node${group.length === 1 ? "" : "s"} · ${esc(copies(served))} served · ` +
|
||||||
|
`${esc(tps(typicalTps))} tok/s · price ${formatInferencePrice(pricing)})</span></div>`;
|
||||||
|
html += table(["node", "shard", "last tps", "tps (1h)", "queue", "served"], group.map(n => {
|
||||||
const modelStats = (n.throughput && (n.throughput[n.hf_repo] || n.throughput[n.model])) || {};
|
const modelStats = (n.throughput && (n.throughput[n.hf_repo] || n.throughput[n.model])) || {};
|
||||||
return [
|
return [
|
||||||
nodeDisplayCell(n),
|
nodeDisplayCell(n),
|
||||||
esc(`${n.shard_start ?? "?"}-${n.shard_end ?? "?"}`),
|
esc(`${n.shard_start ?? "?"}-${n.shard_end ?? "?"}`),
|
||||||
`<span class="num">${esc(tps(modelStats.tokens_per_sec_last_hour))}</span>`,
|
`<span class="num">${esc(tps(nodeLiveTps(n)))}</span>`,
|
||||||
esc(String((n.stats && n.stats.queue_depth) ?? 0)),
|
`<span class="num">${esc(tps(modelStats.tokens_per_sec_last_hour))}</span>`,
|
||||||
`<span class="num">${esc(copies(n.model_supply && n.model_supply.served_model_copies))}</span>`,
|
esc(String((n.stats && n.stats.queue_depth) ?? 0)),
|
||||||
]; }));
|
`<span class="num">${esc(copies(n.model_supply && n.model_supply.served_model_copies))}</span>`,
|
||||||
|
];
|
||||||
|
}));
|
||||||
}
|
}
|
||||||
$("nodes").innerHTML = html;
|
$("nodes").innerHTML = html;
|
||||||
}
|
}
|
||||||
@@ -1006,13 +1093,22 @@ function renderBillingUsage(records) {
|
|||||||
table(["time", "model", "api key", "tokens", "cost (USDT)"], rows);
|
table(["time", "model", "api key", "tokens", "cost (USDT)"], rows);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
let consoleClearedAt = 0;
|
||||||
|
const CONSOLE_MAX_LINES = 1000;
|
||||||
|
|
||||||
|
function clearConsole() {
|
||||||
|
consoleClearedAt = Date.now() / 1000;
|
||||||
|
panelSig.console = null;
|
||||||
|
renderConsole({ events: [] });
|
||||||
|
}
|
||||||
|
|
||||||
function renderConsole(data) {
|
function renderConsole(data) {
|
||||||
const events = (data && data.events) || [];
|
const events = ((data && data.events) || []).filter(event => event.ts > consoleClearedAt);
|
||||||
if (!events.length) {
|
if (!events.length) {
|
||||||
$("console").innerHTML = '<div class="empty">no console events</div>';
|
$("console").innerHTML = '<div class="empty">no console events</div>';
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
$("console").innerHTML = events.slice(-120).map(e => {
|
$("console").innerHTML = events.slice(-CONSOLE_MAX_LINES).map(e => {
|
||||||
const level = String(e.level || "info");
|
const level = String(e.level || "info");
|
||||||
const cls = level === "error" ? "console-level-error" : level === "warn" ? "console-level-warn" : "console-level-info";
|
const cls = level === "error" ? "console-level-error" : level === "warn" ? "console-level-warn" : "console-level-info";
|
||||||
const fields = e.fields && Object.keys(e.fields).length ? " " + JSON.stringify(e.fields) : "";
|
const fields = e.fields && Object.keys(e.fields).length ? " " + JSON.stringify(e.fields) : "";
|
||||||
@@ -1706,7 +1802,7 @@ async function requestSelectedModelLoad() {
|
|||||||
if (!selectedChatModel) return;
|
if (!selectedChatModel) return;
|
||||||
const button = $("request-model-load");
|
const button = $("request-model-load");
|
||||||
if (button) button.disabled = true;
|
if (button) button.disabled = true;
|
||||||
const result = await apiCall("/v1/models/load", "POST", { model: selectedChatModel });
|
const result = await apiCall("/v1/models/load", "POST", { model: selectedChatModel, force: isAdmin });
|
||||||
if (button) button.disabled = false;
|
if (button) button.disabled = false;
|
||||||
if (!result.ok) {
|
if (!result.ok) {
|
||||||
alert(result.data.error || "model load request failed");
|
alert(result.data.error || "model load request failed");
|
||||||
@@ -1716,6 +1812,94 @@ async function requestSelectedModelLoad() {
|
|||||||
$("chat-status").textContent = `load queued on ${short(assignment.node_id || "node")} for layers ${assignment.shard_start}-${assignment.shard_end}`;
|
$("chat-status").textContent = `load queued on ${short(assignment.node_id || "node")} for layers ${assignment.shard_start}-${assignment.shard_end}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function requestAdminModelLoad(model, nodeId) {
|
||||||
|
const result = await apiCall("/v1/models/load", "POST", { model, node_id: nodeId, force: true });
|
||||||
|
if (!result.ok) return showAdminModelPlacementStatus(result.data.error || "model load request failed", true);
|
||||||
|
const assignment = result.data.assignment || {};
|
||||||
|
showAdminModelPlacementStatus(`Load queued on ${short(assignment.node_id || "node")} for ${model}.`);
|
||||||
|
await refreshActiveTab(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function releaseAdminModel(model, nodeId) {
|
||||||
|
const result = await apiCall("/v1/models/release", "POST", { model, node_id: nodeId });
|
||||||
|
if (!result.ok) return showAdminModelPlacementStatus(result.data.error || "model release request failed", true);
|
||||||
|
showAdminModelPlacementStatus(`Release queued for ${result.data.released || 0} node(s) serving ${model}.`);
|
||||||
|
await refreshActiveTab(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
function showAdminModelPlacementStatus(message, isError) {
|
||||||
|
const status = $("admin-model-placement-status");
|
||||||
|
status.textContent = message;
|
||||||
|
status.className = isError ? "bad" : "ok";
|
||||||
|
}
|
||||||
|
|
||||||
|
function gib(bytes) { return bytes == null ? "not reported" : `${(Number(bytes) / 1073741824).toFixed(1)} GiB`; }
|
||||||
|
|
||||||
|
function renderAdminNodePool(map) {
|
||||||
|
const groups = {};
|
||||||
|
for (const node of (map && map.nodes) || []) {
|
||||||
|
const account = node.wallet_address || "unbound account";
|
||||||
|
(groups[account] = groups[account] || []).push(node);
|
||||||
|
}
|
||||||
|
let html = "";
|
||||||
|
for (const [account, nodes] of Object.entries(groups).sort(([a], [b]) => a.localeCompare(b))) {
|
||||||
|
html += `<div style="margin-top:10px"><b>${esc(short(account, 20))}</b> <span class="dim">${nodes.length} node(s)</span></div>`;
|
||||||
|
html += table(["node", "assignment", "state / slots", "RAM", "GPU / VRAM", "model drive"], nodes.map(node => {
|
||||||
|
const hw = node.hardware_profile || {};
|
||||||
|
const cap = node.capacity || {};
|
||||||
|
// The network map keeps reported resource capacity under `capacity`.
|
||||||
|
node.ram_bytes = cap.ram_bytes ?? node.ram_bytes;
|
||||||
|
node.vram_bytes = cap.vram_bytes ?? node.vram_bytes;
|
||||||
|
const disk = hw.model_drive_free_bytes ?? hw.model_path_free_bytes ?? hw.disk_free_bytes;
|
||||||
|
const gpu = hw.gpu_name || (hw.cuda_available ? "CUDA GPU" : "CPU only");
|
||||||
|
return [nodeDisplayCell(node), esc(node.hf_repo || node.model || "unassigned"),
|
||||||
|
esc(`${node.stats?.status || "?"} · ${cap.loaded_slots ?? "?"}/${cap.max_loaded_shards ?? node.max_loaded_shards ?? "?"} slots`),
|
||||||
|
esc(gib(node.ram_bytes || (hw.ram_mb && hw.ram_mb * 1048576))),
|
||||||
|
esc(`${gpu} · ${gib(node.vram_bytes || (hw.vram_mb && hw.vram_mb * 1048576))}`), esc(gib(disk))];
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
$("admin-node-pool").innerHTML = html || '<div class="empty">no nodes registered</div>';
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderAdminModelPlacement(models, map) {
|
||||||
|
const nodes = (map && map.nodes) || [];
|
||||||
|
const rows = ((models && models.data) || []).map(model => {
|
||||||
|
const aliases = new Set([model.id, model.hf_repo, ...(model.aliases || [])].filter(Boolean));
|
||||||
|
const serving = nodes.filter(node => aliases.has(node.model) || aliases.has(node.hf_repo)).length;
|
||||||
|
const downloaded = nodes.filter(node => aliases.has(node.model) || aliases.has(node.hf_repo) ||
|
||||||
|
(node.downloaded_models || []).some(item => aliases.has(item.model) || aliases.has(item.hf_repo))).length;
|
||||||
|
const actions = `<button class="small" data-admin-model-load="${esc(model.id)}">load</button> ` +
|
||||||
|
`<button class="small" data-admin-model-release="${esc(model.id)}"${serving ? "" : " disabled"}>release</button>`;
|
||||||
|
return [esc(model.name || model.id), String(serving), String(downloaded), actions];
|
||||||
|
});
|
||||||
|
$("admin-model-placement").innerHTML = rows.length
|
||||||
|
? table(["model", "serving nodes", "downloaded on nodes", "admin action"], rows)
|
||||||
|
: '<div class="empty">no model presets configured</div>';
|
||||||
|
}
|
||||||
|
|
||||||
|
$("admin-model-placement").addEventListener("click", event => {
|
||||||
|
const load = event.target.closest("[data-admin-model-load]");
|
||||||
|
const release = event.target.closest("[data-admin-model-release]");
|
||||||
|
if (load) void chooseModelPlacementNode("load", load.dataset.adminModelLoad);
|
||||||
|
if (release) void chooseModelPlacementNode("release", release.dataset.adminModelRelease);
|
||||||
|
});
|
||||||
|
|
||||||
|
function chooseModelPlacementNode(action, model) {
|
||||||
|
const dialog = $("model-placement-dialog");
|
||||||
|
const select = $("model-placement-node");
|
||||||
|
const aliases = new Set([model].filter(Boolean));
|
||||||
|
const nodes = (lastNetworkMap?.nodes || []).filter(node => action === "load" || aliases.has(node.model) || aliases.has(node.hf_repo));
|
||||||
|
if (!nodes.length) return showAdminModelPlacementStatus(`No node can ${action} ${model}.`, true);
|
||||||
|
$("model-placement-dialog-title").textContent = `${action === "load" ? "Load" : "Release"} ${model} on a node`;
|
||||||
|
select.innerHTML = nodes.map(node => `<option value="${esc(node.node_id)}">${esc(short(node.friendly_name || node.node_id, 20))} — ${esc(node.hf_repo || node.model || "unassigned")}</option>`).join("");
|
||||||
|
dialog.onclose = () => {
|
||||||
|
if (dialog.returnValue !== "confirm") return;
|
||||||
|
if (action === "load") void requestAdminModelLoad(model, select.value);
|
||||||
|
else void releaseAdminModel(model, select.value);
|
||||||
|
};
|
||||||
|
dialog.showModal();
|
||||||
|
}
|
||||||
|
|
||||||
function chatAuthToken() {
|
function chatAuthToken() {
|
||||||
if (accountApiKeys.length) return accountApiKeys[0];
|
if (accountApiKeys.length) return accountApiKeys[0];
|
||||||
return null;
|
return null;
|
||||||
@@ -2293,21 +2477,21 @@ function markRefreshed() {
|
|||||||
|
|
||||||
async function fetchOverviewTab() {
|
async function fetchOverviewTab() {
|
||||||
const fetches = [
|
const fetches = [
|
||||||
fetchJson("/v1/raft/status"),
|
|
||||||
fetchJson("/v1/network/map"),
|
fetchJson("/v1/network/map"),
|
||||||
|
fetchJson("/v1/models"),
|
||||||
fetchJson("/v1/stats"),
|
fetchJson("/v1/stats"),
|
||||||
fetchJson("/v1/routing"),
|
fetchJson("/v1/routing"),
|
||||||
fetchJson("/v1/console"),
|
fetchJson("/v1/console"),
|
||||||
];
|
];
|
||||||
if (isLoggedIn) fetches.push(apiCall("/v1/account"));
|
if (isLoggedIn) fetches.push(apiCall("/v1/account"));
|
||||||
const results = await Promise.all(fetches);
|
const results = await Promise.all(fetches);
|
||||||
const [raft, map, stats, routing, consoleData, accountResp] = results;
|
const [map, models, stats, routing, consoleData, accountResp] = results;
|
||||||
lastStats = stats;
|
lastStats = stats;
|
||||||
lastRouting = routing;
|
lastRouting = routing;
|
||||||
lastNetworkMap = map;
|
lastNetworkMap = map;
|
||||||
if (accountResp && accountResp.ok) applyAccountSummary(accountResp.data, false);
|
if (accountResp && accountResp.ok) applyAccountSummary(accountResp.data, false);
|
||||||
renderIfChanged("hive", raft, renderHive);
|
const nodesPayload = { map, routing, priceMap: buildModelPriceMap(models) };
|
||||||
renderIfChanged("nodes", map, renderNodes);
|
renderIfChanged("nodes", nodesPayload, renderNodes);
|
||||||
renderIfChanged("stats", stats, renderStats);
|
renderIfChanged("stats", stats, renderStats);
|
||||||
renderIfChanged("routing", routing, renderRouting);
|
renderIfChanged("routing", routing, renderRouting);
|
||||||
const speedModels = Object.keys((routing && routing.models) || {});
|
const speedModels = Object.keys((routing && routing.models) || {});
|
||||||
@@ -2348,16 +2532,23 @@ async function fetchBillingTab() {
|
|||||||
|
|
||||||
async function fetchAdminTab() {
|
async function fetchAdminTab() {
|
||||||
const fetches = [
|
const fetches = [
|
||||||
|
fetchJson("/v1/raft/status"),
|
||||||
fetchJson("/v1/console"),
|
fetchJson("/v1/console"),
|
||||||
fetchJson("/v1/billing/summary"),
|
fetchJson("/v1/billing/summary"),
|
||||||
fetchJson("/v1/registry/wallets"),
|
fetchJson("/v1/registry/wallets"),
|
||||||
|
fetchJson("/v1/models"),
|
||||||
|
fetchJson("/v1/network/map"),
|
||||||
];
|
];
|
||||||
if (isAdmin) fetches.push(apiCall("/v1/admin/accounts"));
|
if (isAdmin) fetches.push(apiCall("/v1/admin/accounts"));
|
||||||
const results = await Promise.all(fetches);
|
const results = await Promise.all(fetches);
|
||||||
const [consoleData, summary, wallets, adminResp] = results;
|
const [raft, consoleData, summary, wallets, models, map, adminResp] = results;
|
||||||
|
if (map) lastNetworkMap = map;
|
||||||
|
renderIfChanged("hive", raft, renderHive);
|
||||||
renderIfChanged("console", consoleData, renderConsole);
|
renderIfChanged("console", consoleData, renderConsole);
|
||||||
renderIfChanged("billing-summary", summary, data => renderBilling(data));
|
renderIfChanged("billing-summary", summary, data => renderBilling(data));
|
||||||
renderIfChanged("fraud", { wallets, summary }, data => renderFraud(data.wallets, data.summary));
|
renderIfChanged("fraud", { wallets, summary }, data => renderFraud(data.wallets, data.summary));
|
||||||
|
renderIfChanged("admin-model-placement", { models, map }, data => renderAdminModelPlacement(data.models, data.map));
|
||||||
|
renderIfChanged("admin-node-pool", map, renderAdminNodePool);
|
||||||
if (adminResp && adminResp.ok) {
|
if (adminResp && adminResp.ok) {
|
||||||
renderIfChanged("admin", adminResp.data.accounts || [], accounts => {
|
renderIfChanged("admin", adminResp.data.accounts || [], accounts => {
|
||||||
const rows = accounts.map(a => {
|
const rows = accounts.map(a => {
|
||||||
|
|||||||
@@ -86,7 +86,7 @@ from .model_files import files_for_layer_range, snapshot_dir_for_repo
|
|||||||
from .raft import RaftNode
|
from .raft import RaftNode
|
||||||
|
|
||||||
|
|
||||||
_CONSOLE_LIMIT = 300
|
_CONSOLE_LIMIT = 1000
|
||||||
_PROXY_PROGRESS_LOG_INTERVAL = 5.0
|
_PROXY_PROGRESS_LOG_INTERVAL = 5.0
|
||||||
_SESSION_COOKIE_NAME = "meshnet_session"
|
_SESSION_COOKIE_NAME = "meshnet_session"
|
||||||
|
|
||||||
@@ -1494,7 +1494,9 @@ def _scale_demanded_models_locked(server: "_TrackerHTTPServer") -> None:
|
|||||||
break
|
break
|
||||||
|
|
||||||
|
|
||||||
def _request_model_load_locked(server: "_TrackerHTTPServer", model_key: str) -> dict | None:
|
def _request_model_load_locked(
|
||||||
|
server: "_TrackerHTTPServer", model_key: str, node_id: str | None = None,
|
||||||
|
) -> dict | None:
|
||||||
"""Queue an explicitly requested model on the best available joined node."""
|
"""Queue an explicitly requested model on the best available joined node."""
|
||||||
resolved_name, preset = _resolve_model_preset(server.model_presets, model_key)
|
resolved_name, preset = _resolve_model_preset(server.model_presets, model_key)
|
||||||
if preset is None or not preset.get("hf_repo"):
|
if preset is None or not preset.get("hf_repo"):
|
||||||
@@ -1510,6 +1512,8 @@ def _request_model_load_locked(server: "_TrackerHTTPServer", model_key: str) ->
|
|||||||
continue
|
continue
|
||||||
host_nodes = [server.registry[item["node_id"]] for item in host["loaded"] if item["node_id"] in server.registry]
|
host_nodes = [server.registry[item["node_id"]] for item in host["loaded"] if item["node_id"] in server.registry]
|
||||||
placeable = [node for node in host_nodes if _has_usable_quantization(node)]
|
placeable = [node for node in host_nodes if _has_usable_quantization(node)]
|
||||||
|
if node_id is not None:
|
||||||
|
placeable = [node for node in placeable if node.node_id == node_id]
|
||||||
if not placeable:
|
if not placeable:
|
||||||
continue
|
continue
|
||||||
anchor = max(placeable, key=lambda node: node.benchmark_tokens_per_sec)
|
anchor = max(placeable, key=lambda node: node.benchmark_tokens_per_sec)
|
||||||
@@ -1528,6 +1532,53 @@ def _request_model_load_locked(server: "_TrackerHTTPServer", model_key: str) ->
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _force_model_load_locked(server: "_TrackerHTTPServer", model_key: str) -> dict | None:
|
||||||
|
"""Replace the fastest ready assignment after an explicit admin eviction."""
|
||||||
|
resolved_name, preset = _resolve_model_preset(server.model_presets, model_key)
|
||||||
|
if preset is None or not preset.get("hf_repo"):
|
||||||
|
return None
|
||||||
|
start, end = _preset_layer_bounds(preset)
|
||||||
|
# An explicit admin eviction is permitted to recover a stuck/loading node
|
||||||
|
# and to use the preset default precision. It must only avoid a node that
|
||||||
|
# already has another assignment in flight.
|
||||||
|
candidates = [node for node in server.registry.values()
|
||||||
|
if node.pending_new_assignment is None]
|
||||||
|
if not candidates:
|
||||||
|
return None
|
||||||
|
node = max(candidates, key=lambda item: item.benchmark_tokens_per_sec)
|
||||||
|
shard_end = min(end, start + max(1, min(_node_layer_capacity(node, preset), end - start + 1)) - 1)
|
||||||
|
quantization = _node_quantization(node, preset)
|
||||||
|
directive = _load_directive(node, str(preset["hf_repo"]), start, shard_end, quantization)
|
||||||
|
replaced = node.hf_repo or node.model
|
||||||
|
node.model, node.hf_repo = resolved_name, str(preset["hf_repo"])
|
||||||
|
node.shard_start, node.shard_end, node.quantization = start, shard_end, quantization
|
||||||
|
node.managed_assignment, node.pending_new_assignment = True, directive
|
||||||
|
node.pending_directives.append(directive)
|
||||||
|
_tracker_log(server, "warn", "model load forced", node_id=node.node_id,
|
||||||
|
model=resolved_name, replaced_model=replaced, shard=f"{start}-{shard_end}")
|
||||||
|
return {"node_id": node.node_id, "model": resolved_name, "hf_repo": preset["hf_repo"],
|
||||||
|
"shard_start": start, "shard_end": shard_end, "replaced_model": replaced}
|
||||||
|
|
||||||
|
|
||||||
|
def _release_model_locked(
|
||||||
|
server: "_TrackerHTTPServer", model_key: str, node_id: str | None = None,
|
||||||
|
) -> int:
|
||||||
|
"""Queue DROP_SHARD for every served shard and remove it from routing immediately."""
|
||||||
|
resolved_name, preset = _resolve_model_preset(server.model_presets, model_key)
|
||||||
|
if preset is None:
|
||||||
|
return 0
|
||||||
|
released = 0
|
||||||
|
for node in server.registry.values():
|
||||||
|
if node_id is not None and node.node_id != node_id:
|
||||||
|
continue
|
||||||
|
if not _node_matches_preset(node, resolved_name, preset) or node.shard_start is None or node.shard_end is None:
|
||||||
|
continue
|
||||||
|
node.pending_directives.append(_drop_directive(node, str(preset.get("hf_repo") or resolved_name), node.shard_start, node.shard_end, node.quantization or "bfloat16"))
|
||||||
|
node.status = "loading"
|
||||||
|
released += 1
|
||||||
|
return released
|
||||||
|
|
||||||
|
|
||||||
def _preferred_node_quantization(
|
def _preferred_node_quantization(
|
||||||
node: _NodeEntry,
|
node: _NodeEntry,
|
||||||
preset: dict,
|
preset: dict,
|
||||||
@@ -3043,6 +3094,9 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
if self.path == "/v1/models/load":
|
if self.path == "/v1/models/load":
|
||||||
self._handle_model_load_request()
|
self._handle_model_load_request()
|
||||||
return
|
return
|
||||||
|
if self.path == "/v1/models/release":
|
||||||
|
self._handle_model_release_request()
|
||||||
|
return
|
||||||
if self.path == "/v1/models/vote":
|
if self.path == "/v1/models/vote":
|
||||||
self._handle_model_coverage_vote()
|
self._handle_model_coverage_vote()
|
||||||
return
|
return
|
||||||
@@ -3145,6 +3199,16 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
self.send_response(404)
|
self.send_response(404)
|
||||||
self.end_headers()
|
self.end_headers()
|
||||||
|
|
||||||
|
def _model_pricing_payload(self, model: str) -> dict | None:
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if server.billing is None:
|
||||||
|
return None
|
||||||
|
in_rate, out_rate = server.billing.prices_for(model)
|
||||||
|
return {
|
||||||
|
"input_per_1k_usdt": in_rate,
|
||||||
|
"output_per_1k_usdt": out_rate,
|
||||||
|
}
|
||||||
|
|
||||||
def _handle_models(self):
|
def _handle_models(self):
|
||||||
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
created = int(time.time())
|
created = int(time.time())
|
||||||
@@ -3160,8 +3224,6 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
seen_ids: set[str] = set()
|
seen_ids: set[str] = set()
|
||||||
for name, preset in server.model_presets.items():
|
for name, preset in server.model_presets.items():
|
||||||
model_nodes = [node for node in alive if _node_matches_preset(node, name, preset)]
|
model_nodes = [node for node in alive if _node_matches_preset(node, name, preset)]
|
||||||
if not model_nodes and not preset.get("recommended"):
|
|
||||||
continue
|
|
||||||
required_start, required_end = _preset_layer_bounds(preset)
|
required_start, required_end = _preset_layer_bounds(preset)
|
||||||
coverage = _coverage_percentage(
|
coverage = _coverage_percentage(
|
||||||
model_nodes,
|
model_nodes,
|
||||||
@@ -3200,6 +3262,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
"shard_coverage_percentage": coverage,
|
"shard_coverage_percentage": coverage,
|
||||||
"served_model_copies": served_copies,
|
"served_model_copies": served_copies,
|
||||||
"quantizations": quantizations,
|
"quantizations": quantizations,
|
||||||
|
"pricing": self._model_pricing_payload(name),
|
||||||
})
|
})
|
||||||
seen_ids.add(name)
|
seen_ids.add(name)
|
||||||
# Note: the preset's hf_repo is deliberately NOT added to seen_ids —
|
# Note: the preset's hf_repo is deliberately NOT added to seen_ids —
|
||||||
@@ -3251,6 +3314,7 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
required_start,
|
required_start,
|
||||||
required_end,
|
required_end,
|
||||||
),
|
),
|
||||||
|
"pricing": self._model_pricing_payload(model_id),
|
||||||
})
|
})
|
||||||
seen_ids.add(model_id)
|
seen_ids.add(model_id)
|
||||||
self._send_json(200, {"object": "list", "data": data})
|
self._send_json(200, {"object": "list", "data": data})
|
||||||
@@ -3310,6 +3374,11 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
"endpoint": node.endpoint,
|
"endpoint": node.endpoint,
|
||||||
"relay_addr": node.relay_addr,
|
"relay_addr": node.relay_addr,
|
||||||
"peer_id": node.peer_id,
|
"peer_id": node.peer_id,
|
||||||
|
"wallet_address": node.wallet_address,
|
||||||
|
"hardware_profile": dict(node.hardware_profile),
|
||||||
|
"ram_bytes": node.ram_bytes,
|
||||||
|
"vram_bytes": node.vram_bytes,
|
||||||
|
"max_loaded_shards": node.max_loaded_shards,
|
||||||
}
|
}
|
||||||
for node in tracker_nodes
|
for node in tracker_nodes
|
||||||
],
|
],
|
||||||
@@ -4819,14 +4888,43 @@ class _TrackerHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
if not isinstance(model, str) or not model.strip():
|
if not isinstance(model, str) or not model.strip():
|
||||||
self._send_json(400, {"error": "model is required"})
|
self._send_json(400, {"error": "model is required"})
|
||||||
return
|
return
|
||||||
|
node_id = body.get("node_id")
|
||||||
|
if node_id is not None and (not isinstance(node_id, str) or not node_id):
|
||||||
|
self._send_json(400, {"error": "node_id must be a non-empty string"})
|
||||||
|
return
|
||||||
with server.lock:
|
with server.lock:
|
||||||
self._purge_expired_nodes()
|
self._purge_expired_nodes()
|
||||||
assignment = _request_model_load_locked(server, model)
|
assignment = _request_model_load_locked(server, model, node_id)
|
||||||
|
if assignment is None and node_id is None and body.get("force") is True:
|
||||||
|
assignment = _force_model_load_locked(server, model)
|
||||||
if assignment is None:
|
if assignment is None:
|
||||||
self._send_json(409, {"error": "no ready joined node has an available model slot and sufficient capacity"})
|
self._send_json(409, {"error": "no ready joined node has an available model slot and sufficient capacity"})
|
||||||
return
|
return
|
||||||
self._send_json(202, {"status": "queued", "assignment": assignment})
|
self._send_json(202, {"status": "queued", "assignment": assignment})
|
||||||
|
|
||||||
|
def _handle_model_release_request(self):
|
||||||
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
|
if not self._require_role("admin", "validator"):
|
||||||
|
return
|
||||||
|
body = self._read_json_body()
|
||||||
|
if body is None:
|
||||||
|
return
|
||||||
|
model = body.get("model")
|
||||||
|
if not isinstance(model, str) or not model.strip():
|
||||||
|
self._send_json(400, {"error": "model is required"})
|
||||||
|
return
|
||||||
|
node_id = body.get("node_id")
|
||||||
|
if node_id is not None and (not isinstance(node_id, str) or not node_id):
|
||||||
|
self._send_json(400, {"error": "node_id must be a non-empty string"})
|
||||||
|
return
|
||||||
|
with server.lock:
|
||||||
|
self._purge_expired_nodes()
|
||||||
|
released = _release_model_locked(server, model, node_id)
|
||||||
|
if not released:
|
||||||
|
self._send_json(404, {"error": "no served shards found for model"})
|
||||||
|
return
|
||||||
|
self._send_json(202, {"status": "release_queued", "released": released})
|
||||||
|
|
||||||
def _handle_model_coverage_vote(self):
|
def _handle_model_coverage_vote(self):
|
||||||
"""Record a rolling wish-list signal for an unavailable precision."""
|
"""Record a rolling wish-list signal for an unavailable precision."""
|
||||||
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
server: _TrackerHTTPServer = self.server # type: ignore[assignment]
|
||||||
@@ -6477,7 +6575,7 @@ class TrackerServer:
|
|||||||
embedded_relay_port: int = 8765,
|
embedded_relay_port: int = 8765,
|
||||||
embedded_relay_max_peers: int = 500,
|
embedded_relay_max_peers: int = 500,
|
||||||
billing: BillingLedger | None = None,
|
billing: BillingLedger | None = None,
|
||||||
enable_billing: bool = False,
|
enable_billing: bool = True,
|
||||||
billing_db: str | None = None,
|
billing_db: str | None = None,
|
||||||
accounts: AccountStore | None = None,
|
accounts: AccountStore | None = None,
|
||||||
accounts_db: str | None = None,
|
accounts_db: str | None = None,
|
||||||
|
|||||||
@@ -396,7 +396,7 @@ def test_accounts_gossip_endpoint_applies_events(account_tracker):
|
|||||||
def test_accounts_endpoints_404_when_disabled():
|
def test_accounts_endpoints_404_when_disabled():
|
||||||
"Accounts endpoints 404 when disabled\n\nTags: accounts, auth, http"
|
"Accounts endpoints 404 when disabled\n\nTags: accounts, auth, http"
|
||||||
|
|
||||||
tracker = TrackerServer() # no accounts, no billing
|
tracker = TrackerServer(enable_billing=False) # no accounts, no billing
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
try:
|
try:
|
||||||
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
with pytest.raises(urllib.error.HTTPError) as exc_info:
|
||||||
|
|||||||
@@ -172,13 +172,13 @@ def test_restart_persistence(tmp_path):
|
|||||||
assert reloaded.snapshot()["protocol_cut"] == pytest.approx(0.02 * 0.10)
|
assert reloaded.snapshot()["protocol_cut"] == pytest.approx(0.02 * 0.10)
|
||||||
|
|
||||||
|
|
||||||
def test_tracker_enables_billing_with_default_db_when_requested(tmp_path, monkeypatch):
|
def test_tracker_enables_billing_with_default_db(tmp_path, monkeypatch):
|
||||||
"Tracker enables billing with default db when requested\n\nTags: billing, http, payments"
|
"Tracker enables billing with default db by default\n\nTags: billing, http, payments"
|
||||||
|
|
||||||
from meshnet_tracker.billing import DEFAULT_BILLING_DB_PATH
|
from meshnet_tracker.billing import DEFAULT_BILLING_DB_PATH
|
||||||
|
|
||||||
monkeypatch.chdir(tmp_path)
|
monkeypatch.chdir(tmp_path)
|
||||||
tracker = TrackerServer(enable_billing=True)
|
tracker = TrackerServer()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
try:
|
try:
|
||||||
# /v1/billing/summary is admin-gated now; just confirm the server is up.
|
# /v1/billing/summary is admin-gated now; just confirm the server is up.
|
||||||
@@ -329,6 +329,7 @@ def billed_tracker():
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
billing=ledger,
|
billing=ledger,
|
||||||
|
enable_billing=False,
|
||||||
hive_secret=HIVE_SECRET,
|
hive_secret=HIVE_SECRET,
|
||||||
)
|
)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
@@ -457,6 +458,7 @@ def test_proxy_chat_caps_inflated_streaming_usage_by_observed_chunks():
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
billing=ledger,
|
billing=ledger,
|
||||||
|
enable_billing=False,
|
||||||
)
|
)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{port}"
|
tracker_url = f"http://127.0.0.1:{port}"
|
||||||
@@ -506,6 +508,7 @@ def test_proxy_chat_splits_payout_by_tracker_assigned_route_span():
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
billing=ledger,
|
billing=ledger,
|
||||||
|
enable_billing=False,
|
||||||
)
|
)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{port}"
|
tracker_url = f"http://127.0.0.1:{port}"
|
||||||
@@ -582,6 +585,7 @@ def test_proxy_chat_rejects_request_above_spend_cap_before_routing():
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
billing=ledger,
|
billing=ledger,
|
||||||
|
enable_billing=False,
|
||||||
max_charge_per_request=0.01,
|
max_charge_per_request=0.01,
|
||||||
)
|
)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
@@ -656,6 +660,7 @@ def test_proxy_chat_records_validation_event_with_plain_route_metadata():
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
contracts=contracts,
|
contracts=contracts,
|
||||||
|
enable_billing=False,
|
||||||
)
|
)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{port}"
|
tracker_url = f"http://127.0.0.1:{port}"
|
||||||
|
|||||||
@@ -39,6 +39,14 @@ def test_dashboard_served_with_all_panels():
|
|||||||
assert "resolveModelGroup" in html
|
assert "resolveModelGroup" in html
|
||||||
assert "buildModelAliasMap" in html
|
assert "buildModelAliasMap" in html
|
||||||
assert "modelAliasKey(raw)" in html
|
assert "modelAliasKey(raw)" in html
|
||||||
|
assert "@media (min-width:900px)" in html
|
||||||
|
assert "grid-template-columns:repeat(4,minmax(0,1fr));" in html
|
||||||
|
assert ".wide { grid-column:span 2; }" in html
|
||||||
|
assert 'onclick="clearConsole()"' in html
|
||||||
|
assert "let consoleClearedAt = 0;" in html
|
||||||
|
assert "max-height:520px; overflow-y:auto; overflow-x:auto;" in html
|
||||||
|
assert "const CONSOLE_MAX_LINES = 1000;" in html
|
||||||
|
assert "events.slice(-CONSOLE_MAX_LINES)" in html
|
||||||
finally:
|
finally:
|
||||||
tracker.stop()
|
tracker.stop()
|
||||||
|
|
||||||
@@ -97,6 +105,29 @@ def test_dashboard_allows_admin_to_request_selected_model_load():
|
|||||||
assert '$("request-model-load").style.display = enabled ? "" : "none"' in html
|
assert '$("request-model-load").style.display = enabled ? "" : "none"' in html
|
||||||
|
|
||||||
|
|
||||||
|
def test_dashboard_exposes_admin_model_inventory_and_release_controls():
|
||||||
|
"Admin placement controls show the full model inventory and can release capacity."
|
||||||
|
html = _dashboard_html()
|
||||||
|
|
||||||
|
assert 'id="admin-model-placement"' in html
|
||||||
|
assert "renderAdminModelPlacement" in html
|
||||||
|
assert '"/v1/models/release"' in html
|
||||||
|
assert "requestAdminModelLoad" in html
|
||||||
|
assert "releaseAdminModel" in html
|
||||||
|
assert 'data-admin-model-load=' in html
|
||||||
|
assert 'data-admin-model-release=' in html
|
||||||
|
assert "admin-model-placement-status" in html
|
||||||
|
assert 'id="admin-node-pool"' in html
|
||||||
|
assert "renderAdminNodePool" in html
|
||||||
|
assert "model drive" in html
|
||||||
|
# RAM and VRAM live in the network-map capacity object, not at node top level.
|
||||||
|
assert "node.ram_bytes = cap.ram_bytes" in html
|
||||||
|
assert "node.vram_bytes = cap.vram_bytes" in html
|
||||||
|
assert 'id="model-placement-dialog"' in html
|
||||||
|
assert "chooseModelPlacementNode" in html
|
||||||
|
assert "node_id: nodeId" in html
|
||||||
|
|
||||||
|
|
||||||
def test_network_map_includes_node_friendly_name():
|
def test_network_map_includes_node_friendly_name():
|
||||||
"Network map includes node friendly name\n\nTags: dashboard, http"
|
"Network map includes node friendly name\n\nTags: dashboard, http"
|
||||||
tracker = TrackerServer()
|
tracker = TrackerServer()
|
||||||
|
|||||||
@@ -253,7 +253,7 @@ def test_proxy_head_is_route_head_and_routing_endpoint_lists_routes():
|
|||||||
threads.append(thread)
|
threads.append(thread)
|
||||||
gpu_stub, cpu_stub = stubs
|
gpu_stub, cpu_stub = stubs
|
||||||
|
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = TrackerServer(enable_billing=False, model_presets={
|
||||||
"qwen3.6-35b-a3b": {
|
"qwen3.6-35b-a3b": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": 39,
|
"layers_end": 39,
|
||||||
@@ -321,7 +321,7 @@ def test_proxy_head_is_route_head_and_routing_endpoint_lists_routes():
|
|||||||
|
|
||||||
def test_admin_model_load_request_queues_directive_on_joined_node():
|
def test_admin_model_load_request_queues_directive_on_joined_node():
|
||||||
"Admin model load request queues directive on joined node\n\nTags: http, performance, routing, tracker"
|
"Admin model load request queues directive on joined node\n\nTags: http, performance, routing, tracker"
|
||||||
tracker = TrackerServer(validator_service_token="test-admin")
|
tracker = TrackerServer(enable_billing=False, validator_service_token="test-admin")
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
try:
|
try:
|
||||||
node = _post_json(
|
node = _post_json(
|
||||||
@@ -355,6 +355,39 @@ def test_admin_model_load_request_queues_directive_on_joined_node():
|
|||||||
assert heartbeat["directives"][0]["model"] == "Qwen/Qwen2.5-0.5B-Instruct"
|
assert heartbeat["directives"][0]["model"] == "Qwen/Qwen2.5-0.5B-Instruct"
|
||||||
|
|
||||||
|
|
||||||
|
def test_admin_can_replace_a_served_model_and_release_it():
|
||||||
|
"Forced admin placement replaces a served shard; release queues DROP_SHARD."
|
||||||
|
tracker = TrackerServer(enable_billing=False, validator_service_token="test-admin")
|
||||||
|
port = tracker.start()
|
||||||
|
try:
|
||||||
|
node = _post_json(
|
||||||
|
f"http://127.0.0.1:{port}/v1/nodes/register",
|
||||||
|
{"endpoint": "http://127.0.0.1:9912", "model": "stub-model",
|
||||||
|
"shard_start": 0, "shard_end": 3, "managed_assignment": True,
|
||||||
|
"max_loaded_shards": 1, "memory_mb": 1,
|
||||||
|
"hardware_profile": {"host_id": "full-host"}},
|
||||||
|
)
|
||||||
|
headers = {"Content-Type": "application/json", "Authorization": "Bearer test-admin"}
|
||||||
|
load = urllib.request.Request(
|
||||||
|
f"http://127.0.0.1:{port}/v1/models/load",
|
||||||
|
data=json.dumps({"model": "qwen2.5-0.5b-instruct", "force": True}).encode(),
|
||||||
|
headers=headers, method="POST")
|
||||||
|
with urllib.request.urlopen(load) as response:
|
||||||
|
assert json.loads(response.read())["assignment"]["node_id"] == node["node_id"]
|
||||||
|
heartbeat = _post_json(f"http://127.0.0.1:{port}/v1/nodes/{node['node_id']}/heartbeat", {})
|
||||||
|
assert heartbeat["directives"][0]["action"] == "LOAD_SHARD"
|
||||||
|
release = urllib.request.Request(
|
||||||
|
f"http://127.0.0.1:{port}/v1/models/release",
|
||||||
|
data=json.dumps({"model": "qwen2.5-0.5b-instruct"}).encode(), headers=headers, method="POST")
|
||||||
|
with urllib.request.urlopen(release) as response:
|
||||||
|
assert json.loads(response.read())["released"] == 1
|
||||||
|
heartbeat = _post_json(f"http://127.0.0.1:{port}/v1/nodes/{node['node_id']}/heartbeat", {})
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
|
||||||
|
assert heartbeat["directives"][0]["action"] == "DROP_SHARD"
|
||||||
|
|
||||||
|
|
||||||
def test_endpoint_key_distinguishes_same_port_different_hosts():
|
def test_endpoint_key_distinguishes_same_port_different_hosts():
|
||||||
"Endpoint key distinguishes same port different hosts\n\nTags: http, performance, routing, tracker"
|
"Endpoint key distinguishes same port different hosts\n\nTags: http, performance, routing, tracker"
|
||||||
from meshnet_node.torch_server import _clamp_downstream_hops, _endpoint_key
|
from meshnet_node.torch_server import _clamp_downstream_hops, _endpoint_key
|
||||||
|
|||||||
@@ -203,6 +203,7 @@ def test_probation_earns_nothing_then_earning_begins():
|
|||||||
},
|
},
|
||||||
contracts=contracts,
|
contracts=contracts,
|
||||||
billing=ledger,
|
billing=ledger,
|
||||||
|
enable_billing=False,
|
||||||
)
|
)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{port}"
|
tracker_url = f"http://127.0.0.1:{port}"
|
||||||
|
|||||||
@@ -60,7 +60,7 @@ def test_bad_node_is_slashed_and_excluded_from_gateway_routes(capsys):
|
|||||||
contracts = LocalSolanaContracts()
|
contracts = LocalSolanaContracts()
|
||||||
contracts.registry.submit_stake("wallet-good", 500)
|
contracts.registry.submit_stake("wallet-good", 500)
|
||||||
contracts.registry.submit_stake("wallet-bad", 500)
|
contracts.registry.submit_stake("wallet-bad", 500)
|
||||||
tracker = TrackerServer(contracts=contracts)
|
tracker = TrackerServer(contracts=contracts, enable_billing=False)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
||||||
|
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ def test_tracker_records_increasing_hop_latency_for_model_and_hardware(model, ha
|
|||||||
|
|
||||||
layer_count, hf_repo = MODELS[model]
|
layer_count, hf_repo = MODELS[model]
|
||||||
nodes, threads = _start_latency_nodes(5)
|
nodes, threads = _start_latency_nodes(5)
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = TrackerServer(enable_billing=False, model_presets={
|
||||||
model: {
|
model: {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": layer_count - 1,
|
"layers_end": layer_count - 1,
|
||||||
@@ -142,7 +142,7 @@ def test_tracker_records_increasing_hop_latency_for_model_and_hardware(model, ha
|
|||||||
|
|
||||||
def test_model_speed_dashboard_includes_visualization_and_route_drilldown():
|
def test_model_speed_dashboard_includes_visualization_and_route_drilldown():
|
||||||
"Model speed dashboard includes visualization and route drilldown\n\nTags: model, node, routing"
|
"Model speed dashboard includes visualization and route drilldown\n\nTags: model, node, routing"
|
||||||
tracker = TrackerServer()
|
tracker = TrackerServer(enable_billing=False)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
try:
|
try:
|
||||||
html = urllib.request.urlopen(f"http://127.0.0.1:{port}/dashboard").read().decode()
|
html = urllib.request.urlopen(f"http://127.0.0.1:{port}/dashboard").read().decode()
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ def _register_node(tracker_url: str, endpoint: str, shard_start: int, shard_end:
|
|||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def two_node_setup():
|
def two_node_setup():
|
||||||
"""Start tracker, two stub nodes, and gateway; yield (gateway_url, node_a, node_b)."""
|
"""Start tracker, two stub nodes, and gateway; yield (gateway_url, node_a, node_b)."""
|
||||||
tracker = TrackerServer()
|
tracker = TrackerServer(enable_billing=False)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
||||||
|
|
||||||
|
|||||||
@@ -1075,6 +1075,190 @@ def test_partial_snapshot_loader_materializes_only_assigned_tensors(tmp_path):
|
|||||||
assert model.model.rotary_emb.to_calls == ["cpu:0"]
|
assert model.model.rotary_emb.to_calls == ["cpu:0"]
|
||||||
|
|
||||||
|
|
||||||
|
def _build_partial_snapshot_fixture(snapshot_dir: Path, *, num_layers: int = 4) -> dict[str, dict[str, int]]:
|
||||||
|
"""Minimal HF snapshot with per-tensor numel metadata for memory-scaling tests."""
|
||||||
|
weight_map: dict[str, str] = {
|
||||||
|
"model.embed_tokens.weight": "shard-head.safetensors",
|
||||||
|
"model.norm.weight": "shard-tail.safetensors",
|
||||||
|
"lm_head.weight": "shard-tail.safetensors",
|
||||||
|
}
|
||||||
|
tensor_numel: dict[str, dict[str, int]] = {
|
||||||
|
"shard-head.safetensors": {"model.embed_tokens.weight": 10_000},
|
||||||
|
"shard-tail.safetensors": {
|
||||||
|
"model.norm.weight": 512,
|
||||||
|
"lm_head.weight": 10_000,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for layer in range(num_layers):
|
||||||
|
rel = f"shard-layer-{layer}.safetensors"
|
||||||
|
weight_map[f"model.layers.{layer}.self_attn.q_proj.weight"] = rel
|
||||||
|
weight_map[f"model.layers.{layer}.mlp.down_proj.weight"] = rel
|
||||||
|
per_layer = 1_000 * (layer + 1)
|
||||||
|
tensor_numel[rel] = {
|
||||||
|
f"model.layers.{layer}.self_attn.q_proj.weight": per_layer,
|
||||||
|
f"model.layers.{layer}.mlp.down_proj.weight": per_layer,
|
||||||
|
}
|
||||||
|
(snapshot_dir / rel).write_bytes(b"stub")
|
||||||
|
|
||||||
|
(snapshot_dir / "config.json").write_text(json.dumps({"num_hidden_layers": num_layers}))
|
||||||
|
(snapshot_dir / "model.safetensors.index.json").write_text(json.dumps({"weight_map": weight_map}))
|
||||||
|
(snapshot_dir / "shard-head.safetensors").write_bytes(b"stub")
|
||||||
|
(snapshot_dir / "shard-tail.safetensors").write_bytes(b"stub")
|
||||||
|
return tensor_numel
|
||||||
|
|
||||||
|
|
||||||
|
def _partial_load_materialized_numel(
|
||||||
|
snapshot_dir: Path,
|
||||||
|
shard_start: int,
|
||||||
|
shard_end: int,
|
||||||
|
tensor_numel: dict[str, dict[str, int]],
|
||||||
|
) -> int:
|
||||||
|
"""Return the summed numel of checkpoint tensors assigned to one shard load."""
|
||||||
|
|
||||||
|
class FakeModule:
|
||||||
|
def to(self, device):
|
||||||
|
return self
|
||||||
|
|
||||||
|
class FakeModel:
|
||||||
|
def __init__(self, num_layers: int):
|
||||||
|
self.model = types.SimpleNamespace(
|
||||||
|
embed_tokens=FakeModule(),
|
||||||
|
layers=[FakeModule() for _ in range(num_layers)],
|
||||||
|
rotary_emb=FakeModule(),
|
||||||
|
norm=FakeModule(),
|
||||||
|
)
|
||||||
|
self.lm_head = FakeModule()
|
||||||
|
|
||||||
|
def tie_weights(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
class AutoConfigStub:
|
||||||
|
@staticmethod
|
||||||
|
def from_pretrained(model_id):
|
||||||
|
assert model_id == str(snapshot_dir)
|
||||||
|
return types.SimpleNamespace(num_hidden_layers=num_layers)
|
||||||
|
|
||||||
|
class AutoModelStub:
|
||||||
|
@staticmethod
|
||||||
|
def from_config(cfg, torch_dtype=None):
|
||||||
|
return FakeModel(cfg.num_hidden_layers)
|
||||||
|
|
||||||
|
class UnusedContext:
|
||||||
|
def __enter__(self):
|
||||||
|
return None
|
||||||
|
|
||||||
|
def __exit__(self, exc_type, exc, tb):
|
||||||
|
return False
|
||||||
|
|
||||||
|
num_layers = json.loads((snapshot_dir / "config.json").read_text())["num_hidden_layers"]
|
||||||
|
loaded_numel = 0
|
||||||
|
|
||||||
|
def fake_set_tensor(module, tensor_name, device, value=None, dtype=None):
|
||||||
|
nonlocal loaded_numel
|
||||||
|
loaded_numel += int(getattr(value, "numel", lambda: 0)())
|
||||||
|
|
||||||
|
class FakeTensor:
|
||||||
|
def __init__(self, numel: int):
|
||||||
|
self._numel = numel
|
||||||
|
|
||||||
|
def numel(self) -> int:
|
||||||
|
return self._numel
|
||||||
|
|
||||||
|
class FakeSafeOpen:
|
||||||
|
def __init__(self, filename, framework, device):
|
||||||
|
self.filename = Path(filename).name
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, exc_type, exc, tb):
|
||||||
|
return False
|
||||||
|
|
||||||
|
def get_tensor(self, tensor_name):
|
||||||
|
return FakeTensor(tensor_numel[self.filename][tensor_name])
|
||||||
|
|
||||||
|
_load_partial_model_from_snapshot(
|
||||||
|
AutoConfigStub,
|
||||||
|
AutoModelStub,
|
||||||
|
types.SimpleNamespace(),
|
||||||
|
str(snapshot_dir),
|
||||||
|
shard_start,
|
||||||
|
shard_end,
|
||||||
|
"bf16",
|
||||||
|
"cpu:0",
|
||||||
|
init_empty_weights_fn=UnusedContext,
|
||||||
|
set_tensor_fn=fake_set_tensor,
|
||||||
|
safe_open_fn=FakeSafeOpen,
|
||||||
|
)
|
||||||
|
return loaded_numel
|
||||||
|
|
||||||
|
|
||||||
|
def test_partial_snapshot_resident_weight_numel_scales_with_shard(tmp_path):
|
||||||
|
"Partial load materializes only assigned-layer weights, not the full checkpoint.\n\nTags: model, node, real-inference"
|
||||||
|
snapshot_dir = tmp_path / "snapshot"
|
||||||
|
snapshot_dir.mkdir()
|
||||||
|
tensor_numel = _build_partial_snapshot_fixture(snapshot_dir, num_layers=4)
|
||||||
|
|
||||||
|
middle_numel = _partial_load_materialized_numel(snapshot_dir, 1, 1, tensor_numel)
|
||||||
|
full_numel = _partial_load_materialized_numel(snapshot_dir, 0, 3, tensor_numel)
|
||||||
|
|
||||||
|
# Layer 1 only: two tensors at 2000 numel each.
|
||||||
|
assert middle_numel == 4_000
|
||||||
|
# Head embed + four layers (two tensors each, increasing sizes) + tail norm/lm_head.
|
||||||
|
assert full_numel == 10_000 + 512 + 10_000 + 2_000 + 4_000 + 6_000 + 8_000
|
||||||
|
assert middle_numel < full_numel // 4
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.integration
|
||||||
|
def test_partial_snapshot_materialized_param_count_with_real_torch(tmp_path):
|
||||||
|
"When torch is installed, partial load leaves unassigned params on meta device.\n\nTags: model, node, real-inference"
|
||||||
|
torch = _require_functional_torch()
|
||||||
|
pytest.importorskip("transformers")
|
||||||
|
pytest.importorskip("safetensors")
|
||||||
|
pytest.importorskip("accelerate")
|
||||||
|
from safetensors.torch import save_file
|
||||||
|
from transformers import AutoConfig, AutoModelForCausalLM, GPT2Config
|
||||||
|
|
||||||
|
n_layer = 4
|
||||||
|
n_embd = 16
|
||||||
|
snapshot_dir = tmp_path / "snapshot"
|
||||||
|
snapshot_dir.mkdir()
|
||||||
|
config = GPT2Config(n_layer=n_layer, n_embd=n_embd, n_head=2, n_positions=32)
|
||||||
|
(snapshot_dir / "config.json").write_text(config.to_json_string())
|
||||||
|
|
||||||
|
weight_map: dict[str, str] = {}
|
||||||
|
for layer in range(n_layer):
|
||||||
|
key = f"transformer.h.{layer}.attn.c_attn.weight"
|
||||||
|
rel = f"layer-{layer}.safetensors"
|
||||||
|
weight_map[key] = rel
|
||||||
|
save_file({key: torch.ones(n_embd, 3 * n_embd)}, snapshot_dir / rel)
|
||||||
|
(snapshot_dir / "model.safetensors.index.json").write_text(json.dumps({"weight_map": weight_map}))
|
||||||
|
|
||||||
|
def count_materialized(model) -> tuple[int, int]:
|
||||||
|
materialized = 0
|
||||||
|
meta = 0
|
||||||
|
for param in model.parameters():
|
||||||
|
if param.device.type == "meta":
|
||||||
|
meta += param.numel()
|
||||||
|
else:
|
||||||
|
materialized += param.numel()
|
||||||
|
return materialized, meta
|
||||||
|
|
||||||
|
middle = _load_partial_model_from_snapshot(
|
||||||
|
AutoConfig,
|
||||||
|
AutoModelForCausalLM,
|
||||||
|
torch,
|
||||||
|
str(snapshot_dir),
|
||||||
|
1,
|
||||||
|
1,
|
||||||
|
torch.float32,
|
||||||
|
torch.device("cpu"),
|
||||||
|
)
|
||||||
|
mid_mat, mid_meta = count_materialized(middle)
|
||||||
|
assert mid_mat == n_embd * 3 * n_embd
|
||||||
|
assert mid_meta > mid_mat * 10
|
||||||
|
|
||||||
|
|
||||||
def test_partial_snapshot_loader_requires_known_layer_count(tmp_path):
|
def test_partial_snapshot_loader_requires_known_layer_count(tmp_path):
|
||||||
"Partial snapshot loader requires known layer count\n\nTags: model, node, real-inference"
|
"Partial snapshot loader requires known layer count\n\nTags: model, node, real-inference"
|
||||||
snapshot_dir = tmp_path / "snapshot"
|
snapshot_dir = tmp_path / "snapshot"
|
||||||
|
|||||||
@@ -32,12 +32,18 @@ def test_matrix_reports_direct_relay_prefill_decode_and_machine_readable_metrics
|
|||||||
assert {"p50_latency_ms", "p95_latency_ms", "payload_bytes", "compression_ratio",
|
assert {"p50_latency_ms", "p95_latency_ms", "payload_bytes", "compression_ratio",
|
||||||
"connection_attempts", "p95_queue_wait_ms"} <= set(run["phases"]["decode"])
|
"connection_attempts", "p95_queue_wait_ms"} <= set(run["phases"]["decode"])
|
||||||
sample = run["samples"][0]
|
sample = run["samples"][0]
|
||||||
|
assert sample["model_ms"] > 0
|
||||||
|
assert sample["encode_ms"] > 0
|
||||||
|
assert sample["activation_decode_ms"] > 0
|
||||||
assert sample["framing_ms"] > 0
|
assert sample["framing_ms"] > 0
|
||||||
assert sample["metadata_ms"] > 0
|
assert sample["metadata_ms"] > 0
|
||||||
assert sample["copy_allocation_ms"] > 0
|
assert sample["copy_allocation_ms"] > 0
|
||||||
assert sample["copy_allocation_bytes"] >= sample["payload_bytes"]
|
assert sample["copy_allocation_bytes"] >= sample["payload_bytes"]
|
||||||
|
assert sample["local_http_forwarding_ms"] > 0
|
||||||
assert len(run["samples"]) == 1 + len(run["output_tokens"])
|
assert len(run["samples"]) == 1 + len(run["output_tokens"])
|
||||||
assert {"tokens_per_sec", "bytes_per_token", "compression_cpu_ms", "peak_buffered_bytes"} <= set(run["phases"]["decode"])
|
assert {"tokens_per_sec", "bytes_per_token", "compression_cpu_ms", "peak_buffered_bytes",
|
||||||
|
"model_execution_ms", "activation_encoding_ms", "activation_decoding_ms",
|
||||||
|
"local_http_forwarding_ms"} <= set(run["phases"]["decode"])
|
||||||
|
|
||||||
|
|
||||||
def test_cached_sessions_reuse_one_connection_and_preserve_stub_tokens():
|
def test_cached_sessions_reuse_one_connection_and_preserve_stub_tokens():
|
||||||
@@ -74,7 +80,10 @@ def test_cli_writes_json_artifact_and_human_summary(tmp_path, capsys):
|
|||||||
report = json.loads(output.read_text())
|
report = json.loads(output.read_text())
|
||||||
assert report["schema_version"] == 1
|
assert report["schema_version"] == 1
|
||||||
assert "Route Session benchmark" in capsys.readouterr().out
|
assert "Route Session benchmark" in capsys.readouterr().out
|
||||||
assert "relay" in format_summary(report)
|
summary = format_summary(report)
|
||||||
|
assert "relay" in summary
|
||||||
|
assert "model/encode/decode" in summary
|
||||||
|
assert "HTTP" in summary
|
||||||
|
|
||||||
|
|
||||||
def test_performance_gate_checks_comparison_identity_session_and_cleanup():
|
def test_performance_gate_checks_comparison_identity_session_and_cleanup():
|
||||||
|
|||||||
@@ -12,7 +12,12 @@ import urllib.request
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from meshnet_contracts import LocalSolanaContracts
|
from meshnet_contracts import LocalSolanaContracts
|
||||||
from meshnet_tracker.billing import BillingLedger
|
from meshnet_tracker.billing import (
|
||||||
|
BillingLedger,
|
||||||
|
DEFAULT_PAYOUT_DUST_FLOOR,
|
||||||
|
DEFAULT_PAYOUT_THRESHOLD,
|
||||||
|
DEFAULT_SETTLE_PERIOD,
|
||||||
|
)
|
||||||
from meshnet_tracker.server import TrackerServer
|
from meshnet_tracker.server import TrackerServer
|
||||||
|
|
||||||
MODEL = "stub-model"
|
MODEL = "stub-model"
|
||||||
@@ -172,3 +177,115 @@ def test_non_leader_never_signs():
|
|||||||
assert ledger.get_node_pending("wallet-a") > 0
|
assert ledger.get_node_pending("wallet-a") > 0
|
||||||
finally:
|
finally:
|
||||||
tracker.stop()
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
|
# Production defaults — sourced from billing.py (cli.py uses the same constants)
|
||||||
|
PROD_SETTLE_PERIOD = DEFAULT_SETTLE_PERIOD
|
||||||
|
PROD_PAYOUT_THRESHOLD = DEFAULT_PAYOUT_THRESHOLD
|
||||||
|
PROD_DUST_FLOOR = DEFAULT_PAYOUT_DUST_FLOOR
|
||||||
|
|
||||||
|
|
||||||
|
def _set_pending_since(ledger: BillingLedger, wallet: str, ts: float) -> None:
|
||||||
|
with ledger._lock:
|
||||||
|
ledger._pending_since[wallet] = ts
|
||||||
|
|
||||||
|
|
||||||
|
def test_prod_settlement_constants():
|
||||||
|
"Production settlement policy is 24 h period, 5 USDT threshold, 0.01 dust floor.\n\nTags: billing, settlement"
|
||||||
|
assert DEFAULT_SETTLE_PERIOD == 86400.0
|
||||||
|
assert DEFAULT_PAYOUT_THRESHOLD == 5.0
|
||||||
|
assert DEFAULT_PAYOUT_DUST_FLOOR == 0.01
|
||||||
|
|
||||||
|
|
||||||
|
def test_payables_24h_period_not_due_before_elapsed():
|
||||||
|
"Sub-threshold pending is not payable until the 24 h period elapses.\n\nTags: billing, settlement"
|
||||||
|
ledger = BillingLedger(starting_credit=100.0, default_price_per_1k=0.02)
|
||||||
|
ledger.charge_request("client", MODEL, 1000, [("node-wallet", 12)])
|
||||||
|
assert ledger.get_node_pending("node-wallet") == pytest.approx(0.018)
|
||||||
|
|
||||||
|
t0 = 1_700_000_000.0
|
||||||
|
_set_pending_since(ledger, "node-wallet", t0)
|
||||||
|
due_early = ledger.payables(
|
||||||
|
threshold=PROD_PAYOUT_THRESHOLD,
|
||||||
|
max_period=PROD_SETTLE_PERIOD,
|
||||||
|
dust_floor=PROD_DUST_FLOOR,
|
||||||
|
now=t0 + PROD_SETTLE_PERIOD - 1.0,
|
||||||
|
)
|
||||||
|
assert due_early == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_payables_24h_period_triggers_at_boundary():
|
||||||
|
"Sub-threshold pending becomes payable exactly after the 24 h period.\n\nTags: billing, settlement"
|
||||||
|
ledger = BillingLedger(starting_credit=100.0, default_price_per_1k=0.02)
|
||||||
|
ledger.charge_request("client", MODEL, 1000, [("node-wallet", 12)])
|
||||||
|
pending = ledger.get_node_pending("node-wallet")
|
||||||
|
|
||||||
|
t0 = 1_700_000_000.0
|
||||||
|
_set_pending_since(ledger, "node-wallet", t0)
|
||||||
|
due = ledger.payables(
|
||||||
|
threshold=PROD_PAYOUT_THRESHOLD,
|
||||||
|
max_period=PROD_SETTLE_PERIOD,
|
||||||
|
dust_floor=PROD_DUST_FLOOR,
|
||||||
|
now=t0 + PROD_SETTLE_PERIOD,
|
||||||
|
)
|
||||||
|
assert due == [("node-wallet", pytest.approx(pending))]
|
||||||
|
|
||||||
|
|
||||||
|
def test_payables_threshold_triggers_before_24h_period():
|
||||||
|
"Pending ≥ 5 USDT pays immediately without waiting the 24 h period.\n\nTags: billing, settlement"
|
||||||
|
ledger = BillingLedger(starting_credit=1000.0, default_price_per_1k=0.02)
|
||||||
|
# cost = 300k * 0.02 / 1k = 6.0 USDT → node share 5.4 USDT
|
||||||
|
ledger.charge_request("client", MODEL, 300_000, [("node-wallet", 12)])
|
||||||
|
assert ledger.get_node_pending("node-wallet") >= PROD_PAYOUT_THRESHOLD
|
||||||
|
|
||||||
|
due = ledger.payables(
|
||||||
|
threshold=PROD_PAYOUT_THRESHOLD,
|
||||||
|
max_period=PROD_SETTLE_PERIOD,
|
||||||
|
dust_floor=PROD_DUST_FLOOR,
|
||||||
|
now=time.time(),
|
||||||
|
)
|
||||||
|
assert len(due) == 1
|
||||||
|
assert due[0][0] == "node-wallet"
|
||||||
|
|
||||||
|
|
||||||
|
def test_prod_defaults_hold_sub_threshold_pending():
|
||||||
|
"Tracker with prod 24 h / 5 USDT settings does not pay tiny balances early.\n\nTags: billing, settlement"
|
||||||
|
ledger = BillingLedger(starting_credit=100.0, default_price_per_1k=0.02)
|
||||||
|
ledger.charge_request("client", MODEL, 1000, [("node-wallet", 12)])
|
||||||
|
treasury = _FakePayoutTreasury()
|
||||||
|
tracker = _make_tracker(
|
||||||
|
ledger,
|
||||||
|
treasury,
|
||||||
|
threshold=PROD_PAYOUT_THRESHOLD,
|
||||||
|
period=PROD_SETTLE_PERIOD,
|
||||||
|
dust=PROD_DUST_FLOOR,
|
||||||
|
)
|
||||||
|
tracker.start()
|
||||||
|
try:
|
||||||
|
time.sleep(0.5)
|
||||||
|
assert treasury.batches == []
|
||||||
|
assert ledger.get_node_pending("node-wallet") > 0
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def test_prod_threshold_triggers_immediate_payout():
|
||||||
|
"Tracker with prod 24 h period still pays when pending ≥ 5 USDT.\n\nTags: billing, settlement"
|
||||||
|
ledger = BillingLedger(starting_credit=1000.0, default_price_per_1k=0.02)
|
||||||
|
ledger.charge_request("client", MODEL, 300_000, [("node-wallet", 12)])
|
||||||
|
treasury = _FakePayoutTreasury()
|
||||||
|
tracker = _make_tracker(
|
||||||
|
ledger,
|
||||||
|
treasury,
|
||||||
|
threshold=PROD_PAYOUT_THRESHOLD,
|
||||||
|
period=PROD_SETTLE_PERIOD,
|
||||||
|
dust=PROD_DUST_FLOOR,
|
||||||
|
)
|
||||||
|
tracker.start()
|
||||||
|
try:
|
||||||
|
assert _wait_for(lambda: treasury.batches)
|
||||||
|
assert ledger.get_node_pending("node-wallet") == pytest.approx(0.0)
|
||||||
|
assert treasury.batches[0][0][0] == "node-wallet"
|
||||||
|
assert treasury.batches[0][0][1] >= PROD_PAYOUT_THRESHOLD
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
|||||||
@@ -28,6 +28,11 @@ from meshnet_tracker.server import (
|
|||||||
_TEST_HIVE_SECRET = "test-hive-secret"
|
_TEST_HIVE_SECRET = "test-hive-secret"
|
||||||
|
|
||||||
|
|
||||||
|
def _tracker(**kwargs):
|
||||||
|
kwargs.setdefault("enable_billing", False)
|
||||||
|
return TrackerServer(**kwargs)
|
||||||
|
|
||||||
|
|
||||||
def _post_json(url: str, payload: dict) -> dict:
|
def _post_json(url: str, payload: dict) -> dict:
|
||||||
data = json.dumps(payload).encode()
|
data = json.dumps(payload).encode()
|
||||||
req = urllib.request.Request(
|
req = urllib.request.Request(
|
||||||
@@ -67,7 +72,7 @@ def test_tracker_send_json_ignores_broken_pipe_after_client_disconnect():
|
|||||||
|
|
||||||
def test_tracker_exposes_registered_model_metadata():
|
def test_tracker_exposes_registered_model_metadata():
|
||||||
"Tracker exposes registered model metadata\n\nTags: http, routing, tracker"
|
"Tracker exposes registered model metadata\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
try:
|
try:
|
||||||
@@ -114,7 +119,7 @@ def test_tracker_exposes_registered_model_metadata():
|
|||||||
|
|
||||||
def test_tracker_lists_recommended_kimi_before_nodes_register():
|
def test_tracker_lists_recommended_kimi_before_nodes_register():
|
||||||
"Tracker lists recommended kimi before nodes register\n\nTags: http, routing, tracker"
|
"Tracker lists recommended kimi before nodes register\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
try:
|
try:
|
||||||
@@ -138,7 +143,7 @@ def test_tracker_lists_recommended_qwen_27b_before_nodes_register():
|
|||||||
|
|
||||||
Tags: http, routing, tracker
|
Tags: http, routing, tracker
|
||||||
"""
|
"""
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
try:
|
try:
|
||||||
@@ -177,7 +182,7 @@ def test_quantization_coverage_uses_higher_precision_shards():
|
|||||||
|
|
||||||
def test_network_map_exposes_pool_size_and_speed_summary():
|
def test_network_map_exposes_pool_size_and_speed_summary():
|
||||||
"Network map exposes pool size and speed summary\n\nTags: http, routing, tracker"
|
"Network map exposes pool size and speed summary\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
try:
|
try:
|
||||||
@@ -219,7 +224,7 @@ def test_network_map_exposes_pool_size_and_speed_summary():
|
|||||||
|
|
||||||
def test_network_map_exposes_served_model_copy_count():
|
def test_network_map_exposes_served_model_copy_count():
|
||||||
"Network map exposes served model copy count\n\nTags: http, routing, tracker"
|
"Network map exposes served model copy count\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
try:
|
try:
|
||||||
@@ -273,7 +278,7 @@ def test_network_map_exposes_served_model_copy_count():
|
|||||||
|
|
||||||
def test_recommended_kimi_becomes_deployable_when_pool_is_large_enough():
|
def test_recommended_kimi_becomes_deployable_when_pool_is_large_enough():
|
||||||
"Recommended kimi becomes deployable when pool is large enough\n\nTags: http, routing, tracker"
|
"Recommended kimi becomes deployable when pool is large enough\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
gib = 1024 * 1024 * 1024
|
gib = 1024 * 1024 * 1024
|
||||||
@@ -302,7 +307,7 @@ def test_recommended_kimi_becomes_deployable_when_pool_is_large_enough():
|
|||||||
|
|
||||||
def test_network_assign_can_start_recommended_kimi_by_short_name():
|
def test_network_assign_can_start_recommended_kimi_by_short_name():
|
||||||
"Network assign can start recommended kimi by short name\n\nTags: http, routing, tracker"
|
"Network assign can start recommended kimi by short name\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
url = f"http://127.0.0.1:{port}"
|
url = f"http://127.0.0.1:{port}"
|
||||||
try:
|
try:
|
||||||
@@ -344,7 +349,7 @@ def test_tracker_serves_health_while_proxy_request_is_in_flight():
|
|||||||
slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
|
slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
|
||||||
slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
|
slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
|
||||||
slow_thread.start()
|
slow_thread.start()
|
||||||
tracker = TrackerServer(heartbeat_timeout=60.0)
|
tracker = _tracker(heartbeat_timeout=60.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
proxy_error = []
|
proxy_error = []
|
||||||
try:
|
try:
|
||||||
@@ -410,7 +415,7 @@ def test_tracker_route_log_counts_proxy_inflight_requests():
|
|||||||
slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
|
slow_node = http.server.HTTPServer(("127.0.0.1", 0), SlowChatHandler)
|
||||||
slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
|
slow_thread = threading.Thread(target=slow_node.serve_forever, daemon=True)
|
||||||
slow_thread.start()
|
slow_thread.start()
|
||||||
tracker = TrackerServer(heartbeat_timeout=60.0)
|
tracker = _tracker(heartbeat_timeout=60.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
errors = []
|
errors = []
|
||||||
try:
|
try:
|
||||||
@@ -498,7 +503,7 @@ def test_tracker_logs_stream_progress_before_request_completes():
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer(heartbeat_timeout=60.0)
|
tracker = _tracker(heartbeat_timeout=60.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
response = None
|
response = None
|
||||||
try:
|
try:
|
||||||
@@ -590,7 +595,7 @@ def test_tracker_stream_survives_idle_gap_between_sse_chunks():
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), IdleStreamingChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), IdleStreamingChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer(heartbeat_timeout=60.0)
|
tracker = _tracker(heartbeat_timeout=60.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
response = None
|
response = None
|
||||||
try:
|
try:
|
||||||
@@ -657,7 +662,7 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), StreamingChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer(heartbeat_timeout=60.0)
|
tracker = _tracker(heartbeat_timeout=60.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
response = None
|
response = None
|
||||||
request_id = None
|
request_id = None
|
||||||
@@ -723,7 +728,7 @@ def test_tracker_dashboard_can_cancel_inflight_proxy():
|
|||||||
|
|
||||||
def test_tracker_routes_hf_model_alias_from_quickstart():
|
def test_tracker_routes_hf_model_alias_from_quickstart():
|
||||||
"The documented qwen2.5-0.5b alias resolves a full HF repo registration.\n\nTags: http, routing, tracker"
|
"The documented qwen2.5-0.5b alias resolves a full HF repo registration.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -775,7 +780,7 @@ def test_tracker_proxy_accepts_hf_model_alias_from_quickstart():
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -833,7 +838,7 @@ def test_tracker_proxy_routes_split_preset_nodes_by_alias():
|
|||||||
head = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
|
head = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
|
||||||
head_thread = threading.Thread(target=head.serve_forever, daemon=True)
|
head_thread = threading.Thread(target=head.serve_forever, daemon=True)
|
||||||
head_thread.start()
|
head_thread.start()
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"qwen3.6-35b-a3b": {
|
"qwen3.6-35b-a3b": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": 39,
|
"layers_end": 39,
|
||||||
@@ -889,7 +894,7 @@ def test_tracker_proxy_routes_split_preset_nodes_by_alias():
|
|||||||
|
|
||||||
def test_tracker_route_endpoint_routes_split_preset_nodes_by_alias():
|
def test_tracker_route_endpoint_routes_split_preset_nodes_by_alias():
|
||||||
"Tracker route endpoint routes split preset nodes by alias\n\nTags: http, routing, tracker"
|
"Tracker route endpoint routes split preset nodes by alias\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"qwen3.6-35b-a3b": {
|
"qwen3.6-35b-a3b": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": 39,
|
"layers_end": 39,
|
||||||
@@ -936,7 +941,7 @@ def test_tracker_route_endpoint_routes_split_preset_nodes_by_alias():
|
|||||||
|
|
||||||
def test_tracker_route_endpoint_ignores_model_case_and_outer_whitespace():
|
def test_tracker_route_endpoint_ignores_model_case_and_outer_whitespace():
|
||||||
"Tracker route endpoint ignores model case and outer whitespace\n\nTags: http, routing, tracker"
|
"Tracker route endpoint ignores model case and outer whitespace\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"qwen3.6-35b-a3b": {
|
"qwen3.6-35b-a3b": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": 39,
|
"layers_end": 39,
|
||||||
@@ -970,7 +975,7 @@ def test_tracker_route_endpoint_ignores_model_case_and_outer_whitespace():
|
|||||||
|
|
||||||
def test_tracker_route_prefers_distributed_over_single_full_shard():
|
def test_tracker_route_prefers_distributed_over_single_full_shard():
|
||||||
"When a full 0-39 node and a partial 0-21 head coexist, /v1/route\n\nTags: http, routing, tracker"
|
"When a full 0-39 node and a partial 0-21 head coexist, /v1/route\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"qwen3.6-35b-a3b": {
|
"qwen3.6-35b-a3b": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": 39,
|
"layers_end": 39,
|
||||||
@@ -1044,7 +1049,7 @@ def test_tracker_proxy_ignores_model_case_and_outer_whitespace():
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), ChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"qwen3.6-35b-a3b": {
|
"qwen3.6-35b-a3b": {
|
||||||
"layers_start": 0,
|
"layers_start": 0,
|
||||||
"layers_end": 39,
|
"layers_end": 39,
|
||||||
@@ -1083,7 +1088,7 @@ def test_tracker_proxy_ignores_model_case_and_outer_whitespace():
|
|||||||
|
|
||||||
def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix():
|
def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix():
|
||||||
"Tracker registration node id includes wallet prefix and stable suffix\n\nTags: http, routing, security, tracker, wallet"
|
"Tracker registration node id includes wallet prefix and stable suffix\n\nTags: http, routing, security, tracker, wallet"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
wallet = "7j77FsPY1evV8tuf4Z73AVrWwxBEW1pvKwi4EvcRD3g"
|
wallet = "7j77FsPY1evV8tuf4Z73AVrWwxBEW1pvKwi4EvcRD3g"
|
||||||
try:
|
try:
|
||||||
@@ -1124,7 +1129,7 @@ def test_tracker_registration_node_id_includes_wallet_prefix_and_stable_suffix()
|
|||||||
|
|
||||||
def test_tracker_node_registration():
|
def test_tracker_node_registration():
|
||||||
"A node can register with the tracker and receives a node_id.\n\nTags: http, routing, tracker"
|
"A node can register with the tracker and receives a node_id.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
resp = _post_json(
|
resp = _post_json(
|
||||||
@@ -1146,7 +1151,7 @@ def test_tracker_node_registration():
|
|||||||
|
|
||||||
def test_tracker_route_selection():
|
def test_tracker_route_selection():
|
||||||
"Tracker returns ordered route when nodes collectively cover all layers.\n\nTags: http, routing, tracker"
|
"Tracker returns ordered route when nodes collectively cover all layers.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -1174,7 +1179,7 @@ def test_tracker_route_selection():
|
|||||||
|
|
||||||
def test_tracker_route_error_no_coverage():
|
def test_tracker_route_error_no_coverage():
|
||||||
"Tracker returns 503 when registered nodes do not cover all required layers.\n\nTags: http, routing, tracker"
|
"Tracker returns 503 when registered nodes do not cover all required layers.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
# Only half the layers covered — route must fail.
|
# Only half the layers covered — route must fail.
|
||||||
@@ -1197,7 +1202,7 @@ def test_tracker_route_error_no_coverage():
|
|||||||
|
|
||||||
def test_tracker_coverage_endpoint_reports_uncovered_ranges():
|
def test_tracker_coverage_endpoint_reports_uncovered_ranges():
|
||||||
"Coverage endpoint returns compressed layer ranges with node counts.\n\nTags: http, routing, tracker"
|
"Coverage endpoint returns compressed layer ranges with node counts.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 6,
|
"total_layers": 6,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1242,7 +1247,7 @@ def test_tracker_coverage_endpoint_reports_uncovered_ranges():
|
|||||||
|
|
||||||
def test_tracker_coverage_endpoint_accepts_registered_hf_repo_or_short_name():
|
def test_tracker_coverage_endpoint_accepts_registered_hf_repo_or_short_name():
|
||||||
"Coverage endpoint supports real HF models registered outside preset catalog.\n\nTags: http, routing, tracker"
|
"Coverage endpoint supports real HF models registered outside preset catalog.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -1279,7 +1284,7 @@ def test_tracker_coverage_endpoint_accepts_registered_hf_repo_or_short_name():
|
|||||||
|
|
||||||
def test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias():
|
def test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias():
|
||||||
"Tracker models endpoint lists registered hf repo and short name alias\n\nTags: http, routing, tracker"
|
"Tracker models endpoint lists registered hf repo and short name alias\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -1302,7 +1307,7 @@ def test_tracker_models_endpoint_lists_registered_hf_repo_and_short_name_alias()
|
|||||||
|
|
||||||
def test_tracker_auto_assigns_new_node_to_uncovered_range_first():
|
def test_tracker_auto_assigns_new_node_to_uncovered_range_first():
|
||||||
"Capability-driven registration fills the first uncovered layer gap.\n\nTags: http, routing, tracker"
|
"Capability-driven registration fills the first uncovered layer gap.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 8,
|
"total_layers": 8,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1331,7 +1336,7 @@ def test_tracker_auto_assigns_new_node_to_uncovered_range_first():
|
|||||||
|
|
||||||
def test_tracker_speed_weighted_vram_assignment_covers_model():
|
def test_tracker_speed_weighted_vram_assignment_covers_model():
|
||||||
"Three auto-assigned nodes reach 100% coverage with widest range on largest VRAM.\n\nTags: http, routing, tracker"
|
"Three auto-assigned nodes reach 100% coverage with widest range on largest VRAM.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 12,
|
"total_layers": 12,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1375,7 +1380,7 @@ def test_tracker_speed_weighted_vram_assignment_covers_model():
|
|||||||
|
|
||||||
def test_tracker_speed_is_primary_when_both_nodes_can_cover_gap():
|
def test_tracker_speed_is_primary_when_both_nodes_can_cover_gap():
|
||||||
"Tracker speed is primary when both nodes can cover gap\n\nTags: http, routing, tracker"
|
"Tracker speed is primary when both nodes can cover gap\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 12,
|
"total_layers": 12,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1410,7 +1415,7 @@ def test_tracker_speed_is_primary_when_both_nodes_can_cover_gap():
|
|||||||
|
|
||||||
def test_tracker_registration_directive_is_not_replayed_on_heartbeat():
|
def test_tracker_registration_directive_is_not_replayed_on_heartbeat():
|
||||||
"Tracker registration directive is not replayed on heartbeat\n\nTags: http, routing, tracker"
|
"Tracker registration directive is not replayed on heartbeat\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 2,
|
"total_layers": 2,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1434,7 +1439,7 @@ def test_tracker_registration_directive_is_not_replayed_on_heartbeat():
|
|||||||
|
|
||||||
def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
|
def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
|
||||||
"A second managed node with capacity for the full model keeps the first copy intact.\n\nTags: http, routing, tracker"
|
"A second managed node with capacity for the full model keeps the first copy intact.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 4,
|
"total_layers": 4,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1472,7 +1477,7 @@ def test_tracker_pool_join_adds_redundant_copy_without_splitting_incumbent():
|
|||||||
|
|
||||||
def test_tracker_explicit_full_copy_join_preserves_existing_serving_node():
|
def test_tracker_explicit_full_copy_join_preserves_existing_serving_node():
|
||||||
"--model style joins with explicit shards add redundancy instead of reshuffling.\n\nTags: http, routing, tracker"
|
"--model style joins with explicit shards add redundancy instead of reshuffling.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(heartbeat_timeout=10.0)
|
tracker = _tracker(heartbeat_timeout=10.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
base_reg = {
|
base_reg = {
|
||||||
@@ -1511,7 +1516,7 @@ def test_tracker_explicit_full_copy_join_preserves_existing_serving_node():
|
|||||||
|
|
||||||
def test_tracker_periodic_rebalance_purges_expired_nodes_without_requests():
|
def test_tracker_periodic_rebalance_purges_expired_nodes_without_requests():
|
||||||
"Tracker periodic rebalance purges expired nodes without requests\n\nTags: http, routing, tracker"
|
"Tracker periodic rebalance purges expired nodes without requests\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(
|
tracker = _tracker(
|
||||||
heartbeat_timeout=0.05,
|
heartbeat_timeout=0.05,
|
||||||
rebalance_interval=0.02,
|
rebalance_interval=0.02,
|
||||||
model_presets={"tiny-model": {"total_layers": 1, "bytes_per_layer": {"bfloat16": 1_000}}},
|
model_presets={"tiny-model": {"total_layers": 1, "bytes_per_layer": {"bfloat16": 1_000}}},
|
||||||
@@ -1533,7 +1538,7 @@ def test_tracker_periodic_rebalance_purges_expired_nodes_without_requests():
|
|||||||
|
|
||||||
def test_tracker_faster_node_receives_wider_range_when_capacity_ties():
|
def test_tracker_faster_node_receives_wider_range_when_capacity_ties():
|
||||||
"Tracker faster node receives wider range when capacity ties\n\nTags: http, routing, tracker"
|
"Tracker faster node receives wider range when capacity ties\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 12,
|
"total_layers": 12,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1564,7 +1569,7 @@ def test_tracker_faster_node_receives_wider_range_when_capacity_ties():
|
|||||||
|
|
||||||
def test_tracker_rebalances_after_middle_range_node_timeout():
|
def test_tracker_rebalances_after_middle_range_node_timeout():
|
||||||
"Killing a middle shard queues LOAD_SHARD and restores coverage.\n\nTags: http, routing, tracker"
|
"Killing a middle shard queues LOAD_SHARD and restores coverage.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(
|
tracker = _tracker(
|
||||||
heartbeat_timeout=0.15,
|
heartbeat_timeout=0.15,
|
||||||
model_presets={
|
model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
@@ -1612,7 +1617,7 @@ def test_tracker_rebalances_after_middle_range_node_timeout():
|
|||||||
|
|
||||||
def test_tracker_rebalances_managed_hf_node_after_peer_timeout():
|
def test_tracker_rebalances_managed_hf_node_after_peer_timeout():
|
||||||
"HF nodes auto-assigned by the tracker receive LOAD_SHARD after a peer dies.\n\nTags: http, routing, tracker"
|
"HF nodes auto-assigned by the tracker receive LOAD_SHARD after a peer dies.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(heartbeat_timeout=0.15, rebalance_interval=10.0)
|
tracker = _tracker(heartbeat_timeout=0.15, rebalance_interval=10.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
managed = _post_json(
|
managed = _post_json(
|
||||||
@@ -1648,7 +1653,7 @@ def test_tracker_rebalances_managed_hf_node_after_peer_timeout():
|
|||||||
|
|
||||||
def test_tracker_route_error_no_nodes():
|
def test_tracker_route_error_no_nodes():
|
||||||
"Tracker returns 503 with clear error when the registry is empty.\n\nTags: http, routing, tracker"
|
"Tracker returns 503 with clear error when the registry is empty.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
@@ -1664,7 +1669,7 @@ def test_tracker_route_error_no_nodes():
|
|||||||
|
|
||||||
def test_tracker_heartbeat_updates_node():
|
def test_tracker_heartbeat_updates_node():
|
||||||
"Sending a heartbeat for a registered node succeeds.\n\nTags: http, routing, tracker"
|
"Sending a heartbeat for a registered node succeeds.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
reg = _post_json(
|
reg = _post_json(
|
||||||
@@ -1684,7 +1689,7 @@ def test_tracker_heartbeat_updates_node():
|
|||||||
|
|
||||||
def test_tracker_heartbeat_stores_current_requests():
|
def test_tracker_heartbeat_stores_current_requests():
|
||||||
"Node-reported in-flight request snapshots appear on the network map.\n\nTags: http, routing, tracker"
|
"Node-reported in-flight request snapshots appear on the network map.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
reg = _post_json(
|
reg = _post_json(
|
||||||
@@ -1749,7 +1754,7 @@ def test_normalize_current_requests_sanitizes_payload():
|
|||||||
|
|
||||||
def test_tracker_heartbeat_expiry():
|
def test_tracker_heartbeat_expiry():
|
||||||
"Nodes that miss their heartbeat window are excluded from routes.\n\nTags: http, routing, tracker"
|
"Nodes that miss their heartbeat window are excluded from routes.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(heartbeat_timeout=0.05) # 50 ms
|
tracker = _tracker(heartbeat_timeout=0.05) # 50 ms
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -1778,7 +1783,7 @@ def test_tracker_heartbeat_expiry():
|
|||||||
|
|
||||||
def test_tracker_heartbeat_expiry_removes_node_from_registry():
|
def test_tracker_heartbeat_expiry_removes_node_from_registry():
|
||||||
"Expired nodes are removed, not merely hidden from route responses.\n\nTags: http, routing, tracker"
|
"Expired nodes are removed, not merely hidden from route responses.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(heartbeat_timeout=0.05)
|
tracker = _tracker(heartbeat_timeout=0.05)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
reg = _post_json(
|
reg = _post_json(
|
||||||
@@ -1806,7 +1811,7 @@ def test_tracker_heartbeat_expiry_removes_node_from_registry():
|
|||||||
|
|
||||||
def test_tracker_route_rejects_non_extending_overlap():
|
def test_tracker_route_rejects_non_extending_overlap():
|
||||||
"Overlapping shards that do not extend coverage cannot form a complete route.\n\nTags: http, routing, tracker"
|
"Overlapping shards that do not extend coverage cannot form a complete route.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -1831,7 +1836,7 @@ def test_tracker_route_rejects_non_extending_overlap():
|
|||||||
|
|
||||||
def test_tracker_console_records_model_not_available():
|
def test_tracker_console_records_model_not_available():
|
||||||
"Tracker console records model not available\n\nTags: http, routing, tracker"
|
"Tracker console records model not available\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
@@ -1856,7 +1861,7 @@ def test_tracker_console_records_model_not_available():
|
|||||||
|
|
||||||
def test_tracker_registration_rejects_invalid_payload():
|
def test_tracker_registration_rejects_invalid_payload():
|
||||||
"Registration errors return a defined JSON 400 response.\n\nTags: http, routing, tracker"
|
"Registration errors return a defined JSON 400 response.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
@@ -1876,7 +1881,7 @@ def test_tracker_registration_rejects_invalid_payload():
|
|||||||
|
|
||||||
def test_tracker_registration_rejects_invalid_max_loaded_shards():
|
def test_tracker_registration_rejects_invalid_max_loaded_shards():
|
||||||
"Tracker registration rejects invalid max loaded shards\n\nTags: http, routing, tracker"
|
"Tracker registration rejects invalid max loaded shards\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
@@ -1896,7 +1901,7 @@ def test_tracker_registration_rejects_invalid_max_loaded_shards():
|
|||||||
|
|
||||||
def test_network_map_exposes_node_capacity_limits():
|
def test_network_map_exposes_node_capacity_limits():
|
||||||
"Network map exposes node capacity limits\n\nTags: http, routing, tracker"
|
"Network map exposes node capacity limits\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 12,
|
"total_layers": 12,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1927,7 +1932,7 @@ def test_network_map_exposes_node_capacity_limits():
|
|||||||
|
|
||||||
def test_tracker_capacity_uses_ram_when_node_has_no_vram():
|
def test_tracker_capacity_uses_ram_when_node_has_no_vram():
|
||||||
"CPU-only nodes should expose RAM-backed shard capacity, not default GPU capacity.\n\nTags: http, routing, tracker"
|
"CPU-only nodes should expose RAM-backed shard capacity, not default GPU capacity.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 20,
|
"total_layers": 20,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1956,7 +1961,7 @@ def test_tracker_capacity_uses_ram_when_node_has_no_vram():
|
|||||||
|
|
||||||
def test_rebalance_keeps_one_active_range_even_when_multiple_slots_advertised():
|
def test_rebalance_keeps_one_active_range_even_when_multiple_slots_advertised():
|
||||||
"max_loaded_shards is exposed but reserved until node runtime supports multi-range serving.\n\nTags: http, routing, tracker"
|
"max_loaded_shards is exposed but reserved until node runtime supports multi-range serving.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 8,
|
"total_layers": 8,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -1983,7 +1988,7 @@ def test_rebalance_keeps_one_active_range_even_when_multiple_slots_advertised():
|
|||||||
|
|
||||||
def test_tracker_routes_only_nodes_for_requested_model():
|
def test_tracker_routes_only_nodes_for_requested_model():
|
||||||
"A node registered for one model cannot satisfy another model's route.\n\nTags: http, routing, tracker"
|
"A node registered for one model cannot satisfy another model's route.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"model-a": {"layers_start": 0, "layers_end": 31},
|
"model-a": {"layers_start": 0, "layers_end": 31},
|
||||||
"model-b": {"layers_start": 0, "layers_end": 31},
|
"model-b": {"layers_start": 0, "layers_end": 31},
|
||||||
})
|
})
|
||||||
@@ -2006,7 +2011,7 @@ def test_tracker_routes_only_nodes_for_requested_model():
|
|||||||
|
|
||||||
def test_tracker_assignment_coverage_is_model_scoped():
|
def test_tracker_assignment_coverage_is_model_scoped():
|
||||||
"Shard assignment gaps are computed only from nodes serving the same model.\n\nTags: http, routing, tracker"
|
"Shard assignment gaps are computed only from nodes serving the same model.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"model-a": {"layers_start": 0, "layers_end": 31},
|
"model-a": {"layers_start": 0, "layers_end": 31},
|
||||||
"model-b": {"layers_start": 0, "layers_end": 31},
|
"model-b": {"layers_start": 0, "layers_end": 31},
|
||||||
})
|
})
|
||||||
@@ -2031,7 +2036,7 @@ def test_tracker_excludes_banned_wallets_from_routes():
|
|||||||
contracts.registry.submit_stake("wallet-a", 1_000)
|
contracts.registry.submit_stake("wallet-a", 1_000)
|
||||||
contracts.registry.submit_stake("wallet-b", 1_000)
|
contracts.registry.submit_stake("wallet-b", 1_000)
|
||||||
contracts.registry.ban_wallet("wallet-b")
|
contracts.registry.ban_wallet("wallet-b")
|
||||||
tracker = TrackerServer(contracts=contracts, minimum_stake=100)
|
tracker = _tracker(contracts=contracts, minimum_stake=100)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_post_json(
|
_post_json(
|
||||||
@@ -2065,7 +2070,7 @@ def test_tracker_rejects_banned_wallet_registration():
|
|||||||
contracts = LocalSolanaContracts()
|
contracts = LocalSolanaContracts()
|
||||||
contracts.registry.submit_stake("wallet-b", 1_000)
|
contracts.registry.submit_stake("wallet-b", 1_000)
|
||||||
contracts.registry.ban_wallet("wallet-b")
|
contracts.registry.ban_wallet("wallet-b")
|
||||||
tracker = TrackerServer(contracts=contracts, minimum_stake=100)
|
tracker = _tracker(contracts=contracts, minimum_stake=100)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
@@ -2111,7 +2116,7 @@ def test_gateway_routes_even_when_compute_node_has_low_stake():
|
|||||||
contracts = LocalSolanaContracts()
|
contracts = LocalSolanaContracts()
|
||||||
contracts.registry.submit_stake("wallet-a", 1_000)
|
contracts.registry.submit_stake("wallet-a", 1_000)
|
||||||
contracts.registry.submit_stake("wallet-b", 5)
|
contracts.registry.submit_stake("wallet-b", 5)
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
||||||
|
|
||||||
@@ -2166,7 +2171,7 @@ def test_gateway_records_compute_attribution_after_inference_session():
|
|||||||
contracts.registry.submit_stake("wallet-a", 1_000)
|
contracts.registry.submit_stake("wallet-a", 1_000)
|
||||||
contracts.registry.submit_stake("wallet-b", 1_000)
|
contracts.registry.submit_stake("wallet-b", 1_000)
|
||||||
contracts.payment.fund_api_key("api-key-a", lamports=10_000)
|
contracts.payment.fund_api_key("api-key-a", lamports=10_000)
|
||||||
tracker = TrackerServer(contracts=contracts, minimum_stake=100)
|
tracker = _tracker(contracts=contracts, minimum_stake=100)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
||||||
|
|
||||||
@@ -2294,7 +2299,7 @@ def test_gateway_returns_json_503_for_malformed_tracker_route():
|
|||||||
|
|
||||||
def test_two_node_pipeline_via_tracker():
|
def test_two_node_pipeline_via_tracker():
|
||||||
"End-to-end: nodes register with tracker; gateway discovers route dynamically.\n\nTags: http, routing, tracker"
|
"End-to-end: nodes register with tracker; gateway discovers route dynamically.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
tracker_url = f"http://127.0.0.1:{tracker_port}"
|
||||||
|
|
||||||
@@ -2351,7 +2356,7 @@ def test_two_node_pipeline_via_tracker():
|
|||||||
|
|
||||||
def test_stats_endpoint_returns_empty_models_initially():
|
def test_stats_endpoint_returns_empty_models_initially():
|
||||||
"Stats endpoint returns empty models initially\n\nTags: http, routing, tracker"
|
"Stats endpoint returns empty models initially\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
try:
|
try:
|
||||||
resp = _get_json(f"http://127.0.0.1:{port}/v1/stats")
|
resp = _get_json(f"http://127.0.0.1:{port}/v1/stats")
|
||||||
@@ -2449,7 +2454,7 @@ def test_stats_sqlite_persistence_survives_restart(tmp_path):
|
|||||||
|
|
||||||
def test_stats_gossip_endpoint_merges_peer_slice():
|
def test_stats_gossip_endpoint_merges_peer_slice():
|
||||||
"Stats gossip endpoint merges peer slice\n\nTags: gossip, http, network, routing, tracker"
|
"Stats gossip endpoint merges peer slice\n\nTags: gossip, http, network, routing, tracker"
|
||||||
tracker = TrackerServer(hive_secret=_TEST_HIVE_SECRET)
|
tracker = _tracker(hive_secret=_TEST_HIVE_SECRET)
|
||||||
port = tracker.start()
|
port = tracker.start()
|
||||||
try:
|
try:
|
||||||
peer_payload = {
|
peer_payload = {
|
||||||
@@ -2610,7 +2615,7 @@ def test_select_route_reputation_never_overrides_coverage():
|
|||||||
|
|
||||||
def test_two_stub_nodes_complete_pipeline_via_tracker():
|
def test_two_stub_nodes_complete_pipeline_via_tracker():
|
||||||
"Integration: two StubNodeServer instances serving complementary shards\n\nTags: http, routing, tracker"
|
"Integration: two StubNodeServer instances serving complementary shards\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"two-node-model": {"layers_start": 0, "layers_end": 3}
|
"two-node-model": {"layers_start": 0, "layers_end": 3}
|
||||||
})
|
})
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
@@ -2732,7 +2737,7 @@ def test_shard_heal_cycle_surviving_node_covers_dead_peers_gap(monkeypatch):
|
|||||||
monkeypatch.setattr(torch_server, "_load_backend", fake_load)
|
monkeypatch.setattr(torch_server, "_load_backend", fake_load)
|
||||||
|
|
||||||
# Use a very short timeout so Node A expires quickly.
|
# Use a very short timeout so Node A expires quickly.
|
||||||
tracker = TrackerServer(heartbeat_timeout=0.15, rebalance_interval=10.0)
|
tracker = _tracker(heartbeat_timeout=0.15, rebalance_interval=10.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
|
|
||||||
node_b = TorchNodeServer(backend=_FakeBackend(shard_start=12, shard_end=23))
|
node_b = TorchNodeServer(backend=_FakeBackend(shard_start=12, shard_end=23))
|
||||||
@@ -2807,7 +2812,7 @@ def test_shard_heal_cycle_surviving_node_covers_dead_peers_gap(monkeypatch):
|
|||||||
|
|
||||||
def test_network_map_exposes_memory_pool():
|
def test_network_map_exposes_memory_pool():
|
||||||
"Network map exposes memory pool\n\nTags: http, routing, tracker"
|
"Network map exposes memory pool\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"tiny-model": {
|
"tiny-model": {
|
||||||
"total_layers": 8,
|
"total_layers": 8,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -2836,7 +2841,7 @@ def test_network_map_exposes_memory_pool():
|
|||||||
|
|
||||||
def test_same_endpoint_can_register_multiple_models():
|
def test_same_endpoint_can_register_multiple_models():
|
||||||
"Same endpoint can register multiple models\n\nTags: http, routing, tracker"
|
"Same endpoint can register multiple models\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
base = {
|
base = {
|
||||||
@@ -2864,9 +2869,46 @@ def test_same_endpoint_can_register_multiple_models():
|
|||||||
tracker.stop()
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def test_explicit_model_placement_targets_only_the_selected_node():
|
||||||
|
"An admin can add and release a model on one chosen multi-model node.\n\nTags: http, routing, tracker"
|
||||||
|
from meshnet_tracker.server import _release_model_locked, _request_model_load_locked
|
||||||
|
|
||||||
|
tracker = _tracker(model_presets={
|
||||||
|
"model-a": {"total_layers": 4, "bytes_per_layer": {"bfloat16": 1_000}, "hf_repo": "org/ModelA"},
|
||||||
|
"model-b": {"total_layers": 4, "bytes_per_layer": {"bfloat16": 1_000}, "hf_repo": "org/ModelB"},
|
||||||
|
})
|
||||||
|
tracker_port = tracker.start()
|
||||||
|
try:
|
||||||
|
registrations = []
|
||||||
|
for port in (9062, 9063):
|
||||||
|
registrations.append(_post_json(
|
||||||
|
f"http://127.0.0.1:{tracker_port}/v1/nodes/register",
|
||||||
|
{"endpoint": f"http://127.0.0.1:{port}", "model": "model-a", "hf_repo": "org/ModelA",
|
||||||
|
"num_layers": 4, "shard_start": 0, "shard_end": 3, "max_loaded_shards": 2,
|
||||||
|
"vram_bytes": 20_000, "ram_bytes": 20_000, "quantizations": ["bfloat16"],
|
||||||
|
"benchmark_tokens_per_sec": 1.0, "hardware_profile": {}, "score": 1.0},
|
||||||
|
))
|
||||||
|
selected, other = (item["node_id"] for item in registrations)
|
||||||
|
|
||||||
|
with tracker._lock:
|
||||||
|
assignment = _request_model_load_locked(tracker._server, "model-b", selected) # type: ignore[arg-type]
|
||||||
|
assert assignment is not None
|
||||||
|
assert assignment["node_id"] == selected
|
||||||
|
assert tracker._registry[selected].pending_new_assignment is not None
|
||||||
|
assert tracker._registry[other].pending_new_assignment is None
|
||||||
|
|
||||||
|
with tracker._lock:
|
||||||
|
released = _release_model_locked(tracker._server, "model-a", selected) # type: ignore[arg-type]
|
||||||
|
assert released == 1
|
||||||
|
assert len(tracker._registry[selected].pending_directives) == 2
|
||||||
|
assert tracker._registry[other].pending_directives == []
|
||||||
|
finally:
|
||||||
|
tracker.stop()
|
||||||
|
|
||||||
|
|
||||||
def test_scale_demanded_models_queues_add_shard_on_spare_host():
|
def test_scale_demanded_models_queues_add_shard_on_spare_host():
|
||||||
"Scale demanded models queues add shard on spare host\n\nTags: http, routing, tracker"
|
"Scale demanded models queues add shard on spare host\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(model_presets={
|
tracker = _tracker(model_presets={
|
||||||
"model-a": {
|
"model-a": {
|
||||||
"total_layers": 4,
|
"total_layers": 4,
|
||||||
"bytes_per_layer": {"bfloat16": 1_000},
|
"bytes_per_layer": {"bfloat16": 1_000},
|
||||||
@@ -2957,7 +2999,7 @@ def _proxy_chat_status(policy: str, quantization: str | None) -> tuple[int, dict
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), _EchoChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), _EchoChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer(capability_policy=policy)
|
tracker = _tracker(capability_policy=policy)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
endpoint = f"http://127.0.0.1:{node.server_address[1]}"
|
endpoint = f"http://127.0.0.1:{node.server_address[1]}"
|
||||||
@@ -3005,7 +3047,7 @@ def test_a_node_declaring_a_null_quantization_is_never_routed():
|
|||||||
node = http.server.HTTPServer(("127.0.0.1", 0), _EchoChatHandler)
|
node = http.server.HTTPServer(("127.0.0.1", 0), _EchoChatHandler)
|
||||||
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
node_thread = threading.Thread(target=node.serve_forever, daemon=True)
|
||||||
node_thread.start()
|
node_thread.start()
|
||||||
tracker = TrackerServer(capability_policy=POLICY_COMPAT)
|
tracker = _tracker(capability_policy=POLICY_COMPAT)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
endpoint = f"http://127.0.0.1:{node.server_address[1]}"
|
endpoint = f"http://127.0.0.1:{node.server_address[1]}"
|
||||||
@@ -3028,7 +3070,7 @@ def test_a_node_declaring_a_null_quantization_is_never_routed():
|
|||||||
|
|
||||||
def test_raft_apply_preserves_a_declared_precision_for_follower_routing():
|
def test_raft_apply_preserves_a_declared_precision_for_follower_routing():
|
||||||
"A follower that drops the declared precision would route unlike the leader.\n\nTags: raft, routing, tracker"
|
"A follower that drops the declared precision would route unlike the leader.\n\nTags: raft, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker.start()
|
tracker.start()
|
||||||
try:
|
try:
|
||||||
tracker._raft_apply("register", {
|
tracker._raft_apply("register", {
|
||||||
@@ -3051,7 +3093,7 @@ def test_raft_apply_preserves_a_declared_precision_for_follower_routing():
|
|||||||
|
|
||||||
def test_raft_apply_grandfathers_a_legacy_registration_that_omits_quantization():
|
def test_raft_apply_grandfathers_a_legacy_registration_that_omits_quantization():
|
||||||
"The follower must reach the same legacy fallback the leader did, not a bare None.\n\nTags: raft, routing, tracker"
|
"The follower must reach the same legacy fallback the leader did, not a bare None.\n\nTags: raft, routing, tracker"
|
||||||
tracker = TrackerServer()
|
tracker = _tracker()
|
||||||
tracker.start()
|
tracker.start()
|
||||||
try:
|
try:
|
||||||
tracker._raft_apply("register", {
|
tracker._raft_apply("register", {
|
||||||
@@ -3071,7 +3113,7 @@ def test_raft_apply_grandfathers_a_legacy_registration_that_omits_quantization()
|
|||||||
|
|
||||||
def test_rebalancing_does_not_resurrect_a_node_that_declared_an_unusable_precision():
|
def test_rebalancing_does_not_resurrect_a_node_that_declared_an_unusable_precision():
|
||||||
"Managed assignment picks a node's precision, so it must not overwrite an explicit unusable one.\n\nTags: http, routing, tracker"
|
"Managed assignment picks a node's precision, so it must not overwrite an explicit unusable one.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(heartbeat_timeout=0.15, rebalance_interval=10.0)
|
tracker = _tracker(heartbeat_timeout=0.15, rebalance_interval=10.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
managed = _post_json(
|
managed = _post_json(
|
||||||
@@ -3120,7 +3162,7 @@ def _register_shard(tracker_port: int, port: int, **overrides) -> dict:
|
|||||||
|
|
||||||
def test_a_node_with_an_unusable_precision_covers_no_layers():
|
def test_a_node_with_an_unusable_precision_covers_no_layers():
|
||||||
"A node no request can reach must not report its shards as served coverage.\n\nTags: http, routing, tracker"
|
"A node no request can reach must not report its shards as served coverage.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(rebalance_interval=10.0)
|
tracker = _tracker(rebalance_interval=10.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
_register_shard(
|
_register_shard(
|
||||||
@@ -3136,7 +3178,7 @@ def test_a_node_with_an_unusable_precision_covers_no_layers():
|
|||||||
|
|
||||||
def test_rebalancing_fills_the_layers_an_unusable_node_only_appears_to_cover():
|
def test_rebalancing_fills_the_layers_an_unusable_node_only_appears_to_cover():
|
||||||
"Counting a dark node as coverage hides the gap, so the rebalancer never fills it.\n\nTags: http, routing, tracker"
|
"Counting a dark node as coverage hides the gap, so the rebalancer never fills it.\n\nTags: http, routing, tracker"
|
||||||
tracker = TrackerServer(rebalance_interval=10.0)
|
tracker = _tracker(rebalance_interval=10.0)
|
||||||
tracker_port = tracker.start()
|
tracker_port = tracker.start()
|
||||||
try:
|
try:
|
||||||
# The dark node claims the model's tail; the managed node holds the head.
|
# The dark node claims the model's tail; the managed node holds the head.
|
||||||
|
|||||||
Reference in New Issue
Block a user