From 95245be51230b3f4e5dc7aaad547e76fa743b283 Mon Sep 17 00:00:00 2001 From: Dobromir Popov Date: Mon, 13 Jul 2026 18:14:21 +0200 Subject: [PATCH] documentation revision --- .claude/memory/MEMORY.md | 2 +- .claude/memory/alpha-hardening-navigation.md | 6 +++--- .claude/memory/project-status.md | 4 +++- .scratch/alpha-hardening/handoff.md | 4 ++-- .../issues/21-honest-noise-calibration-corpus.md | 2 +- .../distributed-gguf-runtime/RALPH-CONTEXT.md | 4 ++-- .scratch/distributed-gguf-runtime/README.md | 2 +- .../decision-framework.md | 2 +- ...fetensors-versus-gguf-performance-contract.md | 2 +- ...02-adopt-the-versioned-grpc-shard-protocol.md | 2 +- ...exact-artifact-and-runtime-recipe-identity.md | 2 +- ...-reproducible-pinned-llama-cpp-patch-stack.md | 2 +- ...ent-dense-llama-range-aware-gguf-ownership.md | 2 +- ...architecture-defined-boundary-input-output.md | 2 +- ...add-isolated-concurrent-local-hot-kv-state.md | 2 +- ...8-build-the-standalone-c-grpc-shard-worker.md | 2 +- ...9-integrate-the-native-worker-with-meshnet.md | 2 +- ...ss-local-real-model-two-process-acceptance.md | 2 +- ...ass-a-real-heterogeneous-two-machine-route.md | 2 +- ...-continuous-batching-and-bounded-admission.md | 2 +- ...failure-cancellation-and-restart-semantics.md | 2 +- ...e-the-gguf-versus-safetensors-release-gate.md | 2 +- ...-add-and-certify-a-qwen3-qwen3-moe-adapter.md | 2 +- ...e-upstream-llama-cpp-collaboration-package.md | 2 +- CONTEXT-MAP.md | 10 ++++++++++ docs/PRD.md | 16 ++++++++-------- docs/adr/0001-pytorch-over-llama-cpp.md | 2 ++ docs/adr/0003-optimistic-fraud-detection.md | 2 ++ ...g-live-progress-and-mixed-topology-routing.md | 4 ++-- .../0022-sharded-per-node-generation-cache.md | 4 +++- .../adr/0024-distributed-gguf-runtime.md | 6 +++++- docs/issues/01-monorepo-scaffold.md | 2 +- docs/issues/02-two-node-shard-pipeline.md | 2 +- .../03-tracker-registration-and-routing.md | 2 +- docs/issues/04-node-client-startup.md | 2 +- docs/issues/05-openai-compatible-gateway.md | 2 +- docs/issues/08-probationary-period-and-bans.md | 2 +- docs/issues/09-p2p-shard-swarm.md | 2 +- docs/issues/10-meshnet-sdk.md | 2 +- ...binary-data-plane-and-peer-weight-transfer.md | 4 +--- docs/issues/36-relay-streamed-chat.md | 4 +--- docs/issues/42-gguf-llamacpp-node-backend.md | 2 +- ...budget-shard-slots-and-dropout-relocation.md} | 7 +++++-- 43 files changed, 78 insertions(+), 57 deletions(-) create mode 100644 CONTEXT-MAP.md rename .scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md => docs/adr/0024-distributed-gguf-runtime.md (92%) rename docs/issues/{20-memory-budget-shard-slots-and-dropout-relocation.md => 48-memory-budget-shard-slots-and-dropout-relocation.md} (94%) diff --git a/.claude/memory/MEMORY.md b/.claude/memory/MEMORY.md index 3ae8bd0..bdeecd9 100644 --- a/.claude/memory/MEMORY.md +++ b/.claude/memory/MEMORY.md @@ -7,4 +7,4 @@ - [Alpha hardening navigation](alpha-hardening-navigation.md) — locked fraud/auth decisions, Bucket-1 order, handoff pointers - **Node capability admission** — `.scratch/node-capability-admission/` (P0 plan: generic doctor/real-forward validation, fail-closed readiness, tracker admission gate; [PRD](../../.scratch/node-capability-admission/PRD.md), [README](../../.scratch/node-capability-admission/README.md), ADR-0023) - **Distributed relay performance** — relay `/rpc` requester sockets are persistent per Route Session and Activation Seam as of 2026-07-10; `request_id` remains unique per activation while `X-Meshnet-Session` remains stable for KV state. Next low-risk priorities: persistent direct/loopback HTTP, seam byte/latency telemetry, then trace-driven zstd tuning. -- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md). +- **Distributed GGUF direction** — benchmark-gated native runtime: compare controlled Transformers/safetensors and whole-model llama.cpp lanes before expensive work; ship only for measured speed or model-fit advantage. Public parallelism is contiguous Shards in an Inference Route; concurrency comes from per-node continuous batching across isolated Route Sessions, while tensor/expert collectives stay inside optional trusted composite providers. Native data plane uses versioned Protobuf over long-lived gRPC/HTTP2 seam streams, with existing relay carrying the same opaque frames when needed. llama.cpp/GGML remains the substrate behind a project-owned standalone worker and small pinned fork; vLLM is an optional complete managed provider and concept donor, not a fork. Nakshatra, `prima.cpp`, `llama-gguf`, LiGGUF and historical GPUStack are source/test donors only. Active plan: [README](../../.scratch/distributed-gguf-runtime/README.md), [architecture](../../.scratch/distributed-gguf-runtime/architecture.md), [PRD](../../.scratch/distributed-gguf-runtime/PRD.md), [Ralph backlog](../../.scratch/distributed-gguf-runtime/prd.json). ADR: [0024](../../docs/adr/0024-distributed-gguf-runtime.md). Research: [landscape](../../docs/research/distributed-gguf-landscape.md), [GitHub follow-up](../../docs/research/distributed-gguf-github-followup.md), [vLLM](../../docs/research/vllm-distributed-gguf-assessment.md). diff --git a/.claude/memory/alpha-hardening-navigation.md b/.claude/memory/alpha-hardening-navigation.md index 4ce5bee..9e9ef77 100644 --- a/.claude/memory/alpha-hardening-navigation.md +++ b/.claude/memory/alpha-hardening-navigation.md @@ -20,9 +20,9 @@ Active workstream (started 2026-07-04): alpha hardening of the money/trust path. **Launch-readiness grilling (2026-07-06):** Locked launch plan — devnet dev/test run now, then **real mainnet SOL/USDT** (not devnet, not a new public token) for the first cohort: friends (API clients) + hired VPS/VPC hosts (our own test infra, not third-party volunteers — stake-free, risk-free if something breaks, not a long-term topology). Pricing: clients are the only party spending real money; nodes only accumulate off-chain credit and get paid in batches (30min dev / 24h later) — a failed distribution leaves funds parked, not lost, so mainnet-vs-devnet mixups are lower-risk than initially assumed. TAI token: do NOT issue/list now — ADR-0002 already locks listing behind $50k volume + 25 nodes/15 wallets plus an unresolved securities-review gate; only a dormant mainnet mint (cheap, ~few $ SOL) for name/branding reservation is in scope, bundled with treasury-key work, not before it. Treasury custody: bare keypair file (current runbook 02) is not acceptable for real funds — plan is **free native SPL multisig** (`spl-token create-multisig`, no protocol fee unlike Squads' 0.5 SOL), 2-of-3 signers, at least one cold/offline, others one-per-hired-VPS-provider to avoid correlated compromise (not yet built — ops task, no issue filed). Stake/slash asymmetry (registry/slash is a local Python adapter per ADR-0007, not on-chain) accepted for now since hired hosts are our own infra and friends aren't node operators — revisit before opening to real third-party node operators. A mainnet-vs-devnet boot guardrail was proposed and explicitly declined by the owner given the safe-by-default money flow above. -**Two new issues from this session, both `ready-for-agent`:** -- **21 — Honest-noise calibration corpus** (`.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md`) rescoped from "prod gate" to a **hard alpha-release blocker**. Confirmed by code read: `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns bool only, no raw divergence value; fleet-dispatch exists but wrong shape (`server.py:2998-3104`, pinned routes + latency, not full-fleet + TOPLOC divergence); storage wrong shape (`registry_events` has no divergence/hardware columns). Three-part build: (1) surface raw TOPLOC distance from audit.py, (2) extend dispatch to hit every registered node with fixed prompt/seed, (3) new SQLite table keyed by node+GPU+dtype. Small-fleet exception granted (N = actual hired-VPS fleet size). Hired VPS hosts stay stake-free until this closes. -- **23 — Dynamic HF-benchmarked pricing** (`.scratch/alpha-hardening/issues/23-dynamic-hf-pricing.md`), high priority but not a release blocker. Pricing today is 100% static (`DEFAULT_PRICE_PER_1K_TOKENS = 0.02`, `billing.py:21`; `model_presets.json` has no per-model price). Target: 80% of cheapest comparable provider on `https://huggingface.co/inference/models` (per-provider-per-model marketplace, `?search=` query param works, no confirmed JSON API — plain scrape attempted first, escalate to headless browser only if the table isn't in raw HTML). Human-verified `hf_aliases` + `hf_verified_match_note` (params/quantization) per model, not auto-discovered matching. Reuses the `_settlement_loop` daemon-thread pattern for a daily refresh; falls back silently to the static default on any failure. +**Two new issues from this session:** +- **21 — Honest-noise calibration corpus** — `Status: ready-for-human` (engineering done 2026-07-06; blocked on human fleet calibration run before mainnet launch). +- **23 — Dynamic HF-benchmarked pricing** — `Status: done` (see `23-dynamic-hf-pricing_completed.md`). Both are already migrated into `.scratch/alpha-hardening/prd.json` (AH-021 updated, AH-023 added) and the README index — ready for Ralph to pick up unattended. diff --git a/.claude/memory/project-status.md b/.claude/memory/project-status.md index 9603fb0..4c53bad 100644 --- a/.claude/memory/project-status.md +++ b/.claude/memory/project-status.md @@ -6,7 +6,9 @@ metadata: type: project --- -# Project Status (2026-07-02) +# Project Status (2026-07-13) + +> Doc reconciliation 2026-07-13: `docs/PRD.md`, ADR numbering (0024 = distributed GGUF), and `docs/issues/` statuses updated to match `docs/prd.json` (US-001…US-035 done). Post-035 work remains in issues 36–48 and `.scratch/`. All 35 user stories in docs/prd.json are done (35/35), including the reward-system arc US-030…US-035 completed 2026-07-02: diff --git a/.scratch/alpha-hardening/handoff.md b/.scratch/alpha-hardening/handoff.md index c44246c..765f83f 100644 --- a/.scratch/alpha-hardening/handoff.md +++ b/.scratch/alpha-hardening/handoff.md @@ -8,7 +8,7 @@ ## 1. Mission / where we are -neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer and a **broken money/trust path**. Three independent audits agreed: unauthenticated gossip, free-credit faucet, double-pay risks, ephemeral bans, and node self-reported accounting undermine alpha release. The owner locked alpha scope (single settlement tracker, open node join, devnet mock-USDT, carried-forward reputation) and a fraud/verification design (TOPLOC adoption, adaptive audits, on-demand hop bisection, persisted graduated reputation, tracker-authoritative accounting). **Research and planning artifacts are complete** (ADRs 0016–0019, 22 issue files, README index). Next: implement Bucket 1 blockers test-first. +neuron-tai is a volunteer-GPU, pipeline-parallel LLM inference network with a working routing layer. Pre-release audits found the money/trust path was not alpha-ready; **Bucket 1 alpha blockers are implemented** (see `.scratch/alpha-hardening/README.md`). Remaining launch gates: issue **21** (human calibration run), post-alpha Bucket 2 (12–15), and active scratch tracks (NCA, perf, distributed GGUF). --- @@ -42,7 +42,7 @@ Point to artifacts — do not re-derive from this handoff. | Path | What it contains | |---|---| | `.scratch/alpha-hardening/README.md` | Issue/ADR index + implementation order | -| `.scratch/alpha-hardening/issues/` | 22 work items (Buckets 1–3) | +| `.scratch/alpha-hardening/issues/` | 25 work items (Buckets 1–3 + perf follow-ups) | | `.scratch/alpha-hardening/research-verifiable-inference.md` | SOTA research, layered alpha scheme (§8), build-vs-adopt (§9) | | `docs/adr/0016–0019` | Alpha scope, auth, fraud, multi-tracker design | | `docs/agents/issue-tracker.md` | Issue file conventions | diff --git a/.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md b/.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md index 8ae2267..3e64004 100644 --- a/.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md +++ b/.scratch/alpha-hardening/issues/21-honest-noise-calibration-corpus.md @@ -16,7 +16,7 @@ Research anchor: `.scratch/alpha-hardening/research-verifiable-inference.md` §8 **Launch context (why this is buildable now, not a research project):** first-launch nodes are hired VPS/VPC hosts under our own direct control (test infrastructure we pay for, not third-party volunteers) — not a long-term topology, but risk-free for calibration purposes since there's no external party to dispute a bad reading. Friends are client-side users of the API in this phase, not node operators. Run the calibration pass against this small, fully-controlled fleet first; hired hosts stay stake-free until it's done, then move to real staking once thresholds derive from their own hardware. -**Current gap (confirmed 2026-07-06 by code read):** none of the three pieces below exist yet. +**Current gap (historical — closed 2026-07-06):** the three engineering pieces below were missing when this issue was filed; all are now implemented and unit-tested. Remaining work is the human calibration run on the live hired-VPS fleet. 1. `verify_activation_proofs()` (`packages/validator/meshnet_validator/audit.py:94-127`) returns a **plain bool** — no raw TOPLOC divergence/distance value is ever computed or surfaced. Every "done" fraud-detection issue (06–10) currently runs on a guessed threshold baked into that bool, not a calibrated one. 2. Fleet dispatch exists but is the wrong shape: `_handle_benchmark_hop_penalty` / `_handle_benchmark_results` (`packages/tracker/meshnet_tracker/server.py:2998-3104`, from the old US-030 latency work) targets pinned 1–3-node *routes* and measures latency, not TOPLOC divergence across *every* registered node. diff --git a/.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md b/.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md index 34efed9..d5c5e6b 100644 --- a/.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md +++ b/.scratch/distributed-gguf-runtime/RALPH-CONTEXT.md @@ -9,7 +9,7 @@ Before changing code, every Ralph agent must: 1. Read this file completely. 2. Read the selected issue under `.scratch/distributed-gguf-runtime/issues/`. -3. Read `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` and the relevant part of `architecture.md`. +3. Read `docs/adr/0024-distributed-gguf-runtime.md` and the relevant part of `architecture.md`. 4. Read `.claude/memory/MEMORY.md` and root `CONTEXT.md` for current project vocabulary and constraints. 5. Inspect the current implementation and tests; do not assume historical scratch text describes live code. 6. Read the evidence/handoff directories for every declared dependency. @@ -296,7 +296,7 @@ Active decisions: - `.scratch/distributed-gguf-runtime/README.md` - `.scratch/distributed-gguf-runtime/implementation-strategy.md` - `.scratch/distributed-gguf-runtime/architecture.md` -- `.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md` +- `docs/adr/0024-distributed-gguf-runtime.md` - `.scratch/distributed-gguf-runtime/PRD.md` - `.scratch/distributed-gguf-runtime/prd.json` diff --git a/.scratch/distributed-gguf-runtime/README.md b/.scratch/distributed-gguf-runtime/README.md index cf64115..27d3deb 100644 --- a/.scratch/distributed-gguf-runtime/README.md +++ b/.scratch/distributed-gguf-runtime/README.md @@ -25,7 +25,7 @@ Transformers/safetensors remains the correctness baseline. vLLM remains an optio - [Current architecture](architecture.md) - [PRD](PRD.md) - [Ralph backlog](prd.json) -- [ADR-0020](ADR-0020-distributed-gguf-runtime.md) +- [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md) - [Milestones](milestones.md) - [Issues](issues/) - [Distributed GGUF research](../../docs/research/distributed-gguf-landscape.md) diff --git a/.scratch/distributed-gguf-runtime/decision-framework.md b/.scratch/distributed-gguf-runtime/decision-framework.md index c8aba2a..4b1098c 100644 --- a/.scratch/distributed-gguf-runtime/decision-framework.md +++ b/.scratch/distributed-gguf-runtime/decision-framework.md @@ -1,6 +1,6 @@ # Distributed GGUF Decision Framework -> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0020](ADR-0020-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale. +> **Superseded for active implementation decisions.** The grill was resolved on 2026-07-13. Use [implementation-strategy.md](implementation-strategy.md), [architecture.md](architecture.md), [ADR-0024](../../docs/adr/0024-distributed-gguf-runtime.md), and [prd.json](prd.json). This file remains as historical decision rationale. This framework is for grilling open decisions. It keeps decisions tied to project vocabulary and implementation gates instead of vague "distributed inference" language. diff --git a/.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md b/.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md index 8751cdf..f191ccc 100644 --- a/.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md +++ b/.scratch/distributed-gguf-runtime/issues/01-lock-the-safetensors-versus-gguf-performance-contract.md @@ -56,4 +56,4 @@ As a runtime engineer, I need a controlled baseline so that GGUF work proceeds f - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md b/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md index d38c8ce..b5cf25c 100644 --- a/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md +++ b/.scratch/distributed-gguf-runtime/issues/02-adopt-the-versioned-grpc-shard-protocol.md @@ -56,4 +56,4 @@ As a node developer, I need a battle-proven streaming protocol so that Python an - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md b/.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md index 46c82cf..2b3d69a 100644 --- a/.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md +++ b/.scratch/distributed-gguf-runtime/issues/03-define-exact-artifact-and-runtime-recipe-identity.md @@ -54,4 +54,4 @@ As the Tracker, I need exact compatibility identity so that only numerically and - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md b/.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md index 3c6c677..ee63716 100644 --- a/.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md +++ b/.scratch/distributed-gguf-runtime/issues/04-create-the-reproducible-pinned-llama-cpp-patch-stack.md @@ -58,4 +58,4 @@ As a maintainer, I need a small auditable fork boundary so that upstream updates - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md b/.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md index 7b0c3e3..5ee4df3 100644 --- a/.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md +++ b/.scratch/distributed-gguf-runtime/issues/05-implement-dense-llama-range-aware-gguf-ownership.md @@ -58,4 +58,4 @@ As a node, I need to map only my assigned dense-Llama Shard so that aggregate co - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md b/.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md index 9f0f4a9..6a1eef4 100644 --- a/.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md +++ b/.scratch/distributed-gguf-runtime/issues/06-implement-architecture-defined-boundary-input-output.md @@ -58,4 +58,4 @@ As a Shard, I need to consume and emit the correct transformer boundary state so - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md b/.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md index c05e751..3ace318 100644 --- a/.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md +++ b/.scratch/distributed-gguf-runtime/issues/07-add-isolated-concurrent-local-hot-kv-state.md @@ -57,4 +57,4 @@ As a client, I need concurrent Route Sessions to retain independent per-Shard ca - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md b/.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md index 663fc9e..0faa6a9 100644 --- a/.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md +++ b/.scratch/distributed-gguf-runtime/issues/08-build-the-standalone-c-grpc-shard-worker.md @@ -62,4 +62,4 @@ As a node runtime, I need one supervised native process so that llama.cpp intern - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md b/.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md index 1c05bdd..039b52b 100644 --- a/.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md +++ b/.scratch/distributed-gguf-runtime/issues/09-integrate-the-native-worker-with-meshnet.md @@ -58,4 +58,4 @@ As the existing node service, I need a GGUF Shard backend adapter so that the Tr - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md b/.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md index 3e8904b..fa2eb63 100644 --- a/.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md +++ b/.scratch/distributed-gguf-runtime/issues/10-pass-local-real-model-two-process-acceptance.md @@ -59,4 +59,4 @@ As a release engineer, I need real local distributed parity before involving net - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md b/.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md index 7cd9dca..8d23aca 100644 --- a/.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md +++ b/.scratch/distributed-gguf-runtime/issues/11-pass-a-real-heterogeneous-two-machine-route.md @@ -59,4 +59,4 @@ As a consumer-hardware operator, I need two physical machines to execute one GGU - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md b/.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md index 6578484..182c20b 100644 --- a/.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md +++ b/.scratch/distributed-gguf-runtime/issues/12-implement-continuous-batching-and-bounded-admission.md @@ -60,4 +60,4 @@ As a node operator, I need active sessions batched safely so that concurrency in - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md b/.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md index 8e681e0..790838c 100644 --- a/.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md +++ b/.scratch/distributed-gguf-runtime/issues/13-harden-failure-cancellation-and-restart-semantics.md @@ -59,4 +59,4 @@ As a client, I need failures to be bounded and explicit so that distributed spee - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md b/.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md index e7a545b..ee55f1c 100644 --- a/.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md +++ b/.scratch/distributed-gguf-runtime/issues/14-enforce-the-gguf-versus-safetensors-release-gate.md @@ -62,4 +62,4 @@ As the product owner, I need an end-to-end comparison so that the native runtime - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md b/.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md index 79d9607..5240d8d 100644 --- a/.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md +++ b/.scratch/distributed-gguf-runtime/issues/15-add-and-certify-a-qwen3-qwen3-moe-adapter.md @@ -58,4 +58,4 @@ As a client seeking top models, I need a separately certified MoE-capable archit - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md b/.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md index c38ce87..b8d5de0 100644 --- a/.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md +++ b/.scratch/distributed-gguf-runtime/issues/16-produce-the-upstream-llama-cpp-collaboration-package.md @@ -57,4 +57,4 @@ As a maintainer, I need narrow upstreamable proposals so that our patch burden c - [PRD](../PRD.md) - [Implementation strategy](../implementation-strategy.md) - [Current architecture](../architecture.md) -- [Architecture decision](../ADR-0020-distributed-gguf-runtime.md) +- [Architecture decision](../../docs/adr/0024-distributed-gguf-runtime.md) diff --git a/CONTEXT-MAP.md b/CONTEXT-MAP.md new file mode 100644 index 0000000..156e7ce --- /dev/null +++ b/CONTEXT-MAP.md @@ -0,0 +1,10 @@ +# Context map + +Multi-context layout is not yet split. Use the root domain vocabulary: + +- **[CONTEXT.md](../CONTEXT.md)** — ubiquitous language for the distributed inference network +- **`docs/adr/`** — system-wide architectural decisions +- **`.scratch//`** — active feature plans and issues +- **`.claude/memory/MEMORY.md`** — agent session index and current workstreams + +Per-context `src//docs/adr/` ADRs will be added when bounded contexts graduate out of the monorepo packages layout. diff --git a/docs/PRD.md b/docs/PRD.md index 4275333..9a5ad10 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done (base program US-001…US-035 complete; see `docs/prd.json`. Post-035 work lives in `docs/issues/36+` and `.scratch/`. Payment/settlement superseded by ADR-0015; fraud by ADR-0018.) # Distributed Inference Network — PRD @@ -8,7 +8,7 @@ Running large language models requires expensive dedicated hardware that most pe ## Solution -A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in SOL or USDC; node operators earn our native token. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start. +A volunteer GPU network where anyone can share their GPU by running a single command and immediately start earning tokens. Nodes each load a shard of a large model; a tracker routes inference requests through the optimal chain of nodes whose shards collectively cover all layers. Developers access the network through an OpenAI-compatible API — a one-line change from any existing LLM integration. Clients pay in **USDT** (alpha: devnet mock-USDT; production: mainnet USDT). Node operators earn USDT payouts from the custodial treasury (ADR-0015); the TAI reward token (ADR-0002) remains deferred. Everything is auto-configured: GPU detection, shard download, wallet creation, and network registration happen automatically on first start. ## User Stories @@ -76,8 +76,8 @@ The codebase is organized as a Python monorepo with the following top-level pack - `packages/contracts` — Solana L2 smart contracts (stake, slash, strike, ban, settlement) - `packages/p2p` — P2P gossip layer and shard swarm seeding -### Inference engine (ADR-0001) -PyTorch with a Petals-style shard pipeline. Each node independently loads its assigned shard from local disk. At inference time, only activation tensors (~8 KB per layer boundary per token) travel between nodes — no model weights cross the network during serving. +### Inference engine (ADR-0001; native GGUF path ADR-0024) +PyTorch with a Petals-style shard pipeline remains the current production backend. A benchmark-gated llama.cpp/GGUF native path is planned in ADR-0024. Each node independently loads its assigned shard from local disk. At inference time, only activation tensors (~8 KB per layer boundary per token) travel between nodes — no model weights cross the network during serving. ### Inference route execution The gateway receives a client request, asks the tracker for an inference route (ordered list of node endpoints covering all layers), opens a persistent TCP session to the first node in the route, streams activation tensors through each node in sequence, and returns the final logits as a streaming chat completion response. @@ -91,11 +91,11 @@ The gateway receives a client request, asks the tracker for an inference route ( 6. Register with tracker (wallet, hardware profile, shard, endpoint) 7. Begin accepting inference connections -### Payment flow -Clients pre-fund an API key with SOL/USDC. The gateway records per-request compute attribution. A settlement transaction runs on Solana L2 at the end of each epoch: client balance is debited, node operators receive our native token proportional to layers served, validators receive a reward share. Solana contracts are the authoritative source for all stake, slash, strike, and ban state (ADR-0002). +### Payment flow (ADR-0015 supersedes ADR-0002 settlement mechanics) +Clients pre-fund an API key with USDT. The tracker meters each request against the off-chain ledger. Periodic settlement batches USDT payouts from the custodial treasury to node operators proportional to work units. Fraud penalties forfeit pending balance (ADR-0018); strike/ban state persists in the tracker registry. TAI token emission remains deferred (ADR-0002 roadmap). -### Fraud detection (ADR-0003) -Validators re-run ~5% of completed requests. If a node's output diverges beyond floating-point tolerance from the reference, the validator submits a slash transaction on-chain. Strike count increments. At the configured strike threshold, the wallet is banned on-chain. New wallets complete N unpaid jobs before earning begins. +### Fraud detection (ADR-0018; historical ADR-0003) +Validators re-run ~5% of completed requests with TOPLOC activation verification. Caught cheaters forfeit pending balance and receive strikes; three strikes bans the wallet. Probation (first N unpaid jobs) remains the anti-sybil re-entry cost. ### Tracker architecture (ADR-0004) Centralized tracker service (HTTP + WebSocket) for fast routing. Nodes gossip state via a lightweight P2P layer so the node client can discover routes during tracker outages. Solana is the authoritative source of truth for all incentive-relevant state. diff --git a/docs/adr/0001-pytorch-over-llama-cpp.md b/docs/adr/0001-pytorch-over-llama-cpp.md index dc0fba7..4a638b0 100644 --- a/docs/adr/0001-pytorch-over-llama-cpp.md +++ b/docs/adr/0001-pytorch-over-llama-cpp.md @@ -1,5 +1,7 @@ # PyTorch over llama.cpp for the inference engine +> **Runtime direction update (2026-07-13):** PyTorch/safetensors remains the current production backend and correctness reference. A benchmark-gated native GGUF path is defined in [ADR-0024](0024-distributed-gguf-runtime.md); it does not replace this ADR until release gates pass. + We started with llama.cpp RPC as the distributed backend (following kyuz0/amd-strix-halo-toolboxes), but switched to PyTorch with a Petals-style shard pipeline. llama.cpp RPC requires the primary node to load the full model and distribute weights over the network at every session start — for a 70B model that's ~70GB over LAN per launch, making tracker-driven node rebalancing prohibitively expensive. PyTorch/Petals lets each node load its shard independently from local disk; only activations (~8KB per layer boundary per token) cross the network at inference time. PyTorch also has same-day support for new model architectures, training support (required for the planned torrent-style fine-tuning feature), and is the engine Petals itself uses for this exact use case. ## Considered Options diff --git a/docs/adr/0003-optimistic-fraud-detection.md b/docs/adr/0003-optimistic-fraud-detection.md index 60f7f8c..9e57d40 100644 --- a/docs/adr/0003-optimistic-fraud-detection.md +++ b/docs/adr/0003-optimistic-fraud-detection.md @@ -1,5 +1,7 @@ # Optimistic trust with stake slashing and strike-based bans +> **Settlement update (2026-07-04):** Alpha uses pending-balance forfeiture instead of stake slashing ([ADR-0015](0015-usdt-custodial-settlement.md)). Fraud detection, TOPLOC audits, and persisted reputation are specified in [ADR-0018](0018-fraud-detection-verification-and-reputation.md). The text below is the historical prototype design. + All inference responses are trusted by default. Validators re-run a random sample (~5%) of requests on reference nodes and compare outputs. Nodes that fail are slashed (stake reduced). Enough strikes result in a permanent on-chain ban. For the prototype, the gateway emits validation events after completed requests. A validation event records the session id, model preset, request messages, observed output, and the route metadata for each node that served the request. The validator samples events with a configurable rate and deterministic seed for tests. Sampled events are re-run against a trusted reference node/reference function; string outputs must match exactly for stub models, while future tensor/model outputs use a configurable floating-point tolerance. diff --git a/docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md b/docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md index 62fe4da..ed1d1e7 100644 --- a/docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md +++ b/docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md @@ -1,6 +1,6 @@ # ADR-0020: Dashboard chat streaming, live request progress, and the mixed-topology routing flaw -## Status: Accepted (chat/streaming/styles implemented); routing flaw documented, fix pending +## Status: Accepted (chat/streaming/styles and mixed-topology routing fix implemented) ## Context @@ -94,7 +94,7 @@ head + full-model downstream is a topology the planner never had to handle befor prior split tests used disjoint shards (0–11 + 12–23) where `shard_start` happened to equal the correct continuation layer. -### Required fix (not yet implemented) +### Required fix (implemented 2026-07-07 — commits `518c259`, `e44abc9`, `1ecc599`; see ADR-0021) 1. **Correct continuation layer:** when hop N ends at layer `e`, hop N+1 must execute from `start_layer = e + 1` regardless of the downstream node's own `shard_start` diff --git a/docs/adr/0022-sharded-per-node-generation-cache.md b/docs/adr/0022-sharded-per-node-generation-cache.md index 5b2df20..f9b2088 100644 --- a/docs/adr/0022-sharded-per-node-generation-cache.md +++ b/docs/adr/0022-sharded-per-node-generation-cache.md @@ -1,6 +1,8 @@ # ADR-0022: Sharded per-node generation cache for distributed PyTorch routes -## Status: Accepted +## Status: Superseded — see [0022-sharded-per-node-kv-cache.md](0022-sharded-per-node-kv-cache.md) + +> Draft alternate header names (`X-Meshnet-Cache-Mode`, `X-Meshnet-Seq-Len`) were not implemented. The accepted wire protocol and implementation use `X-Meshnet-Cache` and `X-Meshnet-Past-Len` per the linked ADR. ## Context diff --git a/.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md b/docs/adr/0024-distributed-gguf-runtime.md similarity index 92% rename from .scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md rename to docs/adr/0024-distributed-gguf-runtime.md index cbfb3c9..14ed027 100644 --- a/.scratch/distributed-gguf-runtime/ADR-0020-distributed-gguf-runtime.md +++ b/docs/adr/0024-distributed-gguf-runtime.md @@ -1,8 +1,12 @@ -# ADR-0020: Lean Native Distributed GGUF Runtime +# ADR-0024: Lean Native Distributed GGUF Runtime Status: Accepted Date: 2026-07-13 +> **Numbering note:** ADR-0020 is reserved for dashboard chat streaming and mixed-topology routing (`docs/adr/0020-chat-streaming-live-progress-and-mixed-topology-routing.md`). This record was originally drafted as ADR-0020 in `.scratch/distributed-gguf-runtime/` and renumbered to avoid the collision. +> +> **Relation to ADR-0001:** PyTorch/safetensors remains the correctness reference and current production backend. This ADR defines a benchmark-gated native GGUF path; it does not revoke ADR-0001 until release gates pass. + ## Context The project currently uses Transformers/safetensors as its real model execution backend. This provides broad architecture coverage and a correctness reference, but reported and observed consumer CPU/GPU inference performance motivates evaluating llama.cpp/GGML and quantized GGUF. diff --git a/docs/issues/01-monorepo-scaffold.md b/docs/issues/01-monorepo-scaffold.md index 83e8047..93cc414 100644 --- a/docs/issues/01-monorepo-scaffold.md +++ b/docs/issues/01-monorepo-scaffold.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 01 — Monorepo scaffold + single-node smoke test diff --git a/docs/issues/02-two-node-shard-pipeline.md b/docs/issues/02-two-node-shard-pipeline.md index 943d0d2..812a96a 100644 --- a/docs/issues/02-two-node-shard-pipeline.md +++ b/docs/issues/02-two-node-shard-pipeline.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 02 — Two-node shard pipeline diff --git a/docs/issues/03-tracker-registration-and-routing.md b/docs/issues/03-tracker-registration-and-routing.md index 224efce..ca58dc3 100644 --- a/docs/issues/03-tracker-registration-and-routing.md +++ b/docs/issues/03-tracker-registration-and-routing.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 03 — Tracker: node registration + route selection diff --git a/docs/issues/04-node-client-startup.md b/docs/issues/04-node-client-startup.md index 28020c2..6abc1ff 100644 --- a/docs/issues/04-node-client-startup.md +++ b/docs/issues/04-node-client-startup.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 04 — Node client startup flow (`meshnet-node start`) diff --git a/docs/issues/05-openai-compatible-gateway.md b/docs/issues/05-openai-compatible-gateway.md index 41db5a7..a5bc224 100644 --- a/docs/issues/05-openai-compatible-gateway.md +++ b/docs/issues/05-openai-compatible-gateway.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 05 — OpenAI-compatible gateway diff --git a/docs/issues/08-probationary-period-and-bans.md b/docs/issues/08-probationary-period-and-bans.md index 8a22394..c2989ee 100644 --- a/docs/issues/08-probationary-period-and-bans.md +++ b/docs/issues/08-probationary-period-and-bans.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done (on-chain registry mechanics superseded — probation/ban enforcement uses tracker registry + ADR-0015/0018) # 08 — Node probationary period + ban enforcement diff --git a/docs/issues/09-p2p-shard-swarm.md b/docs/issues/09-p2p-shard-swarm.md index 37e20e8..9fe303b 100644 --- a/docs/issues/09-p2p-shard-swarm.md +++ b/docs/issues/09-p2p-shard-swarm.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 09 — P2P shard swarm diff --git a/docs/issues/10-meshnet-sdk.md b/docs/issues/10-meshnet-sdk.md index 9def6a5..7b7f825 100644 --- a/docs/issues/10-meshnet-sdk.md +++ b/docs/issues/10-meshnet-sdk.md @@ -1,4 +1,4 @@ -Status: ready-for-agent +Status: done # 10 — `meshnet` Python SDK diff --git a/docs/issues/19-binary-data-plane-and-peer-weight-transfer.md b/docs/issues/19-binary-data-plane-and-peer-weight-transfer.md index f768a72..20d3918 100644 --- a/docs/issues/19-binary-data-plane-and-peer-weight-transfer.md +++ b/docs/issues/19-binary-data-plane-and-peer-weight-transfer.md @@ -1,8 +1,6 @@ # US-019 — Binary data plane and optional peer weight transfer -Status: needs-triage -Priority: Low -Stage: Design parking lot +Status: done (design parking lot; binary activation path shipped in US-011/US-019) ## Context diff --git a/docs/issues/36-relay-streamed-chat.md b/docs/issues/36-relay-streamed-chat.md index ea9efd2..40e4fc5 100644 --- a/docs/issues/36-relay-streamed-chat.md +++ b/docs/issues/36-relay-streamed-chat.md @@ -1,8 +1,6 @@ # US-036 — Streamed chat completions over the relay RPC path -Status: planned -Priority: Critical (blocks public friends-test deployment) -Stage: Designed +Status: done (implemented — `_stream_relayed_frames` in `server.py`; verify on public NAT relay before friends-test) ## Context diff --git a/docs/issues/42-gguf-llamacpp-node-backend.md b/docs/issues/42-gguf-llamacpp-node-backend.md index 544800a..9f8645b 100644 --- a/docs/issues/42-gguf-llamacpp-node-backend.md +++ b/docs/issues/42-gguf-llamacpp-node-backend.md @@ -1,7 +1,7 @@ # US-042 — GGUF/llama.cpp node backend Status: planned -Priority: High (unlocks big MoE models on volunteer hardware — the pool's core value) +Priority: High (whole-model GGUF shortcut; distributed path in [ADR-0024](../adr/0024-distributed-gguf-runtime.md)) Stage: Draft design ## Context diff --git a/docs/issues/20-memory-budget-shard-slots-and-dropout-relocation.md b/docs/issues/48-memory-budget-shard-slots-and-dropout-relocation.md similarity index 94% rename from docs/issues/20-memory-budget-shard-slots-and-dropout-relocation.md rename to docs/issues/48-memory-budget-shard-slots-and-dropout-relocation.md index 74f70cf..b055d08 100644 --- a/docs/issues/20-memory-budget-shard-slots-and-dropout-relocation.md +++ b/docs/issues/48-memory-budget-shard-slots-and-dropout-relocation.md @@ -1,6 +1,8 @@ -Status: ready-for-agent +Status: planned -# US-020 - Memory budget, shard slots, and dropout relocation hardening +# US-048 — Memory budget, shard slots, and dropout relocation hardening + +> Renumbered from duplicate slot `20` (which belongs to tracker-node-hardening / US-020 in `docs/prd.json`). ## Goal @@ -64,3 +66,4 @@ The current runtime still effectively has one active backend shard per node. A n - 2026-06-30: Created after implementing the initial registration plumbing in commit `f1e4ed6` (`--memory`, `--max-shards`, tracker validation). This issue captures the remaining end-to-end behavior so it does not conflict with US-013. - 2026-06-30: Implementation decision: `max_loaded_shards` is currently a validated and exposed capacity field, but multi-range assignment remains reserved because `TorchNodeServer` serves one active backend shard. The tracker therefore emits at most one active range per node while exposing `vram_bytes`, `ram_bytes`, `max_loaded_shards`, quantization, throughput, and computed `max_assignable_layers` in inspection endpoints. +- 2026-07-13: Renumbered from `docs/issues/20-memory-budget-…` to resolve duplicate issue slot 20.