1068 lines
86 KiB
JSON
1068 lines
86 KiB
JSON
{
|
||
"name": "Distributed Inference Network",
|
||
"description": "Distributed inference network: base program US-001…US-035 complete; post-alpha friends-test and LAN-serving arc US-036…US-047 tracked here. Active scratch features (alpha hardening, NCA, distributed GGUF) have separate prd.json files under .scratch/.",
|
||
"branchName": "ralph/distributed-inference-network",
|
||
"userStories": [
|
||
{
|
||
"id": "US-001",
|
||
"title": "01 — Monorepo scaffold + single-node smoke test",
|
||
"description": "Stand up the monorepo package layout and prove that a client HTTP request can travel end-to-end through the stack — gateway → one node serving all layers → valid response — before any real model or distributed logic is wired in. The six top-level packages should exist with minimal stubs: - `packages/node` — node client CLI (`meshnet-node`) - `packages/gateway` — HTTP gateway + route orchestration - `packages/tracker` — node registry and route selection - `packages/sdk` — `meshnet` Python SDK - `packages/contracts` — Solana smart contract wrappers - `packages/p2p` — gossip and shard swarm The smoke test uses a tiny in-process stub model (not a real LLM) so the test is fast and has no external dependencies. The node runs in the same process as the test. The gateway is a real HTTP server. The client sends a real `POST /v1/chat/completions` request and receives a valid OpenAI-format response.",
|
||
"acceptanceCriteria": [
|
||
"`pip install -e packages/node packages/gateway packages/tracker` works from repo root",
|
||
"`meshnet-node`, `meshnet-gateway`, and `meshnet-tracker` CLI entry points exist",
|
||
"A single integration test starts a gateway and one stub node in-process, sends a `POST /v1/chat/completions` request, and asserts a valid OpenAI-format response is returned",
|
||
"The test passes with `pytest` from repo root with no external services running",
|
||
"All six package directories exist with `pyproject.toml` and an importable top-level module",
|
||
"Follow TDD where code is added: add/adjust an observable behavior test before implementation when practical",
|
||
"Run the task-specific tests and ensure they pass",
|
||
"Run `python -m pytest` from repo root and ensure it passes, or document why no test suite exists yet",
|
||
"Keep the implementation scoped to this story; do not implement downstream stories early",
|
||
"Commit only this story's code changes with a focused conventional commit message"
|
||
],
|
||
"priority": 1,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/01-monorepo-scaffold.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Completed by Ralph iteration 7b260695; verified by pytest and editable package installs.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-002",
|
||
"title": "02 — Two-node shard pipeline",
|
||
"description": "Extend the single-node smoke test so that inference is routed through exactly two nodes, each serving half of a stub model's layers. Activation tensors must travel from the gateway to node A, then from node A to node B, and the final output returns to the gateway. This proves the distributed shard pipeline works end-to-end before real models or a real tracker are introduced. The two nodes run in the same process as the test (no real network required). The gateway hardcodes the two-node route for now — dynamic route selection comes in issue 03. The activation tensor protocol (shape, dtype, serialisation format) must be established here, as all later issues depend on it. Use the domain vocabulary from `CONTEXT.md`: the ordered list of nodes is an **inference route**; each node's layer range is a **shard**; the tensors passed between nodes are activations (not \"data\" or \"chunks\").",
|
||
"acceptanceCriteria": [
|
||
"The integration test spins up two stub nodes, each configured with a non-overlapping shard range",
|
||
"A `POST /v1/chat/completions` request results in activation tensors flowing node-A → node-B (verifiable via test assertions or logs)",
|
||
"The gateway assembles the final response correctly from node-B's output",
|
||
"The test passes with no external services running",
|
||
"The activation tensor serialisation format is documented in a short inline comment (shape, dtype, wire format) — this becomes the contract all future nodes implement",
|
||
"Follow TDD where code is added: add/adjust an observable behavior test before implementation when practical",
|
||
"Run the task-specific tests and ensure they pass",
|
||
"Run `python -m pytest` from repo root and ensure it passes, or document why no test suite exists yet",
|
||
"Keep the implementation scoped to this story; do not implement downstream stories early",
|
||
"Commit only this story's code changes with a focused conventional commit message"
|
||
],
|
||
"priority": 2,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/02-two-node-shard-pipeline.md",
|
||
"dependsOn": [
|
||
"US-001"
|
||
],
|
||
"completionNotes": "Tests pass; activation tensor flow via stub nodes verified. New HF-model path tested in test_node_startup.py.",
|
||
"status": "done",
|
||
"status_reason": "Base64 JSON wire format established here was replaced by binary HTTP protocol in US-011. tests/test_two_node_pipeline.py needs verification it exercises the new binary format end-to-end."
|
||
},
|
||
{
|
||
"id": "US-003",
|
||
"title": "03 — Tracker: node registration + route selection",
|
||
"description": "Replace the hardcoded two-node route from issue 02 with a real tracker. Nodes register themselves with the tracker on startup (endpoint, shard range, hardware profile, node score). The gateway queries the tracker for an inference route for a given model preset instead of using a static list. The tracker returns an ordered list of node endpoints whose shards collectively cover all layers. The tracker runs as a lightweight HTTP service. Node score is initially a simple placeholder (e.g. fixed value or random) — real throughput/latency scoring comes later. The tracker must correctly reject route requests when no registered nodes cover a required shard range and return an appropriate error. This issue establishes the tracker's registration and routing API contract, which issues 04 (node startup) and 05 (OpenAI gateway) both depend on.",
|
||
"acceptanceCriteria": [
|
||
"A node can register with the tracker via HTTP, providing its endpoint, shard range, and a hardware profile",
|
||
"The gateway queries the tracker with a model preset name and receives an ordered list of node endpoints forming a complete inference route",
|
||
"The end-to-end integration test from issue 02 still passes with the tracker now in the loop (no hardcoded routes)",
|
||
"The tracker returns a clear error response when no route is available for a requested model preset",
|
||
"Nodes that fail to heartbeat within a configurable window are removed from the registry",
|
||
"The tracker's registration and route-selection HTTP API is defined (paths, request/response shapes) in the tracker package",
|
||
"Follow TDD where code is added: add/adjust an observable behavior test before implementation when practical",
|
||
"Run the task-specific tests and ensure they pass",
|
||
"Run `python -m pytest` from repo root and ensure it passes, or document why no test suite exists yet",
|
||
"Keep the implementation scoped to this story; do not implement downstream stories early",
|
||
"Commit only this story's code changes with a focused conventional commit message"
|
||
],
|
||
"priority": 3,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/03-tracker-registration-and-routing.md",
|
||
"dependsOn": [
|
||
"US-002"
|
||
],
|
||
"completionNotes": "Completed by Ralph iteration 79796dd2; verified by pytest, compileall, editable installs, CLI help, and spec/code reviews.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-004",
|
||
"title": "04 — Node client startup flow (`meshnet-node start`)",
|
||
"description": "Make `meshnet-node start` self-configuring from scratch on a machine with a CUDA-capable GPU (or CPU fallback). The full startup sequence must complete without any manual configuration: 1. Detect GPU model and available VRAM 2. Load an existing Solana wallet from disk, or generate and save a new one 3. Query the tracker for the optimal shard assignment given the hardware profile 4. Download the assigned shard from HuggingFace (`huggingface_hub`) 5. Register with the tracker (wallet address, endpoint, shard range, hardware profile) 6. Begin accepting inference connections The node prints a short status summary on startup: wallet address, assigned shard, model preset, and download progress. No interactive prompts — the entire flow is non-interactive. This is the primary viral growth vector. Every second of friction in this flow costs node operators. The startup sequence must work on Linux with CUDA, and degrade gracefully to CPU on machines without a GPU.",
|
||
"acceptanceCriteria": [
|
||
"`meshnet-node start --tracker http://localhost:8080` completes startup without prompts on a machine with a CUDA GPU",
|
||
"A Solana wallet keypair is generated and saved to `~/.config/meshnet/wallet.json` if none exists",
|
||
"The assigned shard is downloaded from HuggingFace and cached to `~/.cache/meshnet/shards/`",
|
||
"The node registers with the tracker and appears in the tracker's node registry",
|
||
"The node accepts a live inference connection and processes activation tensors correctly after startup",
|
||
"On a CPU-only machine, the node starts with a warning and serves a CPU-appropriate shard",
|
||
"An integration test covers the full startup sequence against a local tracker stub",
|
||
"Follow TDD where code is added: add/adjust an observable behavior test before implementation when practical",
|
||
"Run the task-specific tests and ensure they pass",
|
||
"Run `python -m pytest` from repo root and ensure it passes, or document why no test suite exists yet",
|
||
"Keep the implementation scoped to this story; do not implement downstream stories early",
|
||
"Commit only this story's code changes with a focused conventional commit message"
|
||
],
|
||
"priority": 4,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/04-node-client-startup.md",
|
||
"dependsOn": [
|
||
"US-003"
|
||
],
|
||
"completionNotes": "Completed by Ralph iteration 86510a10; verified by pytest, compileall, editable installs, CLI help, and final review.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-005",
|
||
"title": "05 — OpenAI-compatible gateway",
|
||
"description": "Expose a production-shape HTTP API from the gateway so that any client using the OpenAI Python SDK (or any OpenAI-compatible tool) works by changing only the `base_url`. The gateway translates OpenAI-format requests into inference route execution and streams responses back in OpenAI-format. Endpoints to implement: - `POST /v1/chat/completions` — streaming (`text/event-stream`) and non-streaming - `GET /v1/models` — returns the list of model presets currently routable on the network - `GET /v1/health` — liveness check The gateway selects an inference route from the tracker, executes the shard pipeline, and assembles the streamed response. If the tracker returns no route for the requested model, the gateway responds with a standard OpenAI-format error (`model_not_available`). Authentication (API key → SOL/USDC balance) is a stub in this issue — return 200 for any non-empty `Authorization` header. Real payment gating comes in issue 06.",
|
||
"acceptanceCriteria": [
|
||
"`openai.OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"test\").chat.completions.create(model=\"stub-model\", messages=[...])` returns a valid response",
|
||
"Streaming works: `stream=True` returns `text/event-stream` chunks in OpenAI SSE format",
|
||
"`GET /v1/models` returns a JSON array of available model preset names",
|
||
"A request for an unavailable model returns an OpenAI-format error response with HTTP 503",
|
||
"LangChain `ChatOpenAI(base_url=..., api_key=...)` works against the gateway",
|
||
"An integration test covers streaming and non-streaming paths end-to-end through a real tracker and two stub nodes",
|
||
"Follow TDD where code is added: add/adjust an observable behavior test before implementation when practical",
|
||
"Run the task-specific tests and ensure they pass",
|
||
"Run `python -m pytest` from repo root and ensure it passes, or document why no test suite exists yet",
|
||
"Keep the implementation scoped to this story; do not implement downstream stories early",
|
||
"Commit only this story's code changes with a focused conventional commit message"
|
||
],
|
||
"priority": 5,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/05-openai-compatible-gateway.md",
|
||
"dependsOn": [
|
||
"US-003"
|
||
],
|
||
"completionNotes": "All 6 gateway tests pass including OpenAI SDK, LangChain, streaming, and 503 for unknown model.",
|
||
"status": "done",
|
||
"status_reason": "Gateway is currently the pipeline orchestrator. US-014 moves orchestration to tracker-nodes; gateway becomes a thin load-balancer proxy. Implementation will be superseded — defer rework until US-014 lands."
|
||
},
|
||
{
|
||
"id": "US-006",
|
||
"title": "06 — Solana stake + settlement contracts",
|
||
"description": "Deploy and integrate the Solana smart contracts that make node staking, client payment, and token reward settlement trustless. All development and testing targets **Solana testnet** — never devnet or mainnet during development, to avoid real costs.",
|
||
"acceptanceCriteria": [
|
||
"All contracts deploy successfully to Solana testnet",
|
||
"A node can submit a stake transaction and have its balance reflected in the registry contract",
|
||
"A client can fund an API key account with testnet SOL",
|
||
"After a completed inference session, compute attribution is recorded on-chain with correct node/layer attribution",
|
||
"The epoch settlement transaction correctly distributes token rewards to node operators and deducts client balances",
|
||
"The gateway refuses to route to a node whose stake balance is below the minimum threshold",
|
||
"All contract interactions in tests run against a local Solana test validator (via `solana-test-validator`) — no live testnet required for CI",
|
||
"A `.env.testnet` config points to Solana testnet RPC for manual end-to-end testing",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 6,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/06-solana-stake-and-settlement.md",
|
||
"dependsOn": [
|
||
"US-003"
|
||
],
|
||
"completionNotes": "Completed by fresh Ralph/Codex session c257ffde with controller patches for clarified economics; verified by pytest, compileall, and diff check.",
|
||
"status": "done",
|
||
"status_reason": "Superseded for alpha by ADR-0015 (US-030..US-035): custodial USDT treasury + off-chain ledger replaces the Anchor stake/registry/settlement contracts, and targets Solana devnet (mock-USDT mint) rather than the testnet-only rule in this story's description. See docs/issues/06-solana-stake-and-settlement.md and ADR-0016."
|
||
},
|
||
{
|
||
"id": "US-007",
|
||
"title": "07 — Fraud detection: validator + on-chain slash",
|
||
"description": "Implement the optimistic fraud detection loop. After each inference request completes, the validator process independently decides whether to re-run the request on a reference node (~5% sample rate).",
|
||
"acceptanceCriteria": [
|
||
"The validator process samples ~5% of completed inference requests (configurable)",
|
||
"A node returning a deliberately wrong output is detected and slashed within one validation cycle",
|
||
"The on-chain stake balance of the slashed node decreases by the correct slash amount",
|
||
"The strike count for the slashed node increments on-chain",
|
||
"A node that reaches the strike threshold is excluded from route selection on the next gateway request",
|
||
"A slashed node logs a clear warning to stdout",
|
||
"An integration test: run a deliberately-bad node, send 20 requests, assert at least one slash transaction is submitted and the node is eventually excluded from routes",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 7,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/07-fraud-detection-slash.md",
|
||
"dependsOn": [
|
||
"US-005",
|
||
"US-006"
|
||
],
|
||
"completionNotes": "Completed by fresh Ralph/Codex session 04475912 with controller fix for duplicate slash suppression; verified by pytest, compileall, and diff check.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-008",
|
||
"title": "08 — Node probationary period + ban enforcement",
|
||
"description": "Implement the two anti-sybil mechanisms that make re-entering the network after a ban economically costly.",
|
||
"acceptanceCriteria": [
|
||
"A node with a new wallet receives no token rewards for its first N jobs (verified via settlement contract state)",
|
||
"The node client prints remaining probationary jobs on startup",
|
||
"After N jobs, the next epoch settlement correctly credits the node with token rewards",
|
||
"A wallet with strike count at the threshold is marked banned in the registry contract",
|
||
"The tracker excludes banned wallets from route selection",
|
||
"A banned wallet that attempts to register with the tracker is rejected",
|
||
"An integration test covers: new wallet → N jobs → earning begins; and: strike threshold reached → banned → excluded from routes",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 8,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/08-probationary-period-and-bans.md",
|
||
"dependsOn": [
|
||
"US-007"
|
||
],
|
||
"completionNotes": "Completed by fresh Ralph/Codex session db3f5c10 with controller test fix for banned registration semantics; verified by pytest, compileall, and diff check.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-009",
|
||
"title": "09 — P2P shard swarm",
|
||
"description": "Once a node has downloaded a shard, it seeds that shard to other nodes that are assigned the same shard.",
|
||
"acceptanceCriteria": [
|
||
"A node that has downloaded a shard is listed as a peer for that shard in the tracker",
|
||
"A second node assigned the same shard downloads it from the first node (peer) rather than HuggingFace",
|
||
"If no peers are available, the node falls back to HuggingFace without error",
|
||
"The node client logs whether the shard was downloaded from a peer or HuggingFace",
|
||
"Shard chunks are verified against a checksum before being marked complete",
|
||
"An integration test: node A downloads shard from HuggingFace stub; node B with same assignment downloads from node A",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 9,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/09-p2p-shard-swarm.md",
|
||
"dependsOn": [
|
||
"US-004"
|
||
],
|
||
"completionNotes": "Completed by fresh Ralph/Codex session 243fae88 with controller fix for streaming tar archives; verified by pytest, compileall, and diff check.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-010",
|
||
"title": "10 — `meshnet` Python SDK",
|
||
"description": "A Python SDK (`packages/sdk`) that wraps the OpenAI-compatible gateway and exposes network-specific controls.",
|
||
"acceptanceCriteria": [
|
||
"`pip install meshnet` installs the SDK",
|
||
"`Client.chat.completions.create(...)` works identically to the OpenAI SDK",
|
||
"`client.wallet.balance()` returns the current SOL/USDC balance for the API key",
|
||
"`client.wallet.top_up()` returns a valid Solana payment address",
|
||
"`client.models.available()` returns model presets with shard coverage percentage",
|
||
"`client.estimate_cost(model, tokens)` returns a cost estimate in SOL",
|
||
"`client.request(redundancy=2)` sends to two independent inference routes and returns majority response",
|
||
"The SDK is typed (py.typed, full type stubs)",
|
||
"An integration test covers each SDK method against a local gateway + tracker + stub nodes",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 10,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/10-meshnet-sdk.md",
|
||
"dependsOn": [
|
||
"US-005",
|
||
"US-006"
|
||
],
|
||
"completionNotes": "Completed by fresh Ralph/Codex session ef4eea0e; verified by pytest, compileall, editable SDK install, and diff check.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-011",
|
||
"title": "11 — Binary activation wire format",
|
||
"description": "Replace the base64 JSON activation payload with raw binary HTTP bodies, zstd compression, and chunked prefill (128 tokens/chunk). All nodes and the gateway must be migrated. Stub nodes continue to emit zeroed tensors, just in binary. This is a protocol prerequisite for US-012 (real model backend).",
|
||
"acceptanceCriteria": [
|
||
"Node /forward endpoint reads shape/dtype/session/chunk from HTTP headers; body is raw binary (optionally zstd-compressed)",
|
||
"Node /forward response is raw binary with the same header set",
|
||
"Gateway splits prompts > MESHNET_CHUNK_TOKENS (default 128) into sequential chunks sent through the pipeline",
|
||
"Integration test: 512-token stub activation (4 chunks) through a two-node pipeline returns 4 valid binary chunk responses",
|
||
"zstd Python package added as a dependency to packages/node and packages/gateway",
|
||
"_make_stub_activations replaced with _make_stub_binary_activation(shape, dtype) -> bytes",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 11,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/11-binary-wire-format.md",
|
||
"dependsOn": [
|
||
"US-002"
|
||
],
|
||
"completionNotes": "Completed by fresh Ralph/Codex session 3f3bed75; verified by pytest, compileall, and diff check.",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-012",
|
||
"title": "12 — Real PyTorch model backend",
|
||
"description": "Replace stub node inference with actual transformers layer execution. Node loads HuggingFace SafeTensors model shard (model.model.layers[start:end]), runs real forward passes in bfloat16, handles head (embed_tokens) and tail (lm_head) responsibilities, and supports bitsandbytes NF4/INT8/bfloat16 quantization via --quantization flag. Test model: openai-community/gpt2.",
|
||
"acceptanceCriteria": [
|
||
"meshnet-node start --model-id openai-community/gpt2 --shard-start 0 --shard-end 6 loads the shard without error",
|
||
"Head node (shard_start==0) loads tokenizer and embed_tokens",
|
||
"Tail node (shard_end==total_layers) loads model.norm and lm_head",
|
||
"Two-node GPT-2 integration test returns deterministic coherent text completion",
|
||
"--quantization [bfloat16|int8|nf4] flag supported; bfloat16 default",
|
||
"Node with insufficient VRAM prints clear error and exits",
|
||
"transformers, bitsandbytes, safetensors, accelerate added as node dependencies",
|
||
"Integration tests marked @pytest.mark.integration, skipped in CI without GPU",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 12,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/12-real-pytorch-model-backend.md",
|
||
"dependsOn": [
|
||
"US-011"
|
||
],
|
||
"completionNotes": "Completed by agent",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-013",
|
||
"title": "13 — Coverage-first tracker shard assignment",
|
||
"description": "Upgrade tracker route selection to coverage-first, speed-weighted bin-packing. Tracker maintains a live coverage map per model, issues LOAD_SHARD/DROP_SHARD rebalance directives when coverage drops, and assigns shard ranges using declared VRAM, quantization, and benchmark throughput. A model is only routable when all layer ranges have node_count >= 1.",
|
||
"acceptanceCriteria": [
|
||
"Node registration accepts vram_bytes, ram_bytes, quantizations[], benchmark_tokens_per_sec",
|
||
"GET /v1/coverage/<model_preset> returns list of {start_layer, end_layer, node_count}",
|
||
"Model is unroutable when any layer range has node_count=0",
|
||
"New node gets assigned to highest-priority uncovered range first",
|
||
"Node disconnection triggers LOAD_SHARD directive to idle node within 30 seconds",
|
||
"Shard assignment respects VRAM: assigned_layers <= floor((vram_bytes * 0.8) / bytes_per_layer_at_quant)",
|
||
"Faster node receives wider shard range when both can cover the same gap",
|
||
"Integration test: three nodes with different VRAM show widest range on largest-VRAM node with 100% coverage",
|
||
"Integration test: kill middle-range node → tracker issues LOAD_SHARD → coverage recovers",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 13,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/13-coverage-first-shard-assignment.md",
|
||
"dependsOn": [
|
||
"US-003",
|
||
"US-012"
|
||
],
|
||
"completionNotes": "Completed by agent",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "US-014",
|
||
"title": "14 — Tracker-as-first-layer-node (inference entry point)",
|
||
"description": "Merge inference orchestration into tracker nodes that serve the first-layer shard. A tracker node exposes /v1/chat/completions directly: tokenizes input, runs embed_tokens + layers[0..k], selects onward route from coverage map, forwards binary activations, receives tail output, and streams tokens back. The standalone gateway becomes a thin load-balancer routing to tracker nodes.",
|
||
"acceptanceCriteria": [
|
||
"Node with shard_start==0 (or --tracker-mode flag) exposes /v1/chat/completions alongside /forward",
|
||
"Tracker-node /v1/chat/completions: tokenize → embed → own layers → route selection → forward binary → receive tail → stream SSE",
|
||
"Two tracker nodes for same model each handle requests independently",
|
||
"Gateway proxies /v1/chat/completions to tracker nodes (round-robin), discovered via GET /v1/tracker-nodes/<model_preset>",
|
||
"Integration test: two tracker nodes + two mid-shard nodes for GPT-2; 10 requests via gateway; both tracker nodes receive load; all responses valid",
|
||
"Existing US-005 OpenAI-compatible tests still pass",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 14,
|
||
"passes": true,
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/14-tracker-as-node.md",
|
||
"dependsOn": [
|
||
"US-012",
|
||
"US-013"
|
||
],
|
||
"status": "done",
|
||
"completionNotes": "Implemented: (1) Tracker GET /v1/tracker-nodes/<model> endpoint returning nodes registered with tracker_mode=true at shard_start==0. (2) StubNodeServer and TorchNodeServer now accept tracker_mode/tracker_url params; when tracker_mode=True, /v1/chat/completions is served alongside /forward. (3) TorchNodeServer auto-detects tracker mode when shard_start==0. (4) Gateway _handle_chat_completions checks for tracker-nodes first via _get_tracker_nodes(), proxies round-robin if found, falls back to existing direct pipeline if none (backward compat). (5) CLI --tracker-mode and --tracker-url flags added. (6) Integration test: 2 tracker-nodes + 2 mid-shard nodes for gpt2; 10 requests; round-robin verified (5/5 split); all responses valid OpenAI format. All 78 tests pass."
|
||
},
|
||
{
|
||
"id": "US-015",
|
||
"title": "15 — Ralph: agent-agnostic runner + status-field-aware dashboard",
|
||
"description": "Two improvements to the Ralph workflow tooling. (1) Status-field awareness: replace all passes:true/false reads in ralph_progress.py with the rich status field. Surface to-revise and needs-review stories as an attention list in the dashboard; auto command skips them by default. (2) Agent agnosticism: make the agent selectable per session — codex (existing), claude (Claude Code CLI), openrouter (unified API for GPT-4/Mistral/Llama/DeepSeek via OPENROUTER_API_KEY + --model), or custom (--agent-cmd path to any script). Persist agent choice to .ralph-tui/agent-config.json. When ralph-tui does not natively support a requested agent, ralph_progress.py falls back to a thin adapter that calls the agent API directly with the task prompt. (3) Worktree parallelism: ralph_progress.py auto --parallel [N] creates one git worktree per ready task (git worktree add ../AI-worktree-<id> -b feat/<id>), runs up to N agent sessions concurrently, and merges each branch back to master after the agent exits 0 and tests pass. Non-blocking stories that share no file overlap can safely run in parallel. ralph_progress.py list-parallel shows which open stories have no overlapping dependsOn chains and are safe to run concurrently.",
|
||
"acceptanceCriteria": [
|
||
"All story.get('passes') reads in ralph_progress.py replaced with _is_done() helper that reads status field with passes fallback",
|
||
"_story_sets() returns six buckets: done, attention (to-revise/needs-review), active (in-progress), in-design, ready, blocked",
|
||
"Dashboard shows ⚠ Attention required section listing to-revise/needs-review stories with status_reason",
|
||
"auto command skips to-revise and needs-review stories; --include-revise flag overrides",
|
||
"_review_report includes Attention Required section with status_reason for all affected stories",
|
||
"ralph_progress.py set-agent --agent <name> [--model <model>] writes .ralph-tui/agent-config.json",
|
||
"--agent codex|claude|openrouter|custom accepted by run-next, auto, review subcommands",
|
||
"run-next --agent custom --agent-cmd ./my-agent.sh runs a task via custom script (prompt file as $1)",
|
||
"Saved agent config is loaded as default when --agent is not passed on the CLI",
|
||
"python -m pytest passes from repo root",
|
||
"ralph_progress.py auto --parallel 2 starts two worktrees concurrently for the two highest-priority ready tasks",
|
||
"Each worktree is created at ../AI-worktree-<story-id> on branch feat/<story-id>",
|
||
"After agent exits 0 and pytest passes in the worktree, the branch is merged to master and the worktree removed",
|
||
"ralph_progress.py list-parallel prints the set of open stories with no shared dependency chain (safe to parallelize)",
|
||
"If a worktree merge fails (conflict), the worktree is preserved for manual resolution and reported clearly"
|
||
],
|
||
"priority": 15,
|
||
"passes": true,
|
||
"status": "done",
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/15-ralph-agent-agnostic-status-aware.md",
|
||
"dependsOn": [],
|
||
"completionNotes": "Implemented by agent: status-aware helpers (_is_done, _needs_attention, _is_active, _is_in_design), 6-bucket _story_sets, attention dashboard section, _review_report Attention Required block, auto --include-revise, set-agent subcommand with persistent agent-config.json, _run_openrouter stub, custom agent support, list-parallel subcommand, and auto --parallel N worktree orchestration. All 65 tests pass."
|
||
},
|
||
{
|
||
"id": "US-016",
|
||
"title": "16 — Mining-style node startup CLI + live dashboard",
|
||
"description": "Replace the bare flag-driven node CLI with a wizard-guided first-run experience (like a GPU mining client) followed by a live terminal dashboard once the node is running. On first run, the wizard auto-detects GPU VRAM, presents a curated list of compatible models with VRAM requirements at each quantization level, lets the user pick a download location, and writes a persistent config file so subsequent starts are one command. Once the node is running, the wizard gives way to a rich live status panel showing: GPU temp + VRAM used, tokens/sec, requests served, peers connected, TAI earned (stub until US-006 is live). A Browse HuggingFace option calls the HF Hub API so users can load any HF model beyond the curated list.",
|
||
"acceptanceCriteria": [
|
||
" with no args and no config file enters the interactive setup wizard",
|
||
"Wizard step 1: auto-detect GPU(s) via torch.cuda / torch.version.hip; print GPU name + total VRAM",
|
||
"Wizard step 2: show curated model list (name, HF repo, layers, VRAM@NF4/INT8/BF16); mark models that do NOT fit available VRAM as [too large]",
|
||
"Wizard step 3: offer [B] Browse HuggingFace — calls HF Hub API (huggingface_hub.list_models filtered by pipeline_tag=text-generation, sorted by downloads, top 20) and lets user enter a custom HF repo ID",
|
||
"Wizard step 4: prompt for download directory (default ~/.meshnet/models/); validate writable; show estimated disk usage for chosen model+quantization",
|
||
"Wizard step 5: prompt for tracker URL (default http://localhost:8080); validate connection",
|
||
"Wizard writes ~/.config/meshnet/config.json; second run skips wizard and starts directly",
|
||
"All wizard values overridable via CLI flags: --model, --download-dir, --quantization [bf16|int8|nf4], --tracker, --wallet, --reset-config",
|
||
"Once node is running, wizard clears and a live dashboard renders every 2s (rich.live): GPU util%, VRAM used/total, tokens/sec (EMA), requests served, TAI earned (stub 0.0), peers connected, uptime, current model/shard range",
|
||
"Dashboard exits cleanly on Ctrl-C with a summary line",
|
||
"Works inside WSL2 (no termios/ioctl calls that fail on Windows terminal; fall back to plain-text status if rich is not available)",
|
||
" passes from repo root",
|
||
"Commit only this story changes"
|
||
],
|
||
"priority": 16,
|
||
"status": "done",
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/16-mining-cli-ux.md",
|
||
"dependsOn": [
|
||
"US-004",
|
||
"US-012"
|
||
],
|
||
"completionNotes": "Implemented: mining-style wizard with GPU detection, curated model list (7 models with NF4/INT8/BF16 VRAM requirements), HF Hub browse, persistent config, rich live dashboard with plain-text WSL2 fallback. 19 tests, 97 passed total."
|
||
},
|
||
{
|
||
"id": "US-017",
|
||
"title": "17 — P2P gossip, NAT-traversal relay node, and SSL/TLS",
|
||
"description": "Add a gossip layer so nodes discover each other and propagate coverage-map changes without polling the tracker continuously. Introduce a publicly-hosted relay node (run by the team) that solves NAT traversal using circuit relay (Petals-style) and serves as the bootstrap peer list for new nodes. Encrypt all node-to-node and node-to-tracker communications with TLS. Gossip protocol: WebSocket-based PubSub over wss://, topics: node-join / node-leave / coverage-update / heartbeat. Peer discovery: mDNS (zeroconf) for LAN, public relay bootstrap list for internet. NAT traversal: relay node acts as TCP-level circuit relay when direct connection fails (hole-punching first, relay second). Architecture is designed to migrate to libp2p GossipSub + Kademlia DHT in a future story without breaking the message schema.",
|
||
"acceptanceCriteria": [
|
||
"All HTTP between nodes and tracker uses HTTPS (TLS 1.3); self-signed cert generated on first run and fingerprint pinned in config; relay node uses Let's Encrypt",
|
||
"Nodes broadcast node-join / node-leave events over wss:// to known peers within 1s of registration",
|
||
"mDNS peer discovery (Python zeroconf) finds other meshnet nodes on the same LAN segment without manual tracker URL entry",
|
||
"Public relay bootstrap list (hardcoded relay URL + ) is consulted when no LAN peers found",
|
||
"Relay node is a standalone meshnet package () with CLI: starts a WebSocket relay server + circuit relay + optional tracker proxy",
|
||
"When a node behind NAT cannot accept inbound connections, the relay forwards its traffic; node advertises relay address (relay_url/node_id) to tracker as its effective endpoint",
|
||
"Tracker accepts both direct node URLs and relay-proxied URLs in heartbeat payloads",
|
||
"Integration test: two nodes in separate processes on localhost (simulating NAT) communicate via a local relay process; inference request routes correctly",
|
||
"ADR-0010 documents the gossip protocol, relay architecture, and migration path to libp2p",
|
||
" passes from repo root",
|
||
"Commit only this story changes"
|
||
],
|
||
"priority": 17,
|
||
"status": "done",
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/17-p2p-gossip-relay-ssl.md",
|
||
"dependsOn": [
|
||
"US-013",
|
||
"US-014"
|
||
],
|
||
"completionNotes": "Implemented: packages/p2p (identity, TLS cert+fingerprint, GossipClient WSS PubSub, MdnsDiscovery with zeroconf optional), packages/relay (RelayServer gossip hub + circuit relay proxy, meshnet-relay CLI), tracker extended with relay_addr/cert_fingerprint/peer_id, relay_bootstrap.json, ADR-0010. 18 new tests; 115 total passed."
|
||
},
|
||
{
|
||
"id": "US-018",
|
||
"title": "18 — End-to-end two-machine LAN inference test",
|
||
"description": "Prove the network works across two real machines: the Linux rig (this machine) and a Windows 11 rig running WSL2. One machine runs the tracker + first-shard node (inference entry point). The other machine runs a second-shard node. A client sends a real inference request and receives a streamed response. This story is primarily a test plan + setup guide + test execution script; it produces documented evidence (logs, timing, token output) that real distributed inference works. It also surfaces any real-world issues (port forwarding, CUDA driver version mismatches, WSL2 CUDA passthrough, model download paths) that need fixing.",
|
||
"acceptanceCriteria": [
|
||
"docs/INSTALL_WINDOWS.md exists: step-by-step WSL2 + CUDA + meshnet-node install on Windows 11",
|
||
"docs/TWO_MACHINE_TEST.md exists: how to start tracker on machine A, node on machine B, run inference, interpret output",
|
||
"A test script scripts/test_lan_inference.py: given --tracker-url, --gateway-url, sends 3 chat completion requests, asserts valid OpenAI format, prints token count + latency + which nodes served each request",
|
||
"Both machines can reach each other on LAN (documented: firewall rules, port list)",
|
||
"At least one successful inference recorded: the test script exits 0 with output showing tokens generated and node IDs",
|
||
"Latency breakdown logged: gateway→node-A, node-A→node-B, node-B→gateway (approximate, from server logs)",
|
||
"Known issues during test documented in docs/TWO_MACHINE_TEST.md under a Known Issues section",
|
||
"Commit only this story changes"
|
||
],
|
||
"priority": 18,
|
||
"status": "done",
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/18-two-machine-lan-test.md",
|
||
"dependsOn": [
|
||
"US-016",
|
||
"US-017"
|
||
],
|
||
"completionNotes": "docs/INSTALL_WINDOWS.md: WSL2+CUDA+meshnet-node install guide. docs/TWO_MACHINE_TEST.md: two-machine LAN test procedure with known issues. scripts/test_lan_inference.py: stdlib-only test script, 3 requests, exit 0 on success, auto-discovers gateway from tracker."
|
||
},
|
||
{
|
||
"id": "US-019",
|
||
"title": "19 — Distributed tracker consensus (Raft assignments + CRDT heartbeats)",
|
||
"description": "Replace the single-point-of-failure tracker with a fault-tolerant cluster. Tracker nodes elect a leader via Raft and commit shard assignments as log entries — all tracker nodes agree on who owns what. Node liveness (heartbeats) uses CRDT gossip (eventual consistency, high frequency OK). A node registers with any tracker node; the write is forwarded to the leader and replicated to followers. A 3-node tracker cluster survives one tracker failure without losing assignment state. The relay/gossip layer already built in US-017 handles peer heartbeats; this story wires Raft on top for authoritative assignments.",
|
||
"acceptanceCriteria": [
|
||
"3 tracker nodes can be started and form a Raft cluster (leader election, log replication)",
|
||
"A node registers with any follower — the registration is forwarded to the leader and replicated",
|
||
"Killing the leader causes a new election within 5 seconds; registrations continue working",
|
||
"Shard assignments returned by any tracker node are identical (strong consistency)",
|
||
"Node heartbeats use CRDT gossip (not Raft) — high-frequency, eventual consistency",
|
||
"meshnet-tracker CLI gains --cluster-peers flag to specify peer tracker URLs",
|
||
"Integration test: 3 tracker nodes, kill leader mid-test, verify assignment still works",
|
||
"QUICKSTART.md updated with multi-tracker setup section"
|
||
],
|
||
"priority": 19,
|
||
"status": "done",
|
||
"notes": "Architecture decision: Raft for assignments (strong consistency) + CRDT gossip for liveness (eventual consistency). User approved 2026-06-29.",
|
||
"dependsOn": [
|
||
"US-017"
|
||
],
|
||
"completionNotes": "raft.py: minimal Raft consensus (leader election, log replication, AppendEntries, RequestVote). gossip.py: LWW CRDT gossip for node heartbeats. TrackerServer gains cluster_peers + cluster_self_url params. --cluster-peers and --self-url CLI flags added. 6 integration tests: leader election <1s, follower registration propagation, leader kill + re-election <5s, gossip table."
|
||
},
|
||
{
|
||
"id": "US-020",
|
||
"title": "20 — Tracker + node hardening: BrokenPipe fix, deterministic node IDs, HF coverage",
|
||
"description": "Polish pass on tracker and node after real multi-host LAN testing. Three concrete issues: (1) tracker _send_json crashed with BrokenPipeError when a slow-inference client disconnected mid-stream; (2) node IDs were random UUIDs, making re-registration after tracker restart create phantom entries; (3) HF-repo model coverage endpoint did not handle short-name vs full-repo lookups consistently.",
|
||
"acceptanceCriteria": [
|
||
"BrokenPipeError in tracker _send_json is silently swallowed (client disconnected is not an error)",
|
||
"Node IDs are deterministic (wallet-address + port hash) so re-registration after tracker restart reuses the same ID",
|
||
"GET /v1/coverage/<model> accepts both short names and full HF repo IDs",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 20,
|
||
"status": "done",
|
||
"notes": "Discovered during first two-machine LAN test (US-018 follow-up).",
|
||
"dependsOn": [
|
||
"US-018"
|
||
],
|
||
"completionNotes": "BrokenPipeError wrapped in try/except in tracker and node send paths. Node IDs now sha256(wallet+port)[:16]. Coverage endpoint normalises short/full HF names."
|
||
},
|
||
{
|
||
"id": "US-021",
|
||
"title": "21 — --route-timeout CLI flag for node tracker route lookup",
|
||
"description": "The node's slow-path tracker route lookup used a hard-coded 30-second timeout. On high-latency networks (relay, 5G) or when the tracker is slow to respond, this caused premature failures. Expose it as a CLI flag so operators can tune it per deployment.",
|
||
"acceptanceCriteria": [
|
||
"meshnet-node start accepts --route-timeout <seconds> (float, default 30.0)",
|
||
"route_timeout is passed through to TorchNodeServer and used in tracker /v1/route HTTP call",
|
||
"TorchNodeServer exposes route_timeout as a readable property",
|
||
"Test: setting a non-default timeout is reflected on the running server object",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 21,
|
||
"status": "done",
|
||
"notes": "Small QoL story; acceptance criteria driven by a failing test.",
|
||
"dependsOn": [
|
||
"US-016"
|
||
],
|
||
"completionNotes": "cli.py: --route-timeout added to start_cmd. TorchNodeServer._route_timeout stored, route_timeout property exposed. Test: test_route_timeout_config_is_exposed_on_server."
|
||
},
|
||
{
|
||
"id": "US-022",
|
||
"title": "22 — X-Meshnet-Start-Layer: overlapping shard execution protocol",
|
||
"description": "When two nodes register overlapping shard ranges (e.g. node A covers 0-15 and node B covers 12-23), the naive pipeline re-runs layers 12-15 on node B. The X-Meshnet-Start-Layer header tells each downstream node which layer index to start from, skipping already-computed layers. The tracker injects start_layer into X-Meshnet-Route hops at proxy time.",
|
||
"acceptanceCriteria": [
|
||
"Tracker _handle_proxy_chat builds route hops with start_layer computed from covered_up_to",
|
||
"Node _handle_binary_forward reads X-Meshnet-Start-Layer and passes it to backend.forward_bytes",
|
||
"Node _get_remaining_route parses start_layer from both injected header and /v1/route slow-path",
|
||
"TorchModelShard.forward_bytes accepts optional start_layer and skips layers below it",
|
||
"Test: two-node route with overlapping shards produces correct output without double-computing layers",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 22,
|
||
"status": "done",
|
||
"notes": "Protocol decision: option A (start_layer injected by tracker, not negotiated peer-to-peer). Approved in design session.",
|
||
"dependsOn": [
|
||
"US-014",
|
||
"US-019"
|
||
],
|
||
"completionNotes": "X-Meshnet-Start-Layer header added to forward protocol. Tracker computes covered_up_to as it builds route_hops. model_backend.forward_bytes(start_layer=) implemented. Tests added for overlapping shard scenarios."
|
||
},
|
||
{
|
||
"id": "US-023",
|
||
"title": "23 — Heartbeat stats payload: request counters + dynamic reassignment response",
|
||
"description": "Node heartbeats are currently empty POSTs. Extend them to carry cumulative stats (total_requests, failed_requests, queue_depth, uptime_seconds) so the tracker can make informed load-balancing decisions. The heartbeat response may include a new_assignment field, enabling the tracker to redirect a node to a different shard without a restart.",
|
||
"acceptanceCriteria": [
|
||
"Heartbeat POST body includes total_requests, failed_requests, queue_depth, uptime_seconds, status",
|
||
"TorchNodeServer tracks total_requests, failed_requests, queue_depth with a threading.Lock",
|
||
"Tracker stores the last heartbeat payload per node and exposes it in /v1/nodes (list endpoint)",
|
||
"Heartbeat response may include new_assignment: {model, shard_start, shard_end}; node logs it",
|
||
"Stats survive tracker outage: buffered locally, flushed on next successful heartbeat",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 23,
|
||
"status": "done",
|
||
"notes": "Reassignment is logged only for now; hot-reload (load new shard without restart) is a future story.",
|
||
"dependsOn": [
|
||
"US-016"
|
||
],
|
||
"completionNotes": "torch_server.py: total_requests, failed_requests, queue_depth added with _stats_lock. startup.py _start_heartbeat sends stats dict. Tracker stores last_heartbeat per node. new_assignment field in heartbeat response logged by node."
|
||
},
|
||
{
|
||
"id": "US-024",
|
||
"title": "24 — Enhanced availability map with per-node health details",
|
||
"description": "GET /v1/coverage/<model> returns coarse band-level coverage. Operators need to know which specific nodes are in each band and whether they are healthy (last heartbeat time, queue depth). Extend the response to include a nodes array per band with per-node health details, and add a separate detailed endpoint.",
|
||
"acceptanceCriteria": [
|
||
"GET /v1/coverage/<model> response includes nodes: [{node_id, endpoint, healthy, queue_depth, last_seen_s}] per band",
|
||
"healthy is true iff last heartbeat < heartbeat_timeout seconds ago",
|
||
"Existing band fields (start_layer, end_layer, node_count) are preserved",
|
||
"Tests updated: band assertions check nodes array not just node_count",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 24,
|
||
"status": "done",
|
||
"notes": "Grouped-by-band format chosen (both node list and band metadata in same object). Dead nodes included in response with healthy=false.",
|
||
"dependsOn": [
|
||
"US-013"
|
||
],
|
||
"completionNotes": "_coverage_map_detailed added to tracker. Each band now includes nodes list. Tests updated to check band fields individually rather than exact dict comparison."
|
||
},
|
||
{
|
||
"id": "US-025",
|
||
"title": "25 — Model usage statistics: rolling RPM windows + SQLite persistence",
|
||
"description": "Trackers need to know which models are being requested most often to make smart load-balancing and assignment decisions. Add per-model rolling request-rate counters (last hour, last day, last month) with a circular-bucket implementation. Persist buckets to SQLite so stats survive tracker restarts. Support gossip-based stat sharing between tracker peers (additive merge, per-tracker slices).",
|
||
"acceptanceCriteria": [
|
||
"_RollingCounter: circular buckets, epoch-indexed, stale buckets auto-reset on record()",
|
||
"_StatsCollector: record_request(), get_local_rpms(), merge_peer_rpms(), get_combined_stats()",
|
||
"Stats persisted to SQLite (--stats-db PATH); loaded on startup",
|
||
"GET /v1/stats returns combined stats (local + peer slices) as {model: {rpm_last_hour, rpm_last_day, rpm_last_month}}",
|
||
"POST /v1/stats/gossip accepts a peer's local slice and merges it additively",
|
||
"_handle_proxy_chat records a stat after model is determined",
|
||
"6 unit tests covering counter, collector, merge, SQLite round-trip, gossip endpoint",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 25,
|
||
"status": "done",
|
||
"notes": "Option B chosen: stats stored locally per tracker and synced via gossip (not aggregated centrally). Rolling windows: 60×1min buckets (1 hour), 24×1hr (1 day), 30×1day (~1 month).",
|
||
"dependsOn": [
|
||
"US-017"
|
||
],
|
||
"completionNotes": "_RollingCounter and _ModelStats classes in server.py. _StatsCollector with SQLite save/load. /v1/stats and /v1/stats/gossip endpoints. --stats-db CLI flag. Stats gossip in _stats_loop thread."
|
||
},
|
||
{
|
||
"id": "US-026",
|
||
"title": "26 — Smart model assignment via demand×coverage scoring",
|
||
"description": "The /v1/network/assign endpoint assigns new nodes to whichever model and shard range covers the biggest uncovered gap. This ignores demand: a model with 1000 RPM and 60% coverage should attract more nodes than a zero-traffic model with 50% coverage. Add a scoring formula: score = (demand_rpm + 1.0) × (coverage_deficit + 0.01) so high-demand under-covered models win. price_per_token is reserved in the protocol response at 0.0.",
|
||
"acceptanceCriteria": [
|
||
"/v1/network/assign returns the highest-scoring model+shard using demand×coverage formula",
|
||
"demand_rpm uses combined stats from _StatsCollector (local + peer slices)",
|
||
"coverage_deficit is fraction of layers with zero coverage [0.0, 1.0]",
|
||
"price_per_token: 0.0 included in response (reserved field)",
|
||
"Models with no traffic still compete by coverage (floor: demand_rpm + 1.0)",
|
||
"Test: two models, one high-demand low-coverage wins over low-demand high-coverage",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 26,
|
||
"status": "done",
|
||
"notes": "Score formula chosen after grilling: (demand_rpm + 1.0) × (coverage_deficit + 0.01). The +1.0 floor means coverage alone drives assignment when demand is zero.",
|
||
"dependsOn": [
|
||
"US-025",
|
||
"US-024"
|
||
],
|
||
"completionNotes": "_handle_network_assign updated with scoring. _effective_throughput helper. price_per_token: 0.0 in response. Tests in test_tracker_routing.py."
|
||
},
|
||
{
|
||
"id": "US-027",
|
||
"title": "27 — Throughput-optimized routing: effective throughput as tiebreak",
|
||
"description": "The greedy max-reach route selection picks nodes by shard coverage but does not consider node speed. When two nodes cover the same remaining range, prefer the one with higher effective throughput (benchmark_tokens_per_sec / (queue_depth + 1)). This is a tiebreak only — coverage maximization remains the primary objective.",
|
||
"acceptanceCriteria": [
|
||
"_effective_throughput(node) = benchmark_tokens_per_sec / (queue_depth + 1)",
|
||
"_select_route uses throughput as tiebreak when shard_end is equal",
|
||
"Test: two nodes with same shard range, different throughput — faster node is picked",
|
||
"Existing greedy coverage tests still pass (throughput does not change primary selection)",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 27,
|
||
"status": "done",
|
||
"notes": "Throughput tiebreak only — coverage-maximizing greedy stays primary. queue_depth from last heartbeat.",
|
||
"dependsOn": [
|
||
"US-023"
|
||
],
|
||
"completionNotes": "_effective_throughput added to server.py. _select_route updated: when shard_end equal, max throughput wins. Tests added and existing tests corrected for new tiebreak order."
|
||
},
|
||
{
|
||
"id": "US-028",
|
||
"title": "28 — Routing correctness tests: three-node, overlap, and throughput scenarios",
|
||
"description": "Add a comprehensive test suite for route selection covering: greedy three-node no-overlap ordering, overlapping shard handling, throughput tiebreak, and gap detection. These tests lock in the routing contract so refactors don't silently regress.",
|
||
"acceptanceCriteria": [
|
||
"test_select_route_no_overlap_three_nodes: greedy picks nodes in layer order (A→C→B for ranges 0-7, 8-15, 16-23)",
|
||
"test_select_route_with_overlap: overlapping nodes correctly resolved",
|
||
"test_select_route_throughput_tiebreak: faster node wins when reach is equal",
|
||
"test_select_route_gap_leaves_error: partial coverage returns error",
|
||
"All tests pass without mocking (in-process tracker server)",
|
||
"python -m pytest passes"
|
||
],
|
||
"priority": 28,
|
||
"status": "done",
|
||
"notes": "Tests revealed the greedy picks A→C→B (not A→B→C) for non-overlapping ranges when C starts at a lower layer than B. Test expectation corrected to match algorithm.",
|
||
"dependsOn": [
|
||
"US-027"
|
||
],
|
||
"completionNotes": "7 routing tests added to test_tracker_routing.py. _make_node helper. test_select_route_no_overlap_three_nodes corrected: greedy outputs [A, C, B] when C.shard_start < B.shard_start."
|
||
},
|
||
{
|
||
"id": "US-029",
|
||
"title": "29 — Outbound relay client: NAT/internet pipeline hops",
|
||
"description": "Nodes behind NAT (WSL2, 5G, home routers) register with the tracker including a relay_addr (wss://relay/rpc/{peer_id}). When the head node needs to forward activations to a behind-NAT peer, it must use the relay instead of direct HTTP. Add _relay_hop() to torch_server.py that opens a per-hop WebSocket to the relay's /rpc/{peer_id} endpoint, sends the binary activation (base64-encoded), and returns the response. If relay fails, fall back to direct HTTP.",
|
||
"acceptanceCriteria": [
|
||
"_relay_hop(relay_addr, path, body, headers, timeout) opens WS to relay, sends body_base64, returns (status, headers, body)",
|
||
"_get_remaining_route returns list[dict] with relay_addr field (was list[tuple])",
|
||
"_run_downstream_pipeline dispatches via _relay_hop when hop has relay_addr",
|
||
"Falls back to direct HTTP if relay connection fails (logs warning)",
|
||
"Tracker _handle_proxy_chat includes relay_addr in downstream hop dicts",
|
||
"relay_bridge._handle_request decodes body_base64; response uses body_base64 for binary (octet-stream)",
|
||
"All 157 tests pass",
|
||
"QUICKSTART.md updated with relay NAT/internet section"
|
||
],
|
||
"priority": 29,
|
||
"status": "done",
|
||
"notes": "relay_addr format: wss://relay.../rpc/{peer_id}. Binary activations (bfloat16) base64-encoded through relay JSON protocol — no precision loss. WSL2 nodes now work behind NAT without --advertise-host.",
|
||
"dependsOn": [
|
||
"US-017",
|
||
"US-022"
|
||
],
|
||
"completionNotes": "_relay_hop() added to torch_server.py. _get_remaining_route returns list[dict]. relay_bridge.py updated with body_base64 support. Tracker injects relay_addr into downstream hop dicts. 157 tests pass."
|
||
},
|
||
{
|
||
"id": "US-030",
|
||
"title": "30 — Manual route selection + hop-penalty benchmarking",
|
||
"description": "Two additions to the tracker. (1) Callers can pin an explicit inference route by passing an optional \"route\": [\"<node_id>\", ...] field in the POST /v1/chat/completions body. The tracker uses those nodes in order instead of auto-selecting; clients that omit the field are unaffected. (2) A new privileged POST /v1/benchmark/hop-penalty endpoint runs the same prompt through up to three routes (1-node, 2-node, 3-node) using whatever nodes are registered, records per-hop latency, and appends results to benchmark_results.json in the tracker's working directory. The routing algorithm is not changed — this story is data collection only. Auth is a header-presence stub (non-empty Authorization header required for benchmark endpoints).",
|
||
"acceptanceCriteria": [
|
||
"POST /v1/chat/completions accepts optional \"route\": [node_id, ...] in the request body; if present, tracker routes to those nodes in order; if absent, existing auto-routing is unchanged",
|
||
"Missing or invalid node IDs in route return HTTP 400 with a descriptive error message",
|
||
"POST /v1/benchmark/hop-penalty requires a non-empty Authorization header; missing/empty returns HTTP 401",
|
||
"Benchmark body: {\"model\": \"...\", \"prompt\": \"...\", \"max_new_tokens\": 64 (optional)}",
|
||
"Benchmark fans out to up to 3 routes (1-node, 2-node, 3-node) using currently registered nodes; routes with insufficient coverage are skipped, not errored",
|
||
"Benchmark response includes per-route entries: {\"route\": [node_id, ...], \"total_ms\": float, \"per_hop_ms\": [float, ...], \"tokens_generated\": int}",
|
||
"Results appended to <tracker_cwd>/benchmark_results.json (created if absent) as a JSON array; each entry includes ISO timestamp, model, sha256 of prompt, and per-route breakdown",
|
||
"GET /v1/benchmark/results returns the stored results array; also requires non-empty Authorization header",
|
||
"Integration test: pin a 1-node route and a 2-node route for the same prompt; assert 2-node result has 2 per_hop_ms entries; assert both records appear in benchmark_results.json",
|
||
"Clients that never send route or call /v1/benchmark/* are completely unaffected (existing tests pass unchanged)",
|
||
"python -m pytest passes from repo root",
|
||
"Commit only this story's changes"
|
||
],
|
||
"priority": 30,
|
||
"status": "done",
|
||
"notes": "Source issue: .scratch/distributed-inference-network/issues/30-manual-route-and-hop-benchmark.md. Design decisions grilled 2026-07-01: route via body field, explicit-only benchmark trigger, auth stub, routing algorithm unchanged, persist to JSON file.",
|
||
"dependsOn": [
|
||
"US-014",
|
||
"US-019"
|
||
],
|
||
"completionNotes": "Pinned routes: optional \"route\": [node_id,...] in POST /v1/chat/completions (400 on unknown/invalid ids; absent field unchanged). POST /v1/benchmark/hop-penalty + GET /v1/benchmark/results, both auth-gated (non-empty Authorization). Fans out to 1/2/3-node routes via _find_pinned_route; per-hop latency derived incrementally (k-node total minus (k-1)-node total); appends to benchmark_results.json (path configurable). 6 tests in tests/test_manual_route_benchmark.py."
|
||
},
|
||
{
|
||
"id": "US-031",
|
||
"title": "31 — Billing ledger: per-token pricing, 90/10 split, pending balances",
|
||
"description": "Tracker-side off-chain billing per ADR-0015. Each model preset has a USDT price per 1K tokens in tracker config. On request completion the tracker debits the client's API-key ledger balance and credits 90% to the serving nodes' pending balances proportional to work_units (layers × tokens), 10% to the protocol cut. Requests from API keys with insufficient balance are rejected with 402. Ledger persists in the existing tracker SQLite and replicates across the tracker hive.",
|
||
"acceptanceCriteria": [
|
||
"Per-model price_per_1k_tokens in tracker config with sane defaults",
|
||
"Completed request debits client ledger: price × total_tokens / 1000",
|
||
"90% split across serving nodes by work_units; 10% accrues to protocol_cut",
|
||
"Insufficient balance → HTTP 402 before routing (no free work)",
|
||
"Balances survive tracker restart (SQLite) and replicate to hive followers",
|
||
"Unit tests: single-node route, 3-node split, exhausted balance, restart persistence"
|
||
],
|
||
"priority": 31,
|
||
"status": "done",
|
||
"notes": "Pure off-chain — no Solana calls in this story. Reuses ComputeAttribution/work_units from packages/contracts.",
|
||
"dependsOn": [
|
||
"US-023",
|
||
"US-025"
|
||
],
|
||
"completionNotes": "BillingLedger in packages/tracker/meshnet_tracker/billing.py: event-sourced USDT ledger (credit/charge/payout/forfeit events, id-deduped), SQLite persistence, per-model price_per_1k_tokens (preset key or prices dict), 90/10 split by work units, walletless shares to protocol cut. server.py: 401/402 gate before routing, billing on non-streaming/streaming/relayed completion paths, GET /v1/billing/summary, POST /v1/billing/gossip, event push in _stats_loop (cursor advances only when all peers reached). CLI --billing-db. settle_node_payout/forfeit_pending are the US-033/034 hooks. 11 new tests in tests/test_billing_ledger.py; suite 195 passed (3 pre-existing ModuleNotFoundError: openai failures unrelated)."
|
||
},
|
||
{
|
||
"id": "US-032",
|
||
"title": "32 — Devnet custodial treasury: mock-USDT mint + deposit watcher",
|
||
"description": "Real Solana adapter behind the packages/contracts boundary, custodial model per ADR-0015. A setup script creates the mock-USDT SPL mint (6 decimals) and treasury token account on devnet. Clients register a wallet pubkey against their API key; the deposit watcher polls the treasury token account and credits the sender's API-key ledger balance on confirmed USDT transfers. Mint address, RPC URL, and treasury keypair path are config (.env.devnet).",
|
||
"acceptanceCriteria": [
|
||
"scripts/devnet_setup.py creates mint + treasury ATA and prints .env.devnet values",
|
||
"POST /v1/wallet/register binds client wallet pubkey to API key",
|
||
"Deposit watcher credits ledger within one poll interval of a confirmed transfer",
|
||
"Duplicate/replayed transactions are credited exactly once (signature dedupe)",
|
||
"Local solana-test-validator integration test covers mint→deposit→credit flow",
|
||
"Deterministic local adapter still works for CI without any validator"
|
||
],
|
||
"priority": 32,
|
||
"status": "done",
|
||
"notes": "Supersedes 'testnet never devnet' note in issue 06. solana-py + spl-token client libs.",
|
||
"dependsOn": [
|
||
"US-031"
|
||
],
|
||
"completionNotes": "SolanaCustodialTreasury in packages/contracts/meshnet_contracts/solana_adapter.py: urllib JSON-RPC client + solders signing (installed solana-py 0.40 has no sync client), deposit parsing from jsonParsed token balances, batched payouts, devnet helpers (create_mock_usdt_mint, mint_mock_usdt). scripts/devnet_setup.py writes .env.devnet. POST /v1/wallet/register binds wallet→api_key (replicated ledger 'bind' events). Deposit watcher thread credits exactly-once via deposit-<signature> event ids. Tests: fake-treasury watcher tests + skipif-gated solana-test-validator e2e."
|
||
},
|
||
{
|
||
"id": "US-033",
|
||
"title": "33 — Settlement loop: leader-only batched USDT payouts",
|
||
"description": "The Raft leader runs the settlement loop per ADR-0015: pay a node when pending ≥ payout_threshold OR time since its last payout ≥ max_period, whichever fires first, with a dust floor. Payouts are batched SPL transfers treasury→node wallet on devnet. Dev defaults: period 60s, threshold ~0; prod defaults: period 24h, threshold few USDT — all dynamic config. Settlement history (tx signature, node, amount, epoch) persists and replicates.",
|
||
"acceptanceCriteria": [
|
||
"Only the Raft leader settles; followers never sign (asserted in a 3-tracker test)",
|
||
"Trigger: pending ≥ threshold OR elapsed ≥ max_period; dust floor respected",
|
||
"Batched SPL transfers land on devnet; pending balances zeroed atomically with recorded tx signature",
|
||
"Failed/timeout transactions retry without double-pay (idempotent by settlement id)",
|
||
"Settlement history queryable via tracker HTTP endpoint",
|
||
"End-to-end devnet test: fund client → run inference → observe node wallet USDT balance increase"
|
||
],
|
||
"priority": 33,
|
||
"status": "done",
|
||
"notes": "Mining-pool standard: threshold + max-interval, dust floor. Treasury key only on settlement-capable trackers.",
|
||
"dependsOn": [
|
||
"US-019",
|
||
"US-032"
|
||
],
|
||
"completionNotes": "Settlement loop in TrackerServer: leader-only (raft.is_leader; standalone settles), trigger pending≥threshold OR pending age≥settle_period, dust floor. Pending debited via payout events referencing settlement id BEFORE send; unconfirmed batches resent with same id → no double-pay. confirm via 'settlement' event with tx signature. GET /v1/billing/settlements. CLI: --settle-period/--payout-threshold/--payout-dust-floor (prod 24h/5/0.01; dev 60/0). Banned wallets skipped. 6 tests in tests/test_settlement_loop.py incl. non-leader-never-signs."
|
||
},
|
||
{
|
||
"id": "US-034",
|
||
"title": "34 — Hardened proof-of-work: pending-balance forfeiture penalty",
|
||
"description": "Wire the validator's ~5% sampling to the new penalty per ADR-0015: on confirmed output divergence the node's entire pending balance is forfeited to the protocol cut, a strike is recorded, and three strikes ban the wallet (tracker rejects registration and excludes from routes). Probationary period (first N jobs unpaid) is retained as the re-entry cost. Penalty math documented: at 5% sampling, forfeiting ~a day's pending earnings ≫ 20× per-job cheat gain.",
|
||
"acceptanceCriteria": [
|
||
"Validator divergence → pending balance forfeited to protocol_cut in the same ledger transaction as the strike",
|
||
"3rd strike bans wallet: registration rejected, excluded from all routes",
|
||
"Banned wallet's unpaid pending balance is not paid out at next settlement",
|
||
"Probation: first N jobs (default 50) accrue no pending balance; node CLI shows remaining",
|
||
"Integration test: deliberately-bad node loses pending, accrues strikes, gets banned within 60 requests",
|
||
"Slash/forfeiture events visible in tracker logs and settlement history"
|
||
],
|
||
"priority": 34,
|
||
"status": "done",
|
||
"notes": "No stake deposit — pending balance IS the collateral. Amends ADR-0003 penalty; sampling mechanics unchanged.",
|
||
"dependsOn": [
|
||
"US-031"
|
||
],
|
||
"completionNotes": "ValidatorProcess(billing=ledger) forfeits entire pending balance on divergence (same cycle as strike). Tracker POST /v1/billing/forfeit (auth stub) for remote validators: forfeit + strike + ban state. Probation wired into _bill_completed: wallets with probationary_jobs_remaining>0 have their share redirected to protocol cut; record_completed_job per request. Penalty math in packages/validator/README.md. 5 tests in tests/test_forfeiture_penalty.py; banned-never-paid asserted in settlement tests."
|
||
},
|
||
{
|
||
"id": "US-035",
|
||
"title": "35 — Tracker web dashboard",
|
||
"description": "Web dashboard served by the tracker (single-page, no build step — plain HTML/JS polling tracker HTTP endpoints). Shows: hive membership and Raft leader, node registry with health/scores/coverage map, client ledger balances, node pending balances, settlement history with devnet explorer links, strikes/bans, and rolling RPM stats. Read-only in this story; every tracker in the mesh can serve it from its replicated state.",
|
||
"acceptanceCriteria": [
|
||
"GET /dashboard serves the UI from any tracker (leader or follower)",
|
||
"Panels: hive/leader, nodes+coverage, client balances, pending payouts, settlement history (with tx links), strikes/bans, RPM stats",
|
||
"Auto-refresh ≤5s without page reload",
|
||
"No new build toolchain — static assets embedded in the tracker package",
|
||
"Works against a 3-tracker hive in the two-machine LAN test setup"
|
||
],
|
||
"priority": 35,
|
||
"status": "done",
|
||
"notes": "CLI dashboard already exists from US-016; this is the web counterpart. Write ops (price config, manual settle) deferred.",
|
||
"dependsOn": [
|
||
"US-031",
|
||
"US-033"
|
||
],
|
||
"completionNotes": "GET /dashboard served from embedded dashboard.html (package-data, no build step) by any tracker. Panels: hive/leader (raft status), nodes+coverage grouped by model, client balances, node pending + protocol cut, settlement history with devnet explorer links, strikes/bans/forfeitures (GET /v1/registry/wallets + snapshot forfeits), RPM stats. 4s auto-refresh via fetch polling. 3 tests in tests/test_dashboard.py."
|
||
},
|
||
{
|
||
"id": "US-036",
|
||
"title": "36 — Streamed chat completions over the relay RPC path",
|
||
"description": "Public NAT deployments proxy every chat request tracker → relay → head node. Implement true multi-frame SSE streaming over the relay WebSocket so clients see live tokens and relayed streams bill through the same SSE accounting loop as direct proxy streams. Inter-node /forward activation hops stay single-frame (ADR-0014).",
|
||
"acceptanceCriteria": [
|
||
"stream: true chat via relay delivers SSE chunks incrementally (≥2 distinct frame arrivals before [DONE])",
|
||
"Relayed streamed request records nonzero billed tokens and node work credit",
|
||
"Non-streamed relayed requests and /forward binary hops behave exactly as before (single frame, body_base64 intact)",
|
||
"Legacy single-frame response from an old node is accepted as terminal",
|
||
"Idle stream (no frame for 120 s) returns 504 and cleans up the relay-side queue",
|
||
"Extend tests/test_gossip_and_relay.py alongside test_relay_rpc_round_trips_http_request_to_peer",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 36,
|
||
"status": "needs-review",
|
||
"notes": "Source issue: docs/issues/36-relay-streamed-chat.md. Implemented via _stream_relayed_frames in server.py; verify on public NAT relay before friends-test.",
|
||
"dependsOn": [
|
||
"US-029",
|
||
"US-031"
|
||
],
|
||
"completionNotes": "Multi-frame relay-http-response protocol; node relay_bridge line-by-line SSE emit; relay server per-request asyncio.Queue; tracker _stream_relayed_frames with SSE billing parity. Client mid-stream disconnect accepted limitation for alpha."
|
||
},
|
||
{
|
||
"id": "US-037",
|
||
"title": "37 — Concurrent request handling in the node relay bridge",
|
||
"description": "RelayHttpBridge currently handles relay-http-request envelopes serially, blocking up to 300 s per request. Off-LAN a node can be head of one route and downstream hop of another — overlapping routes through a shared node break. Dispatch on a bounded ThreadPoolExecutor (default 8, configurable) with per-frame WS send locking compatible with US-036 streaming.",
|
||
"acceptanceCriteria": [
|
||
"While one relayed request is in flight, a second relay-http-request to the same node completes without waiting for the first",
|
||
"Responses are correctly matched by request_id when they complete out of order",
|
||
"More than N simultaneous requests queue and all eventually complete; thread count never exceeds N workers",
|
||
"Bridge survives a relay reconnect with workers still in flight (no crash, no deadlock; orphaned responses dropped)",
|
||
"Configurable via meshnet-node start --relay-concurrency N (env MESHNET_RELAY_CONCURRENCY)",
|
||
"Extend tests/test_gossip_and_relay.py",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 37,
|
||
"status": "open",
|
||
"notes": "Source issue: docs/issues/37-relay-bridge-concurrency.md. Critical for public friends-test; blocks concurrent head + hop on same node.",
|
||
"dependsOn": [
|
||
"US-036"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-038",
|
||
"title": "38 — Tracker cluster join via a single seed peer",
|
||
"description": "Tracker cluster membership is static today — a newcomer configured with only one existing peer is never learned by the rest of the hive and quorum math diverges. A joining tracker configured with any one live seed announces via hive-HMAC-signed POST /v1/cluster/join; membership changes replicate through the Raft log and persist across restarts.",
|
||
"acceptanceCriteria": [
|
||
"Start trackers A+B; start C with only A as seed → within one election timeout A, B, and C report the same 3-peer membership on GET /v1/cluster/peers, and a value proposed on C commits on A and B",
|
||
"Join without a valid hive signature is rejected with 403; join to a follower is forwarded to the leader",
|
||
"Restarting C with its seed offline rejoins from persisted membership",
|
||
"Standalone tracker (no seeds) behaves exactly as today",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 38,
|
||
"status": "open",
|
||
"notes": "Source issue: docs/issues/38-tracker-seed-join.md. Out of scope: peer removal, joint consensus, automatic seed retry.",
|
||
"dependsOn": [
|
||
"US-013",
|
||
"US-017"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-039",
|
||
"title": "39 — Caller Credit granted once per account; chat requires account keys",
|
||
"description": "DEFAULT_STARTING_CREDIT=0 and no grant path leaves every fresh public tracker request at 402. Grant Caller Credit once per account on first API key creation via deterministic event id caller-credit-{account_id}; chat on accounts-enabled trackers requires a real active sk-mesh- key (401 for invented bearers).",
|
||
"acceptanceCriteria": [
|
||
"Fresh account → first key → key has --starting-credit balance; chat succeeds",
|
||
"Second key on the same account → no additional credit",
|
||
"Revoke-and-recreate keys → still no additional credit (deterministic event id)",
|
||
"Random bearer string on an accounts-enabled tracker → 401, never 402/free work",
|
||
"Tracker without accounts store: gate behavior unchanged",
|
||
"--starting-credit 0 disables the grant entirely (mainnet posture)",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 39,
|
||
"status": "open",
|
||
"notes": "Source issue: docs/issues/39-caller-credit-account-keys.md. Critical for friends-test inference.",
|
||
"dependsOn": [
|
||
"US-031",
|
||
"US-035"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-040",
|
||
"title": "40 — Devnet top-up button on the dashboard",
|
||
"description": "After Caller Credit (US-039) is spent, devnet friends need a dashboard faucet refill without on-chain USDT deposits. POST /v1/account/topup (session-authenticated) credits a configured fixed amount per click; flag off returns 404 and hides the button.",
|
||
"acceptanceCriteria": [
|
||
"Flag off: endpoint 404s, dashboard shows no top-up button",
|
||
"Flag on: logged-in user tops up own key, balance rises by exactly N",
|
||
"Topping up another account's key → 403",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 40,
|
||
"status": "open",
|
||
"notes": "Source issue: docs/issues/40-devnet-dashboard-topup.md. Mainnet deployments set --devnet-topup 0.",
|
||
"dependsOn": [
|
||
"US-039"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-041",
|
||
"title": "41 — Account wallet: browser-extension signing, in-browser generation, export-only",
|
||
"description": "Accounts need a visible wallet for deposit attribution without the tracker ever holding private keys. Dashboard integrates Solana wallet-adapter connect+nonce proof, or in-browser keypair generation with one-time export; no private-key import endpoint.",
|
||
"acceptanceCriteria": [
|
||
"Connect-extension flow stores a verified pubkey (rejects unsigned/mismatched nonce proofs)",
|
||
"Generate flow: pubkey lands on the account; private key is never sent to the tracker, export works",
|
||
"No endpoint or UI accepts a private key",
|
||
"Deposits to the shown address credit the account's keys via the existing watcher",
|
||
"Address visible on the account panel after either flow",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 41,
|
||
"status": "open",
|
||
"notes": "Source issue: docs/issues/41-account-wallet-keypair.md. Not needed for devnet friends test; needed before mainnet.",
|
||
"dependsOn": [
|
||
"US-032",
|
||
"US-039"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-042",
|
||
"title": "42 — GGUF/llama.cpp node backend (phase C whole-model first)",
|
||
"description": "Node backend is transformers-only today; large MoE models on consumer hardware require GGUF via llama.cpp. Phase C: whole-model GGUF nodes (single-hop routes) first; partial-layer distributed GGUF deferred to ADR-0024. Also: GGUF catalog entries, Strix Halo/Vulkan hardware detection, download dir applies to GGUF files.",
|
||
"acceptanceCriteria": [
|
||
"A node with --gguf <repo-or-path> --quant IQ3_XXS serves /v1/chat/completions via llama.cpp with GPU offload where available",
|
||
"Tracker treats it as a full-coverage node (single-hop routes, billing works)",
|
||
"Streamed responses work through the tracker proxy and the relay (US-036)",
|
||
"python -m pytest passes from repo root (llama.cpp behind an optional extra)"
|
||
],
|
||
"priority": 42,
|
||
"status": "in-design",
|
||
"notes": "Source issue: docs/issues/42-gguf-llamacpp-node-backend.md. Distributed native path: ADR-0024. Sequencing: phase C before A/B investigation.",
|
||
"dependsOn": [
|
||
"US-036"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-043",
|
||
"title": "43 — Dashboard model search and model cards",
|
||
"description": "Dashboard lacks model-centric discovery. Add server-side HF search proxy merged with tracker presets and live coverage; model cards show architecture, coverage gaps, pricing, memory per quant, and a request-this-model action. Featured section driven by CURATED_MODELS including GGUF once US-042 lands.",
|
||
"acceptanceCriteria": [
|
||
"Searching a HF repo id or free text returns results without the browser calling HF directly",
|
||
"A served model's card shows live coverage and a working chat-now state",
|
||
"An unserved model's card shows the request action and estimated memory per quant",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 43,
|
||
"status": "open",
|
||
"notes": "Source issue: docs/issues/43-dashboard-model-search-cards.md. Post-deploy polish.",
|
||
"dependsOn": [
|
||
"US-035"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-044",
|
||
"title": "44 — Tracker as model-file source; nodes download only their shard",
|
||
"description": "Second nodes joining a fleet today download entire HF snapshots even for small shard assignments. Tracker --models-dir advertises layer-scoped safetensors subsets; nodes race tracker/peer sources before HF allow_patterns fallback. Hard half remaining: meta-device partial model materialization so resident memory scales with shard size, not full model size.",
|
||
"acceptanceCriteria": [
|
||
"Tracker started with --models-dir / MESHNET_MODELS_DIR advertises local model-file sources in assignment responses",
|
||
"Tracker serves a tar stream (or per-file API) containing only safetensors files for the assigned layer range plus config/tokenizer/index metadata",
|
||
"Node downloader tries exact-shard peers, then tracker/peer file subsets, then HF snapshot_download with allow_patterns — never silently full-repo when layer index is available",
|
||
"Two-machine test: machine B receives only its assigned range from machine A — nothing fetched from HF",
|
||
"Machine B resident memory scales with its shard size, not model size",
|
||
"Checksums verified end-to-end; corrupted transfer falls back cleanly",
|
||
"Single-node/full-model flows unchanged",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 44,
|
||
"status": "in-progress",
|
||
"notes": "Source issue: docs/issues/44-tracker-shard-source-partial-download.md. Download path largely implemented 2026-07-06; partial LOAD (meta-device materialization) and two-machine acceptance remain.",
|
||
"dependsOn": [
|
||
"US-004",
|
||
"US-012"
|
||
],
|
||
"completionNotes": "Tracker models-dir indexing, layer-scoped tar stream, HF allow_patterns client-side from remote index, per-file download API with retries, symlink dereference in tar writers. Remaining: true partial model load and live two-machine verification."
|
||
},
|
||
{
|
||
"id": "US-045",
|
||
"title": "45 — Dual-rate billing: separate input and output token prices",
|
||
"description": "Ledger has one price_per_1k_tokens and stream vs non-stream paths disagree on input vs output counting. Charge both input and output tokens at separate rates per model; HF pricing refresher applies 80% of each marketplace side separately.",
|
||
"acceptanceCriteria": [
|
||
"Streamed and non-streamed requests for the same exchange bill the same split (input charged in both)",
|
||
"A model with asymmetric provider rates bills input and output differently; usage_for / billing events expose the split",
|
||
"Old persisted billing events replay byte-identically (balances unchanged)",
|
||
"HF refresh sets both rates from the marketplace row, not the average",
|
||
"Spend cap (--max-charge-per-request) uses the dual rates",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 45,
|
||
"status": "in-progress",
|
||
"notes": "Source issue: docs/issues/45-dual-rate-billing.md. Billing correctness before friends test.",
|
||
"dependsOn": [
|
||
"US-031"
|
||
]
|
||
},
|
||
{
|
||
"id": "US-046",
|
||
"title": "46 — Tracker .env awareness + first-node auto-join bootstrap",
|
||
"description": "Fresh trackers return 503 on auto-join because deployability ignores the joining caller's hardware, and meshnet-tracker ignores .env MESHNET_DOWNLOAD_DIR. Fix empty-registry bootstrap, tracker env loading parity with node CLI, models-dir fallback chain, and tar dereference for HF symlink snapshots.",
|
||
"acceptanceCriteria": [
|
||
"Fresh tracker (empty registry) + caller with enough memory for a recommended preset → /v1/network/assign returns 200 with model_sources populated when tracker holds a local snapshot",
|
||
"Fresh tracker + caller too small for any recommended preset → still 503",
|
||
"meshnet-tracker start in a directory with .env setting MESHNET_DOWNLOAD_DIR serves /v1/model-files/download from that dir with no extra flags",
|
||
"Explicit --models-dir and MESHNET_MODELS_DIR still take precedence",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 46,
|
||
"status": "needs-review",
|
||
"notes": "Source issue: docs/issues/46-tracker-env-and-first-node-autojoin.md. Verified live 2026-07-06.",
|
||
"dependsOn": [
|
||
"US-044"
|
||
],
|
||
"completionNotes": "Empty-registry synthesizes caller as candidate node; tracker loads .env; models_dir falls back MESHNET_MODELS_DIR → MESHNET_DOWNLOAD_DIR; tar dereference=True. Pytest passes aside from known port-7000 env conflict."
|
||
},
|
||
{
|
||
"id": "US-047",
|
||
"title": "47 — Tracker-first model downloads: visibility, sane timeouts, RAM-based sizing",
|
||
"description": "Explicit --model startup should skip pointless auto-join; tracker/peer sources preferred over HF with visible progress and 30 s socket timeouts; client abort during tar stream logs one line; CPU nodes size shards from RAM not phantom GPU VRAM; per-file downloads for robustness over fragile multi-GB tar streams.",
|
||
"acceptanceCriteria": [
|
||
"Node started with explicit --model never queries /v1/network/assign and never prints auto-join unavailable",
|
||
"Tracker/peer model source preferred; HF only when no source, all sources fail, or --tracker-source-disabled",
|
||
"Tracker-source downloads print progress every 512 MB and print exception + URL on failure",
|
||
"A ≥2 s read stall no longer aborts a tracker model-source download (30 s socket timeout)",
|
||
"Client disconnect during /v1/model-files/download logs one line on the tracker, no traceback",
|
||
"CPU node with big RAM gets a RAM-sized shard: sizing paths ignore VRAM unless device=cuda",
|
||
"Live two-machine retest: Windows node downloads from tracker at LAN speed with RAM-sized shard assignment",
|
||
"python -m pytest passes from repo root"
|
||
],
|
||
"priority": 47,
|
||
"status": "in-progress",
|
||
"notes": "Source issue: docs/issues/47-model-source-download-visibility.md. Engineering largely complete 2026-07-06; live two-machine retest pending.",
|
||
"dependsOn": [
|
||
"US-044",
|
||
"US-046"
|
||
],
|
||
"completionNotes": "Skip auto-join when model explicit; sequential source try with progress; 30 s model-source timeout; assignment_vram_mb for CPU; per-file /v1/model-files/download with manifest and retries. Remaining: live Windows two-machine retest."
|
||
}
|
||
],
|
||
"metadata": {
|
||
"updatedAt": "2026-07-13T16:15:00.000Z",
|
||
"statusVocabulary": {
|
||
"open": "Not started",
|
||
"in-design": "Decisions pending before implementation can begin",
|
||
"in-progress": "Agent actively working on it",
|
||
"needs-review": "Implemented, awaiting human review or verification",
|
||
"to-revise": "Previously done, needs rework due to design or architecture changes",
|
||
"done": "Implemented, tested, and verified",
|
||
"blocked": "Waiting on unresolved external dependency"
|
||
}
|
||
}
|
||
} |